JP7410402B2 - Visual inspection system - Google Patents

Description

The present disclosure relates to an image-based visual inspection system.

In the manufacturing process, visual inspections are performed on various objects. In the past, appearance inspections were performed visually by humans, but in recent years, automation has been progressing.

In automatic visual inspection, it is common to image an object with a camera configured with a sophisticated optical system, and automatically determine whether the object is defective through image processing by a computer. In such an automatic visual inspection system, determination is made by replacing the degree of defect in the acquired image with a numerical value and comparing the numerical value with a preset threshold value. Further, in an automatic appearance inspection system, any item whose characteristic amount can be imaged, such as surface scratches, dirt, shape, and posture, can be inspected.

Automatic visual inspection systems are required to improve detection accuracy to prevent defective products from being shipped. Detection accuracy is strongly influenced by inspection parameters, such as imaging conditions such as an illumination method during inspection, and quality of captured images such as variations in color and shape of objects. Furthermore, the detection accuracy changes depending on where the cutoff point (threshold value) for defect determination is set.

Japanese Patent Application Publication No. 2006-098365

However, when the detection accuracy is increased in a conventional automatic appearance inspection system, over-determination (over-determination) in which an originally good product is determined to be defective becomes more likely to occur, and the amount of over-determination also increases.

In this way, there is a trade-off relationship between detection accuracy and the amount of overdetermination in automatic visual inspection, but it is realistic to set inspection parameters and thresholds that can maximize detection accuracy while minimizing the amount of overdetermination. Very difficult.

For this reason, during mass production, a certain number of over-judged products are allowed to occur, even though the true number of defects is extremely small. Since it is necessary to make judgments, there is a problem in that productivity decreases.

An object of the present disclosure is to make it possible to suppress a decrease in productivity due to the occurrence of over-judged products in visual inspection.

A first aspect of the present disclosure includes a primary inspection unit (10) that performs a defect determination without using machine learning based on an image taken of an object, and an object determined to be defective by the primary inspection unit (10). A visual inspection system comprising a secondary inspection section (20) that uses a first machine learning model (21) to distinguish between true defective products and over-judged products based on an image of the object. be.

In the first aspect, the secondary inspection section (20) uses the first machine learning model (21) to determine whether the object is over-judged based on the image of the object determined to be defective by the primary inspection section (10). Re-evaluate whether or not. Therefore, the occurrence of over-judged products can be suppressed, and the number of man-hours for manually re-inspecting whether or not there are over-judged products at the final stage of inspection can be reduced, so that it is possible to suppress a decrease in productivity.

A second aspect of the present disclosure is that, in the first aspect, machine learning is performed on the first machine learning model (21) in advance using images of non-defective products as training data, and the secondary inspection unit ( 20) is based on the difference between the non-defective image generated by the first machine learning model (21) from the image of the object determined to be defective and the image of the object determined to be defective, This is an appearance inspection system that is characterized by separating true defective products from over-determined products.

In the second aspect, if the difference between the image of the object determined to be defective and the image of the good item generated by the first machine learning model (21) exceeds a predetermined value, it is determined that the object is truly defective. can be determined. Therefore, it is possible to accurately determine whether the object is defective without being sensitive to slight changes (positional deviation, etc.) in the image of the object.

In a third aspect of the present disclosure, in the second aspect, the secondary inspection unit (20) examines each of the entire image and/or a specific portion of the image of the object determined to be defective. This is an appearance inspection system that evaluates the difference between the image and a non-defective image generated by a machine learning model (21).

In the third aspect, for example, when inspecting parts of various shapes at once on a printed circuit board on which parts are mounted, the installation position of the parts, etc., is determined for the entire image (macroscopic part) between the non-defective image and the For the solder joints of leads, etc., it is possible to evaluate the difference between a specific part (microscopic part) of the image and a non-defective image. As a result, even microscopic parts that tend to be buried in macroscopic parts can be inspected all at once, improving detection power.

A fourth aspect of the present disclosure is that, in the second or third aspect, the first machine learning model (21) uses the second machine learning model (31) based on line drawing information including at least an outline. The visual inspection system further includes a learning data generation unit (30) that generates a large number of non-defective images for learning.

In the fourth aspect, there is no need to prepare a large number of real images of the object for learning of the first machine learning model (21). Therefore, for example, the first machine learning model (21) can be trained even before the prototype is manufactured and the actual product does not exist, so it is possible to improve the inspection quality of the prototype and shorten the trial production period.

In a fifth aspect of the present disclosure, in the second or third aspect, based on at least one real image, a second machine learning model (31) is used to transform the first machine learning model (21). The external appearance inspection system further includes a learning data generation section (30) that generates a large number of non-defective images for learning.

In the fifth aspect, there is no need to prepare a large number of real images of the object for learning of the first machine learning model (21). In other words, the first machine learning model (21) can be trained using a small number of real images.

In a sixth aspect of the present disclosure, in the fourth aspect, the second machine learning model (31) is configured to perform machine learning in advance to generate a texture from hint information given for each part of the line drawing information. is performed, and the learning data generation unit (30) adds noise to the texture generated by the second machine learning model (31) to generate a large number of non-defective images for learning. It is an inspection system.

In the sixth aspect, the second machine learning model (31) is trained in advance to generate a texture for each part of line drawing information, that is, for each basic part. It is possible to obtain good product images for training of the machine learning model (21). Therefore, the learning of the first machine learning model (21) can be advanced at the time of starting use of the system, thereby improving convenience and responsiveness for system users.

In a seventh aspect of the present disclosure, in any one of the first to sixth aspects, the first machine learning model (21) is selected from a plurality of learning models subjected to machine learning according to the inspection item. The secondary inspection unit (20) is an appearance inspection system characterized in that the plurality of learning models are hierarchically combined and used.

In the seventh aspect, by using a hierarchical combination of multiple learning models subjected to machine learning according to inspection items, it is possible to accurately and efficiently classify true defective products and over-judged products. I can do it.

In an eighth aspect of the present disclosure, in any one of the first to seventh aspects, a plurality of the first machine learning models (21) are prepared in advance for each category of objects, and the primary inspection unit ( 10) determines the category of the object from the image taken of the object, and the secondary inspection section (20) determines the category of the object corresponding to the category of the object determined by the primary inspection section (10). This is an appearance inspection system characterized by using the first machine learning model (21).

In the eighth aspect, when the objects to be inspected have various shapes, the learning of the first machine learning model (21) can be optimized for each category in which objects with similar shapes are grouped. . Therefore, true defective products and over-determined products can be separated with high accuracy.

FIG. 1 is an overall configuration diagram of a visual inspection system according to an embodiment. FIG. 2 is a diagram showing good and defective solder joints of electronic component leads mounted on a printed circuit board. FIG. 3 is a diagram illustrating how the visual inspection system according to the embodiment evaluates the difference between the entire image of the object and the specific portion thereof with respect to the non-defective image. FIG. 4 is a flowchart of visual inspection by the visual inspection system according to the embodiment. FIG. 5 is an overall configuration diagram of a visual inspection system according to Modification 1. FIG. 6 is a diagram illustrating how the learning data generation unit of the visual inspection system according to Modification 1 generates a plurality of non-defective images based on line drawing information. FIG. 7 is a diagram illustrating how the learning data generation unit of the visual inspection system according to Modification 1 generates a plurality of non-defective images based on an actual image. FIG. 8 is a flowchart of non-defective product image generation by the learning data generation unit of the visual inspection system according to Modification 1. FIG. 9 is an overall configuration diagram of a visual inspection system according to a second modification. FIG. 10 is an overall configuration diagram of a visual inspection system according to modification 3. FIG. 11 is a diagram showing how machine learning of a learning model is performed for each category in the visual inspection system according to Modification 3.

(Summary of embodiment)
In the visual inspection system according to the embodiment described below, based on the image of the object determined to be defective by a conventional automatic visual inspection device (primary inspection section) that does not use a learning model such as a neural network for the determination itself, A secondary inspection unit using a learning model performs secondary judgment in real time to separate true defective products from over-judged products.

As a result, it is possible to efficiently perform the determination and separation of over-determined products while making use of the detection power (high detection accuracy) of inspection items that the conventional automatic visual inspection apparatus is good at.

For example, by generating an image of a presumed non-defective product of the part to be inspected using a learning model and quantifying the difference between the non-defective image and the image of the object acquired in the primary inspection department, re-judgment of an over-judged product can be performed. You may do so.

Furthermore, the machine learning of the learning model may be optimized according to the inspection item or the category of the object, and these learning models may be hierarchically combined or used selectively. As a result, even when a plurality of inspection items are inspected at once on the premise of objects having various shapes, it is possible to perform re-judgment of over-judged products with high accuracy.

Furthermore, when there are two or more types of failure modes of the object, and the feature amount of each failure mode appears separately in the macroscopic part, which is the whole (or most part) of the image, and the microscopic part, which is a part of the image. , the macroscopic part and the microscopic part may be separated and determined separately. Thereby, it is possible to avoid a situation in which the feature amount of the microscopic portion is buried in the feature amount of the macroscopic portion and cannot be detected.

In addition, a learning model different from the learning model that re-judges over-judged products is used to generate the necessary amount of learning images from line drawing information (for example, design drawing information) of the object and a small number of actual images. It's okay. As a result, even if the inspection target changes due to the introduction of a new model, it is possible to prepare a considerable amount of learning data in advance, so you can quickly develop a visual inspection system that can handle the new target. can be completed. Further, if a unique texture exists for each part made of different materials, the texture may be generated using another learning model from hint (label) information given to each part of the line drawing information. Thereby, it is possible to generate a learning image that is close to a real image.

(Embodiment)
Embodiments will be described with reference to the drawings. As shown in FIG. 1, the visual inspection system (100) according to the present embodiment includes a primary inspection section (10) that performs defective determination without using machine learning based on an image taken of the object (1); The secondary inspection section (20) uses the first machine learning model (21) to separate true defective products from over-judged products based on the image of the object determined to be defective by the primary inspection section (10). ).

As the primary inspection section (10), for example, a general optical appearance inspection device that does not use a machine learning model for defect determination itself can be used. The primary inspection section (10) includes, for example, an imaging section (11) such as a camera. The primary inspection section (10) may include a storage section such as a hard disk, for example, to store data such as captured images. Although the primary inspection unit (10) does not use a learning model for defect determination itself, it may use machine learning when determining inspection thresholds and inspection parameters.

The primary inspection section (10) and the secondary inspection section (20) each have a processing section for making a defective determination. Each processing unit includes, for example, a processor and a memory that stores programs and information for operating the processor. The primary inspection section (10) and the secondary inspection section (20) are configured to be able to exchange data such as images with each other. The secondary inspection section (20) may have a storage section such as a hard disk, for example, to store data such as images transmitted from the primary inspection section (10). Furthermore, in the visual inspection system (100), the primary inspection section (10) and the secondary inspection section (20) may have common storage sections.

The first machine learning model (21) used in the secondary inspection unit (20) may be, for example, a neural network such as a multilayer perceptron, a support vector machine, a discriminant function, a Bayesian network, etc., but is not particularly limited. It's not something you can do.

When the first machine learning model (21) is configured as a neural network, the first machine learning model (21) has, for example, three layers: an input layer, a middle layer, and an output layer. Each layer includes one or more neurons, each neuron in the input layer is connected to each neuron in the intermediate layer, and each neuron in the intermediate layer is connected to each neuron in the output layer. An image of an object determined to be defective by the primary inspection unit (10) is input to each neuron in the input layer.

Before the secondary inspection unit (20) uses the first machine learning model (21) to re-determine over-judged products, the secondary inspection unit (20) or another information processing device Machine learning of the machine learning model (21), that is, calculation of model data, for example, calculation of model data indicating settings of a neural network. The model data includes, for example, the number of layers in the neural network, the number of neurons (nodes) included in each layer, and coupling coefficients (coupling weights) between neurons. The secondary inspection unit (20) sets the first machine learning model (21) using the model data, and uses the set first machine learning model (21) (that is, the trained model) to determine the overdecision. The product will be re-evaluated.

When the first machine learning model (21) is configured as a neural network, in machine learning of the neural network, first, training data of input information is input to the input layer, and data is propagated from the input layer to the output layer. , obtain output information from the output layer. Next, using the obtained output information and the training data of the output information, the coupling coefficient between the input layer and the output layer and the bias assigned to the neurons of the intermediate layer are calculated. For example, the coupling coefficient and bias are adjusted so that the difference between the output information and the teacher data becomes smaller.

As a result of the above machine learning, model data including adjusted coupling coefficients and biases is generated. The model data includes, for example, the number of layers in the neural network, the number of neurons belonging to each layer, coupling coefficients, and biases. The generated model data is stored in the storage section of the secondary inspection section (20), for example. The secondary inspection unit (20) sets a neural network based on the stored model data before re-judging over-judgment products. That is, the secondary inspection unit (20) sets the number of layers, the number of neurons, the coupling coefficient, and the bias in the neural network constituting the first machine learning model (21) to values specified in the model data. . In this way, the secondary inspection unit (20) re-determines over-determined products using the first machine learning model (21) using model data.

In this embodiment, machine learning is performed on the first machine learning model (21) in advance using images of non-defective products as training data, and Based on the image, the first machine learning model (21) may be able to generate a non-defective image corresponding to the image. In this case, for example, a generative adversarial network (GAN) may be used as the first machine learning model (21). In the case of a truly defective product, the difference between the actual image and the non-defective image generated by the first machine learning model (21) is measured, and this is used to set an arbitrary threshold. The next inspection section (20) can distinguish between true defective products and over-determined products.

Hereinafter, a case will be described as an example in which a printed circuit board on which a large number of electronic components are mounted by soldering or the like is used as an object (1) to perform a defect determination by the appearance inspection system (100) of this embodiment. Specifically, the mounting state of a plurality of electronic components (several hundred or more per board) having sizes of several hundred to several thousand μm and different shapes is inspected for several tens of seconds per board. Here, there is a trade-off relationship between the inspection accuracy and the amount of overdetermination, but by rejudging (secondary inspection) of overdetermination products using a learning model as in this embodiment, the inspection accuracy can be improved. It is possible to effectively reduce the amount of overdetermination while maintaining the same.

FIG. 2 is a diagram showing good and defective solder joints of leads in electronic components mounted on a printed circuit board. As shown in Fig. 2, when the electrode (3) on the printed circuit board (2) and the lead (4) of the electronic component are joined by solder (5), for the non-defective states (a) and (b), Various defective product states (c) to (l) may occur. For example, in a defective product state (g), a pinhole (blowhole) (6) occurs in the solder (5), and in a defective product state (h), a foreign object (7) gets mixed into the solder (5), resulting in a defective product. In the state (k), another lead (8) is joined to the solder (5), and in the defective product state (l), a solder ball (9) is formed.

In contrast to the entire or most part of an image of a printed circuit board (hereinafter also referred to as the "macroscopic part"), the solder joints of the leads as shown in FIG. , sometimes called the "microscopic part"). However, when features that appear in the macroscopic area, such as misalignment of components, and features that appear in the microscopic area, such as poor solder joints of leads, appear at the same time, the characteristics of the microscopic area There is a possibility that the amount is buried in the feature amount of the macroscopic part and cannot be detected.

Accordingly, in the present embodiment, as shown in FIG. Evaluate the difference (difference image) with the non-defective image generated by (21). FIG. 3 also shows the results (frequency) of determining whether a large number of objects (1) are defective (NG) based on the amount of difference (degree of abnormality) for each of the macroscopic part and the microscopic part. In this way, by separately determining the defectiveness of the macroscopic part and the microscopic part, it becomes possible to evaluate the degree of abnormality of the microscopic part, which tends to be overshadowed by the degree of abnormality of the macroscopic part.

In the example shown in FIG. 3, the difference between the image of the object (1), which was determined to be defective by the primary inspection section (10), and the image of the good item was evaluated for both the macroscopic part and the microscopic part. Instead, the difference between the macroscopic part and the microscopic part with respect to the non-defective image may be evaluated.

Alternatively, a separately prepared machine learning model may be used to recognize the position of each microscopic part in the image of the object (1). This is because even if the dimensional accuracy of individual electronic components is good, if the placement of the same component on the board is different, the background circuit pattern and printed characters on the board surface will also be different, so the inspection range will be small. This is because it is difficult to set the visual field artificially. At the same time, the fact that position recognition cannot be performed using a separately prepared learning model means that an abnormality such as misalignment of parts has been detected. Accordingly, by automatically recognizing the microscopic part using the first machine learning model (21), inspection accuracy can be improved.

FIG. 4 is a flowchart of visual inspection by the visual inspection system (100) of this embodiment.

First, in step S1, it is determined whether or not defect determination has been completed for all components (parts and parts to be inspected) of the object (1). If all components have been determined to be defective, the process ends.

If there is a component for which the defect determination has not been completed, in step S2, the primary inspection section (10) (for example, an optical appearance inspection device) performs a defect determination for the component to be determined. If there is no defect in the component, a non-defective product is determined in step S9, and the process returns to step S1. If the component is defective, in step S3, the secondary inspection section (20) acquires an image of the object (1) including the component from the primary inspection section (10). In addition, when the primary inspection section (10) has a storage section, the secondary inspection section (20) does not acquire images for each component from the primary inspection section (20), but acquires all images from the primary inspection section (20). Images may be acquired for all components at once.

The secondary inspection unit (20) that has acquired the image of the target object (1) may select (switch) the first machine learning model (21) corresponding to the component in step S4. The selection of this first machine learning model (21) will be described in detail in Modified Example 3 below.

Next, in step S5, the secondary inspection unit (20) uses the first machine learning model (21) to detect defects (for example, misalignment) in the component in the macroscopic part of the image of the object (1). If there is a defect, a defective product is determined in step S8, and the process returns to step S1. If there is no defect in step S5, the secondary inspection unit (20) uses the first machine learning model (21) to check the microscopic part related to the component in the image of the object (1) in step S6. (e.g. solder joints).

If the position of the microscopic unit cannot be recognized in step S6, a defective product is determined in step S8, and the process returns to step S1. If the position of the microscopic section can be recognized in step S6, the secondary inspection section (20) uses the first machine learning model (21) in step S7 to determine whether the component is defective or not. For example, whether there is a solder joint defect) is evaluated based on the difference from the above-mentioned non-defective image. If there is a defect, a defective product is determined in step S8, and the process returns to step S1. If there is no defect in step S7, the secondary inspection section (20) makes a non-defective determination in step S9, and returns to step S1.

In the present embodiment, the processes of steps S2 to S9 described above are repeatedly performed for each component until it is determined in step S1 that the defect determination has been completed for all components.

-Effects of embodiment-
According to the present embodiment, the secondary inspection section (20) uses the first machine learning model (21) to perform an overperformance based on the image of the object (1) determined to be defective by the primary inspection section (10). Re-determine whether the product is a certified product. Therefore, the occurrence of over-judged products can be suppressed, and the number of man-hours required to re-inspect whether the over-judged products are present at the final stage of inspection can be reduced, thereby suppressing the decline in productivity caused by the occurrence of over-judged products. This is particularly effective when there is a large amount of overdetermination.

In addition, re-inspections are often visual inspection tasks, but in this embodiment, visual inspection personnel who are responsible for these tasks may become fatigued due to repeated simple tasks, or may overlook or make assumptions. Since this can be suppressed, it is also possible to constantly prevent a decrease in detection power.

In addition, in this embodiment, machine learning is performed on the first machine learning model (21) using images of non-defective products as training data in advance, and the objects determined to be defective by the primary inspection section (10) are A non-defective image is generated from the image of (1) using the first machine learning model (21), and a true defect is determined based on the difference between the non-defective image and the image of the object (1) determined to be defective. Good products and over-determined products may be separated. In this way, if the difference between the image of the object (1) determined to be defective and the corresponding image of the non-defective item exceeds a predetermined value, it can be determined that the object (1) is a truly defective item. Therefore, it is possible to accurately determine whether the object (1) is defective without being sensitive to slight changes (positional deviation, etc.) in the image.

Further, in the present embodiment, the secondary inspection unit (20) inspects each of the macroscopic part (the whole or most part) and/or the microscopic part (specific part) of the image of the object (1) determined to be defective. , the difference from the non-defective image generated by the first machine learning model (21) may be evaluated. In this way, for example, when inspecting printed circuit boards on which components are mounted and components of various shapes at once, the installation position of the components can be determined by evaluating the difference between the macroscopic part of the image and the non-defective image. However, for the solder joints of leads, etc., it is possible to evaluate the difference between the microscopic part of the image and the non-defective image. As a result, even microscopic parts that tend to be buried in macroscopic parts can be inspected all at once, improving detection power.

(Modification 1)
Modification 1 will be explained with reference to the drawings. As shown in FIG. 5, the appearance inspection system (100) according to this modification is different from the embodiment shown in FIG. It is equipped with a learning data generation unit (30) that generates a large number of good product images for learning of the machine learning model (21). In FIG. 5, the same components as in the embodiment shown in FIG. 1 are given the same reference numerals.

The second machine learning model (31) may generate a non-defective image based on line drawing information (e.g., design drawing information) that includes at least an outline, as shown in FIG. As shown in , a non-defective product image may be generated based on at least one real image.

By the way, since electronic parts are industrial products, the dimensions, shape, and surface of the board on which they are assembled are basically the same for each type of part, and the texture (hue, brightness, (texture, etc.) is easy to determine. Therefore, in this modification, when the second machine learning model (31) generates a non-defective image based on the line drawing information, the second machine learning model (31) is given a hint ( Machine learning may be performed in advance to generate textures from label) information. In this way, by adding noise adjusted to a predetermined amount to the texture generated by the second machine learning model (31), it is possible to generate a large number of non-defective images that are close to real images.

FIG. 8 is an example of a flowchart of non-defective product image generation by the learning data generation unit (30) of the visual inspection system (100) of this modification. In addition, in FIG. 8, "line drawing", "label", "texture", and "generated image" are schematically shown so as to correspond to each step of the flow diagram.

First, in step S11, the learning data generation unit (30) acquires line drawing information (original image) including at least an outline.

Next, in step S12, label information for each part of the line drawing information is provided to the learning data generation unit (30), and the second machine learning model (31) generates texture for each part from the label information. The texture may be generated in advance and the obtained texture may be stored in the database D1.

Next, in step S13, the learning data generation unit (30) reads out the region-specific textures stored in the database D1, and adds noise to the read textures.

Next, in step S14, the learning data generation unit (30) generates a large number of non-defective images by combining textures for each region to which noise has been added, and in step S15, outputs the generated non-defective images. The outputted non-defective product images are stored, for example, in the secondary inspection unit (20) or the storage unit of another information processing device, and these non-defective product images are used as the first machine learning model (21) described in the above embodiment. used as training data in machine learning.

-Effects of modification 1-
In this modified example described above, in addition to the effects similar to those of the embodiment described above, the following effects can be obtained.

According to this modification, the learning data generation unit (30) uses the second machine learning model (31) to generate a learning model for the first machine learning model (21) based on the line drawing information including at least the outline. Generate many images of non-defective products. Therefore, there is no need to prepare a large number of real images of the object (1) for learning the first machine learning model (21). Therefore, it is possible to train the first machine learning model (21) even in the absence of the actual product before prototype production, for example, so that it is possible to improve the inspection quality and the learning for prototypes with a small number of actual products or in the early stage of mass production. It becomes possible to shorten the period until completion.

Further, according to this modification, the learning data generation unit (30) uses the second machine learning model (31) based on at least one real image to Generate many images of non-defective products. Therefore, there is no need to prepare a large number of real images of the object for learning the first machine learning model (21). In other words, the first machine learning model (21) can be trained using a small number of real images.

Further, according to this modification, the second machine learning model (31) is trained in advance to generate textures from hint information given for each part (that is, basic parts) of line drawing information. As a result, it is possible to obtain good product images for learning of the first machine learning model (21) for many new models. Therefore, since the first machine learning model (21) can be trained at the time the system starts to be used, convenience and responsiveness can be guaranteed to the final system user even before use. .

On the other hand, if the learning data generation unit (30) is not provided, if the object to be inspected is new or the actual object does not exist, it may not be possible to prepare images that will serve as training data for the first machine learning model (21). , or only a limited amount of learning data can be prepared. As a result, it may not be possible to train the first machine learning model (21) that can handle newly applied parts in new models, etc., or the learning will be insufficient, resulting in over-judged parts. The re-judgment accuracy will decrease.

On the other hand, as mentioned above, in this modified example, it is possible to generate multiple teacher images from line drawing information of electronic components with clear dimensions, shapes, standards, etc. and from a small number of real images, so a considerable amount of It is possible to start learning the first machine learning model (21) even before obtaining the actual machine learning model (21). In other words, by pre-learning the first machine learning model (21) about basic parts, etc., it is possible to eliminate the user's concerns about the preparation lead time.

In addition, even if the dimensional accuracy of individual electronic components, which are industrial products, is good, slight misalignment of the components can cause various shapes to appear in the shape of solder, which is molten metal. Also, it is possible to flexibly determine whether the product is defective by using the first machine learning model (21) that has been trained using the non-defective product images of this modification.

(Modification 2)
Modification 2 will be explained with reference to the drawings. As shown in FIG. 9, the appearance inspection system (100) according to this modification is different from the embodiment shown in FIG. The secondary inspection section (20) uses a hierarchical combination of multiple learning models. In FIG. 9, the same components as those in the embodiment shown in FIG. 1 are given the same reference numerals.

In the appearance inspection system (100) of this modified example, as shown in FIG. When the test item 1 to N is received, a learning model optimized for each item is used to determine whether the test item is defective. Only when there are no defects in all items, the judgment result (25) of "good product" is output.

-Effects of modification 2-
In this modified example described above, in addition to the effects similar to those of the embodiment described above, the following effects can be obtained. In other words, by using a hierarchical combination of multiple learning models that have been optimally machine learned according to inspection items, it is possible to accurately and efficiently distinguish between true defective products and over-identified products. . Furthermore, if a certain item is found to be defective, the inspection can be efficiently completed by determining "defective (NG)" without determining whether the next item is defective. Conversely, it is also possible to collect desired result information by programming so that one or more test items can be arbitrarily selected depending on the subject.

(Modification 3)
Modification 3 will be explained with reference to the drawings. As shown in FIG. 10, the appearance inspection system (100) according to this modification differs from the embodiment shown in FIG. 1 in the following points. That is, a plurality of first machine learning models (21) are prepared in advance for each category in which objects with similar shapes are grouped, and the primary inspection section (10) uses the images of the object (1) to determine the corresponding model. The category of the object (1) is determined, and the secondary inspection section (20) uses the first machine learning model (21) corresponding to the category of the object (1) determined by the primary inspection section (10). use In FIG. 10, the same components as in the embodiment shown in FIG. 1 are given the same reference numerals.

In the appearance inspection system (100) of this modification, as shown in FIG. , and upon receiving the category information (16) of the object (1), the selection unit (22) selects a learning model (determination model) corresponding to the category information (16), and selects the selected determination model. A defective judgment is made using , and the judgment result (25) is output.

In this modification, as shown in FIG. 10, determination models 1 to N are prepared for each of N categories of component groups 1 to N. Further, as shown in FIG. 11, each of the determination models 1 to N is generated by performing learning in advance using learning data (image data set) prepared for each category.

-Effects of modification 3-
In this modified example described above, in addition to the effects similar to those of the embodiment described above, the following effects can be obtained. That is, when the objects (1) to be inspected have various shapes, the learning of the first machine learning model (21) can be optimized for each category in which objects with similar shapes are grouped. Therefore, true defective products and over-determined products can be separated with high accuracy.

《Other embodiments》
In the above embodiment (including variations; the same applies hereinafter), the object (1) is a printed circuit board on which many electronic components are mounted by soldering etc., but the object (1) is particularly Needless to say, it is not limited. For example, the appearance inspection system (100) of the above embodiment may be used for shipping inspection of fresh vegetables as objects (1). In shipping inspection of fresh vegetables, there are cases where it is desired to inspect both the characteristics that appear on the whole vegetable, such as the color of the leaves, which indicates freshness, and the characteristics that appear in a specific part, such as the diameter of a stump, which indicates the growing period. By using the appearance inspection system (100), features that appear on the whole vegetable and features that appear on specific parts can be divided into macroscopic parts and microscopic parts, respectively, and can be accurately and efficiently inspected.

Although the embodiments and modifications have been described above, it will be understood that various changes in form and details can be made without departing from the spirit and scope of the claims. Furthermore, the above embodiments and modifications may be combined or replaced as appropriate, as long as the functionality of the object of the present disclosure is not impaired.

As described above, the present disclosure is useful for visual inspection systems.

10 Primary inspection section 20 Secondary inspection section 21 First machine learning model 30 Learning data generation section 31 Second machine learning model 100 Visual inspection system

Claims (6)

A primary inspection section (10) that performs defective judgments based on images of objects without using machine learning;
a secondary inspection section (10) that uses a first machine learning model (21) to separate true defective products from over-determined products based on images of objects determined to be defective by the primary inspection section (10); 20) and
Machine learning is performed on the first machine learning model (21) in advance using images of non-defective products as training data,
The secondary inspection unit (20) generates a non-defective image generated by the first machine learning model (21) from the image of the object determined to be defective, and an image of the object determined to be defective. Based on the difference between
The secondary inspection unit (20) compares each of the entire image and/or specific portion of the image of the object determined to be defective with the non-defective image generated by the first machine learning model (21). evaluate,
After determining whether the entire image is defective, the secondary inspection section (20) recognizes the position of the specific portion of the image if there is no defect in the entire image, and then determines the position of the specific portion of the image. Determine whether a specific part is defective
An appearance inspection system characterized by: In claim 1 ,
a learning data generation unit (30) that uses a second machine learning model (31) to generate a large number of good product images for learning of the first machine learning model (21) based on line drawing information including at least a contour; An appearance inspection system further comprising: In claim 1 ,
further comprising a learning data generation unit (30) that uses a second machine learning model (31) to generate a large number of non-defective images for learning of the first machine learning model (21) based on at least one real image; An appearance inspection system comprising: In claim 2 ,
Machine learning is performed on the second machine learning model (31) in advance to generate a texture from hint information given for each part of the line drawing information,
The visual inspection system is characterized in that the learning data generation unit (30) generates a large number of non-defective images for learning by adding noise to the texture generated by the second machine learning model (31). In any one of claims 1 to 4 ,
The first machine learning model (21) is composed of a plurality of learning models subjected to machine learning according to inspection items,
The appearance inspection system is characterized in that the secondary inspection section (20) uses a hierarchical combination of the plurality of learning models. In any one of claims 1 to 5 ,
A plurality of the first machine learning models (21) are prepared in advance for each category of objects, and
The primary inspection unit (10) determines the category of the object from the image taken of the object,
The appearance inspection system is characterized in that the secondary inspection section (20) uses the first machine learning model (21) corresponding to the category of the object determined by the primary inspection section (10).

Images