JP7614053B2 - Learning device, estimation device, defect inspection device, and defect inspection method - Google Patents

Description

The present invention relates to a learning device, an estimation device, a defect inspection device, and a defect inspection method.

A packaging material called embossed carrier tape has been known for storing tiny electronic components (see Patent Document 1 below).

JP 2003-136545 A

The above-mentioned embossed carrier tape needs to be inspected for defects such as the adhesion of foreign matter to the surface and defects such as burrs in the recesses and edges that house electronic components. In order to inspect the appearance of the embossed carrier tape with high precision, it is necessary to use clear images, but since there are multiple types of defects and embossed carrier tapes come in a variety of shapes, thicknesses, and materials, it is necessary to carefully set the lighting conditions of the lighting device that are optimal for inspecting defects for each embossed carrier tape.

One of the objectives of the present invention is to provide a learning device, an estimation device, a defect inspection device, and a defect inspection method that can inspect objects with defects with high accuracy.

A learning device according to one aspect of the present invention includes an input unit for inputting defect type information indicating the type of defect of an object, lighting condition information regarding the lighting conditions under which a lighting device suitable for the type of defect to be detected in the object irradiates light, and object information for identifying the object, and a model construction unit for constructing a model that is machine-learned using the defect type information, lighting condition information, and object information input by the input unit as training data, and that outputs the lighting condition information regarding the optimal conditions for the object when the defect type information and object information are input.

An estimation device according to one aspect of the present invention includes an input unit that inputs defect type information indicating a type of defect of an object and object information for identifying the object, and an estimation unit that inputs the defect type information and object information input by the input unit into a model that has been machine-learned using the defect type information, the object information, and lighting condition information related to the lighting conditions under which a lighting device corresponding to the type of defect of the object irradiates light, as training data, and estimates optimal lighting conditions for the object based on the lighting condition information related to the optimal conditions for the object output from the model.

A defect inspection device according to one aspect of the present invention includes a lighting device that irradiates light onto an object, an inspection unit that inspects the object, an input unit that inputs defect type information indicating the type of defect of the object and object information for identifying the object, an estimation unit that inputs the defect type information and object information input by the input unit into a model that has been machine-learned using the defect type information, the object information, and lighting condition information related to the lighting conditions under which the lighting device irradiates light corresponding to the type of defect of the object as training data, and estimates optimal lighting conditions for the object based on the lighting condition information related to the optimal conditions for the object output from the model, and a control unit that controls the lighting device based on the lighting conditions estimated by the estimation unit.

A defect inspection method according to one aspect of the present invention includes the steps of : a defect inspection device inputting defect type information indicating a type of defect of an object, lighting condition information related to lighting conditions under which an illumination device corresponding to the type of defect of the object irradiates light, and object information for identifying the object; a step of constructing a model in which the defect inspection device has performed machine learning using the input defect type information, lighting condition information, and object information as training data, the model outputting the lighting condition information related to optimal conditions for the object when the defect type information and the object information are input; a step of inputting the defect type information and the object information into the model when the defect inspection device inputs the defect type information and the object information for identifying the object, and estimating optimal lighting conditions for the object based on the lighting condition information related to optimal conditions for the object output from the model; a step of the defect inspection device controlling the illumination device based on the estimated lighting conditions; and a step of the defect inspection device inspecting the object.

The present invention provides a learning device, an estimation device, a defect inspection device, and a defect inspection method that can inspect objects with defects with high accuracy.

FIG. 1 is a diagram illustrating an example of a schematic configuration of a defect inspection system according to an embodiment. 1 is a block diagram illustrating an example of a functional configuration of a defect inspection system according to an embodiment. FIG. 2 is a block diagram showing an example of a learning unit in the embodiment. FIG. 1 is a diagram illustrating the relationship between input variables and output variables in a convolutional neural network. FIG. 2 is a block diagram showing an example of an estimation unit 330 in the embodiment. FIG. 2 is a top view showing an example of an arrangement of lighting devices in the embodiment. FIG. 11 is a top view showing another example of the arrangement of the lighting device in the embodiment. FIG. 2 is a side view of the lighting device according to the embodiment. FIG. 11 is another side view of the lighting device according to the embodiment. 1A is a top view showing an example of an embossed carrier tape 10 according to an embodiment, and FIG. 1B is an example of an embossed carrier tape 10 having no defects. FIG. 11 is a diagram showing a relationship between an illumination condition, a defect type, and correlation data in the embodiment. FIG. FIG. 13 is a diagram showing another configuration of the defect inspection system 1 according to the embodiment. FIG. 11 is a diagram for explaining a process of performing an estimation process using a convolutional neural network in the embodiment. FIG. 1 is a diagram for explaining a learning process using a convolutional neural network in an embodiment.

Below, embodiments of the learning device, estimation device, defect inspection device, and defect inspection method will be described with reference to the drawings.

[Defect inspection overview]
The defect inspection system 1 of the embodiment inspects defects in an object. In the embodiment, the object is, for example, an embossed carrier tape. The embossed carrier tape is made of plastic and has storage pockets (recesses) for storing chip-type electronic components formed continuously. The defect inspection system 1 inspects defects in the storage pockets and the like by acquiring information based on the shape of the embossed carrier tape while irradiating the embossed carrier tape with light. Note that, although the present embodiment measures the dimensions of shapes such as recesses, the present invention is not limited thereto and can also be applied to applications for measuring shapes other than recesses.

[Configuration of defect inspection system 1]
1 is a diagram showing an example of a schematic configuration of a defect inspection system 1 in an embodiment. The defect inspection system 1 includes, for example, an illumination device 100 and an imaging device 200 arranged above and below a production line L. The production line L is a moving means for delivering an embossed carrier tape 10 on which recesses 10a are formed. When measuring the dimensions of the embossed carrier tape 10, the defect inspection system 1 images the embossed carrier tape 10 with the imaging device 200 while irradiating the embossed carrier tape 10 with light from the illumination device 100. The defect inspection system 1 measures the dimensions of the recesses 10a based on the captured image or the like.

FIG. 2 is a block diagram showing an example of the functional configuration of the defect inspection system 1 of the embodiment. The defect inspection system 1 includes, for example, a lighting device 100, an imaging device 200, a defect inspection device 300, an information input unit 400, a teacher data storage unit 500, and a notification unit 600.

The defect inspection device 300 includes, for example, a defect inspection unit 310, a learning unit 320, an estimation unit 330, and an illumination control unit 340. The functional units such as the defect inspection unit 310, the learning unit 320, the estimation unit 330, and the illumination control unit 340 are realized by a processor such as a CPU (Central Processing Unit) executing a program stored in a program memory. In addition, some or all of these functional units may be realized by hardware such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array), or may be realized by software and hardware working together. The program may be stored in advance in a storage device (storage device with a non-transient storage medium) such as a hard disk drive (HDD) or flash memory of the defect inspection device 300, or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the defect inspection device 300 by mounting the storage medium (non-transient storage medium) in a drive device.

The defect inspection unit 310 inspects the embossed carrier tape 10 for defects using the image captured by the imaging device 200. The defect inspection unit 310 inspects whether the embossed carrier tape 10 has defects, for example, based on a correlation value based on a comparison between a reference image and the captured image. The reference image is, for example, an image of a non-defective product. The learning unit 320 constructs an estimation model that estimates optimal lighting conditions for the embossed carrier tape 10. The estimation unit 330 performs calculations using the estimation model to estimate optimal lighting conditions for defect inspection of the embossed carrier tape 10. The lighting control unit 340 controls the lighting device 100 according to the lighting conditions estimated by the estimation unit 330.

The information input unit 400 is, for example, a user interface device that is operated by a user. The information input unit 400 accepts object information and defect type information based on the user's operation and outputs them to the defect inspection device 300.

The teacher data storage unit 500 is realized by a storage device such as a HDD, a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory), or a RAM (Random Access Memory). The teacher data storage unit 500 may be realized by a SAN (Storage Area Network) or a NAS (Network Attached Storage). The teacher data storage unit 500 stores defect type information, illumination condition information, and object information as teacher data. The defect type information is information indicating the type of defect in the embossed carrier tape 10. The lighting condition information includes at least one of the lighting conditions, such as the arrangement of the multiple lights constituting the lighting device 100, the light intensity, the light angle, and the light wavelength. The object information is information indicating a portion of the embossed carrier tape 10 as the object, but is not limited to this. The object information may be physical property information including at least one of the shape, color, transparency, surface properties, molding method, and mold shape of the embossed carrier tape 10.

The notification unit 600 is, for example, a display or a speaker, and notifies, for example, the results of defective inspections, the results of estimating lighting conditions, and the results of controlling the lighting device 100.

FIG. 3 is a block diagram showing an example of the learning unit 320 in the embodiment. The learning unit 320 includes, for example, a teacher data acquisition unit 321, a model construction unit 322, a convolutional neural network (CNN) 323, and a learning result storage unit 324.

Figure 4 is a diagram showing the relationship between input variables and output variables in the convolutional neural network 323. The input variables are defect type information and object information, and the output variables are lighting condition information. In Figure 4, the input variables and output variables included in each row correspond to one embossed carrier tape 10. For example, the shape information of the embossed carrier tape 10 with product information "1" is a1, the transparency information is b1, the surface property information is c1, and the defect type information is d1, and the position information of the optimal lighting device 100 for inspecting defects in the embossed carrier tape 10 with product information "1" is A1, the wavelength information is B1, and the intensity information is C1.

The teacher data acquisition unit 321 acquires defect type information, object information, and illumination condition information as teacher data from the teacher data storage unit 500. The model construction unit 322 inputs the defect type information and object information to the convolutional neural network 323, and learns the processing parameters of the convolutional neural network 323 so as to output output variables from the convolutional neural network 323. Specifically, the model construction unit 322 uses the teacher data to recursively calculate (update) the processing parameters so as to reduce the difference between the output variables corresponding to the input variables shown in FIG. 4 and the target output. The model construction unit 322 performs, for example, deep learning to acquire the processing parameters. Deep learning is machine learning using a multi-layered neural network, particularly a neural network with three or more layers. In the embodiment, the convolutional neural network 323 is used as the neural network with a multi-layered structure. The model construction unit 322 saves the updated processing parameters in the learning result storage unit 324.

FIG. 5 is a block diagram showing an example of the estimation unit 330 in the embodiment. The estimation unit 330 includes an information acquisition unit 331, an estimation processing unit 332, a convolutional neural network 333, and an estimation result storage unit 334. The information acquisition unit 331 acquires defect type information and/or object information from the information input unit 400. The convolutional neural network 333 is an estimation model in which processing parameters stored in the learning result storage unit 324 are set. When the convolutional neural network 333 receives defect type information and object information, it outputs illumination condition information. The estimation processing unit 332 stores the estimation result output from the convolutional neural network 333 in the estimation result storage unit 334. The estimation result includes information such as the arrangement, wavelength, and intensity of the illumination device 100 as information corresponding to the illumination condition information shown in FIG. 4.

In the above-described embodiment, defect type information and object information are described as input variables, but the learning unit 320 may construct the convolutional neural network 323 so that the object information is an input variable and the network outputs illumination condition information for inspecting defects that may occur in the object indicated by the object information. The learning unit 320 may also construct the convolutional neural network 323 so that the network inputs a portion of the embossed carrier tape 10 as object information and outputs illumination condition information for inspecting defects that may occur in the portion.

FIG. 6 is a top view showing an example of the arrangement of the lighting device 100 in the embodiment. In FIG. 6, for example, one circle indicates one lighting element 100a. The lighting device 100 is configured, for example, by arranging multiple lighting elements 100a radially from a virtual central circle. The lighting device 100 is arranged so that the center of the central circle intersects with the production line L. FIG. 7 is a top view showing another example of the arrangement of the lighting device 100 in the embodiment. The lighting device 100 is configured, for example, by arranging multiple lighting elements 100a along multiple straight lines extending vertically from each side of a virtual rectangle. The lighting device 100 is arranged so that the center of gravity of the rectangle intersects with the production line L. The defect inspection system 1 can, for example, switch the arrangement of FIG. 6 or FIG. 7 based on the estimation result according to the object information of the embossed carrier tape 10, adjust the wavelength of the light irradiated from each lighting element 100a, or adjust the intensity of the light irradiated by each lighting element 100a.

8 and 9 are side views of the lighting device 100 in the embodiment. As shown in FIG. 8, the lighting device 100 has multiple lighting elements 100a arranged along upper and lower arcs relative to the embossed carrier tape 10 transported on the production line L. Each lighting element 100a may be positioned so that the irradiation light is directed toward a predetermined range of the production line L, and may be dimmed so that the irradiation light is directed toward the predetermined range of the production line L. A certain lighting element 100a irradiates the irradiation light to the predetermined range via a mirror 110. The imaging device 200 is arranged in a position to image the predetermined range of the production line L. As shown in FIG. 9, the defect inspection system 1 can also irradiate the embossed carrier tape 10 with irradiation light from some of the multiple lighting elements 100a based on the estimation result, for example.

10 is a top view showing an example of an embossed carrier tape 10 of an embodiment, where (a) is an example of an embossed carrier tape 10 without defects, and (b) is an example of an embossed carrier tape 10 with defects. The embossed carrier tape 10 has a plurality of recesses 10a and holes 10b formed therein. The defect inspection system 1 detects defects such as deformation of the recesses 10a, burrs in the holes 10b, burrs at the ends of the embossed carrier tape 10, foreign matter on the surface of the embossed carrier tape 10, and damage to the surface of the embossed carrier tape 10, as shown in FIG. 10(b). For this purpose, the defect inspection system 1 can estimate optimal lighting conditions for inspecting the type of defect by learning defect type information as an input variable. In addition, the defect inspection system 1 can estimate optimal lighting conditions for inspecting the defect in order to inspect the defect that may occur in the portion of the embossed carrier tape 10.

Figure 11 shows the relationship between the illumination conditions, the defect type, and the correlation data in the embodiment. The waveform in Figure 11 is the correlation data, and the horizontal axis in the correlation data is the correlation value, and the vertical axis is the frequency. The correlation data in Figure 11 includes a waveform corresponding to a good image (right side in the figure) and a waveform corresponding to a defective image (left side in the figure). The good image is an image of an embossed carrier tape 10 without defects. The defective image is an image of an embossed carrier tape 10 with defects. The waveform corresponding to the good image has a peak value corresponding to the correlation value (first correlation value c1) between the reference image and the good image. The waveform corresponding to the defective image has a peak value corresponding to the correlation value (second correlation value c2) between the reference image and the defective image. The reference image is an image of an embossed carrier tape 10 without defects captured in advance under illumination conditions of an arbitrary angle, waveform, and brightness. The first correlation value c1 is a value higher than the second correlation value c2. The more suitable the lighting conditions are for the type of defect, the greater the difference between the first correlation value c1 and the second correlation value c2.

The correlation data includes a first correlation value range between a reference image and an image of an object without defects, and a second correlation value range between the reference image and an image of an object with defects. The first correlation value range and the second correlation value range are narrower the more suitable the lighting conditions are for the type of defect in the object. For example, for defect type a, the correlation value range between the reference image and the good image under lighting condition A is narrower than the correlation value range between the reference image and the good image under lighting condition C.

For example, if the optimal lighting condition for inspecting defect type a is A, the reference image is an image of the embossed carrier tape 10 captured under lighting condition A. The defect inspection unit 310 calculates a correlation value between an image of the embossed carrier tape 10 without defects captured under lighting condition A and the reference image, and if the calculated correlation value is within a predetermined range from the pre-stored correlation value c1, it determines that the embossed carrier tape 10 does not have a defect of defect type a. The defect inspection unit 310 calculates a correlation value between an image of the embossed carrier tape 10 with defect type a captured under lighting condition A and the reference image, and if the calculated correlation value is within a predetermined range from the pre-stored correlation value c2, it determines that the embossed carrier tape 10 has a defect of defect type a.

From the above, it is desirable that the learning unit 320 uses the above-mentioned correlation data to construct the convolutional neural network 323 so that, when defect type information is input, it can output lighting condition information related to lighting conditions that can obtain a low correlation with an image of an object having the defect, and can obtain a high correlation with a reference image obtained by imaging an object without the defect. Furthermore, when defect type information is input, the estimation unit 330 can output lighting condition information related to lighting conditions that can obtain a high correlation with the reference image.

Figure 12 is a diagram showing another configuration of the defect inspection system 1 of the embodiment. The invention according to claim 1 of the present application described above may estimate the optimal lighting conditions for one defect type and control the lighting device 100 based on the estimated lighting conditions, but is not limited to this. The defect inspection system 1 may include a plurality of sets of lighting devices 100 and imaging devices 200 (a set of 100a and 200b, a set of 100b and 200b, ...). The defect inspection system 1 inspects the embossed carrier tape 10 for defect type a, and then inspects the embossed carrier tape 10 for defect type b. At this time, when the defect inspection system 1 receives object information, it estimates the optimal lighting conditions for the defect type a based on the object information, controls the lighting device 100a, and inspects the defect type a based on the image captured by the imaging device 200a, and estimates the optimal lighting conditions for the defect type b based on the object information, controls the lighting device 100b, and inspects the defect type a based on the image captured by the imaging device 200b. This allows the defect inspection system 1 to inspect multiple types of defects.

[Convolutional Neural Network]
Hereinafter, a description will be given of an example of the convolutional neural network 323 and the convolutional neural network 333 in the embodiment. In this description, the convolutional neural network 323 and the convolutional neural network 333 will be collectively referred to as a "convolutional neural network."

As described above, the model construction unit 322 sets the teacher data as an input variable to be input to the input layer of the learning convolutional neural network 323, and sets the lighting condition information as an output variable to be output from the output layer. The model construction unit 322 performs machine learning using a learning data set of defect type information, object information, and lighting condition information. The estimation processing unit 332 inputs object information to the input layer of the trained convolutional neural network 333, and acquires lighting condition information from the output layer.

FIG. 13 is a diagram for explaining the process of performing the estimation process using the convolutional neural network in the embodiment. The convolutional neural network includes, for example, layers L0, L1, L2, Li, and LI. Layer L0 is also called the input layer, layers L1 to Li are also called intermediate layers or hidden layers, and layer LI is also called the output layer. In the convolutional neural network, object information is input to the input layer L0. The defect type information and object information are represented by a matrix D11 in which the parameters of various information are the positions of the matrix. Each element of the matrix D11 includes an R (red) sub-value, a G (green) sub-value, and a B (blue) sub-value as sub-values of the parameters corresponding to the positions of the matrix. The first intermediate layer L1 is a layer in which the convolution process (also called the filtering process) and the pooling process are performed.

(Convolution Processing)
An example of the convolution process of the intermediate layer L1 will be described. The convolution process is a process of applying a filter to the original parameters to output a feature map. Specifically, the input parameters are divided into a submatrix D121 of R, a submatrix D122 of B, and a submatrix D123 of G. Each of the submatrix D121, D122, and D123 (each of which is also referred to as a "submatrix D12") is multiplied by an element of the submatrix of s rows and t columns and an element of a convolution matrix CM1 (also called a kernel) of s rows and t columns for each submatrix of s rows and t columns, and then added to calculate a first parameter value. The first parameter values calculated in each submatrix D12 are multiplied by a weighting coefficient and then added to calculate a second parameter value. The second parameter value is set as each element of the convolution matrix D131 as a matrix element corresponding to the position of the submatrix. In each sub-matrix D12, the position of the submatrix is shifted for each element (sub-parameter), so that the second parameter value at each position is calculated, and all matrix elements of the convolution matrix D131 are calculated.

FIG. 13 shows an example of a convolution neural network using a 3-row, 3-column convolution matrix CM (convolution matrix) 1, and the convolution parameter value D1311 is calculated as a first parameter value for the 3-row, 3-column submatrix from the 2nd row to the 4th row and the 2nd column to the 4th column of each submatrix D12. A weighting factor is calculated and added to each of the first parameter values of the submatrices D121, D122, and D123, and a second parameter value is calculated as the matrix element in the 2nd row and the 2nd column of the convolution matrix D131. Similarly, the second parameter value of the matrix element in the 3rd row and the 2nd column of the convolution matrix D131 is calculated from the submatrix from the 3rd row to the 5th row and the 2nd column to the 4th column. Similarly, the convolution matrix D132, ... is calculated using other weighting factors or other convolution matrices.

(Pooling)
An example of the pooling process of the intermediate layer L1 will be described. The pooling process is a process of reducing a parameter group (matrix) while retaining the characteristics of the defect type information and the target information. Specifically, for each region PM (pooling matrix) of u rows and v columns in the convolution matrix D131, a representative value of the matrix elements in the region is calculated. The representative value is, for example, the maximum value of the matrix elements in the region. The representative value is set to each element of the CNN matrix D141 as a matrix element corresponding to the position of the region PM. The region in the convolution matrix D131 is shifted for each region PM, so that a representative value at each position is calculated, and all elements of the CNN matrix D141 are calculated.

Figure 13 shows an example of a convolutional neural network using a region PM of 2 rows and 2 columns, for example. For the 2 rows and 2 columns region PM from the 3rd to 4th row and the 3rd to 4th column of the convolution matrix D131, the maximum value in the region PM is calculated as a representative value. This representative value is set to the matrix element in the 2nd row and 2nd column of the CNN matrix D141. Similarly, a representative value of the matrix element in the 3rd row and 2nd column of the CNN matrix D141 is calculated from the submatrix from the 5th to 6th row and the 2nd to 4th column. Similarly, CNN matrices D142, ... are calculated from the convolution matrix D132, ....

Each matrix element (N elements) of the CNN matrices D141, D142, ... is arranged in a predetermined order to generate a vector X. The vector X includes N elements _xn (n = 1, 2, 3, ... N). The vector X corresponds to the vector u ⁽⁰⁾ .

The hidden layer Li represents the i-th (i=2, ...) hidden layer. The hidden layer Li includes a vector u ⁽ⁱ⁾ . A vector z ⁽ⁱ⁾ is output from each node of the hidden layer Li. The vector z ⁽ⁱ⁾ has a value input to a function f(u ⁽ⁱ⁾ ) for which the vector u ⁽ⁱ⁾ is an activation function. The vector u ⁽ⁱ⁾ has a value obtained by multiplying the vector z ^(i-1) output from the (i-1)th hidden layer node by the weight matrix W ⁽ⁱ⁾ and adding the value to the vector b ⁽ⁱ⁾ . The vector b ⁽ⁱ⁾ is a bias.

The output layer LI includes a vector z ^(I-1) . The output of the output layer LI is M elements of _ym (m=1, 2, ... M). That is, the output layer LI outputs a vector Y ( _y1 , _y2 , _y3 , ... _yM ) including M elements of _ym . As described above, the convolutional neural network outputs a vector Y as an output variable when parameters of defect type information and object information are input as input variables. The vector Y in the embodiment represents illumination condition information.

Figure 14 is a diagram for explaining the learning process using a convolutional neural network in an embodiment. The vector output from the first intermediate layer L1 for the parameter values of the object information in the learning data set is defined as vector [X]. The vector representing the determined class in the learning data set is defined as vector [Y]. The determined class is the output coefficient (illumination condition information of the arrangement, wavelength, and intensity of the lighting elements) shown in Figure 4 in the embodiment.

An initial value is set for the weight matrix W ⁽ⁱ⁾ . When a vector [X] is input to the second intermediate layer L2 based on the defect type information and the target object information being input to the input layer L0, a vector Y(X) based on the vector [X] is output from the output layer LI. The error E between the vector Y(X) and the vector [Y] is calculated using a loss function. The gradient ΔE _i of the i-th layer is calculated using the output z _i from each layer and the error signal δ _i . The error signal δ _i is calculated using the error signal δ _i-1 . Note that the process of calculating the error signal on the input layer side from the error signal on the output layer side from the output layer side toward the input layer L0 is also called backpropagation. The weight matrix W ⁽ⁱ⁾ is updated based on the gradient ΔE _i . Similarly, in the first intermediate layer L1, the convolution matrix CM or the weight coefficient is updated.

[Estimation model settings]
The model construction unit 322 sets processing parameters for the convolutional neural network, such as the number of layers, the number of nodes in each layer, the node connection method between each layer, the activation function, the error function, the gradient descent algorithm, the pooling region, the kernel, the weighting coefficient, and the weighting matrix. The model construction unit 322 sets, for example, three layers (I=3) as the number of layers. The model construction unit 322 sets, as the number of nodes in each layer (also referred to as the "node number"), the number of elements of the vector X (node number N) to 800, the number of nodes in the second intermediate layer (i=2) to 500, and the number of nodes in the output layer (i=3) to 10. However, the embodiment is not limited to this, and the total number may be four or more layers, and a different value may be set for the number of nodes.

The model construction unit 322 creates 20 convolution matrices CM with 5 rows and 5 columns, and sets a region PM with 2 rows and 2 columns. However, the embodiment is not limited to this, and a different number of matrices or a different number of convolution matrices CM may be set. Also, a region PM with a different number of matrices may be set. The model construction unit 322 may perform more convolution processes or pooling processes.

The model construction unit 322 sets full connections as the connections of each layer of the convolutional neural network. However, the embodiment is not limited to this, and the connections of some or all layers may be set to non-full connections. The model construction unit 322 sets a sigmoid function as the activation function of all layers. However, the embodiment is not limited to this, and the activation function of each layer may be other activation functions such as a step function, linear combination, soft sine, soft plus, ramp function, truncated power function, polynomial, absolute value, radial basis function, wavelet, maxout, etc. Also, the activation function of a certain layer may be a different type from that of other layers.

The model construction unit 322 sets squared loss (mean squared error) as the error function. However, the present invention is not limited to this, and the error function may be cross entropy, τ-quantile loss, Huber loss, or ε-sensitivity loss (ε-tolerance error function). In addition, the model construction unit 322 sets SGD (stochastic gradient descent) as the algorithm (gradient descent algorithm) for calculating the gradient. However, the present invention is not limited to this, and the gradient descent algorithm may be momentum (inertia term) SDG, AdaGrad, RMSprop, AdaDelta, Adam (Adaptive moment estimation), etc.

The model construction unit 322 may set other neural networks, such as a perceptron neural network, a recurrent neural network (RNN), a residual network (ResNet), etc., in addition to a convolutional neural network (CNN). The model construction unit 322 may also set some or all of the supervised learning estimation models, such as a decision tree, a regression tree, a random forest, a gradient boosting tree, a linear regression, a logistic regression, or an SVM (support vector machine).

[Effects of the embodiment]
As described above, according to the embodiment, it is possible to realize a learning device (learning unit 320) including: a teacher data acquisition unit 321 that inputs defect type information indicating the type of defect in the embossed carrier tape 10, lighting condition information related to the lighting conditions under which the lighting device 100 irradiates light corresponding to the type of defect in the embossed carrier tape 10, and object information for identifying the embossed carrier tape 10; and a model construction unit 322 that constructs a model that is machine-learned using the defect type information, lighting condition information, and object information input by the teacher data acquisition unit 321 as teacher data, and that outputs lighting condition information related to the conditions optimal for the embossed carrier tape 10 when the defect type information and/or the object information is input. According to this embodiment, it is possible to acquire lighting condition information optimal for the embossed carrier tape 10 to be measured simply by inputting information about the embossed carrier tape 10 to be measured and the defect type into the model, so that the embossed carrier tape 10 can be inspected with high accuracy.

However, since the accuracy and reproducibility of the inspection change significantly depending on the lighting conditions, it may be difficult to find the optimal lighting conditions. Conventionally, one lighting condition has been improved so that multiple defects that may occur in the embossed carrier tape 10 can be inspected as much as possible, and the improvement of the lighting conditions has been estimated based on the experience of an expert. However, the optimal lighting conditions differ depending on the type of defect, the material such as the color and transparency of the embossed carrier tape 10, the surface property (reflective, glossy or non-glossy), the shape (flat or uneven), the location, and the characteristics of the embossed carrier tape 10 such as the surface shape and back shape of the embossed carrier tape 10, so it may take a long time to estimate the optimal lighting conditions for multiple inspections with high accuracy. In contrast, according to the embodiment, the estimation time for the lighting conditions can be shortened by constructing the above-mentioned model.

In addition, according to the embodiment, the position of the lighting device 100, the light intensity, the light angle, and the light wavelength are used as the lighting condition information, so that it is possible to estimate the optimal position of the lighting device 100, the light intensity, the light angle, and the light wavelength. Furthermore, according to the embodiment, the part of the embossed carrier tape 10 is used as the object information, so that it is possible to estimate the optimal lighting conditions for defect inspection of the part of the embossed carrier tape 10. Furthermore, according to the embodiment, the physical property information of the embossed carrier tape 10, such as the shape, color, transmittance, surface properties, molding method, and mold shape, is used as the object information, so that it is possible to estimate the optimal lighting conditions for defect inspection of the physical property information.

Furthermore, according to the embodiment, an estimation device (estimation unit 330) including an estimation processing unit 332 can be realized. According to this embodiment, it is possible to obtain optimal lighting condition information for the embossed carrier tape 10 to be measured simply by inputting information about the type of defect and the embossed carrier tape 10 to be measured into the model, so that the embossed carrier tape 10 can be inspected with high accuracy.

The program of the defect inspection device 300 in one aspect of the present invention may be a program that controls one or more processors such as a CPU (a program that causes a computer to function) so as to realize the functions shown in each of the above-mentioned embodiments and variations related to one aspect of the present invention. The information handled by each of these devices may be temporarily stored in RAM during processing, and then stored in various storage devices such as flash memory, SSD, and HDD, and may be read, modified, and written by the CPU as necessary.

The learning unit 320 or the estimation unit 330 of the defect inspection device 300 in each of the above-mentioned embodiments and modifications may be partially or entirely realized by a computer having one or more processors. In that case, the control function may be realized by recording a program for realizing the control function in a computer-readable recording medium, and reading and executing the program recorded in the recording medium into a computer system. In addition, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may also include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in such a case. The above-mentioned program may be a program for realizing part of the above-mentioned functions, or may be a program that can be realized in combination with a program already recorded in the computer system.

In addition, a part or all of the learning unit 320 or the estimation unit 330 provided in the defect inspection device 300 in each of the above-mentioned embodiments and modified examples may be realized as an LSI, which is typically an integrated circuit, or as a chip set. Furthermore, the integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit and/or a general-purpose processor. Furthermore, if an integrated circuit technology that can replace LSI appears due to advances in semiconductor technology, it is also possible to use an integrated circuit based on that technology.

Although the above describes in detail each embodiment and modification as one aspect of this invention with reference to the drawings, the specific configuration is not limited to each embodiment and modification, and also includes design changes within the scope of the gist of this invention. Furthermore, various modifications of one aspect of the present invention are possible within the scope of the claims, and embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. Also included are configurations in which elements described in the above embodiments and modifications are substituted with elements that have the same effect.

For example, one aspect of the present invention may be realized by combining some or all of the above embodiments.

The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made without departing from the spirit of the present invention.

1 Defect inspection system 100 Illumination device 200 Imaging device 300 Defect inspection device 310 Defect inspection section 320 Learning section 330 Estimation section 340 Illumination control section 400 Information input section 500 Teacher data storage section 600 Notification section

Claims (10)

an input unit for inputting defect type information indicating a type of defect of an object, lighting condition information regarding lighting conditions under which a lighting device irradiates light corresponding to the type of defect of the object, and object information for identifying the object;
a model construction unit that constructs a model by machine learning using the defect type information, the illumination condition information, and the object information input by the input unit as training data, the model outputting the illumination condition information related to the optimum condition for the object when the defect type information and the object information are input;
A learning device comprising: the illumination condition information is data indicating a relationship between a type of defect of the object, a condition under which the illumination device irradiates light, and correlation data based on a correlation between a reference image and an image representing a defect state of the object,
the model construction unit constructs the model so as to output the illumination condition information relating to an illumination condition that can obtain a high correlation with a reference image when the defect type information is input.
The learning device according to claim 1 . The learning device according to claim 2, wherein the correlation data includes a first correlation value between a reference image and an image of an object without a defect, and a second correlation value between the reference image and an image of an object with a defect, and is information indicating that the first correlation value is higher than the second correlation value. The learning device according to claim 2, wherein the correlation data includes a first correlation value range between a reference image and an image of an object without a defect, and a second correlation value range between the reference image and an image of an object with a defect, and the first correlation value range and the second correlation value range are information indicating that the lighting conditions are narrow enough to be suitable for the type of defect of the object. The learning device according to claim 1, wherein the lighting condition information includes at least one of the following conditions: the arrangement of the lighting device, the intensity of light, the angle of light, and the wavelength of light. The learning device according to claim 1, wherein the object information indicates a part of the object, and the lighting condition information indicates optimal lighting conditions for defect inspection of the part of the object. The learning device of claim 1, wherein the object information indicates physical property information including at least one of the shape, color, transparency, surface properties, molding method, and mold shape of the object, and the lighting condition information indicates optimal lighting conditions for defect inspection of the object indicated by the physical property information. an input unit for inputting defect type information indicating a type of defect in an object and object information for identifying the object;
an estimation unit that inputs the defect type information and the object information input by the input unit into a model that has been machine-learned using teacher data including the defect type information, the object information, and lighting condition information related to lighting conditions under which a lighting device corresponding to a type of defect of the object irradiates light, and estimates optimal lighting conditions for the object based on the lighting condition information related to optimal conditions for the object output from the model;
An estimation device comprising: A lighting device that irradiates light onto an object;
An inspection unit that inspects the object;
an input unit for inputting defect type information indicating a type of defect of the object and object information for identifying the object;
an estimation unit that inputs the defect type information and the object information input by the input unit into a model that has been machine-learned using teacher data including the defect type information, the object information, and lighting condition information related to lighting conditions under which a lighting device corresponding to a type of defect of the object irradiates light, and estimates optimal lighting conditions for the object based on the lighting condition information related to optimal conditions for the object output from the model;
a control unit that controls the lighting device based on the lighting condition estimated by the estimation unit;
A defect inspection device comprising: a step of inputting defect type information indicating a type of defect of an object, lighting condition information regarding lighting conditions under which a lighting device irradiates light corresponding to the type of defect of the object, and object information for identifying the object by a defect inspection device;
constructing a model in which the defect inspection device has machine-learned the input defect type information, the illumination condition information, and the object information using training data, the model outputting the illumination condition information relating to the optimum conditions for the object when the defect type information and the object information are input;
a step of inputting the defect type information and object information for identifying the object to the defect inspection device, inputting the defect type information and the object information to the model, and estimating optimal illumination conditions for the object based on the illumination condition information relating to optimal conditions for the object output from the model;
The defect inspection device controls the lighting device based on the estimated lighting conditions;
The defect inspection device inspects the object;
A defect inspection method including:

Images