JP7708463B2 - Image inspection method and image inspection device - Google Patents

Description

The present disclosure relates to an image inspection method and an image inspection device. In particular, the present disclosure relates to an image inspection method and an image inspection device that detects discrepancies arising from differences in shooting conditions when comparing two images.

Traditionally, in order to provide products such as food and medicines safely to consumers, it has been necessary to manage each product during the manufacturing process. To manage each product, it is necessary to identify each product during the manufacturing process. Typically, to identify an product, an identification number is attached to the product or its packaging container. However, even if an identification number is attached to a container that packages a certain number of products, there are cases in which an identification number is not attached to each individual product.

For example, medicines that may be harmful to humans depending on how they are used may need to be strictly managed in individual tablets to prevent them from being taken out of the country. Commercially available medicines are often sold in a format in which multiple tablets are stored in a packaging container. Each of the multiple tablets stored in the packaging container is packaged using a press through pack (PTP) with a label of the same design, and each packaging label is not assigned an identification number.

In recent years, technology has been developed to identify individual items by accurately distinguishing minute color shading on the surface of the item and/or its packaging label (e.g. uneven or faded printing on the packaging label). The above-mentioned identification technology is able to identify individual items by detecting differences in specific areas that cannot be distinguished by the human eye, even when packaging labels have the same design.

In order to identify individual items (tablets of the above-mentioned medicine) in the actual manufacturing process, it is necessary to generate a reference image in advance by photographing each tablet individually using an imaging device such as a camera. When identifying each tablet, a comparison image is generated by photographing each tablet individually, and the comparison image is compared with the reference image to determine whether the two match, thereby identifying each tablet.

JP 2016-018340 A

In the above-mentioned identification technology, the brightness values of both the reference image and the comparison image are calculated on a pixel-by-pixel basis, and it is determined whether the calculated brightness values are within a certain range to determine whether the two match. In cases where multiple tablets are packaged with packaging labels of the same design, the brightness values are calculated from the shades of the color of the packaging labels (which cannot be distinguished by the human eye). Therefore, if the position and/or inclination of the imaging device is not the same when photographing the tablets to generate the comparison image and when photographing the tablets to generate the reference image, the calculated brightness values will be different, and as a result, the same tablets may be determined not to match.

Patent Document 1 discloses a technique for acquiring an image of an object, extracting the edges of the image, and calculating the amount of blur of the edge based on the brightness difference of pixels within a predetermined area including the extracted edge. The technique disclosed in Patent Document 1 makes it possible to detect the occurrence of tilt of the imaging device when photographing a tablet to generate a comparison image, compared to when photographing the tablet to generate a reference image.

However, the technology disclosed in Patent Document 1 can detect blurring of edge parts of a generated image, but does not detect blurring of other parts.

A computing device according to one embodiment is a computing device connected to an imaging device, which receives a first image including at least a pattern array from the imaging device, the first image being generated by photographing an object to be photographed, calculates a first set of luminance values for the pattern array from the first image, receives a second image including at least the pattern array from the imaging device, the second image being generated by photographing an object identical to the object to be photographed, calculates a second set of luminance values for the pattern array from the second image, calculates a set of difference values between the first set of luminance values and the second set of luminance values, and detects a mismatch between the first image and the second image by comparing the set of difference values with a predetermined threshold.

In another embodiment, a method executed by a computer device includes a computer device connected to an imaging device, the computer device receiving a first image including at least a pattern array from the imaging device, the first image being generated by photographing an object to be photographed, calculating a first coordinate position for the pattern array from the first image, receiving a second image including at least the pattern array from the imaging device, the second image being generated by photographing an object to be photographed that is the same as the object to be photographed, calculating a second coordinate position for the pattern array from the first image, calculating a difference value between the first coordinate position and the second coordinate position, and detecting a mismatch between the first image and the second image by comparing the difference value with a predetermined threshold value.

The image inspection system according to the embodiment can detect discrepancies between two images, and can therefore detect discrepancies that arise from differences in the shooting conditions.

FIG. 1 is a block diagram showing an example of the configuration of an image inspection system. FIG. 13 is a block diagram showing an example of a pattern sheet. 13 is a flowchart illustrating an example of a reference image analysis process. FIG. 13 is a diagram showing an example of an intra-dot region. FIG. 13 is a diagram showing an example of an inter-dot region. 11A and 11B are diagrams illustrating examples of approximation straight lines of dot rows and dot columns. 11A and 11B are diagrams illustrating an example of a state in which a subject is photographed using a pattern sheet. 13 is a flowchart illustrating an example of a comparative image analysis process. FIG. 2A is a diagram showing an example of a pattern arrangement in a reference image, and FIG. 2B is a diagram showing an example of a pattern arrangement in a comparison image. FIG. 13 is a diagram showing an example of a state in which dot rows are compared. FIG. 2A is a diagram showing an example of a pattern arrangement in a reference image, and FIG. 2B is a diagram showing an example of a pattern arrangement in a comparison image. FIG. 1A is a diagram showing an example of an inter-dot region between adjacent dots in a reference image, and FIG. 1B is a diagram showing an example of an inter-dot region between adjacent dots in a comparison image.

Below, an image inspection system according to one embodiment will be described in detail with reference to the attached drawings. For example, in a process of identifying an object based on two images generated by photographing the object, the image inspection system detects discrepancies between the two images resulting from differences in the photographing conditions. Hereinafter, in this embodiment, any object to be photographed will be referred to as a "photographed subject."

For example, in a pharmaceutical tablet manufacturing process, in order to identify each tablet individually, a previously generated image of each tablet (reference image (first image)) is compared with an image generated during the actual manufacturing process (comparison image (second image)). In order for two images generated by photographing the same object (tablet) to match, the imaging device needs to photograph areas in the same relative positions within the object when generating the two images.

In addition, when generating two images, the imaging device must capture the subject at the same position and angle. If the above-mentioned conditions are not met, two images generated by capturing images of the same subject may not match. Furthermore, even if the above-mentioned conditions are met, two images generated by capturing images of the same subject may not match due to differences in brightness at the locations where the images are captured when the two images are generated.

The image inspection system detects the above-mentioned inconsistencies, for example, at the early stages of the pharmaceutical manufacturing process, and therefore detects differences in the shooting conditions. The inconsistencies between the two images are detected based on the brightness values of the pattern arrays in the images, etc.

Differences in shooting conditions are factors that cause differences in two images generated by photographing the same subject. As described above, differences in shooting conditions include differences in the position of the shooting area when photographing the subject to generate the two images (reference image and comparison image), differences in the position and/or inclination of the imaging device, and differences in the brightness of the location where the images are taken.

First, the configuration of the image inspection system 100 will be described with reference to FIG. 1. The image inspection system 100 shown in FIG. 1 includes a computer device 1, an imaging device 2, an input device 3, and an output device 4. The computer device 1 is connected to each of the imaging device 2, the input device 3, and the output device 4.

The computer device 1 is an information processing device including at least a processor, a memory, a storage device, and an input/output device. The computer device 1 may be a general-purpose computer such as a personal computer. The processor reads a program stored in the storage device into the memory and executes the program. The program is a computer program including computer-readable instructions that implement the image inspection method according to this embodiment. The input/output device inputs and outputs data between the imaging device 2, the input device 3, and the output device 4, which will be described later.

The imaging device 2 is a camera for photographing an object, including one or more CCD image sensors or CMOS image sensors. The imaging device 2 is coupled to the computer device 1, and transmits an image generated by photographing the object to the computer device 1. In order to generate an image with high pixel count, it is desirable for the imaging device 2 to have more image sensors arranged. The imaging device 2 also includes an array of LEDs (not shown). The LED array irradiates the object with light. By providing the imaging device 2 with an array in which more LEDs are evenly arranged, the entire object can be irradiated with light, generating a clearer image.

The input device 3 assists the user in inputting information when the user interacts with the computer device 1. The input device 3 is implemented by a mouse and a keyboard, etc. Similarly, the output device 4 assists the user in outputting information to the user when the user interacts with the computer device 1. The output device 4 is implemented by a display screen, etc.

In this embodiment, the computer device 1 and the imaging device 2 are directly connected to each other, but the present invention is not limited to such an example. The computer device 1 and the imaging device 2 may be connected to each other through a network.

Although not shown, the image inspection system 100 may include a support member (holder) for supporting the imaging device 2. The support member supports the imaging device 2 so that the focal point of the imaging device 2 is on the subject to be photographed. The image inspection system 100 may also include a jig for fixing the subject to be photographed. The jig fixes the subject to be photographed so that the focal point of the imaging device 2 is on a specific area within the subject to be photographed.

The imaging device 2 may also include an acceleration sensor and a gyro sensor. The acceleration sensor and the gyro sensor detect the angular velocity when the imaging device 2 is tilted, and transmit the angular velocity signal to the computing device 1.

Furthermore, the image inspection system 100 may include a brightness sensor. The brightness sensor is installed at any position within the environment in which the imaging device 2 captures the subject. The brightness sensor detects the brightness in the environment and transmits it as a brightness signal to the computing device 1.

Next, an example of a pattern sheet PS will be described with reference to FIG. 2. The pattern sheet PS is photographed by the imaging device 2 in order to detect mismatches between two images. The pattern sheet PS includes a pattern array PA. The pattern array PA includes a collection of dots formed at equal intervals. It is desirable that the dots in the pattern array PA are densely arranged in order to increase the accuracy of detecting differences in the photographing conditions. In this embodiment, the dots are arranged at equal intervals in units of 1 mm.

In this embodiment, the pattern array PA corresponds to a set of dots, but it may be a set of patterns of any shape other than dots. For example, the pattern array PA may be a set of patterns of any shape, such as a square, triangle, or diamond.

The imaging device 2 generates a reference image and a comparison image by photographing the pattern sheet PS. For example, when the image inspection system 100 is applied to a manufacturing process of an article (to identify individual articles), the image inspection system 100 generates a reference image in advance by photographing the pattern sheet PS. Then, prior to the actual manufacturing process, the image inspection system 100 generates a comparison image by photographing the same pattern sheet PS as when the reference image was generated. By comparing the brightness values, etc., of the two images generated by photographing the same pattern sheet PS, discrepancies between the two images are detected. Details will be described later.

Next, an example of the reference image analysis process executed by the image inspection system 100 will be described with reference to the flowchart shown in FIG. 3. In the reference image analysis process, the imaging device 2 captures an image of a specific area of the pattern sheet PS to generate a reference image. The specific area of the pattern sheet PS corresponds to an area that includes at least the pattern array PA. Hereinafter, this specific area is referred to as the capture area IA. Furthermore, the pattern sheet PS corresponds to the capture target O.

First, the imaging device 2 captures an image of the imaging area IA of the imaging target O (pattern sheet PS) (step 301). In this embodiment, the entire area of the pattern array PA shown in FIG. 2 corresponds to the imaging area IA.

Next, the imaging device 2 generates an image corresponding to the imaging area IA (step 302). This image serves as a reference image RI to be compared with a comparison image described later. Once the reference image RI is generated, the imaging device 2 transmits the reference image RI to the computer device 1, and the control device of the computer device 1 receives the reference image RI via the input/output device.

The control device of the computing device 1 then stores the reference image RI in a storage device (step 303). The control device then reads out the reference image RI and identifies the individual dots in the pattern array PA (step 304). The dots may be identified, for example, by binarizing the reference image RI and labeling the binarized image. This process recognizes the circular portions of all dots in the pattern array PA. That is, a set of coordinate positions of pixels corresponding to the outer perimeter of the dots is calculated.

Next, the control device calculates the center coordinates of each identified dot (step S75). The center coordinates are derived, for example, by calculating the diameter of the dot from the coordinate position of the outer frame portion of the dot identified in step 304, and then calculating the center coordinates of that diameter.

Next, the control device calculates an area within a predetermined range within the dot from the center coordinate calculated in step 305 (step 306). In this embodiment, this area corresponds to a square area having a predetermined size centered on the center coordinate. This area is referred to as the "intra-dot area (intra-pattern area) DA _intra ". Through this process, a set of coordinate positions of pixels corresponding to the intra-dot area DA _intra is calculated.

Fig. 4 shows an example of the intra-dot area DA _intra . As shown in Fig. 4, the intra-dot area DA _intra corresponds to a square area having a predetermined size centered on central coordinates (intra-dot central coordinates (intra-pattern central coordinates) CC _intra ) in the dot D. In this embodiment, the intra-dot area DA _intra corresponds to a square, but may be any shape other than a square.

Next, the control device calculates the luminance values for all pixels in the intra-dot area DA _intra , and calculates the average (intra-dot average luminance value (intra-pattern average luminance value)) (step 307). The luminance value is calculated by taking a weighted average of each pixel's RGB values according to equation (1). The coefficients used in equation (1) (0.298912 for R, 0.586611 for G, and 0.114478 for B) are derived from empirical rules about how the human eye sees things.
Brightness value = 0.298912 × R + 0.586611 × G + 0.114478 × B (1)

Next, the control device calculates an area within a predetermined range between adjacent dots from the coordinates identified in step 304 and the center coordinates calculated in step 305 (step 308). In this embodiment, this area corresponds to a square area having a predetermined size centered on the inter-dot center coordinates calculated by calculating the inter-dot center coordinates (inter-pattern center coordinates) between the center coordinates of each of the two adjacent dots. This area is called the "inter-dot area (inter-pattern area) DA _inter ". Through this process, a set of coordinate positions of pixels corresponding to the inter-dot area DA _inter is calculated.

Fig. 5 shows an example of the inter-dot area DA _inter . As shown in Fig. 5, the inter-dot area DA _inter corresponds to a square area having _a predetermined size centered on the central coordinate ( _inter -dot central coordinate CC _inter ) between the intra-dot central coordinates (intra-dot central coordinate CC _intra1 and intra-dot central coordinate CC _intra2 ) of adjacent dots (dot D 1 and dot D 2 ). In this embodiment, the inter-dot area DA _inter corresponds to a square, but may be any shape other than a square.

Next, the control device calculates the luminance value for all pixels in the inter-dot area DA _inter , and calculates the average (inter-dot average luminance value (inter-pattern average luminance value)) (step 309). The luminance value is calculated according to the above-mentioned formula (1).

Next, the control device repeats the process of steps 305 to 307 for all dots in the pattern array PA in the reference image RI that were recognized in step 304 (step 310). This process calculates a set of average intra-dot luminance values for all dots in the pattern array PA. Each calculated intra-dot average luminance value is stored in the storage device in association with a value indicating each dot in the pattern array PA (e.g., a dot identification number (pattern identification number)).

The control device also repeats the processes of steps 308 and 309 for all adjacent dots in the pattern array PA in the reference image RI that were recognized in step 304 (step 311). This process calculates a set of average inter-dot luminance values for all adjacent dots in the pattern array PA. Each of the calculated inter-dot average luminance values is stored in the storage device in association with a value indicating each inter-dot set in the pattern array PA (e.g., an inter-dot set identification number (inter-pattern set identification number)).

The control device further identifies a set of dots (dot row (pattern row)) at the X coordinate in the pattern array PA, and calculates an approximation line connecting the center coordinates of each dot in the dot row (step 312). Figure 6 shows an example of an approximation line for a specific dot row.

As described above, in the pattern array PA, the dots are evenly spaced. Therefore, as shown in FIG. 6, in the dot rows in the pattern array PA, the dots are arranged to form a straight line in the direction of the arrow X. This straight line corresponds to the approximated straight line.

This process is repeated for all rows in the pattern array PA. This process calculates a set of approximation lines for all dot rows in the pattern array PA. The calculated approximation lines are stored in a storage device in association with values indicating each dot row in the pattern array PA (e.g., dot row identification numbers (pattern row identification numbers)).

Similarly, the control device identifies a set of dots (a dot row (a pattern row)) at the Y coordinate in the pattern array PA, and calculates an approximation line connecting the center coordinates of each dot in the dot row (step 313). Figure 6 shows an example of an approximation line for a specific dot row.

As described above, in the pattern array PA, the dots are evenly spaced. Therefore, as shown in FIG. 6, in the dot row in the pattern array PA, the dots are arranged to form a straight line in the direction of the arrow Y. This straight line corresponds to the approximated straight line.

This process is repeated for all columns in the pattern array PA. This process calculates a set of approximation lines for all dot columns in the pattern array PA. The calculated approximation lines are stored in a storage device in association with a value indicating each dot column in the pattern array PA (e.g., a dot column identification number (pattern column identification number)).

By the reference image analysis process described above, the average intra-dot luminance value, the average inter-dot luminance value, and the approximate straight line for all dots in the pattern array PA in the reference image RI are stored in the storage device of the computer device 1. In the comparison image analysis process described below, the average intra-dot luminance value, the average inter-dot luminance value, and the approximate straight line for all dots in the array PA are similarly calculated for the comparison image. Then, the average intra-dot luminance value, the average inter-dot luminance value, and the approximate straight line for the comparison image are compared with the average intra-dot luminance value, the average inter-dot luminance value, and the approximate straight line for the reference image.

In the reference image analysis process, an example has been shown in which the imaging device 2 captures the pattern sheet PS as the subject O, but the present invention is not limited to such an example. For example, the pattern sheet PS may exist separately from the subject O (such as a pharmaceutical tablet) and be captured overlapping the subject O. The same applies to the comparative image analysis process described below. In this case, the pattern sheet PS is composed of a transparent sheet.

Figure 7 shows an example of a state in which a pattern sheet PS is used to photograph an object O. As shown in Figure 7, the pattern sheet PS is placed between the imaging device 2 and the object O. Light L irradiated from the imaging device 2 passes through the pattern sheet PS and reaches the object O. The images (reference image and comparison image) generated by photographing the object O in this manner will include the pattern array PA and the packaging label, for example, if the object O is a pharmaceutical tablet (packaging label).

In addition, instead of using a pattern sheet PS, a pattern arrangement may be formed in advance on the object to be photographed O. In this case, if the object to be photographed O is a pharmaceutical tablet (packaging label), the pattern arrangement will be formed on the packaging sheet.

Next, the comparative image analysis process executed by the image inspection system 100 will be described with reference to the flowchart shown in FIG. 8. The comparative image analysis process is executed, for example, at a preliminary stage of the pharmaceutical manufacturing process. In the comparative image analysis process, the imaging device 2 captures an image of the same imaging area IA of the pattern sheet PS as the reference image RI, thereby generating a comparative image.

In principle, a comparison image generated by photographing the same pattern sheet PS as the pattern sheet PS on which the reference image RI is based will match the reference image RI. However, due to differences in the photographing conditions between photographing the pattern sheet PS to generate the reference image RI (hereinafter referred to as the first photographing) and photographing the pattern sheet PS to generate the comparison image (hereinafter referred to as the second photographing), the two images may not match.

In the process shown in FIG. 8, the brightness values of the comparison image and the reference image are compared to detect discrepancies between the two. By performing this process prior to the drug manufacturing process, differences in the shooting conditions can be detected. By detecting differences in the shooting conditions prior to the manufacturing process, measures such as correcting the imaging device 2 to an appropriate position can be taken.

First, the imaging device 2 captures an image of the imaging area IA of the imaging target O (pattern sheet PS) (step 801). The imaging area IA corresponds to the same area as the imaging area IA in the reference image RI described in FIG. 3.

Next, the imaging device 2 generates an image corresponding to the imaging area IA (step 802). This image corresponds to a comparison image CI to be compared with the reference image RI. Once the comparison image CI is generated, the imaging device 2 transmits the comparison image CI to the computer device 1, and the control device of the computer device 1 receives the reference image RI via the input/output device.

Next, in the computer device 1, the processes of steps 803 to 813 are executed on the comparison image CI, but these processes are similar to the processes of steps 303 to 313 described in FIG. 3, so the description is omitted. In other words, the control device of the computer device 1 calculates the average intra-dot luminance value, the average inter-dot luminance value, and the approximation line for all dots in the pattern array PA in the comparison image CI.

The control device then compares the approximation line for the specific dot row calculated in step 812 with the approximation line for the corresponding dot row calculated for the reference image, and calculates the difference between the two. It also determines whether this difference is within a predetermined threshold (step 814). The difference between the approximation lines for the two images (reference image RI and comparison image CI) is calculated for all dot rows, and it is determined whether it is within the threshold.

Similarly, the control device compares the approximation line for the specific dot row calculated in step 810 with the approximation line for the corresponding dot row calculated for the reference image, and calculates the difference between the two. It also determines whether this difference value is within a predetermined threshold value (step 815). The difference value of the approximation lines for the two images is calculated for all dot rows, and it is determined whether it is within the threshold value.

With reference to Figures 9 and 10, an example of the difference in the approximation line between a row of dots in the reference image RI and a row of dots in the comparison image CI is described.

Figure 9(a) shows an example of the state during the first shooting, and the pattern array PA in the reference image RI generated as a result. The arrows in the pattern array PA indicate approximate straight lines for specific dot rows. As shown in Figure 9(a), during the first shooting, the imaging device 2 faces directly toward the pattern sheet PS, which is the subject of the shooting, and shoots it. As a result, in the generated reference image RI, the pattern array PA is located almost in the center of the reference image RI.

Figure 9(b) shows an example of the state during the first shooting, and the pattern array PA in the comparison image CI generated as a result. The arrows in the pattern array PA indicate the approximation line for a specific dot row. This approximation line corresponds to the approximation line shown in Figure 9(a). As shown in Figure 9(b), during the first shooting, the imaging device 2 shoots at an angle with respect to the pattern sheet PS to be shot. As a result, in the generated comparison image CI, the pattern array PA is positioned laterally shifted in the comparison image CI compared to the pattern array in the comparison image RI.

Figure 10 shows an example of a comparison between an approximation line for a specific dot row calculated for the reference image RI (reference image approximation line RA) and an approximation line for the corresponding dot row calculated for the comparison image CI (comparison image approximation line CA). In Figure 10, the reference image approximation line RA and the comparison image approximation line CA are shown on the same plane to clearly show that the comparison image approximation line CA is shifted to the side. Note that to distinguish between the two, the dot rows in the reference image RI are shown filled in black, and the dot rows in the comparison image CI are shown shaded.

As shown in FIG. 10, when the reference image approximation line RA and the comparison image approximation line CA are compared, they are separated by a difference Di. Since the reference image approximation line RA serves as the reference, the difference Di means that the comparison image approximation line CA is shifted to the side by Di. A difference value indicating this difference is calculated for all dot rows and dot columns in steps 814 and 815.

In this embodiment, the approximate straight lines for the dot rows and dot columns in the pattern array are compared, but the present invention is not limited to such an example. For example, an approximate straight line (e.g., a straight line inclined at approximately 45 degrees) connecting any two points in the pattern array PA may be calculated for both the reference image RI and the comparison image CI, and the two may be compared.

Also, for example, the central coordinates of each dot in the reference image RI may be compared with the central coordinates of each dot in the comparison image CI. In this case, although the calculation load increases, it is possible to detect, for example, that a mismatch occurs only in a specific area within the image. In other words, in the processes of steps 814 and 815, the coordinate positions of the dots in the reference image RI and the dots in the comparison image CI are compared.

By processing steps 814 and 815, it is possible to detect an image mismatch between the reference image RI and the comparison image CI. As described above, this mismatch is caused by the imaging device 2 being tilted with respect to the pattern sheet PS in the second shooting compared to the first shooting. Therefore, by detecting a mismatch between the images, it is possible to determine the difference in the shooting conditions.

Returning to the explanation of FIG. 8, the control device compares the average dot-to-dot luminance value and the average dot-to-dot luminance value for a specific dot calculated in step 808 with the average dot-to-dot luminance value and the average dot-to-dot luminance value for the corresponding dot calculated for the reference image, and calculates a difference between the two. It also determines whether this difference value is within a predetermined threshold value (step 816). The average dot-to-dot luminance value and the difference value between the average dot-to-dot luminance value for the two images (reference image RI and comparison image CI) are calculated for all dots, and it is determined whether they are within the threshold value.

With reference to Figures 11 and 12, an example of the difference in the intra-dot average luminance value and the inter-dot average luminance value between the dots in the pattern array PA in the reference image RI and the dots in the pattern array PA in the comparison image CI will be described. Note that the example shown in Figures 11 and 12 is a different example from the example shown in Figures 9 and 10.

Figure 11(a) shows an example of a pattern array PA in a reference image RI. As shown in Figure 11(a), each dot in the pattern array PA has approximately the same size.

Figure 11(b) shows an example of the pattern array PA in the comparison image CI. As shown in Figure 11(b), the size of each dot in the pattern array PA is larger in the area surrounded by the dashed line compared to the dots in the reference image RI. This is due to, for example, the distance between the imaging device 2 and the pattern sheet PS during the first shooting being different from that distance at the location during the second shooting.

Fig. 12(a) shows an example of a reference image inter-dot area DA _RDinter between adjacent dots in a reference image RI, and Fig. 12(b) shows an example of a comparison image inter-dot area DA _CDinter between adjacent dots in a comparison image CI.

12, the reference image dot-to-dot region DA _RDinter does not overlap with either of the two dots (reference image dots _RD1 and _RD2 ) in the reference image RI. In other words, there is no black portion in the reference image dot-to-dot region DA _RDinter .

On the other hand, the comparative image inter-dot area DA _CDinter partially overlaps with both of the two dots (comparison image dots _CD1 and _CD2 ) in the comparative image CI. In other words, there is a black portion in the comparative image inter-dot area DA _CDinter .

Therefore, the average inter-dot luminance value for the inter-dot region DA _RDinter of the reference image differs from the average inter-dot luminance value for the inter-dot region DA _CDinter of the comparison image by a certain difference value. By setting a threshold within this difference value range, the difference between the two will exceed the threshold value.

By setting the threshold value as described above, since the reference image dot _RD1 and the comparison image dot _CD1 have substantially the same central coordinates, the difference value between the average intra-dot luminance values for the intra-dot area DA _intra of both dots does not exceed the threshold value. The same is true for the reference image dot _RD2 and the comparison image dot _CD2 .

On the other hand, when the comparison image dot inter-area DA _CDinter is compared with the reference image dot inter-area DA _RDinter , the difference between the average luminance values of the two exceeds the threshold value, and as a result, a mismatch between the reference image and the comparison image is detected.

By processing step 816, as described above, it is possible to detect an image mismatch between the reference image RI and the comparison image CI even when the central coordinates of the dots in the two images are approximately the same but the sizes of the dots are different. As described above, this mismatch is caused by the difference in the distance between the imaging device 2 and the pattern sheet PS during the first and second shooting. Therefore, by detecting a mismatch between the images, it is possible to determine the difference in the shooting conditions.

In this embodiment, the average intra-dot luminance value and the average inter-dot luminance value are compared for all dots in the pattern array PA for the reference image RI and the comparison image CI, but it is not necessary to compare all dots. For example, the comparison may be performed only for dots within a specified area in both the reference image RI and the comparison image CI.

In addition, in this embodiment, the approximate straight lines, the average intra-dot luminance value, and the average inter-dot luminance value for the dot rows and dot columns of the reference image RI and the comparison image CI are compared, but it is not necessary to compare all of them. Any one or more of the approximate straight lines, the average intra-dot luminance value, and the average inter-dot luminance value for the dot rows and dot columns may be compared.

As described above, the image inspection system according to the embodiment has been described. According to this embodiment, it is possible to detect discrepancies occurring in any area within two images.

The difference values for the above-mentioned approximate straight line, the average intra-dot luminance value, and the average inter-dot luminance value may be output to an output device 4 coupled to the computer device 1. By outputting the difference values, the user can, for example, determine to what extent the imaging device 2 is tilted, and can take this into consideration when retaking the second image capture.

In addition, during the first or second shooting, the tilt of the imaging device 2 and/or the brightness of the location where the image is shot may be detected and stored in the storage device of the computing device 1 in association with the difference values for the approximation line, the average brightness value within the dot, and the average brightness value between the dots. In this way, the tilt of the imaging device 2 corresponding to the difference value can be output to the output device 4 and fed back to the user.

Please note that the hardware components described in the above embodiment are merely illustrative and other configurations are possible. Also, the steps described in the above embodiment do not necessarily have to be performed in the order described, but may be performed in any order. Furthermore, additional steps may be added without departing from the basic concept of the present invention.

The image inspection system 100 according to one embodiment of the present invention is implemented by a computer program executed by the computer device 1, and the computer program may be stored in a non-transitory storage medium. Examples of non-transitory storage media include read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disk devices, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs).

Claims (5)

A computing device coupled to an imaging device, comprising:
receiving a first image including a pattern array that is a set of patterns having an arbitrary shape from the imaging device, the first image being generated by photographing an object to be photographed;
calculating a first coordinate position, which is a central coordinate of each pattern in the pattern array, from the first image;
receiving a second image including at least the pattern array from the imaging device, the second image being generated by photographing an object identical to the object to be photographed;
calculating second coordinate positions, which are center coordinates of each of the patterns in the pattern array, from the second image;
Calculating a difference between the first coordinate position and the second coordinate position;
detecting a mismatch between the first image and the second image by comparing the difference value with a predetermined threshold.
A computing device comprising: A computing device coupled to an imaging device, comprising:
receiving a first image including a pattern array that is a set of patterns having an arbitrary shape from the imaging device, the first image being generated by photographing an object to be photographed;
calculating a first approximation line connecting a plurality of patterns in the pattern array from the first image based on first coordinates that are center coordinates of each pattern in the pattern array;
receiving a second image including at least the pattern array from the imaging device, the second image being generated by photographing an object identical to the object to be photographed;
calculating a second approximation line connecting a plurality of patterns in the pattern array from the second image based on second coordinates that are center coordinates of each of the patterns in the pattern array;
detecting a mismatch between the first image and the second image by comparing the first approximation line with the second approximation line;
A computing device comprising: 1. A method performed by a computing device in communication with an imaging device, comprising:
receiving a first image including a pattern array, which is a set of patterns having an arbitrary shape, from the imaging device, the first image being generated by photographing an object;
calculating a first coordinate position, which is a central coordinate of each pattern in the pattern array, from the first image;
receiving a second image from the imaging device, the second image including at least the pattern array, the second image being generated by photographing an identical subject to the subject;
calculating, from the second image, second coordinate positions that are center coordinates of each of the patterns in the pattern array;
calculating a difference value between the first coordinate position and the second coordinate position;
detecting a mismatch between the first image and the second image by comparing the difference value with a predetermined threshold;
23. A method comprising: 1. A method performed by a computing device in communication with an imaging device, comprising:
receiving a first image including a pattern array, which is a set of patterns having an arbitrary shape, from the imaging device, the first image being generated by photographing an object;
calculating a first approximation line connecting a plurality of patterns in the pattern array from the first image based on first coordinates that are center coordinates of each pattern in the pattern array;
receiving a second image from the imaging device, the second image including at least the pattern array, the second image being generated by photographing an identical subject to the subject;
calculating, from the second image, a second approximation line connecting a plurality of patterns in the pattern array based on second coordinates that are center coordinates of each of the patterns in the pattern array;
detecting a mismatch between the first image and the second image by comparing the first approximation line with the second approximation line;
23. A method comprising : A computer program comprising computer executable instructions which, when executed by a computing device, cause the computing device to carry out a method according to claim 3 or 4 .