JP7396160B2 - Inspection equipment, inspection program and inspection method - Google Patents

Description

The present invention relates to an inspection device, an inspection program, and an inspection method.

BACKGROUND ART Conventionally, in appearance inspections when shipping specific products (for example, products such as fruits, vegetables, bread, flowers, etc.), pass/fail judgments have been made by focusing on the color of the product's appearance. For this reason, in an inspection device that performs a visual inspection of the item, for example, by matching a target image taken of the item with a predetermined model image (an image taken in advance of an item that is determined to be acceptable), it is possible to determine whether the item has passed or failed. Judgment is being made.

Japanese Patent Application Publication No. 2005-4612 Special Publication No. 2004-502250

However, in the case of the above-mentioned articles, even if the article is determined to pass the visual inspection, the color of the exterior is not uniform. This is because, for example, there is a certain degree of individual difference in the external color, such as uneven baking of bread or the pattern of a flower, and completely the same color cannot be reproduced. For this reason, there is a limit to the inspection accuracy in the inspection method of matching with a predetermined model image. In addition, in the case of the above-mentioned articles, there is a problem in that even if the appearance of the entire article is compared, local abnormalities are buried in individual differences among the articles and are difficult to detect.

One aspect of the present invention is to improve inspection accuracy in appearance inspection of articles.

According to one aspect, the inspection device includes:
a dividing unit that divides a circumscribed rectangular area of the item, extracted from a target image obtained by photographing the item to be inspected, into a plurality of area images;
a classification unit that classifies the plurality of area images into a plurality of groups according to a distance from a reference position of the circumscribed rectangular area;
a calculation unit that calculates the ratio of the number of pixels of each color for each group by totaling the pixels included in the area image classified into each group for each color;
The ratio of the number of pixels of each color in each group calculated in advance for a model image taken of an article that is determined to be acceptable is compared with the ratio of the number of pixels of each color in each group calculated by the calculation unit, and the ratio of the number of pixels of each color in each group calculated by the calculation unit is compared. and a comparison section that determines whether the target article passes or fails.

Inspection accuracy can be improved in appearance inspection of articles.

FIG. 1 is a diagram showing an example of a system configuration of an inspection system. It is a diagram showing an example of the hardware configuration of the inspection device. FIG. 3 is a diagram showing an example of a functional configuration of a generation unit of the inspection device. FIG. 7 is a diagram illustrating a specific example of processing by an image data acquisition unit, a compression unit, and an RGB value conversion unit. FIG. 7 is a diagram illustrating a specific example of processing by an area dividing unit. FIG. 2 is a first diagram showing a specific example of processing by a group generation unit. FIG. 6 is a diagram illustrating a specific example of processing by a color ratio calculation unit. FIG. 7 is a diagram illustrating a specific example of processing by a registration unit. It is a figure showing an example of the functional composition of the judgment part of an inspection device. FIG. 7 is a diagram illustrating a specific example of processing by a comparison unit. It is a figure which shows an example of a comparison result. 7 is a flowchart showing the flow of reference data generation processing. It is a flowchart which shows the flow of determination processing. FIG. 2 is a second diagram showing a specific example of processing by a group generation unit.

Each embodiment will be described below with reference to the accompanying drawings. Note that, in this specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, thereby omitting redundant explanation.

[First embodiment]
<System configuration of inspection system>
First, the system configuration of an inspection system including the inspection apparatus according to the first embodiment will be described. FIG. 1 is a diagram showing an example of the system configuration of an inspection system.

As shown in FIG. 1, the inspection system 100 includes an imaging device 110 and an inspection device 120. The imaging device 110 and the inspection device 120 are connected via wire or wirelessly.

The imaging device 110 transmits image data including a model image taken in advance of an article determined to be acceptable to the inspection device 120. In addition, the imaging device 110 transmits image data including target images taken of the inspection target articles 141, 142, 143, . . . to the inspection device 120. In the first embodiment, it is assumed that the items 141, 142, 143, .

The inspection device 120 processes image data transmitted from the imaging device 110. An inspection program including a generation program and a determination program is installed in the inspection device 120, and by executing the program, the inspection device 120 functions as a generation section 121 and a determination section 122.

The generation unit 121 generates reference data based on the model image included in the image data. Furthermore, the generation unit 121 stores the generated reference data in the reference data storage unit 123.

The determination unit 122 determines whether the article to be inspected passes or fails based on the target image included in the image data. Note that the determination unit 122 refers to the reference data stored in the reference data storage unit 123 when determining whether the article to be inspected passes or fails.

<Hardware configuration of inspection device>
Next, the hardware configuration of the inspection device 120 will be explained. FIG. 2 is a diagram showing an example of the hardware configuration of the inspection device. As shown in FIG. 2, the inspection device 120 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, and a RAM (Random Access Memory) 203. CPU201, ROM202, and RAM203 form what is called a computer.

The inspection device 120 also includes an auxiliary storage device 204, an operating device 205, a display device 206, an I/F (Interface) device 207, and a drive device 208. Note that each piece of hardware of the inspection device 120 is interconnected via a bus 209.

The CPU 201 is an arithmetic device that executes various programs installed in the auxiliary storage device 204 (for example, an inspection program including a generation program and a determination program).

ROM202 is a nonvolatile memory. The ROM 202 functions as a main storage device that stores various programs, data, etc. necessary for the CPU 201 to execute various programs installed in the auxiliary storage device 204 . Specifically, the ROM 202 stores boot programs such as BIOS (Basic Input/Output System) and EFI (Extensible Firmware Interface).

The RAM 203 is a volatile memory such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory). The RAM 203 functions as a main storage device that provides a work area in which various programs installed in the auxiliary storage device 204 are expanded when executed by the CPU 201 .

The auxiliary storage device 204 is an auxiliary storage device that stores various programs, data generated by executing the various programs, and the like. For example, the reference data storage unit 123 is implemented in the auxiliary storage device 204.

The operating device 205 is an input device for the administrator of the inspection device 120 to input various instructions to the inspection device 120. The display device 206 is a display device that displays the internal state of the inspection device 120 and the like. The I/F device 207 is a connection device for connecting to the imaging device 110.

The drive device 208 is a device for setting the recording medium 210. The recording medium 210 here includes a medium for recording information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, or a magneto-optical disk. Further, the recording medium 210 may include a semiconductor memory or the like that electrically records information, such as a ROM or a flash memory.

Note that the drive device 208 may be used when installing various programs in the auxiliary storage device 204. Specifically, the administrator of the inspection device 120 sets the distributed recording medium 210 in the drive device 208, and the drive device 208 reads various programs recorded on the recording medium 210, so that the auxiliary storage device 204 Various programs may be installed on the computer.

<Functional configuration of the generation section of the inspection device>
Next, the functional configuration of the generation unit 121 of the inspection device 120 will be explained. FIG. 3 is a diagram illustrating an example of the functional configuration of the generation unit of the inspection device. As shown in FIG. 3, the generation unit 121 includes an image data acquisition unit 301, a compression unit 302, an RGB value conversion unit 303, an area division unit 304, a group generation unit 305, a color ratio calculation unit 306, and a registration unit 307.

The image data acquisition unit 301 acquires a model image from among the image data transmitted from the imaging device 110, and extracts a circumscribed rectangular area of the article determined to be acceptable from the acquired model image, thereby generating a rectangular image. Output.

The compression unit 302 divides the rectangular image into partial images having a predetermined number of pixels, and combines the predetermined number of pixels included in each partial image into one pixel. At this time, the compression unit 302 calculates a pixel value (RGB value) of 1 for each partial image using the pixel values of a predetermined number of pixels included in each partial image. Thereby, the compression unit 302 compresses the rectangular image into an image in which each partial image has one pixel, and generates a compressed rectangular image.

The RGB value conversion unit 303 converts the pixel values (RGB values) of each partial image of the compressed rectangular image to generate a color-converted rectangular image. Furthermore, the RGB value conversion unit 303 notifies the area division unit 304 of the color-converted rectangular image.

The area dividing unit 304 divides the color-converted rectangular image into a plurality of area images (including a predetermined number of partial images). Further, the area dividing unit 304 notifies the group generating unit 305 of a plurality of area images obtained by dividing the rectangular image into areas.

The group generation unit 305 classifies the plurality of area images notified by the area division unit 304 into a plurality of groups according to the distance from the reference position of the rectangular image. Furthermore, the group generation unit 305 notifies the color ratio calculation unit 306 of each area image classified into each group.

The color ratio calculation unit 306 totalizes a plurality of pixels (partial images) included in the area images classified into each group for each color (calculates the number of pixels (number of partial images) of each color for each group). Thereby, the color ratio calculation unit 306 calculates the ratio (color ratio) of the number of pixels (number of partial images) of each color in each group.

The registration unit 307 stores the color ratio of each group as reference data in the reference data storage unit 123.

<Specific example of processing of each part of the generation unit of the inspection device>
Next, a specific example of the processing of each section of the generation section 121 (the image data acquisition section 301 to the registration section 307) of the inspection apparatus 120 will be described.

(1) Specific example of processing by image data acquisition unit, compression unit, and RGB value conversion unit First, a specific example of processing by the image data acquisition unit 301, compression unit 302, and RGB value conversion unit 303 of the generation unit 121 will be described. FIG. 4 is a diagram showing a specific example of processing by the image data acquisition unit, compression unit, and RGB value conversion unit.

As shown in FIG. 4, the image data acquisition unit 301 acquires a model image 410 transmitted from the imaging device 110, and extracts a circumscribed rectangular area of an article 411 determined to be acceptable from the acquired model image 410. Then, a rectangular image 420 is output.

Further, as shown in FIG. 4, the compression unit 302 divides the rectangular image 420 into partial images (for example, partial image 421) of a predetermined number of pixels (9 pixels in the example of FIG. 4), and A predetermined number of pixels (nine pixels in the example of FIG. 4) are collectively defined as one pixel.

Note that when compressing a predetermined number of pixels (9 pixels in the example of FIG. 4) included in a partial image (for example, partial image 421) into one pixel, the compression unit 302 calculates the average pixel value of the predetermined number of pixels. Calculate the value. As a result, the compression unit 302 generates a partial image 422 having one pixel value (in the example of FIG. 4, R value="210") (hereinafter, the partial image 422 will be one pixel).

Note that the compression unit 302 similarly calculates the average value of the pixel values of a predetermined number of pixels for the G value and the B value, and performs similar processing on all partial images included in the rectangular image 420. Thereby, the compression unit 302 compresses the rectangular image 420 into an image in which each partial image has one pixel, and generates a compressed rectangular image.

Further, as shown in FIG. 4, the RGB value conversion unit 303 converts the pixel values (RGB values) of each partial image for the compressed rectangular image. In this embodiment, for example, the R values of 256 gradations from 0 to 255 are divided into 11 gradation ranges, and the R values belonging to each gradation range are color-converted to the representative value of each gradation range. . The example in Figure 4 is
・Color conversion of R values belonging to the gradation range = 0 to 24 to representative value = 0,
・Color conversion of the R value belonging to the gradation range = 25 to 49 to the representative value = 25,
・Color conversion of the R value belonging to the gradation range = 50 to 74 to a representative value = 50,
・Color conversion of the R value belonging to the gradation range = 75 to 99 to the representative value = 75,
・Color conversion of R values belonging to the gradation range = 100 to 124 to a representative value = 100,
・Color conversion of the R value belonging to the gradation range = 125 to 149 to the representative value = 125,
etc. As a result, the R value of 256 gradations is color-converted into 11 representative values.

Similarly, the RGB value conversion unit 303 divides the G values of 256 gradations from 0 to 255 into 11 gradation ranges, and converts the G values belonging to each gradation range into representative values of each gradation range. Convert color to value. As a result, the 256 gradations of G values are color-converted into 11 representative values.

Similarly, the RGB value conversion unit 303 divides the B values of 256 gradations from 0 to 255 into 11 gradation ranges, and converts the B values belonging to each gradation range into representative values of each gradation range. Convert color to value. As a result, the 256 gradation B values are color-converted into 11 representative values.

Note that, according to the RGB value conversion unit 303, color information of 256×256×256=16,777,216 patterns is converted to color information of 11×11×11=1,331 patterns.

In FIG. 4, an article 412 that is determined to be acceptable and is included in a color-converted rectangular image 430 shows an image after color conversion has been performed by the RGB value conversion unit 303.

(2) Specific example of processing by area dividing unit Next, a specific example of processing by area dividing unit 304 of generation unit 121 will be described. FIG. 5 is a diagram showing a specific example of processing by the area dividing section. As shown in FIG. 5, upon acquiring a color-converted rectangular image 430, the area dividing unit 304 divides the image into a plurality of area images. The example in FIG. 5 shows how the color-converted rectangular image 430 is divided into 36 regional images by dividing it into 6 parts in the vertical direction and 6 parts in the horizontal direction.

Furthermore, the region dividing unit 304 notifies the group generating unit 305 of a plurality of region images (36 region images in the example of FIG. 5) obtained by dividing the color-converted rectangular image 430 into regions.

(3) Specific example of processing by group generation unit Next, a specific example of processing by group generation unit 305 of generation unit 121 will be described. FIG. 6 is a first diagram showing a specific example of processing by the group generation unit. As shown in FIG. 6, the group generation unit 305 divides a plurality of region images obtained by region-dividing the color-converted rectangular image 430 into a plurality of groups (three groups in the example of FIG. 6). Classify.

The example in FIG. 6 shows how the group generation unit 305 classifies a plurality of area images into three groups according to the distance from the reference position, using the center position of the color-converted rectangular image 430 as the reference position. It shows.

Specifically, the group generation unit 305 classifies a group of area images surrounding the center position of the color-converted rectangular image 430 into group 1. The example in FIG. 6 shows how four area image groups located along arrow 601 are classified into group 1.

Next, the group generation unit 305 groups a group of area images surrounding the center position of the color-converted rectangular image 430 and located one column outside the area images classified into group 1. Classified into 2. The example in FIG. 6 shows how a group of 12 regional images located along arrow 602 are classified into group 2.

Next, the group generation unit 305 groups a group of area images surrounding the center position of the color-converted rectangular image 430 and located one column outside the area images classified into group 2. Classified into 3. The example in FIG. 6 shows how a group of 20 area images located along arrow 603 are classified into group 3.

(4) Specific example of processing by color ratio calculation unit Next, a specific example of processing by color ratio calculation unit 306 of generation unit 121 will be described. FIG. 7 is a diagram illustrating a specific example of processing by the color ratio calculation unit.

As shown in FIG. 7, the color ratio calculation unit 306 calculates the area images classified into each group (4 area images classified into group 1, 12 area images classified into group 2, 12 area images classified into group 3). 20 classified area images) are obtained.

Next, the color ratio calculation unit 306 aggregates the plurality of pixels (partial images) included in the area images classified into each group for each color, thereby calculating the number of pixels of each color (number of partial images) for each group. Calculate the ratio (color ratio).

In FIG. 7, a pie chart 701 aggregates multiple partial images included in the four area images classified into group 1 for each color, and calculates the ratio (color ratio) of the number of pixels (number of partial images) of each color. This is the graph shown.

Similarly, a pie chart 702 aggregates a plurality of partial images included in the 12 area images classified into group 2 for each color, and shows the ratio (color ratio) of the number of pixels (number of partial images) of each color. This is a graph.

Similarly, a pie chart 703 aggregates multiple partial images included in the 20 area images classified into group 3 for each color, and shows the ratio (color ratio) of the number of pixels (number of partial images) of each color. This is a graph.

(5) Specific example of processing by registration unit Next, a specific example of processing by the registration unit 307 of the generation unit 121 will be described. FIG. 8 is a diagram showing a specific example of processing by the registration unit.

As shown in FIG. 8, the registration unit 307 acquires the color ratio of each group for a plurality of articles (acceptable articles) that are determined to be acceptable. The example in FIG. 8 shows how the color ratios of each group are obtained for accepted articles 1, 2, 3, . . . .

According to the example of FIG. 8, the color ratio of each group obtained for the accepted article 1 is registered in the table 801 as reference data and stored in the reference data storage unit 123. As shown in FIG. 8, the table 801 has "color information 1" to "color information N" arranged in the horizontal direction as information items, and "group 1" to "group 3" arranged in the vertical direction. Ru. Note that according to the example of FIG. 4 described above, the color information included in the color-converted rectangular image is 1,331 patterns, so N=1331.

Further, according to the example of FIG. 8, the grade of the accepted article 1 ("grade 1") is also registered in the table 801.

Similarly, according to the example of FIG. 8, the color ratio of each group acquired for the accepted article 2 is registered in the table 802 as reference data and stored in the reference data storage unit 123. Further, according to the example of FIG. 8, the grade of the accepted article 2 ("Grade 2") is also registered in the table 802.

Similarly, according to the example of FIG. 8, the color ratios of each group acquired for the accepted article 3 are registered in the table 803 as reference data and stored in the reference data storage unit 123. Further, according to the example of FIG. 8, the grade of the accepted article 3 ("Grade 3") is also registered in the table 803.

<Functional configuration of the determination section of the inspection device>
Next, the functional configuration of the determination unit 122 of the inspection device 120 will be explained. FIG. 9 is a diagram showing an example of the functional configuration of the determination section of the inspection device. As shown in FIG. 9, the determination unit 122 includes an image data acquisition unit 901, a compression unit 902, and an RGB value conversion unit 903 (an example of a conversion unit). Further, the determination unit 122 includes an area division unit 904 (an example of a division unit), a group generation unit 905 (an example of a classification unit), a color ratio calculation unit 906 (an example of a calculation unit), and a comparison unit 907.

Note that the functions of each unit from the image data acquisition unit 901 to the color ratio calculation unit 906 are the same as the functions of each unit from the image data acquisition unit 301 to the color ratio calculation unit 306 described using FIG. The explanation will be omitted and only the comparison unit 907 will be explained.

When the comparison unit 907 obtains the color ratio of each group for the article to be inspected from the color ratio calculation unit 906, the comparison unit 907 receives reference data (tables 801, 802, 803, etc.) from the reference data storage unit 123. read out. In addition, the comparison unit 907 compares the color ratio of each group obtained for the item to be inspected with the color ratio of each group for each passed item registered in the tables 801, 802, 803, etc. do. Further, the comparison unit 907 determines whether the article to be inspected passes or fails based on the comparison result, and if it is determined to be acceptable, determines the grade of the article to be inspected.

<Specific example of processing of each part of the determination unit of the inspection device>
Next, a specific example of the processing of each section (here, the comparison section 907) of the determination section 122 of the inspection apparatus 120 will be described.

FIG. 10 is a diagram illustrating a specific example of processing by the comparison unit. As shown in FIG. 10, the comparison unit 907 compares the color ratio of each group acquired for the article to be inspected with the color ratio of each group registered in the table 801. Similarly, the comparison unit 907 compares the color ratio of each group acquired for the article to be inspected with the color ratio of each group registered in the table 802. Similarly, the comparison unit 907 compares the color ratio of each group acquired for the article to be inspected with the color ratio of each group registered in the table 803.

In this manner, the comparison unit 907 performs a table-by-table comparison with all the tables stored as reference data in the reference data storage unit 123. In addition, the comparison unit 907 determines whether the article to be inspected passes or fails based on the comparison results for each table, and if it is determined to pass, determines the grade of the article to be inspected.

FIG. 11 is a diagram showing an example of the comparison results, and shows a comparison between the color ratio of each group for passing article 1 (numeral 1100) and the color ratio of each group for the article to be inspected (numeral 1110). It shows.

In FIG. 11, pie charts 1101 to 1103 indicate the color ratio of each group for the accepted article 1. Further, pie charts 1111 to 1113 show the color ratio of each group for the article to be inspected.

In addition, in FIG. 11, a comparison result 1121 shows the comparison result of the color ratio of group 1, a comparison result 1122 shows the comparison result of the color ratio of group 2, and a comparison result 1123 shows the comparison result of the color ratio of group 3. , respectively.

As shown in comparison results 1121 to 1123, when comparing the color ratios for each table, the comparison unit 907 compares the color ratios for each group. Furthermore, as shown in comparison results 1121 to 1123, when comparing the color ratios in each group, the comparison unit 907 performs the comparison for each color and calculates the absolute value of the difference. Furthermore, as shown in comparison results 1121 to 1123, the comparison unit 907 adds the absolute values of the differences calculated for each color, and calculates the total of the absolute values of the differences for each group.

In this way, by dividing the objects into groups and comparing the color ratios of each color, it is possible to avoid a situation where local abnormalities in the object to be inspected are buried in individual differences among the objects. As a result, the difference between an article containing a local abnormality and an acceptable article can be made clearer than when comparing the overall appearance of the article by color.

Note that in the comparison unit 907, if the sum of the absolute values of the differences is large in any group (for example, if the sum of the absolute values of the differences is greater than or equal to a predetermined threshold), the item to be inspected is determined to be defective. It is determined that the candidate is a successful candidate. In addition, in the comparison unit 907, if the sum of the absolute values of the differences is small for any group (for example, if the sum of the absolute values of the differences is less than a predetermined threshold), the item to be inspected passes the test. It is determined that there is.

Note that when it is determined that the item to be inspected is a candidate for failure in comparison with all the tables stored as standard data in the standard data storage unit 123, the comparison unit 907 determines that the item to be inspected is a failure candidate. It is determined that the test has passed.

On the other hand, if it is determined that the item to be inspected is acceptable in comparison with any one or more tables among the tables stored as standard data in the standard data storage unit 123, the comparison unit 907 It is determined that the following items pass the test.

In addition, if there are multiple tables that have been determined to pass the inspection target article, the comparison unit 907 identifies the table with the minimum sum of the absolute values of the differences (in other words, identifies the most similar table), and Determine the grade associated with the table. Thereby, the comparison unit 907 determines the grade of the article to be inspected.

<Flow of standard data generation process>
Next, the flow of reference data generation processing by the generation unit 121 of the inspection device 120 will be explained. FIG. 12 is a flowchart showing the flow of the reference data generation process.

In step S1201, the image data acquisition unit 301 of the generation unit 121 inputs "1" to a counter n that counts articles determined to be acceptable.

In step S1202, the image data acquisition unit 301 of the generation unit 121 acquires the image data of the accepted item n as a model image, extracts a circumscribed rectangular area of the accepted item n from the acquired model image, and outputs a rectangular image. .

In step S1203, the compression unit 302 of the generation unit 121 divides the rectangular image into partial images each having a predetermined number of pixels, and combines the predetermined number of pixels included in each partial image into one pixel. Compress the partial image into an image of one pixel.

In step S1204, the RGB value conversion unit 303 of the generation unit 121 performs color conversion on the compressed rectangular image by converting the pixel values (RGB values) of each partial image.

In step S1205, the area division unit 304 of the generation unit 121 divides the color-converted rectangular image into a plurality of area images.

In step S1206, the group generation unit 305 of the generation unit 121 classifies each area image obtained by area division into a plurality of groups according to the distance from the reference position of the rectangular image.

In step S1207, the color ratio calculation unit 306 of the generation unit 121 totalizes the plurality of partial images included in each area image classified into the plurality of groups for each color, and calculates the color ratio of each group.

In step S1208, the registration unit 307 of the generation unit 121 registers the color ratio of each group in the table, and stores it in the reference data storage unit 123 as reference data.

In step S1209, the image data acquisition unit 301 of the generation unit 121 determines whether the color ratio of each group has been registered in the table for all the articles determined to be passed (all the passed articles). In step S1209, if it is determined that there is a passed article whose color ratio of each group is not registered in the table (in the case of No in step S1209), the process advances to step S1210.

In step S1210, the image data acquisition unit 301 of the generation unit 121 increments the counter n, and then returns to step S1202.

On the other hand, if it is determined in step S1209 that the color ratios of each group have been registered in the table for all acceptable articles (in the case of YES in step S1209), the reference data generation process is ended.

<Flow of judgment process>
Next, the flow of the determination process by the determination unit 122 of the inspection device 120 will be explained. FIG. 13 is a flowchart showing the flow of the determination process.

In step S1301, the comparison unit 907 of the determination unit 122 reads reference data from the reference data storage unit 123.

In step S1302, the image data acquisition unit 901 of the determination unit 122 acquires a target image obtained by photographing the article to be inspected, and extracts a circumscribed rectangular area of the article to be inspected from the acquired target image, thereby creating a rectangular image. Output.

In step S1303, the compression unit 902 of the determination unit 122 divides the rectangular image into partial images each having a predetermined number of pixels, and combines the predetermined number of pixels included in each partial image into one pixel. Compress the partial image into an image of one pixel.

In step S1304, the RGB value conversion unit 903 of the determination unit 122 performs color conversion on the compressed rectangular image by converting the pixel values (RGB values) of each partial image.

In step S1305, the area division unit 904 of the determination unit 122 divides the color-converted rectangular image into a plurality of area images.

In step S1306, the group generation unit 905 of the determination unit 122 classifies each region image obtained by region division into a plurality of groups according to the distance from the reference position of the rectangular image.

In step S1307, the color ratio calculation unit 906 of the determination unit 122 totalizes the plurality of partial images included in each area image classified into the plurality of groups for each color, and calculates the color ratio of each group.

In step S1308, the comparison unit 907 of the determination unit 122 compares the color ratio of the article to be inspected and the color ratio of each table included in the reference data for each group and color, and compares the absolute difference for each group. Calculate the sum of values.

In step S1309, the comparison unit 907 of the determination unit 122 determines whether each table included in the reference data includes a group with a large sum of absolute values of differences in color ratios. In step S1309, if it is determined that any table included in the reference data includes a group with a large sum of absolute values of differences in color ratios (if YES in step S1309), step Proceed to S1310.

In step S1310, the comparison unit 907 of the determination unit 122 determines that the item to be inspected is rejected.

On the other hand, if it is determined in step S1309 that there is a table that does not include a group with a large sum of the absolute values of differences in color ratios among the tables included in the reference data (in the case of No in step S1309), ), the process advances to step S1311.

In step S1311, the comparison unit 907 of the determination unit 122 determines that the article to be inspected passes the test.

In step S1312, the comparison unit 907 of the determination unit 122 identifies a table that does not include a group with a large sum of absolute values of differences in color proportions. Further, the comparing unit 907 of the determining unit 122 determines the grade of the article to be inspected by identifying the grade associated with the specified table.

Note that if there are multiple tables that do not include a group in which the sum of the absolute values of the differences in color proportions is large, the comparison unit 907 of the determination unit 122 selects the table in which the sum of the absolute values of the differences in color proportions of each group is the smallest. Identify. Then, the comparing unit 907 of the determining unit 122 determines the grade of the article to be inspected by identifying the grade associated with the specified table.

In step S1313, the image data acquisition unit 901 of the determination unit 122 determines whether to end the determination process. If it is determined in step S1313 to continue the determination process (in the case of No in step S1313), the process returns to step S1302. On the other hand, if it is determined in step S1313 that the determination process is to be terminated (in the case of Yes in step S1313), the determination process is terminated.

As is clear from the above description, the inspection device 120 according to the first embodiment divides the circumscribed rectangular area of the item, which is extracted from the target image of the item to be inspected, into a plurality of area images. do. Furthermore, the inspection device 120 according to the first embodiment classifies the plurality of region images into a plurality of groups according to the distance from the reference position of the circumscribed rectangular region. Furthermore, the inspection apparatus 120 according to the first embodiment calculates the color ratio for each group by totaling each partial image included in the area images classified into each group for each color. Furthermore, the inspection device 120 according to the first embodiment calculates the color ratio of each group calculated in advance for a model image taken of an article determined to be passed, and the color of each group calculated for the article to be inspected. The ratio is compared to determine whether the item to be inspected passes or fails.

In this manner, the inspection device 120 according to the first embodiment compares the color ratios of the articles to be inspected and the color ratios of the passed articles for each group. Thereby, according to the inspection device 120 according to the first embodiment, it is possible to avoid a situation where local abnormalities in the article to be inspected are buried in individual differences among the articles.

As a result, according to the first embodiment, the difference between an article containing a local abnormality and an acceptable article can be made clearer than when comparing the appearance of the entire article by color. . That is, according to the first embodiment, it is possible to improve the inspection accuracy in the appearance inspection of articles.

[Second embodiment]
In the first embodiment described above, the unique characteristics of the article were not particularly considered during the reference data generation process and the determination process. However, some articles may have unique characteristics that should be considered during visual inspection.

For example, when the article is a fruit, the calyx portion is greatly different in color from the portion other than the calyx, and each fruit has a specific color. On the other hand, depending on how the fruit is placed during the visual inspection, the position of the fruit's stem will be different for each image data. Therefore, when region images are divided into groups, they may be classified into any group, and the color ratios will vary greatly in the classified groups.

Therefore, in the second embodiment, when dividing area images into groups, area images including fruit halves (area images having a specific color) are excluded from grouping targets (classification targets). The second embodiment will be described below, focusing on the differences from the first embodiment.

<Specific example of processing by group generation unit>
FIG. 14 is a second diagram showing a specific example of processing by the group generation unit. Note that in FIG. 14, the group generation section 1400 will be described as a group generation section included in the generation section 121, but it is assumed that the group generation section included in the determination section 122 also has a similar function.

As shown in FIG. 14, the group generation unit 1400 divides the color-converted rectangular image 1430 into a plurality of regional images and classifies them into a plurality of groups.

Here, in the example of FIG. 14, among the plurality of area images included in the color-converted rectangular image 1430, area image 1431 includes the calyx of the fruit to be inspected. In this case, the group generation unit 1400 excludes the region image 1431 from the 12 region images classified into group 2, and classifies the 11 region images into group 2. In FIG. 14, reference numeral 1401 indicates that a region image 1431 is excluded from the region image group classified into group 2.

In this way, by classifying the region images in consideration of the unique characteristics of the article, it is possible to calculate an appropriate color ratio as the color ratio for each group. As a result, according to the second embodiment, it is possible to further improve the inspection accuracy in the appearance inspection of articles.

[Other embodiments]
In the first and second embodiments described above, when dividing area images into groups, the center position of the rectangular image is used as the reference position. However, the reference position is not limited to the center position of the rectangular image, and may be, for example, the upper end position or the lower end position of the rectangular image. Alternatively, it may be the left end position or right end position of a rectangular image.

Furthermore, in the first and second embodiments described above, the compression units 302 and 902 respectively compress the rectangular image by combining 3 pixels x 3 pixels (9 pixels) included in the partial image into one pixel. Although the case has been described, the compression method by the compression units 302 and 902 is not limited to this.

Furthermore, in the first and second embodiments described above, the generation unit 121 and the determination unit 122 are described as having the compression units 302 and 902, respectively. However, the generation unit 121 and the determination unit 122 do not need to include the compression units 302 and 902. In this case, the color ratio calculation units 306 and 906 use the ratio of the number of uncompressed pixels as the ratio of the number of pixels of each color, instead of the ratio of the number of pixels after compression (that is, the ratio of the number of partial images). It will be calculated as a percentage.

Further, in the first and second embodiments described above, the case where the RGB value conversion units 303 and 903 respectively convert 25 gradations into 1 gradation has been described, but the color conversion by the RGB value conversion units 303 and 903 The method is not limited to this.

Furthermore, in the first and second embodiments described above, the generation unit 121 and the determination unit 122 are described as having RGB value conversion units 303 and 903, respectively. However, the generation unit 121 and the determination unit 122 do not need to include the RGB value conversion units 303 and 903. In this case, the color ratio calculation units 306 and 906 calculate the ratio of the number of pixels of each color in the 16,777,216 pattern as the ratio of the number of pixels of each color instead of the ratio of the number of pixels of each color in the 1,331 pattern. It will be calculated as a color ratio.

In addition, in the first and second embodiments described above, the case where the area dividing units 304 and 904 respectively divide the rectangular image into six vertically and horizontally into six, but the number of divisions is six. Not limited.

Furthermore, in the first and second embodiments described above, the comparison unit 907 calculates the sum of the absolute values of the differences between each color for each group, and performs the comparison based on the calculated total value. However, the comparison method of the comparison unit 907 is not limited to this. For example, the sum of the absolute values of the differences between the colors may be normalized for each group, and the comparison may be performed based on the normalized total value.

Further, in the first embodiment, the case where the object to be inspected is a fruit has been described, but the object to be inspected is not limited to this, and the object to be inspected is not limited to this, but may be an object to be inspected based on the color of its appearance, such as vegetables, bread, flowers, etc. , any other article may be used as long as it is an article for which a pass/fail determination is to be made.

Note that, in the disclosed technology, forms such as those described below are possible.
(Additional note 1)
a dividing unit that divides a circumscribed rectangular area of the item, extracted from a target image obtained by photographing the item to be inspected, into a plurality of area images;
a classification unit that classifies the plurality of area images into a plurality of groups according to a distance from a reference position of the circumscribed rectangular area;
a calculation unit that calculates the ratio of the number of pixels of each color for each group by totaling the pixels included in the area image classified into each group for each color;
The ratio of the number of pixels of each color in each group calculated in advance for a model image taken of an article that is determined to be acceptable is compared with the ratio of the number of pixels of each color in each group calculated by the calculation unit, and the ratio of the number of pixels of each color in each group calculated by the calculation unit is compared. An inspection device comprising: a comparison section for determining pass/fail of a target article;
(Additional note 2)
The comparison section is
As a result of comparing each group, if it is determined that the difference in the ratio of the number of pixels of each color in any group is greater than or equal to a predetermined threshold value, the item to be inspected is determined to be rejected. Inspection equipment as described.
(Additional note 3)
The comparison section is
As a result of comparing each group, if it is determined that the difference in the ratio of the number of pixels of each color in all groups is less than a predetermined threshold value, the article to be inspected is determined to pass. Inspection equipment.
(Additional note 4)
The comparison section is
If the item to be inspected is determined to pass the ratio of the number of pixels of each color in each group calculated in advance for multiple model images, the difference in the ratio of the number of pixels of each color in each group is the smallest. The inspection device according to supplementary note 3, wherein the grade of the article to be inspected is determined by specifying a model image that becomes .
(Appendix 5)
Divide the circumscribed rectangular area of the object, extracted from the target image of the object to be inspected, into partial images each having a predetermined number of pixels, and summarize the pixel values of the predetermined number of pixels included in each partial image. further comprising a compression unit that compresses a circumscribed rectangular area of the article by calculating a pixel value of 1,
The calculation unit is
The inspection device according to supplementary note 1, wherein the ratio of the number of pixels is calculated by setting the partial image as one pixel.
(Appendix 6)
further comprising a conversion unit that converts the pixel value of each partial image included in the circumscribed rectangular area of the article compressed by the compression unit into a predetermined representative value in the gradation range to which the pixel value belongs,
The calculation unit is
The inspection device according to appendix 5, wherein the ratio of the number of pixels of each color is calculated by summing up each color converted to the representative value.
(Appendix 7)
The inspection device according to supplementary note 1, wherein the classification unit excludes area images containing a predetermined color from classification targets when classifying the area images into a plurality of groups.
(Appendix 8)
The inspection device according to appendix 1, wherein the reference position of the circumscribed rectangular area includes any one of a center position, an upper end position, a lower end position, a left end position, and a right end position of the circumscribed rectangular area.
(Appendix 9)
dividing a circumscribed rectangular region of the object, extracted from a target image obtained by photographing the object to be inspected, into a plurality of region images;
classifying the plurality of area images into a plurality of groups according to the distance from the reference position of the circumscribed rectangular area;
By aggregating the pixels included in the area images classified into each group for each color, the ratio of the number of pixels of each color is calculated for each group,
The ratio of the number of pixels of each color in each group, calculated in advance for a model image taken of an item that is determined to be acceptable, is compared with the calculated ratio of the number of pixels of each color in each group, and the item to be inspected is determined. Judging pass/fail of
An inspection program that causes a computer to perform processing.
(Appendix 10)
dividing a circumscribed rectangular region of the object, extracted from a target image obtained by photographing the object to be inspected, into a plurality of region images;
classifying the plurality of area images into a plurality of groups according to the distance from the reference position of the circumscribed rectangular area;
By aggregating the pixels included in the area images classified into each group for each color, the ratio of the number of pixels of each color is calculated for each group,
The ratio of the number of pixels of each color in each group, calculated in advance for a model image taken of an item that is determined to be acceptable, is compared with the calculated ratio of the number of pixels of each color in each group, and the item to be inspected is determined. Judging pass/fail of
An inspection method in which processing is performed by a computer.

Note that the present invention is not limited to the configurations shown here, such as combinations of other elements with the configurations listed in the above embodiments. These points can be modified without departing from the spirit of the present invention, and can be appropriately determined depending on the application thereof.

100: Inspection system 110: Imaging device 120: Inspection device 121: Generation unit 122: Determination unit 301: Image data acquisition unit 302: Compression unit 303: RGB value conversion unit 304: Area division unit 305: Group generation unit 306: Color ratio Calculation unit 307: Registration unit 430: Rectangular image 801-803: Table 901: Image data acquisition unit 902: Compression unit 903: RGB value conversion unit 904: Area division unit 905: Group generation unit 906: Color ratio calculation unit 907: Comparison Parts 1121 to 1123: Comparison results 1400: Group generation part

Claims (8)

a dividing unit that divides a circumscribed rectangular area of the item, extracted from a target image obtained by photographing the item to be inspected, into a plurality of area images;
a classification unit that classifies the plurality of area images into a plurality of groups according to a distance from a reference position of the circumscribed rectangular area;
a calculation unit that calculates the ratio of the number of pixels of each color for each group by totaling the pixels included in the area image classified into each group for each color;
The ratio of the number of pixels of each color in each group calculated in advance for a model image taken of an article that is determined to be acceptable is compared with the ratio of the number of pixels of each color in each group calculated by the calculation unit, and the ratio of the number of pixels of each color in each group calculated by the calculation unit is compared. An inspection device comprising: a comparison section for determining pass/fail of a target article; The comparison section is
As a result of comparing each group, if it is determined that the difference in the ratio of the number of pixels of each color in any group is equal to or greater than a predetermined threshold value, the article to be inspected is determined to be rejected. The inspection device described in . The comparison section is
According to claim 1, the article to be inspected is determined to pass when it is determined that the difference in the ratio of the number of pixels of each color is less than a predetermined threshold value in all groups as a result of comparison for each group. inspection equipment. The comparison section is
If the item to be inspected is determined to pass the ratio of the number of pixels of each color in each group calculated in advance for multiple model images, the difference in the ratio of the number of pixels of each color in each group is the smallest. 4. The inspection device according to claim 3, wherein the grade of the article to be inspected is determined by specifying a model image that becomes . Divide the circumscribed rectangular area of the object, extracted from the target image of the object to be inspected, into partial images each having a predetermined number of pixels, and summarize the pixel values of the predetermined number of pixels included in each partial image. further comprising a compression unit that compresses a circumscribed rectangular area of the article by calculating a pixel value of 1,
The calculation unit is
The inspection device according to claim 1, wherein the ratio of the number of pixels is calculated by setting the partial image as one pixel. further comprising a conversion unit that converts the pixel value of each partial image included in the circumscribed rectangular area of the article compressed by the compression unit into a predetermined representative value in the gradation range to which the pixel value belongs,
The inspection device according to claim 5, wherein the calculation unit calculates the ratio of the number of pixels of each color by totaling the converted values for each color. dividing a circumscribed rectangular region of the object, extracted from a target image obtained by photographing the object to be inspected, into a plurality of region images;
classifying the plurality of area images into a plurality of groups according to the distance from the reference position of the circumscribed rectangular area;
By aggregating the pixels included in the area images classified into each group for each color, the ratio of the number of pixels of each color is calculated for each group,
The ratio of the number of pixels of each color in each group, calculated in advance for a model image taken of an item that is determined to be acceptable, is compared with the calculated ratio of the number of pixels of each color in each group, and the item to be inspected is determined. Judging pass/fail of
An inspection program that causes a computer to perform processing. dividing a circumscribed rectangular region of the object, extracted from a target image obtained by photographing the object to be inspected, into a plurality of region images;
classifying the plurality of area images into a plurality of groups according to the distance from the reference position of the circumscribed rectangular area;
By aggregating the pixels included in the area images classified into each group for each color, the ratio of the number of pixels of each color is calculated for each group,
The ratio of the number of pixels of each color in each group, calculated in advance for a model image taken of an item that is determined to be acceptable, is compared with the calculated ratio of the number of pixels of each color in each group, and the item to be inspected is determined. Judging pass/fail of
An inspection method in which processing is performed by a computer.

Images