JP6898546B2 - Information processing equipment, control methods, and programs - Google Patents

Description

The present invention relates to a technique for discriminating scratches on a product.

In order to measure the degree of perfection of a product, there is a method of verifying the product by processing an image obtained by a user taking a picture of the product with a camera or the like.

As an example of the method, when inspecting the presence or absence of bolts for fixing the supporting glass and the rigid overhead wire to the hanging bracket, the brightness gradient direction at each pixel position is calculated for the image obtained by photographing these and made into a template. There is a method of inspecting the presence or absence of bolts by comparing the obtained luminance gradient directions (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2014-126361

However, the image taken by the product may contain noise due to the shooting, and there is a problem that it is difficult to discriminate the scratches existing in the product itself due to this noise.

In the invention described in Patent Document 1, there is no description or suggestion regarding the determination of the scratches existing in the product itself, and the scratch patterns can be varied by creating a template with respect to the expected luminance gradient direction of the scratches. Therefore, it is not realistic.

Therefore, an object of the present invention is to provide an information processing device, a control method, and a program capable of accurately discriminating the existence of linear pixel groups in an image.

In order to achieve the above object, the image is divided into a plurality of regions having a predetermined shape and size based on an arbitrary start position, and based on the gradient direction and the gradient intensity of the brightness of each pixel in the divided regions. The accumulating means for accumulating the gradient intensities of pixels having the same gradient direction, and the determining means for determining whether or not the pixels satisfying a predetermined condition as the cumulative result of the accumulating means are linear. The classification means is characterized in that the image is divided into a plurality of new regions of the predetermined shape and size based on the start position shifted from the start position in an arbitrary direction.

According to the present invention, there is an effect that the existence of linear pixel groups can be accurately determined in an image.

It is a system block diagram which shows an example of the structure of the linear scratch presence / absence determination system of this invention. It is a block diagram which shows the hardware configuration of an information processing apparatus. It is a block diagram which shows an example of the software structure of the linear scratch presence / absence determination system. It is a flowchart which shows the process of the linear scratch presence / absence determination in the linear scratch presence / absence determination system. It is an image of an image with linear scratches and an image without linear scratches at the time of image acquisition. This is an image when the gradient direction of the brightness is calculated for each pixel of the image. It is an image when the image is divided. This is an image when the image is divided by shifting the start position to the lower right. It is a histogram which accumulated the gradient intensity in the same gradient direction for each division area. This is an image when pixels included in the gradient direction with a cumulative value of a certain value or more are acquired and plotted. This is an image when the result of determining the presence or absence of scratches is output to the screen. It is a flowchart which shows the process of the linear scratch presence / absence determination in the linear scratch presence / absence determination system. This is an image when an image of a certain size is cut out with the coordinates of the endpoints as the center. This is an image when searching for the existence of pixels on the same straight line at the edge of the cutout image.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a system configuration diagram showing an example of the configuration of the linear scratch presence / absence determination system 100 of the present invention.

FIG. 1 shows a configuration in which a camera 102 and a PC 104, which is an information processing device, are connected via an image transfer cable (USB, Ethernet, camera link, etc.). Even if it does not have such a configuration, it may be configured as an information processing device equipped with a camera.

The PC 104 acquires an image obtained by photographing an image to be photographed by the camera 102 via an image transfer cable, and executes a process related to determination of the presence or absence of linear scratches in the image.

Further, the linear scratch presence / absence determination system 100 of the present embodiment may have a configuration in which the camera 102 having the above-described configuration does not exist.

In that case, the image stored in the memory (RAM) of the PC 104 is subjected to the process of determining the presence or absence of linear scratches in the image.

Hereinafter, the hardware configuration of the information processing apparatus applicable to the PC 104 shown in FIG. 1 will be described with reference to FIG.

FIG. 2 is a block diagram showing a hardware configuration of an information processing device applicable to the PC 104 shown in FIG.

In FIG. 2, 201 is a CPU that comprehensively controls each device and controller connected to the system bus 204. Further, in the ROM 202 or the external memory 212, a BIOS (Basic Input / Output System) which is a control program of the CPU 201, an operating system program (hereinafter referred to as an OS), and a function to be executed by each PC will be described later. Various programs etc. are stored.

Reference numeral 203 denotes a RAM, which functions as a main memory, a work area, and the like of the CPU 201. The CPU 201 realizes various operations by loading a program or the like necessary for executing a process from the ROM 202 or the external memory 212 into the RAM 203 and executing the loaded program.

Reference numeral 205 denotes an input controller, which controls input from a keyboard (KB) 210, a pointing device such as a mouse (not shown), or the like.

Reference numeral 206 denotes a video controller that controls the display on a display such as the display 211.

Reference numeral 207 is a memory controller, which is an external storage device (hard disk (HD)) for storing various data, a flexible disk (FD), or a compact flash (registered trademark) memory connected to a PCMCIA card slot via an adapter. Controls access to memory 212.

Reference numeral 208 denotes a communication I / F controller, which controls reception of image data from an external PC 213 via a network (TCP / IP).

Reference numeral 209 denotes an image I / F controller, which controls reception of image data from the camera 102 via an image transfer cable (USB, Ethernet, camera link, etc.).

Various programs described later for realizing the present invention are recorded in the RAM 203 and executed by the CPU 201.

The image data used when the program is executed is stored in the ROM 202, the external memory 212, the external PC 213, and the camera 102 depending on the application, and is stored in the RAM 203 via various controllers when the program is executed.

Next, a functional block diagram of the PC104 as the information processing apparatus of the present invention will be described with reference to FIG.

The detailed control processed by each functional block will be described with reference to the flowchart described later.

The image acquisition means 301 acquires image data from the ROM 202, the external memory 212, the external PC 213, and the camera 102 according to the application, and stores the image data in the RAM 203.

The image processing means 302 executes image processing for determining the presence or absence of linear scratches on the image data stored in the RAM 203 by the image acquisition means 301.

The discrimination result output function 303 displays the discrimination result executed by the image processing means 302 on the display 211. Alternatively, the determination result is stored in the RAM 203 as a file.

The details of each function will be described in each process of the flowchart.

Hereinafter, the linear scratch presence / absence determination process in the linear scratch presence / absence determination system 100 of the present embodiment will be described with reference to FIG.

In step S101, the image acquisition means 301 acquires image data from the ROM 202, the external memory 212, the external PC 213, or the camera 102.

The image data includes an image with linear scratches or an image without linear scratches (see FIG. 5).

The image shown in FIG. 5 shows, for example, an example of an image obtained when a semiconductor wafer is photographed by a camera 102, the left side shows an image having linear scratches, and the right side shows a linear image. It shows an image without scratches.

In step S102, the image processing means 302 calculates the gradient direction of the brightness for each pixel of the image acquired in step S101 (see FIG. 6).

In the image shown in FIG. 6, the gradient direction of the brightness is calculated for each pixel of the image shown in FIG. On the left side, the left side indicates the luminance gradient direction for an image having linear scratches, and the right side indicates the luminance gradient direction for an image without linear scratches.

Further, in step S103, the image processing means 302 calculates the gradient intensity of the brightness for each pixel of the image acquired in step S101.

In the subsequent processing, the presence or absence of linear scratches is determined using the gradient direction of the luminance acquired in steps 102 and 103 and the gradient intensity. However, when the size of the linear scratches that can exist is smaller than the size of the entire image, the cumulative value of the gradient intensity due to the linear scratches may be smaller than the cumulative value of the gradient intensity due to noise.

In order to correspond to the above case, the cumulative value of the gradient intensity of the brightness is calculated for each arbitrary division region instead of the entire image. From step S104 to step S107, a method of classifying images will be described.

In step S104, the image processing means 302 divides the image by positioning the upper left corner of the image at the start position. The size (width and height) of one division area is set based on the average size of linear scratches that may exist in the image. For example, as shown in FIG. 7, when the size of one division area is 1/4 of the image size, 16 (4 × 4) division areas are created.

In step S105, the image processing means 302 shifts the start position from the upper left of the image to the right by 1/2 of the division area size, and divides the image so that the division area does not protrude from the image. For example, if the size of one division area is 1/4 of the image size, 12 (3 × 4) division areas are created.

In step S106, the image processing means 302 shifts the start position from the upper left of the image to 1/2 of the division area size, and divides the image so that the division area does not protrude from the image. For example, if the size of one division area is 1/4 of the image size, 12 (4 × 3) division areas are created.

In step S107, the image processing means 302 shifts the start position from the upper left of the image to the lower right by 1/2 of the division area size, and divides the image so that the division area does not protrude from the image. For example, as shown in FIG. 8, when the size of one division area is 1/4 of the image size, nine (3 × 3) division areas are created.

The compartmentalized areas created in steps S104 to S107 overlap each other. The purpose of creating by overlapping the division areas is to cover most of the linear scratches in one division area so that one linear scratch in the image does not straddle a plurality of division areas as much as possible. Is to be included in.

The shift amount from step S105 to step S107 is 1/2 of the division area size, but in order to satisfy the above object, a shift amount such as 1/4, 1/8, 1/16, etc. is added to divide the division area. May be increased.

In step S108, as shown in FIG. 9, in the image processing means 302, for each division region created in steps S104 to S107, the gradient intensity of the pixel brightness calculated in steps S102 and S103 in the same gradient direction is determined. Accumulate.

The histogram shown in FIG. 9 shows the first region in which the image is divided in steps S104 to S107 from the left, the M-th region in which the image is divided in the center, and the image on the right. The Nth region is shown.

In this histogram, the horizontal axis indicates the gradient intensity of the pixel, and the vertical axis indicates the gradient intensity of the pixel.

In step S109, the image processing means 302 calculates the maximum value from the cumulative value of each division area calculated in step S108 or the cumulative value of all the division areas. Which of the above cumulative values is used can be changed according to the application.

In step S110, the image processing means 302 calculates the average / variance from the cumulative value of each division region calculated in step S108 or the cumulative value of all division regions. Which of the above cumulative values is used can be changed according to the application.

In step S111, the image processing means 302 determines whether or not the maximum value calculated in step S109 is less than a certain value. If it is less than a certain value, the process proceeds to step S112, and if it is more than a certain value, the process proceeds to step S113.

In step S112, the image processing means 302 determines whether or not the variance calculated in step S110 is less than a certain value. If it is less than a certain value, the process proceeds to step S115, and if it is more than a certain value, the process proceeds to step S113.

In step S113, the image processing means 302 refers to the cumulative value of each division region calculated in step S108, and plots the positions of the pixels included in the gradient direction in which the cumulative value is a certain value or more on the image (see FIG. 10). ). It is expected that many pixels included in the linear scratches will be acquired by this processing and plotted on the image.

In FIG. 10, from each histogram of the division area L and the division area M shown on the left, the positions of the pixels whose cumulative values are included in the gradient direction of a certain value or more are plotted on the image as shown on the right. ..

In step S114, the image processing means 302 uses the image created in step S113 to determine whether or not the plotted pixels exist on the same straight line. In determining whether or not they exist on the same straight line, the straight line is detected by existing image processing techniques (Hough transform, RANSAC, etc.), and the number and ratio of pixels existing on the straight line are used.

If it is determined that the plotted pixels are on the same straight line, the process proceeds to step S116, and if not, the process proceeds to step S115.

In step S115, the image processing means 302 determines that there are no linear scratches in the image, and in step S116, the image processing means 302 determines that there are linear scratches in the image.

In step S117, the discrimination result output function 303 outputs the discrimination result from the judgment results of steps S115 and S116 to a display, a file, or the like.

As the discrimination result, as shown in the judgment result confirmation screen 400 shown in FIG. 11, there are linear scratches / no (or NG / OK) 401, center position 402, size 403, length 404, strength 405, direction 406, and the like. .. When the discrimination result is displayed on the display, the discrimination result information may be shown on the image.
[Modification example]

Hereinafter, with reference to FIG. 12, additional processing of the image processing function in the above-mentioned linear scratch presence / absence determination system will be described.

The flowchart shown in FIG. 12 is an improvement of the flowchart shown in FIG.

If it is determined in step S114 of the flowchart shown in FIG. 4 that the plotted pixels exist on the same straight line, the process proceeds to step S201 of the flowchart shown in FIG.

In step S201, the image processing means 302 acquires two endpoints of pixels existing on the same straight line.

In step 202, the image processing means 302 cuts out an image of a constant size centered on the coordinates of the two endpoints, as shown in FIG. The constant size is calculated from the average size of linear scratches that may exist in the image.

In step S203, the image processing means 302 refers only to the pixels in the top, bottom, left, and right edges of the cropped image, and determines whether or not the plotted pixels exist in the pixels.

In the case of linear scratches, the pixels are connected to each other to cause scratches. Therefore, as shown in the upper part of FIG. 14, there are pixels plotted in one row at the top, bottom, left, and right edges of the cutout image. The pixels are not connected, and as shown in the lower part of FIG. 14, there is no plotted image. In this process, each of the cut-out images of the two end points is individually determined.

In step S204, if the image processing means 302 has an image plotted in both of the cut-out images of the two endpoints in step S203, it is determined that the image is a linear scratch, and the process proceeds to step S116 of the flowchart shown in FIG. .. If not, it is determined that there is noise, and the process proceeds to step S115 of the flowchart of FIG.

As described above, according to the present invention, it is possible to accurately discriminate scratches on an object by using an image obtained by photographing the object.

It should be noted that the structure and contents of the various data described above are not limited to this, and it goes without saying that the structure and contents are various depending on the intended use and purpose.

Although one embodiment has been described above, the present invention can take an embodiment as, for example, a method, a program, a recording medium, or the like.

Further, the program in the present invention is a program in which a computer can execute the processing method of the flowchart shown in FIG. The program in the present invention may be a program for each processing method of each device shown in FIG. 7.

As described above, a recording medium on which a program that realizes the functions of the above-described embodiment is recorded is supplied to the system or device, and the computer (or CPU or MPU) of the system or device stores the program in the recording medium. Needless to say, the object of the present invention can be achieved by reading and executing.

In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium storing the program constitutes the present invention.

Recording media for supplying programs include, for example, flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, and silicon. Disks, solid state drives, etc. can be used.

Further, by executing the program read by the computer, not only the function of the above-described embodiment is realized, but also the OS (operating system) or the like running on the computer is actually operated based on the instruction of the program. Needless to say, there are cases where a part or all of the processing is performed and the processing realizes the functions of the above-described embodiment.

Further, after the program read from the recording medium is written in the memory provided in the function expansion board inserted in the computer or the function expansion unit connected to the computer, the function expansion board is based on the instruction of the program code. It goes without saying that there are cases where the CPU or the like provided in the function expansion unit performs a part or all of the actual processing, and the processing realizes the functions of the above-described embodiment.

It goes without saying that it can also be applied when it is achieved by supplying a program to a system or device. In this case, by reading the recording medium in which the program for achieving the present invention is stored into the system or the device, the system or the device can enjoy the effect of the present invention.

Further, by downloading and reading a program for achieving the present invention from a server, database, or the like on the network by a communication program, the system or device can enjoy the effect of the present invention.

It should be noted that all the configurations in which each of the above-described embodiments and modifications thereof are combined are also included in the present invention.

100 Linear scratch presence / absence determination system 102 Camera 104 Information processing device 201 CPU
202 RAM
203 ROM
204 System Bus 205 Input Controller 206 Video Controller 207 Memory Controller

208 Communication I / F (interface) controller 209 Input unit 210 Display device 211 External memory

Claims (6)

A classification means for dividing an image into a plurality of regions having a predetermined shape and size based on an arbitrary start position, and
A cumulative means for accumulating the gradient intensities of pixels having the same gradient direction based on the gradient direction and the gradient intensity of the brightness of each pixel in the divided region.
A determination means for determining whether or not a pixel whose cumulative result of the accumulation means satisfies a predetermined condition is linear.
With
The information processing apparatus is characterized in that the classification means divides the image into a plurality of new regions having a predetermined shape and size based on a start position shifted from the start position in an arbitrary direction. The information processing apparatus according to claim 1 , further comprising an analysis means for analyzing whether or not a linear scratch is present due to a connection between linear pixels determined by the determination means. The information processing apparatus according to claim 1 or 2 , wherein the determination means determines whether or not a pixel having a cumulative result of the cumulative means of a certain value or more becomes linear. The information processing apparatus according to any one of claims 1 to 3 , further comprising an output means for outputting a determination result obtained by the determination means. It is a control method of an information processing device that processes images.
A division step that divides an image into multiple areas of predetermined shape and size based on an arbitrary starting position, and
A cumulative step of accumulating the gradient intensities of pixels having a similar gradient direction based on the gradient direction and the gradient intensity of the pixels in the divided region.
A determination step for determining whether or not a pixel whose cumulative result of the cumulative step satisfies a predetermined condition is linear, and a determination step.
With
The division step is a control method of an information processing apparatus, characterized in that the image is divided into a plurality of new regions having a predetermined shape and size based on a start position shifted from the start position in an arbitrary direction. Computer,
A classification means for dividing an image into a plurality of regions having a predetermined shape and size based on an arbitrary start position, and
A cumulative means for accumulating the gradient intensities of pixels having a similar gradient direction based on the gradient direction and the gradient intensity of the pixels in the divided region.
The cumulative result of the cumulative means is made to function as a determination means for determining whether or not the pixels satisfying a predetermined condition are linear.
The classification means is a program characterized in that the image is divided into a plurality of new regions having a predetermined shape and size based on a start position shifted from the start position in an arbitrary direction.

Images