JP4095801B2 - Image processing apparatus and method based on image vector definition - Google Patents

Description

[0001]
[Industrial application fields]
The present invention relates to an image processing apparatus and method used in various fields such as part inspection and plant monitoring, and more particularly to an image processing apparatus and method for detecting an object whose boundary with the background is unclear.
[0002]
[Prior art]
Conventionally, in a system to which an image processing technique is applied, various processing algorithms have been developed according to the purpose of processing, such as object shape extraction and area division, and the captured object and imaging conditions. Various basic techniques are used as basic image processing techniques for constructing algorithms. In particular, the following methods have been used for the statistical property evaluation of images, the division of connected regions in images, the detection of graphic patterns or specific objects.
[0003]
That is, as shown in FIG. 11, the conventional image processing apparatus records an image pickup apparatus 1 for imaging or imaging a subject, an image processing computer 2 for executing image processing, a display apparatus 3 for displaying a processing result, and result data. It is composed of an external storage device 4 for saving. The image processing computer 2 includes an A / D conversion function 5 for converting into a digital image, a frame memory 6 for temporarily storing an input image, various image processing functions 7 for performing image conversion, feature extraction, image recognition, and the like, A central processing unit 8 that performs these operations, a system control bus 9 that transfers data, and an image transfer bus 10 are included.
[0004]
In the image processing apparatus configured as described above, image quality is evaluated, connected regions are divided, and an object is detected by various image processing methods. As the most general method for evaluating the image quality, measurement of a gray scale histogram is used. The density histogram is a frequency distribution of the density values of each pixel. In the evaluation, statistical numerical values such as maximum value, minimum value, and average value are used as indicators. Since the image data is discrete data, it is effective as a method for evaluating the properties of the entire image.
[0005]
Further, as a technique for evaluating the connectivity of pixels in a grayscale image, conventionally, after the grayscale image is separated into an object and a background by binarization threshold processing, the connected components are labeled with the same labeling processing. Although it is very effective as a method for evaluating the connectivity in a binarized image, it is not for evaluating a grayscale image.
[0006]
As a method for detecting a graphic pattern or a specific object from an image, a matching process using a gray template is generally used. The template matching process is to create an m × n template image and extract a region having the highest correlation value by a process such as correlation calculation. In this case, since the correlation calculation between the pixels is executed in the entire search range, it takes a long time to process. In order to detect an object more flexibly, it is necessary to change conditions such as a template size and rotation, and it is necessary to examine a matching algorithm.
[0007]
These methods are examples, but they are not necessarily sufficient as a method for more effectively using the connection information of each pixel constituting the image.
[0008]
[Problems to be solved by the invention]
In the conventional image processing apparatus described above, a technique for more effectively utilizing the connection information of each pixel constituting an image has not been proposed. In particular, when detecting an object whose boundary with the background is unclear, detecting the boundary line has been a problem.
[0009]
Therefore a first object of the present invention, as a means to evaluate the quality by image processing, by measuring the histogram of the image vector, Chi lifting metrics contour linkage information in the image, and the graphic pattern or identified from the image An object of the present invention is to provide an apparatus for detecting a specific object at high speed and flexibly using a matching process using an image vector template as means for detecting the object .
[0012]
Furthermore, a second object of the present invention is to use a matching process using an image vector template and a vector template updating process with time change as means for continuously detecting an object whose shape changes in time series, An object of the present invention is to provide an apparatus for continuously detecting a specific object.
[0013]
A third object of the present invention is to provide a method having an evaluation index of contour connection information in an image by measuring a histogram of an image vector as means for evaluating image quality by image processing.
[0014]
[Means for Solving the Problems]
In order to achieve the above object, an image processing apparatus and method according to the present invention employ the following configuration.
[0015]
The invention of the device described in claim 1
Data forming means for forming image data based on the amount of reflection of electromagnetic waves from the object;
For the image data from the data forming means, an image vector calculation means for obtaining information on the gradient and direction of the brightness by defining the vector amount of the image from the gray value of each pixel and its neighboring pixels,
Vector processing means for measuring the frequency distribution of the vector length or the frequency distribution in the vector direction from the vector distribution calculated by the image vector calculating means, and quantifying the contour information in the object;
A result display means for displaying a result representing a vector in each pixel;
A result recording means for recording an image vector calculation result and a vector frequency distribution measurement result;
The vector processing means converts the conversion table showing the relationship between the distance of the template geometry and any shape that is vector representation to the reference position coordinates arbitrarily set and each vector quantity, template the conversion table As described above, the template figure is extracted and recognized by voting to the reference position coordinates corresponding to the vector amount of each pixel in the processing target .
The invention of the method described in claim 3 is
Form image data based on the amount of reflection of electromagnetic waves from the object,
For the image data, by defining the vector amount of the image from the gray value of each pixel and its neighboring pixels, obtain information on the gradient and direction of the brightness,
Measure the frequency distribution of the vector length or the frequency distribution in the vector direction from the vector distribution obtained by the definition of the vector quantity to quantify the contour information in the object,
Displaying the result of representing the vector at each pixel;
Record image vector calculation results and vector frequency distribution measurement results,
In the image processing method as described above,
Converts a vector-represented geometric figure and a template of arbitrary shape into a conversion table showing the relationship between each vector quantity and the distance to a reference position coordinate that is arbitrarily set, and each pixel in the processing target using the conversion table as a template A template figure is extracted and recognized by voting to a reference position coordinate corresponding to the vector amount .
[0018]
According to a fourth aspect of the present invention, in the image processing apparatus according to the third aspect, when the shape of the object changes with time, the template is obtained from the result of evaluating the consistency of the arbitrary-shaped template represented in vector. An image processing apparatus is provided that performs tracking detection of a target object that changes in time series by realizing a function for updating the image.
[0019]
In invention of Claim 3 ,
Form image data based on the amount of reflection of electromagnetic waves from the object,
For the image data, by defining the vector amount of the image from the gray value of each pixel and its neighboring pixels, obtain information on the gradient and direction of the brightness,
Measure the frequency distribution of the vector length or the frequency distribution in the vector direction from the vector distribution obtained by the definition of the vector quantity to quantify the contour information in the object,
Displaying the result of representing the vector at each pixel;
Record image vector calculation results and vector frequency distribution measurement results,
In the image processing method as described above,
Provided is an image processing method characterized by extracting and recognizing a template figure by evaluating vector consistency between a vector-represented geometric figure and an arbitrarily shaped template and a vector image to be processed. .
[0020]
SUMMARY OF THE INVENTION
FIG. 1 shows a basic configuration of an image processing apparatus of the present invention. As shown in FIG. 1, the image processing apparatus of the present invention obtains an electromagnetic wave reflected from an object, for example, light (whether it is visible light or invisible light such as ultraviolet or infrared). Imaging device 1 that forms imaged or imaged data, image processing computer 2 that executes image processing, display device 3 that displays processing results, and external storage device 4 that records and stores the result data. .
[0021]
The image processing computer 2 includes an A / D conversion unit 5 that converts a digital image, a frame memory 6 that temporarily stores an input image, various image processing units 7 that perform image conversion, feature extraction, and image recognition, A central processing unit 8 that performs these operations, a system control bus 9 that transfers data, and an image transfer bus 10 are included.
[0022]
In the image processing apparatus shown in FIG. 1, image data processing is performed as shown in FIGS. As shown in FIGS. 2A to 2E, a vector image Pv is obtained from the input image Pin, a frequency distribution is obtained from the vector distribution, an edge component sharpness is obtained from the vector length frequency distribution, and the frequency in the vector direction is obtained. The inclination of the object can be obtained from the distribution. By monitoring the change in the image vector distribution in the entire image or in the local area, it is possible to detect a change in the state of the imaging target.
[0023]
In this image processing apparatus, as preprocessing in the image processing algorithm, an image vector definition in which the luminance value of a pixel is considered as one vector potential is obtained by the following equation (1).
Ri, j = {(Ii, j + 1−Ii, j−1) / 2, − (Ii + 1, j−Ii−1, j) / 2}} (1)
[0024]
2A to 2E show examples of image vector distribution. The upright image input image Pin shown in FIG. 2A and the vector frequency of the vector image Pv are shown in FIG. As shown in the left side view, it is concentrated at 90 degrees and 270 degrees. In addition, the input image Pin of the tilted image in FIG. 2B and the vector frequency of the vector image Pv are concentrated at about 60 degrees and about 240 degrees as shown in the right side view of FIG. . Similarly, in the image in which the boundary line with the background in FIG. 2D is unclear, the frequency of the vector length is expressed as shown in FIG.
[0025]
DETAILED DESCRIPTION OF THE INVENTION
FIGS. 3A and 3B and FIGS. 4A and 4B show a first embodiment of the image processing apparatus according to the present invention. In the first embodiment, as shown in FIG. 3A, an imaging device 1 that converts information on an object in space into video or time-series image data, and data from the imaging device 1 is A / D converted. Alternatively, an image processing computer 2 that executes various processes using an image vector definition by fetching into the frame memory in an image format by frame transfer via a medium, a processing result display device 3 that presents processing results to an operator, and processing The result recording device 4 stores the result in a format that can be referred to and searched.
[0026]
In the first embodiment, the inclination of the substrate to be inspected is corrected. When performing an automatic appearance inspection of electronic components, a substrate to be inspected is placed on the inspection stage and a detailed inspection regarding various items is performed. At that time, in order to perform comparison with a normal pattern prepared in advance, it is necessary to install the inspection object in the same direction as the normal pattern. Therefore, the substrate that is the inspection object is photographed by the imaging device installed above the rotation stage, and the digital input image Pin is captured by the image processing device. Then, a vector calculation process is performed on the input image to form a vector image Pv, and a processing result image Pout is formed.
[0027]
By this vector calculation process, as shown in FIG. 4B, the frequency distribution for each vector angle of the input image is measured. In the input image, if the direction of the IC on the inspection board is arranged in the vertical and horizontal directions, the frequency distribution in the vector direction is inevitably 0 °, 90 °, 180 °, 270. The frequency should be concentrated at each position of °.
[0028]
On the other hand, when the image is taken in an inclined state as in the input image Pin in FIG. 4A, the frequency distribution in the vector direction is different from the normal case. The orientation of the inspection substrate can be corrected by obtaining the inclination angle of the inspection object from this distribution difference and feeding it back to the rotary stage.
[0029]
FIG. 3B shows a flow of processing operation. According to the first embodiment, after the start, the input image is captured in step S11, and the image vector calculation process is performed in step S12.
[0030]
Next, a vector histogram measurement process is performed in step S13, and the process proceeds to step S14 to perform an inclination correction process. Subsequently, the processing result is stored and output in step S15, and the operation ends (end).
[0031]
5A, 5B, and 6 show a second embodiment of the image processing apparatus according to the present invention. As shown in FIG. 5A, the second embodiment is configured as an apparatus similar to the first embodiment. In the second embodiment, the shape of the molten pool formed on the weld base material X22 is accurately measured by melting the welding rod X21 that is an electrode.
[0032]
FIG. 6 shows various images in this image processing. In this case, in the observed image, the welding rod X21 can be identified with good contrast, but the shape is difficult to extract because the gradient between the weld pool boundary and the weld base material is gentle. The image vector amount in all pixels of the near-infrared image is calculated by the above equation (1). And the image which displayed the vector amount by the arrow is the vector image Pv.
[0033]
Moreover, the image Pc subjected to the closed region extraction process has a shape obtained by removing the contour shape of the welding rod from the molten pool shape. In this case, the segmentation of the region is performed by retrieving the connected component of the vector from the vector amount of each pixel and its neighboring pixels in the vector distribution, extracting the region closed by the vector component from the retrieval result, and evaluating the connectivity of the region And do the extraction.
[0034]
Thereby, a molten pool and the boundary part of a base material are distinguishable. A processing result image Pout is obtained by combining the closed region extraction processing image Pc with the background. The identified image Pout is displayed on the processing result display device 3, and recording processing to the result recording device 4 is also performed for history management and database creation.
[0035]
As shown in FIG. 5B, in order to observe the shape of the molten pool, the near-infrared image Pin (FIG. 6) imaged by the imaging device 1 is stored in the frame memory 4 in step S21 after the start. Then, an image vector calculation process is performed in step S22, and the process proceeds to step S23 to perform a vector search process. Next, after performing the closed region extraction process in step S24, the region area measurement process is performed in step S25. Then, in step S26, the result is stored and output, and the end is reached.
[0036]
Since the second embodiment uses a search method characterized by a vector, it is possible to detect an object whose boundary with the background is unclear and to identify a specific object from the image. it can.
[0037]
FIGS. 7A and 7B and FIGS. 8A and 8B show a third embodiment of the image processing apparatus according to the present invention. As shown in FIG. 7A, the third embodiment is composed of the same apparatus as the first embodiment. In the third embodiment, the indicated value of the circular meter X3 is automatically read.
[0038]
As shown in the flowchart of FIG. 7B, after the start, the instruction value of the circular meter X3 is photographed by the imaging device 1 in step S31. Then, the process proceeds to step S32, and the image data obtained by the imaging apparatus 1 is converted into a 640 × 480 pixel, 8-bit grayscale image by using the A / D converter (not shown) for one frame of the video signal by the image processing computer 2. And stored in the frame memory 4 (FIG. 8A). The image vector amount in all pixels of the grayscale image is calculated by the above-described equation (1). Then, the vector amount is displayed as an arrow to form a vector image Pv (FIG. 8A).
[0039]
Next, in step S33, a vector template having a shape to be detected from the target image is generated. As a figure to be used as a template, a geometric figure such as a quadrangle, a triangle, a circle and an ellipse or an arbitrary figure representing a shape specific to the object is used (FIG. 8B).
[0040]
For extraction of the circular meter, a circular template is used to detect a frame portion (black portion) of the circular meter. At this time, the image vectors of the circular outer frame and the inner frame are represented by a clockwise closed loop and a counterclockwise closed loop. For this reason, a vector template in which a clockwise rotation and a counterclockwise rotation are paired is prepared, and matching processing is performed with the vector distribution of the target image (step S34).
[0041]
In this matching process, a conversion table showing the relationship between the vector amount and the distance from the adopted template to the reference position coordinate is created in advance, and a vote is given to the coordinate value corresponding to the vector amount of each pixel in the inspection image. . At this time, since the size of the circular template is expected to change as the position of the imaging system moves, the position of the imaging system is provided by providing a setting change function for the minimum magnification, maximum magnification, and search step in the template. Even when the zoom rate is changed, the frame portion of the circular meter can be extracted flexibly.
[0042]
Next, the position (orientation) of the pointer is obtained. An isosceles triangle template is used for extracting the pointer position, and the size is set according to the magnification at the time of circular extraction. In the case of a pointer, in order to obtain the direction information indicated by the pointer, the template is rotated around the center position of the circle, and the voting process is similarly performed. By selecting the position with the largest vote value and restoring the shape, it is possible to measure the direction of the pointer and display it on the processing result display device 3 (FIG. 8A). In step S35, the result of processing is recorded in the result recording device 4 for history management and database creation.
[0043]
Since the third embodiment uses a vector quantity to search for figures and arbitrary shapes, a specified geometric figure can be extracted and recognized, and a specific object can be identified from the image. can do.
[0044]
9A, 9B, and 10 show a fourth embodiment of the image processing apparatus according to the present invention. As shown in FIG. 9A, the fourth embodiment is configured by the same apparatus as the first embodiment. In the fourth embodiment, tracking detection of a vehicle traveling on a road is performed, and traffic volume, speed, and vehicle movement are measured.
[0045]
In the fourth embodiment, the imaging device 1 attached above the road captures the driving lane and captures the input image Pin. Then, the image at _{Δt 0} is converted into a digital image by an A / D converter (not shown). At this time, after automatically detecting the vehicle portion to be detected, the image processing computer 2 performs image vector conversion to create a vector template. Then, capture images of at subsequent delta _t1 to the time delta _t0 to the image processing computer 2 performs the matching processing of the vector using the previous template, the processing result to obtain an image Pout.
[0046]
This updates the resulting region as a new template, as well as to process the image o'clock delta _t2. After repeating this until Δ _tn within the time period appearing on the screen, the measurement result is stored in the database 4.
[0047]
Since the fourth embodiment uses a method of updating a template defined by a vector amount in time series, it can flexibly cope with changes in appearance as the vehicle moves forward, and in time series. An object whose shape changes can be continuously detected.
[0048]
【The invention's effect】
As described above, the image processing apparatus of the present invention has the following effects.
[0049]
In the first aspect of the invention, by measuring a histogram of vector components in an image detected from an object, it can be used as a new index for evaluating the statistical properties of the image. In addition, since the rotation angle and the like can be extracted when an image is acquired by a camera or the like, application to automatic tilt correction and the like is possible. Moreover, by performing template matching based on the vector definition, it can be used as a new means for detecting an object of an arbitrary figure from the search image.
[0052]
Further, in the invention described in claim 2 , the object is continuously detected from the time-series image by performing template matching based on the vector definition in the invention described in claim 1 and updating the vector template according to the time change. It can be used as a new means to do this.
[0053]
In the invention described in claim 3 , by measuring the histogram of the vector component in the image detected from the object, it can be used as a new index for evaluating the statistical properties of the image. Further, the template figure can be extracted and recognized by evaluating the vector consistency between the vector-represented geometric figure and arbitrarily shaped template and the vector image to be processed.
[Brief description of the drawings]
FIG. 1 is an explanatory diagram showing a basic configuration of an image processing apparatus according to the present invention.
FIGS. 2A to 2E are views showing examples of image vector distribution of the image processing apparatus according to the present invention.
FIG. 3A is a diagram showing the configuration of the first embodiment of the present invention, and FIG. 3B is a flowchart showing the operation thereof.
FIG. 4A is a diagram showing an example of a processed image according to the first embodiment of the present invention, and FIG. 4B is a diagram showing a vector frequency thereof.
FIG. 5 (A) is a diagram showing a configuration of a second embodiment of the present invention, and FIG. 5 (B) is a flowchart showing its operation.
FIG. 6 is a view showing an example of a processed image according to the second embodiment of the present invention.
FIG. 7A is a diagram showing a configuration of a third embodiment of the present invention, and FIG. 7B is a flowchart showing its operation.
FIG. 8A is a view showing an example of a processed image according to the third embodiment of the present invention, and FIG. 8B is a view showing a template used for processing.
FIG. 9A is a diagram showing a configuration of a fourth embodiment of the present invention, and FIG. 9B is a flowchart showing its operation.
FIG. 10 is a diagram showing an example of a processed image according to the fourth embodiment of the present invention.
FIG. 11 is a diagram illustrating a configuration of a conventional image processing apparatus.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Imaging device 2 Image processing computer 3 Display device 4 Result recording device 5 A / D converter 6 Image frame memory 7 Various image processing 8 Central processing unit 9 System control bus 10 Image transfer bus 11 Image vector arithmetic unit Pin Input image Pv Vector image Pc Closed region extraction processing image Pout Processing result image X1 Inspection board X21 Welding rod X22 Welding base material X3 Circular meter X4 Vehicle

Claims (3)

Data forming means for forming image data based on the amount of reflection of electromagnetic waves from the object;
For the image data from the data forming means, an image vector computing means for obtaining information on the gradient and direction of the brightness by defining the vector amount of the image from the gray value of each pixel and its neighboring pixels,
Vector processing means for measuring the frequency distribution of the vector length or the frequency distribution in the vector direction from the vector distribution calculated by the image vector calculating means, and quantifying the contour information in the object;
A result display means for displaying a result representing a vector in each pixel;
A result recording means for recording an image vector calculation result and a vector frequency distribution measurement result;
The vector processing means converts the conversion table showing the relationship between the distance of the template geometry and any shape that is vector representation to the reference position coordinates arbitrarily set and each vector quantity, template the conversion table An image processing apparatus for extracting and recognizing a template figure by voting to a reference position coordinate corresponding to a vector amount of each pixel in a processing target . The image processing apparatus according to claim 1.
When the shape of an object changes with time, it is possible to track and detect an object that changes in time series by updating the template based on the result of evaluating the consistency of a template with an arbitrary shape expressed in vector. An image processing apparatus characterized by performing the processing. Create image data based on the amount of reflection of electromagnetic waves from the object,
For the image data, by defining the vector amount of the image from the gray value of each pixel and its neighboring pixels, information on the brightness gradient and direction is obtained,
Measure the frequency distribution of the vector length or the frequency distribution in the vector direction from the vector distribution obtained by the definition of the vector quantity to quantify the contour information in the object,
Displaying the result of representing the vector at each pixel;
Record image vector calculation results and vector frequency distribution measurement results,
In the image processing method as described above,
The vector-represented geometric figure and arbitrary-shaped template are converted into a conversion table showing the relationship between each vector quantity and the distance to the arbitrarily set reference position coordinates, and the conversion table is used as a template for each processing target. An image processing method comprising extracting and recognizing a template figure by voting to a reference position coordinate corresponding to a pixel vector amount .

Images