JP3822570B2 - Image processing apparatus and method - Google Patents

Description

[0001]
[Industrial application fields]
The present invention relates to an image processing apparatus and a method thereof.
[0002]
[Prior art]
Conventionally, a range finder has been proposed as an apparatus for observing a three-dimensional position.
[0003]
The principle of a range finder that measures a three-dimensional position using a laser spot as an optical pattern will be described with reference to FIG.
[0004]
The laser beam 518 emitted from the laser light source 517 goes straight until it reaches the object. If the object 519 is present on the beam straight line, the beam 518 generates a spot at the point 520 and forms an image at the point 524 on the imaging surface 523.
[0005]
When the object 521 is present instead of the object 519, the beam 518 generates a spot at the point 522 and forms an image at the point 525 on the imaging surface 523. This point is extracted as a point having the maximum peak value in the luminance distribution on the imaging surface.
[0006]
At this time, the spatial position of the point 520 is obtained when the intersection coordinate value of the beam straight line 518 and the line of sight 520-524 is obtained, and the spatial position of the point 522 is obtained when the intersection coordinate value of the beam straight line 518 and the line of sight 522-525 is obtained. .
[0007]
This is the principle of range finder coordinate calculation.
[0008]
In this range finder, the pattern projection means and the image input means need to be arranged at different positions. Therefore, only the places on the object where both can be seen simultaneously can be input. Due to the shape of the object surface, a place where both of them cannot be seen by chance remains as a place that cannot be entered.
[0009]
Such a part is called occlusion.
[0010]
An example of this occlusion is shown in FIG.
[0011]
The laser beam 527 emitted from the laser light source 526 generates a spot at a point 529 on the object 528. However, this spot is blocked by the point 530 on the object 528 and does not reach the point 532 on the imaging surface 531. As a result, since the image of the point 529 is not detected, the spatial position of this point cannot be calculated. This is occlusion.
[0012]
As a method for dealing with the above occlusion, for example, a survival method proposed in IEICE TRANS. Vol.
[0013]
This will be described with reference to FIG.
[0014]
The image input means is placed in two places, and the imaging surfaces 538 and 540 are installed.
[0015]
The laser beam 534 emitted from the laser light source 533 goes straight until it reaches the object. If the object 535 is present on the beam straight line, the laser beam 534 should generate a spot at the point 536 and form images at points 539 and 541 on the imaging surface 538 and the imaging surface 540.
[0016]
However, as shown in FIG. 8, the spot generated at the point 536 is blocked by the point 537 and does not reach the point 539.
[0017]
On the other hand, since an image is formed at the point 541 on the imaging surface 540, the spatial position of the point 536 can be obtained based on this point by the principle of coordinate calculation.
[0018]
A method of obtaining a spatial position from an input obtained by using a plurality of image input means in this way is called a “survival method”, and occlusion can be avoided by this method.
[0019]
A conventional apparatus having a plurality of imaging surfaces (or a plurality of optical paths) implements this survival method.
[0020]
In general, if a primary reflection spot formed directly on the object surface by the laser beam is observed, the spatial coordinate value calculated by the range finder principle correctly gives the coordinate value of the sample point on the object surface.
[0021]
However, when this laser beam is further reflected and a secondary or subsequent spot formed by hitting another part of the object surface shows the strongest reflection, this spot is observed and an erroneous spatial coordinate point may be calculated. Such noise is called spike noise.
[0022]
The principle of spike noise generation is shown in FIG.
[0023]
The laser beam 543 emitted from the laser light source 542 generates a primary spot at a point 545 of the object 544 and forms an image at a point 546 on the imaging surface 550.
[0024]
On the other hand, the laser beam 543 is reflected at a point 545 to generate a secondary spot at a point 547 on the object, and an image is formed at a point 548 on the imaging surface 550.
[0025]
If the luminance of the point 548 happens to be brighter than the point 546, the point having the maximum peak value in the luminance distribution appearing on the imaging surface is the point 548, and the point 549 is obtained by calculating the spatial position from this point. As a result, there is an illusion that there is a sample point on the object surface at a point 549 that floats in the air instead of the actual point 545. This is spike noise.
[0026]
Currently, a low-pass filter is used as a method for dealing with the spike noise.
[0027]
This is because if there is a point that protrudes excessively compared to the data around the object surface, it is regarded as spike noise, and the calculated value of this point is rejected and recalculated from the surrounding points using a local average, etc. Is the method.
[0028]
However, this method has three problems.
[0029]
(1) Necessary ambient data is required to determine whether the noise is spike noise. In other words, it is impossible to determine whether or not there is spike noise by looking only at that point and not to remove it.
[0030]
(2) There is no index for determining a threshold for determining spike noise. Therefore, the current situation is that the threshold is determined by human judgment.
[0031]
(3) In the case of an object rich in undulations, it is difficult to determine whether spike noise is present even when compared with surrounding data. Moreover, when occlusion and spike noise occur simultaneously, it is impossible to avoid both.
[0032]
[Problems to be solved by the invention]
As described above, the conventional range finder has problems such as a large amount of data required for dealing with spike noise and a spike detection threshold value that cannot be automatically determined.
[0033]
Although there were methods for avoiding occlusion and spike noise, it was impossible to deal with both at the same time.
[0034]
The present invention has been made in view of such problems, and an object of the present invention is to provide an image processing apparatus with improved convenience by solving the above problems.
[0035]
[Means for Solving the Problems]
According to the first aspect of the present invention, a light pattern irradiating means for irradiating a light pattern toward the surface of the observed object and an image of the reflected light of the light pattern reflected by the surface of the observed object. Two or more imaging means, a peak detection means for detecting one or more luminance peaks estimated as reflected light on the surface of the object to be observed from the images captured by the imaging means, and the peak The trajectory of the reflected light is calculated from the luminance peak detected by the detection means, the trajectory at the time of irradiation of the light pattern that is the source of reflection of the reflected light is calculated, and the intersection of the two trajectories is calculated. position calculating means for calculating the position coordinates in the space where the observation object exists, classifies the intersection obtained position coordinates is distributed within a predetermined distance from the position calculating unit into one cluster, A score calculating means for said number of intersections included in the cluster with the score of this cluster, among the clusters obtained by the score calculating means, the average coordinate calculating an average coordinate of the intersection constituting the highest scoring clusters Image processing comprising: calculation means; and output means for outputting the average coordinates calculated by the average coordinate calculation means as the coordinates of points on the surface of the observed object sampled by the optical pattern Device.
[0038]
[Operation]
The image processing apparatus of the present invention will be described.
[0039]
Two or more imaging means captures the light pattern of the surface of the non-observed object.
[0040]
The peak detection means detects one or more luminance peaks estimated as an optical pattern image from the images picked up by these image pickup means.
[0041]
The position calculation means calculates one coordinate of a point in the space corresponding to the peak from the peak detected by the peak detection means and the locus of the optical pattern. That is, the intersection of the peak and the trace of the optical pattern is set as one coordinate.
[0042]
The score calculation means distributes the points obtained from the position calculation means to clusters distributed within a predetermined distance, and sets the number of points included in each cluster as the score of this cluster.
[0043]
The average coordinate detection means calculates the average coordinates of the points constituting the cluster having the highest score among the clusters obtained by the score calculation means.
[0044]
The output means outputs the average coordinates are calculated by the average coordinate detecting means, as the coordinates of a point on the object to be observed object sampled by the light pattern.
[0045]
According to this image processing apparatus, since the points constituting the cluster with the highest score are the coordinates of the points on the sampled object to be sampled, spike noise can be removed.
[0046]
【Example】
(First embodiment)
A first embodiment of a three-dimensional sensor device according to an image processing apparatus of the present invention will be described with reference to the drawings.
[0047]
FIG. 1 is an overall configuration diagram of the three-dimensional sensor device of the present embodiment.
[0048]
Reference numeral 501 denotes an optical pattern projection unit for forcibly generating an optical pattern on the object surface, such as a laser light source.
[0049]
Reference numerals 502 and 505 denote imaging units which are first to n-th image input means for photographing a generated pattern.
[0050]
Reference numerals 503 and 506 denote peak detection units for calculating the luminance distribution of the captured image and detecting the luminance peak value therefrom.
[0051]
Reference numerals 504 and 507 denote position calculation units for calculating the spatial coordinates of the sample points on the object from the points detected by the peak detection unit.
[0052]
Reference numeral 508 denotes a score calculation for classifying points on the object surface calculated from the first to n-th imaging units 502 and 505 into clusters in the ε range and determining the score of the cluster based on the number of points in the cluster. Part.
[0053]
Reference numeral 509 denotes an output unit that outputs the representative point of the cluster that has obtained the highest score in the score calculation unit as spatial position information.
[0054]
The principle of the “majority decision method” that makes it possible to avoid spike noise proposed as the present invention will be described with reference to FIG.
[0055]
In the majority method, the image pickup units 555 and 557 of the camera as image input means are placed. In the case of FIG. 2, it is assumed to be placed in two places for the sake of simplicity.
[0056]
The laser beam 552 emitted from the laser light source 551 reaches the point 554 of the object 553 and generates a primary spot. Then, an image is formed at a point 556 on the imaging surface 555 and a point 558 on the imaging surface 557.
[0057]
On the other hand, the laser beam 552 is reflected at the point 554 to form a secondary spot at the point 559, and an image is formed at the point 560 on the imaging surface 555 and the point 561 on the imaging surface 557.
[0058]
Therefore, when peak detection is performed from the imaging units 555 and 557, four points 556, 558, 560, and 561 are listed as candidate points.
[0059]
Next, spatial coordinates are calculated from these four points and the laser beam 552.
[0060]
Here, assuming that the sample point on the object surface is a true point, even if the position is calculated separately from the two cameras, the position of the true point should indicate almost the same point. However, since an error at the time of setting the camera or a quantization error is included, the same point is not necessarily shown.
[0061]
Therefore, a cluster in the ε range centering on each point is generated. The number of points included in the cluster is taken as the score of this cluster, the spatial coordinates of the points in the cluster with the highest score are averaged, and the averaged spatial coordinates are regarded as true points.
[0062]
In the case of FIG. 2, the trajectory of the beam emitted from the laser light source 551 and the intersection of the trajectories (552-556, 552-558, 552-560, 552-561) of the beam reflected from the object 553 and obtained five points. To obtain the spatial coordinates. That is, a point 554 is calculated from the points 556 and 558, a point far from the object is calculated from the point 560, and a point 562 is calculated from the point 561.
[0063]
When each cluster is generated, since the cluster including the point 554 obtains the highest two points, it is confirmed that the point 554 is a true point and the other points are spike noises.
[0064]
The above method is called the majority method. This method can be used from two or more imaging units, and the greater the number, the greater the stability.
[0065]
(Second embodiment)
Next, a second embodiment of the present invention will be described with reference to FIGS.
[0066]
In this embodiment, both the survival method and the majority method are used in order to be able to avoid both occlusion and spike noise at the same time.
[0067]
In this method, three or more image input means are arranged. FIG. 3 shows an example in which three cameras are arranged as image input means, and the imaging surfaces of the cameras are placed at 568, 570, and 572.
[0068]
In FIG. 3, a laser beam 564 emitted from a laser light source 563 reaches a point 566 on an object 565 and generates a primary spot.
[0069]
At this time, the point 569 on the imaging surface 568 is occluded by the point 567 and an image is not formed. However, since the images are captured by a plurality of cameras, the point 571 on the imaging surfaces 570 and 572 is obtained in accordance with the principle of the survival method. And 573 are images. Therefore, occlusion is avoided (D01 in FIG. 4).
[0070]
On the other hand, the laser beam 564 is reflected at a point 566 to form a secondary spot at a point 574, and images are formed at points 575, 576, and 577 on the three imaging surfaces 568, 570, and 572, and spike noise is generated.
[0071]
Here, a point where the brightness value of surrounding points is lower than that of itself is extracted as a peak candidate (D02 in FIG. 4), and threshold processing is performed to detect a brightness peak from the peak candidates.
[0072]
In the case of FIG. 3, when peak detection is performed from three imaging units, five points 571, 573, 575, 576, and 577 are detected as points having luminance peak values (D03 in FIG. 4).
[0073]
Accordingly, the trajectory of the beam emitted from the light source 563 and the intersection of the beam trajectories (566-571, 566-573, 574-575, 574-576, 574-577) reflected from the object 565 to obtain these five points are obtained. By obtaining, the spatial coordinates around the points 566, 578 and 579 are calculated (D04 in FIG. 4).
[0074]
Similar to the majority method, a cluster in the ε range is generated from this intersection. In the case of FIG. 3, a cluster including the point 566, a cluster including the point 578, and a cluster including the point 579 are generated (D05 in FIG. 4).
[0075]
Next, for each cluster, the number of points included in the cluster is regarded as a score, and score calculation is performed (D06 in FIG. 4).
[0076]
In FIG. 3, since the cluster including the point 566 calculated from the points 571 and 573 obtains the maximum two points, a true point exists in the cluster including the point 566. By this score calculation, the points 578 and 579 which were spike noises were removed (D07 in FIG. 4).
[0077]
Finally, the true point is determined by averaging the points in the cluster including the point 566 (D08 in FIG. 4).
[0078]
Thus, even if occlusion and spike noise occur at the same time, it is possible to avoid both at the same time by using the survival method and the majority method together by placing three or more cameras.
[0079]
(Third embodiment)
A third embodiment of the present invention will be described below.
[0080]
As shown in FIG. 5, a plurality of image pickup units 511 or 512 as the image input means are arranged, and the images picked up by each are sequentially processed by the peak detection unit 513 to detect the luminance peak for all the images. . Based on the peak detected by the peak detection, the position detection unit 514 calculates and accumulates the position coordinates of the points in the space. When the processing of a plurality of images taken at the same time is finished, the accumulated position coordinates are output to the point calculation unit 515, the true point is calculated by calculating the score, and the result is output to the output unit 516.
[0081]
(Modification)
In addition, this invention is not limited to said Example, It can implement by changing variously. That is, as long as at least the majority method can be executed, the quantity relationship among the imaging means, the peak detection means, and the position calculation means can be variously set.
[0082]
For example, it is possible to adopt a configuration with n imaging means, m peak detecting means, and one position calculating means.
[0083]
In short, in the present invention, there is no difference in the structural details as long as at least the majority method is feasible or the majority method and the survival method are compatible.
[0084]
【The invention's effect】
According to the present invention, spike noise can be removed by the majority method using the points constituting the highest-scoring cluster as the coordinates of the points on the sampled object to be sampled, so that the spatial position on the object to be observed can be reliably determined. Can be calculated.
[Brief description of the drawings]
FIG. 1 is an overall configuration diagram of a three-dimensional sensor device according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a majority method for avoiding spike noise in a three-dimensional sensor device;
FIG. 3 is a diagram illustrating an operation when occlusion and spike noise occur in the three-dimensional sensor device.
FIG. 4 is a flowchart showing a flow of processing in the three-dimensional sensor device.
FIG. 5 shows an overall configuration of another embodiment of the three-dimensional sensor device according to the second invention.
FIG. 6 is a principle view of a range finder.
FIG. 7 is a diagram showing the principle of occlusion generation in a range finder.
FIG. 8 is a diagram illustrating the principle of a survival method for avoiding occlusion in a range finder.
FIG. 9 is a diagram showing the principle of spike noise generation in a range finder.
[Explanation of symbols]
501 Optical pattern projection unit 502 First imaging unit 503 First peak detection unit 504 First position calculation unit 505 n-th imaging unit 506 n-th peak detection unit 507 n-th position calculation unit 508 score calculation unit 509 Output section

Claims (3)

A light pattern irradiating means for irradiating a light pattern toward the surface of the object to be observed;
And two or more imaging means for imaging the reflected light of the light pattern in which the irradiation light patterns are reflected by the object to be observed object surface,
Peak detecting means for detecting one or more luminance peaks estimated as reflected light on the surface of the observed object of the light pattern from the images respectively captured by the imaging means;
The trajectory of the reflected light is calculated from the luminance peak detected by the peak detection means, and the trajectory at the time of irradiation of the light pattern that is the source of reflection of the reflected light is calculated. About the intersection of these two trajectories Position calculating means for calculating position coordinates in a space where the observed object exists ;
Classifying the intersections in which the position coordinates obtained from the position calculation means are distributed within a predetermined distance into one cluster, and using the number of the intersections included in each cluster as the score of the cluster, score calculation means;
Among the clusters obtained by the score calculation means, average coordinate calculation means for calculating the average coordinates of the intersections constituting the cluster with the highest score;
An image processing apparatus comprising: output means for outputting the average coordinates calculated by the average coordinate calculation means as coordinates of points on the surface of the observed object sampled by the optical pattern. Having said imaging means three or more, and, the said reflected light from the observation object surface, the image processing apparatus according to claim 1, wherein the at least one imaging means is arranged so as to be captured . An image processing method performed by an image processing apparatus,
A light pattern irradiation step of irradiating a light pattern toward the surface of the object to be observed;
And two or more imaging step of imaging the reflected light of the light pattern in which the irradiation light patterns are reflected by the object to be observed object surface,
A peak detection step for detecting one or more luminance peaks estimated as reflected light on the surface of the observed object of the light pattern from images captured in each of these imaging steps;
The trajectory of the reflected light is calculated from the luminance peak detected in this peak detection step, the trajectory at the time of irradiation of the light pattern that is the source of reflection of the reflected light is calculated, and the intersection of these two trajectories is calculated. a position calculation step of calculating the position coordinates in the space in which the pre-Symbol the observed object is present,
Classifying the intersections in which the position coordinates obtained in the position calculation step are distributed within a predetermined distance into one cluster, and calculating the number of intersections included in each cluster as the score of the cluster; and
Of the clusters obtained in this score calculation step, an average coordinate calculation step for calculating the average coordinates of the intersections constituting the cluster with the highest score;
An image processing method comprising: an output step of outputting the average coordinates calculated in the average coordinate calculation step as coordinates of points on the surface of the observed object sampled by the light pattern.

Images