JP7276166B2 - Three-dimensional measuring device - Google Patents

Description

The present invention relates to a three-dimensional measuring device that measures the three-dimensional shape of an object using a phase shift method.

A device using a phase shift method is known as a three-dimensional measuring device for measuring the three-dimensional shape of an object to be measured. The phase shift method is a technique for performing triangulation by projecting a plurality of phase-shifted fringe pattern images.

Patent Literature 1 discloses a technique for shortening the time required to measure a three-dimensional shape using a three-dimensional measuring device. The three-dimensional measurement apparatus disclosed in Patent Document 1 assigns the fringes of each phase to light of different wavelengths (eg, red, green, and blue), and projects a fringe pattern image obtained by synthesizing these onto a measurement object. The object to be measured on which this fringe pattern image is projected is photographed with a color camera. Then, by extracting each color component from the photographed image, phase calculation is performed in one photographing.

Japanese Patent No. 3723057

In addition to three-dimensional measurement of an object, there is a demand for photographing a two-dimensional image of the object in order to improve the accuracy of object recognition or shape recognition, improve visibility, and the like. Note that in this specification, a two-dimensional image means an image captured in a state in which a striped pattern is not actively generated. For example, a two-dimensional image is an image captured under white light irradiation or natural light. In order to capture a two-dimensional image, it is conceivable to separately capture a two-dimensional image before and after capturing by projecting the striped light.

However, if this is done, the time at which the striped light is projected for three-dimensional measurement and the time at which the two-dimensional image is captured will not match. As a result, when the shape of an object such as a robot changes over time, or when the object moves, the shape and position of the object will differ between the 3D image and the 2D image, making image processing difficult or difficult. There was a fear that the calculation processing would increase.

The present disclosure has been made based on this situation, and an object thereof is to provide a three-dimensional measuring apparatus capable of performing three-dimensional measurement and obtaining a two-dimensional image in one shot.

The above objects are achieved by the combination of features stated in the independent claims, and the sub-claims define further advantageous embodiments. The symbols in parentheses described in the claims indicate the corresponding relationship with specific means described in the embodiments described later as one aspect, and do not limit the disclosed technical scope.

A three-dimensional measuring device for achieving the above purpose is
a projector (20) for projecting a composite striped pattern image obtained by combining a plurality of single-color striped pattern images;
A camera (30) for capturing in color the composite fringe pattern image projected onto the measurement object,
A three-dimensional measuring device that measures the three-dimensional shape of an object to be measured by a phase shift method based on an image captured by a camera,
a projection control unit (S11) for projecting a composite fringe pattern image from the projector toward the measurement object;
The luminance value of the photographed image captured by the camera in which the composite stripe pattern image is projected onto the measurement object, and is the luminance value of the photographed image for each color pixel corresponding to each of the multiple color filters provided in the camera. a captured image acquisition unit (S12) to acquire;
a phase calculation unit (S14) that calculates the phase of the luminance value for each pixel based on the luminance value of the captured image for each color pixel acquired by the captured image acquisition unit;
A coordinate determination unit (S15, S16) that determines the three-dimensional coordinates of the measurement object based on the phase calculated by the phase calculation unit;
a projection brightness determination unit (S22) that determines a projection brightness value (I _f ) for each color pixel of the composite fringe pattern image projected onto the measurement object based on the phase for each pixel calculated by the phase calculation unit;
The brightness value (I _b ) of the captured image for each color pixel is multiplied by a correction coefficient for making the projection brightness value determined by the projection brightness determination unit a predetermined constant brightness value, thereby correcting the two-dimensional image. and a correction unit (S24, S24-1) for generating.

In this three-dimensional measuring device, a composite fringe pattern image is projected onto an object to be measured, and an image of the object to be measured at that time is captured by a camera. Then, the luminance value of the captured image for each color pixel is acquired from the captured image captured by the camera. Accordingly, since the phase shift method can be applied, the three-dimensional coordinates of the object to be measured can be determined.

Furthermore, this three-dimensional measurement apparatus utilizes the phase for each pixel calculated to measure the three-dimensional shape of the object to be measured, and the phase for each color pixel of the composite fringe pattern image projected onto the object to be measured. Determines the projection luminance value. A two-dimensional image is generated by multiplying the luminance value of the captured image for each color pixel by a correction coefficient for making the projection luminance value a constant luminance value.

In this manner, since a two-dimensional image is generated by correcting an image captured by projecting a composite fringe pattern image, three-dimensional measurement and acquisition of a two-dimensional image can be performed in a single image capturing.

The correction coefficient can be a correction coefficient for making the projection luminance value determined by the projection luminance determination unit the maximum luminance value of the monochromatic stripe pattern image.

When the correction coefficients are set to such coefficients, the two-dimensional image obtained by correction using the correction coefficients can be bright and highly visible.

The projection brightness determination unit can determine the projection brightness value by substituting the phase calculated by the phase calculation unit into a preset calculation formula that can calculate the projection brightness value from the phase.

The projection brightness value is the brightness value of the image projected from the projector. Therefore, if the brightness change of the monochromatic stripe pattern is represented by a formula, the projected brightness value can be determined from the formula.

The projection brightness determination section can determine the projection brightness value based on a table or graph in which the projection brightness value is determined from the phase and the phase calculated by the phase calculation section.

If it is a table or a graph, it is possible to create a highly accurate table or graph that corresponds to fluctuations that occur in the actual environment, such as differences in brightness between color pixels. Therefore, it is possible to determine the projection brightness value with high accuracy according to the actual environment.

Furthermore, the three-dimensional measuring device
Stores a luminance value ratio table that expresses, for each color pixel, the luminance value ratio that indicates the ratio of the luminance values detected by the color pixels corresponding to the color filters of a plurality of colors provided in the camera when the camera captures a monochrome image. a table storage unit (11) for
Based on the brightness value of the captured image for each color pixel acquired by the captured image acquisition unit and the brightness value ratio indicated by the brightness value ratio table, the brightness values of other color components are removed from the brightness values of the captured image for each color pixel. and a other color removal unit (S12-1) that corrects to
The correction unit (S24-1) can set the luminance value to be multiplied by the correction coefficient as the luminance value for each color pixel after correction by the other color removal unit.

In this way, the luminance value to be multiplied by the correction coefficient is the luminance value from which the luminance values of the other color components are removed, so that the two-dimensional image is an image in which the effect of the projected composite fringe pattern is further reduced. can be

It is a figure which shows the structure of the three-dimensional measuring device 1 of 1st Embodiment. It is a figure which shows a standard striped pattern image. It is a figure which shows the change of the luminance value of each monochromatic striped pattern image on the same y pixel coordinate. It is a figure which shows the process which measures a three-dimensional shape in 1st Embodiment. It is a graph which shows the relationship between phase (theta) and height coordinate _zm . It is a figure explaining the calculation method of a horizontal coordinate ( _xm , _ym ). It is a figure which shows the two-dimensional image generation process in 1st Embodiment. It is a figure which shows the structure of the three-dimensional measuring device 100 of 2nd Embodiment. It is a figure which shows the change of the luminance value with respect to x pixel coordinate when imaging|photographing a single-color striped pattern image. FIG. 10 is a diagram showing changes in luminance value with respect to x-pixel coordinates when a standard striped turn image is captured; It is a figure which shows the luminance value ratio table generation method. FIG. 12 is a diagram showing an example of a luminance value ratio determined in S32 of FIG. 11; FIG. 12 is a diagram showing an example of a luminance value ratio table generated in S35 of FIG. 11; FIG. It is a figure which shows the effect by correction|amendment using a luminance value ratio table. It is a figure which shows the process which measures a three-dimensional shape in 1st Embodiment. It is a figure which shows the two-dimensional image generation process in 1st Embodiment.

<First embodiment>
Hereinafter, embodiments will be described based on the drawings. FIG. 1 is a diagram showing the configuration of a three-dimensional measuring device 1 according to the first embodiment. The three-dimensional measuring device 1 includes a control device 10, a projector 20, and a camera 30. A three-dimensional measurement apparatus 1 measures the three-dimensional shape of a measurement object 5 placed on a workbench 2 using a phase shift method. The three-dimensional measuring device 1 also generates a two-dimensional image of the object 5 to be measured. The two-dimensional image is an image of the measurement object 5 photographed by the camera 30 while the projector 20 is not projecting an image or is projecting monochromatic light with no luminance change.

The upper surface of the workbench 2 is flat, and the object 5 to be measured is positioned at an arbitrary position on the workbench 2 . The three-dimensional measuring device 1 is used, for example, as the eyes of a robot when the robot is caused to perform picking, assembly work, product inspection, and the like.

The controller 10 can be equipped with a computer. Control device 10 generates image data, which is data of an image projected by projector 20 , and outputs the image data to projector 20 . The image projected by the projector 20 includes a striped pattern image.

Further, the control device 10 acquires image data representing an image captured by the camera 30 while the stripe pattern image is projected from the projector 20 onto the measurement object 5 . Then, based on the image data, the three-dimensional shape of the measurement object 5 is measured by the phase shift method.

The stripe pattern image projected onto the measurement object 5 by the projector 20 in order to measure the three-dimensional shape is the standard stripe pattern image shown in FIG. The standard striped pattern image is a striped pattern image obtained by synthesizing three-color single-color striped pattern images in which the brightness is changed in a brightness range of 0 to 255 for each of red, green, and blue. The standard striped pattern image is an example of a composite striped pattern image.

A monochromatic striped pattern image is an image in which the luminance of any one of red, green, and blue varies sinusoidally in one direction of the image, and the luminance is constant in the direction perpendicular to the one direction. Further, the standard striped pattern image is such that the phases of the three-color single-color striped pattern images are shifted by a predetermined angle.

As an example, the phase of the red monochromatic stripe pattern image is the most advanced, and the phase of the green monochromatic stripe pattern image is delayed by 2π/3. The single-color fringe pattern image of blue is delayed in phase from the single-color fringe pattern image of green by 2π/3. FIG. 3 shows the change in luminance value of each single-color striped pattern image on the same y-pixel coordinates. Since the standard striped pattern image is an image that evenly includes single-color striped pattern images of red, green, and blue, the colors change in a rainbow-like manner as the x-pixel coordinates change.

The projector 20 is a projector capable of projecting color images. The camera 30 is a camera capable of capturing color images. The camera 30 is a digital camera, and has a large number of light detection elements such as photodiodes on its light receiving surface. Each photodetector corresponds to one pixel. It is assumed that the x direction of the image projected by the projector 20 and the x direction of the image captured by the camera 30 are matched.

An RGB color filter is provided in the light arrival direction of the photodetector. The RGB color filter is formed by arranging any one of red, green, and blue color filters in the light arrival direction of each photodetector. The arrangement of red, green, and blue color filters generally follows the Bayer arrangement. The distance between projector 20 and camera 30 is measured in advance.

[Processing for measuring three-dimensional shape]
Next, processing for measuring a three-dimensional shape will be described. FIG. 4 shows processing for measuring a three-dimensional shape. The processing shown in FIG. 4 is executed by the control device 10 based on the user's operation. In step (hereinafter, step is omitted) S11 which is a process of the projection control unit, the standard fringe pattern image is projected onto the measurement object 5, and the image of the measurement object 5 at that time is photographed by the camera 30. FIG.

In S12, which is a process performed by the captured image acquisition unit, the brightness value (hereinafter referred to as captured brightness value) _{Ib of} the captured image for each of the three colors of red, green, and blue is acquired. This means obtaining data indicating the shooting luminance value _Ib for each color pixel. The data output by the color pixels is sometimes called RAW data. A color pixel means a pixel that detects red light, a pixel that detects green light, or a pixel that detects blue light, or a generic term for them.

In S13, the shooting luminance value _Ib of each color is normalized. Normalization is a process of aligning the maximum luminance and minimum luminance of each color. In S14, which is a process as a phase calculation unit, based on the normalized photographing luminance value _Ib for each color pixel at each pixel coordinate (x, y), from Equation 1, at each pixel coordinate (x, y) Calculate the phase θ(x, y).

In Expression 1, N is the total number of phase shifts, and n is the number of phase shifts of the captured image acquired for each color. In the fringe pattern image, n is 0 for the color with the earliest phase, n is 1 for the color with the second earliest phase, and n is 2 for the color with the slowest phase. The total number of phase shifts N is three. Also, a(x, y) is the luminance amplitude, b(x, y) is the background luminance, and θ(x, y) is the phase θ at n=0.

In Equation 1, there are three unknowns, a(x, y), b(x, y), and θ(x, y). Therefore, if the photographing luminance value _Ib of each coordinate (x, y) for each color normalized in S13 is used, three unknowns including the phase θ, a (x, y), b (x, y), θ(x, y) can be calculated for each pixel coordinate.

S15 and S16 are processes of the coordinate determination unit. In S15, the height coordinate _zm of the coordinate measurement point P is determined from the phase θ(x, y) of each pixel coordinate (x, y) calculated in S14. A coordinate measurement point P is a point on the surface of the object 5 to be measured or the workbench 2 .

The height coordinate _zm is the distance from the plane containing the projector 20 and the camera 30 to the object. The height coordinate _zm is determined using the graph showing the relationship between the phase θ and the height coordinate _zm shown in FIG. 5 and the phase θ calculated in S14. The graph shown in FIG. 5 can be created if the coordinates of the projector 20, the coordinates of the camera 30, the height coordinate z _m , and the length of one cycle of the stripe pattern image on the reference plane are known. The reference plane is a plane that is parallel to the projection plane, which is the surface of the workbench 2, and that is at a distance of z _m from the projector 20 and the camera 30. FIG.

If the projector 20 and the camera 30 are fixed, the coordinates of the projector 20 and the coordinates of the camera 30 are known. Also, the height coordinate _zm up to the reference plane is a given value. Furthermore, if the height coordinate _zm to the reference plane is determined, the length of one cycle of the stripe pattern image on the reference plane is also determined. Therefore, the graph shown in FIG. 5 can be obtained in advance.

The graph shown in FIG. 5 is obtained in advance, and in S15, the phase θ (x, y) calculated in S14 is applied to the graph shown in FIG. Determine _zm . It should be noted that the height coordinate _zm cannot be determined if it is unknown what cycle the phase θ is. However, the height coordinate _zm at a certain coordinate measurement point P changes continuously with respect to the height coordinate _zm of the coordinate measurement point P adjacent to that coordinate measurement point P. FIG. Therefore, it is possible to measure the three-dimensional shape of the measurement object 5 even if it is unknown what period the phase θ is. Further, since the height coordinate _zm of the workbench 2 is known, the height coordinate _zm of the coordinate measurement point P may be determined by comparing with the height coordinate _zm of the workbench 2 .

In S16, the horizontal coordinates ( _xm , _ym ) are determined for the coordinate measurement point P whose height coordinate _zm was determined in S15. The height coordinate _zm determined in S15 is associated with the pixel coordinates (x, y). Once the pixel coordinates (x, y) are determined, the directions (δ _x , δ _y ) with respect to the camera 30 are determined. As shown in FIG. 6, _δx is the angle between _{the direction from the camera 30 to the coordinate measuring point P and the zm axis on the xmzm plane .} Although _δy is not shown in FIG. 6, _δy is the angle between the _{zm axis} on the _ymzm plane in the direction from the camera 30 toward the coordinate measurement point P. As can be seen from FIG. 6, the horizontal coordinates (x _m , y _m ) can be calculated from the height coordinates z _m and δ _m , δ _y by geometric calculation.

The three-dimensional shape of the measurement object 5 can be measured by executing the processing from S14 to S16 for each pixel coordinate (x, y).

[Two-dimensional image generation processing]
Next, two-dimensional image generation processing will be described. FIG. 7 shows the two-dimensional image generation processing. The two-dimensional image generation process is executed following the process of measuring the three-dimensional shape. The control device 10 also executes the two-dimensional image generation processing.

In S21, the phase θ for each color pixel is determined for each pixel coordinate (x, y). In S14, the phase θ at n=0 is calculated for each pixel coordinate (x, y). Therefore, in S21, Equation 1 is further used to calculate the phase θ at n=1 and n=2 for each pixel coordinate (x, y).

In S22, which is a process performed by the projection brightness determination unit, the phase θ for each color pixel at each pixel coordinate (x, y) determined in S21 is substituted into Equation 2 to determine the color pixel at each pixel coordinate (x, y). Determine another projection luminance value _If . In Formula 2, n and N have the same meanings as in Formula 1. α is the amplitude of the single-color striped pattern image for generating the composite striped pattern image projected by the projector 20, and β is the central value of the luminance value of the single-color striped pattern image. These α and β are preset values. Therefore, Equation 2 is a calculation formula that can calculate the projection luminance value _If from the phase θ.

The projection luminance value _If calculated by Equation 2 means the luminance value of the composite fringe pattern image projected by the projector 20 on the surface of the measurement object 5 .

In S23, a correction coefficient C for each color pixel of each pixel coordinate (x, y) is determined. The correction coefficient C is calculated using Equation (3). The numerator of Equation 3 means the highest luminance value of the monochromatic striped pattern image included in the composite striped pattern image. Therefore, the correction coefficient C is a correction coefficient for making the _projection luminance value If the maximum luminance value, which is a constant luminance value.

In S24, which is a process performed by the correction unit, the correction coefficient C determined in S23 is multiplied by the shooting brightness value _Ib for each color pixel at each pixel coordinate (x, y) determined in S12. The luminance value corrected by the correction coefficient C becomes the shooting luminance value _Ib in a state where the luminance value of the projected fringe pattern image is corrected to a constant luminance value. In other words, the luminance value after being corrected by the correction coefficient C is the luminance value photographed with light projected without luminance change. Therefore, a two-dimensional image is generated by the correction in S24.

[Summary of the first embodiment]
The three-dimensional measuring apparatus 1 of the present embodiment described above projects the standard fringe pattern image onto the measurement object 5, and the camera 30 captures the image of the measurement object 5 at that time (S11). Then, the three-dimensional coordinates of the measurement object 5 are determined by the phase shift method by acquiring the brightness values of the captured image for each color pixel from the captured image captured by the camera 30 (S15, S16).

Further, the three-dimensional measurement apparatus 1 utilizes the phase θ for each pixel calculated to measure the three-dimensional shape of the measurement object 5, and the color of the composite fringe pattern image projected onto the measurement object 5. A projection luminance value _If for each pixel is determined (S22). A two-dimensional image is generated by multiplying the photographing luminance value _Ib for each color pixel by a correction coefficient _C for making the projection luminance value If a constant luminance value (S24). Therefore, it is possible to obtain a two-dimensional image without photographing an image separately from photographing for determining the three-dimensional coordinates of the object 5 to be measured.

Further, the correction coefficient C is a coefficient that makes the projection luminance value _If the maximum luminance value. Therefore, the two-dimensional image can be made bright and highly visible.

<Second embodiment>
Next, a second embodiment will be described. In the following description of the second embodiment, the elements having the same reference numerals as those used so far are the same as the elements having the same reference numerals in the previous embodiments unless otherwise specified. Moreover, when only part of the configuration is described, the previously described embodiments can be applied to the other portions of the configuration.

FIG. 8 shows a three-dimensional measuring device 100 of the second embodiment. In the three-dimensional measuring device 100 of the second embodiment, the control device 10 has a table storage section 11 . The table storage unit 11 is a writable non-volatile storage unit. The table storage unit 11 stores a luminance value ratio table generated by executing FIG. In the second embodiment, correction using this luminance value ratio table is also performed.

First, the reason for performing correction using the luminance value ratio table will be described. In FIG. 9, a single-color striped pattern image of red, green, and blue is projected from the projector 20 onto the surface of the workbench 2, and each color pixel is detected with respect to the x-pixel coordinate when the single-colored striped pattern image is photographed by the camera 30. It shows the change in luminance value.

In FIG. 9, red pixels are denoted as R pixels, green pixels as G pixels, and blue pixels as B pixels. The same notation is used in the figures after FIG. 10 . Each of the three waveforms shown in FIG. 9 is a waveform close to a sine wave.

FIG. 10 is also a diagram showing changes in luminance values detected by each color pixel with respect to x pixel coordinates. However, FIG. 10 is a diagram when the standard striped pattern image is projected from the projector 20 onto the surface of the workbench 2 and the standard striped pattern image is photographed by the camera 30 .

Each waveform shown in FIG. 10 is a waveform considerably deviated from a sine wave. The reason for this is that the red, green, and blue filters of the camera 30 do not only transmit red, green, and blue light, but also transmit some other colors of light. Also, in the case where the projector 20 is configured to include color filters, each of the color filters included in the projector 20 also transmits light of other colors to some extent. Therefore, when the projector 20 is configured to have a color filter, when the projector 20 projects an image, the red, green, and blue lights are superimposed on the lights of other colors. .

[Description of method for generating luminance value ratio table]
Therefore, in this embodiment, the brightness value ratio table generation method shown in FIG. 11 is executed to generate a brightness value ratio table for correcting the shooting brightness value _Ib detected by each color pixel. FIG. 11 is a flow chart showing each step of the luminance value ratio table generating method. The brightness value ratio table generation method is executed once each time the combination of the projector 20 and the camera 30 changes. Moreover, the brightness value ratio table generation is executed in a state where the measurement object 5 is not placed on the workbench 2 . Since the object 5 to be measured is not placed on the workbench 2, the projector 20 projects an image on the plane surface of the workbench 2, and the camera 30 captures the projection surface.

In S31, the projector 20 projects a uniform brightness image in which the entire projection image is monochromatic and the brightness is uniform over the entire surface, and the camera 30 captures the uniform brightness image. A uniform luminance image is a correction monochromatic image for creating a luminance value ratio table.

The colors of the uniform luminance image are red, green and blue, corresponding to the color filters provided by camera 30 . In S31 once, a uniform luminance image of any one of these three colors is used. The order of colors to be used is set in advance.

In S32, the photographing luminance value _Ib for each pixel is acquired from the camera 30. FIG. The shooting brightness value _Ib to be acquired is a brightness value for each of red pixels, green pixels, and blue pixels. The luminance value ratio of the color pixels is determined from the acquired photographing luminance values _Ib . For example, if the color of the uniform luminance image projected in S31 is red, the luminance value output by the red pixel will be the largest value. Therefore, the luminance value ratio of the luminance values output by the other color pixels is determined by assuming that the luminance value output by the red pixel is 100%.

FIG. 12 shows an example of the luminance value ratio determined in S32. In FIG. 12, wavelength R, wavelength G, and wavelength B mean that the colors of the uniform luminance image projected by the projector 20 are red, green, and blue, respectively. Wavelength R, wavelength G, and wavelength B in FIG. 13 have the same meaning.

In the example shown in FIG. 12, even when a red uniform luminance image is projected, the luminance value of the green pixel is detected to be 60% of that of the red pixel, and the luminance value of the blue pixel is 10% of that of the red pixel. is detected.

In S33, it is determined whether or not uniform brightness images have been projected for all three colors. If the judgment result of S33 is NO, it will progress to S34. In S34, the color is changed to a color out of red, green, and blue for which no uniform luminance image is projected. After changing the color, S31 and subsequent steps are executed.

If the determination result of S33 is YES, the process proceeds to S35. At S35, a luminance value ratio table is generated. The generated brightness value ratio table is stored in the table storage unit 11 . FIG. 13 shows an example of the luminance value ratio table.

Each luminance value ratio shown in FIG. 12 is obtained by projecting a uniform luminance image from the projector 20 . A uniform luminance image is a monochromatic image. Each luminance value ratio shown in FIG. 12 is a ratio of luminance values detected by each color pixel when the uniform luminance image is captured by the camera 30 . The luminance value ratio table shown in FIG. 13 tabulates each luminance value ratio shown in FIG.

FIG. 14 shows the effect of correction using the luminance value ratio table. In FIG. 14, the dashed-dotted line is the brightness value output by the red pixel of the camera 30 when the standard stripe pattern image is projected. The solid line is the luminance value after correcting the luminance value indicated by the dashed-dotted line using the luminance value ratio table. The dotted line is the luminance value output by the red pixels of the camera 30 when projecting a red monochromatic striped pattern image.

Since even red pixels detect light of other colors, the luminance value of the red pixels when the standard striped pattern image is captured is, for example, near x-pixel coordinate 430, and light of other colors is detected. It is possible to observe an increase in the luminance value due to the influence of However, the waveform after correction is very similar to the waveform obtained when a single-color striped pattern image of red is captured. Therefore, it can be seen that the correction removes the luminance of the other color components.

The corrected luminance value is obtained by solving the equation shown below. In the following formula, R _d , G _d , and B _d are luminance values output by respective color pixels before correction, and R, G, and B are luminance values after correction. R _d , G _d and B _d are values detected by a set of R, G and B pixels. In addition, in the Bayer array, there are two G pixels in one pixel set. The brightness values detected by these two pixels may be averaged and used, or only one of them may be used. α _RG , α _RB , α _GR , α _GB , α _BR , and α _BG are values shown in the luminance value ratio table.

The above simultaneous equations have only three unknowns, R, G, and B. Therefore, R, G, and B, which are luminance values after correction, can be obtained by solving the simultaneous equations.

FIG. 15 shows three-dimensional measurement processing executed in the second embodiment. FIG. 15 differs from FIG. 4 only in that S12-1 is added to the processing shown in FIG.

In S12-1, which is a process performed by the other color removal unit, based on the shooting brightness value _Ib of each color acquired in S12 and the brightness value ratio indicated by the brightness value ratio table stored in the table storage unit 11, Correction is performed by removing the luminance values of other color components from the shooting luminance value _Ib for each color pixel. Specifically, the shooting luminance values I _b of each color acquired in S12 are R _d , G _d , and B _d in Equation 4, and the luminance value ratio indicated by the luminance value ratio table is the coefficient α in Equation 4. Calculate R, G, and B of 4. This processing removes at least part of the other color components.

In S13, the shooting luminance value I _b of each color calculated in S12-1 is normalized, and in S14, at least part of the other color components are removed, and the normalized shooting luminance value I _b is used to obtain the first Similar to the embodiment, the phase θ(x, y) at each pixel coordinate (x, y) is calculated. Subsequent processing is the same as in the first embodiment.

FIG. 16 shows the two-dimensional image generation processing executed in the second embodiment. FIG. 16 differs from the process shown in FIG. 7 in that S24-1 is executed instead of S24. In S24-1, which is a process performed by the correction unit, the correction coefficient C determined in S23 is applied to each pixel coordinate (x, y) from which at least a part of the luminance value of the other color component determined in S12-1 is removed. is multiplied by the photographic luminance value _Ib for each color pixel. A two-dimensional image is generated by the correction in S24-1.

[Summary of the second embodiment]
In the second embodiment described above, the table storage unit 11 stores the luminance value ratio table (FIG. 13). This brightness value ratio table indicates the ratio of brightness values detected by three color pixels of the camera 30 when a uniform brightness image, which is a monochromatic image, is captured.

When measuring the three-dimensional shape of the measurement object 5, this luminance value ratio table is used to correct the luminance values of other color components from the photographing luminance value _Ib for each color pixel (S12-1). ). Then, the phase θ is calculated based on the normalized shooting luminance value _Ib from which at least part of the other color components has been removed (S14).

Even if the camera 30 captures a standard striped pattern image in which three single-color striped pattern images are combined, the captured brightness value _Ib for calculating the phase The degree of influence is reduced. Therefore, the accuracy of the three-dimensional shape of the measurement object 5 measured based on the shooting brightness value _Ib of each color is improved.

Also in the two-dimensional image generation process, the brightness value to be multiplied by the correction coefficient C is the shooting brightness value I _b obtained in S12-1, that is, the shooting brightness value from which at least part of the brightness value of the other color component is removed. _Ib . Therefore, it is possible to generate a two-dimensional image in which the influence of the striped pattern is removed more than in the first embodiment.

Although the embodiments have been described above, the disclosed technology is not limited to the above-described embodiments, and the following modifications are also included in the disclosed scope. Various modifications can be made.

<Modification 1>
In the embodiment, the projection luminance value _If is calculated from the phase θ using the formula shown in Equation 2. However, the relationship between the phase θ and the projection luminance value _If may be stored in a table or graph form. This table or graph can be created from Equation 2, but may also be created based on actual measurements.

If a table or _graph showing the relationship between the phase θ and the projected luminance value If is created based on actual measurement values, an accurate table or graph corresponding to variations occurring in the actual environment, such as differences in brightness between color pixels, can be obtained. Can create graphs.

<Modification 2>
In the second embodiment, the uniform luminance image has been described as the monochromatic image for correction. However, the monochromatic image for correction is not limited to the uniform luminance image. For example, a single-color gradation pattern image or a single-color striped pattern image may be used as the correction single-color image. A monochromatic gradation pattern image is a monochromatic image whose luminance continuously increases or an image whose luminance continuously decreases with respect to a change in pixel coordinates of the projector 20 in one direction. The single-color striped pattern image is a single-color image in which the luminance continuously repeats increase and decrease in response to changes in the pixel coordinates of the projector 20 in one direction.

Furthermore, using these single-color gradation pattern images and single-color striped pattern images, it is also possible to determine an allowable luminance range in which a change in the luminance value ratio between a plurality of color pixels with respect to the photographing luminance value _Ib is permissible. In this case, the brightness value ratio table is determined using the brightness value ratio within the allowable brightness range, the brightness value change range of the single-color striped pattern image is corrected to the allowable brightness range, and based on the corrected single-color striped pattern image, Generate a composite stripe pattern image.

<Modification 3>
In S12-1 of the embodiment, after substituting the shooting luminance value I _b acquired in S12 into R _d , G d , _and B _d of Equation 4, the simultaneous equations of Equation 4 are solved to obtain R, G, and B. was However, before substituting values for R _d , G _d , and B _d , the simultaneous equations shown in Equation 4 are solved for R, G, and B, and in the equations solved for R, G, and B, The acquired shooting luminance value _Ib may be substituted.

<Modification 4>
In the embodiment, the phase θ(x, y) at each pixel coordinate (x, y) is calculated based on the normalized shooting luminance value _Ib for each color pixel at each pixel coordinate (x, y). rice field. However, the phase θ(x, y) at each pixel coordinate (x, y) may be calculated based on the non-normalized shooting luminance value _Ib .

<Modification 5>
In the embodiment, the correction coefficient C is a coefficient that makes the projected luminance value If the maximum luminance value of the monochromatic _fringe pattern image. However, the correction coefficient C may be a coefficient that sets the projection luminance _value If to a constant luminance value other than the maximum luminance value of the monochromatic striped pattern image, for example, the central luminance value.

1: Three-dimensional measuring device 2: Workbench 5: Object to be measured 10: Control device 11: Table storage unit 20 Projector 30: Camera S11: Projection control unit S12: Captured image acquisition unit S12-1: Other color removal unit S14: Phase calculation unit S15, S16: Coordinate determination unit S22: Projection brightness determination unit S24, S24-1: Correction unit I _f : Projection brightness value I _b : Shooting brightness value

Claims (5)

a projector (20) for projecting a composite striped pattern image obtained by combining a plurality of single-color striped pattern images;
A camera (30) for capturing in color the composite fringe pattern image projected onto the measurement object,
A three-dimensional measuring device that measures the three-dimensional shape of the measurement object by a phase shift method based on the image captured by the camera,
a projection control unit (S11) for projecting the composite fringe pattern image from the projector toward the object to be measured;
A luminance value of an image captured by the camera of the state in which the composite stripe pattern image is projected onto the measurement object, the captured image being obtained by color pixels corresponding to color filters of a plurality of colors provided by the camera. a captured image acquisition unit (S12) for acquiring the luminance value of an image;
a phase calculation unit (S14) for calculating the phase of the luminance value for each pixel based on the luminance value of the photographed image for each color pixel acquired by the photographed image acquisition unit;
a coordinate determination unit (S15, S16) that determines the three-dimensional coordinates of the measurement object based on the phase calculated by the phase calculation unit;
A projection _luminance determination unit ( S22) and
By performing correction by multiplying the luminance value (I _b ) of the captured image for each color pixel by a correction coefficient for making the projection luminance value determined by the projection luminance determination unit a predetermined constant luminance value, A three-dimensional measurement device comprising a correction unit (S24, S24-1) that generates a two-dimensional image. The three-dimensional measuring device according to claim 1,
The three-dimensional measuring device, wherein the correction coefficient is a correction coefficient for making the projection luminance value determined by the projection luminance determination unit the maximum luminance value of the monochromatic stripe pattern image. The three-dimensional measuring device according to claim 1 or 2,
The projection luminance determination unit determines the projection luminance value by substituting the phase calculated by the phase calculation unit into a preset calculation formula capable of calculating the projection luminance value from the phase. The three-dimensional measuring device according to claim 1 or 2,
The three-dimensional measuring device, wherein the projection brightness determination unit determines the projection brightness value based on a table or graph in which the projection brightness value is determined from the phase and the phase calculated by the phase calculation unit. The three-dimensional measuring device according to any one of claims 1 to 4,
A luminance value ratio representing a luminance value ratio indicating a ratio of luminance values detected by color pixels respectively corresponding to color filters of a plurality of colors provided in the camera when the camera captures a monochrome image, for each of the color pixels. a table storage unit (11) for storing tables;
Based on the luminance value of the photographed image for each color pixel acquired by the photographed image acquiring unit and the luminance value ratio indicated by the luminance value ratio table, the luminance value of the photographed image for each color pixel is extracted from the photographed image for another color. a non-color removal unit (S12-1) that performs correction to remove the luminance value of the component,
The correcting unit (S24-1) is a three-dimensional measuring device, wherein the luminance value to be multiplied by the correction coefficient is the luminance value for each color pixel corrected by the other color removing unit.

Images