JP6486010B2 - Shape measuring apparatus and shape measuring method - Google Patents

Description

  The present invention relates to a method and an apparatus for reducing three-dimensional measurement error with respect to a change in reflectance of an object surface.

  As a technology for measuring the three-dimensional shape of an object, Non-Patent Document 1 performs measurement from a plurality of directions by using a plurality of cameras and mirrors, and a technology for avoiding the influence of halation and self-hiding by combining measurement results. Is disclosed. By using a mirror, it is possible to perform measurement from more than the number of cameras.

  In addition, according to Patent Document 1, when shape measurement is performed using the lattice projection method, an error occurs near the boundary between regions having different reflectances, and a region in which an error occurs due to defocusing is expanded. And the reflectance of the object is known and constant, and a table of correction relationships is created in advance from the relationship between the shooting luminance and the error, thereby reducing the error that occurs near the boundary of the regions with different reflectances. It is disclosed.

  In addition, in Patent Document 2, when shape measurement is performed using the lattice projection method, an error occurs in the vicinity of the boundary between regions having different reflectances, and a region in which an error occurs due to defocusing is enlarged. A method is disclosed for reducing errors that occur near the boundaries of regions of differing reflectivity by using two grating projectors that are arranged centrally and symmetrically.

JP 2008-241483 A Unexamined-Japanese-Patent No. 2009-264862

Matsumoto Shingo, Fujigaki Motoharu, Kuwatani Akira, Three-dimensional shape measurement from multiple directions using mirrors by full space tabulation method, Proceedings of Stress and Strain Measurement and Strength Evaluation Symposium, 42 (2011), 191-196 .

  The techniques disclosed in the prior art documents described in the background art have the following problems. Although the technique disclosed in Non-Patent Document 1 can avoid the influence of halation and self-conclusion, it can not cope with errors generated near the boundary of regions with different reflectances. The technology disclosed in Patent Document 1 can only cope with uniform defocusing, so it is difficult to apply it to measurement objects other than flat surfaces. In order to synthesize the measurement results of the two grid projectors by simple averaging, measurement with high accuracy can not be expected unless the grid projectors capable of projecting parallel rays are arranged symmetrically.

  Therefore, in view of the above-mentioned problems of the prior art, an object of the present invention is to measure the reflectance of an object surface capable of reducing an error generated near the boundary of regions having different reflectances in measurement by the grid projection method. It is an object of the present invention to provide a method and apparatus for reducing three-dimensional measurement error with respect to change.

The invention according to claim 1 of the present application is
A three-dimensional measurement method for reducing three-dimensional measurement error due to change in reflectance of an object surface, comprising:
The weight is obtained in advance for each pixel using a camera and a plurality of grid projectors installed around the camera, and the above obtained using the grid projectors based on the previously obtained weights. This is a three-dimensional measurement method in which weighted averaging of height information on the object surface is performed for each pixel.
The invention according to claim 2 is
A three-dimensional measurement method for performing a reduction of the three-dimensional measurement error to changes in the reflectivity of the object surface,
Weights are obtained in advance for each pixel using a camera and a plurality of grid projectors installed around the camera, and are obtained using the grid projectors based on the weights obtained in advance. It is a three-dimensional measurement method in which weighted averaging of space coordinate information of the object surface is performed for each pixel .

  According to the present invention, it is possible to reduce an error generated near the boundary of different reflectances, reduce an error generated near the boundary of different materials, and reduce an error generated near the boundary of the stepped portion. It is possible to provide a three-dimensional measurement error reduction method and apparatus for changes in reflectance of

It is a figure which shows that a measurement error arises in the measurement method which combined the phase shift method with the lattice projection method. It is a figure which compares and shows the image which image | photographed the sample under indoor light, and a measurement result. It is a figure which shows the comparison of the luminance part distribution of 1 line, and a measurement result. FIG. 5 shows a sample with different reflectance patterns. It is a figure which shows the phase-analysis result of boundary vicinity of a different reflectance area | region. It is a figure which shows the comparison of the brightness | luminance and phase in line A of FIG. It is a figure which shows the relationship between a reflectance and a phase. It is a figure which shows the viewing angle of a pixel, and defocusing of a lens. It is a figure which shows reproducing the influence of the change of space decomposition by smoothing. It is a figure which shows the sum of the phase which has several amplitude. It is a figure which shows the relationship of a phase and z coordinate. It is a figure which shows the example of arrangement | positioning of one camera and two grating | lattice projection apparatuses. It is a figure which shows the example of the relationship between a true value, a measurement result, and a measurement error. It is a figure which shows acquisition of (DELTA) (theta) and (DELTA) z which used the reference plane. It is a figure which shows the inclination with respect to the pixel of the phase of a projection grating. It is a figure which shows the inclination with respect to z coordinate of the phase of the projection grating | lattice in a certain pixel. It is a figure which shows the case where a camera is in the center. It is a figure which shows the case where a camera is not in the center. It is a figure which shows the case where there are three grating | lattice projection apparatuses. It is a figure which shows the experimental shape measurement apparatus used for the experiment which concerns on this invention. It is a figure which shows a measurement sample. It is a figure which shows the measurement conditions in this experiment. It is a figure which shows the calculated (DELTA) (theta) distribution image. It is a figure which shows the calculated (DELTA) z distribution image. Calculating distribution image of weights W ₁ and W ₂ which is a diagram showing a. It is a diagram illustrating a distribution of weights W ₁ and W ₂ of the line A unit. It is a figure which shows a lattice projection image. It is a figure which shows a measurement result. It is a figure which shows the comparison of a synthetic | combination result. It is a figure which shows the comparison of the measurement result before and behind synthetic | combination. It is a figure which shows a comparison result. It is a figure showing an experiment shape measuring device. It is a figure which shows a measurement sample. It is a figure which shows a measurement result. It is a figure which shows weight distribution. It is a figure which shows a synthetic | combination result. It is a figure which shows the comparison of the measurement result of a synthetic | combination result.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(1) Measurement error due to reflectance Here, the error occurring near the boundary of different reflectances which can be reduced by the present invention will be described. First, Section 1-1 describes the relationship between reflectance distribution and measurement error. Next, in Section 1-2, the influence of the phase analysis method by the phase shift method on the reflectance distribution in the periphery will be described. And, in section 1-3, the factor of the error is considered from the influence of the spatial resolution of the camera because this error is caused by the difference in reflectance with the surroundings.

1-1 Measurement error In the measurement method that combines the grid projection method and the phase shift method, there have been cases where errors have occurred so far. An example is shown in FIG. FIG. 1 (a) is a planar measurement sample, and FIG. 1 (b) is a three-dimensional display of the measurement result. It can be seen from FIG. 1 (b) that the pattern of the sample is raised in the measurement result.

  Here, attention is focused on the position of the pixel where this error occurs. The sample was photographed under white room light and compared with the height distribution image of the measurement results. This is shown in FIG. It is FIG. 3 which extracted and compared the data of 1 line from this. It can be seen from FIG. 3 that the boundary position of the change in reflectance caused by the pattern corresponds to the position of the measurement error. Such measurement errors also occur due to the effects of dirt and the like, which may lead to false detection of defects in the field of quality inspection.

1-2 Relationship between Results of Phase Analysis and Reflectance Distribution In the analysis of a grating image using the phase shift method, independent phase values can be calculated for each pixel without being affected by background light. Since it is not necessary to refer to peripheral pixels, measurement of discontinuous objects is also possible. However, as described in Section 1-1, in the vicinity of the boundary of a region of different reflectance, such as a pattern of an object, there is an influence on the phase analysis result due to the difference in reflectance from the periphery.

  The influence of the phase analysis result is confirmed by the reflectance of the measurement sample. FIG. 4 shows a sample with different reflectance patterns. A grid pattern is projected on this sample and phase analysis is performed. The result of the analysis is shown in FIG. A comparison of the distribution of reflectance and the phase distribution extracted from line A and line B in FIG. 5 is shown in FIG. As shown in FIG. 6, the phase pattern shows irregular changes near the boundary of the regions of different reflectance. From this, it can be understood that the phase analysis result is influenced by the reflectance distribution in the periphery.

1-3 Cause of error From Section 1-2, it has been confirmed that an error occurs in the part where the difference in reflectance with the surrounding area occurs. However, in phase analysis using the phase shift method, it is possible to calculate an independent phase value for each pixel without referring to the peripheral pixels. Therefore, at the phase of phase analysis, it is not conceivable to be affected by such surroundings. Therefore, the spatial resolution of the camera at the time of lattice image photography can be considered as a factor of an error.

  Factors that determine the spatial resolution of the camera include the defocus of the lens and the size of the viewing angle of one pixel. This is shown in FIG. First, each pixel of the camera has a viewing angle, and it is not possible to capture a perfect point. On the other hand, when defocusing occurs, the viewing angle of one pixel is further increased. The change in spatial resolution of the camera due to these factors is considered to greatly affect the measurement accuracy.

2 Derivation of error amount by reflectance distribution and spatial resolution Here, the error generated near the boundary of different reflectance is derived. First, in Section 2-1, the influence of reflectance distribution and spatial resolution is examined, and the phase error is expressed in a general formula. Next, Section 2-2 describes a method of converting the phase error obtained in the previous section into a measurement error.

2-1 Derivation of Phase Error 2-1-1 Influence of Smoothing of Three Elements Regarding the amount of error near the boundary of reflectance, the relationship between the difference in reflectance and the amount of phase change is derived. First, assuming that the amplitude is A _x and the bias is B _x , the change in luminance at the time of four-time shift is expressed as Formula 1 to Formula 4.

  Next, from the relationship of spatial resolution, the case of being affected by the surrounding luminance is considered as smoothing of the three luminances. FIG. 9 is a diagram showing that the influence of the change of spatial resolution is reproduced by smoothing. In consideration of the influence of the peripheral luminance value in which the initial phase is shifted at the Δθ interval, the luminance change at the time of shift is expressed as Formula 5 to Formula 8.

  At this time, equation 9 is obtained from the equations 5 to 8 using the phase shift method.

If this is illustrated on a complex plane, it will become FIG. θ '(x) is a complex vector of amplitudes _Ai-1 , _Ai , _{Ai + 1} , and phases θ (x)-· · · (x) · · (x) + ··· It is the phase when averaged.
Further, assuming that the phase error amount is θ _E (x), several tens equations are established.

  At this time, assuming that the distance between each element is sufficiently small, Expression 11 holds.

  In addition, equation 12 has become one.

From this, it is understood that the phase error amount θ _E (x) is proportional to the magnitude of the phase difference Δθ between each element, regardless of the original phase value θ (x). That is, regardless of the original phase value, the phase error amount derived this time is inversely proportional to the pitch size of the projection grating, and becomes larger as the grating becomes finer.

2-1-2 Influence of Smoothing on n Elements Finally, the phase error amount θ _En (x) in the case of n elements (n is an odd number) is determined, and a general formula representing the phase error is obtained. Considering the sum of n phases having different amplitudes, Equation 13 is obtained.

Therefore, the phase error amount θ _En (x) is expressed by Equation 14:

Here, according to Eq. 12, the phase error amount θ _E (x) does not depend on the original phase value θ (x), so Eq. 15 holds when θ (x) = 0.

  At this time, assuming that the distance between each element is sufficiently small and Δθ is substantially equal to 0, Expression 16 holds.

In addition, Equation 17 also holds.

Further, assuming that the proportional coefficient of the phase error to the phase difference between each element is E _n (x), the equations (18) and (19) hold.

E _n is a coefficient which varies with the amplitude distribution of the spatial resolution of the camera and the measurement object. On the other hand, Δθ means the phase difference between each element, that is, the inclination of the phase of the projection grating. E _n is a coefficient for obtaining the phase error θ _E from the inclination Δθ of the phase of the projection grating, and is referred to as a phase error coefficient here.

2-2 Derivation of measurement error From this section, find the error in pixel units. When considered in pixel units, the inclination Δθ of the phase of the projection grating is the inclination of the projection grating phase at one pixel. From Eq. 19, the phase error can be derived for each pixel. Moreover, in order to obtain the value of the measurement error from this equation, it is necessary to convert the phase value into the z coordinate value. FIG. 11 shows the relationship between the phase of the projection grid and the z-coordinate value.

The direction of the phase change of the projection grating is determined by the direction of the phase shift. This time, it is assumed that the phase increases in the positive direction of the x axis. At this time, assuming that the phase value of the intersection point P between the sight line L and the x axis is θ _p , a point at which the lattice phase changes by 2π on the sight line L becomes the measurable range z _{max in the} z axis direction. Here, if the amount of change in z-coordinate per unit phase and Delta] z, the measurement error value z _e is obtained by the product of the value theta _En and Delta] z of the phase error. This is shown in equation 20. The value of Δz is because different by positional relationship grid projection device and the camera, even the same phase error theta _En, the value and the sign of the measurement error z _e changes.

However, in order to accurately derive the phase error coefficient E _n in number 20 expression, information of the grating amplitude distribution of the characteristic and subpixel resolution of the lens is required. However, it is difficult to obtain these from actual photographed images. Therefore, in order to reduce measurement error, a method which does not require the calculation of the phase error coefficients E _n is required.

3 In the plurality of reduced 3-1 error reduction principle section of the measurement error due to the grating projection apparatus, described reduction method of measurement error that does not require a phase error coefficient E _n. Phase error coefficient E _n is because in the section determined by the relationship between the camera and the measurement object does not change even by changing the position of the grid projection device. Therefore, z _{e in} Eq. 20 is determined by the inclination Δθ of the phase of the projection grating and the variation Δz of the z coordinate per unit phase.

  From this, it is considered effective to use two or more measurement results to reduce the measurement error. Grid projection is performed from different directions, and errors are reduced from the plurality of measurement results obtained. First, Δθ and Δz affected by the arrangement of the grating projection device are determined before measurement. Then, weighted averaging is performed based on these, and reduction of an error can be realized by combining a plurality of measurement results. The details of the error reduction method will be shown below, taking the case of using two grid projectors as an example.

As shown in FIG. 12, it is assumed that one camera and two grid projectors are arranged. At this time, assuming that the true value is z, the measurement results for the true value z of each of the lattice projection devices are z _m1 and z _m2 , and the measurement errors thereof are z _e1 and z _e2 . At this time, when the true value is expressed using the respective measurement results and the measurement error, it is expressed as Expression 21 and Expression 22.

If E _n is eliminated from these equations, equation 23 is obtained.

From equation 23, without using the phase error coefficients E _n, it can be from two measurement results obtain the true value. This means that combining using weighted averaging is performed on each measurement result. Equations for calculating weights are shown in Equations 24 and 25.

As described above, the combining process according to the present invention can reflect the difference between the projection grid pitches and the position of the grid projector in the weight distribution. Therefore, it is possible to combine the measurement results even under measurement conditions in which the grating pitch and the measurement range in each grating projection apparatus are different. Thus, the method of the present invention does not require precise alignment of the camera and the grid projector.
Further, a relationship similar to the relationship between the error of the z coordinate and the phase error holds also for the relationship between the error of the x, y coordinates and the phase error. Therefore, the error reduction method using weighted averaging is also effective for x, y, z coordinates (space coordinates).

3-2 Calculation Method of Δθ and Δz The method according to the present invention requires calculation of the inclination Δθ of the phase of the projection grating and the change amount Δz of the z-coordinate value per unit phase. Here, a method of using a flat plate as a reference plane as shown in FIG. 14 is shown as an example of the Δθ and Δz calculation method.

3-2-1 Calculation of the inclination (Δθ) of the phase of the projection grating Since the inclination Δθ of the phase of the projection grating is the inclination of the phase of the projection grating with respect to the pixels, The calculation can be performed by using an equation. FIG. 15 is a diagram showing the inclination of the phase of the projection grating with respect to the pixels.

3-2-2 Calculation of variation (z) of z-coordinate value per unit phase The variation? Z of z-coordinate value per unit phase is the inclination of the phase of the projection grid with respect to the z-coordinate. Therefore, the calculation can be performed from the amount of change in the z-coordinate direction of the projected phase distribution of the grating. This time, using the reference plane arranged at each z coordinate position, the relationship between the z coordinate and the phase was obtained for each pixel. Then, an approximate straight line was obtained for each pixel using the least squares method, and the reciprocal of the slope of this straight line was used. FIG. 16 is a diagram showing the inclination of the phase of the projection grid with respect to the z coordinate at a certain pixel. By using Δθ and Δz obtained as described above, weights W ₁ and W ₂ can be obtained from Eq. 24 and Eq. 25.

3-3-3 Error reduction using three or more grating projectors This section describes error reduction using three or more grating projectors. In the method of the present invention, when the number of lattice projection apparatuses is increased to three or more, the weight can not be obtained from Eq. Therefore, weights are derived in the case of using three or more grid projectors using a pseudo inverse matrix. When the number of grid projection device is D stage, the relationship of total measurement result z _m and the measurement error z _e and the true value z in each grid projection device is expressed as the number 27 formula to several 29 expression.

  If this is expressed in a determinant, it is expressed by equation 30.

  Here, assuming that the ratio of errors included in each measurement result is a matrix R, it is expressed by Equation 31.

When the matrix R is not regular, the pseudo inverse matrix R ⁺ of R is expressed by Equation 32.

  The equation 30 can be transformed into the equation 33 by using the equation 32.

  From the above equation, a combination equation in error reduction using three or more grating projectors can be obtained as in Equation 34.

3-4 Arrangement Example of Lattice Projection Apparatus The method of the present invention can reflect the arrangement of a plurality of lattice projection apparatuses in the weight, so that errors can be reduced in various arrangements. Here, an example of the assumed arrangement of cameras and grid projectors is shown. FIG. 17 shows the case where the camera is at the center. FIG. 18 shows the case where the camera is not at the center. FIG. 19 is a view showing the case where there are three grid projectors.

4 Error reduction experiment 4-1 Measurement of planar sample In this section, measurement errors that occur near boundaries of different reflectances are reduced by a shape measurement device that uses two grid projectors. An experimental shape measuring apparatus used for this experiment is shown in FIG. 20, and a measurement sample is shown in FIG. In this case, the angles of the grid projectors are changed from left to right because the projection grids from the two grid projectors perform measurement under different conditions. The measurement conditions in this experiment are shown in FIG. The average measurement range in the z-axis direction is 9.7 mm for Projector 1 and 16.9 mm for Projector 2.

  In order to combine measurement results, it is necessary to obtain the values of Δθ and Δz used at the time of combining. For these, an image taken at the time of calibration for obtaining the correspondence between the phase and the z coordinate is used. This time, as shown in FIG. 22, in the calibration, images of 51 reference surface phase distributions are obtained.

  First, 51 Δθ distribution images are obtained from the calibration result. As in the previous chapter, using Equation 35, a distribution image of the inclination Δθ of the phase of the projection grating can be obtained. The 51 distribution images of Δθ obtained are averaged to obtain one image. Furthermore, noise reduction is performed by smoothing processing using the least squares method. The results are shown in FIG.

Next, the Δz distribution is calculated. The Δz distribution is obtained by determining the phase change in the z coordinate axis direction in the calibration data using the least squares method. The results are shown in FIG.

Then, using the equations (24) and (25), the weight distribution used for the composition was obtained from the images of FIG. 23 and FIG. This is shown in FIG. Figure 25 is a diagram showing a distribution image of weights W ₁ and W ₂ calculated. Moreover, what extracted and compared the data of the line A part from FIG. 25 is shown in FIG. From this, it can be seen that the weighting is different in the two grid projectors.

  The sample of FIG. 25 was measured using each of the grid projectors, and combining was performed by weighted averaging using the calculated weight distribution. The image of the projection grid in the measurement experiment performed this time is shown in FIG. 27, and the measurement result is shown in FIG. Using this result, we compared the synthesis by weighted average of the proposed method and the synthesis by simple average, which is the conventional method. The synthesis result by the simple average is shown in FIG. 29 (a), and the synthesis result by the weighted average is shown in FIG. 29 (b).

  The data of line B was extracted, and the error reduction by combination was confirmed. This is shown in FIG. Although the synthesis by the simple average generated a part where the error remained, the synthesis by the weighted average did not occur. Moreover, the value at i = 326 was extracted from this graph, and the change of the measurement accuracy by the combining method was confirmed. The true value was determined by linear approximation using the least squares method and was set to 6.007 mm. At this time, the values of 10 pixels near the error were not used in order to make the line not influenced by the error. The results of extracting and comparing the values of one line from FIG. 30 are shown in FIG.

  The effectiveness of the present invention can be confirmed from FIG. Although the measurement error was 0.257 mm in the case of Projector 1 alone, it could be reduced to 0.030 mm by combining by weighted averaging using the present invention. On the other hand, the measurement error in the case of simple averaging was 0.056 mm. The synthesis result according to the present invention was able to reduce the measurement error to about half of the synthesis result by simple averaging. Therefore, it can be said that the error reduction using this method is effective to the error generated at the boundary of different reflectances.

4-2 Measurement of Electronic Substrate An attempt was made to reduce the measurement error according to the present invention using a measurement apparatus using two grid projectors. An apparatus used for the experiment is shown in FIG. 32, and an electronic substrate having a measurement sample is shown in FIG. The influence of white letters on the black chip of this substrate on the measurement results was compared before and after the error reduction. The measurement result by each grating | lattice projection apparatus is shown in FIG. Also, FIG. 35 shows the weight distribution used for the synthesis, and FIG. 36 shows the synthesis result. The data of Line A is extracted from the measurement results before and after synthesis, and the comparison is shown in FIG. It can be seen that the variation caused by the measurement error is reduced.

1 Projection device 2 Projection device

Claims (2)

A three-dimensional measurement method for reducing three-dimensional measurement error due to change in reflectance of an object surface, comprising:
The weight is obtained in advance for each pixel using a camera and a plurality of grid projectors installed around the camera, and the above obtained using the grid projectors based on the previously obtained weights. A three-dimensional measurement method that performs weighted averaging of height information of the object surface for each pixel. A three-dimensional measurement method for reducing three-dimensional measurement error due to change in reflectance of an object surface , comprising:
Using a camera and a plurality of grid projectors placed around the camera,
A three-dimensional measuring method in which weights are previously obtained for each pixel, and weighted averaging of spatial coordinate information of the object surface obtained using each of the grid projectors based on the previously calculated weights is performed for each pixel .