JP7444396B2 - 3D image measurement system, 3D image measurement method, 3D image measurement program, and recording medium - Google Patents

Description

The present invention relates to a three-dimensional image measurement system, a three-dimensional image measurement method, a three-dimensional image measurement program, and a recording medium for measuring the shape and dimensions of a construction site in fields such as architecture, civil engineering, bridge and tunnel construction, etc.

There are many needs for updating shutters and adding new shutters in warehouses, garages, shops, underground malls, tunnels, etc. When renewing or adding new shutters, etc., the shutters are often fabricated and installed according to parameters such as the shape and size of the site where they actually need to be installed, rather than based on design drawings. Conventionally, most on-site parameters have been measured by workers using a tape measure or the like and recorded on paper. With such on-site measurement methods, there are individual differences in measurement results. Furthermore, it is difficult to measure at high places or over a wide area. For this reason, shutters and the like manufactured based on the measurement results may not be able to be installed on site.

Conventionally, there are measurement methods using laser rangefinders, ultrasonic rangefinders, etc. However, these measuring devices have problems such as being able to only measure the straight distance between two points from the measuring device to the measurement target point, making it difficult to measure details or specific parts. Furthermore, only the distance between two points can be measured in one measurement, and measurement time is required to measure many field parameters as described above.

Furthermore, as a measurement method, it is possible to use image measurement technology. Image measurement techniques can be generally classified into two types of measurement methods: two-dimensional image measurement and three-dimensional image measurement. However, since the manufacturing and installation of shutters and the like requires the three-dimensional shape and parameters of the construction site, it is difficult to apply two-dimensional image measurement, and it is necessary to use three-dimensional image measurement technology.

Typical three-dimensional image measurement methods include methods such as a stereoscopic measurement method based on binocular vision, a measurement method based on patterned light projection, and a lens focusing method based on adjustment of the focal length of a lens.

Pattern light projection measurement is a method of calculating three-dimensional information of a measurement target by projecting pattern light for measurement onto a measurement target and analyzing its reflection pattern. However, when the object to be measured is the shutter described above, the size of the shutter is large, ranging from several meters to over 10 meters, and it is difficult to create and project a pattern of light that covers the entire shutter. Difficult to apply to measurement.

On the other hand, the lens focusing method is often used for microscopic measurements, etc., and can simultaneously realize three-dimensional and two-dimensional measurements of surface shapes by adjusting the focus and photographing multiple times. However, measurement requires lens adjustment and photographing several to several dozen times, and it is necessary to know the depth distance value corresponding to each focal point in advance, so it is considered unsuitable for shutter measurement.

The stereo measurement method is a method in which photos of a measurement target are taken with two cameras from different viewpoints, and three-dimensional information of the measurement target point is calculated by image analysis. According to this stereo measurement method, non-contact measurement can be performed using only a camera, and it is possible to measure objects of different sizes by adjusting the focus of the lens, so it is considered suitable for shutter measurement.

Camera system calibration, which determines camera system parameters such as camera focal length, is one of the most important techniques in stereo measurement. When measuring construction sites such as shutters, it is necessary to take pictures while changing the focal length of the camera lens depending on the size of the measurement target and the position of the person measuring it. Therefore, it is necessary to calibrate the camera system each time measurement is performed, and it is difficult to calibrate the camera system in advance, and the method of determining camera parameters is difficult to apply to measurements of shutters and the like.

The calibration method described in Non-Patent Document 1 has been frequently used in recent years, and has been compiled into a library as described in Non-Patent Document 2. When performing on-site measurements, one method is to fix the focal length and calibrate it after adjusting the focal length of the camera for shooting, but it is necessary to calibrate it every time you take a photo. This calibration method is not suitable for field measurements.

Research on image measurement of buildings has been conducted for some time, and several patents have been filed. In Patent Document 1, the present inventor has proposed a method of taking two photographs of a measurement target by raising and lowering an elevator on a tripod, and performing three-dimensional image measurement. This method is time-consuming because it requires the user to operate the tripod's elevator and take two photographs, one above and one above, each time a photograph is taken. Furthermore, since the camera is fixed on a tripod, the photographing angle cannot be freely adjusted, resulting in a narrow photographic range.

Further, for example, Patent Document 2 proposes a three-dimensional image measurement method for a covering material of a building. However, the measurement purpose of this three-dimensional image measurement method is the shape and thickness of the covering material, and cannot measure the dimensions of an object to be measured such as a shutter. Further, for example, Patent Document 3 proposes a method of generating three-dimensional data of the shape of a building based on a stereo measurement method using two cameras. However, the purpose of this method is to measure the shape of buildings based on straight line extraction, and is not suitable for precision measurement of the dimensions of objects to be measured such as shutters.

Furthermore, when a shutter or the like is to be attached to a building, a tunnel, or the like, the attachment position is specified, and a shutter or the like whose shape and dimensions match the specified attachment position is manufactured and attached. In order to identify the mounting position, methods such as painting or attaching a marker are used. However, these methods not only have problems with the work time, but also have problems with the difficulty of painting and marking high places, and with aesthetic issues as it is necessary to leave the paint and markers until the construction is completed.

Japanese Patent Application Publication No. 2005-164434 JP2020-76585A International Publication No. 2013/002280

Z. Zhang, A Flexible New Technique for Camera Calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 22: pp. 1330-1334, 2000 OpenCV, Camera Calibration and 3D Reconstruction, Internet <URL: https://docs.opencv.org/master/d9/d0c/group__calib3d.html#ga3207604e4b1a1758aa66acb6ed5aa65d>

An object of the present invention is to make it possible to measure the shape and dimensions of a measurement target by taking a plurality of photos of the measurement target at a location where it is easy to take pictures, and using three-dimensional image measurement technology.

The three-dimensional image measurement system of the present invention includes a light projection device that projects linear light, a plumb bob device in which a weight is suspended from a string-like member having feature points, a string-like member of the plumb-bang device, and a light projector. A photographing means for photographing the object to be measured from a plurality of viewpoints, including linear light projected from the arbitrary viewpoint dynamic calibration means for determining internal parameters including a focal length of the photographing means and external parameters of the photographing posture and position of the photographing means based on the linear light projected from the device; and a light projection device. measurement point specifying means for projecting a linear light onto the measurement object to identify the measurement target point on the measurement object; and three-dimensional information calculation means for determining the three-dimensional coordinates of the measurement target point on the object.

In the three-dimensional image measurement system of the present invention, a plumb bob device and a light projecting device are used together, and a plurality of objects to be measured including the string member of the plumb bob device and the linear light projected from the light projecting device are Photographs are taken from each viewpoint, and using these photographed images, internal parameters including the focal length of the photographing means are determined based on the feature points on the string-like member of the plumb-bob device and the linear light projected from the light projector. and the external parameters of the photographing posture and photographing position of the photographing means, and calculate the three-dimensional coordinates of the measurement target point on the measurement object using the external parameters for the image photographed by the photographing means using the internal parameters. By determining this, three-dimensional image measurement can be performed.

That is, in the three-dimensional image measurement system of the present invention, internal parameters including the focal length of the photographing means and external parameters of the photographing posture and photographing position of the photographing means are determined, and an image photographed by the photographing means is determined using the internal parameters. On the other hand, since the three-dimensional coordinates of the measurement target point on the measurement object are determined using external parameters, the three-dimensional image can be maintained even if the focal length of the lens changes during on-site measurement, or the shooting posture or shooting position changes. It is possible to perform measurements.

It is preferable that the three-dimensional image measurement system of the present invention includes a photographed image converting means for projectively transforming the image photographed by the photographing means on the basis that the string member of the plumb bob device is perpendicular to the ground. . As a result, even if the image is taken from an arbitrary viewpoint, an image that looks as if the object to be measured is taken from directly in front can be obtained.

The plumb-bob device consists of a transparent plumb-bob tube through which a string-like member is passed, a base on which the plumb-bob tube stands vertically, a horizontal adjuster that adjusts the top surface of the base horizontally, and a plumb-bob tube with a diameter that is about the same as the inner diameter of the plumb-bob tube. A fixing plate having a hole and provided on the plumb bob tube, an adjustment plate having a hole large enough for the string-like member to pass through and provided on the fixing plate, and an adjustment plate provided on the adjustment plate, The structure may include a length adjustment mechanism that adjusts the length of the string member. Thereby, even when there is a wind, the plumb bob tube does not sway, and the string-like member therein can be prevented from swaying. In addition, by adjusting the relative position of the adjustment plate and the fixed plate in a plane, it is possible to ensure that the string-like member of the plumb-bob device and the weight do not come into contact with the plumb-bob tube even if the plumb-bob tube cannot be installed vertically. It becomes possible to do so.

Further, the plumb-bob device may be configured to include a support for suspending a string-like member, a transparent plumb-bob tube through which the string-like member is passed, and a plumbstone tube suspended from the support by a connecting string-like member. can. This prevents the swinging of the plumb bob from affecting the base, string-like member, and support of the plumb-bob device, and even when there is wind, the string-like member inside the plumb-bob can be You can prevent it from shaking.

It is desirable that the range of the center angle θ (90°<θ<180°) of the wall of the plumb tube be a single color that makes the string-like member stand out. This makes it easier to extract the string-like member of the plumb bob device during image processing.

The three-dimensional image measurement method of the present invention uses a string-like member of a plumb bobbing device in which a weight is suspended from a string-like member having feature points, and linear light projected from a light projection device that projects linear light. The object to be measured is photographed from multiple viewpoints using a photographing means, and the images taken by the photographing means are used to identify the characteristic points on the string-like member of the plumb-bob device and the linear light projected from the projector. Based on this, the internal parameters including the focal length of the photographing means and the external parameters of the photographing posture and photographing position of the photographing means are determined. and determining the three-dimensional coordinates of the measurement target point on the measurement object using external parameters with respect to the image photographed by the photographing means using internal parameters. do. According to the three-dimensional image measurement method of the present invention, it is possible to achieve the same effects as the three-dimensional image measurement system of the present invention described above.

The three-dimensional image measurement program of the present invention uses a string-like member of a plumb bobbing device in which a weight is suspended from a string-like member having feature points, and linear light projected from a light projection device that projects linear light. The photographing means uses images of the object to be measured, including the object, taken from a plurality of viewpoints by the photographing means, based on the feature points on the string-like member of the plumb-bob device and the linear light projected from the light projecting device. arbitrary viewpoint dynamic calibration means for determining internal parameters including the focal length of the camera and external parameters of the photographing posture and position of the photographing means; measurement point specifying means for specifying the above measurement target point, and three-dimensional information for determining the three-dimensional coordinates of the measurement target point on the measurement object using external parameters for the image taken by the photographing means using internal parameters. This is to make the computer function as a calculation means. A computer that executes this program can produce the same functions and effects as the three-dimensional image measurement system of the present invention described above. Note that the three-dimensional image measurement program of the present invention can also be read and executed by a computer via a computer-readable recording medium.

(1) Photograph the object to be measured, including the string-like member of the plumb-down device with a weight suspended from a string-like member having feature points, and the linear light projected from the light projection device that projects the linear light. The images are taken from a plurality of viewpoints by the means, and the image taken by the photographing means is used to determine the image of the photographing means based on the feature points on the string-like member of the plumb-bob device and the linear light projected from the light projecting device. Obtaining internal parameters including a focal length and external parameters of the photographing posture and photographing position of the photographing means, projecting linear light from a light projecting device onto a measurement target, and identifying a measurement target point on the measurement target; The configuration uses external parameters to determine the three-dimensional coordinates of the measurement target point on the measurement target for the image taken by the imaging means using internal parameters, so the focal length of the lens changes during on-site measurement, and the It is possible to perform 3D image measurement even if the posture or shooting position changes, and it is possible to measure the shape and dimensions of the measurement target by taking multiple photos of the measurement target in a location that is easy to take. becomes possible.

(2) With the configuration that projectively transforms the image based on the fact that the string-like member of the plumb-bob device is perpendicular to the ground in the image photographed by the photographing means, even if the image is photographed from any viewpoint, it can be measured. An image that looks like the object was taken directly in front of you can be obtained.

(3) The plumb-bob device includes a transparent plumb-bob tube through which a string-like member is passed, a base on which the plumb-bob tube stands vertically, a horizontal adjuster that adjusts the top surface of the base horizontally, and an inner diameter of the plumb-bang tube. A fixing plate that has holes of the same size and is installed on top of the plumb tube, a hole that is large enough for a string member to pass through, and an adjustment plate that is installed on top of the fixation plate, and an adjustment plate that is installed on top of the adjustment plate. The structure includes a length adjustment mechanism that adjusts the length of the string-like member, so that even in the presence of wind, the plumb bob tube does not sway and the string-like member therein does not sway. This makes it possible to eliminate the influence of wind and measure the shape and dimensions of the object with higher precision.

(4) The plumb-bob device has a structure in which the plumb-bob device has a support for suspending a string-like member, a transparent plumb-bob tube through which the string-like member is passed, and a plumb-bang tube that is suspended from the support by a connecting string-like member. It is possible to prevent the shaking of the plumb bob from affecting the base, string-like member, and support of the plumb-bob device, and to prevent the string-like member in the plumb-bob from shaking even when there is wind. This makes it possible to eliminate the influence of wind and measure the shape and dimensions of the object to be measured with higher precision.

(5) With a configuration in which the range of the central angle θ (90° < θ < 180°) of the wall of the plumb-bob tube is a single color that makes the string-like member stand out, the string-like member of the plumb-bob device can be easily removed during image processing. This makes it easier to extract the shape and dimensions of the object to be measured, making it possible to measure the shape and dimensions of the object with higher precision.

FIG. 1 is an explanatory diagram showing an overview of a three-dimensional image measurement system in an embodiment of the present invention. FIG. 2 is a diagram showing an example of dimensions that need to be measured in the shutter mounting area of FIG. 1; FIG. 2 is a functional block diagram of the computer in FIG. 1. FIG. It is an explanatory view showing an example of arrangement of a laser marker and a plumb-bang device. It is a figure showing example 1 of a plumb bob device. It is a figure which shows the example 2 of a plumb-bob device. It is a figure which shows Example 3 of a plumb-bob device. FIG. 3 is a cross-sectional view showing an example of a plumb bob barrel. 8 is an image diagram of adjusting the center position of the string-like member in the plumb-down swing device of FIG. 7. FIG. FIG. 2 is a flow diagram showing the flow of three-dimensional image measurement by the three-dimensional image measurement system in the embodiment of the present invention. FIG. 2 is a diagram showing an example in which a measurement target object in an embodiment of the present invention is photographed obliquely.

FIG. 1 is an explanatory diagram showing an overview of a three-dimensional image measurement system according to an embodiment of the present invention. In FIG. 1, the three-dimensional image measurement system in this embodiment has a mounting area (shaded area; hereinafter referred to as "shutter mounting area") Z2 when a shutter or the like is mounted on a measurement target Z1. It is for measuring shape and dimensions. The three-dimensional image measurement system includes a plurality of cameras 1 as photographing means for photographing a measurement target Z1 from a plurality of different photographing positions, and a calculator 2 such as a personal computer (PC) to which the cameras 1 are connected and performs various calculations. , a laser marking device 3 (see FIG. 4) as a projection device for projecting linear light, and a plumb bob device 4 (see FIG. 4).

FIG. 2 is a diagram showing an example of dimensions that need to be measured in the shutter mounting area Z2 of FIG. 1. In FIG. 2, W is the horizontal width of the opening Z3 to which a shutter or the like is attached, and H is the vertical width of the opening Z3. X1 to X5 are the dimensions of the periphery of the opening Z3 necessary for mounting a shutter or the like.

FIG. 3 is a functional block diagram of the computer 2 in FIG. 1. As shown in FIG. 3, the computer 2 includes a photographed image converting means 22 for applying projective transformation to the photographed image so that when the measurement target Z1 is photographed from an oblique angle by the camera 1, the photographed image is as if it had been photographed directly from the front. , arbitrary viewpoint dynamic calibration to correctly determine the parameters of camera 1 at the time of shooting when selecting a shooting location or adjusting the focal length of the lens of camera 1 in order to take clear pictures of the measurement target. A plurality of photographs taken from different viewpoints are used using the parameters of the camera 1 obtained from the means 23, the measurement point specifying means 24 that automatically or manually determines the measurement target point, and the arbitrary viewpoint dynamic calibration means 23. Three-dimensional information calculation means 25 that analyzes images and calculates three-dimensional coordinates of measurement points, measurement result output means 26 that outputs measurement results, and stores captured images, primary results of image processing, and final measurement results. storage means 27.

FIG. 4 is an explanatory diagram showing an example of the arrangement of the laser marking device 3 and the plumb-bang device 4. As shown in FIG.
As shown in FIG. 4, the two plumb-down swing devices 4 and the laser marking device 3 are installed at a location where the photographing scene of the shutter mounting area Z2 of the measurement target Z1 can be photographed at the same time as the shutter mounting area Z2. The plumb bob device 4 is placed near the shutter attachment area Z2 of the measurement target Z1. In the arrangement example shown in FIG. 4, two plumb-down swing devices 4 are arranged on the left and right so as to sandwich an opening Z3 into which a shutter or the like is attached.

The laser marker 3 is placed between the two plumb-down devices 4. From the laser marker 3, a laser line (hereinafter referred to as "horizontal line") 3A as a linear light extending in the horizontal direction and two linear lights spatially orthogonal to this and mutually orthogonal are emitted. Laser lines (hereinafter referred to as "rectangular lines") 3B and 3C are projected.

Markers 10 and 11 as characteristic points are attached to the upper and lower parts of the string member 8 of the plumb swing device 4, respectively. The position of the lower marker 11 is adjusted so that it coincides with the horizontal line 3A projected by the laser marker 3 (see FIG. 7). Further, the upper sign 10 is installed at a predetermined constant distance H (see FIG. 7) from the lower sign 11. The interval H is determined before measurement according to the size of the measurement target Z1. It is convenient to set the distance H to a value that is easy to calculate, such as 1 meter, 1.5 meters, or 2 meters. Note that it is also possible to align the horizontal line 3A projected from the laser marker 3 with the upper marker 10 and set the lower marker 11 at a position spaced a certain distance H from the upper marker 10.

The string member 8 and the markers 10, 11 of the plumb bobbing device 4, and the horizontal line 3A and rectangular lines 3B, 3C projected by the laser marker 3 are used for image correction, camera system calibration, and measurement target point identification. used for.

FIG. 5 shows an example of the plumb-bang device 4. As shown in FIG. The plumb bobbing device 4 includes a support 7 erected on a base 6, a string member 8 suspended from the support 7, and signs 10 and 11 attached to the upper and lower parts of the string member 8, respectively. A weight 9 is suspended below the member 8. In this embodiment, the string-like member 8 uses a strong wire. Further, in order to make the support column 7 stand as perpendicular to the ground as possible, a leveling device 5 is installed below the base 6 to adjust the upper surface of the base 6 horizontally. The horizontal adjuster 5 consists of three or more legs whose height can be adjusted individually.

FIG. 6 shows another example of the plumb-bang device according to the embodiment of the present invention. In FIG. 6, parts common to those of the plumb-down swing device 4 described above are designated by the same reference numerals, and detailed explanation thereof will be omitted. In the plumb-down swing device 4A shown in FIG. 6, a plumb-down cylinder 12 is installed to reduce the influence of wind. The string member 8 of the plumb bob device 4A is passed through the plumb bob tube 12. In this embodiment, the plumb tube 12 is made of transparent acrylic material. The plumb tube 12 is suspended from the support 7 using a tether-like member 13. In this plumb-down swing device 4A, when there is a wind, the plumb-bob tube 12 swings, but the string-like member 8 therein hardly swings, or the swing width is strongly suppressed.

FIG. 7 shows yet another example of the plumb-bang device according to the embodiment of the present invention. In FIG. 7, parts common to those of the plumb-down swing devices 4 and 4A described above are denoted by the same reference numerals, and detailed explanation thereof will be omitted. The plumb-bob device 4B shown in FIG. 7 is also designed to reduce the influence of wind, but the plumb-bang device 4A shown in FIG. The tube 12 is vertically erected. Further, a fixing plate 14 is provided on the plumb bob cylinder 12. The fixing plate 14 has a hole 14A as large as the inner diameter of the plumb tube 12, through which the string member 8 is passed.

Further, an adjustment plate 15 is provided on the fixed plate 14. The adjustment plate 15 has a hole 15A small enough to allow the string member 8 to pass through. A hand winch 16 is provided on the adjustment plate 15. The hand winch 16 functions as a vertical position adjustment mechanism that adjusts the vertical position of the weight 9 of the string member 8 and the markers 10 and 11 by adjusting the length of the string member 8. Further, in order to simplify the setting of the laser marking device 3, the laser marking device 3 is installed on the same base 6 as the plumb bobbing device 4B.

In the plumb-bob device 4B of this embodiment, the plumb-bob barrel 12 is provided on the fixed plate 14 without using the support 7, so that the plumb-bang barrel 12 does not sway even when there is wind. The string-like member 8 therein also does not sway.

FIG. 8 is a cross-sectional view showing an example of the plumb tube 12 in FIGS. 6 and 7. In this embodiment, a cylindrical transparent acrylic material with a radius R is used as the plumb tube 12. On the back side of this plumb bob tube 12, that is, on the side far from the camera 1, a single color background sticker 17 is pasted to make the string-like member 8 stand out within the range of the central angle θ. A background sticker 17 is pasted along the wall of the plumb tube 12. In this embodiment, the value of the central angle θ of the range where the background sticker 17 is pasted is set to be larger than 90 degrees and smaller than 180 degrees. The purpose of installing the background sticker 17 is to make it easier to extract the string-like members 8 of the plumb bob devices 4A and 4B during image processing.

In this way, when using the plumb-down devices 4A and 4B with the plumb-bob barrel 12 as shown in FIG. The installation angle of the plumb-down cylinder 12 is adjusted so that the background stickers 17 are both located behind the string-like members 8 of the plumb-down swing devices 4A and 4B.

FIG. 9 is an image diagram of adjusting the center position of the string-like member 8 in the plumb-down swing device 4B of FIG. If the ground at the measurement site is not level, it is difficult to make the plumb bob 12 stand perpendicular to the ground only by adjusting the height using the leveling device 5 attached below the base 6. In particular, if the inclination angle of the plumb-bob tube 12 is large, the string-like member 8 and the weight 9 may hit the wall of the plumb-bob tube 12, making it impossible to function. Therefore, in this embodiment, the relative positions of the fixing plate 14 and the adjusting plate 15 can be adjusted so that the plumb bob barrel 12 can operate normally even if it has a certain degree of inclination. In this plumb bob device 4B, the weight 9 is arranged near the center position of the circle at the bottom of the plumb bob barrel 12 by shifting the string-like member passing center of the adjusting plate 15 and the string-like member passing center of the fixed plate 14. to be done.

Next, each means 22 to 27 of the camera 1 and the computer 2 will be explained in detail.

[Camera 1]
As shown in FIG. 1, two or more photographing positions where the shutter attachment area Z2 of the measurement target Z1 can be photographed are selected, the focal length of the lens of the camera 1 is adjusted, and a plurality of photographs are taken. The selection of the photographing position is arbitrary, but in order to show the entire shutter mounting area Z2, the photographing positions are near the front (photographing position A2 in Fig. 1), on the left side (photographing position A1 in Fig. 1), and on the right side (photographing position A3 in Fig. 1). It is preferable to do something like this. In addition, images are taken not only of the shutter attachment area Z2 but also of a part of the laser line (horizontal line 3A and rectangular lines 3B, 3C) projected by the string member 8 of the plumb bobbing device 4 and the laser marking device 3. It is necessary to hold it.

When photographing the measurement target Z1 at each photographing position A1 to A3, the lens focal length of the camera 1 can be adjusted. In order to improve measurement accuracy, it is desirable that the focal length of the camera 1, once adjusted, not be changed when photographing at each photographing position.

[Photographed image conversion means 22]
When displaying the measurement results as an image, it is natural to use a photograph of the measurement object Z1 taken from directly in front. However, the condition of photographing the measurement target Z1 from directly in front is too strict and cannot be realized theoretically. Furthermore, since the photographing position of the camera 1 is not limited to being directly in front of the object to be measured Z1, the photographing position may be taken from an oblique viewing angle or from below. For this reason, the photographed measurement object Z1 is almost always oblique in the image.

FIG. 11 is an example in which the measurement target Z1 according to the embodiment of the present invention is photographed obliquely. The photographing position is on the left side of the measurement object Z1 and is not directly in front of the measurement object Z1, so the measurement object Z1 is photographed diagonally. Furthermore, the image on the left side of the image is close to the camera 1, so it is photographed in a large size, and the image on the right side is far from the camera 1, so it is photographed in a small size.

Therefore, the photographed image converting means 22 performs projective transformation on the photographed image of the measurement target Z1 with reference to the fact that the string member 8 of the plumb bobbing device 4 is perpendicular to the ground. In the image that has been projectively transformed by the photographed image converting means 22, the image appears to be photographed directly in front of the measurement target Z1, regardless of the photographing position and the photographing viewing angle. The following formula is used for projective transformation.

Here, (x, y) are the pixel coordinates of the photographed image at each photographing position, (x', y') are the pixel coordinates after adjustment by projective transformation, and a, b, c, d, t _x , t _y are It is an adjustment parameter.

The adjustment parameters are determined using the fact that the string-like member 8 of the plumb-down swing device 4 in the photographed image is always perpendicular to the ground. Specifically, as shown in the shaded area shown in FIG. 11, a total of four markers 10 and 11 on the left plumb-bob device 4 and right plumb-bang device 4 are formed into a rectangle with vertices C1, C2, C3, and C4. make. Although the quadrangles on the image are not rectangles, the quadrangles C1, C2, C3, and C4 in real space are rectangles and have the following characteristics.

(1) C1-C2 is parallel to the ground.
(2) C3-C4 is parallel to the ground.
(3) C1-C3 is perpendicular to the ground.
(4) C2-C4 is perpendicular to the ground.
(5) The length of C1-C2 is equal to the length of C3-C4.
(6) The length of C1-C3 is equal to the length of C2-C4.

Using these characteristics as constraint conditions, each unknown parameter in Equation 1 above can be determined. Using these determined parameters, Equation 1 is applied to each pixel of the captured image from an arbitrary viewpoint, projective transformation is performed on the image, and the captured image is normalized. As a result of such processing, the image as shown in FIG. 11 is corrected into an image as shown in FIG. 4.

[Arbitrary viewpoint dynamic calibration means 23]
The arbitrary viewpoint dynamic calibration means 23 calculates the focal length and shooting angle of the lens of the camera 1 used when taking a photo of the measurement object Z1 from an arbitrary position, the world coordinate values of the shooting location, so-called three-dimensional This is a means for determining camera system parameters during image measurement.

In the three-dimensional image measurement system according to the present embodiment, in order to take a picture of the measurement target Z1 in an optimal state with the camera 1, the focal length of the lens 1, the direction and distance of the photograph, etc. are changed every time the photograph is taken. That is, the parameters of the camera system may change each time a photograph is taken, and the camera system needs to be calibrated every time a photograph is taken.

For this reason, the arbitrary viewpoint dynamic calibration means 23 uses the markers 10 and 11 on the string member 8 of the plumb bobbing device 4, which are photographed at the same time as the measurement target object Z1, for all photographed images, and The internal parameters of the camera 1, including the focal length of the lens, and the external parameters of the photographing posture and position are determined.

Specifically, the arbitrary viewpoint dynamic calibration means 23 uses the camera calibration model described in Non-Patent Document 1 as shown in Equation 2 below, and uses the internal parameters including the focal length of the camera 1 and the image capturing of the camera 1. The posture and external parameters of the photographing position are determined.

Here, (u, v) are the pixel coordinates of the target point on the image sensor of camera 1, s is the scale coefficient corresponding to the magnification, and (X, Y) are the three-dimensional world coordinates of the target point on the calibration plane. , A is an internal matrix of camera 1, r ₁ is a rotation matrix about the X axis, r ₂ is a rotation matrix about the Y axis, and t is a translation matrix. r ₁ and r ₂ are called rotation parameters, t is called a translation parameter, and these are collectively called external parameters of the camera 1.

The internal matrix A of the camera is expressed by Equation 3 below.

The internal parameters f _x , f _y , c _x , and c _y of the camera 1 shown in Equation 3 can be obtained by the method described in Non-Patent Document 1 and Non-Patent Document 2. However, in this embodiment, since the focal length of the lens of the camera 1 is adjusted when photographing the measurement target Z1, f _x and f _y in Equation 3 may change each time the measurement is performed.

Therefore, in this embodiment, the arbitrary viewpoint dynamic calibration means 23 calculates the focal length change coefficient k _f at each focal length of the lens (variable focal length lens) as follows when calibrating the internal parameters.

In addition, during actual measurement, the arbitrary viewpoint dynamic calibration means 23 pays attention to the distance H between the upper and lower markers 10 and 11 of the plumb bob device 4 in the photographed image normalized by the photographed image conversion means 22, and From the above correspondence between the number of pixels and the actual length, f _y is determined, and from k _f determined during calibration, f _x is determined based on Equation 4.

Note that the external parameters r ₁ , r ₂ , and t are obtained using the rectangle of the vertices C1, C2, C3, and C4 in FIG. 11 in the same way as the photographed image converting means 22. However, the following two additional constraints are added to (1) to (6) above.
(7) The length of C1-C3 is a preset interval H.
(8) The length of C1-C2 is H/k _f .

That is, the arbitrary viewpoint dynamic calibration means 23 calculates that the string members 8 of the two plumb-down swing devices 4 are parallel in the real world, and that the distance between the two markers 10 and 11 on each string member 8 is in reality. Under the known condition that the interval H is preset in the world, the internal parameters f _x , f _y , c _x , c _y and the external parameters (rotation parameters r ₁ , r ₂ , and translation parameters Find t).

The arbitrary viewpoint dynamic calibration means 23 uses a plurality of photographed images taken from each photographing position, and under the above-mentioned eight constraint conditions, calculates the vertices C1 and C2 instead of the chessboard of the method of Non-Patent Document 1. , C3, and C4, r ₁ , r ₂ , and t are determined, and the external parameters are calibrated.

[Measurement point specifying means 24]
In order to specify a measurement point, the measurement point specifying means 24 projects a laser line from the laser marker 3 onto the measurement object Z1, and specifies the measurement target point on the measurement object Z1. Specifically, as shown in FIG. 4, an optical marker is created on the measurement target Z1 by projecting a horizontal line 3A and two rectangular lines 3B and 3C orthogonal thereto from the laser marker 3. The image photographed by the camera 1 includes these optical markers, and the measurement point specifying means 24 specifies the measurement target point from the optical markers photographed during image processing.

In addition, measurement target points that are difficult to project the laser line from the laser marker 4, such as curved parts, or measurement target points that require measurement and are located in areas where no other laser line is projected, can be measured by clicking the mouse at the image measurement stage. It can be directly specified and specified using methods such as .

In addition, when installing the laser marking device 3, it is necessary to devise ways to easily specify the measurement target point on the measurement target object Z1. Furthermore, if necessary, it is also possible to use two or more laser marking devices 3 or to use another type of light projecting device in place of the laser marking device 3.

[Three-dimensional information calculation means 25]
The three-dimensional information calculation means 25 calculates the external parameters of the camera 1 obtained by the arbitrary viewpoint dynamic calibration means 23 for the plurality of images of the measurement object Z1 taken by the camera 1 at each of the photographing positions A1 to A3. By using the three-dimensional image measurement method, the three-dimensional coordinates of the measurement target point specified by the measurement point specifying means 24, the measurement target point specified by mouse click, and other measurement target points that need to be measured are automatically determined. Calculated as follows.

In addition, the three-dimensional information calculation means 25 calculates not only the measurement target point but also the distance H between the upper and lower markers 10 and 11 of the plumb bob device 4 in each photographed image from the above parameters, and combines the calculation results with the markers 10 and 11. The value of the above parameter is corrected based on the difference from the true value of the interval H between 11 and 11. This further improves the accuracy of three-dimensional measurement.

[Measurement result output means 26]
The measurement result output means 26 marks the measurement target point on the calibrated photographed image processed by the photographed image conversion means 22, and based on the three-dimensional world coordinates of the measurement target point calculated by the three-dimensional information calculation means 25, The distance and area between the required measurement target points are displayed on the monitor. Additionally, these measurement results are output in file formats such as image format and text format, and saved in a database.

Measurement results stored in the database can be recalled at any time, and processing such as adding or deleting measurement points is also possible. It is also possible to convert the three-dimensional image measurement results into a CAD data format and view them using specific commercially available software or free software.

[Storage means 27]
The storage means 27 is mainly composed of a computer such as a PC, and a data storage device such as an SSD or HDD. As shown in FIG. 3, the storage means 27 is mainly used for capturing images shot by the camera 1, shooting image conversion means 22, arbitrary viewpoint dynamic calibration means 23, measurement point specifying means 24, and three-dimensional information calculation means 25. It has functions such as saving and outputting image processing results.

Next, a three-dimensional image measuring method using the three-dimensional image measuring system having the above configuration will be explained. FIG. 10 is a flowchart showing the flow of three-dimensional image measurement by the three-dimensional image measurement system in this embodiment.

For measurement, first, the plumb bob device 4 and the laser marker 3 are installed near the measurement target Z1 (S100). Subsequently, a plurality of photographs of the measurement target Z1 are taken from different viewpoints using the camera 1 (S101). Note that when photographing the measurement object Z1, the focal length of the lens of the camera 1 is adjusted using the internal parameters determined by the arbitrary viewpoint dynamic calibration means 23. Next, the photographed photograph is sent from the camera 1 to the computer 2 (S102). Transmission methods include wired communication, wireless communication, and direct transmission using a memory card.

In the computer 2, the photographed image conversion means 22 applies projective transformation to the photographed photograph in order to make it easier to see (S103). Furthermore, the measurement point specifying means 24 specifies a measurement target point from the photographed image (S104), and the three-dimensional coordinates of the specified measurement target point are calculated using external parameters determined by the arbitrary viewpoint dynamic calibration means 23 (S107). The three-dimensional information calculation means 25 calculates the information using the three-dimensional information calculation means 25 (S105). Finally, the measurement result output means 26 outputs the measurement result as an image or text file and creates a database (S106).

As described above, in the three-dimensional image measurement system according to the present embodiment, the plumb-down device 4 and the laser marking device 3 are used together, and the The measurement object Z1 including the horizontal line 3A and the rectangular lines 3B and 3C is photographed from a plurality of viewpoints, and the marks 10 and 11 on the string member 8 of the plumb bob device 4 are identified using the photographed images. Based on the horizontal line 3A and rectangular lines 3B and 3C projected from the laser marking device 3, internal parameters including the focal length of the camera 1 and external parameters of the shooting attitude and shooting position of the camera 1 are determined. Three-dimensional image measurement can be performed by determining the three-dimensional coordinates of the measurement target point on the measurement target Z1 using the external parameters for the image photographed by the camera 1 using the parameters.

That is, in the three-dimensional image measurement system according to the present embodiment, simple devices such as the plumb-bob device 4 and laser marking device 3 are used, and a general-purpose photographing device such as a commercially available digital camera is used as the camera 1. Photographs of the construction site necessary for installing shutters, fire doors, etc. are taken, and then the computer 2 uses image processing to calculate parameters such as the shape and dimensions of the site necessary for manufacturing and installing shutters, etc., and uses image measurement results. It is possible to create a database, improve measurement accuracy, and identify measurement target points easily at the photo-taking stage, at the subsequent image processing stage, and when needed in the future. . In addition, this three-dimensional image measurement system uses an optical marker using the laser marking device 3, so it is possible to identify the mounting position of the shutter etc. from the captured image without applying paint or attaching markers as in the conventional method. , it is possible to measure parameters such as shape and dimensions.

In particular, in the three-dimensional image measurement system according to the present embodiment, internal parameters including the focal length of the camera 1 and external parameters of the photographing posture and photographing position of the camera 1 are determined, and the internal parameters are used to obtain the internal parameters including the focal length of the camera 1. Since the three-dimensional coordinates of the measurement target point on the measurement target Z1 are determined from the image using external parameters, the focal length of the lens of the camera 1 may change during on-site measurement, or the shooting posture or shooting position may change. It is possible to perform three-dimensional image measurement even if the

Therefore, when photographing on-site, it is possible to take a photograph of the measurement target Z1 from a place where it can be easily seen, and there is no need to decide the photographing location or line of sight angle in advance, and the camera 1 can be adjusted as necessary. You can adjust the focal length of the lens and choose the shooting location.

Furthermore, in the three-dimensional image measurement system according to the present embodiment, since the image is projectively transformed based on the fact that the string member 8 of the plumb bobbing device 4 in the image photographed by the camera 1 is perpendicular to the ground, any viewpoint can be used. Even if the image is taken from the front, an image that looks like the object to be measured is taken from directly in front can be obtained.

The effects of the three-dimensional image measurement system in this embodiment are as follows.
(1) Using inexpensive commercially available equipment, high-speed and highly accurate three-dimensional image measurement can be achieved with simple operations such as photographing, and non-contact measurement of buildings, tunnels, etc. will be possible.
(2) When photographing, there is no need to decide the focal length of the camera or the photographing location in advance, and the photograph can be taken at a location where the object to be measured can be clearly photographed and at the necessary lens focal length. This can reduce the time and effort required for measurement.
(3) It is possible to fully automatically calculate the three-dimensional coordinates of the measurement target point, and automatically calculate the distance, area, etc. between the necessary measurement points.

The three-dimensional image measurement system, three-dimensional image measurement method, and three-dimensional image measurement program of the present invention are useful for three-dimensional measurement in fields such as architecture, civil engineering, bridge and tunnel construction. In particular, when installing shutters on building garages, building fire doors on buildings, building fire doors on evacuation shafts during tunnel construction, etc., it is necessary to measure the shape and dimensions of buildings, bridges, and tunnels. It is suitable for fields such as shape measurement and quality control.

1 Camera 2 Calculator 3 Laser marker 4, 4A, 4B Plumb bob device 5 Horizontal adjuster 6 Base 7 Support 8 String member 9 Weight 10, 11 Sign 12 Plumb tube 13 Tether string member 14 Fixing plate 14A Hole 15 Adjustment plate 15A Hole 16 Hand winch 17 Background sticker 22 Photographed image conversion means 23 Arbitrary viewpoint dynamic calibration means 24 Measurement point specifying means 25 Three-dimensional information calculation means 26 Measurement result output means 27 Storage means Z1 Measurement object Z2 Shutter mounting area Z3 opening

Claims (14)

a light projection device that projects linear light;
A plumb bobbing device in which a weight is suspended from a string-like member having characteristic points;
a photographing means for photographing the measurement object from a plurality of viewpoints, including the string member of the plumb-down swing device and the linear light projected from the light projector;
Using an image photographed by the photographing means, an internal image including the focal length of the photographing means is determined based on the feature points on the string-like member of the plumb-bob device and the linear light projected from the light projecting device. arbitrary viewpoint dynamic calibration means for determining parameters and external parameters of the photographing posture and photographing position of the photographing means;
Measurement point specifying means for projecting linear light from the light projecting device onto the measurement object to specify a measurement target point on the measurement object;
3D image measurement, comprising: 3D information calculation means for calculating the 3D coordinates of the measurement target point on the measurement object using the external parameters with respect to the image photographed by the photographing means using the internal parameters; system. The three-dimensional image measurement system according to claim 1, further comprising a photographed image converting means for projectively transforming the image photographed by the photographing means on the basis that the string-like member of the plumb-down swing device is perpendicular to the ground in the image photographed by the photographing means. . The light projection device projects linear light extending in the horizontal direction,
The plumb bobbing device has marks as the characteristic points on the upper and lower parts of the string-like member, respectively, and the position of one of the marks corresponds to the horizontal linear light irradiated from the light projector. 3. The three-dimensional image measurement system according to claim 1, wherein the other position is set at a position spaced a certain distance from the mark. The plumb-bob device includes a transparent plumb-bob tube through which the string-like member is passed, a base on which the plumb-bob tube is erected vertically, a horizontal adjuster for horizontally adjusting the top surface of the base, and a plumb-bang tube for adjusting the top surface of the base to be horizontal. a fixing plate having a hole about the same diameter as the inner diameter and provided on the plumb-down cylinder; an adjusting plate having a hole large enough for the string-like member to pass through and provided on the fixing plate; The three-dimensional image measurement system according to claim 3, further comprising a length adjustment mechanism provided on the adjustment plate and adjusting the length of the string-like member. 3. The plumb-bob device includes a support for suspending the string-like member, and a transparent plumb-bob tube through which the string-like member is passed, and the plumb-bang tube is suspended from the support by a connecting string-like member. The three-dimensional image measurement system described. 6. The three-dimensional image measurement system according to claim 4, wherein a range of a central angle θ (90°<θ<180°) of the wall of the plumb tube is a single color that makes the string-like member stand out. 4. Three-dimensional image measurement according to claim 3, wherein there are two plumb-down swing devices, which are set at regular intervals in the photographic scene of the measurement target, and whose upper and lower markers have the same height. system. The light projection device projects linear light extending in the horizontal direction,
The plumb bobbing device has marks as the characteristic points on the upper and lower parts of the string-like member, respectively, and the position of one of the marks corresponds to the horizontal linear light irradiated from the light projector. and the other position is set at a certain distance from the said sign,
The plumb-down swing devices are two, and are set at a certain interval in the photographing scene of the measurement target, and the height of the upper and lower markers of each is the same,
The photographed image converting means converts a rectangle whose vertices are the four marks of the string-like members of the two plumb-down swing devices in the image photographed by the photographing means, so that the vertical sides are perpendicular to the ground and the horizontal sides are perpendicular to the ground. 3. The three-dimensional image measurement system according to claim 2 , wherein the three-dimensional image measurement system calibrates the image so that it is parallel to the ground. The arbitrary viewpoint dynamic calibration means is configured to determine that the string-like members of the two plumb-down swing devices in the image photographed by the photographing means are parallel in the real world, and the distance between the two marks on each string-like member. 7. A focal length as the internal parameter of the photographing means and a rotation parameter and a translation parameter as the external parameters are determined under the known condition that is a preset interval in the real world. The three-dimensional image measurement system described. The measurement point specifying means creates an optical marker on the measurement object by projecting a linear light extending in the horizontal direction from the light projecting device and two linear lights spatially orthogonal to this and mutually orthogonal. The three-dimensional image measurement system according to claim 1 or 2, wherein the measurement target point is specified from an image including the three-dimensional image measurement system. The three-dimensional information calculation means uses external parameters of the photographing means obtained by the arbitrary viewpoint dynamic calibration means to calculate the three-dimensional information of the plurality of images of the measurement object photographed by the photographing means. 3. The three-dimensional image measurement system according to claim 1, wherein the three-dimensional coordinates of the measurement target point specified by the measurement point specifying means and the measurement target point specified by a mouse click are automatically calculated. A means for photographing the object to be measured including the string-like member of the plumb-shape device in which a weight is suspended from a string-like member having characteristic points and the linear light projected from the light projection device that projects the linear light. By taking pictures from multiple viewpoints,
Using an image photographed by the photographing means, an internal image including the focal length of the photographing means is determined based on the feature points on the string-like member of the plumb-bob device and the linear light projected from the light projecting device. determining parameters and external parameters of a photographing posture and a photographing position of the photographing means;
projecting linear light from the light projecting device onto the measurement target to identify a measurement target point on the measurement target;
A three-dimensional image measuring method comprising determining three-dimensional coordinates of the measurement target point on the measurement object using the external parameters with respect to an image photographed by the photographing means using the internal parameters. The object to be measured includes the string-like member of the plumb bobbing device in which a weight is suspended from a string-like member having characteristic points, and the linear light projected from the light projection device that projects the linear light. Using images taken from a plurality of viewpoints respectively, the focal length of the photographing means is determined based on the feature points on the string-like member of the plumb-bob device and the linear light projected from the light projecting device. arbitrary viewpoint dynamic calibration means for determining internal parameters including internal parameters and external parameters of a photographing posture and a photographing position of the photographing means;
Measurement point specifying means for projecting linear light from the light projecting device onto the measurement object to specify a measurement target point on the measurement object;
for causing a computer to function as a three-dimensional information calculation means for calculating three-dimensional coordinates of the measurement target point on the measurement object using the external parameters with respect to an image photographed by the photographing means using the internal parameters; 3D image measurement program. A computer-readable recording medium on which the three-dimensional image measurement program according to claim 13 is recorded.

Images