JP6573196B2 - Distance information correction apparatus, distance information correction method, and distance information correction program - Google Patents

Description

  The present invention relates to a distance information correction apparatus, a distance information correction method, and a distance information correction program that correct a desired depth image (distance image) by correction processing.

  Research on three-dimensional shape measurement has been actively conducted since ancient times, and in recent years, distance sensors that can easily acquire three-dimensional distance information are becoming popular. It is also widely used by general users, for example, a game controller equipped with a human gesture recognition function using a distance sensor has appeared. On the other hand, it has been studied for a long time to acquire a three-dimensional shape of a whole body, such as a Buddha image, and to preserve a cultural asset as a digital image by imaging a single subject so that it is surrounded by multiple distance sensors. Has been.

  However, methods using the Structured Light method and the Time of Flight method have been developed for the distance sensor method, but any method can be used in the depth direction, such as sensing noise between pixels or distortion in the distance direction. It is difficult to measure accurately. In particular, when using a plurality of distance sensors, there is an individual difference between the sensors, and thus a problem arises in that a deviation occurs when the three-dimensional shape acquired by each sensor is back-projected into one common three-dimensional space. It has been known. In addition, a camera for acquiring color information is often used together with a distance sensor, and the calibration error between the camera and the distance sensor may affect the final three-dimensional shape restoration result. Are known.

  When using multiple distance sensors and multiple cameras at the same time to estimate the colored 3D shape of the subject, the depth value of each distance sensor contains an error, so the 3D shape restoration results are accurate. Has the problem of lacking.

  In particular, since it is inevitable that an error is included in the distance sensor, efforts have been made to restore the three-dimensional shape from only the image information of the camera. However, when the distance sensor is not used, it is necessary to restore the three-dimensional shape from the image information using stereo matching, a view volume intersection method, or the like. In the method using stereo matching, since the restoration accuracy depends on the texture of the subject, for example, an object having a uniform color cannot be restored in principle (for example, non-patent literature). 1). Also, a method for accurately performing camera calibration for a distance sensor has been proposed (see, for example, Non-Patent Document 2), which improves the position accuracy of the camera and performs depth correction in the depth direction. Not something to do.

Michael Bleyer, Christoph Rhemann, and Carsten Rother. "PatchMatch Stereo-Stereo Matching with Slanted Support Windows." BMVC. Vol. 11. 2011. Cha Zhang, Zhengyou Zhang. "Calibration between depth and color sensors for commodity depth cameras." Computer Vision and Machine Learning with RGB-D Sensors. Springer International Publishing, 2014. 47-64.

  In order to accurately restore the three-dimensional shape, the accuracy of the depth image (distance image) by the depth sensor (distance sensor) needs to be high. However, in the methods described in Non-Patent Documents 1 and 2, the depth value acquired by the depth sensor (distance sensor) is corrected from the result of constructing a pre-imaging environment configured with a known shape and capturing images. There is a problem that you can't.

  The present invention has been made in view of such circumstances, and a distance information correction apparatus, a distance information correction method, and a distance information correction program capable of acquiring a more accurate three-dimensional shape by correcting a depth image. The purpose is to provide.

  According to one embodiment of the present invention, input information that is a pair of a depth image including a correction depth image and a correction target depth image and a color image including a correction color image captured simultaneously with the correction depth image is acquired. An input information acquisition unit and a sensor position that inputs known three-dimensional shape information and estimates the position of the sensor when the correction color image is acquired among the color images acquired by the input information acquisition unit An estimation unit; a reference depth image projection unit that projects a known shape as a reference to obtain a reference depth image based on the estimated position of the sensor; and the correction depth image acquired by the reference depth image and the input information unit. Using the correction parameter calculation means for calculating the depth correction parameter and the depth correction parameter calculated by the correction parameter calculation means. A distance information correction device and a depth value correction means for correcting the correction target depth image Te.

  One aspect of the present invention is the distance information correction device, wherein the input information acquisition unit acquires the depth value of the depth image using the known shape object whose surface is not black.

  One aspect of the present invention is the distance information correction device, wherein the depth value correction unit collectively corrects the depth value of the entire image using the depth correction parameter obtained by the correction parameter calculation unit. .

One aspect of the present invention is the distance information correction device, wherein the correction parameter calculation unit compares the reference depth image with the correction depth image acquired by the input information unit , and determines the comparison result. A depth correction parameter is calculated for each pixel based on the correction value, and the depth value correction unit corrects the correction target depth image using the depth correction parameter for each pixel calculated by the correction parameter calculation unit.

  According to one embodiment of the present invention, input information that is a pair of a depth image including a correction depth image and a correction target depth image and a color image including a correction color image captured simultaneously with the correction depth image is acquired. Input information acquisition means; auxiliary camera position estimation means for estimating the position of the auxiliary camera; and position of the camera that has captured the correction color image based on the position of the auxiliary camera that has been estimated Sensor position propagation means for estimating the reference depth image projection means for obtaining a reference depth image by projecting a reference known shape based on the estimated position of the camera, the reference depth image and the input information means A correction parameter calculation means for comparing the depth image for correction and calculating a depth correction parameter; and the depth calculated by the correction parameter calculation means. A distance information correction device and a depth value correction means for correcting the correction target depth image using a scan correction parameter.

  One aspect of the present invention is a distance information correction method performed by a distance information correction apparatus that corrects a depth image, which is captured simultaneously with a depth image including a correction depth image and a correction target depth image, and the correction depth image. An input information acquisition step for acquiring input information that is a pair with a color image including a correction color image; and input of known three-dimensional shape information, and among the color images acquired by the input information acquisition step, A sensor position estimating step for estimating the position of the sensor when the correction color image is acquired, and a reference depth image projecting step for obtaining a reference depth image by projecting a known shape as a reference based on the estimated position of the sensor; A correction parameter for calculating a depth correction parameter by comparing the reference depth image with the correction depth image acquired in the input information step. A data calculating step, the distance information correction method and a depth value correction step of correcting the correction target depth image using the depth correction parameter calculated by the correction parameter calculating step.

  One aspect of the present invention is a distance information correction method performed by a distance information correction apparatus that corrects a depth image, which is captured simultaneously with a depth image including a correction depth image and a correction target depth image, and the correction depth image. An input information acquisition step for acquiring input information that is a pair with a color image including a correction color image, an auxiliary camera position estimation step for providing an auxiliary camera and estimating the position of the auxiliary camera, and the position estimated A sensor position propagation step for estimating the position of the camera that has captured the correction color image based on the position of the auxiliary camera, and a reference known shape is projected by projecting a reference known shape based on the estimated position of the camera. Depth correction parameters are calculated by comparing the depth image projection step, the reference depth image, and the depth image for correction acquired in the input information step. A correction parameter calculating step of calculating a distance information correction method and a depth value correction step of correcting the correction target depth image using the depth correction parameter calculated by the correction parameter calculating step.

  One aspect of the present invention is a distance information correction program for causing a computer to function with the distance information correction device according to any one of claims 1 to 6.

  According to the present invention, since the depth value acquired by the distance sensor can be corrected, an effect that a more accurate three-dimensional shape can be acquired is obtained.

It is a block diagram which shows the structure of the distance information correction apparatus 1 by 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the distance information correction apparatus 1 shown in FIG. It is a figure which shows the outline of operation | movement of the input information acquisition part 11 shown in FIG. It is a figure which shows the operation | movement outline | summary of the reference depth projection part 13 and the correction parameter calculation part 14 which are shown in FIG. It is a block diagram which shows the structure of the distance information correction apparatus 1 by 2nd Embodiment of this invention. It is a figure which shows the outline of operation | movement of the input information acquisition part 11 shown in FIG. It is a figure which shows the operation | movement outline of the auxiliary camera position estimation part 16 and the sensor position propagation | transmission part 17 which are shown in FIG. It is a block diagram which shows the structure of the distance information correction apparatus 1 by 3rd Embodiment of this invention.

<First Embodiment>
Hereinafter, a distance information correction apparatus according to a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a distance information correction apparatus in the embodiment. The distance information correction device 1 is configured by a computer device, and includes an input information acquisition unit 11, a sensor position estimation unit 12, a reference depth projection unit 13, a correction parameter calculation unit 14, and a depth value correction unit 15 as shown in FIG. I have.

  The operation of the distance information correction apparatus 1 shown in FIG. 1 will be described with reference to FIG. FIG. 2 is a flowchart showing the operation of the distance information correction apparatus 1 shown in FIG.

  The input information acquisition unit 11 is a pair of depth image (distance image) and color image captured at the same time from a depth sensor connected to the distance information correction device 1 or recorded data (including depth information) captured in advance. Are obtained (N sheets: N is a natural number of 2 or more) (step S1). Hereinafter, the depth image and the color image captured for performing the depth correction are referred to as a correction depth image and a correction color image. Further, the depth image to be subjected to depth correction is hereinafter referred to as a correction target depth image (see FIG. 3). The depth image to be corrected is a depth image to be corrected of M (M is a natural number) or more.

  The subject captured in the correction depth image and the correction color image needs to be captured by an object having a known shape. Any shape object may be used as long as the shape is known, but a color / material other than black that does not absorb near-infrared light is used. For example, a check pattern composed of white and gray may be used as a candidate. In addition, a poster image whose actual size and texture are known may be simply used as a known shape object by pasting it on a flat wall or floor.

  Note that the correction color image is captured from the same viewpoint as the paired correction depth image. Since a general distance sensor acquires a near-infrared image from the same viewpoint in order to calculate a depth image, a near-infrared image can be used as a color image. In the case of a laser type distance sensor other than the near-infrared image, a laser reflection intensity image can be used as a color image. In addition, a visible image can be used as a color image if the distance sensor has an image element sensor that can acquire color information and depth information at the same position. In order to acquire color information and depth information at the same position, instead of acquiring an image by arranging a means for acquiring a color image and a means for acquiring a depth image in parallel, the means for acquiring a color image and the depth image are combined. The imaging viewpoint of the means for acquiring needs to be completely coincident.

  Next, the sensor position estimation unit 12 includes a plurality of color images for correction acquired by the input information acquisition unit 11 and three-dimensional shape information of a known object (for example, if a check pattern is used, each lattice point has a three-dimensional shape. The position / orientation of the sensor (imaging means) when a plurality of correction color images are acquired is estimated (step S2). The position / orientation estimated here is a relative position / orientation from the known object, and changes if the position / orientation of the known object changes without moving the position of the imaging means.

  There are N color images for correction, and Lm (L is a natural number) of 3D points on the surface of a known shape object, the mth three-dimensional point is Pm, on the nth correction color image corresponding to Pm Let Qn, m be the two-dimensional corresponding point. First, L corresponding points are detected from all N color images for correction. For example, by using SIFT or SURF called natural feature points, two-dimensional corresponding points between the known texture and the captured color image for correction can be acquired.

  If the subject is a check pattern, a lattice point detection method may be used. Next, from the relationship between Qn, m and Pm, the translation parameter Tn rotation parameter Rn of the sensor, the internal parameter A of the distance sensor, and the lens distortion coefficient matrix D are calculated for each of the N correction color images. Here, Tn is 3 × 1, Rn is 3 × 3, A is 3 × 3, D is 1 × n (n is the number of parameters of the distortion model, and is a value set in advance, for example, 8. ). Further, these parameters are obtained based on the absolute position of the space where the known object is installed. Since this calculation is widely known as a PnP problem (camera calibration problem), a detailed description of the calculation is omitted.

  Next, the reference depth projection unit 13 receives the three-dimensional shape of the known object in the sensor position estimation unit 12, the calculated Tn, Rn, A, D, and the correction color image acquired by the input information acquisition unit 11, and For all the correction depth images, a reference depth image composed of pixels having true depth values and a mask image for indicating a comparison target region are generated.

First, a known shape object is projected onto the viewpoint of the correction depth image using Tn, Rn, A, and D. An image in which the _pz value of the expression (1) in the projection calculation is stored for each pixel is created and used as a reference depth image (step S3).
In a projection calculation of a three-dimensional point (P _x , P _y , P _z ) on an object having a known shape, an image is created in which the P _z value obtained by solving equation (1) is stored for each pixel, and this is used as a reference depth. An image. Here, s is called a scale factor, and it is generally known that it disappears during the calculation.

  Furthermore, a mask image for distinguishing a pixel only at a pixel (px, py) position where a known shape object is projected from a pixel included in the reference depth image and a pixel that is not so is generated. Any different value may be used as the pixel value of the mask image. In the following description, it is assumed that the pixel value of the mask image for a pixel onto which a known shape object is projected is 1 and the pixel value of a pixel that is not so is 0.

  Next, it is determined whether or not the processing has been completed for all the correction images. If the processing has not been completed, the process returns to step S2, and the processing is repeated until completion (step S4).

  Next, the correction parameter calculation unit 14 receives the correction depth image acquired by the input information acquisition unit 11, the reference depth image and the mask image acquired by the reference depth projection unit 13, and inputs the correction depth image and the reference depth. The images are compared (step S5), and a depth correction parameter for correcting the correction target depth image is estimated (step S6; see FIG. 4).

First, the depth value of the pixel (i, j) included in Cn, Cn is set to Cn (i, j), and the reference depth image is set to Dn, pixel (n). Let the depth value of i, j) be Dn (i, j). For the mask image, a set of pixels having a pixel value of 1 is represented as Q. The energy function J shown below is minimized so that Cn (i, j) is closest to Dn (i, j) in all pixels.
_{^{J = (1/2) Σ N n}} Σ (i, j) ∈M ‖Dn (i, j) -F (Cn (i, j)) ‖ ² (2)

In the present embodiment, assuming that there is a linear error with respect to the reference depth image as the function F, a description will be given by adopting a model of Dn (i, j) = aCn (i, j) + b. In addition, a model proportional to the reciprocal of the depth value, such as Dn (i, j) = a / (Cn (i, j) + b), may be used. The energy function J to be minimized when using a linear model is shown below.
_{^{J = (1/2) Σ N n}} Σ (i, j) ∈M ‖Dn (i, j) -aCn (i, j) -b) ‖ ² (3)

  Correction parameters a and b are calculated so as to minimize J. Since an analytical solution exists for J by using a general least square method, it can be solved by a general calculation method. Detailed description is omitted.

  Next, the depth value correction unit 15 receives the correction parameters a and b calculated by the correction parameter calculation unit 14 and the correction target depth image acquired by the input information acquisition unit 11 as input, and each pixel of the correction target depth image. The depth value is corrected (step S7).

  First, Gm (i, j) is the depth value of the pixel (i, j) included in the mth correction target depth image Gm out of M, and the depth value of the pixel of the corrected depth image is similarly set. Hereinafter, it is expressed as Vm (i, j). Similar to the correction parameter calculation unit 14, when a linear model is used, the calculation for performing the correction is Vm (i, j) = aGm (i, j) + b. Even when different models are used, the depth value can be corrected by performing the same process.

  After performing the above correction for all pixels, Gm is stored as a corrected depth image. Further, the above-described processing is performed on all M correction target depth images, and the corrected depth images are output or stored, and the processing is terminated.

Second Embodiment
Next, a distance information correction apparatus according to a second embodiment of the present invention will be described. FIG. 5 is a block diagram showing a configuration of a distance information correction apparatus according to the second embodiment of the present invention. In this figure, the same parts as those in the apparatus shown in FIG. The apparatus shown in this figure is different from the apparatus shown in FIG. 1 in that an auxiliary camera position estimation unit 16 and a sensor position propagation unit 17 are newly provided.

  In the first embodiment that does not use an auxiliary camera, a near-infrared image can be used as a correction color image in the case of a general distance sensor. However, since near-infrared images generally have low resolution and high noise, it may be difficult to detect two-dimensional corresponding points of known-shaped objects in situations where known-shaped objects are located far away. It is done. The configuration of the first embodiment is sufficient if only correction at a position of about 1 m, which is relatively close to the camera, is performed without requiring correction at a distance.

  Therefore, in the second embodiment, in addition to the first embodiment, an auxiliary camera is added to the input information acquisition unit, and the sensor position is estimated by the auxiliary camera, thereby greatly expanding the acquisition range of the correction color image. Is possible. Accordingly, an auxiliary camera position estimation unit 16 and a sensor position propagation unit 17 are added instead of the sensor position estimation unit.

  The operation of the input information acquisition unit 11 shown in FIG. 5 will be described with reference to FIG. FIG. 6 is a diagram showing an outline of the operation of the input information acquisition unit 11 shown in FIG. The input information acquisition unit 11 acquires a plurality of pairs of depth images and color images captured at the same time from the auxiliary camera and depth sensor connected to the distance information correction apparatus 1 or recorded data captured in advance. Thereafter, the color image captured from the sensor for estimating the sensor position is referred to as a color image for position estimation, and the depth image and the color image captured for performing depth correction are referred to as a correction depth image and a correction color. This is called an image. In addition, a depth image that is a target for depth correction is hereinafter referred to as a correction target depth image. Furthermore, among the color images captured from the auxiliary camera, the image captured for the auxiliary camera position estimation is the auxiliary camera position estimation color image, and the color image captured at the same time as the correction color image. This is referred to as an auxiliary camera correction color image.

  The subject captured in the auxiliary camera position estimation color image and the position estimation color image needs to capture an object having a known shape. Any shape object may be used as long as the shape is known. The subject captured in the correction depth image and the correction color image needs to capture an object having a known shape. Any shape object may be used as long as the shape is known, but a color / material other than black that does not absorb near-infrared light is used. For example, a check pattern composed of white and gray may be used as a candidate. In addition, by pasting a poster image with a known actual size and texture on a flat wall or floor, it can be easily used as a known shape object. By using a subject other than black, the depth value can be measured more accurately.

  Note that the correction color image is captured from the same viewpoint as the paired correction depth image. Since a general distance sensor acquires a near-infrared image from the same viewpoint in order to calculate a depth image, a near-infrared image can be used as a color image. In the case of a laser type distance sensor other than the near-infrared image, a laser reflection intensity image can be used as a color image. In addition, a visible image can be used as a color image of a sensor as long as it is a distance sensor having an image element sensor that can acquire color information and depth information at the same position.

  Next, operations of the auxiliary camera position estimation unit 16 and the sensor position propagation unit 17 shown in FIG. 5 will be described with reference to FIG. FIG. 7 is a diagram showing an outline of operations of the auxiliary camera position estimation unit 16 and the sensor position propagation unit 17 shown in FIG. The auxiliary camera position estimation unit 16 estimates the camera parameters of the auxiliary camera and the distance sensor connected to the input information acquisition unit 11. First, the auxiliary camera translation parameter Tc_l, rotation parameter Rc_l, and distance sensor internal parameter Ac for each of a plurality of L auxiliary camera position estimation color images (L is a natural number) captured by the input information acquisition unit 11; A lens distortion coefficient matrix Dc is calculated. Since the calculation method solves the PnP problem in the same manner as the sensor position estimation unit 12 of the first embodiment, detailed description thereof is omitted. Next, similarly for the distance sensor, the translation parameter Td_l, the rotation parameter Rd_l, the internal parameter Ad of the distance sensor, and the lens distortion coefficient matrix Dd are calculated using L position estimation color images. To do.

  Next, the sensor position propagation unit 17 calculates each correction color image from the translation parameter Tc_l and rotation parameter Rc_l of the auxiliary camera estimated by the auxiliary camera position estimation unit 16 and the translation parameter Td_l and rotation parameter Rd_l of the sensor. Estimate the camera parameters. First, a set of parameters is selected from Tc_l, Rc_l, Td_l, and Rd_l calculated for each of the L auxiliary camera position estimation color images and the sensor position estimation color image. Tc, Rc, Td , Rd. Any method may be used as the selection criterion. However, since it is desirable to select the parameter estimated with the highest accuracy, for example, a method using a reprojection error as an index can be cited.

Alternatively, a combination in which an object having a known shape is arranged closest to or farthest from the camera may be selected. Next, the translation parameter Tr_n and the rotation parameter Rr_n are estimated for each of the N auxiliary camera correction color images. Then, according to the following equations (4) to (7), a translation parameter Tn rotation parameter Rn, an internal parameter A of the distance sensor, and a lens distortion coefficient matrix D are calculated as parameters of the sensor correction color image.
Tn = Td + RdRc ⁻¹ (Tr_n−Tc) (4)
Rn = RdRc ⁻¹ Rr_n (5)
A = Ad (6)
D = Dd (7)

<Third Embodiment>
Next, a distance information correction apparatus according to a third embodiment of the present invention will be described. FIG. 8 is a block diagram showing a configuration of a distance information correction apparatus according to the third embodiment of the present invention. In this figure, the same parts as those in the apparatus shown in FIG. The apparatus shown in this figure is different from the apparatus shown in FIG. 1 in that a pixel correction parameter calculation unit 18 is newly provided.

  In the first embodiment, the depth correction parameter is calculated collectively for the entire image. However, in a case where the estimation accuracy of the lens distortion parameter is low, or a case where the depth correction parameter increases as the distance from the center other than the image center increases due to the characteristics of the distance measurement method, it is sometimes difficult to perform depth correction that handles the entire image collectively. Therefore, in the third embodiment, more accurate depth correction can be performed by calculating the depth correction parameter for each pixel. In the configuration shown in FIG. 8, a pixel correction parameter calculation unit 18 is provided instead of the correction parameter calculation unit 14.

The pixel correction parameter calculation unit 18 receives the correction depth image acquired by the input information acquisition unit 11, the reference depth image acquired by the reference depth projection unit 13, and the mask image, and inputs the correction depth image and the reference depth image. A comparison is performed to calculate a correction parameter for correcting the correction target depth image. First, the nth depth image for correction among N images is Cn, the depth value of the pixel (i, j) included in Cn is Cn (i, j), and the reference depth image is similarly Dn, pixel ( Let the depth value of i, j) be Dn (i, j). For the mask image, a set of pixels having a pixel value of 1 is represented as Q. In all the pixels, the energy function J (i, j) shown below is minimized so that Cn (i, j) is closest to Dn (i, j).
J (i, j) = ( 1/2) Σ N n ‖Dn (i, j) -F (Cn (i, j)) ‖ ² (8)

In this embodiment, assuming that there is a linear error with respect to the reference depth image as the function F, Dn (i, j) = A (i, j) Cn (i, j) + B (i, j) Adopt a model and explain. In addition, a model proportional to the reciprocal of the depth value such as Dn (i, j) = A (i, j) / (Cn (i, j) + B (i, j)) may be used. The energy function J to be minimized when the linear model is used is shown in the following equation (9).
J (i, j) = ( 1/2) Σ N n ‖Dn (i, j) -A (i, j) Cn (i, j) -B (i, j) ‖ ² (9)

  Correction parameters A (i, j) and B (i, j) are calculated so as to minimize J (i, j), but a general least square method is used for J (i, j). Since there is an analytical solution at, it can be solved by a general calculation method, and thus detailed explanation is omitted.

  The depth value correction unit 15 receives the correction parameters A (i, j) and B (i, j) calculated by the pixel correction parameter calculation unit 18 and the correction target depth image acquired by the input information acquisition unit 11 as inputs. Then, the depth value of each pixel of the correction target depth image is corrected. First, Gm (i, j) is the depth value of the pixel (i, j) included in the mth correction target depth image Gm out of M, and the depth value of the pixel of the corrected depth image is similarly set. Hereinafter, it is expressed as Vm (i, j). Similar to the pixel correction parameter calculation unit 18, when a linear model is used, the calculation for performing the correction is Vm (i, j) = A (i, j) Gm (i, j) + B (i, j). Become.

  Even when different models are used, the depth value can be corrected by performing the same process. After performing the above correction for all pixels, Gm is stored as a corrected depth image. Further, the above-described processing is performed on all M correction target depth images, and the corrected depth images are output or stored, and the processing is terminated.

  As described above, when performing a depth image correction method that is one of the results obtained by using the three-dimensional shape measurement technique, using a set of a color image and a depth image captured at the same time, The depth value is corrected for each pixel of the depth image.

  According to this configuration, a more accurate three-dimensional shape can be acquired by correcting a desired distance image.

  You may make it implement | achieve all or one part of the distance information correction apparatus 1 in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be a program for realizing a part of the above-described functions, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system. It may be realized using hardware such as PLD (Programmable Logic Device) or FPGA (Field Programmable Gate Array).

  As mentioned above, although embodiment of this invention has been described with reference to drawings, the said embodiment is only the illustration of this invention, and it is clear that this invention is not limited to the said embodiment. is there. Therefore, additions, omissions, substitutions, and other modifications of the components may be made without departing from the technical idea and scope of the present invention.

  The present invention can be applied to applications where it is essential to acquire a more accurate three-dimensional shape by correcting a desired distance image.

  DESCRIPTION OF SYMBOLS 1 ... Distance information correction apparatus, 11 ... Input information acquisition part, 12 ... Sensor position estimation part, 13 ... Reference depth projection part, 14 ... Correction parameter calculation part, 15 ... Depth Value correction unit, 16 ... auxiliary camera position estimation unit, 17 ... sensor position propagation unit, 18 ... pixel correction parameter calculation unit

Claims (8)

Input information acquisition means for acquiring input information which is a pair of a depth image including a correction depth image and a correction target depth image and a color image including a correction color image captured simultaneously with the correction depth image;
Sensor position estimation means that inputs information of a known three-dimensional shape object and estimates the position of the sensor when the correction color image is acquired among the color images acquired by the input information acquisition means;
A reference depth image projecting means for projecting a known shape as a reference to obtain a reference depth image based on the estimated position of the sensor;
Correction parameter calculation means for comparing the reference depth image and the correction depth image acquired by the input information acquisition means to calculate a depth correction parameter;
A distance information correction apparatus comprising: a depth value correction unit that corrects the depth image to be corrected using the depth correction parameter calculated by the correction parameter calculation unit.   The distance information correction apparatus according to claim 1, wherein the input information acquisition unit acquires the depth value of the depth image using the known three-dimensional shape object whose surface is not black.   The distance information correction apparatus according to claim 1, wherein the depth value correction unit collectively corrects the depth value of the entire image using the depth correction parameter obtained by the correction parameter calculation unit. The correction parameter calculation means compares the reference depth image with the correction depth image acquired by the input information acquisition means, calculates a depth correction parameter for each pixel based on the comparison result,
The distance information correction device according to claim 1, wherein the depth value correction unit corrects the correction target depth image using the depth correction parameter for each pixel calculated by the correction parameter calculation unit. Input information acquisition means for acquiring input information which is a pair of a depth image including a correction depth image and a correction target depth image and a color image including a correction color image captured simultaneously with the correction depth image;
An auxiliary camera position estimating means for providing an auxiliary camera and estimating the position of the auxiliary camera;
Sensor position propagation means for estimating the position of the camera that has captured the correction color image based on the position of the auxiliary camera that has been estimated;
A reference depth image projecting means for projecting a known shape as a reference to obtain a reference depth image based on the estimated position of the camera;
Correction parameter calculation means for comparing the reference depth image and the correction depth image acquired by the input information acquisition means to calculate a depth correction parameter;
A distance information correction apparatus comprising: a depth value correction unit that corrects the depth image to be corrected using the depth correction parameter calculated by the correction parameter calculation unit. A distance information correction method performed by a distance information correction device for correcting a depth image,
An input information acquisition step for acquiring input information which is a pair of a depth image including a correction depth image and a correction target depth image and a color image including a correction color image captured simultaneously with the correction depth image;
A sensor position estimation step for inputting information of a known three-dimensional shape object and estimating the position of the sensor when the correction color image is acquired among the color images acquired by the input information acquisition step;
A reference depth image projecting step for obtaining a reference depth image by projecting a reference known shape based on the estimated position of the sensor;
A correction parameter calculation step of calculating a depth correction parameter by comparing the reference depth image and the correction depth image acquired by the input information acquisition step;
A distance information correction method comprising: a depth value correction step of correcting the correction target depth image using the depth correction parameter calculated by the correction parameter calculation step. A distance information correction method performed by a distance information correction device for correcting a depth image,
An input information acquisition step for acquiring input information which is a pair of a depth image including a correction depth image and a correction target depth image and a color image including a correction color image captured simultaneously with the correction depth image;
An auxiliary camera position estimating step for providing an auxiliary camera and estimating the position of the auxiliary camera;
A sensor position propagation step for estimating the position of the camera that has captured the correction color image based on the position of the auxiliary camera whose position has been estimated;
A reference depth image projecting step for projecting a known shape as a reference to obtain a reference depth image based on the estimated position of the camera;
A correction parameter calculation step of calculating a depth correction parameter by comparing the reference depth image and the correction depth image acquired by the input information acquisition step;
A distance information correction method comprising: a depth value correction step of correcting the correction target depth image using the depth correction parameter calculated by the correction parameter calculation step.   A distance information correction program for causing a computer to function as the distance information correction device according to any one of claims 1 to 5.