JP7729595B2 - Coordinate alignment system for range image sensors and other measurement systems - Google Patents

Description

The present invention relates to the alignment of a motion measurement system using a distance image sensor with other measurement systems.

In the fields of sports and rehabilitation, three-dimensional motion analysis using motion capture is performed to record and analyze the movements of subjects. While optical motion capture and motion capture using inertial sensors have been well-known motion capture technologies, in recent years, so-called markerless motion capture, which does not use markers or inertial sensors, has emerged. For example, a markerless motion capture technique uses infrared or LiDAR (Light Detection and Ranging) to acquire a single distance image (depth image) using a Time of Flight (TOF) method, and then estimates joint positions from the acquired depth image (Non-Patent Document 1). Skeletal information estimation based on distance images is also described in Patent Documents 1 to 3, for example. In recent years, relatively inexpensive motion capture using distance imaging has appeared on the market (Non-Patent Document 2), and its use in clinical settings is expected (Non-Patent Document 3).

Meanwhile, measurement systems using force plates and pressure distribution sensors, which use floor reaction force sensors, are known as measurement systems for analyzing a subject's gait (Patent Document 4, Patent Document 5). It would be useful to perform synchronized measurements between a motion analysis system using a distance image sensor and another system (e.g., a pressure distribution sensor or force plate) and use the mutual information to gain a deeper understanding of the subject's movements. For example, in the case of rehabilitation, a comprehensive understanding of the subject's movements using a distance image sensor and other devices can provide useful information for determining an appropriate treatment plan.

However, ordinary distance image sensors do not have a means for aligning coordinates with other devices, and are unable to perform measurements in the same coordinate system. Patent Document 6 is an example of prior art for a function for aligning a motion analysis device with other external devices. This method can be used not only with force plates, but also with other devices. However, for the following reasons, this method cannot be applied to motion measurement systems that use infrared distance image sensors.

First, with a range image sensor, it is not possible to obtain the distance from an infrared reflective marker, and therefore the position of the marker cannot be identified. Through experiments conducted by the inventors, it was discovered that when attempting to obtain the distance from an infrared reflective material using a range image sensor, it is not possible to calculate an appropriate distance. Second, because range image sensors do not use the DLT method (adopted in Patent Document 6), the calculation formula used in Patent Document 6 cannot be used.

Patent No. 6617830 Patent No. 5211069 Patent No. 6433149 Patent Publication No. 2001-29329 Patent Publication No. 2006-284404 Patent Publication No. 2013-162911

J. Shotton et al., “Real-Time Human Pose Recognition in Parts from a Single Depth Image,” Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), IEEE CS Press, 2011, pp. 1297-1304. Z. Zhang. Microsoft kinect sensor and its effect. IEEE MultiMedia, 19(2):4-10, Feb 2012. Application of 3D motion analysis using markerless motion capture Koichi Haruna, Keisuke Kon, Jun Inagaki, Yoichiro Sato Journal of the Japanese Society of Prosthetics and Orthotics, Vol. 35, No. 1, pp. 17-23, 2019

The present invention aims to solve the above-mentioned problems when achieving coordinate alignment between a motion analysis system using a distance image sensor and another measurement system (such as a pressure distribution sheet).

The first technical means adopted by the present invention is:
a target surface including a first reflecting surface at the center and a second reflecting surface provided so as to surround the first reflecting surface;
the second reflecting surface is formed from an infrared reflecting material;
It is a distance measuring tool used in conjunction with an infrared distance image sensor.
In one embodiment, the target surface and the first reflecting surface are concentric circles, and the second reflecting surface is ring-shaped.
In one aspect, the orientation of the target surface is variable.

The second technical means adopted by the present invention is:
A distance measurement method using a camera, a distance image sensor using infrared rays, and a distance measurement jig,
The distance measuring jig is
a target surface including a first reflecting surface at the center and a second reflecting surface provided so as to surround the first reflecting surface;
the second reflecting surface is formed from an infrared reflecting material;
detecting an area of the target surface by image processing based on the image acquired by the camera, and detecting a specific point on the first reflecting surface;
acquiring a distance to the specific point on the first reflecting surface using infrared rays emitted from the distance image sensor;
A ranging method including:
In one aspect, the image processing comprises:
acquiring a first image including a distance measuring jig installed at a predetermined position using a camera whose position and orientation are fixed;
acquiring a second image with the camera without the distance measuring jig installed;
detecting a region of the target surface based on a difference in brightness between the first image and the second image, and detecting a specific point on the first reflecting surface;
It consists of:
In one embodiment, the particular point is the center of the target surface.
In one embodiment, the camera is an infrared camera.

The third technical means adopted by the present invention is to obtain the coordinate values of a specific point on the target using a distance measurement method;
The 3D coordinate values (U, V, D) of the depth image acquired by the distance image sensor can be converted into 3D coordinate values (X, Y, Z) of a three-dimensional coordinate system,
Using the distance image sensor and the distance measurement method, obtain 3D coordinate values (U, V, D) on a depth image of the specific point on the target surface;
The 3D coordinate values (X, Y, Z) in the three-dimensional coordinate system corresponding to the 3D coordinate values (U, V, D) are obtained.

The fourth technical means adopted by the present invention is:
A coordinate alignment system in a composite measurement system including a first measurement system having a distance image sensor using infrared rays and a second measurement system,
the coordinate alignment system includes three calibration jigs detachably installed at predetermined positions of the second measurement system, and a camera;
Each calibration jig is
a target surface including a first reflecting surface at the center and a second reflecting surface provided so as to surround the first reflecting surface;
the second reflecting surface is formed from an infrared reflecting material;
a specific point on the first reflecting surface is a target,
the three calibration jigs are installed in the second measurement system in a positional relationship such that the first target, the second target, and the third target form vertices of a scalene triangle;
In the first measurement system, 3D coordinate values (U, V, D) of the depth image acquired by the distance image sensor can be converted into 3D coordinate values (X, Y, Z) of a first coordinate system;
the coordinate alignment system includes a processing unit and a storage unit;
the storage unit stores reference coordinate values of each target in the first coordinate system (X, Y, Z) of the range image sensor when the first coordinate system of the first measurement system and the second coordinate system of the second measurement system are coordinate-aligned;
The processing unit
detecting an area of the target surface by image processing based on the image acquired by the camera, and detecting a specific point on the first reflecting surface as a target;
Using the range image sensor, obtain measurement coordinate values (X, Y, Z) of each target on a first coordinate system;
The system is configured to associate measurement coordinate values with reference coordinate values for each target and obtain a transformation formula that matches the measurement coordinate values with the reference coordinate values.

The present invention makes it possible to provide a system that implements a function for aligning the coordinates of a motion analysis system using a distance image sensor with another measurement system (such as a pressure distribution sheet). By aligning the coordinate systems of a motion analysis system using a distance image sensor with another measurement system (such as a pressure distribution sheet), the usefulness of the combined system can be increased. For example, in rehabilitation, gait analysis can be performed using the combined system to obtain information useful for determining treatment guidelines for patients, etc.

FIG. 1 is an overall view of a composite measurement system including a first measurement system and a second measurement system using a range image sensor. 10A and 10B are diagrams showing a jig for acquiring coordinate values of a distance measuring jig according to the present embodiment. FIG. 2 is a diagram showing an installation jig for the distance measuring jig according to the embodiment. This figure shows a distance measurement jig related to this embodiment, where the upper figure is a different oblique view of the distance measurement jig, the lower figure (left) shows an installation jig, and the lower figure (right) shows the state in which a coordinate value acquisition jig is attached to the installation jig. FIG. 10 is a diagram showing the attached state of the installation jig. 10A to 10C are diagrams illustrating steps for acquiring coordinate values of a target using a distance measuring jig according to the present embodiment. 1 is a schematic diagram of a coordinate alignment system according to an embodiment of the present invention.

[A] Composite Measurement System The composite measurement system according to this embodiment will now be described. As shown in Figure 1, the measurement system according to this embodiment consists of a first measurement system and a second measurement system. The first measurement system is a motion measurement system using a distance image sensor, and the second measurement system is a pressure distribution sheet equipped with a pressure distribution sensor.

The first measurement system includes a distance image sensor that acquires a depth image of the object, and a 3D joint position estimation means that acquires the 3D joint positions of the object based on the depth image of the object acquired by the distance image sensor, and measures the movement of the object walking on the pressure distribution sheet.

The 3D joint position estimation means according to this embodiment can be configured from a computer equipped with a memory and a processor. The 3D joint position estimation means according to the first measurement system acquires a person area based on depth data (distance image data) and estimates joint positions by inputting distance information about the person area into a classifier consisting of a trained model. A program that estimates the 3D joint positions of an object based on the object's depth data is stored in the computer's memory, and the program is executed by the processor to estimate and output the 3D joint positions of the object. Estimating the three-dimensional coordinate values of an object's joints based on a depth image of the object is itself well known, and is described in Patent Documents 1 to 3 and Non-Patent Document 1.

In this specification, the 3D coordinate values of the range image sensor are defined as the coordinate values (Xc Yc, Zc) obtained using the range image sensor. The 2D coordinate values of the range image sensor are defined as the coordinate values (U, V, D) obtained when the coordinate values are displayed in 2D. The coordinate values (U, V) are the coordinate values on the monitor, and D is the depth distance from the sensor. The range image sensor of the first measurement system implements a conversion function from the 2D coordinate values (U, V, D) to the 3D coordinate values (Xc, Yc, Zc). The first coordinate system of the first measurement system in Figure 1 is the 3D coordinate values (Xc Yc, Zc) of the range image sensor.

The first measurement system may include a distance image sensor that acquires a depth image (distance image) of the object, as well as a video camera (RGB camera) that acquires a color image (RGB image) of the object. In this case, it is desirable that the coordinate values of the depth image and the 3D coordinate values of the video camera are mutually convertible. In one aspect, a 2D joint position estimation program that estimates 2D joint positions based on an RGB image is stored in computer memory, and pose estimation (2D joint position estimation) is performed to acquire the 2D joint positions of the object based on the color image (RGB image) of the object acquired by the video camera. The 3D joint positions estimated from the distance image sensor may be corrected using the 2D joint positions estimated from the camera image. In this case, the distance image sensor and video camera may be integrated into a composite sensor, or the distance image sensor and video camera may be provided separately. Kinect (registered trademark) is a typical example of such a composite sensor system. Kinect is a composite sensor consisting of an infrared projector, a depth camera, and a color image camera, and the infrared projector and infrared camera form a distance image sensor (depth sensor) (it does not have a pose estimation method based on RGB images).

The pressure distribution sheet serving as the second measurement system acquires the plantar load distribution of a subject walking on the sheet. The plantar load distribution is acquired according to a second coordinate system (xy plane). In this embodiment, the 3D coordinate system of the second measurement system is assumed to be (Xs, Ys, 0), since the system (pressure distribution sheet) is placed on the floor for use. The second coordinate system of the second measurement system in Figure 1 is the 3D coordinate value (Xs, Ys, 0) of the pressure distribution sheet. The second measurement system includes a computer including a processor that processes data acquired by the load sensor of the pressure distribution sheet and memory for storing the data. The second measurement system has an external input function (in one embodiment, an external input terminal of a synchronization cable) and is configured to input a synchronization signal from the first measurement system (master). Note that the second measurement system may also be configured as the master to output a synchronization signal to the first measurement system. While the second measurement system in this embodiment is a pressure distribution sheet, the second measurement system is not limited to a pressure distribution sheet and may be, for example, a force plate.

The first coordinate system of the first measurement system and the second coordinate system of the second measurement system are coordinate-aligned, and the measurement data acquired by the first measurement system and the measurement data acquired by the second measurement system are synchronized. Gait analysis of the subject is performed based on the motion data of the subject while walking (including posture data at a certain point in time) acquired by the first measurement system and the time and distance factors while walking acquired by the second measurement system.

One synchronization method is, for example, to prepare a jig that outputs a synchronization signal at the same time as illuminating an LED at the moment measurement starts, and to synchronize the start with the moment the LED lights up on the color camera of the range image sensor of the first measurement system (which is pre-synchronized with the range image sensor) and the moment the second measurement system receives the synchronization signal. In this embodiment, synchronization is achieved by sounding a buzzer with a speaker. Various known means can be used as appropriate to synchronize the first and second measurement systems.

[B] Coordinate Alignment System The coordinate alignment system includes a composite measurement system consisting of a first measurement system and a second measurement system each equipped with a distance image sensor, as well as a camera (infrared camera), three distance measurement jigs (calibration jigs), and a computer equipped with a memory and a processor. The computer constituting the composite measurement system and the computer constituting the coordinate alignment system may be the same computer.

[B-1] Distance measurement jig (calibration jig)
A distance measuring jig according to this embodiment will be described. The distance measuring jig according to this embodiment is a distance measuring jig used together with a distance image sensor that uses infrared rays. The distance image sensor according to this embodiment is a distance measuring means that irradiates an object with light from a built-in infrared LED and measures the time it takes for the light to reflect and return to determine the distance. The distance measuring jig according to this embodiment is composed of a coordinate value acquisition jig 1 and an installation jig 2 for the coordinate value acquisition jig (hereinafter referred to as the "installation jig"). The distance measuring jig according to this embodiment is a calibration jig used to align the first coordinate system of a first measurement system and the second coordinate system of a second measurement system.

As shown in Figure 2, the coordinate value acquisition jig 1 comprises a main body 3 including a target surface, a support portion 4 of the main body 3, and a base 5 located at the bottom of the support portion 4. The target surface comprises a first reflective surface 6 in the center and a second reflective surface 7 surrounding the first reflective surface 6, with the second reflective surface 7 being made of an infrared-reflective material. Through research by the inventors, we have discovered that when attempting to acquire distance from an infrared-reflective material using a distance image sensor, an appropriate distance cannot be calculated. Therefore, even if the spherical infrared reflective marker disclosed in Patent Document 6 is used as a calibration jig, the distance D of the coordinate values (U, V, D) of the distance image cannot be acquired. To solve this problem, we developed a distance measurement jig (calibration jig) with a target surface consisting of two reflective surfaces (first reflective surface 6 and second reflective surface 7) with different physical properties.

The infrared reflective material of the second reflective surface 7 is a material that produces a high-brightness area when infrared light is projected onto it and a camera (infrared camera) is used (a material from which distance cannot be measured using an infrared distance image sensor). Infrared reflective materials are well known, and for example, the second reflective surface 7 can be formed by applying commercially available infrared reflective tape to a predetermined location on the target surface.

The material of the first reflective surface 6 surrounded by the second reflective surface 7 must be a material that allows the infrared distance image sensor to obtain distance by reflecting infrared rays (it is not a material that absorbs or scatters infrared rays). Due to the nature of infrared rays, it is understood that many materials around us in everyday life meet this condition.

In the distance measurement jig according to this embodiment, the target surface and the first reflecting surface 6 are concentric circles, and the second reflecting surface 7 is ring-shaped. The center of the target surface, i.e., the center of the first reflecting surface 6, is the distance measurement target, and the distance measurement jig measures the distance to this distance measurement target. In this embodiment, a strip of infrared reflective tape is attached to the second reflecting surface 7, and the distance to the second reflecting surface 7 (the area of the infrared reflective tape) on the target surface cannot be obtained by the range image sensor. A key feature of this embodiment is that the central region of the target surface (first reflecting surface 6) does not have infrared reflective tape attached. While the target surface appears as a high-brightness area due to halation in the infrared reflective camera, the center of the high-brightness area does not have infrared reflective tape attached, so the distance to the center can be obtained by the range image sensor. In other words, the range image sensor can obtain the target's coordinate values (U, V, D).

In one embodiment, the width of the ring-shaped second reflective surface 7 is set so that brightness is high all the way to the center. Specifically, the infrared reflective tape is set to a width that reflects infrared light output from the distance image sensor, causing halation in the infrared reflective camera, resulting in high brightness all the way to the center. As described below, when detecting the target surface area through image processing, the difference in brightness between images acquired before and after placing the jig is taken, and the area with the largest difference is detected as the target surface. Since the center position of the constant brightness area is calculated at this time, it is desirable that the area be illuminated all the way to the center. However, it will be understood by those skilled in the art that even if the area is not necessarily illuminated all the way to the center, the target surface can be detected by designing the image processing program.

The main body 3 of the distance measuring jig is a vertically oriented disk, the base 5 is a horizontally oriented disk, and the support part 4 consists of a raised part 40 and a square plate-shaped base part 41 at the lower end of the raised part 40, with the base part 41 being integrated with the upper surface of the base 5.

The installation jig 2 is a jig that, when combined with the coordinate value acquisition jig 1, aligns the first coordinate system of the first measurement system (distance image sensor) with the second coordinate system of the second measurement system (pressure distribution sheet). As shown in Figure 3, the installation jig 2 comprises a main body 8 that is rectangular in plan view, a circular opening 80 formed in the center of the main body 8, and a fixing part 9 extending from one side of the rectangular main body 8, and can be attached to a predetermined position on the pressure distribution sheet via the fixing part 9.

The main body 8 of the installation jig 2 in this embodiment is a sheet-like or plate-like body of a predetermined thickness. The fixing portion 9 is thinner than the thickness of the main body 8, and is fixed to the pressure distribution sheet by abutting the end face of the main body 8 against the edge of the pressure distribution sheet, placing the main body 8 on the floor, and overlapping the fixing portion 9 on the pressure distribution sheet. The installation jig 2 is designed to be fixed to a predetermined position with high reproducibility by utilizing the configuration and shape of the second measurement system that constitutes the integrated measurement system, and the installation jig 2 in this embodiment is configured to be fixed to a specific position on the pressure distribution sheet with high reproducibility. In this embodiment, the shape is designed to be fixed to the connection between the pressure distribution sensors, allowing it to be attached to a specific position on the pressure distribution sheet with high precision. The fixing portion 9 of the installation jig 2 according to this embodiment has a U-shaped groove 90 in plan view that receives a pair of protruding elements on the pressure distribution sheet. By receiving the protruding elements in the groove and abutting the end face of the main body 8 against the edge of the pressure distribution sheet, the position of the installation jig (i.e., the distance measurement jig or calibration jig) relative to the pressure distribution sheet is determined. Note that the illustrated configuration of the installation jig 2 (particularly the configuration of the fixing portion 9) is merely one example, and it will be understood by those skilled in the art that the fixing portion 9 can be designed to correspond to the specific configuration of the second measurement system that makes up the composite measurement system, for example.

By placing the base 5 of the coordinate value acquisition jig 1 in the circular opening 80 of the main body 8 of the installation jig 2, the coordinate value acquisition jig 1 is installed at a predetermined position on the pressure distribution sheet. The bottom surface of the base 5 of the coordinate value acquisition jig 1 is placed on the floor surface on which the pressure distribution sheet is placed. Because the disc-shaped base 5 is located within the circular opening 80, the base 6 can rotate within the opening 80. In other words, with the ranging jig attached at a predetermined position on the pressure distribution sheet, the orientation of the target surface of the coordinate value acquisition jig 1 can be changed.

As described above, the base 5 of the coordinate value acquisition jig 1 is rotatable within the opening 80 of the main body 8 of the installation jig 2, making the coordinate value acquisition jig 1 rotatable relative to the main body 8 of the installation jig 2. In other words, the target surface of the coordinate value acquisition jig 1 is rotatably supported, and the orientation of the target surface can be changed by rotating the coordinate value acquisition jig 1. Therefore, the orientation of the target surface can be changed to match the orientation of the range image sensor. Note that the coordinate value acquisition jig and the installation jig may be integrated (the base of the coordinate value acquisition jig doubles as the installation jig) to form a ranging jig, and the support part of the coordinate value acquisition jig may be rotatable relative to the base. Furthermore, although the target surface is vertical in this embodiment, the inclination of the target surface may be variable.

In this embodiment, a concentric target surface and first reflecting surface 6 are used, but the shapes of the target surface and first reflecting surface 6 are not limited to circles. The target surface consists of a central first reflecting surface 6 and a second reflecting surface 7 surrounding the first reflecting surface 6. The second reflecting surface 7 is made of an infrared reflective material. Any shape is acceptable as long as it allows a specific point (e.g., the center) on the target surface (first reflecting surface 6) to be calculated based on the area of the target surface extracted from the image; for example, the target surface and first reflecting surface 6 may be rectangular (concentric rectangles).

In this embodiment, the base 5 of the coordinate value acquisition jig 1 is disk-shaped, and by rotating the coordinate value acquisition jig 1 relative to the installation jig 2, the target surface can be made to face the lens surface of the camera, and a mechanism is provided in which the position of the midpoint does not change even when calculated for the planar projection image onto the camera.

[B-2] Camera: In this embodiment, the target surface of the distance measuring jig is detected by image processing based on images acquired using an infrared camera. Note that the target surface can be extracted using a normal camera because its brightness is stronger than other areas. While an infrared camera is not required, it is advantageous to use an infrared camera image to more effectively detect the target surface area. The coordinate system of the range image sensor and the coordinate system of the infrared camera (camera) are matched in advance. Conversion information between the 2D coordinate system of the range image sensor and the 2D coordinate system of the camera is obtained in advance and is convertible. Furthermore, the 2D coordinate system of the range image sensor (the coordinate system of the depth image) and the 3D coordinate system are convertible.

[B-3] Acquisition of Coordinate Values of Targets of Distance Measuring Jig (Calibration Jig) The steps for acquiring coordinate values of targets of a distance measuring jig (calibration jig) will be described with reference to FIG.
(a) An infrared camera is used to obtain an image including a distance measuring jig placed at a predetermined position on the pressure distribution sheet as a first image.
(a) An image of the pressure distribution sheet without the distance measuring jig is acquired as a second image using an infrared camera (at the same position and posture). The order in which the first image and the second image are acquired is not limited.
(c) By image processing, the area of the target surface of the distance measuring jig is detected based on the difference in brightness between the first image and the second image.
(d) Based on the detected area, the coordinate values (U, V) of a specific point (center) on the first reflecting surface 6 of the target surface are obtained.
(e) Using a distance image sensor, the distance D to a specific point (center) on the first reflecting surface 6 is obtained.
(f) The coordinate values (U, V, D) of a specific point on the target surface are converted into coordinate values (Xc, Yc, Zc).

[B-4] Coordinate Alignment Step The coordinate alignment step will now be described. Three calibration jigs are installed in predetermined positions on the pressure distribution sheet in advance, and the three-dimensional coordinate values of the three targets (first target, second target, third target) when coordinate alignment is performed are set as reference coordinate values. During measurement, the coordinates of the three targets (first target, second target, third target) on the three calibration jigs are measured, and the measured coordinate values of the three targets are obtained. Coordinate alignment is performed by translating and rotating the targets so that their measured coordinate values match the set reference coordinate values.

The following contents are set in advance and stored in the memory of the computer.
(i) Information about where the origin position of the pressure distribution sheet corresponds to on the distance image sensor. The amount of parallel translation in the X direction and the Y direction (for example, in mm) and the amount of rotation (deg) are set in the configuration file.
(ii) Information about where the coordinate values of the calibration jig are located when the pressure distribution sheet and the coordinate system of the range image sensor are aligned. In the configuration file, set the coordinate values of the 3D coordinate system of the range image sensor of the three targets as reference coordinate values.

The three points set in each coordinate ^system _are ^Psc1 = ( ^Xsc1 , _{Ysc1 ,} ^Zsc1 ), ^Psc2 = ₍ ^Xsc2 , ^Ysc2 , _Zsc2 ), _Psc3 = ( _{Xsc3, Ysc3, Zsc3) (coordinate values after movement). At this time, the lengths of each side of the triangle formed by these three points are set to be different. The three points, Psc1 = (} ^Xsc1 _{, Ysc1 , Zsc1), Psc2 = (Xsc2, Ysc2 ,} ^{Zsc2 ) , Psc3 =} ₍ ^{Xsc3 , Ysc3 , Zsc3 ) ,} _can ^be _set ^reproducibly _on ^the sheet _{and can} be any _{three points that} are not on _a straight line ^. In this embodiment, P ^s _c1 is a point that can be fixed to the seat in the seat coordinate system, P ^s _c2 is a point in the seat coordinate system that has the same Y coordinate value as P ^s _c1 , and P ^s _c3 is a point in the seat coordinate system that has the same X coordinate value as P ^s _c2 .

Before measurement, the 3D coordinate values of the three calibration jigs are obtained. An infrared image is captured as the first image without the calibration jigs in place. The three calibration jigs are placed in their designated positions, and an infrared image is captured as the second image. The difference in brightness between the first and second images is calculated, and positions above a certain value are identified as target surfaces, and the coordinate values (U, V) of the center of each target surface are obtained. The 2D image from the range image sensor is used to obtain (U, V, D), and conversion is performed to obtain coordinate values (X _c1 , Y _c1 , Z _c1 ), coordinate values (X _c2 , Y _c2 , Z _c2 ), and coordinate values (X _c3 , Y _c3 , Z _c3 ). The correspondence between these three coordinate values and the first, second, and third targets is obtained using the lengths of the sides of a triangle formed by the three points. In this way, the coordinate values of the three points before transformation are obtained: _Pc1 = ( _Xc1 , Yc1, _Zc1 ), _Pc2 = ( _Xc2 , _Yc2 , _Zc2 ), _Pc3 = _{( Xc3} , _Yc3 , _Zc3 ) (coordinate values before movement).

The measurement coordinate values Pc1 _{, Pc2} , and _Pc3 of the three targets are converted to the coordinate values ^Psc1 _, ^Psc1 _{, and} ^Psc1 _of the setting files of the three targets. Specifically, the following calculation is performed.
(1) Regarding the coordinate values before movement,
V ^s _e1 = V ^s _2→1 = ( P ^s _c2 − P ^s _c1 ) / | P ^s _c2 − P ^s _c1 |
V ^s _3→1 = ( P ^s _c3 − P ^s _c1 ) / | P ^s _c3 − P ^s _c1 |,
Let V ^s _e2 be the unit normal vector to V ^s _2→1 and V ^s _3→1 .
The normal vector between ^Vs _e2 and ^Vs _e1 is defined as ^Vs _e3 .
Let the matrix M ^s be M ^s = ( V ^s _e1 , V ^s _e2 , V ^s _e3 ).

(2) Regarding the coordinate values after movement,
V ^c _e1 = V _2→1 = ( P _c2 − P _c1 ) / | P _c2 − P _c1 |
Let V _3→1 = ( P _c3 − P _c1 ) / | P _c3 − P _c1 |,
Let V ^c _e2 be the unit normal vector to V _2→1 and V _3→1 .
The normal vector between V ^c _e2 and V ^c _e1 is V ^c _e3 .
Let the matrix ^Mc be ^Mc = ( ^Vc _e1 , ^Vc _e2 , ^Vc _e3 ).

(3) For the coordinate value before movement P ₀ = (X _p , Y _p , Z _p ) ^T , the transformation that places P _c1 at the origin and makes the three points coincide with the floor surface is P ₀ ' = (M ^s ) ^-1 (P ₀ -P _C1 ).
(4) The formula for converting the coordinates converted in (3) into the converted coordinate value P ₀ ″ is P ₀ ″=M ^c P ₀ ′+P _C1 .
By processing (3) and (4) simultaneously, P 0 '' can be calculated as P ₀ '' = M ^c (M ^s ) ^-1 (P ₀ -P _C1 ) + P _C1 .
This calculation makes it possible to align the coordinate system of the pressure distribution sheet and the coordinate system of the range image sensor for any point. After that, by translating it using the "information about where the origin position of the pressure distribution sheet corresponds on the range image sensor (information in the setting file)" in the setting file, it becomes possible to place the origin of the pressure distribution sheet at any location relative to the origin of the range image sensor.

Explanation of symbols

REFERENCE SIGNS LIST 1 Coordinate value acquisition jig 2 Installation jig for coordinate value acquisition jig 3 Main body of coordinate value acquisition jig 4 Support part 5 Base 6 First reflection surface 7 Second reflection surface 8 Main body of installation jig

Claims (6)

A coordinate alignment system in a composite measurement system including a first measurement system, which is a motion measurement system equipped with a distance image sensor that irradiates an object with infrared light and measures the time it takes for the infrared light to reflect and return to determine the distance, and a second measurement system made of a pressure distribution sheet ,
the coordinate alignment system comprises three calibration jigs detachably installed at predetermined positions on the pressure distribution sheet , an infrared camera, and the distance image sensor ;
Each calibration jig is
a target surface including a first reflecting surface at the center and a second reflecting surface provided so as to surround the first reflecting surface;
the second reflecting surface is formed from an infrared reflecting material;
the first reflecting surface is formed of a material that allows the distance image sensor to acquire distance by reflecting infrared rays, and a specific point on the first reflecting surface serves as a target;
Each calibration jig is placed on the pressure distribution sheet with the target surface in a vertical position,
the three calibration jigs are installed on the pressure distribution sheet in a positional relationship such that the first target, the second target, and the third target are vertices of a scalene triangle;
In the first measurement system, 3D coordinate values (U, V, D) of the depth image acquired by the distance image sensor can be converted into 3D coordinate values (X, Y, Z) of a first coordinate system;
the coordinate alignment system includes a processing unit and a storage unit;
the storage unit stores reference coordinate values of each target in the first coordinate system (X, Y, Z) of the range image sensor when the first coordinate system of the first measurement system and the second coordinate system of the second measurement system are coordinate-aligned;
The processing unit
detecting an area of the target surface by image processing based on the image acquired by the infrared camera, and detecting a specific point on the first reflecting surface as a target;
Using the 3D coordinate values (U, V, D) of each target acquired by the range image sensor, measurement coordinate values (X, Y, Z) of each target on a first coordinate system are acquired;
and obtaining a transformation formula for matching measurement coordinate values and reference coordinate values of each target, and for matching the measurement coordinate values with the reference coordinate values.
Coordinate alignment system. the target surface and the first reflecting surface are concentric circles, and the second reflecting surface is ring-shaped;
The coordinate alignment system of claim 1 . The orientation of the target surface is variable.
The coordinate alignment system of claim 1 . The tilt of the target surface is variable.
The coordinate alignment system of claim 1 . The specific point is the center of the target surface.
The coordinate alignment system of claim 2 . The image processing
acquiring a first image including the calibration jig installed at a predetermined position using an infrared camera whose position and orientation are fixed;
acquiring a second image with the infrared camera without the calibration jig installed;
detecting a region of the target surface based on a difference in brightness between the first image and the second image, and detecting a specific point on the first reflecting surface;
Consists of:
The coordinate alignment system according to any one of claims 1 to 5 .