JP4667900B2 - 3D analysis method from 2D image and system to execute it - Google Patents

Description

  The present invention relates to a method for analyzing three-dimensional information of a subject from a single two-dimensional image and a system for executing the method, and in particular, a method suitable for grasping the three-dimensional motion of a subject from time-series two-dimensional image information and About the system.

Conventionally, as a technique for obtaining three-dimensional information from a two-dimensional image photographed by a camera, it is known to use a plurality of images from different viewing angles. A technique for extracting stereoscopic images and three-dimensional shape information by stereo viewing, and a technique for obtaining and analyzing a front image and a side image by arranging two cameras at different viewing angles.
In Patent Document 1, when extracting three-dimensional information by stereo vision, it is possible to extract three-dimensional information without performing multiple photographing with the same sensor, and it is easy to automate parameter setting used for extraction. A “stereoscopic three-dimensional information extraction device” that aims to be able to do this is disclosed. As shown in FIG. 5, from the parameter setting means, the distance between the collection position * 1 of the SAR sensor and the observation area, the distance between the collection position * 2 of the optical sensor and the observation area, the collection position * 1 of the SAR sensor Correspondence between various observation parameters such as the distance of the collection position * 2 and the optical sensor, and a plurality of feature points existing in the SAR image * 3 from the collection position * 1 and the optical sensor image * 4 from the collection position * 2 Information is manually entered by the operator. The image processing means performs a trigonometric function calculation based on the information given from the parameter setting means for a plurality of feature points in the two images * 3, * 4, and the two two-dimensional images * 3, * The three-dimensional information of the feature points in 4 is extracted. Since it is based on stereo vision, two two-dimensional images are required.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for easily generating three-dimensional data on a computer from a two-dimensional image. The image processing unit of the data generation apparatus includes a “storage unit, a coordinate system setting unit that sets a partitioned space coordinate system, a virtual space coordinate system, and a screen coordinate system for a two-dimensional image, and a partitioned space coordinate system, respectively. A first coordinate conversion equation setting unit for setting a coordinate conversion equation from the virtual space coordinate system to the virtual coordinate system, a second coordinate conversion equation setting unit for setting a coordinate conversion equation from the virtual space coordinate system to the screen coordinate system, and a reference Based on the rectangle setting unit, each coordinate conversion formula, the coordinates of the reference rectangle, etc., the coefficient calculation unit that calculates the coefficient of each coordinate conversion formula, and the three-dimensional data of the two-dimensional image based on the coefficient of each coordinate conversion formula Including a data generation unit to be generated ". Since the present invention relates to a data generation apparatus that generates three-dimensional data of a subject that is a partitioned space such as a captured room, the subject is stationary such as a ceiling, a floor, a wall, and a pillar, and its direction and length The target is known. Although it has been suggested that 3D information is analyzed from 2D images using the known data, the 2D images are premised on a subject space defined by at least two orthogonal planes orthogonal to each other. .

Further, Patent Document 3 discloses a three-dimensional data creation device that can create three-dimensional data with high accuracy with simple configuration and simple processing from photographs and perspectives used in architectural design and the like. A data creation method and apparatus are disclosed. This device is a digital image input device and scanner that captures a still image of a space whose spatial structure is unknown, and the captured still image as a projected image of the space, and is based on the positional relationship of straight lines in the still image. A means for determining a viewpoint for the still image, a means for creating three-dimensional data of a spatial structure in the still image based on the still image and viewpoint information, and an image processing computer for outputting the created three-dimensional data. Is. In addition, an image as shown in FIG. 6 is shown, and in the specification [0016], “If there is one line whose position and length are known in the image information shown in the photograph or perspective, You can know the length and size of all the parts, and for other parts such as furniture and baskets, you can usually guess and supplement some information such as the height, distance from the wall, angle with the wall, etc. Therefore, it is possible to estimate the positional relationship fairly accurately based on the information that has been estimated and supplemented, thereby estimating the mutual positional relationship of all the objects in the photograph, and It can be obtained. "
Japanese Patent Laid-Open No. 2004-21427 “Three-Dimensional Information Extraction Device by Stereo Vision” Published on January 22, 2004 JP, 2004-280196, A "Data generation device and its method" Published on October 7, 2004 Japanese Patent Laid-Open No. 2000-76453 “Method and Apparatus for Creating Three-dimensional Data” Published on March 14, 2000 Especially, description of paragraph [0016]

As described above, Patent Documents 1 and 2 disclose a method of obtaining three-dimensional information of a subject using a two-dimensional image, subject size and direction information. However, these techniques are based on the premise that the subject is stationary and the direction is known, and this method cannot perform dynamic three-dimensional analysis of the subject. However, in recent years, in the sports world, it is possible to objectively understand the needs of athletes in three dimensions to help them improve their skills and the movements of animals from a biological, medical, and pharmaceutical perspective. There is a strong need to do it. However, the methods as disclosed in Patent Documents 2 and 3 are intended for a stationary special-form subject and cannot meet these needs. Although it is possible to respond to this, it is necessary to install two or more cameras, and the facilities are large, which makes it difficult to shoot and calculate data.
The problem to be solved by the present invention is that a subject that changes the position of the limbs depending on the time, such as a human being or an animal, more generally, a subject composed of line segments joined by a free joint, is obtained from a single two-dimensional image. Another object is to provide a method for calculating original information, a method for analyzing the motion of a subject from a series of two-dimensional images captured by a single camera, and a system for implementing these methods.

A method for calculating three-dimensional information from a two-dimensional image according to the present invention is based on a two-dimensional image obtained by imaging a line segment connected by a free joint and the length information of the line segment. The original information is calculated on the basis of one connecting portion position ,
The three-dimensional spatial coordinates X-axis, Y-axis, and Z-axis are related to the two-dimensional camera coordinates U-axis and V-axis so that the X-axis and U-axis are parallel, and the Z-axis and V-axis are parallel. The camera coordinates (U _m , V _m ) with respect to the three-dimensional position (X _m , Y _m , Z _m ) of M _m in the relationship between the joint M _m having a length of 1 _km and the other end M _k While calculating the (X _k , Z _k ) coordinate value of M _k using the depth direction coefficient from the (U _k , V _k ) information ,
Y _k = ± {l _km ² − (X _k −X _m ) ² − (Z _k −Z _m ) ² } ^1/2 + Y _m
The depth information Y _k of the other end M _k of the line segment is calculated based on the following formula to obtain the three-dimensional position information (X _k , Y _k , Z _k ) of the other end M _k , and further using this position as a reference The three-dimensional position of each line segment during imaging is sequentially calculated .
The method of dynamically analyzing three-dimensional information from a two-dimensional image according to the present invention is based on a two-dimensional image captured in time series with the same optical system, and a line segment in each image obtained by the above method. For the three-dimensional information, the offset amount in the optical axis direction of both images is calculated from the positional deviation information between the images of the connecting portion based on the two images, and the three-dimensional information of the line segment is corrected, The position change information of each line segment was obtained.
Another method for dynamically analyzing three-dimensional information from a two-dimensional image according to the present invention is a method for analyzing the motion of a subject three-dimensionally. A two-dimensional image captured in time series with the same optical system is used. A step of obtaining three-dimensional information of a line segment in each image obtained by the method according to claim 1, a step of calculating a center point position of each line segment in each image, and two images of the center position Calculating an offset amount in the optical axis direction of both images for each line segment from a gap between them, and sequentially correcting the calculated offset amount from a connecting portion based on the three-dimensional information of the line segment to sequentially correct each line segment The step of obtaining position change information was taken.
A further different method for dynamically analyzing three-dimensional information from a two-dimensional image according to the present invention is obtained by the method according to claim 1 on the basis of two-dimensional images taken in time series with the same optical system. For the three-dimensional information of the line segment in each image, an accelerometer is attached to the reference connecting part, a vector is calculated from the detected value of the accelerometer, and an offset amount in the optical axis direction of both images is calculated to calculate the line segment. The 3D information was corrected to obtain position change information for each line segment.

A system for analyzing three-dimensional information of a subject from a two-dimensional image according to the present invention includes a camera that outputs a captured two-dimensional image as information, means for inputting subject line segment information and line segment selection information, two-dimensional Calculating means for calculating the three-dimensional information of the line segment from the line segment information on the image and the length information of the line segment , the calculating means comprising three-dimensional spatial coordinates X-axis, Y-axis, Z-axis; two-dimensional camera coordinate U axis, X-axis and U-axis the relation between V axis, taken as Z-axis and V-axis are parallel, joint M _m and the other one of the connecting length is l _miles The depth direction coefficient is used from the camera coordinate (U _m , V _m ) and (U _k , V _k ) information with reference to the three-dimensional position (X _m , Y _m , Z _m ) of M _m in relation to the end M _k. To calculate the (X _k , Z _k ) coordinate value of M _k ,
Y _k = ± {l _km ² − (X _k −X _m ) ² − (Z _k −Z _m ) ² } ^1/2 + Y _m
The depth information Y _k of the other end M _k of the line segment is calculated based on the following formula to obtain the three-dimensional position information (X _k , Y _k , Z _k ) of the other end M _k , and further using this position as a reference The three-dimensional position of each line segment during imaging is sequentially calculated .
Further, if necessary, a system including a memory for storing two-dimensional images taken in time series and storing analysis data, a display unit for displaying two-dimensional images, and means for processing analysis results Present.
In the recording medium according to the present invention, the relationship between the three-dimensional spatial coordinates X-axis, Y-axis and Z-axis and the two-dimensional camera coordinates U-axis and V-axis is such that the X-axis and U-axis are parallel and the Z-axis and V-axis are parallel. The camera coordinates with reference to the three-dimensional position (X _m , Y _m , Z _m ) of M _m in the relationship between the joint M _m having one connected length of 1 _km and the other end M _k While calculating the (X _k , Z _k ) coordinate value of M _k using the depth direction coefficient from the (U _m , V _m ) and (U _k , V _k ) information ,
Y _k = ± {l _km ² − (X _k −X _m ) ² − (Z _k −Z _m ) ² } ^1/2 + Y _m
The depth information Y _k of the other end M _k of the line segment is calculated based on the following formula to obtain the three-dimensional position information (X _k , Y _k , Z _k ) of the other end M _k , and further using this position as a reference The three-dimensional position of each line segment at the time of imaging is calculated from the line segment position information connected by a free joint on the two-dimensional image and the length information of the line segment. A program for calculating information is written in a computer-readable form. Furthermore, if necessary, position change information is obtained from each two-dimensional image captured in time series, and the optical axis direction offset amount between the images is calculated, position correction is performed, and the movement of the subject is three-dimensional. To which the program to analyze automatically is added.

A method for calculating three-dimensional information from a two-dimensional image according to the present invention is based on a two-dimensional image obtained by imaging a line segment connected by a free joint and the length information of the line segment. Since the original information is calculated, not only a three-dimensional image for one line segment can be obtained, but also three-dimensional information for a plurality of line segments connected by free joints is obtained from the two-dimensional image. be able to.
Further, the method of analyzing 3D information from the 2D image according to the present invention is a method of changing the position of each line segment from the 3D information of each line segment obtained by the above method for a 2D image captured in time series. Since the information is obtained and the motion of the subject is analyzed three-dimensionally, a series of motions about a plurality of line segments connected by free joints can be analyzed. The position in the optical axis direction, which is difficult to grasp accurately from the two-dimensional image information, can be corrected by taking data for each line segment, or by providing a plurality of accelerometers.
If the subject is an animal, the joint corresponds to the joint, and the line segment corresponds to the limb, the body of the athlete responds to the strong needs from the sports world and biological, medical, and pharmaceutical viewpoints. It is possible to provide a method for three-dimensionally capturing a target movement from a two-dimensional image that can be easily obtained and a method for objectively grasping the movement of an animal.

A system for analyzing three-dimensional information of a subject from a two-dimensional image according to the present invention includes a camera that outputs a captured two-dimensional image as information, means for inputting subject line segment information and line segment selection information, two-dimensional The above calculation function is based on a general-purpose camera and a personal computer, since it only includes calculation means for calculating three-dimensional information of the line segment from the line segment information on the image and the length information of the line segment. It is possible to realize by providing. Furthermore, if necessary, a system including a memory for storing two-dimensional images taken in time series and storing analysis data, a display unit for displaying two-dimensional images, and means for processing analysis results In this case, it is possible not only to display a time-series image as a moving image but also to display an analysis result in a suitably processed form.
The recording medium according to the present invention is a computer-readable program that calculates the three-dimensional information of the line segment at the time of imaging from the line segment information combined by the free joint on the two-dimensional image and the length information of the line segment Therefore, simply preparing a general-purpose camera and a general-purpose computer and reading the computer into the computer can easily analyze the three-dimensional information of the subject from the two-dimensional image according to the present invention. A system can be constructed. Furthermore, if necessary, the analysis result is obtained if a program for obtaining the position change information of each line segment from each two-dimensional image captured in time series and analyzing the motion of the subject in three dimensions is added. A system for analyzing the three-dimensional information of the subject from the two-dimensional image according to the present invention, which can be displayed in an appropriately processed form, can be constructed.

  FIG. 1 shows a model example of “line segments joined by free joints” handled in the present invention. In this example, three line segments AC, BC, and CD are connected to a free joint (joint) C, and the other end D of the line segment CD is also a joint D. In addition to the line segment CD, a line segment DE is coupled, and the line segment EF and line segment EG are coupled to the joint E in addition to the line segment DE. It is assumed that each line segment can be freely turned and rotated at the joint portion like a ball bearing. Assume that the image taken from the front direction at the time when there is a “line segment joined by free joint” model having such a structure is (1) in the figure. The three-dimensional information of each line segment cannot be grasped from this one two-dimensional image. However, if there is an image of (2) in the figure taken from the side direction where the viewing angle for photographing this model is changed by 90 °, the three-dimensional information of each line segment can be grasped from these two pieces of image information. This will be understood as a matter of course.

A method for analyzing three-dimensional information from one two-dimensional image presented by the present invention will be described. As shown in FIG. 2, the subject in the three-dimensional space is imaged as a two-dimensional image on the imaging surface of the camera. Since the relationship between the spatial coordinates (X axis, Y axis, Z axis) and camera coordinates (U axis, V axis) at that time is simple, the X axis and the U axis are parallel to the Z axis and the V axis. If it is taken, it can be expressed by the following relationship.
X = a (Y) U, Z = b (Y) V (1)
Here, a (Y) and b (Y) are coefficients in the Y-axis direction, that is, the depth direction.
Assuming that a captured image of a line segment having a length l _k-m with markers M _m and M _k attached to both ends as shown in FIG. 3 is obtained, the camera coordinates (U axis, V axis) The corresponding spatial coordinate information (X axis, Y axis, Z axis) is calculated from the two-dimensional image. The two-dimensional information is the camera coordinate values (U _m , V _m ) and (U _k , V _k ) of the markers M _m and M _k . When these are used to calculate the X and Z values of the spatial coordinate system from the above relational expression, in general, calculation cannot be performed without depth information, but the length of the line segment is l _k−. When it is known that _m , the following relationship is established.
^{_{l k-m 2 = a (}} 0) 2 (U k -U m) 2 + b (0) 2 (V k -V m) 2 ‥‥‥‥‥ (2)
The coefficients of a (0) and b (0) are calculated from this equation. If there are multiple line segments of known length on the two-dimensional image, a more accurate count value can be obtained by the least square method. I can do it.

となり、解が２つ得られる。このことは所定長さの線分は一方の端部を基準とした場合他端の奥行き位置は基準点より手前側に来る場合と奥側に位置する場合の二つがあり得ることを意味している。実際の線分のＹ座標位置Ｙ_ｋはそのいずれか一方であるので、その選択が必要となるが、対象である線分が動物の体節であるとか動きの予測がつく機械要素であったりすれば、実際の画像からそれを特定することはさほど困難なことではない。また、線分が他の被写体の前にあるか後にあるかといった位置情報や、時系列的連続画像である場合などは動きの連続性の中から判断できるなど、実際の画像からそれを特定することが可能となる。
被写体モデルが図１に示したような節で結合された線分の集合体であるときは１つの関節位置を基準としてそれぞれの線分端部の位置を順次算出することが出来る。 Next, calculation of depth information will be described. Now, to calculate the depth information _{Y k} of the other end _{M k} positions of _{M m} is one end of a line segment of interest _{(X m, Y m, Z} m) as a reference. Because the relationship of the following equation (3) holds for the line segment

And two solutions are obtained. This means that when a line segment of a predetermined length is based on one end, the depth position of the other end can be two cases, that is, a position closer to the front than the reference point and a position closer to the back. Yes. Since the actual Y coordinate position Y _k of the line segment is one of them, it is necessary to select it. However, the target line segment is an animal body segment or a machine element that can predict movement. Then, it is not so difficult to identify it from the actual image. In addition, the position information such as whether the line segment is in front of or behind another subject, and if it is a time-series continuous image, it can be determined from the continuity of movement, etc. It becomes possible.
When the subject model is an aggregate of line segments connected by nodes as shown in FIG. 1, the position of each line segment end can be calculated sequentially with one joint position as a reference.

ここで、ΔＵｋ＝(Ｕ_ｋ,１−Ｕ_{ｋ＋１,１})であり、ΔＶｋ＝(Ｖ_ｋ,１−Ｖ_{ｋ＋１,１})であって、ｋは２以上の対象線分の数である。このａ(０)とｂ(０)は、本来的にｙの関数であるが、カメラが観察空間において固定され、カメラパラメータも不変の状態での使用を前提とすれば、結像位置は実空間座標において同じｙ値、この場合０の位置にあるものとして定数として扱うことが出来る。 Now, in order to analyze the movement of the body at the moment the athlete crosses the bar in a high jump competition, a digital camera was installed at a predetermined position with respect to the pole and the bar, and continuous pictures were taken before and after. First, select the athlete's shoulders, elbows, wrists, left and right ear tips, specific parts of the spine from the neck to the waist, hip joints, knees, ankles, shoe tips, etc. Click to obtain 2D position information on the image as the pixel position. A marker is placed on the clicked specific position so that it can be identified which part is indicated by the marker number i, and frame information j for identifying which frame is an image is also set as a point (U _{i , j} , V _{i, j} ). By this method, points are extracted from all continuous images.
Next, the length between the specific points of the player, that is, the body segment information corresponding to the line segment is acquired. The coefficients a (0) and b (0) are calculated by putting the known line segment information into the equation (2). In this case, use the one that can accurately obtain the dimensions such as poles and bars rather than using the body segment information. Is advantageous. In the imaging system, the imaging position is set to 0, and the first point (U _{i, 1} , V _{i, 1} ) and the second point (U _2,1 , V _2,1 ) in the first frame image are connected. If the length of the target line segment in the real space is l ₁ , the above equation (2) can be expressed as l ₁ ² = a (0) ² (U _1,1 −U _2,1 ) ² + b (0) ² (V _1,1 -V _2,1 ) ² (2) '
If there are a plurality of line segments, the same calculation is performed and a (0) and b (0) are calculated using the least square method.

Here, ΔUk = (U _{k, 1} −U _{k + 1,1} ), ΔVk = (V _{k, 1} −V _{k + 1,1} ), and k is the number of target line segments of 2 or more. These a (0) and b (0) are inherently functions of y. However, if the camera is fixed in the observation space and the camera parameters are assumed to be unchanged, the imaging position is actual. The same y value in spatial coordinates, in this case, it can be treated as a constant as being at position 0.

If the coefficients a (0) and b (0) are obtained, a certain point in the human body to be measured in the image, for example, the point on the right shoulder is set as the reference origin (0, 0) in the expression (3). Based on the calculation of equation (4), a solution is obtained. Since the relationship of the previous equation (3) holds, the solution of equation (4) is obtained. Among the two obtained solutions, a solution that seems to be appropriate as a human motion is artificially selected while observing the image information comprehensively. If the image cannot be selected from a single frame image, the possible data is accumulated together until it can be identified by the subsequent image. The position calculation of each point is sequentially performed, and three-dimensional information about all measurement points is accumulated.
For the second and lower frames, the information about the j images can be obtained by obtaining the three-dimensional position information of the measurement points for each image with the reference point and the selected right shoulder position as the origin by the same method.

  The data for the above-mentioned j images have the origin at the position of the specific point (in this embodiment, the right shoulder), but the specific point itself is not a fixed point in real space. As work, the original three-dimensional information as real space coordinates can be obtained by calculating the movement of this specific point and correcting the origin information of each image as a difference offset. In this calculation, two images with different imaging times are overlapped, the difference in the origin position is calculated in the same way as the line segment information, and the positional deviation in the Y-axis direction is calculated, and the origin position deviation between each image is calculated as a difference. Make corrections as offsets. Since these j images are images of the same setting at the same camera, the camera parameters are the same and can be calculated from position information on each image without actually overlapping the images. If the 3D information for each image is perfect and the positional relationship of a specific point, which is one point on each image, is known, in theory, this provides time-series 3D information for each point of the subject. Will be.

  However, when the calculation is actually performed in this process, the calculated position of the tip of the left shoe farthest from the right shoulder position, which is a specific point, has a large difference from the actual position. This is because the operation of specifying each point on the image is performed by manually marking the image displayed on the display by a person, so that position detection involves a slight error. Each point position information calculated with the reference point as the origin based on the position information contains an error, and each point position is calculated sequentially from the reference point. Will become larger. This error is greatly generated at the Y-axis direction position in the environment of the camera optical system. Therefore, in this embodiment, in order to guarantee the offset error in the Y-axis direction, a method of correcting the Y-direction position for each line segment is adopted.

  First, as a first step, the center positions of all connected line segments are calculated. As a second step, a Y-direction offset amount is calculated from the calculated amount of misalignment between frames at each center position. In the third step, correction of each point is executed by applying Y-direction correction for each line segment sequentially from the right shoulder origin position with the offset amount calculated for each line segment.

  In the previous embodiment, a method of calculating the center position of the line segment in the Y-direction offset amount between frames and calculating time-series continuity from the deviation of the point was adopted. A method is adopted in which an accelerometer is attached to such a point, a movement vector is calculated based on the acceleration information, and time-series continuity is calculated. However, even in this method, the information of only one accelerometer causes a problem that an error occurs in a line segment far from the accelerometer mounting position as in the Y direction calculation from only the previous origin position data. Therefore, in this embodiment, the accuracy is improved by attaching a plurality of accelerometers to remote points.

When analyzing human movements with the method of the present invention, if the four points of the left and right shoulder positions and the left and right hip bone positions are selected as the reference points, the position of the center of gravity and the orientation and twist of the body can be grasped. It was found that 3D analysis from 2D images can be performed very effectively.
In the above description, the analysis is performed based on the two-dimensional image captured for the purpose of analysis from the beginning. However, the present invention is not limited to this, and if line segment information of the subject can be obtained from images captured in the past. It is possible to perform analysis by the method of the present invention. Even from the recording film of famous players, analysis is possible if there is physique data. In this case, the position information can be obtained by digitizing an image and using an image ranging technique on the display. When there is no physique data, if the longest line segment data in a series of images is regarded as a direction parallel to the XZ plane, the body motion can be analyzed as appropriate.

The system that performs the information for analyzing the three-dimensional information of the subject from the two-dimensional image according to the present invention does not require a large-scale device. As shown in FIG. 4, if a digital camera and a recording medium storing a program for executing the method of the present invention are prepared, a system can be realized using a general-purpose personal computer. The camera outputs the captured two-dimensional image as information to a memory in the personal computer. This image can be monitored on a personal computer display. The subject line segment information and line segment selection information are input using a keyboard or mouse of a personal computer. On the two-dimensional image displayed on the display, the point at the end of the line segment is marked by clicking with the mouse, and the two-dimensional coordinate position of the point is read, and all the calculations performed in the present invention are executed. The software to be stored is stored in a recording medium. If this recording medium is set in a personal computer and installed in the personal computer, this recording medium is no longer necessary. Based on the line segment information on the two-dimensional image and the actual length information of the previously input line segment, the three-dimensional information of the line segment is calculated by the computing means of the personal computer according to the procedure of the method described above. Is done.
The analysis result is stored in a memory and displayed as a line segment image or is three-dimensional coordinate data. Therefore, images from different angles can be easily processed by a computer graphics technique. Further, the center of gravity position of the body can be identified from the four points of the left and right shoulder positions and the left and right hip bone positions described above, and the center of gravity movement in continuous motion can be observed.

  The present invention has mainly explained the analysis of athletes' movements. In addition, in the medical field, it is used as judgment data for body movement diagnosis and rehabilitation, and in the field of biology, animal movement observation, machine In the field, it can be applied to line segments in a wide range of fields, such as observation of the operating state of the mechanism.

It is a conceptual diagram of the connected line segment structure which this invention handles. It is a figure explaining a space coordinate system and a camera coordinate system. It is a figure explaining the relationship between the real space coordinate of a line segment structure, and a camera coordinate. It is a figure explaining the system which performs the method of this invention. It is a figure explaining the three-dimensional information analysis by the conventional stereo method. It is a figure explaining the conventional method of analyzing three-dimensional information from a two-dimensional image.

Explanation of symbols

A, B, C..., G line segment end point l ₁ , l ₂ ..., L _i line segment length M _k , M _m Mark point

Claims (10)

And a two-dimensional image and length information of the segment of the captured segments linked by free joints, the three-dimensional position of the line at the time of imaging, met method to calculate on the basis of the one connection position And
The three-dimensional spatial coordinates X-axis, Y-axis, and Z-axis are related to the two-dimensional camera coordinates U-axis and V-axis so that the X-axis and U-axis are parallel, and the Z-axis and V-axis are parallel. The camera coordinates (U _m , V _m ) with respect to the three-dimensional position (X _m , Y _m , Z _m ) of M _m in the relationship between the joint M _m having a length of 1 _km and the other end M _k While calculating the (X _k , Z _k ) coordinate value of M _k using the depth direction coefficient from the (U _k , V _k ) information ,
Y _k = ± {l _km ² − (X _k −X _m ) ² − (Z _k −Z _m ) ² } ^1/2 + Y _m
The depth information Y _k of the other end M _k of the line segment is calculated based on the following formula to obtain the three-dimensional position information (X _k , Y _k , Z _k ) of the other end M _k , and further using this position as a reference A method of sequentially calculating the three-dimensional position of each line segment during imaging.   Based on the two-dimensional images captured in time series by the same optical system, the three-dimensional information of the line segments in each image obtained by the method of claim 1 Calculates the offset amount in the optical axis direction of both images from the positional deviation information between the images, corrects the three-dimensional information of the line segment, obtains the position change information of each line segment, and moves the subject in three dimensions How to parse.   A step of obtaining three-dimensional information of a line segment in each image obtained by the method according to claim 1, based on a two-dimensional image captured in time series by the same optical system, and each line segment in each image A step of calculating a center point position, a step of calculating an offset amount in the optical axis direction of both images from a deviation between the two images of the center position for each line segment, and a connection based on the three-dimensional information of the line segment A step of three-dimensionally analyzing the motion of the subject, comprising: sequentially correcting the calculated offset amount from a unit to obtain position change information of each line segment.   An accelerometer is attached to the reference connecting portion for the three-dimensional information of the line segment in each image obtained by the method according to claim 1 based on a two-dimensional image captured in time series with the same optical system. The vector is calculated from the detected values of the accelerometer, the offset amount in the optical axis direction of both images is calculated, the three-dimensional information of the line segment is corrected, the position change information of each line segment is obtained, and the motion of the subject is calculated. A three-dimensional analysis method.   The three-dimensional object according to any one of claims 1 to 4, wherein the subject is an animal including a human, the joint is a joint, and the line segment is a limb body to analyze the posture and movement of the animal. analysis method.   The three-dimensional object according to claim 5, wherein the subject is a human and the orientation of the person and the twist of the body are analyzed by using the left and right shoulder tip positions and the left and right hip bone tip positions as specific points. analysis method. The line segment from the camera that outputs the captured two-dimensional image as information, means for inputting line segment information and line segment selection information of the subject, line segment information on the two-dimensional image, and length information of the line segment and an arithmetic means for calculating three-dimensional information of said arithmetic means spatial coordinates X-axis of the three-dimensional, Y-axis, Z-axis and two-dimensional camera coordinate U axis, X-axis and U-axis the relation between V axis but taken as Z-axis and V-axis are parallel, three-dimensional position of the M _m 1 single concatenated length in relation to the joint M _m and the other M _k is _{l km (X m, Y m} , With Z _m ) as a reference, the (X _k , Z _k ) coordinate value of M _k is calculated from the camera coordinate (U _m , V _m ) and (U _k , V _k ) information using the depth direction coefficient ,
Y _k = ± {l _km ² − (X _k −X _m ) ² − (Z _k −Z _m ) ² } ^1/2 + Y _m
The depth information Y _k of the other end M _k of the line segment is calculated based on the following formula to obtain the three-dimensional position information (X _k , Y _k , Z _k ) of the other end M _k , and further using this position as a reference A system for analyzing three-dimensional information of a subject from a two-dimensional image that sequentially calculates a three-dimensional position of each line segment at the time of imaging .   The two-dimensional image according to claim 7, comprising a memory for accumulating two-dimensional images taken in time series and storing analysis data, a display unit for displaying the two-dimensional image, and means for processing the analysis result. A system that analyzes 3D information of a subject from an image. The three-dimensional spatial coordinates X-axis, Y-axis, and Z-axis are related to the two-dimensional camera coordinates U-axis and V-axis so that the X-axis and U-axis are parallel, and the Z-axis and V-axis are parallel. The camera coordinates (U _m , V _m ) with respect to the three-dimensional position (X _m , Y _m , Z _m ) of M _m in the relationship between the joint M _m having a length of 1 _km and the other end M _k While calculating the (X _k , Z _k ) coordinate value of M _k using the depth direction coefficient from the (U _k , V _k ) information ,
Y _k = ± {l _km ² − (X _k −X _m ) ² − (Z _k −Z _m ) ² } ^1/2 + Y _m
The depth information Y _k of the other end M _k of the line segment is calculated based on the following formula to obtain the three-dimensional position information (X _k , Y _k , Z _k ) of the other end M _k , and further using this position as a reference The three-dimensional position of the line segment at the time of imaging is calculated from the line segment position information connected by a free joint on the two-dimensional image and the length information of the line segment , which sequentially calculates the three-dimensional position of each line segment at the time of imaging. A recording medium in which a program for calculating information is written in a computer-readable form.   A program that obtains position change information for each line segment from each two-dimensional image captured in time series, calculates the amount of offset in the optical axis direction between the images, corrects the position, and analyzes the motion of the subject in three dimensions The recording medium according to claim 9, to which is added.

Images