JP7578323B2 - Calculation device - Google Patents

Description

The present invention relates to a calculation device, a calculation method, and a recording medium.

It is known to perform gait analysis of users based on the data.

For example, Patent Document 1 is an example of a document that describes human gait analysis. Patent Document 1 describes a gait analysis device that has a data acquisition unit that acquires two types of image data from a depth sensor, a skeletal information creation unit that creates skeletal information based on the image data acquired by the data acquisition unit, a correction processing unit that corrects the skeletal information created by the skeletal information creation unit, and an analysis processing unit that analyzes the user's gait using the corrected skeletal information.

International Publication No. 2017/170832

One example of an index that may be desired to be calculated when performing gait analysis is the bending of the waist. However, for example, when performing analysis based on image data without using a depth sensor as described in Patent Document 1, it is difficult to calculate an index value corresponding to the bending of the waist because it is not possible to obtain information in the depth direction.

Therefore, the object of the present invention is to provide a judgment device, judgment method, and recording medium that solves the problem of the difficulty in calculating an index value corresponding to the bending of the waist based on image data.

In order to achieve this object, a calculation device according to one embodiment of the present disclosure includes:
a height calculation unit that calculates a height of a person in the image data based on the image data;
a bending calculation unit that acquires a hand length that is a length connecting the left and right hands of the person in the image data, and calculates an index value indicating bending of the waist of the person in the image data based on the acquired hand length and the height calculated by the height calculation unit;
The configuration has the following:

In addition, a calculation method according to another embodiment of the present disclosure includes:
An information processing device,
Calculating a height of a person in the image data based on the image data;
The system is configured to obtain a wrist length, which is the length connecting the left and right wrists of the person in the image data, and calculate an index value indicating the curvature of the waist of the person in the image data based on the obtained wrist length and the calculated height.

In addition, a recording medium according to another embodiment of the present disclosure includes:
In the information processing device,
Calculating a height of a person in the image data based on the image data;
This is a computer-readable recording medium having recorded thereon a program for implementing a process of acquiring a wrist length, which is the length connecting the left and right wrists of the person in the image data, and calculating an index value indicating the curvature of the waist of the person in the image data based on the acquired wrist length and the calculated height.

With each of the configurations described above, it is possible to calculate an index value corresponding to the bending of the waist based on image data.

FIG. 1 is a diagram illustrating a configuration example of a determination system according to a first embodiment of the present disclosure. 1 is a diagram for explaining an example of image data acquired in a determination system; 2 is a block diagram showing an example of the configuration of the determination device shown in FIG. 1 . FIG. 11 is a diagram showing an example of skeletal information. FIG. 11 is a diagram showing an example of ground contact determination information. 10A and 10B are diagrams for explaining examples of determination made by a ground contact determining unit; 11A and 11B are diagrams for explaining an example of determination by a ground contact timing determination unit. 10 is a flowchart showing an example of a process performed by a ground contact determination unit. 10 is a flowchart showing an example of a process performed by a ground contact timing determination unit. FIG. 11 is a diagram illustrating a configuration example of a calculation system according to a second embodiment of the present disclosure. FIG. 11 is a block diagram showing an example of the configuration of the calculation device shown in FIG. 10 . FIG. 13 is a diagram showing an example of hand length and height. 13 is a flowchart illustrating an example of the operation of the calculation device. FIG. 13 is a diagram illustrating a configuration example of a calculation system according to a third embodiment of the present disclosure. FIG. 15 is a block diagram showing an example of the configuration of the calculation device shown in FIG. 14 . FIG. 13 is a diagram showing an example of height. FIG. 1 is a diagram showing an example of a component. FIG. 13 is a diagram showing another example of a component. 13 is a flowchart illustrating an example of the operation of the calculation device. FIG. 13 is a diagram illustrating an example of a hardware configuration of a calculation device according to a fourth embodiment of the present disclosure. FIG. 2 is a block diagram showing an example of the configuration of a calculation device. FIG. 13 is a block diagram showing another example of the configuration of the calculation device.

[First embodiment]
A first embodiment of the present disclosure will be described with reference to Figs. 1 to 9. Fig. 1 is a diagram showing a configuration example of a determination system 100. Fig. 2 is a diagram for explaining an example of image data acquired in the determination system 100. Fig. 3 is a block diagram showing a configuration example of a determination device 300. Fig. 4 is a diagram showing an example of skeletal information 323. Fig. 5 is a diagram showing an example of ground contact determination information 324. Fig. 6 is a diagram for explaining an example of determination by the ground contact determination unit 333. Fig. 7 is a diagram for explaining an example of determination by the ground contact timing determination unit 334. Fig. 8 is a flowchart showing an example of processing by the ground contact determination unit 333. Fig. 9 is a flowchart showing an example of processing by the ground contact timing determination unit 334.

In the first embodiment of the present disclosure, a determination system 100 is described that determines the timing when the feet of a walking person touch the ground based on image data acquired using an imaging device such as a smartphone 200. The determination system 100 acquires time-series image data showing the person walking from the back of the screen to the front. As described below, the determination system 100 recognizes the position of the skeleton from each of the multiple image data. Then, the determination system 100 determines the timing when the feet touch the ground based on the degree of change in the recognized positions.

The timing of landing determined by the determination system 100 can be used, for example, to perform various gait analyses such as the angle of each joint at the time the foot touches the floor, indicators such as whether the toes are lifted from the floor, pitch, stride, and stride cycle. The timing of landing determined by the determination system 100 may be used for purposes other than those exemplified above.

Figure 1 shows an example configuration of a determination system 100. Referring to Figure 1, the determination system 100 has, for example, a smartphone 200 and a determination device 300. As shown in Figure 1, the smartphone 200 and the determination device 300 are connected to each other, for example, wirelessly or via a wire, so as to be able to communicate with each other.

The smartphone 200 functions as an imaging device that captures images of a person walking. The smartphone 200 may be a smartphone having general functions such as a camera function for acquiring image data, a touch panel for displaying the screen, and various sensors such as a GPS sensor and an acceleration sensor.

In this embodiment, as shown in FIG. 2, the smartphone 200 captures an image of a person walking in a forward-backward direction, such as from the back of the screen to the front. In other words, the smartphone 200 acquires time-series image data showing a person walking in a forward-backward direction. The smartphone 200 also transmits the acquired image data to the determination device 300. The smartphone 200 may associate information indicating the date and time when the image data was acquired with the image data, and transmit the associated data to the determination device 300.

The determination device 300 is an information processing device that determines the timing when the feet of a walking person touch the ground based on image data acquired by the smartphone 200. For example, the determination device 300 is a server device. The determination device 300 may be a single information processing device, or may be realized, for example, on the cloud.

Figure 3 shows an example of the configuration of the determination device 300. Referring to Figure 3, the determination device 300 has, as main components, for example, a communication I/F unit 310, a memory unit 320, and a calculation processing unit 330.

The communication I/F unit 310 is composed of a data communication circuit. The communication I/F unit 310 performs data communication with an external device, a smartphone 200, etc., connected via a communication line.

The storage unit 320 is a storage device such as a hard disk or memory. The storage unit 320 stores processing information and a program 326 required for various processes in the calculation processing unit 330. The program 326 is read into the calculation processing unit 330 and executed to realize various processing units. The program 326 is read in advance from an external device or recording medium via a data input/output function such as the communication I/F unit 310, and is stored in the storage unit 320. Main information stored in the storage unit 320 includes, for example, a trained model 321, image information 322, skeletal information 323, ground contact judgment information 324, and ground contact timing information 325.

The trained model 321 is a trained model used by the skeleton recognition unit 332 when performing skeleton recognition. The trained model 321 is generated in advance, for example, in an external device by performing machine learning using teacher data such as image data containing skeleton coordinates, and is acquired from an external device via the communication I/F unit 310 and stored in the memory unit 320. The trained model 321 may be generated using a known method. The trained model 321 may be updated by a re-learning process using additional teacher data.

The image information 322 includes time-series image data acquired by the camera of the smartphone 200. The image information 322 is generated and updated by the image acquisition unit 331 acquiring image data, for example.

For example, in the image information 322, identification information for identifying the image data is associated with the image data. The image information 322 may include information other than the above examples, such as information indicating the date and time when the smartphone 200 acquired the image data.

Skeleton information 323 includes information indicating the coordinates of each part of a person recognized by skeleton recognition unit 332. Skeleton information 323 includes, for example, information indicating the coordinates of each part of each piece of image data in a time series. For example, skeleton information 323 is generated and updated as a result of processing by skeleton recognition unit 332.

Figure 4 shows an example of skeletal information 323. Referring to Figure 4, in skeletal information 323, for example, identification information is associated with position information of each part. Here, the identification information is, for example, information indicating image data used when recognizing the skeletal information. The identification information may correspond to identification information for identifying image data in image information 322. In addition, the position information of each part includes information indicating the coordinates of each part in the image data, such as the position of the pelvis.

The parts included in the position information of each part correspond to the trained model 321. For example, in FIG. 4, the parts are pelvis, center of spine, . . ., right knee, left knee, . . ., right ankle, left ankle, . . .
are shown as examples. The position information of each part can include about 30 parts, such as the right shoulder, . . ., the left elbow, . . . (other parts may be included). In this embodiment, the position information of each part includes information indicating the coordinates of at least the right ankle and the left ankle. The parts included in the position information of each part may be other than those exemplified in FIG. 4 and the like.

The grounding judgment information 324 includes information indicating the result of the judgment made by the grounding judgment unit 333. For example, the grounding judgment information 324 is generated and updated as the result of the judgment made by the grounding judgment unit 333.

Figure 5 shows an example of ground contact judgment information 324. As shown in Figure 5, for example, in the ground contact judgment information 324, identification information is associated with a ground contact judgment result. Here, the identification information is information indicating the image data for which the ground contact judgment was performed, etc. The identification information may correspond to identification information for identifying the image data in the image information 322. Furthermore, the ground contact judgment result indicates the result of the ground contact judgment. For example, the ground contact judgment result includes "left foot" indicating the result of determining that the left foot is on the ground, "right foot" indicating the result of determining that the right foot is on the ground, "-" indicating that it has been determined that the foot is excluded from the ground contact judgment, and the like. The ground contact judgment result may indicate other than the above examples.

The ground contact timing information 325 includes information indicating the result of the determination by the ground contact timing determination unit 334. For example, the ground contact timing information 325 is generated and updated as the result of the determination by the ground contact timing determination unit 334.

For example, the ground contact timing information 325 includes information indicating the timing when the foot touched the ground. Here, the information indicating the timing when the foot touched the ground is, for example, at least one of identification information indicating image data that has been determined to have touched the ground, information indicating the acquisition date and time of the image data that has been determined to have touched the ground, and the like. The ground contact timing information 325 may be, for example, a correspondence between information indicating the foot that touched the ground and information indicating the timing when the foot touched the ground.

The calculation processing unit 330 has a calculation device such as a CPU and its peripheral circuits. The calculation processing unit 330 reads and executes the program 326 from the storage unit 320, thereby implementing various processing units by having the above hardware and the program 326 work together. The main processing units implemented by the calculation processing unit 330 include, for example, an image acquisition unit 331, a skeletal recognition unit 332, a ground contact determination unit 333, a ground contact timing determination unit 334, and an output unit 335.

The image acquisition unit 331 acquires image data acquired by the smartphone 200 from the smartphone 200 via the communication I/F unit 310. For example, the image acquisition unit 331 acquires time-series image data from the smartphone 200. In addition, the image acquisition unit 331 stores the acquired image data in the memory unit 320 as image information 322.

The skeleton recognition unit 332 uses the trained model 321 to recognize the skeleton of the person to be measured for walking posture in the image data. For example, the skeleton recognition unit 332 recognizes the skeleton for each piece of image data in time series. The skeleton recognition unit 332 may be configured to recognize the skeleton for image data extracted from the time series image data based on a predetermined condition. For example, the skeleton recognition unit 332 recognizes each part such as the upper part of the spine, right shoulder, left shoulder, right elbow, left elbow, right wrist, left wrist, right hand, left hand, etc. The skeleton recognition unit 332 also calculates the coordinates in the screen data of each recognized part. Then, the skeleton recognition unit 332 associates the recognition and calculation results with identification information for identifying the image data, etc., and stores them in the storage unit 320 as skeleton information 323. In this way, the skeleton recognition unit 332 obtains information indicating the position of each part of the person in the image data.

The parts recognized by the skeleton recognition unit 332 correspond to the trained model 321 (teaching data used when training the trained model 321). Therefore, the skeleton recognition unit 332 may recognize parts other than those exemplified above according to the trained model 321.

The ground contact determination unit 333 determines which foot is on the ground based on the skeleton of the person indicated by the skeleton information 323. For example, the ground contact determination unit 333 determines which of the right and left feet of a walking person is on the ground based on the change in the coordinates of the right and left ankles included in the skeleton information 323. For example, the ground contact determination unit 333 refers to the skeleton information 323 to obtain information indicating the position, such as the coordinates of the ankles. Then, when the change in the coordinates of the ankles satisfies a predetermined condition, the ground contact determination unit 333 determines that the foot with the ankle that satisfies the condition is on the ground. The ground contact determination unit 333 also stores information indicating the result of the determination in the storage unit 320 as ground contact determination information 324.

Figure 6 shows an example of changes in the Y coordinates (the vertical coordinates of the image data) of the right and left ankles. For example, in Figure 6, the X axis shows values according to time, and the Y axis shows the vertical coordinates (Y coordinates) in the image data. For example, in the case of Figure 6, the larger the Y axis value, the more the right ankle or left ankle is located toward the bottom of the image data (i.e., the closer the right ankle or left ankle is located to the front of the screen).

Generally, when a person walks, the following movements are repeated alternately: one of the right and left feet remains on the ground while the other moves forward, both the right and left feet touch the ground, one of the right and left feet moves forward while the other remains on the ground, both the right and left feet touch the ground. In other words, in the case of this embodiment, it is assumed that the following states are repeated: the Y coordinate of the right ankle is constant and the Y coordinate of the left ankle increases, the Y coordinate of the right ankle and the Y coordinate of the left ankle are constant, the Y coordinate of the left ankle is constant and the Y coordinate of the right ankle increases, and the Y coordinate of the right ankle and the Y coordinate of the left ankle are constant.

Therefore, the ground contact determination unit 333 determines which foot is on the ground based on the change in the Y coordinate of the right ankle and the left ankle. In other words, when the Y coordinate of the ankle can be evaluated as being constant, the ground contact determination unit 333 determines that the foot having the ankle is on the ground. Specifically, for example, the ground contact determination unit 333 compares the Y coordinate of the ankle in the image data to be determined with the Y coordinate of the ankle in the image data immediately preceding the target image data in the time series. Then, when the displacement (amount of change) from the Y coordinate of the ankle in the immediately preceding image data is equal to or less than a predetermined value, the ground contact determination unit 333 determines that the Y coordinate is constant and that the foot having the ankle with the constant Y coordinate is on the ground. The ground contact determination unit 333 may determine that the Y coordinate is constant by a method other than the above example. The range of displacement in which the ground contact determination unit 333 determines that the Y coordinate is constant may be set arbitrarily.

The ground contact determination unit 333 may be configured to determine that the foot other than the foot determined to be on the ground has started to move based on the Y coordinate of the ankle, and to exclude the foot from the ground contact determination. For example, in the case of walking, the right foot generally touches the ground, and then the left foot starts to move. The ground contact determination unit 333 may determine that the left foot has started to move when the Y coordinate of the left ankle starts to increase after the Y coordinate of the right ankle has become constant. Specifically, for example, the ground contact determination unit 333 can determine that the left foot has started to move and exclude the left ankle from the ground contact determination when the displacement of the Y coordinate of the left ankle from the previous image data in the time series exceeds the displacement of the Y coordinate of the right ankle from the previous image data after the Y coordinate of the right ankle has become constant, and can exclude the left foot from the ground contact determination.

In addition, generally, when a person walks, the new foot that touches the ground moves forward more than the foot that is already on the ground. That is, in the case of this embodiment, the Y coordinate of the ankle of the new foot that touches the ground is greater than the Y coordinate of the ankle of the foot that is already on the ground. Therefore, the ground contact determination unit 333 may be configured to determine whether or not the ankle is on the ground by focusing only on the ankle with the larger Y coordinate of the right or left ankle.

Furthermore, the ground contact determination unit 333 may be configured to refer to the coordinates of the knee in addition to the coordinates of the ankle when making a ground contact determination. For example, the ground contact determination unit 333 may be configured to determine that the foot to be determined to be grounded is excluded from the ground contact determination when a change in the coordinate of the knee satisfies a predetermined condition. Specifically, for example, the ground contact determination unit 333 may be configured to determine that the foot to be determined to be grounded is excluded from the ground contact determination when the displacement of the Y coordinate of the knee from the previous image data in the time series image data is 0 or less.

For example, as described above, the ground contact determination unit 333 performs ground contact determination based on the change in the Y coordinate of the ankle. Note that the ground contact determination unit 333 may be configured to perform ground contact determination based on the change in the skeleton other than those exemplified above, such as using the Y coordinate of the toe.

The ground contact timing determination unit 334 determines the timing when the foot touches the ground based on the foot placement state determined based on the person's skeleton indicated by the skeleton information 323. For example, the ground contact timing determination unit 334 determines the timing when the foot touches the ground based on a change in the foot touching the ground indicated by the ground contact determination information 324. The ground contact timing determination unit 334 then stores the determination result in the memory unit 320 as ground contact timing information 325.

For example, the ground contact timing determination unit 334 determines that the foot has touched the ground at the timing when the ground contact foot indicated by the ground contact determination result is switched to another foot. For example, referring to FIG. 7, the "left foot" is on the ground at identification information "124" and "125", and is excluded from ground contact determination at identification information "126" and "127". Then, it is determined that the "right foot" is on the ground at identification information "128". That is, in the case of FIG. 7, the foot on the ground is switched from the left foot to the right foot at the timing of identification information "128". Therefore, the ground contact timing determination unit 334 determines that the right foot has touched the ground at the timing of identification information "128".

The output unit 335 outputs the ground contact timing information 325, the ground contact judgment information 324, etc. For example, the output unit 335 can output at least one of the above information to an external device, the smartphone 200, etc.

The above is an example of the configuration of the determination device 300. Next, the operation of the determination device 300 will be explained with reference to Figures 8 and 9.

First, an example of the operation of the ground contact determination unit 333 will be described with reference to Fig. 8. Referring to Fig. 8, the ground contact determination unit 333 compares the Y coordinate of the knee in the image data to be determined with the Y coordinate of the knee in the image data immediately preceding the object of determination in the time series image data (step S101). If the displacement from the Y coordinate of the knee in the immediately preceding image data exceeds 0 (step S101, Yes), the ground contact determination unit 333 checks whether the displacement of the ankle satisfies a condition (step S102).

If the ankle displacement satisfies the condition (step S102, Yes), the ground contact determination unit 333 determines that the foot that satisfies the condition is on the ground (step S103). On the other hand, if the displacement of the Y coordinate of the knee is 0 or less (step S101, No) or if the ankle displacement does not satisfy the condition (step S102, No), the ground contact determination unit 333 determines that the foot that does not satisfy the condition is not on the ground.

The above is an example of the operation of the ground contact determination unit 333. The ground contact determination unit 333 can determine that the displacement of the ankle satisfies the condition, for example, when the Y coordinate of the ankle can be evaluated as constant based on the change in the Y coordinate, and when it is not determined that the foot other than the foot determined to be on the ground has started to move, or when the change in knee coordinate satisfies the condition. The ground contact determination unit 333 may also be configured to identify an ankle that focuses on the Y coordinate of the ankle, and perform the above-mentioned confirmation on the identified ankle.

Next, an example of the operation of the ground contact timing determination unit 334 will be described with reference to Fig. 9. Referring to Fig. 9, the ground contact timing determination unit 334 checks whether the ground contact foot indicated by the ground contact determination result has changed by referring to the ground contact timing information 325 (step S201).

If the foot has changed (step S201, Yes), the ground contact timing determination unit 334 determines that the changed timing is the ground contact timing (step S202). On the other hand, if the foot on the ground indicated by the determination result has not changed (step S201), the ground contact timing determination unit 334 does not perform the above determination.

The above is an example of processing by the ground contact timing determination unit 334.

In this way, the determination device 300 has a skeleton recognition unit 332, a ground contact determination unit 333, and a ground contact timing determination unit 334. With this configuration, the ground contact timing determination unit 334 can determine the timing at which the foot touches the ground based on the determination result by the ground contact determination unit 333 using the coordinates of the skeleton recognized by the skeleton recognition unit 332. In other words, with the above configuration, it becomes possible to grasp the timing at which the foot touches the ground based on image data.

The determination device 300 may be configured to determine the ground contact timing based on the results of skeletal recognition by an external device that performs skeletal recognition. When configured in this manner, the determination device 300 does not need to have the function of the skeletal recognition unit 332.

Second Embodiment
Next, a second embodiment of the present disclosure will be described with reference to Fig. 10 to Fig. 13. Fig. 10 is a diagram showing a configuration example of a calculation system 400. Fig. 11 is a block diagram showing a configuration example of a calculation device 500. Fig. 12 is a diagram showing an example of effort length and height. Fig. 13 is a flowchart showing an operation example of the calculation device 500.

In a second embodiment of the present disclosure, a calculation system 400 is described that calculates an index value indicating the bending of the waist of a person in image data based on image data acquired using an imaging device such as a smartphone 200. As in the first embodiment, the calculation system 400 acquires image data showing a person walking from the back of the screen to the front, and recognizes the position of the skeleton from the image data. Then, the calculation system 400 calculates an index value indicating the bending of the waist based on the recognition result.

As described later, the calculation system 400 described in this embodiment acquires the hand length, which is the sum of the in-image distances of the skeletal lengths connecting each part present in the left and right hands, and the height, which is the in-image distance between the head and the toes. Then, the calculation system 400 calculates an index value indicating the bending of the waist based on the acquired hand length and height. If both the hand length and the height are assumed to be height, the height changes as the waist is bent, while the hand length does not change even if the waist is bent. In addition, the length of both hands when spread out to the left and right corresponds to the height. The calculation system 400 calculates an index value indicating the bending of the waist by using the above. It is known that the ratio of the hand length to the height changes depending on the age, height, etc. Therefore, the calculation system 400 may be configured to add a predetermined correction value to the hand length according to the attributes of the person such as age, height, and sex, and then calculate an index value indicating the bending of the waist based on the corrected hand length and height.

The calculation system 400 described in this embodiment can function as the determination system 100 described in the first embodiment. In other words, the calculation system 400 may have a configuration for determining the ground contact timing.

Figure 10 shows an example configuration of the calculation system 400. Referring to Figure 10, the calculation system 400 has, for example, a smartphone 200 and a calculation device 500. As shown in Figure 10, the smartphone 200 and the calculation device 500 are connected to each other, for example, wirelessly or via a wire, so as to be able to communicate with each other.

The configuration of the smartphone 200 is the same as that of the first embodiment. Therefore, a description of the smartphone 200 is omitted.

The calculation device 500 is an information processing device that calculates an index value indicating the bending of the waist based on image data acquired by the smartphone 200. For example, the calculation device 500 is a server device. The calculation device 500 may be a single information processing device, or may be realized, for example, on the cloud.

Figure 11 shows an example configuration of the calculation device 500. Referring to Figure 11, the calculation device 500 has, as main components, for example, a communication I/F unit 510, a memory unit 520, and a calculation processing unit 530.

The communication I/F unit 510 is composed of a data communication circuit. The communication I/F unit 510 performs data communication with an external device, a smartphone 200, etc., connected via a communication line.

The storage unit 520 is a storage device such as a hard disk or memory. The storage unit 520 stores processing information and a program 527 required for various processes in the calculation processing unit 530. The program 527 is read into the calculation processing unit 530 and executed to realize various processing units. The program 527 is read in advance from an external device or recording medium via a data input/output function such as the communication I/F unit 510, and is stored in the storage unit 520. Main information stored in the storage unit 520 includes, for example, a trained model 521, image information 522, skeletal information 523, hand length information 524, height information 525, and waist bending value information 526.

The trained model 521, image information 522, and skeletal information 523 are similar to the trained model 321, image information 322, and skeletal information 323 described in the first embodiment.

The effort length information 524 indicates the effort length, which is the sum of the in-image distances of the skeletal lengths connecting the parts between the left and right hands, such as the palms, wrists, elbows, and shoulders. The effort length information 524 is calculated in advance, for example, based on image data acquired with the hand extended downward so that the effort length is the longest, and is acquired from an external device via the communication I/F unit 510 and stored in the memory unit 520. The effort length information 524 may be generated or updated as a result of a calculation process by the effort length calculation unit 533.

For example, the effort length information 524 includes information indicating the effort length. The effort length information 524 may include information indicating the effort length obtained for each time-series image data. For example, the effort length information 524 may be information that associates identification information indicating the image data used to calculate the effort length with information indicating the effort length.

Height information 525 indicates the height of a person in the image data, which is the in-image distance between the head (forehead) and the toes (e.g., the lower of the left and right). Height information 525 is generated and updated, for example, as a result of a calculation process by height calculation unit 534.

For example, height information 525 includes information indicating the height obtained for each time-series image data. Height information 525 may be, for example, a combination of identification information indicating the image data used to calculate the height and information indicating the height. Height information 525 may also be, for example, a combination of identification information indicating image data that satisfies a predetermined condition and information indicating the height.

The waist bending value information 526 indicates a waist bending value, which is an index value indicating the bending of the waist. The waist bending value information 526 is generated and updated, for example, as a result of a calculation process by the bending calculation unit 535.

For example, waist flexion value information 526 includes information indicating the waist flexion value. Waist flexion value information 526 may also include information indicating the waist flexion value acquired for each time-series image data. Waist flexion value information 526 may be, for example, a combination of identification information indicating image data used to calculate the waist flexion value and information indicating the waist flexion value. Waist flexion value information 526 may also be, for example, a combination of identification information indicating image data that satisfies a predetermined condition and information indicating the waist flexion value.

The calculation processing unit 530 has a calculation device such as a CPU and its peripheral circuits. The calculation processing unit 530 reads and executes the program 527 from the storage unit 520, thereby implementing various processing units by having the above hardware and the program 527 work together. The main processing units implemented by the calculation processing unit 530 include, for example, an image acquisition unit 531, a skeleton recognition unit 532, a hand length calculation unit 533, a height calculation unit 534, a bend calculation unit 535, and an output unit 536.

The image acquisition unit 531 and the skeleton recognition unit 532 can perform processing similar to the image acquisition unit 331 and the skeleton recognition unit 332 described in the first embodiment.

The effort length calculation unit 533 calculates the effort length, which is the sum of the in-image distances of the skeleton lengths connecting the left and right efforts, based on the skeleton information 523. Then, the effort length calculation unit 533 stores the calculated effort length in the memory unit 520 as effort length information 524.

For example, FIG. 12 shows an example of effort length. Referring to FIG. 12, the effort length calculation unit 533 refers to the skeletal information 523 to acquire information indicating the position of each part between the left and right hands, such as the palm, wrist, elbow, and shoulder. Then, based on the acquired information, the effort length calculation unit 533 calculates the sum of the distances between each part when adjacent parts are connected by a straight line. For example, as described above, the effort length calculation unit 533 calculates the effort length based on the position of each part between the left and right hands.

The effort length calculation unit 533 can perform the above-mentioned effort length calculation process for each image data in the time series. In other words, the effort length calculation unit 533 may be configured to calculate the effort length for each image data. The effort length calculation unit 533 may be configured to calculate the effort length for image data extracted under any condition from the image data in the time series.

The effort length calculation unit 533 may calculate the effort length using parts other than those exemplified above. Information indicating the effort length may be calculated and acquired in advance. When information indicating the effort length is acquired in advance, the calculation device 500 does not need to have the effort length calculation unit 533.

The height calculation unit 534 calculates the height, which is the distance in the image from the head to the toes, based on the skeletal information 523. The height calculation unit 534 then stores the calculated height in the memory unit 520 as height information 525.

For example, FIG. 12 shows an example of height. Referring to FIG. 12, the height calculation unit 534 acquires information indicating the positions of the forehead, which is the highest part, and the left and right toes by referring to the skeletal information 523. Then, based on the acquired information, the height calculation unit 534 calculates the distance between the forehead and the lower of the left and right toes in a straight line. Alternatively, based on the acquired information, the height calculation unit 534 calculates the difference in the Y-axis direction (the up and down direction in FIG. 12) between the forehead and the lower of the left and right toes. For example, as described above, the height calculation unit 534 calculates the height based on the positions of the head and feet.

The height calculation unit 534, like the effort length calculation unit 533, can perform the above-mentioned height calculation process for each image data in the time series. In other words, the height calculation unit 534 may be configured to calculate the height in the image data for each image data. The height calculation unit 534 may be configured to calculate the height in image data extracted under any condition from the image data in the time series.

The height calculation unit 534 may calculate the height using a part other than the above examples. For example, the height calculation unit 534 may calculate the length between parts including the waist between the neck and the ankle as the height.

The bending calculation unit 535 calculates a waist bending value, which is an index value indicating the bending of the waist, based on the hand length indicated by the hand length information 524 and the height indicated by the height information 525. The bending calculation unit 535 then stores the calculated waist bending value in the memory unit 520 as waist bending value information 526.

For example, the bend calculation unit 535 calculates the waist bend value by dividing the hand length by the height. In other words, the bend calculation unit 535 calculates the waist bend value by calculating (hand length)/(height). For example, the bend calculation unit 535 can calculate the waist bend value using the hand length and height calculated for each image data.

It is assumed that the effort length is shortened by waving the hands while walking. Therefore, the bending calculation unit 535 may calculate the bending value of the waist using an effort length selected from the effort length information 524 according to a predetermined criterion, such as using the longest effort length among the effort lengths included in the effort length information 524. In other words, the effort length used by the bending calculation unit 535 when calculating the bending value of the waist may be an effort length selected according to a predetermined criterion among the effort lengths included in the effort length information 524. The selection criterion may be set arbitrarily, such as using the longest one. In addition, the effort length information 524 may include only one effort length acquired in advance. Therefore, the effort length used by the bending calculation unit 535 when calculating the bending value of the waist may be, for example, a constant value.

The bending calculation unit 535 may also use the result of the determination of the landing timing described in the first embodiment to identify image data that is the target of the calculation of the waist bending value. For example, the bending calculation unit 535 may be configured to calculate the waist bending value using the height acquired based on the image data determined to be the landing timing. The bending calculation unit 535 may also be configured to calculate the waist bending value using the height acquired based on a predetermined number of frames of image data from the image data determined to be the landing timing. In this way, by configuring the bending value of the waist to be calculated based on the image data specified based on the determination result of the landing timing (based on the height), it is possible to calculate a more appropriate waist bending value as an index to be compared with other people. The bending calculation unit 535 may also be configured to calculate the waist bending value for each image data in the time series and to calculate the average value of the calculated waist bending values.

The output unit 536 outputs hand length information 524, height information 525, waist bending value information 526, etc. For example, the output unit 536 can output at least one of the above pieces of information to an external device, a smartphone 200, etc.

The above is an example configuration of the calculation device 500. Next, an example operation of the calculation device 500 will be described with reference to FIG. 13.

Referring to Figure 13, the bend calculation unit 535 obtains the hand length indicated by the hand length information 524 and the height indicated by the height information 525 (step S301).

The bending calculation unit 535 calculates a bending value of the waist, which is an index value indicating bending of the waist, based on the hand length and the height (step S302). For example, the bending calculation unit 535 calculates the bending value of the waist by dividing the hand length by the height.

The above is an example of the operation of the calculation device 500.

In this way, the calculation device 500 has a bending calculation unit 535. With this configuration, the calculation device 500 can calculate the bending value of the waist based on the acquired hand length and height. In other words, with the above configuration, an index value corresponding to the bending of the waist can be calculated based on the image data.

The calculation device 500 can have the function of the determination device 300 described in the first embodiment. The calculation device 500 can also have the same modified example as the determination device 300 described in the first embodiment.

[Third embodiment]
Next, a third embodiment of the present disclosure will be described with reference to Fig. 14 to Fig. 19. Fig. 14 is a diagram showing a configuration example of a calculation system 600. Fig. 15 is a block diagram showing a configuration example of a calculation device 700. Fig. 16 is a diagram showing an example of height. Figs. 17 and 18 are diagrams showing examples of foot components. Fig. 19 is a flowchart showing an operation example of the calculation device 700.

In the third embodiment of the present disclosure, a calculation system 600 is described that calculates an index value indicating the state of an object of interest based on image data acquired using an imaging device such as a smartphone 200. As in the first and second embodiments, the calculation system 600 acquires image data showing a person walking from the back of the screen to the front, and recognizes the position of the skeleton from the image data. Then, the calculation system 600 calculates an index value indicating the state of the object based on the recognition result.

For example, the calculation system 600 described in this embodiment focuses on the foot and calculates an index value indicating the state of the toes, such as the rise and direction of the toes. Specifically, for example, the calculation system 600 acquires the difference in the X-axis direction and the difference in the Y-axis direction between the positions of the ankle and the toes, which are parts present in the foot of interest, and the comparison length to be compared. Then, the calculation system 600 calculates an index value indicating the direction of the toes based on the difference in the X-axis direction and the comparison length, and calculates an index value indicating the rise of the toes based on the difference in the Y-axis direction and the comparison length. The closer a person is to the camera, which is an imaging device, the larger the person is captured. Therefore, even if the toes are raised in the same way or point in the same direction, the length in the image data differs depending on the proximity of the person to the imaging device. The calculation system 600 makes it possible to calculate an index value that solves the above problem by comparing the difference in the X-axis direction and the difference in the Y-axis direction between the positions of the ankle and the toes with the comparison length, which is the length to be compared.

In this embodiment, the foot is taken as an example of the object of interest. However, the object of interest may be any part other than the foot. In other words, the calculation system 600 may use the difference in the X-axis direction or the Y-axis direction between the positions of parts present in the object of interest other than the ankle and the toes. The comparison length refers to the length between parts whose length in the image data varies depending on the proximity of the person to the camera, which is the imaging device, when the image data is acquired, but whose length is constant for the person. The comparison length can be, for example, one of the height (not taking into account the bending of the waist) which is the distance in the image from the head to the toes described in the second embodiment, the hand length, the height which is the distance in the image from the head to the pelvis, and the like. The comparison length may be the length between parts other than those exemplified above.

The calculation system 600 described in this embodiment can have the functions of the determination system 100 described in the first embodiment and the calculation system 400 described in the second embodiment. In other words, the calculation system 600 may have a configuration for determining the ground contact timing and a configuration for calculating the hip bending value.

Figure 14 shows an example configuration of a calculation system 600. Referring to Figure 14, the calculation system 600 has, for example, a smartphone 200 and a calculation device 700. As shown in Figure 14, the smartphone 200 and the calculation device 700 are connected to each other, for example, wirelessly or by wire, so as to be able to communicate with each other.

The configuration of the smartphone 200 is the same as that of the first embodiment. Therefore, a description of the smartphone 200 is omitted.

The calculation device 700 is an information processing device that calculates index values indicating the lift of the toes and the direction of the toes based on image data acquired by the smartphone 200. For example, the calculation device 700 is a server device. The calculation device 700 may be a single information processing device, or may be realized, for example, on the cloud. Note that the calculation device 700 may calculate only one of the index value indicating the lift of the toes and the index value indicating the direction of the toes.

Figure 15 shows an example configuration of the calculation device 700. Referring to Figure 15, the calculation device 700 has, as main components, for example, a communication I/F unit 710, a memory unit 720, and an arithmetic processing unit 730.

The communication I/F unit 710 is composed of a data communication circuit. The communication I/F unit 710 performs data communication with an external device, a smartphone 200, etc., connected via a communication line.

The storage unit 720 is a storage device such as a hard disk or memory. The storage unit 720 stores processing information and a program 727 necessary for various processes in the calculation processing unit 730. The program 727 is read into the calculation processing unit 730 and executed to realize various processing units. The program 727 is read in advance from an external device or recording medium via a data input/output function such as the communication I/F unit 710, and is stored in the storage unit 720. Main information stored in the storage unit 720 includes, for example, a trained model 721, image information 722, skeletal information 723, comparative length information 724, foot component information 725, and toe data information 726.

The trained model 721, image information 722, and skeletal information 723 are similar to the trained model 321, image information 322, and skeletal information 323 described in the first embodiment.

Comparative length information 724 indicates a comparative length, which is a length between parts of a person whose length is constant for that person, although the length in the image data varies depending on the proximity of the person to the camera, which is an imaging device, when the image data is acquired. For example, comparative length information 724 indicates one of a height, which is the distance in the image from the head to the toes, a hand length, a second height, which is the distance in the image from the head to the pelvis, and the like. Comparative length information 724 is generated/updated, for example, as a result of a calculation process by comparative length calculation unit 733.

For example, the comparison length information 724 includes information indicating the comparison length obtained for each time-series image data. For example, the comparison length information 724 may be information that associates identification information indicating the image data used to calculate the comparison length with information indicating the comparison length.

Foot component information 725 indicates inter-part lengths such as the X component, which is the difference in the X-axis direction between the positions of the ankle and the toes, and the Y component, which is the difference in the Y-axis direction between the positions of the ankle and the toes. Foot component information 725 is generated and updated, for example, as a result of a calculation process by foot component calculation unit 734. Note that foot component information 725 may include only one of the X component and the Y component.

For example, the foot component information 725 includes information indicating the X component and the Y component obtained for each time-series image data. For example, the comparison length information 724 may be information that associates identification information indicating the image data used to calculate the X component and the Y component with information indicating the X component and the Y component.

The toe data information 726 indicates an index value indicating the lift of the toes and an index value indicating the direction of the toes. The toe data information 726 is generated and updated, for example, as a result of a calculation process by the toe data calculation unit 735.

For example, the toe data information 726 includes an index value indicating the rise of the toes and an index value indicating the direction of the toes. The toe data information 726 includes an index value indicating the rise of the toes and an index value indicating the direction of the toes acquired for each time-series image data. The height information 525 may be, for example, a correspondence between identification information indicating the image data used to calculate the index value and information indicating an index value indicating the rise of the toes and an index value indicating the direction of the toes. The toe data information 726 may also be, for example, a correspondence between identification information indicating image data that satisfies a predetermined condition and information indicating an index value indicating the rise of the toes and an index value indicating the direction of the toes.

The calculation processing unit 730 has a calculation device such as a CPU and its peripheral circuits. The calculation processing unit 730 reads and executes the program 727 from the storage unit 720, thereby implementing various processing units by having the above hardware and the program 727 work together. The main processing units implemented by the calculation processing unit 730 include, for example, an image acquisition unit 731, a skeleton recognition unit 732, a comparison length calculation unit 733, a foot component calculation unit 734, a toe data calculation unit 735, and an output unit 736.

The image acquisition unit 731 and the skeleton recognition unit 732 can perform processing similar to the image acquisition unit 331 and the skeleton recognition unit 332 described in the first embodiment.

The comparative length calculation unit 733 calculates a comparative length, which is one of the following: height, which is the in-image distance from the head to the toes, hand length, second height, which is the in-image distance from the head to the pelvis, etc., based on the skeletal information 723. Then, the comparative length calculation unit 733 stores the calculated comparative length in the storage unit 720 as comparative length information 724.

For example, FIG. 16 shows an example of height, which is an example of the comparative length. Referring to FIG. 16, the comparative length calculation unit 733 acquires information indicating the positions of the forehead, which is the highest part, and the left and right toes by referring to the skeletal information 523. Then, the comparative length calculation unit 733 calculates the distance between the forehead and the lower of the left and right toes with a straight line based on the acquired information. Alternatively, the comparative length calculation unit 733 calculates the difference in the Y-axis direction (up and down direction in FIG. 12) between the forehead and the lower of the left and right toes based on the acquired information. For example, as described above, the comparative length calculation unit 733 calculates the height based on the positions of the head and the feet. Note that the comparative length calculation unit 733 may be configured to calculate the hand length or the second height, which is the distance in the image from the head to the pelvis, as the comparative length by similar processing.

The foot component calculation unit 734 calculates at least one of the lengths between parts, such as the X component, which is the difference in the X-axis direction between the positions of the ankle and the toes, and the Y component, which is the difference in the Y-axis direction between the positions of the ankle and the toes, by referring to the skeletal information 523. The foot component calculation unit 734 then stores the calculated foot components in the storage unit 720 as foot component information 725.

For example, FIG. 17 shows an example of the Y component. Referring to FIG. 17, the foot component calculation unit 734 acquires information indicating the positions of the ankle and toes by referring to the skeletal information 523. Then, based on the acquired information, the foot component calculation unit 734 calculates the difference in the Y-axis direction (the up and down direction in FIG. 17) between the ankle and toes. For example, as described above, the foot component calculation unit 734 calculates the Y component based on the positions of the ankle and toes.

Figure 18 also shows an example of the X component. With reference to Figure 18, the foot component calculation unit 734 acquires information indicating the positions of the ankle and toes by referring to the skeletal information 523. Then, based on the acquired information, the foot component calculation unit 734 calculates the difference in the X-axis direction (left and right direction in Figure 18) between the ankle and toes. For example, as described above, the foot component calculation unit 734 calculates the X component based on the positions of the ankle and toes.

The foot component calculation unit 734 may be configured to calculate the Y and X components of each of the left and right feet, for example, or may be configured to calculate the Y and X components of only the lower foot of the left and right feet. The foot component calculation unit 734 may be configured to determine whether or not to calculate the foot components using the result of the contact timing determination described in the first embodiment. For example, the foot component calculation unit 734 can be configured to calculate the Y and X components based on image data determined to be contact timing. The foot component calculation unit 734 may be configured to calculate the Y and X components based on a predetermined number of frames of image data from the image data determined to be contact timing.

The toe data calculation unit 735 calculates an index value indicating the rise of the toes based on the comparison length indicated by the comparison length 724 and the Y component indicated by the foot component information 725. The toe data calculation unit 735 also calculates an index value indicating the direction of the toes based on the comparison length indicated by the comparison length 724 and the X component indicated by the foot component information 725. The toe data calculation unit 735 then stores the calculated index value indicating the rise of the toes and the index value indicating the direction of the toes in the storage unit 720 as toe data information 726.

For example, the toe data calculation unit 735 calculates an index value indicating the rise of the toes by dividing the Y component by the comparison length. In other words, the toe data calculation unit 735 calculates an index value indicating the rise of the toes by calculating (Y component)/(comparison length). For example, the toe data calculation unit 735 can calculate an index value indicating the rise of the toes using the Y component calculated for each image data and the comparison length.

For example, the toe data calculation unit 735 calculates an index value indicating the orientation of the toes by dividing the X component by the comparison length. In other words, the toe data calculation unit 735 calculates an index value indicating the orientation of the toes by calculating (X component)/(comparison length). For example, the toe data calculation unit 735 can calculate an index value indicating the orientation of the toes by using the X component calculated for each image data and the comparison length.

The toe data calculation unit 735 may use the result of the determination of the ground contact timing described in the first embodiment to specify image data to be used for calculating the index value indicating the toe lift and the index value indicating the toe direction. For example, the toe data calculation unit 735 may be configured to calculate the index value indicating the toe lift and the index value indicating the toe direction using the Y component or X component and the comparison length acquired based on the image data determined to be the ground contact timing. The toe data calculation unit 735 may be configured to calculate the index value indicating the toe lift and the index value indicating the toe direction using the Y component or X component and the comparison length acquired based on a predetermined number of frames of image data from the image data determined to be the ground contact timing. In this way, by performing calculations based on the image data specified based on the result of the determination of the ground contact timing, it becomes possible to evaluate the toe lift and the toe direction when the foot touches the ground.

The output unit 736 outputs the comparison length information 724, the foot component information 725, the toe data information 726, etc. For example, the output unit 736 can output at least one of the above information to an external device, the smartphone 200, etc.

The above is an example configuration of the calculation device 700. Next, an example operation of the calculation device 700 will be described with reference to FIG. 19.

Referring to Figure 19, the toe data calculation unit 735 obtains the comparison length indicated by the comparison length information and the Y component and X component indicated by the foot component information 725 (step S401).

The toe data calculation unit 735 calculates toe data based on the comparison length and the Y component and the X component. For example, the toe data calculation unit 735 calculates an index value indicating the lift of the toes based on the comparison length and the Y component. Also, for example, the toe data calculation unit 735 calculates an index value indicating the direction of the toes based on the comparison length and the X component.

The above is an example of the operation of the calculation device 700.

In this way, the calculation device 700 has a toe data calculation unit 735. With this configuration, the calculation device 700 can calculate an index value indicating the lift of the toes and an index value indicating the direction of the toes based on the acquired comparison length and the Y component and X component. In other words, with the above configuration, it is possible to calculate an index value indicating the state of the lift and direction of the toes based on image data.

In addition, when the object of interest is any part other than the foot, the calculation device 700 may be configured to calculate the length between parts of the object of interest other than the ankle and the toes as the Y component and the X component.

The calculation device 700 can have the function of the determination device 300 described in the first embodiment. The calculation device 700 can have the same modified example as the determination device 300 described in the first embodiment. Similarly, the calculation device 700 can have the function of the calculation device 500 described in the second embodiment. The calculation device 700 can have the same modified example as the calculation device 500 described in the second embodiment.

[Fourth embodiment]
Next, a fourth embodiment of the present disclosure will be described with reference to Fig. 20 to Fig. 22. In the fourth embodiment of the present disclosure, an overview of the configuration of a calculation device 800 that is an information processing device will be described.

Fig. 20 shows an example of a hardware configuration of the calculation device 800. Referring to Fig. 20, the calculation device 800 has, as an example, the following hardware configuration.
・CPU (Central Processing Unit) 801 (arithmetic unit)
ROM (Read Only Memory) 802 (storage device)
RAM (Random Access Memory) 803 (storage device)
Program group 804 loaded into RAM 803
A storage device 805 for storing the programs 804
A drive device 806 that reads and writes data from and to a recording medium 810 outside the information processing device.
A communication interface 807 that connects to a communication network 811 outside the information processing device
An input/output interface 808 for inputting and outputting data
A bus 809 connecting each component

In addition, the calculation device 800 can realize the functions of the height calculation unit 821 and the curvature calculation unit 822 shown in FIG. 21 by the CPU 801 acquiring the program group 804 and executing it. The program group 804 is, for example, stored in advance in the storage device 805 or ROM 802, and is loaded and executed by the CPU 801 into the RAM 803, etc., as necessary. The program group 804 may be supplied to the CPU 8801 via the communication network 811, or may be stored in advance on the recording medium 810, and the drive device 806 may read out the programs and supply them to the CPU 801.

Note that FIG. 20 shows an example of the hardware configuration of the computing device 800. The hardware configuration of the computing device 800 is not limited to the above-described case. For example, the computing device 800 may be configured with a part of the above-described configuration, such as not having the drive device 806.

The height calculation unit 821 calculates the height of a person in the image data based on the image data.

The bending calculation unit 822 acquires a hand length, which is the length connecting the left and right hands of the person in the image data. The bending calculation unit 822 also calculates an index value indicating the bending of the waist of the person in the image data based on the acquired hand length and the height calculated by the height calculation unit 821.

Thus, the calculation device 800 has a height calculation unit 821 and a bend calculation unit 822. The bend calculation unit 822 is configured to acquire a hand length. With this configuration, the bend calculation unit 822 can calculate an index value indicating the bending of the waist of the person in the image data based on the acquired hand length and the height calculated by the height calculation unit 821. As a result, it becomes possible to calculate an index value corresponding to the bending of the waist based on the image data.

An information processing device such as the calculation device 800 described above can be realized by incorporating a predetermined program into the information processing device. Specifically, a program that is another form of the present invention is a program for implementing a process in an information processing device that calculates the height of a person in image data based on the image data, obtains a wrist length that is the length connecting the left and right wrists of the person in the image data, and calculates an index value that indicates the bending of the waist of the person in the image data based on the obtained wrist length and the calculated height.

In addition, the calculation method executed by the above-mentioned information processing device is such that the information processing device calculates the height of the person in the image data based on the image data, obtains a wrist length which is the length connecting the left and right wrists of the person in the image data, and calculates an index value indicating the curvature of the waist of the person in the image data based on the obtained wrist length and the calculated height.

Even if the invention is a program (or recording medium) or a calculation method having the above-mentioned configuration, it has the same action and effect as the above-mentioned case, and therefore can achieve the same purpose as the calculation device 800 having the above-mentioned curvature calculation unit 822.

The calculation device 800 may also be configured to realize the functions of the component calculation unit 831, comparison length calculation unit 832, and index value calculation unit 833 shown in FIG. 22 by the CPU 801 acquiring the program group 804 and executing it.

The component calculation unit 831 calculates the length between parts of a person in the image data that exists within a target object based on the image data.

The comparison length calculation unit 832 calculates the comparison length to be compared based on the image data.

The index value calculation unit 833 calculates an index value indicating the state of the object of interest based on the inter-part length calculated by the component calculation unit 831 and the comparison length calculated by the comparison length calculation unit 832.

In this way, the calculation device 800 can have a component calculation unit 831, a comparison length calculation unit 832, and an index value calculation unit 833. With this configuration, the index value calculation unit 833 can calculate an index value indicating the state of the target of interest based on the inter-part length calculated by the component calculation unit 831 and the comparison length calculated by the comparison length calculation unit 832. As a result, it becomes possible to calculate an index value according to the state of the target of interest based on the image data.

An information processing device such as the calculation device 800 described above can be realized by incorporating a predetermined program into the information processing device. Specifically, a program that is another form of the present invention is a program for implementing a process in an information processing device that calculates, based on image data, a part-to-part length between parts that exist within a target object of interest for a person in the image data, calculates a comparison length to be compared based on the image data, and calculates an index value indicating the state of the target object of interest based on the calculated part-to-part length and the calculated comparison length.

In addition, the calculation method executed by the above-mentioned information processing device is such that the information processing device calculates, based on image data, the part-to-part length between parts that exist within the target object of interest for the person in the image data, calculates, based on the image data, a comparison length to be compared, and calculates an index value indicating the state of the target object of interest based on the calculated part-to-part length and the calculated comparison length.

Even if the invention is a program (or recording medium) or a calculation method having the above-mentioned configuration, it has the same action and effect as the above-mentioned case, and therefore can achieve the same purpose as the calculation device 800 having the above-mentioned index value calculation unit 833.

<Additional Notes>
A part or all of the above-described embodiments can be described as follows: A calculation device and the like according to the present invention will be outlined below. However, the present invention is not limited to the following configuration.

(Appendix 1)
a height calculation unit that calculates a height of a person in the image data based on the image data;
a bending calculation unit that acquires a hand length that is a length connecting the left and right hands of the person in the image data, and calculates an index value indicating bending of the waist of the person in the image data based on the acquired hand length and the height calculated by the height calculation unit;
A computing device comprising:
(Appendix 2)
The calculation device according to claim 1, wherein the bend calculation unit calculates the index value by dividing the hand length by the height.
(Appendix 3)
a labor-effort length calculation unit that calculates the labor-effort length of a person in the image data based on the image data,
The calculation device according to claim 1 or 2, wherein the bend calculation unit calculates the index value based on the hand length calculated by the hand length calculation unit and the height calculated by the height calculation unit.
(Appendix 4)
The calculation device according to claim 3, wherein the hand length calculation unit calculates the hand length based on information indicating coordinates of each part of the person in the image data recognized based on the image data.
(Appendix 5)
The calculation device according to claim 3 or 4, wherein the bend calculation unit calculates the index value based on the hand length selected based on a predetermined criterion from the hand lengths calculated by the hand length calculation unit and the height calculated by the height calculation unit.
(Appendix 6)
The bending calculation unit acquires the hand length stored in advance in a storage device, and calculates the index value based on the acquired hand length and the height calculated by the height calculation unit. The calculation device described in Appendix 1 or Appendix 2.
(Appendix 7)
The calculation device according to any one of Supplementary Note 1 to Supplementary Note 6, wherein the bend calculation unit obtains ground contact timing information indicating a timing at which the foot of the person in the image data touches the ground, and calculates the index value based on the height calculated based on the image data identified based on the ground contact timing information.
(Appendix 8)
The calculation device according to any one of Supplementary Note 1 to Supplementary Note 7, wherein the height calculation unit calculates the height based on information indicating coordinates of each part of the person in the image data recognized based on the image data.
(Appendix 9)
An information processing device,
Calculating a height of a person in the image data based on the image data;
A calculation method comprising: acquiring a wrist length, which is the length connecting the left and right wrists of the person in the image data; and calculating an index value indicating the curvature of the waist of the person in the image data based on the acquired wrist length and the calculated height.
(Appendix 10)
In the information processing device,
Calculating a height of a person in the image data based on the image data;
A program for implementing a process of acquiring a wrist length, which is the length connecting the left and right wrists of the person in the image data, and calculating an index value indicating the curvature of the waist of the person in the image data based on the acquired wrist length and the calculated height.

The programs described in the above embodiments and appendices may be stored in a storage device or a computer-readable recording medium. For example, the recording medium may be a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

Although the present invention has been described above with reference to the above-mentioned embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

In addition, this invention claims the benefit of priority based on patent application No. 2021-055313, filed in Japan on March 29, 2021, and all contents described in that patent application are incorporated herein by reference.

100 Determination system 200 Smartphone 300 Determination device 310 Communication I/F unit 320 Memory unit 321 Trained model 322 Image information 323 Skeleton information 324 Landing judgment information 325 Landing timing information 326 Program 330 Calculation processing unit 331 Image acquisition unit 332 Skeleton recognition unit 333 Landing judgment unit 334 Landing timing judgment unit 335 Output unit 400 Calculation system 500 Calculation device 510 Communication I/F unit 520 Memory unit 521 Trained model 522 Image information 523 Skeleton information 524 Hand length information 525 Height information 526 Waist bending value information 527 Program 530 Calculation processing unit 531 Image acquisition unit 532 Skeleton recognition unit 533 Hand length calculation unit 534 Height calculation unit 535 Bend calculation unit 536 Output unit 600 Calculation system 700 Calculation device 710 Communication I/F unit 720 Memory unit 721 Trained model 722 Image information 723 Skeleton information 724 Comparative length information 725 Foot component information 726 Toe data information 727 Program 730 Calculation processing unit 731 Image acquisition unit 732 Skeleton recognition unit 733 Comparative length calculation unit 734 Foot component calculation unit 735 Toe data calculation unit 736 Output unit 800 Calculation device 801 CPU
802 ROM
803 RAM
804 Program group 805 Storage device 806 Drive device 807 Communication interface 808 Input/output interface 809 Bus 810 Recording medium 811 Communication network 821 Height calculation unit 822 Curvature calculation unit 831 Component calculation unit 832 Comparative length calculation unit 833 Index value calculation unit

Claims (10)

a height calculation unit that calculates a height of a person in the image data based on the image data;
a bending calculation unit that acquires a hand length that is a length connecting the left and right hands of the person in the image data, and calculates an index value indicating bending of the waist of the person in the image data based on the acquired hand length and the height calculated by the height calculation unit;
A computing device comprising: The calculation device according to claim 1 , wherein the bend calculation unit calculates the index value by dividing the hand length by the height. a labor-effort length calculation unit that calculates the labor-effort length of a person in the image data based on the image data,
The calculation device according to claim 1 , wherein the bend calculation unit calculates the index value based on the hand-to-hand length calculated by the hand-to-hand length calculation unit and the height calculated by the height calculation unit. The calculation device according to claim 3 , wherein the hand length calculation unit calculates the hand length based on information indicating coordinates of each part of the person in the image data recognized based on the image data. The calculation device according to claim 3 , wherein the bend calculation unit calculates the index value based on the hand length selected from the hand lengths calculated by the hand length calculation unit according to a predetermined criterion and the height calculated by the height calculation unit. The calculation device according to claim 1 , wherein the bend calculation unit acquires the hand length stored in advance in a storage device, and calculates the index value based on the acquired hand length and the height calculated by the height calculation unit. 2. The calculation device according to claim 1, wherein the bend calculation unit acquires ground contact timing information indicating a timing at which the foot of the person in the image data touches the ground, and calculates the index value based on the height calculated based on the image data specified based on the ground contact timing information. The calculation device according to claim 1 , wherein the height calculation unit calculates the height based on information indicating coordinates of each part of the person in the image data that is recognized based on the image data. An information processing device,
Calculating the height of a person in the image data based on the image data;
A calculation method for calculating an index value indicating the curvature of the waist of the person in the image data based on an arm length, which is the length connecting the left and right arms of the person in the image data, and the calculated height. In the information processing device,
Calculating a height of a person in the image data based on the image data;
A program for implementing a process of acquiring a wrist length, which is the length connecting the left and right wrists of the person in the image data, and calculating an index value indicating the curvature of the waist of the person in the image data based on the acquired wrist length and the calculated height .

Images