JP7630118B2 - Fatigue estimation system, fatigue estimation method, posture estimation device, and program - Google Patents

Description

The present disclosure relates to a fatigue estimation system for estimating a subject's level of fatigue, a posture estimation device used in the estimation system, a fatigue estimation method, and a program.

In recent years, there have been many cases in which accumulated fatigue has led to poor health, injuries, accidents, and the like. In response to this, attention has been focused on technology that can prevent poor health, injuries, accidents, and the like by estimating the level of fatigue. For example, Patent Document 1 discloses a fatigue determination device as a fatigue estimation system for estimating the degree of fatigue, which determines the presence or absence of fatigue and the type of fatigue based on force measurement and bioelectrical impedance measurement.

JP 2017-023311 A

However, in the conventional fatigue assessment device exemplified in the above-mentioned Patent Document 1, the posture estimated may not be appropriate, and the degree of fatigue may not be estimated. Therefore, the present disclosure provides a fatigue estimation system that more appropriately estimates posture and estimates the degree of fatigue.

A fatigue estimation system according to one aspect of the present disclosure includes a plurality of imaging devices, each of which outputs images capturing different parts of a subject's body, a posture estimation device that estimates the posture of the subject based on the images output by each of the plurality of imaging devices, and a fatigue estimation device that estimates and outputs a fatigue level of the subject based on the estimated results of the subject's posture.

Furthermore, a posture estimation device according to one aspect of the present disclosure is the posture estimation device described above.

In addition, a fatigue estimation method according to one aspect of the present disclosure is a fatigue estimation method executed by a fatigue estimation device, which acquires images from each of a plurality of imaging devices, each of which outputs images capturing different parts of a subject's body parts, estimates joint positions of body parts including one body part and the other body part as the posture of the subject based on one image of the plurality of images capturing one body part and another image of the plurality of images capturing another body part sharing at least one joint with the one body part, and estimates a fatigue level of the subject based on the estimated posture of the subject and outputs the result to an output device connected to the fatigue estimation device.

Furthermore, a program relating to one aspect of the present disclosure is a program for causing a computer to execute the fatigue estimation method described above.

A fatigue estimation system according to one embodiment of the present disclosure can more appropriately estimate posture and estimate fatigue levels.

FIG. 1A is a first diagram for explaining estimation of a fatigue level according to an embodiment. FIG. 1B is a second diagram for explaining estimation of a fatigue level according to the embodiment. FIG. 1C is a third diagram for explaining estimation of a fatigue level according to the embodiment. FIG. 2 is a block diagram illustrating a functional configuration of the fatigue estimation system according to the embodiment. FIG. 3 is a flowchart showing a method for estimating a fatigue level according to the embodiment. FIG. 4A shows a subject standing still in posture A. FIG. 4B shows a subject resting in posture B. FIG. 5A is a first diagram illustrating an estimated accumulation of a subject's fatigue level in the embodiment. FIG. 5B is a second diagram illustrating the estimated accumulation of fatigue level of a subject in the embodiment. FIG. 6 is a diagram illustrating an example of synthesis of joint positions according to the embodiment. FIG. 7 is a diagram showing an example of displaying an estimation result according to the embodiment.

The following describes the embodiments in detail with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection forms, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in an independent claim are described as optional components.

Note that each figure is a schematic diagram and is not necessarily a precise illustration. In addition, in each figure, the same reference numerals are used for substantially the same configurations, and duplicate explanations may be omitted or simplified.

(Embodiment)
[overview]
In the fatigue estimation system 200 (see FIG. 2 described later) in this embodiment, an image of the subject 11 (see FIG. 1A described later) whose fatigue level is to be estimated is captured, the posture of the subject is estimated, and the fatigue level of the subject 11 is estimated from the estimated posture. Here, depending on the positional relationship between the subject 11 and the imaging device 201 (see FIG. 1A described later) for acquiring the image, there may be an obstruction between them, making it difficult to obtain an image of the subject 11 required to estimate the posture of the subject 11. As a result, in some cases, it may be impossible to estimate the fatigue level of the subject 11.

Therefore, in the present disclosure, multiple imaging devices 201, each capable of imaging at least a portion of the subject 11, are used to compensate for missing parts in an image of the subject 11 required to estimate the posture of the subject 11. In this way, a fatigue estimation system 200 is provided that enables estimation of the fatigue level of the subject 11 even when the posture of the subject 11 cannot be estimated with one imaging device 201. In addition, as will be described in detail later, in the present disclosure, in order to estimate fatigue of the subject 11, the load on at least one of the muscles and joints and the deteriorating blood flow are estimated from the estimated posture of the subject 11.

Therefore, in the present disclosure, the posture estimation device used for posture estimation can also be used to estimate muscle load, joint load, and the degree of deterioration of blood flow. In addition, the posture estimation device in the present disclosure can be applied as a device for estimating the posture of the subject 11 for any other purpose. That is, the posture estimation device of the present disclosure can generate appropriate information on the posture of the subject 11 (two-dimensional skeletal information, three-dimensional skeletal information, and feature values such as the angle between the neck and the spine and the angle between the spine and the lower leg) even from images captured by the imaging devices 201 arranged such that only a part of the body of the subject 11 is captured in each imaging device 201, and this can be applied to any purpose.

[Fatigue estimation system]
The overall configuration of a fatigue estimation system 200 according to an embodiment will be described below. Fig. 1A is a first diagram for explaining fatigue level estimation according to an embodiment. Fig. 1B is a second diagram for explaining fatigue level estimation according to an embodiment. Fig. 1C is a third diagram for explaining fatigue level estimation according to an embodiment.

In an embodiment, the fatigue estimation system 200 in the present disclosure is a system that estimates a fatigue level of a subject 11 by using an image output by imaging the subject 11 using an imaging device 201. The imaging device 201 is not limited in form as long as it is a camera that images the subject 11 and outputs the image, and may be realized by a fixed camera installed on a wall or ceiling of a building or the like as shown in FIG. 1A.

Here, the subject is seated on a chair 12. In the fatigue estimation system 200 of the present disclosure, the fatigue level of the subject 11 is estimated based on the fatigue accumulated by the subject 11 being in a fixed, stationary posture. In other words, the fatigue accumulated by the load on at least one of the muscles and joints and the worsening blood flow (hereinafter also referred to as a decrease in blood flow rate) due to the fixed posture is estimated. Therefore, the subject 11 is in a stationary posture in a sitting, lying or standing position for at least a certain period of time. The certain period is, for example, several tens of seconds or a few seconds, which is the minimum period during which fatigue can be estimated by the fatigue estimation system 200. Such a period is determined depending on the processing capacity of the imaging device 201 and the fatigue estimation device 100 (see FIG. 2 described later) that constitute the fatigue estimation system 200.

Examples of subjects 11 who assume such a stationary posture include desk workers in an office, drivers steering a moving vehicle, people performing strength training using a load in a stationary posture, residents of facilities such as hospitals, passengers and crew of an airplane, etc.

The image captured and output by the imaging device 201 is processed by the estimation device 100, and the posture of the subject 11 (e.g., joint position 11a) is estimated as shown in FIG. 1B. The estimated posture of the subject 11 is output as a rigid link model, for example. Specifically, as shown in FIG. 1B, skeletons shown as straight lines are connected by joints shown by black dots, and the posture of the subject 11 can be reproduced by the positional relationship between two skeletons connected by one joint. The posture is estimated by image recognition, and is output as the above-mentioned rigid link model based on the positional relationship between the joints.

By fitting the estimated rigid link model to a musculoskeletal model 11b as shown in FIG. 1C, the load on at least one of the muscles and joints of each body part is calculated as an estimate in order to maintain the positional relationship according to the estimated posture for each body part of the muscles that pull the skeletons together and the joints that reversibly connect the positional relationship between the skeletons. Since the estimated load on at least one of the muscles and joints of each body part accumulates the longer the static posture is maintained, the degree of fatigue caused by the subject 11 maintaining the static posture is calculated by a calculation using the estimated load and the duration. In the following description, "at least one of the muscles and joints" is also expressed as "muscles and/or joints".

In addition, in this embodiment, in addition to the estimated value of the load on the muscles and/or joints, the fatigue level can be estimated based on the estimated value of the blood flow of the subject 11. In the following description, an example of estimating the fatigue level of the subject 11 using the estimated values of the load on the muscles and the load on the joints will be mainly described, but it is also possible to combine the estimated value of the blood flow to estimate the fatigue level of the subject 11 with higher accuracy. Furthermore, the fatigue level of the subject 11 can be estimated using any one of the estimated values of the load on the muscles, the load on the joints, and the blood flow of the subject 11. In addition, in the present disclosure, the estimation of the fatigue level of the subject 11 based on the posture of the subject 11 is not limited to the above example, and any existing technology related to the estimation of fatigue level can be applied.

As an example, in this embodiment, the fatigue estimation system 200 estimates the posture of the subject 11, and then estimates at least one of the load on the muscles, the load on the joints, and the blood flow rate of the subject 11 based on the duration of the posture. The fatigue estimation system 200 estimates the fatigue level of the subject 11 based on at least one of the estimated values of the load on the muscles, the load on the joints, and the blood flow rate of the subject 11. Hereinafter, for simplification, the estimated value of the load may be simply expressed as the load or the estimated value. Furthermore, when the estimated value includes an estimated value of the blood flow rate, the load may be read as the blood flow rate, and a large load may be replaced with a decrease in the blood flow rate, and a small load may be replaced with an increase in the blood flow rate.

As described above, the blood flow rate is information for quantifying the blood flow that deteriorates when the subject 11 maintains a posture. The lower the blood flow rate, the worse the blood flow of the subject 11 is, and the blood flow rate can be used as an index of fatigue caused by the deterioration of blood flow. The blood flow rate may be obtained as an absolute value at the time of measurement, or may be obtained as a relative value between two different points in time. For example, the degree of deterioration of the blood flow of the subject 11 can be estimated based on the posture of the subject 11 and the relative value of the blood flow rate at two points in time, the start and end points of the posture. In addition, since there is a correlation between the posture of the subject 11, the duration of the posture, and the deterioration of blood flow, the blood flow rate of the subject may be estimated simply from the posture of the subject 11 and the duration of the posture.

In the following description, the above-mentioned musculoskeletal model is used to estimate at least one of the load on muscles, the load on joints, and the blood flow rate from the posture of the subject 11. However, in addition to the above-mentioned musculoskeletal model, a method using actual measurement data can also be applied as a method for estimating the load on muscles, the load on joints, and the blood flow rate from the posture. This actual measurement data is a database constructed by accumulating the actual measurement values of the load on muscles, the load on joints, and the blood flow rate measured for each posture in association with the posture. In this case, the fatigue estimation system 200 can input the estimated posture of the subject 11 into the database, and obtain the actual measurement values of the load on muscles, the load on joints, and the blood flow rate in the corresponding posture as output.

The actual measurement data may be constructed using actual measurement values for each individual, taking into account individual differences among the subjects 11, or the big data obtained from an unspecified number of subjects may be qualified and constructed to suit each subject 11 through analytical processing such as statistical analysis or machine learning.

Next, the functional configuration of the fatigue estimation system 200 in the present disclosure will be described with reference to Figure 2. Figure 2 is a block diagram showing the functional configuration of the fatigue estimation system in accordance with an embodiment.

As shown in FIG. 2, the fatigue estimation system 200 in the present disclosure comprises a fatigue estimation device 100, multiple imaging devices 201, a display device 205, and a recovery device 206.

The estimation device 100 includes an acquisition unit 101, a recognition unit 102, a posture estimation unit 105, a first calculation unit 106, a second calculation unit 107, a fatigue estimation unit 108, and an output unit 109.

The acquisition unit 101 is a communication module that is connected to each of the multiple imaging devices 201 and acquires images of the subject 11 from each of the multiple imaging devices 201. The connection between the acquisition unit 101 and the imaging devices 201 is made by wire or wirelessly, and there is no particular limitation on the method of communication made through the connection.

In addition to the imaging device 201 described above, the fatigue estimation device 100 may also include a communication module that is connected to a timing device and acquires time from the timing device.

The fatigue estimation device 100 may also be provided with a communication module that is connected to a pressure sensor and acquires pressure distribution from the pressure sensor.

The fatigue estimation device 100 may also be provided with a communication module that is connected to a reception device and acquires personal information from the reception device.

The identification unit 102 is a processing unit that is realized by executing a predetermined program using a processor and a memory. The identification unit 102 is installed to realize an identification function for identifying one subject 11 from other persons. The function of the identification unit 102 will be described in detail later.

The posture estimation unit 105 is a processing unit that is realized by executing a predetermined program using a processor and memory. Through processing by the posture estimation unit 105, the posture of the subject 11 is estimated based on the image acquired by the acquisition unit 101 and the additionally acquired pressure distribution, etc. The posture estimation unit 105 is an example of a posture estimation device. In other words, the posture estimation device in this embodiment is realized by being built into the fatigue estimation device 100 as the posture estimation unit 105.

The first calculation unit 106 is a processing unit that is realized by executing a predetermined program using a processor and a memory. Through processing by the first calculation unit 106, the load on each muscle and/or joint is calculated based on the estimated posture of the subject 11 and additionally acquired personal information.

The second calculation unit 107 is a processing unit that is realized by executing a predetermined program using a processor and a memory. Through processing by the second calculation unit 107, the amount of recovery of fatigue in each muscle and/or joint is calculated based on the amount of change in the estimated change in posture of the subject 11.

The fatigue estimation unit 108 is a processing unit that is realized by executing a predetermined program using a processor and a memory. The fatigue estimation unit 108 estimates the fatigue level of the subject 11 based on the duration of the estimated posture using the posture estimated by the posture estimation unit 105 and time acquired from a timing device or an internal oscillator.

The output unit 109 is a communication module that is connected to the display device 205 and the recovery device 206, and outputs content based on the fatigue level estimation result by the fatigue estimation device 100 to the display device 205 and the recovery device 206. The connection between the output unit 109 and the display device 205 or the recovery device 206 is made by wire or wirelessly, and there is no particular limitation on the method of communication made through the connection.

In addition, when the posture estimation unit 105 fails to estimate the posture of the subject 11, the output unit 109 has a function of notifying an external party that there is a subject 11 for whom a fatigue level cannot be estimated. In other words, the output unit 109 outputs the estimation result of the posture estimation unit 105 indicating that estimation of the posture of the subject 11 has failed.

At this time, the output unit 109 outputs the above estimation result as notification information to notify the presence of a subject 11 whose posture estimation has failed. Based on this notification information, the administrator of the fatigue estimation system 200 can take measures such as estimating the fatigue level of the subject 11 whose posture estimation has failed using a different system or asking directly. In this way, the output unit 109 is configured in advance so that the fatigue level of the subject 11 whose posture estimation has failed is not overlooked.

As described above, the imaging device 201 is a device that captures an image of the subject 11 and outputs the image, and is realized by a camera. As the imaging device 201, an existing camera such as a security camera or a fixed camera in the space in which the fatigue estimation system 200 is applied may be used, or a dedicated camera may be newly installed. Such an imaging device 201 is an example of an information output device that outputs an image as information related to the position of the body parts of the subject 11. Therefore, the output information is an image, and is information including the positional relationship of the body parts of the subject 11 on the projected imaging element.

A timing device is a device that measures time, and is realized by a clock. The time measured by a timing device may be an absolute time, or may be the elapsed time from a relative starting point. The timing device may be realized in any form as long as it can measure the time between the time when the subject 11's stillness is detected and the time when the fatigue level is estimated (i.e., the duration of the still posture).

The pressure sensor is a sensor having a detection surface, and measures the pressure applied to each of the unit detection surfaces that divide the detection surface into one or more unit detection surfaces. The pressure sensor measures the pressure for each unit detection surface in this manner, and outputs the pressure distribution on the detection surface. The pressure sensor is positioned so that the subject 11 is positioned on the detection surface.

For example, the pressure sensor is provided on the seat and backrest of a chair on which the subject 11 sits. Also, for example, the pressure sensor may have a marker on its detection surface, and guide the subject 11 onto the detection surface by displaying, for example, "Please sit on the marker." Also, by guiding the subject 11 onto the detection surface of a pressure sensor provided on a portion of the floor in this manner, the pressure sensor may output the pressure distribution of the subject 11 on the floor. Note that since the pressure distribution is used for the purpose of improving the accuracy of the estimation of the degree of fatigue, the fatigue estimation system 200 may be realized without a pressure sensor if sufficient accuracy is ensured.

The reception device is a user interface that accepts input of personal information of the subject 11, and is realized by an input device such as a touch panel or a keyboard. The personal information includes at least one of age, sex, height, weight, muscle mass, stress level, body fat percentage, and exercise proficiency. The age of the subject 11 may be a specific numerical value, may be an age band divided in 10-year intervals such as teens, twenties, and thirties, may be a two-band age band with a specified age boundary such as 59 years or younger or 60 years or older, or may be something else.

The gender of the subject 11 is selected from either male or female, whichever is appropriate for the subject 11. The height and weight are accepted as the numerical values of the height and weight of the subject 11, respectively. The muscle mass is accepted as the muscle composition ratio of the subject 11 measured using a body composition scale or the like. The stress level of the subject 11 is selected by the subject 11 himself from options such as high, medium, and low as the subjective level of stress felt by the subject 11.

In addition, the body fat percentage of subject 11 is the ratio of the weight of body fat to the body weight of subject 11, and is expressed, for example, as a percentage.

In addition, the subject's 11 exercise proficiency may be quantified by the score the subject 11 gets when performing a specified exercise program, or by the state of the exercise that the subject 11 usually engages in. In the former case, it is quantified, for example, by the time it takes to perform 10 back exercises, the time it takes to run 50 meters, or the distance of a long throw. In the latter case, it is quantified, for example, by how many days a week the subject exercises, or how many hours the subject exercises. Since the personal information is used for the purpose of improving the accuracy of the fatigue level estimation, the fatigue estimation system 200 may be realized without a reception device if sufficient accuracy is ensured.

The display device 205 is a device for displaying the content based on the fatigue level estimation result output by the output unit 109. The display device 205 displays an image showing the content based on the fatigue level estimation result, for example, by a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel. The content displayed by the display device 205 will be described later. Furthermore, when the fatigue estimation system 200 is configured to only reduce the fatigue level of the subject 11 using the recovery device 206, it is sufficient to include only the recovery device 206, and the display device 205 is not required.

The recovery device 206 is a device that reduces the fatigue level of the subject 11 by promoting blood circulation in the subject 11. Specifically, the recovery device 206 actively changes the posture of the seated subject 11 by applying voltage, pressurizing, vibrating, or heating, or by changing the arrangement of each part of the chair 12 using a mechanism provided in the chair 12. In this way, the recovery device 206 changes the load on at least one of the muscles and joints of the subject 11, and also promotes blood circulation. In terms of blood flow, the promotion of blood circulation in this way reduces the effect of poor blood flow due to the subject 11 being in a stationary posture, and the fatigue level is restored. The recovery device 206 is attached to or contacted in advance with an appropriate body part of the subject 11 depending on the configuration of the device.

When promoting blood circulation in subject 11 by heating, the entire space around subject 11 is heated, and in such a case, it is not necessary for the device to be attached to or in contact with an appropriate body part of subject 11. Furthermore, when fatigue estimation system 200 is configured only to display the fatigue level estimation result to subject 11, it is sufficient to include only display device 205, and recovery device 206 is not required.

[Action]
Next, estimation of the fatigue level of the subject 11 using the fatigue estimation system 200 in the embodiment will be described with reference to Fig. 3 to Fig. 6. Fig. 3 is a flowchart showing a method for estimating a fatigue level according to the embodiment.

The fatigue estimation system 200 first acquires personal information of the subject 11. The personal information is acquired by the subject 11 himself/herself or an administrator managing the subject 11's fatigue level by inputting it into a reception device. The input personal information of the subject 11 is stored in a storage device or the like (not shown) and is read out and used when estimating the fatigue level.

The fatigue estimation system 200 detects the subject 11 using the imaging device 201. The subject 11 is detected by counting the people within the angle of view of the camera, which is the imaging device 201, so that all people within the angle of view are considered to be the subject 11. That is, here, an operation of acquiring the number of people to be detected (A) is performed (S101). Next, the number of people (B) whose postures can be estimated from only one image, without processing the image, is calculated (S102). Then, the fatigue estimation device 100 determines whether A and B match (S103). If the fatigue estimation device 100 determines that A and B match (Yes in S103), that is, if the postures of all the people to be detected can be estimated from one image, each of them estimates the posture of the subject 11 (here, estimates the joint position) from one image (S108). Then, the fatigue estimation unit 108 estimates the fatigue level of the subject 11 (S109).

Steps S108 and S109 are performed as follows. First, the fatigue estimation system 200 acquires an image output by the imaging device 201 using the acquisition unit 101. If it is detected in the acquired image that the subject 11 is stationary (in a stationary posture), the posture of the subject 11 is estimated in the estimation device 100. Specifically, first, the pressure distribution applied to the detection surface from the pressure sensor is acquired.

The posture estimation unit 105 estimates the posture of the subject 11 based on the acquired image and pressure distribution. For example, when biased pressure is applied, the pressure distribution is used to correct the estimated posture so that the bias is formed. Next, the first calculation unit 106 calculates the load on each muscle and/or joint of the subject 11 from the posture estimation result. At this time, the load is corrected and calculated using personal information acquired in advance. Note that the estimation of the posture of the subject 11 has been explained using FIG. 1B, and the calculation of the load has been explained using FIG. 1C, so a detailed explanation will be omitted.

When correcting the load amount using personal information, for example, the load amount is decreased the closer the age of subject 11 is to the peak age of muscle development, and increased the further the age is from that peak age. Such peak values may be based on the gender of subject 11. Furthermore, the load amount may be decreased if subject 11 is male, and increased if subject 11 is female. Furthermore, the load amount may be decreased the smaller the height and weight of subject 11 are, and increased the larger the height and weight are.

In addition, the load may be set so that the greater the muscle mass of subject 11 is in the composition ratio, the less the load, and the smaller the muscle mass is in the composition ratio, the more the load. In addition, the lower the stress level of subject 11 is, the less the load, and the higher the stress level, the more the load. In addition, the higher the body fat percentage of subject 11 is, the more the load, and the lower the body fat percentage, the less the load. In addition, the higher the exercise proficiency of subject 11 is, the less the load, and the lower the exercise proficiency, the more the load.

Here, the duration of the stationary posture of the subject 11 is measured based on the time obtained from the timing device. The fatigue estimation unit 108 adds the load calculated above each unit time of the duration, and estimates the fatigue level of the subject 11 at that point in time. These processes are continued until the stationary state of the subject 11 is released. Specifically, whether the stationary state has been released is determined based on whether the posture estimated by the posture estimation unit 105 has been changed from a certain stationary posture.

If it is not determined that the stationary state has been released, the fatigue level of the subject 11 is accumulated as long as the stationary posture continues by measuring the duration and adding the load. In other words, the fatigue estimation unit 108 estimates the fatigue level of the subject 11 using an increasing function of the fatigue level with a slope corresponding to the calculated load with respect to the duration. Therefore, the greater the calculated load, the greater the increase in the fatigue level of the subject 11 per unit time. Note that in this accumulation of fatigue level, the fatigue level of the subject 11 is initialized (set to fatigue level 0) at the start of the stationary posture, which is the starting point.

On the other hand, if it is determined that the stationary state has been released, the posture estimation unit 105 calculates the amount of change in posture from the original stationary state posture to the changed current posture. The amount of change in posture is calculated for each muscle and/or joint, similar to the load amount described above. When posture changes in this way, the load on at least one of the muscles and joints changes, and in terms of blood flow, the worsening blood flow is temporarily alleviated, and the fatigue level of the subject 11 begins to recover. The fatigue level reduced by recovery is related to the amount of change in posture. Accordingly, the second calculation unit 107 calculates the amount of recovery, which is the degree of recovery of fatigue level, based on the amount of change in posture.

Based on the time obtained from the timing device, the change time, which is the time during which the change in posture of the subject 11 continues, is measured. The relationship between the recovery amount and the change time is the same as the relationship between the load amount and the duration, and the recovery amount of the subject 11 is accumulated as long as the change in posture continues. In other words, when the posture of the subject 11 changes in this way, the fatigue estimation unit 108 estimates the fatigue level of the subject 11 by subtracting the recovery amount every time a unit of time elapses.

The calculation of the recovery amount, measurement of the change time, and estimation of the subject's 11 fatigue level are continued until the subject's 11 posture stops. Specifically, it is determined whether the posture estimated by the posture estimation unit 105 is a certain stationary posture. If the subject's 11 not being stationary is detected, the recovery amount is calculated, the change time is measured, and the recovery amount is subtracted, accumulating so that the subject's 11 fatigue level is recovered as long as the posture change continues.

In other words, the fatigue estimation unit 108 estimates the fatigue level of the subject 11 using a fatigue level decreasing function having a slope corresponding to the calculated recovery amount with respect to the change time. Since the recovery amount of the fatigue level depends on the amount of change in posture, the greater the amount of change in posture, the greater the decrease in the fatigue level of the subject 11 per unit time.

On the other hand, if the subject 11 is detected to be stationary, the posture and fatigue level are estimated again for the new stationary posture. In this way, the fatigue estimation system 200 calculates the fatigue level of the subject 11 based on the image and taking into account the duration of the stationary posture, so that the fatigue level of the subject 11 can be estimated with less burden on the subject 11 and with higher accuracy.

The above will be explained in more detail using Figures 4A to 5B. Figure 4A is a diagram showing a subject standing still in posture A. Figure 4B is a diagram showing a subject standing still in posture B.

The subject 11 shown in Figures 4A and 4B is in a static posture, seated in a chair 12, similar to that shown in Figure 1A. Note that in Figures 4A and 4B, a table, PC, etc., which are not shown, are actually present, but only the subject 11 and the chair 12 are shown here. The static posture of subject 11 shown in Figure 4A is posture A, which places a relatively heavy load on the shoulders. On the other hand, the static posture of subject 11 shown in Figure 4B is posture B, which places a relatively light load on the shoulders.

The fatigue level estimated for the subject 11 resting in such posture A or posture B is accumulated as shown in Figures 5A and 5B. Figure 5A is a first diagram illustrating the accumulation of the estimated fatigue level of the subject according to the embodiment. Also, Figure 5B is a second diagram illustrating the accumulation of the estimated fatigue level of the subject according to the embodiment.

As shown in Figure 5A, when the subject 11 remains stationary in posture A shown in Figure 4A or posture B shown in Figure 4B, the fatigue level of the subject 11 is expressed by a linear function whose slope is the amount of load calculated from the posture.

As described above, posture A is a posture that imposes a greater load on the subject 11 than posture B. Therefore, for example, in a certain muscle of the subject 11 (here, the muscle related to shoulder movement), the load on posture A (the slope of the straight line in posture A) is greater than the load on posture B (the slope of the straight line in posture B). For this reason, in posture A, subject 11 accumulates (accumulates) a greater degree of fatigue in a shorter period of time than when he or she is stationary in posture B.

On the other hand, as shown in Figure 5B, when the posture of subject 11 changes from posture A shown in Figure 4A to posture B shown in Figure 4B, the fatigue level of subject 11 is expressed by a function that combines a linear function whose slope is the amount of load calculated from the posture and a linear function whose slope is the amount of change in posture.

Therefore, for example, while subject 11 is stationary in posture A, the fatigue level of subject 11 is estimated as an accumulation (addition) of fatigue level by an increasing function with a positive slope corresponding to the load of posture A, as in FIG. 5A, and the accumulation (addition) turns to recovery (decrease) at the change point where subject 11 starts to change his posture. Subject 11's fatigue level recovers (decreases) by an amount shown as the change width in the figure during the period when the change in posture continues, shown as the change time in the figure, by a decreasing function with a negative slope corresponding to the amount of change in posture. After the change point where subject 11's posture stops again in posture B, subject 11's fatigue level is estimated as an accumulation (addition) of fatigue level by an increasing function with a positive slope corresponding to the load of posture B.

In this way, the fatigue estimation system 200 in this embodiment estimates the fatigue level of the subject 11, reflecting accumulation and recovery in response to the subject 11's stillness and changes in posture.

Returning to the explanation of the flowchart in FIG. 3, in step S103, if the fatigue estimation device 100 determines that A and B do not match (No in S103), that is, if the persons to be detected include a person whose posture cannot be estimated from a single image, the fatigue estimation device 100 identifies such a person (S104). Then, the posture estimation unit 105 selects and acquires multiple images that can be used to estimate the posture of the identified person (hereinafter, the subject 11 whose fatigue level is estimated through synthesis of joint positions) (S105). Here, if the posture of the subject 11 cannot be estimated even using multiple images (No in S106), the fatigue level estimation process for this subject 11 is terminated. At this time, for example, notification information described as a function of the output unit 109 is output.

For subject 11 whose posture can be estimated by using multiple images (Yes in S106), the posture of subject 11 is estimated as shown in FIG. 6 below. FIG. 6 is a diagram illustrating an example of joint position synthesis according to an embodiment. FIG. 6 shows information generated while the posture of subject 11 (right end) is estimated based on multiple acquired images (left end).

In addition, in FIG. 6, the image processing system is divided into upper and lower sections from the left to the center, and one image 90a of the multiple images and a joint position 11c of one body part estimated from the first image 90a are shown on the upper section. Similarly, in FIG. 6, another image 90b of the multiple images and a joint position 11d of the other body part estimated from the other image 90b are shown on the lower section. First, in step S105, the posture estimation unit 105 acquires multiple images (here, the first image 90a and the other image 90b). The first image 90a and the other image 90b are images of the one body part and the other body part that are part of the body part of the subject 11, respectively.

In addition, one body part and another body part are body parts that share at least one joint with each other. Therefore, if a combination of multiple images that share at least one joint with each other cannot be identified, the answer to step S106 above becomes No and the process ends.

As shown in FIG. 3, the posture estimation unit 105 estimates the joint positions of the body parts of the subject 11 that are partially reflected in each of the multiple images (S107). In the example of FIG. 6, based on one image 90a, the joint position 11c of the one body part reflected in the one image 90a is estimated, and first information 90c including this is generated. Similarly, in the example of FIG. 6, based on the other image 90b, the joint position 11d of the other body part reflected in the other image 90b is estimated, and second information 90d including this is generated. Then, the joint position 11c of the one body part and the joint position 11d of the other body part are synthesized to estimate the joint position 11a of the body part of the subject 11 including the one body part and the other body part. In this way, the posture estimation unit 105 can estimate the posture of the subject 11 using multiple images in which some body parts of the subject 11 are reflected.

Now, we will continue to refer to Figure 6 to explain the synthesis of joint positions. To synthesize the joint positions of the subject 11, at least two or more common coordinates are required between the joint position 11c of one body part and the joint position 11d of another body part. One of these common coordinates is the joint position of a shared joint, which is one of the joints shared between one body part and another body part. Then, by using a reference coordinate shared within the above-mentioned multiple images in addition to the joint position of the shared joint, it is possible to match the scale of the multiple images and the posture (orientation) of the body parts within the images.

As the reference coordinate, the joint position of a joint other than the shared joint among the joints shared between the one body part and the other body part is used. In this example, in a situation where two or more joints are shared between the one body part and the other body part, the posture estimation unit 105 adjusts the joint position 11c in the one body part and the joint position 11d in the other body part so that the direction and magnitude of the vector extending from one joint to the other joint between these joints match. Then, the posture estimation unit 105 estimates the joint position 11a of the body part of the subject 11 by superimposing and synthesizing the joint position 11c in the one body part and the joint position 11d in the other body part with the shared joint as a reference, in a state where the posture (direction) and scale are matched.

In addition to the above, a marker that appears in both the first image 90a and the other image 90b may be used as the reference coordinate. In the example of FIG. 6, an object marker M that moves in common to both the first image 90a and the other image 90b is shown. The object marker M may be a dedicated object, or any object that exists in space and can be identified as the same object by multiple imaging devices 201 may be used. If the spatial coordinates of this object marker M are used as the reference coordinates, the pose (direction) and scale can be matched, and the objects can be superimposed and synthesized based on the shared joint, as in the above example. For this purpose, the pose estimation unit 105 may calculate the spatial coordinates Mc of the object marker M from the first image 90a and calculate the spatial coordinates Md of the object marker M from the other image 90b.

As another example of a marker, a line marker L may be used. For example, the line marker L is realized by attaching a piece of tape or the like to a location within space. The line marker L is configured so that both ends are in common positions and extend between these ends. Therefore, in an example in which the line marker L is present, the same effect as above can be obtained even if there are no shared joints in the body parts of the subject 11.

Further, further effects can be obtained by utilizing the existence of these markers in real space. That is, in the fatigue estimation system 200, the subject 11 can be identified from other people who are staying in the same space as the subject 11 based on the positional relationship between the markers and the subject 11. In this embodiment, processing such as superimposing the joint positions of each body part of the subject 11 from multiple images is performed. Therefore, when another person (e.g., another person 11z in FIG. 6) is staying in the same space as the subject 11, if the joint positions of one body part of the subject 11 are superimposed on the joint positions of another body part of the other person, the posture of the subject 11 cannot be estimated correctly.

Therefore, the fatigue estimation system 200 is equipped with an identification unit 102, which allows the subject 11 to be identified and some joint positions of the subject 11 to be reliably superimposed on other joint positions of the subject 11. For this purpose, the identification unit 102 identifies the relative positions of the subject 11 and the marker in the image acquired from the imaging device 201 (that the subject 11 is at a position a certain distance in a certain direction relative to the marker, and another person 11z is at a position a different distance in a different direction relative to the marker). This process requires only identification with the spatial resolution of one person, so the processing load is smaller and it can be easily implemented compared to the process of estimating the position of each joint.

As described above, in this embodiment, even if an image shows only a portion of the subject's 11 body part, by combining and complementing multiple images, it is possible to estimate an appropriate posture that can be used to estimate the level of fatigue.

Furthermore, when the above-described method estimates the joint positions of body parts including a part of the body part of subject 11 and another part of the body part as the posture of subject 11, it may not be possible to estimate the posture of other parts that are not included in the part and the other part of the body part of subject 11. For example, the above-described process cannot estimate the posture of an invisible part that is not shown in any of the multiple images, i.e., is not captured in any of the images.

In this case, for example, the joint positions of some and other parts of the body parts of the subject 11 estimated as described above may be input into a machine learning model in which the joint positions of the entire human body are learned, thereby estimating the joint positions of the entire body as the posture of the entire body of the subject 11. Furthermore, even if it is not the entire body, the posture of the subject 11 may be estimated by adding only a part of the invisible parts that has sufficient reliability as the output of the machine learning model. In this way, the posture estimation unit 105 may be configured to estimate the joint positions of the body parts of the subject 11, including the invisible parts that are not captured in the multiple images, as the posture of the subject 11.

Returning to Fig. 3 again, in this manner, the posture estimation unit 105 uses multiple images showing some body parts of the subject 11 to perform processing such as synthesis to estimate the posture of the subject 11 (S108). Then, proceeding to step S109, the fatigue estimation unit 108 estimates the fatigue level of the subject 11 as already described.

Next, an example of the output of the output unit 109 based on the estimated fatigue level will be described. Figure 7 is a diagram showing an example of the display of the estimation result in the embodiment.

As shown in FIG. 7, in the fatigue estimation system 200, the estimated fatigue level of the subject 11 can be displayed and fed back using the display device 205. Specifically, as shown in FIG. 7, by visualizing the fatigue level of the subject 11, it is possible to visually grasp the degree of fatigue of the subject 11. In the figure, a doll imitating the subject 11 and the fatigue levels of the subject 11's shoulders, back, and lower back are displayed together by the display device 205. To make it easier for the subject 11 to intuitively grasp the fatigue level, the fatigue level of the shoulders is shown as "stiff shoulder level," the fatigue level of the back as "back pain level," and the fatigue level of the lower back as "lower back pain level."

Here, in the display in the figure, the fatigue levels of three parts of the subject 11 are displayed all at once, but the fatigue levels of these three parts are estimated from an image captured at one time. In other words, the estimation device 100 estimates the fatigue level for the muscles and/or joints in each of multiple body parts including the first part (e.g., shoulders), the second part (e.g., back), and the third part (e.g., waist) of the subject 11 from one posture of the subject 11. Therefore, even if the posture of the subject 11 is constant, the fatigue levels accumulated in the muscles and/or joints differ for each body part, but the fatigue estimation system 200 can estimate such different fatigue levels simultaneously and individually.

1C, in this embodiment, the load is calculated for each muscle and/or joint of the subject 11, and therefore, if there are no limitations on processing resources, the fatigue level of each muscle and/or joint can be estimated. Therefore, there is no limit to the number of body parts whose fatigue level is estimated from an image captured at one time, and it may be one, two, four or more.

The estimation device 100 calculates the load amount for each of a plurality of body parts, and in one posture of the subject 11, it is possible to estimate the fatigue level of a first part (the above-mentioned shoulder stiffness level) based on the load amount calculated for the first part, the fatigue level of a second part (the above-mentioned back pain level) based on the load amount calculated for the second part, and the fatigue level of a third part (the above-mentioned lower back pain level) based on the load amount calculated for the third part.

In the example shown in the figure, the degree of stiff shoulders is estimated from the load on the trapezius muscle, the degree of back pain is estimated from the fatigue of the latissimus dorsi muscle, and the degree of lower back pain is estimated from the load on the lumbar paraspinal muscles. In this way, one fatigue level may be estimated from the load on one muscle and/or joint, but one fatigue level may also be estimated from the combined load on multiple muscles and/or joints. For example, the degree of stiff shoulders (i.e., one fatigue level of the shoulder) may be estimated from the average value of the load on the trapezius muscle, levator scapulae muscle, rhomboid muscle, and deltoid muscle. In addition, in estimating the fatigue level, a more realistic estimation of the fatigue level may be performed by weighting the load on the muscle and/or joint that has a particularly large effect on the fatigue level of the body part in accordance with the magnitude of the effect, rather than using a simple average value.

The fatigue level estimated in this manner may be shown as a relative position on a reference meter with a minimum value of 0 and a maximum value of 100, as shown in the figure. Here, a reference value is set at a predetermined position on the reference meter. Such a reference value is set to the relative position (or around it, etc.) of the fatigue level at which subjective symptoms such as pain may occur in a typical subject 11, which has been quantified in advance by epidemiological surveys, etc. Therefore, the reference value may be set to a different value depending on the fatigue level of each body part.

Furthermore, when the estimated fatigue level of the subject 11 reaches a reference value, the display device 205 may display a warning to the subject 11 as an estimated result. The reference value here is an example of a first threshold value. In the figure, as an example of such a warning, "The degree of stiff shoulders has exceeded the reference value," is shown at the bottom of the display device 205. In connection with such a warning, the display device 205 may also display a specific method of dealing with the situation, such as "We recommend taking a break," as also shown in the figure.

In addition to the above-mentioned configuration in which the estimation result is displayed to the subject 11 to encourage the subject 11 to deal with the accumulated fatigue level, a configuration in which the fatigue estimation system 200 actively recovers the fatigue level of the subject 11 can also be considered. Specifically, the fatigue level of the subject 11 is recovered by the operation of the recovery device 206 shown in FIG. 2. The specific configuration of the recovery device 206 is as described above, so an explanation is omitted. When the estimated fatigue level of the subject 11 reaches a reference value, the recovery device 206 operates to change the load on at least one of the muscles and joints of the subject 11 and promote blood circulation to reduce the subject's fatigue level. The reference value here is an example of a third threshold value, and may be the same as or different from either the first threshold value or the second threshold value.

[Effects, etc.]
As described above, the fatigue estimation system 200 in this embodiment comprises a plurality of imaging devices 201, each of which outputs images capturing different parts of the body of the subject 11, a posture estimation device (posture estimation unit 105) that estimates the posture of the subject 11 based on the multiple images output by each of the multiple imaging devices 201, and a fatigue estimation device 100 that estimates and outputs the fatigue level of the subject 11 based on the estimated result of the posture of the subject 11.

In such a fatigue estimation system 200, in a situation where an image showing the entire body of the subject 11 in a single image cannot be obtained due to conditions such as the placement position of the imaging device 201, making it difficult to estimate the posture of the subject 11 from a single image, the posture estimation unit 105 can appropriately estimate the posture of the subject 11 by combining and using another image. Then, based on the posture appropriately estimated by the posture estimation unit 105, it becomes possible to more appropriately estimate the fatigue level of the subject 11.

Furthermore, for example, the posture estimation unit 105 may estimate a joint position 11c in one body part based on one image 90a in which one body part is captured among the multiple images, and estimate a joint position 11d in the other body part based on another image 90b in which another body part that shares at least one joint with the one body part is captured among the multiple images, and may combine the estimated joint position 11c in the one body part and the estimated joint position 11d in the other body part to estimate the joint position 11a of the body part including the one body part and the other body part as the posture of the subject 11.

According to this, the joint position 11a of the body part can be estimated using one image 90a in which one body part is captured among the multiple images, and another image 90b in which another body part is captured. Here, the joint position 11c of the one body part is estimated from the one image 90a, the joint position 11d of the other body part is estimated from the other image 90b, and these are combined to estimate the joint position 11a of the body part including the one body part and the other body part. In this way, the joint position 11a including all of the other body parts and the one body part not included in the one image 90a and not included in the other image 90b can be estimated.

In addition, for example, in synthesizing an estimated joint position 11c in one body part and an estimated joint position 11d in another body part, at least one of the relative posture and relative scale between the joint position 11c in one body part and the joint position 11d in the other body part may be determined based on the relative position of a shared joint, which is a joint shared between one body part and the other body part, and a reference coordinate shared within multiple images, and the shared joint and the reference coordinate may be superimposed in a state in which at least one of the determined relative posture and relative scale between the joint position 11c in one body part and the joint position 11d in the other body part is applied.

According to this, by superimposing the relative positions of the shared joint and the reference coordinates, it is possible to synthesize the joint position 11c in one body part from one image 90a, and the joint position 11d in another body part from the other image 90b. At this time, depending on the placement position of the imaging device 201, a situation may occur in which at least one of the posture and scale of the joint position 11c in one body part and the joint position 11d in the other body part differs between the two images. In the above aspect, the above synthesis can be performed after adjusting the posture and scale, etc., that differ between the two images by applying the relative posture (e.g., the amount of rotation to eliminate the difference in relative orientation) and relative scale (e.g., the ratio of enlargement/reduction to eliminate the difference in relative scale) between the joint position 11c in one body part and the joint position 11d in the other body part.

For example, the reference coordinate may be the joint position of a joint other than the shared joint.

According to this, by using the joint position of a joint other than the shared joint as a reference coordinate and superimposing the relative positions of the shared joint and the reference coordinate, it is possible to synthesize a joint position 11c of one body part from one image 90a, and a joint position 11d of another body part from another image 90b.

For example, the reference coordinates may also be the spatial coordinates of a marker placed at a position that is captured in all of the multiple images.

According to this, by using the spatial coordinates of a marker placed at a position that is captured in all of the multiple images as reference coordinates and superimposing the relative positions of the shared joint and the reference coordinates, it is possible to synthesize a joint position 11c of one body part from one image 90a, and a joint position 11d of another body part from another image 90b.

For example, the device may further include an identification unit 102 that identifies the subject 11 from other people 11z who are present in the same space as the subject 11 based on the positional relationship between the marker and the subject 11.

This makes it possible to distinguish the subject 11 from other persons 11z who are present in the same space as the subject 11 based on the positional relationship between the marker and the subject 11, and to estimate the posture of only the subject 11. As a result, even if other persons 11z are present, it is possible to estimate the fatigue level of only the subject 11.

In addition, for example, when the posture estimation unit 105 fails to estimate the posture of the subject 11, it outputs an estimation result indicating that estimation of the posture of the subject 11 has failed, and the fatigue estimation system 200 may further include an output unit 109 that outputs notification information to notify the presence of a subject 11 whose posture estimation has failed, based on the estimation result indicating that estimation of the posture of the subject 11 has failed.

This makes it possible to know the presence of a subject 11 whose posture estimation has failed based on the notified notification information.

For example, the posture estimation unit 105 may estimate joint positions of some body parts of the subject 11 based on multiple images, input the estimated joint positions of some body parts of the subject 11 into a machine learning model in which the joint positions of a human's entire body have been learned, and estimate the joint positions 11a of body parts of the subject 11, including invisible parts that are not captured in the multiple images, as the posture of the subject 11.

According to this, by using a machine learning model that has learned the joint positions of a person's entire body, it is possible to estimate joint positions 11a for body parts, including invisible parts of the subject 11, that cannot be estimated using any combination of multiple images.

For example, the posture estimation unit 105 may further obtain the number of detection targets within the area, and calculate the number of detection targets whose posture can be estimated from only one image output from one of the imaging devices 201 that captures the image within the area, and when the number of detection targets and the number of posture-estimateable targets differ, the posture estimation unit 105 may estimate the posture of the detection targets 11 by treating the detection targets whose posture is not estimable as the target 11.

This allows the above posture estimation using multiple images only when the number of detection subjects in an area differs from the number of posture-estimateable subjects, i.e., only when there is a subject 11 among the detection subjects whose posture cannot be estimated without combining multiple images. This allows the processing resources for posture estimation to be reduced, and the fatigue estimation system 200 can be realized with a simple configuration.

Furthermore, the posture estimation device in this embodiment is the posture estimation unit 105 described above.

According to this, in a situation where an image showing the entire body of the subject 11 cannot be obtained in a single image due to conditions such as the placement position of the imaging device 201, and it is difficult to estimate the posture of the subject 11 from a single image, the posture of the subject 11 can be appropriately estimated by combining and using another image.

In addition, the fatigue estimation method in this embodiment is a fatigue estimation method executed by the fatigue estimation device 100, which acquires images from each of a plurality of imaging devices 201, each of which outputs images capturing a different part of the body part of the subject 11, and estimates the joint position 11a of the body part including the one body part and the other body part as the posture of the subject 11 based on one image 90a of the plurality of images capturing one body part and another image 90b of the plurality of images capturing another body part that shares at least one joint with the one body part, and estimates the fatigue level of the subject 11 based on the estimated posture of the subject 11 and outputs the result to an output device connected to the fatigue estimation device 100.

This can achieve the same effect as the fatigue estimation system 200 described above.

In addition, the program in this embodiment is a program for causing a computer to execute the fatigue estimation method described above.

This allows the same effect as the fatigue estimation system 200 described above to be achieved using a computer.

Other Embodiments
Although the embodiments have been described above, the present disclosure is not limited to the above-described embodiments.

For example, in the above embodiment, the processing performed by a specific processing unit may be executed by another processing unit. Also, the order of multiple processes may be changed, or multiple processes may be executed in parallel.

Furthermore, the fatigue estimation system or posture estimation device in the present disclosure may be realized by multiple devices each having a portion of the multiple components, or may be realized by a single device having all of the multiple components. Also, some of the functions of a component may be realized as the functions of another component, and each function may be distributed in any manner among the components. Any form having a configuration that is provided with all of the functions that can substantially realize the fatigue estimation system or posture estimation device of the present disclosure is included in the present disclosure.

In the above embodiment, each component may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

Each component may also be realized by hardware. For example, each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Each of these circuits may also be a general-purpose circuit, or a dedicated circuit.

In addition, the general or specific aspects of the present disclosure may be realized in a system, an apparatus, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM. In addition, the general or specific aspects of the present disclosure may be realized in any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

In the above embodiment, the subject's posture is estimated from an image using a rigid link model generated by image recognition, the load is calculated from the estimated subject's posture, and the subject's fatigue level is estimated based on the load and duration, but the method of estimating fatigue level is not limited to this. Any existing method may be used as a method of estimating the subject's posture from an image, and any existing method may be used as a method of estimating the load from the subject's posture.

In addition, in the above embodiment, the increasing function and the decreasing function are described as being linear first-order functions, but this is not limited thereto. The increasing function may be a curved function as long as the fatigue level increases over time. Furthermore, the decreasing function may be a curved function as long as the fatigue level decreases over time.

The estimation device described above estimates the subject's level of fatigue using estimated values of muscle load, joint load, and blood flow rate estimated from the subject's posture, but it is also possible to achieve a more accurate estimation of fatigue level by correcting the estimated values with values measured using a measuring device. Specifically, the estimation device obtains measurement values that correspond to the estimated values and are based on the measurement results of measuring the subject using a measuring device.

The detection device may be, for example, an electromyograph, a muscle hardness meter, a pressure gauge, or a near-infrared spectrometer, and may obtain measurements related to the load on the muscle, the load on the joint, and the blood flow rate by measurement. For example, an electromyograph may estimate the muscle movement corresponding to the potential measured by potential measurement. In other words, the estimated muscle movement may be obtained as the measurement value. The estimated muscle movement may be converted into the load on the muscle, and therefore the estimated muscle load may be corrected by the measurement value. The correction here may involve, for example, taking the average of the estimated value and the measurement value, selecting either the estimated value or the measurement value, or substituting the estimated value into the correlation function between the estimated value and the measurement value.

A muscle hardness meter can estimate muscle hardness based on the stress exerted when pressure is applied to the muscle. The estimated muscle hardness can be converted into the amount of load on the muscle, and can be used to correct the estimated value in the same way as above.

A pressure gauge can obtain measurements of the pressure being applied to parts of the subject's body. Such pressure parameters can be input into a musculoskeletal model. Inputting additional parameters such as pressure improves the estimation accuracy of the musculoskeletal model, allowing the estimated values estimated using the musculoskeletal model to be corrected with higher accuracy.

The near-infrared spectrometer can obtain a measurement value obtained by spectroscopically measuring the blood flow rate of the subject. As in the above embodiment, when the estimated value does not include the blood flow rate, the estimated value may be corrected by combining the blood flow rate measured by the infrared spectrometer. Even when the estimated value includes the blood flow rate, the measured blood flow rate may be used when the reliability of the estimated value of the blood flow rate is low.

In this way, by using measurement values corresponding to estimates obtained from a different aspect and making corrections to make the estimates more accurate, it is possible to make a more accurate estimate of the subject's level of fatigue.

In addition, a fatigue factor identification system that identifies the factors behind a subject's fatigue may be configured using the fatigue estimation system described in the above embodiment. Conventionally, with devices or systems that estimate the degree of fatigue as "degree of stiff shoulders" and "degree of lower back pain", etc., it has been difficult to identify the way muscles and joints are used that are the factors behind such "degree of stiff shoulders" and "degree of lower back pain" (i.e., the posture that is the factor). Therefore, by using the fatigue estimation system in the present disclosure, the above problem can be addressed.

That is, in the fatigue factor identification system of the present disclosure, a body part in which fatigue is likely to accumulate in a static posture taken by the subject (a body part with a large estimated amount of promoting various types of fatigue) is identified as a fatigue factor part. Furthermore, the fatigue factor identification system may simply identify a fatigue factor part in one static posture taken by the subject, or may identify a fatigue factor posture with the largest estimated amount of the fatigue factor part from among multiple static postures taken by the subject. In addition, a recommended posture to replace the identified fatigue factor posture may be presented, or a fatigue recovery operation using a recovery device may be performed on the fatigue factor part in the fatigue factor posture.

The fatigue factor identification system includes the fatigue estimation system described in the above embodiment and a storage device for storing information related to the estimated fatigue level. Such a storage device is realized, for example, using a semiconductor memory or the like, and each main storage unit constituting the fatigue estimation system may be used, or a new storage device may be provided that is communicatively connected to the estimation device.

The present disclosure may also be realized as a fatigue estimation method executed by a fatigue estimation system or an estimation device. The present disclosure may also be realized as a program for causing a computer to execute such a fatigue estimation method, or as a computer-readable non-transitory recording medium on which such a program is recorded.

In addition, this disclosure also includes forms obtained by applying various modifications to the embodiments that may occur to a person skilled in the art, or forms realized by arbitrarily combining the components and functions of each embodiment without departing from the spirit of this disclosure.

11 Subject 11a Joint position 11c Joint position in one body part 11d Joint position in another body part 11z Another person 90a One image 90b Another image 100 Fatigue estimation device 101 Acquisition unit 102 Identification unit 105 Posture estimation unit (posture estimation device)
109 Output unit 200 Fatigue estimation system 201 Imaging device Mc, Md Spatial coordinates

Claims (7)

A plurality of image capturing devices each outputting an image capturing a different part of a body part of a subject;
a posture estimation device that estimates a posture of the subject based on a plurality of the images output from each of the plurality of imaging devices;
a fatigue estimation device that estimates and outputs a fatigue level of the subject based on an estimation result of the posture of the subject,
The plurality of images include an image in which one body part is captured, and in which a first part that is a part of another body part sharing at least one joint with the one body part is not captured, and another image in which the other body part is captured, and in which a second part that is a part of the one body part is not captured,
The posture estimation device includes:
estimating a first joint position in the one body part including a joint of the second part based on the one image of the plurality of images;
estimating a second joint position in the other body part including a joint of the first part based on the other image of the plurality of images;
combining the estimated first joint position and the estimated second joint position to estimate joint positions of body parts including the one body part and the other body part as a posture of the subject;
In the synthesis of the estimated first joint position and the estimated second joint position ,
determining at least one of a relative orientation and a relative scale between the first joint position and the second joint position based on a relative position between a shared joint, which is a joint shared between the one body part and the other body part, and a reference coordinate, which is a joint position of a joint different from the shared joint, which is a reference coordinate shared among a plurality of the images;
a fatigue estimation system for estimating fatigue in which the joint of the second part and the joint of the first part are complemented by each other by superimposing the shared joint and the reference coordinates in a state in which at least one of a relative posture and a relative scale between the determined first joint position and the determined second joint position is applied. when the posture estimation device fails to estimate the posture of the subject, outputting the estimation result indicating that the estimation of the posture of the subject has failed;
The fatigue estimation system according to claim 1 , further comprising an output unit that outputs notification information for notifying the presence of the subject whose posture estimation has failed, based on the estimation result indicating that estimation of the subject's posture has failed. The posture estimation device includes:
Estimating joint positions in a body part of the subject based on the plurality of images;
The estimated joint positions of the part of the body of the subject are input into a machine learning model that has learned the joint positions of the entire human body;
The fatigue estimation system according to claim 1 or 2, further comprising: estimating, as the posture of the subject, joint positions of body parts including invisible body parts that are not captured in the plurality of images among the body parts of the subject. The posture estimation device further comprises:
Obtain the number of people to be detected in the area,
Calculating the number of posture-estimable persons whose postures can be estimated from only one image output from one of the imaging devices that captures images within the area, among the detection targets;
The fatigue estimation system according to claim 1 or 2, wherein when the number of the detection subjects is different from the number of the posture-estimateable persons, a person among the detection subjects whose posture is not estimable is set as the subject, and the posture of the subject is estimated. The posture estimation device according to claim 1 . A fatigue estimation method executed by a fatigue estimation device, comprising:
acquiring images from a plurality of image capture devices each outputting an image of a different part of a body part of a subject;
based on one image of the plurality of images in which one body part is captured and which does not include a first part that is a part of another body part sharing at least one joint with the one body part, and another image of the plurality of images in which the other body part is captured and which does not include a second part that is a part of the one body part, joint positions of the body parts including the one body part and the other body part are estimated as a posture of the subject,
Based on the result of estimating the posture of the subject, a fatigue level of the subject is estimated and output to an output device connected to the fatigue estimation device. In estimating the posture of the subject,
estimating a first joint position in the one body part including a joint of the second part based on the one image of the plurality of images;
estimating a second joint position in the other body part including a joint of the first part based on the other image among the plurality of images;
combining the estimated first joint position and the estimated second joint position to estimate joint positions of body parts including the one body part and the other body part as a posture of the subject;
In the synthesis of the estimated first joint position and the estimated second joint position ,
determining at least one of a relative orientation and a relative scale between the first joint position and the second joint position based on a relative position between a shared joint, which is a joint shared between the one body part and the other body part, and a reference coordinate, which is a joint position of a joint different from the shared joint, which is a reference coordinate shared among a plurality of the images;
a fatigue estimation method for estimating fatigue, in which the joint of the second part and the joint of the first part are complemented by each other by superimposing the shared joint and the reference coordinates in a state in which at least one of a relative posture and a relative scale between the determined first joint position and the determined second joint position is applied. A program for causing a computer to execute the fatigue estimation method according to claim 6.

Images