JP7616366B2 - Force estimation device, force estimation method, and program - Google Patents

Description

The present invention relates to a force estimation device, a force estimation method and a program.

There is a known technology that uses sound waves to estimate muscle strength. For example, Non-Patent Document 1 discloses a device that estimates a user's strength by attaching two piezoelectric elements to the back of the user's hand, one of which serves as a speaker and the other as a microphone, sending and receiving ultrasonic waves, extracting features through frequency analysis, and applying a machine learning algorithm based on the extracted features.

Kubo, Yuki, et al. "AudioTouch: Minimally Invasive Sensing of Micro-Gestures via Active Bio-Acoustic Sensing." Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services. 2019.

Conventionally, when the device is used for a long period of time, the temperature of the piezoelectric element changes due to heat generation, body heat, etc., which causes the acoustic characteristics to change and results in drift (baseline fluctuation) errors in the force estimate.

The disclosed technology aims to improve the accuracy of force estimates.

The disclosed technology is a force estimation device for estimating force related to a muscle of an estimation target, the force estimation device comprising: an acoustic wave transmitting unit that transmits acoustic waves; an acoustic wave receiving unit that receives the acoustic waves; a force estimation unit that analyzes the received acoustic waves to calculate an estimate of the force of the muscle of the estimation target; and an offset correction unit that calculates a corrected estimate value by subtracting a set offset correction value from the estimate value , the force estimation device further comprising: a weakness determination unit that determines whether the muscle of the estimation target is in a weakened state; and an offset correction value setting unit that sets the offset correction value based on the force estimate in the time period determined to be in the weakened state, the acoustic wave receiving unit extracts audible sound waves from the received acoustic waves, and the weakness determination unit calculates an absolute value of the amplitude of the extracted audible sound waves, and if a value obtained by integrating the calculated absolute value over a fixed window width is equal to or less than a predetermined threshold value, the force estimation device determines that the fixed window width section was a section of a weakened state .

The accuracy of force estimates can be improved.

FIG. 2 is a functional configuration diagram of the force estimation device. 13 is a flowchart showing an example of the flow of a force estimation process. 11 is a flowchart showing an example of the flow of a correction value setting process. FIG. 2 is a diagram illustrating an example of a hardware configuration of a force estimation device. FIG. 11 is a diagram showing an example of a measurement result of a sound wave. FIG. 13 is a diagram showing an example of a result of an experiment on force estimation. FIG. 2 illustrates an example of a hardware configuration of a computer.

Hereinafter, an embodiment of the present invention (the present embodiment) will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applicable is not limited to the following embodiment.

The force estimation device according to this embodiment is a device that uses sound waves to estimate force related to the muscle strength of an estimation target, such as grip strength, leg strength, back strength, explosive power, etc. The estimation target is assumed to be a user or a human other than the user, but it does not have to be human as long as it has muscles, and can also be a non-hominidae animal such as a dog or monkey. The following explanation will be given on the assumption that the estimation target is a human.

(Functional configuration of the force estimation device)
1 is a functional block diagram of a force estimation device 10. The force estimation device 10 includes an ultrasonic wave transmitter 11, an ultrasonic wave receiver 12, a force estimation unit 13, a force relaxation determination unit 14, an offset correction value setting unit 15, and an offset correction unit 16.

The sound wave transmitting unit 11 transmits sound waves. The transmitted sound waves include ultrasonic waves and sound waves in the audible range, and are transmitted from a position close to the surface of the skin of the subject to be estimated.

The sound wave receiving unit 12 receives sound waves. Specifically, the sound wave receiving unit 12 receives ultrasonic waves and sound waves in the audible range, and extracts ultrasonic waves or sound waves in the audible range from the received sound waves.

The force estimation unit 13 estimates the force of the estimation target. Specifically, the force estimation unit 13 calculates a frequency feature from the received sound wave and inputs the calculated frequency feature into a regression model to obtain an estimate of the force.

The weakness determination unit 14 determines whether the estimation target is in a state of weakness based on the sound waves received by the sound wave receiving unit 12. Specifically, the weakness determination unit 14 calculates the absolute value of the amplitude of the sound waves in the audible range extracted by the sound wave receiving unit 12, and calculates a value (integrated muscle sound) by integrating the calculated absolute value over a certain window width. Here, in order to increase the signal-to-noise ratio of the integrated muscle sound, the weakness determination unit 14 may calculate the integrated muscle sound after applying a low-pass filter with a cutoff frequency of about 100 Hz. If the integrated muscle sound is equal to or less than a predetermined threshold, the weakness determination unit 14 determines that the section corresponding to the calculated certain window width is a section in which the estimation target is in a state of weakness (weakness section).

The offset correction value setting unit 15 sets the offset correction value based on the estimated force value in the relaxation section. Specifically, the offset correction value setting unit 15 calculates the average value of the estimated force value in the relaxation section, and sets the calculated average value as the offset correction value.

The offset correction unit 16 performs offset correction based on the force estimate calculated by the force estimation unit 13 and the offset setting value. Specifically, the offset correction unit 16 subtracts the offset setting value from the force estimate value to obtain the corrected estimate value.

(Operation of the Force Estimation Device)
Next, the operation of the force estimation device 10 will be described with reference to the drawings. The force estimation device 10 starts a force estimation process in response to a user's operation or the like. Note that it is assumed that an offset correction value has been set by an offset correction value setting process described later or based on the results of an experiment or the like when the force estimation process starts.

Figure 2 is a flowchart showing an example of the flow of the force estimation process. The sound wave transmitting unit 11 transmits sound waves (step S11). The sound waves transmitted here include ultrasonic waves. For example, the sound wave transmitting unit 11 uses a piezoelectric element to transmit sweep waves in the ultrasonic band (e.g., 20-40 kHz) at regular intervals (e.g., 20 ms intervals).

Next, the ultrasonic wave receiving unit 12 receives the ultrasonic waves (step S12). For example, the ultrasonic wave receiving unit 12 receives ultrasonic waves using a piezoelectric element different from the piezoelectric element used for transmission.

Next, the force estimation unit 13 estimates the force using a regression model based on the frequency feature of the sound wave (step S13). Specifically, the force estimation unit 13 performs a fast Fourier transform (FFT) on the sound wave received in step S12 for every fixed number of samples (e.g., 4,096 samples) to calculate a power spectrum, and the calculated power spectrum is treated as a vector indicating the frequency feature (feature vector).

The force estimation unit 13 then inputs the feature vector into a regression model to obtain an estimate of the force. The regression model is a trained model based on machine learning, such as SupportVectorRegression. This model utilizes the fact that when ultrasound propagates through the skin, the acoustic characteristics change if there is deformation of the skin in response to the amount of force applied by the subject.

Next, the offset correction unit 16 subtracts the offset correction value from the force estimate value to obtain
The corrected estimated value is output (step S14).

Next, we will explain the correction value setting process for setting the offset correction value. The correction value setting process is executed before the force estimation process starts.

Figure 3 is a flowchart showing an example of the flow of the correction value setting process. The sound wave transmitting unit 11 transmits sound waves (step S21). The sound waves transmitted here include ultrasonic waves and sound waves in the audible range. Next, the sound wave receiving unit 12 receives the sound waves and extracts sound waves in the audible range from the received sound waves (step S22).

Next, the force estimation unit 13 calculates the absolute value of the amplitude of the extracted sound wave in the audible range, and calculates a value (integrated muscle sound) by integrating the calculated absolute value over a fixed window width (step S23). The fixed window width is a time interval of a preset time (e.g., 1 second).

Next, the relaxation determination unit 14 determines whether the integrated muscle sound is equal to or less than a threshold value (step S24). The threshold value is set in advance based on the results of experiments, etc. If the relaxation determination unit 14 determines that the integrated muscle sound is not equal to or less than the threshold value (step S24: No), the force estimation device 10 returns to the processing of step S21 and executes the processing for the next section.

If the relaxation determination unit 14 determines that the integrated muscle sound is equal to or lower than the threshold (step S24: Yes), the force estimation unit 13 estimates the force using a regression model based on the frequency features of the sound waves in that section (relaxation section) (step S25). The process of step S25 is the same as step S13 of the force estimation process shown in FIG. 2.

In addition, the force estimation unit 13 may execute the processing of step S24 in advance before the determination by the force relaxation determination unit 14 and store the execution result.

Next, the offset correction value setting unit 15 sets the average value of the force estimate for that section (relaxation section) as the offset correction value (step S26).

(Hardware Configuration of Force Estimation Device)
4 is a diagram showing an example of a hardware configuration of the force estimation apparatus 10. The force estimation apparatus 10 includes a computer 101, a speaker 102, and a microphone 103.

The computer 101 is a PC (Personal Computer), a smartphone, a server device, etc. The speaker 102 is a device that transmits sound waves. The sound wave transmitting unit 11 is realized by the computer 101 controlling the speaker 102.

The microphone 103 is a device that receives sound waves. The sound wave receiving unit 12 is realized by the computer 101 controlling the microphone 103.

For example, the force estimation device 10, as in Non-Patent Document 1, estimates grip strength by attaching a piezoelectric element that acts as a speaker 102 and a microphone 103 to the back of the hand. For example, if the frequency characteristics of the piezoelectric element that corresponds to microphone 103 have sufficient gain in both the low frequency band of 100 Hz or less and the high frequency band of 20-40 kHz, a single piezoelectric element can be used for both. Also, if the piezoelectric element has frequency characteristics that cause significant attenuation in one of the frequency bands, a separate acoustic receiving device with different frequency characteristics that complements it can be attached, and the piezoelectric element and acoustic receiving device can function together as microphone 103.

(Sound wave measurement results)
Fig. 5 is a diagram showing an example of the measurement results of sound waves. Fig. 5 shows an example of measuring muscle sounds on the back of the hand with a microphone attached to the back of the hand. The "relaxed state" in Fig. 5 is a state in which no force is exerted, and the "gripping strength" section is a state in which force is exerted. When no force is exerted, the amplitude of the sound waves is small, but when force is exerted, the amplitude of the sound waves becomes large due to the influence of the muscle sounds.

From these measurement results, it can be said that the method of determining whether or not a person is in a state of relaxation based on the amplitude of sound waves is effective.

(Force estimation experiment results)
6 is a diagram showing an example of the experimental results of force estimation. In the experiment, the estimated values of grip force were calculated before and after the offset correction process. The dashed line 901 shows the estimated value before correction. The solid line 902 shows the corrected value. The dotted line 903 shows the estimated value after correction.

The correct value is the grip force value measured by the force sensor. The estimated value before correction estimated the grip force to be approximately 5 N in the section where the correct value was 0 N, and estimated the grip force to be approximately 10 N in the section where the correct value was 5 N.

In contrast, the estimated value after offset correction was calculated by subtracting the offset correction value of 5N from the force estimate, resulting in an estimated value close to the correct value. As a result of the offset correction value setting process, the force estimation device 10 determined that the section where the correct value was 0N was a force relaxation section, and set the offset correction value to 5N.

As a result of this experiment, the corrected estimate was closer to the correct value than the uncorrected estimate.

(Example of computer hardware configuration)
Each of the functional units of the force estimation device 10 described above can be realized by causing the computer 101 to execute a program describing the processing contents described in this embodiment. Note that this "computer" may be a physical machine or a virtual machine on the cloud. When a virtual machine is used, the "hardware" described here is virtual hardware.

The above program can be recorded on a computer-readable recording medium (such as a portable memory) and can be stored or distributed. The above program can also be provided via a network such as the Internet or e-mail.

Figure 7 is a diagram showing an example of the hardware configuration of the computer. The computer in Figure 7 has a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, etc., which are all connected to each other via a bus B.

The program that realizes the processing on the computer is provided by a recording medium 1001, such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 via the drive device 1000 into the auxiliary storage device 1002. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

When a program start instruction is received, the memory device 1003 reads out and stores the program from the auxiliary storage device 1002. The CPU 1004 realizes the functions related to the device according to the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network. The display device 1006 displays a GUI (Graphical User Interface) or the like according to a program. The input device 1007 is composed of a keyboard, a mouse, a button, a touch panel, or the like, and is used to input various operation instructions. The output device 1008 outputs the calculation results. The above computer may be equipped with a GPU (Graphics Processing Unit) or a TPU (Tensor processing unit) instead of the CPU 1004, or may be equipped with a GPU or TPU in addition to the CPU 1004. In that case, the processing may be shared and executed, for example, with the GPU or TPU executing processing that requires special calculations and the CPU 1004 executing other processing.

(Effects of this embodiment)
According to the force estimation device 10 of the present embodiment, the offset correction value is subtracted from the force estimate to obtain a corrected force estimate, thereby improving the accuracy of the force estimate.

The force estimation device 10 also determines whether the estimation target was in a relaxed state, and sets an offset correction value based on the force estimate in the time period in which the force estimate was determined to be in a relaxed state. This allows the error occurring in the force estimate to be set as an offset correction value, thereby further improving the force estimate.

(Summary of the embodiment)
This specification describes at least the force estimation device, the force estimation method, and the program described in the following sections.
(Section 1)
A force estimation device for estimating a force related to a muscle to be estimated, comprising:
A sound wave transmitting unit that transmits sound waves;
A sonic wave receiving unit that receives the sonic waves;
a force estimation unit that analyzes the received sound wave to calculate an estimate of the force of the estimation target;
and an offset correction unit that calculates a corrected estimated value by subtracting a set offset correction value from the estimated value.
Force estimator.
(Section 2)
a weakness determination unit that determines whether the estimation target is in a weakness state;
and an offset correction value setting unit that sets the offset correction value based on an estimated value of force in a time period determined to be the weak force state.
2. A force estimation device as claimed in claim 1.
(Section 3)
The sound wave receiving unit extracts sound waves in an audible range from the received sound waves,
The weakness determination unit calculates an absolute value of the amplitude of the extracted sound wave in the audible range, and when a value obtained by integrating the calculated absolute value over a certain window width is equal to or smaller than a predetermined threshold value, determines that the section of the certain window width is a section in a weakness state.
3. A force estimation device as defined in claim 2.
(Section 4)
the force estimator calculates an estimate of a force of the estimation target in a section determined to be in the relaxed state;
the offset correction value setting unit calculates an average value of the force estimate value in the section, and sets the calculated average value as the offset correction value;
4. The force estimation device according to claim 2 or 3.
(Section 5)
The force estimation unit calculates a power spectrum by performing a fast Fourier transform on the received sound wave for every fixed number of samples, and inputs the calculated power spectrum into a regression model as a vector indicating a frequency feature amount to obtain an estimate of the force.
5. A force estimation device according to any one of claims 1 to 4.
(Section 6)
1. A computer implemented force estimation method comprising:
transmitting sound waves;
receiving the acoustic waves;
Analysing the received sound waves to calculate an estimate of the force to be estimated;
and calculating a value obtained by subtracting the set offset correction value from the estimated value as a corrected estimated value.
Force estimation methods.
(Section 7)
A program for causing a computer to function as each unit in the force estimation device according to any one of claims 1 to 5.

Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and variations are possible within the scope of the gist of the present invention as described in the claims.

10 Force estimation device 11 Sound wave transmission unit 12 Sound wave reception unit 13 Force estimation unit 14 Force relaxation determination unit 15 Offset correction value setting unit 16 Offset correction unit 101 Computer 102 Speaker 103 Microphone 1000 Drive device 1001 Recording medium 1002 Auxiliary storage device 1003 Memory device 1004 CPU
1005 Interface device 1006 Display device 1007 Input device 1008 Output device

Claims (5)

A force estimation device for estimating a force related to a muscle to be estimated, comprising:
A sound wave transmitting unit that transmits sound waves;
A sonic wave receiving unit that receives the sonic waves;
a force estimation unit that analyzes the received sound wave to calculate an estimate of the force of the estimation target;
an offset correction unit that calculates a corrected estimated value by subtracting a set offset correction value from the estimated value ,
a weakness determination unit that determines whether the estimation target is in a weakness state;
and an offset correction value setting unit that sets the offset correction value based on an estimated value of force in a time period determined to be the weak force state,
The sound wave receiving unit extracts sound waves in an audible range from the received sound waves,
The weakness determination unit calculates an absolute value of the amplitude of the extracted sound wave in the audible range, and when a value obtained by integrating the calculated absolute value over a certain window width is equal to or smaller than a predetermined threshold value, determines that the section of the certain window width is a section in a weakness state.
Force estimator. the force estimator calculates an estimate of a force of the estimation target in a section determined to be in the relaxed state;
the offset correction value setting unit calculates an average value of the force estimate value in the section, and sets the calculated average value as the offset correction value;
The force estimation device according to claim 1 . The force estimation unit calculates a power spectrum by performing a fast Fourier transform on the received sound wave for every fixed number of samples, and inputs the calculated power spectrum into a regression model as a vector indicating a frequency feature amount to obtain an estimate of the force.
The force estimation device according to claim 1 or 2 . 1. A computer implemented force estimation method comprising:
a sonic wave transmitting step of transmitting a sonic wave;
a sonic wave receiving step of receiving the sonic wave;
a force estimating step of analyzing the received sound wave to calculate an estimate of a force to be estimated;
an offset correction step of calculating a corrected estimated value by subtracting a set offset correction value from the estimated value ,
a weakness determination step of determining whether the estimation target is in a weakness state;
and setting the offset correction value based on an estimated value of the force in the time period determined to be the weak force state.
In the sound wave receiving step, audible sound waves are extracted from the received sound waves;
In the step of determining whether or not the sound waves are weak, an absolute value of the amplitude of the extracted sound waves in the audible range is calculated, and if a value obtained by integrating the calculated absolute value over a certain window width is equal to or smaller than a predetermined threshold value, the section of the certain window width is determined to be a section in which the sound waves are weak.
Force estimation methods. A program for causing a computer to function as each unit in the force estimation device according to any one of claims 1 to 3 .

Images