JP7777010B2 - Measurement device, respiratory rate measurement method, and program - Google Patents

Description

This disclosure relates to a measurement device, a respiratory rate measurement method, and a program.

Patent Document 1 discloses a technology for measuring a subject's respiratory rate by detecting periodic vibrations that can be considered to represent breathing from a moving image of the subject.

JP 2014-171574 A

When conducting screening tests to assess health status by measuring respiratory rate at stores, event venues, etc., the shorter the measurement time required to measure each person's respiratory rate, the more efficiently it will be possible to measure the respiratory rates of a large number of people.

The technology disclosed in Patent Document 1 requires changing the time window for calculating the power spectrum depending on the minimum respiratory rate in the measurement range in order to detect periodic vibrations that can be considered respiration from the captured video of the subject. Furthermore, with the technology disclosed in Patent Document 1, the smaller the minimum respiratory rate in the measurement range, the larger the time window for calculating the power spectrum is set. As a result, with the technology disclosed in Patent Document 1, the smaller the minimum respiratory rate in the measurement range, the longer the measurement time required to measure the respiratory rate. Furthermore, with the technology disclosed in Patent Document 1, there is a risk that an incorrect respiratory rate will be calculated due to the influence of noise contained in the video. Therefore, one aspect of the present disclosure aims to provide a measurement device, respiratory rate measurement method, and program that can output a respiratory rate with guaranteed reliability while reducing measurement time.

A measurement device according to one aspect of the present disclosure includes an imaging unit that captures an image of a living body to acquire a moving image; a respiratory rate calculation unit that calculates the respiratory rate by detecting the oscillation period of a respiratory signal contained in the time changes of multiple pixel values contained in the moving image; a reliability calculation unit that calculates the reliability of the respiratory rate from the results of comparing the respiratory signal with a noise signal contained in the time changes; and an output unit that outputs the respiratory rate if the reliability is equal to or greater than a threshold.

A respiratory rate measurement method according to one aspect of the present disclosure includes the steps of capturing an image of a living body to obtain a moving image, calculating the respiratory rate by detecting the oscillation period of a respiratory signal contained in the time-varying changes in a plurality of pixel values contained in the moving image, calculating the reliability of the respiratory rate from the results of comparing the respiratory signal with a noise signal contained in the time-varying changes, and outputting the respiratory rate if the reliability is equal to or greater than a threshold.

A program according to one aspect of the present disclosure causes a computer to perform the following functions: capturing an image of a living body to acquire a moving image; calculating a respiratory rate by detecting the oscillation period of a respiratory signal contained in the time-varying changes in multiple pixel values contained in the moving image; calculating the reliability of the respiratory rate from the results of comparing the respiratory signal with a noise signal contained in the time-varying changes; and outputting the respiratory rate if the reliability is equal to or greater than a threshold value.

FIG. 1 is a diagram showing an example of how the measurement device is used. 1 is a block diagram showing an example of the configuration of a measurement device according to a first embodiment. 4 is a flowchart showing an example of the operation of the measurement device according to the first embodiment. 1 shows an example of images constituting a moving image. 10 is a graph showing an example of a time-series signal indicating a change in moving speed over time. 5B is a graph showing an example of a change over time in the autocorrelation value of the time-series signal illustrated in FIG. 5A. 10 is a graph showing an example of a change over time in an autocorrelation value for a time-series signal indicating a change over time in a moving speed. 6B is a graph showing an example of the power spectrum of the time-series signal illustrated in FIG. 6A. 4 is a flowchart illustrating an example of the operation of the measurement device according to the first embodiment, following FIG. 3 . 10A and 10B are diagrams illustrating an example of a time-series signal generated from moving images acquired by capturing images of a living body at different measurement times, and a power spectrum. FIG. 10 is a block diagram showing an example of the configuration of a measurement device according to a second embodiment. 10 is a flowchart showing an example of the operation of the measurement device according to the second embodiment. FIG. 10 is a block diagram showing an example of the configuration of a measurement device according to a third embodiment. 10 is a flowchart showing an example of the operation of the measurement device according to the third embodiment. FIG. 1 is a diagram showing an example of an image including images of multiple faces.

(First embodiment)
A first embodiment will be described with reference to Figures 1 to 8. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant explanations will be omitted.

Figure 1 is a diagram showing an example of how the measuring device 100 according to this embodiment is used. The measuring device 100 includes an imaging unit 101. The measuring device 100 calculates the respiratory rate from an image acquired by the imaging unit 101. For example, the measuring device 100 may be a PC (Personal Computer), a smartphone, a tablet terminal, a terminal dedicated to measuring the respiratory rate, etc.

The imaging unit 101 captures an image of the living body 102 to obtain a moving image 211 (see Figure 2). The moving image 211 includes an image of the face and chest of the living body 102. For example, the imaging unit 101 is configured with a CCD (Charged Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) image sensor. The imaging unit 101 may also be configured with a camera image sensor that includes RGB (Red Green Blue) filters.

Figure 2 is a block diagram showing an example of the configuration of the measurement device 100. The measurement device 100 includes an imaging unit 101, a storage unit 201, and a control unit 202.

The imaging unit 101 captures images of the living body 102 during the measurement time to obtain a moving image 211.

The storage unit 201 is a recording medium capable of recording various data, programs, etc., and is composed of, for example, a hard disk, SSD (Solid State Drive), semiconductor memory, etc.

The control unit 202 executes various processes in accordance with the programs and data stored in the memory unit 201. The control unit 202 is realized, for example, by a processor such as a CPU (Central Processing Unit).

The control unit 202 includes a region identification unit 203, a signal generation unit 204, a respiratory rate calculation unit 205, a reliability calculation unit 206, a determination unit 207, an output unit 208, and a measurement time setting unit 209.

The region identification unit 203 identifies a region 212 containing an image of the chest of the living body 102 from each image constituting the moving image 211.

The signal generating unit 204 generates a time series signal 213 that indicates the temporal changes in multiple pixel values contained in the moving image 211. The multiple pixel values are pixel values within an area 212 that includes an image of the chest of the living body 102. The time series signal 213 includes a respiratory signal 213a and a noise signal 213b. The respiratory signal 213a indicates the temporal changes in pixel values due to respiratory movements of the living body 102. The noise signal 213b indicates the temporal changes in pixel values due to noise caused by lighting, etc.

The respiratory rate calculation unit 205 calculates the respiratory rate 214 by detecting the oscillation period of the respiratory signal 213a contained in the temporal changes in the values of multiple pixels contained in the moving image 211.

The reliability calculation unit 206 calculates the reliability 215 of the respiratory rate 214 from the comparison result between the respiratory signal 213a and the noise signal 213b.

The determination unit 207 determines whether the reliability 215 is greater than or equal to a threshold value.

If the reliability 215 is greater than or equal to a threshold, the output unit 208 outputs the respiratory rate 213.

The measurement time setting unit 209 sets the measurement time. Furthermore, if the reliability 215 is lower than the threshold, the measurement time setting unit 209 extends the measurement time.

Figure 3 is a flowchart showing an example of the operation of the measurement device 100 according to this embodiment. In this example, when the image capturing unit 101 is activated, the control unit 202 starts the processing of step S301 illustrated in Figure 3. The imaging range of the image capturing unit 101 is assumed to include the face and chest of the living body 102. Furthermore, at the time the processing of step S301 is started, the measurement time setting unit 209 is assumed to have set the measurement time as an initial value. For example, the initial value of the measurement time is 3 seconds.

In step S301, the imaging unit 301 captures images of the living body 102 for the measurement time set by the measurement time setting unit 209, acquires a moving image 211, and stores the acquired moving image 211 in the storage unit 201. For example, the imaging unit 101 captures images of the living body 102 for the measurement time at 30 to 60 fps (frames per second) and acquires the moving image 211. The moving image 211 includes an image of the face and chest of the living body 102.

In step S302, the region identification unit 203 determines whether or not an image of the face of the living body 102 can be detected from each image constituting the moving image 211. Each image constituting the moving image 211 is an image of each frame constituting the moving image 211. For example, the region identification unit 203 detects feature points such as the eyes, nose, and mouth from each image constituting the moving image 211, and detects an image of the face based on the detected feature points. For example, the region identification unit 203 detects an image of the face using a method such as a cascade detector, a support vector machine, or a CNN (Convolutional Neural Network).

If a face image is not detected in step S302, the control unit 202 returns the process to step S301. In other words, the imaging unit 301 repeats capturing images of the living body 102 for the measurement time until a face image of the living body 102 is detected from each image constituting the moving image 211.

If a face image is detected in step S302, in step S303 the region identification unit 203 identifies a region 212 including an image of the chest of the living body 102 based on the position of the face image detected in step S302. For example, region 212 is a region of a predetermined size located below the position of the region including the face image in the +Y direction (see Figure 4).

Figure 4 shows an example of an image 401 constituting the moving image 211. The region 212 shown in Figure 4 is a region of a predetermined size below the position of the face image 402 included in the image 401. For example, the region of a predetermined size is a rectangular region whose length in the vertical Y direction is the same as the region including the face image 402, and whose length in the horizontal X direction is twice as long as the region including the face image 402. For example, as shown in Figure 4, the region 212 includes an image of the chest of the living body 102. Note that the region 212 only needs to include a portion of the chest of the living body 102, and does not have to include the entire chest of the living body 102.

When the living organism 102 inhales, the rib cage expands. Furthermore, when the living organism 102 exhales, the rib cage contracts. In other words, when the living organism 102 breathes, it performs a respiratory movement that moves the chest and shoulders up and down. Therefore, when the imaging unit 101 captures an image of the living organism 102 and acquires a moving image 211, the pixel values of an area 212 included in the moving image 211 change over time due to the respiratory movement.

Therefore, in the subsequent step S304, the signal generation unit 204 generates a time series signal 213 that indicates the temporal change in pixel values of multiple pixels within the region 212. The temporal change is indicated by the fluctuation in the multiple pixel values included in the video 211, or the fluctuation in a velocity vector determined from the multiple pixel values included in the video 211.

For example, the signal generation unit 204 calculates a representative value of the pixel values of any of the R (red), G (green), or B (blue) pixels within the region 212 for each image that constitutes the moving image 211. The representative value may be the average, median, mode, etc.

Alternatively, the signal generation unit 204 may calculate, for each image constituting the moving image 211, a representative value by multiplying the average pixel values of each R, G, and B pixel in the region 212 by a weighting coefficient and adding the resulting values.

The signal generating unit 204 then generates a time-series signal 213 that indicates the change over time of the calculated representative value. Alternatively, the signal generating unit 204 may perform noise suppression processing on the signal that indicates the change over time of the calculated representative value for each predetermined time, and generate the noise-suppressed signal as the time-series signal 213. For example, the signal generating unit 204 may input the signal that indicates the change over time of the calculated representative value for each predetermined time into an autocorrelation function to calculate the autocorrelation value for each predetermined time. The signal generating unit 204 may then generate a signal that indicates the change over time of the autocorrelation value as the time-series signal 213. Alternatively, the signal generating unit 204 may generate the time-series signal 213 by inputting the signal that indicates the change over time of the calculated representative value for each predetermined time into a low-pass filter that extracts frequency components below a predetermined frequency. This allows the signal generating unit 204 to generate a time-series signal 213 from which frequency components above the predetermined frequency have been removed.

Alternatively, the signal generation unit 204 may calculate a vector V(i) indicating the movement speed in the Y direction for each image constituting the moving image 211. The vector V(i) indicates the movement speed in the Y direction from the time when the image of frame i-1 constituting the moving image 211 is captured to the time when the image of frame i is captured.

For example, the signal generation unit 204 calculates a vector V(i) indicating a movement speed in the Y direction for a plurality of pixels in a region 212 included in an image of frame i constituting the moving image 211 using a method such as block matching or a gradient method. Specifically, the signal generation unit 204 calculates a representative value of the vector V(i, Pix _x,y ) for all pixels Pix _x,y in the region 212 as the vector V(i) for the image of frame i constituting the moving image 211. The pixel Pix _x,y indicates a pixel included in the region 212 whose coordinate values are x and y. The vector V(i, Pix _x,y ) indicates the movement speed in the Y direction of the pixel Pix _x,y in the region 212 included in the image of frame i. For example, the representative value of the vector V(i, Pix _x,y ) is the average value of the vector V(i, Pix _x,y ) for all pixels Pix _x,y in the region 212.

Then, the signal generation unit 204 generates a time-series signal 213 that indicates the change over time of the vector V(i). Alternatively, the signal generation unit 204 may perform noise suppression processing on the signal that indicates the change over time of the vector V(i) and generate the noise-suppressed signal as the time-series signal 213.

For example, the signal generating unit 204 may input a signal indicating the time change in the movement speed in the Y direction for each predetermined time into an autocorrelation function to calculate the autocorrelation value for each predetermined time. The signal generating unit 204 may then generate a signal indicating the time change in the autocorrelation value as the time series signal 213. Alternatively, the signal generating unit 204 may generate the time series signal 213 by inputting the signal indicating the time change in the movement speed in the Y direction into a low-pass filter that extracts frequency components below a predetermined frequency.

Figure 5A is a graph showing an example of a time-series signal indicating changes in movement speed over time. In Figure 5A, the horizontal axis represents time, and the vertical axis represents signal values indicating movement speed. Time-series signal 501 is time-series signal 213 indicating movement speed in the Y direction between frames included in moving image 211.

Figure 5B is a graph showing an example of the change over time in the autocorrelation value for the time-series signal 501 shown in Figure 5A. In Figure 5B, the horizontal axis represents time, and the vertical axis represents the autocorrelation value. Time-series signal 502 is time-series signal 213 that shows the change over time in the autocorrelation value for the time-series signal 501 shown in Figure 5A. As shown in Figure 5B, noise contained in the time-series signal 501 shown in Figure 5A is suppressed and smoothed in time-series signal 502.

In the following step S305, the signal generation unit 204 performs frequency analysis on the time series signal 213 and calculates the power spectrum of the time series signal 213.

In step S306, the respiratory rate calculation unit 205 calculates the respiratory rate 214 from the vibration period of the respiratory signal 213a included in the time-series signal 213. Specifically, the respiratory rate calculation unit 205 calculates the respiratory rate 214 from the maximum peak frequency in the power spectrum calculated in step S305. More specifically, the respiratory rate calculation unit 205 calculates the respiratory rate by taking the maximum peak frequency within the target frequency range in the power spectrum calculated in step S305 as the respiratory frequency. For example, the target frequency range is a frequency range of 2 to 200 breaths per minute.

In step S307, the reliability calculation unit 206 calculates the magnitude of the respiratory signal 213a from the time-series signal 213. For example, the magnitude of the respiratory signal 213a is the power spectrum value of the maximum peak frequency in the time-series signal 213. In other words, the magnitude of the respiratory signal 213a is the power spectrum value of the maximum peak frequency within the target frequency range in the power spectrum calculated in step S306.

Alternatively, the magnitude of the respiratory signal 213a may be a statistical value of power spectrum values within a predetermined range including the maximum peak frequency in the time series signal 213. For example, the predetermined range is a range from a frequency 0.1 Hz lower than the maximum peak frequency to a frequency 0.1 Hz higher than the maximum peak frequency. For example, the statistical value of power spectrum values within the predetermined range is the sum of the power spectrum values within the predetermined range.

In step S308, the reliability calculation unit 206 calculates the magnitude of the noise signal 213b from the time-series signal 213. For example, the magnitude of the noise signal 213b is the power spectrum value of the second or lower peak frequency in the time-series signal 213. In other words, the magnitude of the noise signal 213b is the power spectrum value of the second or lower peak frequency in the power spectrum value calculated in step S306.

For example, the magnitude of noise signal 213b is the power spectrum value of the second largest peak frequency in time series signal 213. Alternatively, the magnitude of noise signal 213b may be the power spectrum value showing the third largest peak in time series signal 213.

Alternatively, if the magnitude of the respiratory signal 213a is a statistical value of power spectral values within a predetermined range including the maximum peak frequency in the time series signal 213, the magnitude of the noise signal 213b may be a statistical value of power spectral values outside the predetermined range. For example, the statistical value of power spectral values outside the predetermined range is the sum of the power spectral values outside the predetermined range. Note that the reliability calculation unit 206 may calculate the magnitude of the noise signal 213b by combining multiple calculation methods.

In step S309, the reliability calculation unit 206 calculates the reliability 215 from the ratio between the magnitude of the respiratory signal 213a calculated in step S307 and the magnitude of the noise signal 213b calculated in step S308. For example, the reliability calculation unit 206 calculates the ratio between the magnitude of the respiratory signal 213a and the magnitude of the noise signal 213b as the reliability 215. The reliability 215 indicates the signal-to-noise ratio (SNR). In other words, a larger value of the reliability 215 indicates that, in the temporal changes in the multiple pixel values included in the moving image 211 indicated by the time-series signal 213, the influence of the temporal changes in pixel values due to the respiratory movement of the living body 102 is greater than the influence of the temporal changes in pixel values due to noise. The control unit 202 then proceeds to step S701, an example of which is shown in FIG. 7.

Figure 6A is a graph showing an example of the change over time in the autocorrelation value of a time-series signal that indicates the change over time in moving speed. In Figure 6A, the horizontal axis represents time, and the vertical axis represents the autocorrelation value. Time-series signal 601 is time-series signal 213 that indicates the autocorrelation value of the moving speed in the Y direction for each predetermined time period for the same position within region 212.

Figure 6B is a graph showing an example of the power spectrum of the time-series signal 601 shown in Figure 6A. In Figure 6B, the horizontal axis represents frequency, and the vertical axis represents power spectrum values. Power spectrum 602 shows the power spectrum of the time-series signal 601. Power spectrum value P611 is the magnitude of the power spectrum of the maximum peak frequency in the time-series signal 601. In other words, power spectrum value P611 indicates the magnitude of the respiratory signal 213a.

Power spectrum value P612 is the magnitude of the second largest power spectrum in the time series signal 213 illustrated in Figure 6A. In other words, power spectrum value P612 indicates the magnitude of noise signal 213b.

The reliability calculation unit 206 calculates the reliability 215 from the ratio between the power spectrum value P611 and the power spectrum value P612. For example, the reliability calculation unit 206 calculates the ratio between the power spectrum value P611 and the power spectrum value P612 as the reliability 215.

Next, we will continue to explain the operation of the measurement device 100 with reference to Figure 7.

In step S701, the determination unit 207 determines whether the reliability 215 is equal to or greater than a threshold. The reliability 215 being equal to or greater than the threshold indicates that, in the temporal changes in the pixel values included in the moving image 211 indicated by the time-series signal 213, the influence of the temporal changes in the pixel values due to the respiratory movement of the living body 102 is sufficiently greater than the influence of the temporal changes in the pixel values due to noise. On the other hand, if the reliability 215 is lower than the threshold, there is a risk that, in the temporal changes in the pixel values included in the moving image 211 indicated by the time-series signal 213, the influence of the temporal changes in the pixel values due to noise is relatively greater.

If the reliability 215 is lower than the threshold in step S701, the measurement time setting unit 209 extends the measurement time by a predetermined time in step S702. For example, the predetermined time is the time it takes for the imaging unit 101 to acquire one frame of image. The control unit 202 then returns the process to step S301 illustrated in FIG. 3. In other words, the measurement time setting unit 209 extends the measurement time, continues imaging the living body 102, and repeatedly calculates the respiratory rate 214 until the reliability 215 becomes equal to or greater than the threshold.

On the other hand, if the reliability 215 is equal to or greater than the threshold in step S701, the output unit 208 outputs the respiratory rate 214 calculated in step S306 illustrated in FIG. 3 in step S703. For example, the output unit 208 displays text indicating the respiratory rate 214 on an LCD display or the like. Alternatively, for example, the output unit 208 may output audio indicating the respiratory rate 214 from a speaker. The control unit 202 then terminates the process of measuring the respiratory rate 214.

In other words, when the influence of the temporal change in pixel values due to the respiratory movement of the living body 102 in the temporal change in the values of multiple pixels included in the moving image 211, as indicated by the time-series signal 213, is sufficiently greater than the influence of the temporal change in pixel values due to noise, the output unit 208 outputs the respiratory rate 214. This allows the measuring device 100 according to this embodiment to guarantee the reliability of the output respiratory rate 214.

Figure 8 shows an example of time series signals and power spectra generated from moving images 211 acquired by capturing images of a living body 102 at different measurement times. In Figure 8, for time series signals 801, 803, and 805, the horizontal axis represents time and the vertical axis represents the autocorrelation value. Also in Figure 8, for power spectrum 802, 804, and 806, the horizontal axis represents frequency and the vertical axis represents the power spectrum value.

Time series signal 801 represents time series signal 213 generated from video image 211 acquired at measurement time T1. Power spectrum 802 represents the power spectrum of time series signal 801.

For example, in step S307, the reliability calculation unit 206 calculates the power spectrum value P811 of the maximum peak frequency in the time-series signal 801 as the magnitude of the respiratory signal 213a. Furthermore, in step S308, the reliability calculation unit 206 calculates the power spectrum value P812 of the second peak frequency in the time-series signal 801 as the magnitude of the noise signal 213b. In this case, in step S309, the reliability calculation unit 206 calculates the reliability 215 from the ratio of the power spectrum value P811 to the power spectrum value P812. If the reliability 215 is lower than the threshold, in step S702, the measurement time setting unit 209 extends the measurement time to time T2, which is longer than time T1.

Time series signal 803 represents time series signal 213 generated from video image 211 acquired at measurement time T2. Power spectrum 804 represents the power spectrum of time series signal 803.

For example, in step S307, the reliability calculation unit 206 calculates the power spectrum value P813 of the maximum peak frequency in the time-series signal 803 as the magnitude of the respiratory signal 213a. Furthermore, in step S308, the reliability calculation unit 206 calculates the power spectrum value P814 of the second peak frequency in the time-series signal 803 as the magnitude of the noise signal 213b. In this case, in step S309, the reliability calculation unit 206 calculates the reliability 215 from the ratio of the power spectrum value P813 to the power spectrum value P814. If the reliability 215 is lower than the threshold, in step S702, the measurement time setting unit 209 extends the measurement time to time T3, which is longer than time T2.

Time series signal 805 represents time series signal 213 generated from video image 211 acquired at measurement time T3. Power spectrum 806 represents the power spectrum of time series signal 805.

For example, in step S307, the reliability calculation unit 206 calculates the power spectrum value P815 of the maximum peak frequency in the time series signal 805 as the magnitude of the respiratory signal 213a. Furthermore, in step S308, the reliability calculation unit 206 calculates the power spectrum value P816 of the second peak frequency in the time series signal 805 as the magnitude of the noise signal 213b. In this case, in step S309, the reliability calculation unit 206 calculates the reliability 215 from the ratio between the power spectrum value P815 and the power spectrum value P816.

If the reliability 215 is equal to or greater than the threshold, in step S703 the output unit 208 outputs the respiratory rate 214 calculated from the maximum peak frequency PEFQ in the time-series signal 805. The control unit 202 then terminates the process of measuring the respiratory rate 214. In other words, the measurement device 100 changes the measurement time according to the reliability 215, and outputs the respiratory rate 214 if the reliability 215 is equal to or greater than the threshold. This allows the measurement device 100 to output the respiratory rate 214 with guaranteed reliability while reducing the measurement time.

(Modification 1 of the first embodiment)
As a first modification of the measuring device 100 according to the present embodiment, when the measurement time setting unit 209 extends the measurement time, the control unit 202 may execute the processes of steps S305 to S309 and the process of step S701 each time a predetermined number of frames of images (two or more frames) are acquired. That is, when the measurement time setting unit 209 extends the measurement time, each time a predetermined number of frames of images are acquired, the respiratory rate calculation unit 205 may calculate the respiratory rate 214, the reliability calculation unit 206 may calculate the reliability 215, and the determination unit 207 may determine whether the reliability 215 is equal to or greater than a threshold. As a result, when the measurement time setting unit 209 extends the measurement time, the measuring device 100 according to this first modification can reduce the processing time compared to when the respiratory rate 214 and the reliability 215 are calculated each time one frame of images is acquired.

(Modification 2 of the First Embodiment)
In a second modification of the measuring device 100 according to this embodiment, the measurement time setting unit 209 may set at least one selected from a group consisting of a lower limit and an upper limit of the measurement time. For example, the lower limit of the measurement time is 3 seconds. Also, for example, the upper limit of the measurement time is 20 seconds.

Let us say that the measurement time setting unit 209 has set an upper limit for the measurement time. In this case, if the imaging unit 101 continues to image the living body 102 for the measurement time, which is the upper limit, and the reliability 215 does not become equal to or greater than the threshold, the control unit 202 ends the process of measuring the respiratory rate 214. Furthermore, the output unit 208 outputs a message indicating an error. For example, the output unit 208 displays a message such as "The respiratory rate could not be measured. Please measure again" on an LCD display or the like. Alternatively, the output unit 208 may output a sound indicating the message from a speaker.

Second Embodiment
The second embodiment will be described with reference to Figures 9 and 10. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant explanations will be omitted. Configurations and processes having substantially the same functions as those of other embodiments will be denoted by the same reference numerals, and explanations will be omitted, and differences from other embodiments will be described.

Figure 9 is a block diagram showing an example of the configuration of a measurement device 100 according to this embodiment. The difference between the measurement device 100 shown in Figure 9 and the measurement device 100 shown in Figure 2 is that the measurement device 100 shown in Figure 9 includes a measurement time setting unit 901 instead of the measurement time setting unit 209.

The measurement time setting unit 901 sets the lower and upper limits of the measurement time to the same value. In other words, the measurement time setting unit 901 does not extend the measurement time.

The respiratory rate calculation unit 205 in this embodiment terminates the process of calculating the respiratory rate 214 if the reliability 215 is lower than the threshold value.

In this embodiment, the output unit 208 outputs a message indicating an error if the reliability 215 is lower than the threshold.

Figure 10 is a flowchart showing an example of the operation of the measurement device 100 according to this embodiment.

When the reliability calculation unit 206 calculates the reliability 215 from the ratio between the magnitude of the breathing signal 213a and the magnitude of the noise signal 213b in step S309 illustrated in FIG. 3, the determination unit 207 determines in step S1001 whether the reliability 215 is equal to or greater than a threshold value.

If the reliability 215 is equal to or greater than the threshold in step S1001, the output unit 208 outputs the respiratory rate 214 calculated in step S306 in step S1002. The processing of steps S1001 and S1002 illustrated in FIG. 10 is similar to the processing of steps S701 and S703 illustrated in FIG. 7, and therefore detailed description thereof will be omitted.

On the other hand, if the reliability 215 is lower than the threshold in step S1001, the output unit 208 outputs a message indicating an error in step S1003. For example, the output unit 208 displays a message such as "Respiration rate could not be measured. Please measure again" on an LCD display or the like. Alternatively, the output unit 208 may output a sound indicating the message from a speaker.

In order for the measuring device 100 to measure the respiratory rate 214 of the living body 102, the living body 102 needs to remain within the imaging range of the imaging unit 101. Therefore, the longer the measurement time, the longer the living body 102 needs to remain within the imaging range of the imaging unit 101. However, the measuring device 100 according to this embodiment does not extend the measurement time, and outputs a message indicating an error if the reliability 215 is lower than the threshold value. As a result, the measuring device 100 according to this embodiment can limit the time required to measure the respiratory rate by not extending the measurement time. Furthermore, the measuring device 100 according to this embodiment can reduce the burden on the living body 102 by not extending the measurement time.

(Third embodiment)
The third embodiment will be described with reference to Figures 11 to 13. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant explanations will be omitted. Configurations and processes having substantially the same functions as those of other embodiments will be denoted by the same reference numerals, and explanations will be omitted, and differences from other embodiments will be described.

Figure 11 is a block diagram showing an example of the configuration of a measurement device 100 according to this embodiment. The difference between the measurement device 100 shown in Figure 11 and the measurement device 100 shown in Figure 2 is that the measurement device 100 shown in Figure 11 includes a distance calculation unit 1101.

When the video 211 contains multiple facial images, the distance calculation unit 1101 calculates the distance 1111 between the multiple facial images.

The output unit 208 in this embodiment outputs a message indicating an error if the minimum value of the distance 1111 between multiple face images is less than or equal to a predetermined distance.

The respiratory rate calculation unit 205 in this embodiment terminates the process of calculating the respiratory rate 214 if the minimum value of the distance 1111 is equal to or less than a predetermined distance.

Figure 12 is a flowchart showing an example of the operation of the measurement device 100 according to this embodiment. In this example, when the measurement device 100 is started, the control unit 202 starts the processing of step S301 illustrated in Figure 12. Furthermore, at the time when the control unit 202 starts the processing of step S301, the measurement time setting unit 209 is set to an initial measurement time. The processing of steps S1201 to S1202 is similar to the processing of steps S301 to S302 illustrated in Figure 3, and therefore detailed description will be omitted.

If a face image is detected in step S1202, in step S1203 the distance calculation unit 1101 determines whether or not multiple face images are included in the video 211. For example, if multiple face images are included in at least one image constituting the video 211, the distance calculation unit 1101 determines that multiple face images are included in the video 211.

If the video 211 does not contain multiple facial images in step S1203, the control unit 202 proceeds to step S303, an example of which is shown in Figure 3.

On the other hand, if the video 211 contains multiple facial images in step S1203, the distance calculation unit 1101 calculates the distance 1111 between the multiple facial images in step S1204.

If the minimum value of the distance 1111 between multiple facial images is equal to or less than a predetermined distance, the region identification unit 203 may erroneously identify a region that includes part of the regions 212 of multiple adjacent living organisms 102 as the region 212 of one living organism 102. In this case, the signal generation unit 204 may erroneously generate a signal indicating the temporal change in pixel values due to the respiratory movement of multiple adjacent living organisms 102 as a time-series signal 213 for one living organism 102.

Therefore, in step S1205, the distance calculation unit 1001 determines whether the minimum value of the distance 1111 between multiple face images is less than or equal to a predetermined distance.

If the minimum value of the distance 1111 between the multiple facial images is equal to or less than the predetermined distance in step S1205, the output unit 208 outputs a message indicating an error in step S1206. For example, the output unit 208 displays a message such as "Please keep an appropriate distance from the person next to you" on an LCD display or the like. Alternatively, the output unit 208 may output a sound indicating the message from a speaker. Then, the control unit 202 ends the process of measuring the respiratory rate 214. In other words, if the minimum value of the distance 1111 between the multiple facial images is equal to or less than the predetermined distance, the respiratory rate calculation unit 205 ends the process of calculating the respiratory rate 214.

On the other hand, if the minimum value of the distance 1111 between multiple face images is longer than the predetermined distance in step S1205, the control unit 202 executes the processes of steps S303 to S309 illustrated in FIG. 3 for each image of the living body 102 included in the video 211. Then, in step S701 illustrated in FIG. 7, the control unit 202 determines whether the reliability 215 for each image of the living body 102 included in the video 211 is greater than or equal to a threshold value.

For images of living organisms 102 whose reliability 215 is equal to or greater than the threshold, the output unit 208 outputs the respiratory rate 214 in step S703 illustrated in FIG. 7. For images of living organisms 102 whose reliability 215 is lower than the threshold, the measurement time setting unit 209 extends the measurement time in step S702 illustrated in FIG. 7. The control unit 202 then returns the process to step S301 illustrated in FIG. 3. In other words, the imaging unit 101 continues to capture images of living organisms 102 whose reliability 215 is lower than the threshold, and the control unit 202 continues to measure the respiratory rate 214 for those living organisms 102.

Figure 13 is a diagram showing an example of image 1310 including face images 1301 to 1306. Distance 1111a is the distance between face image 1301 and face image 1302. Distance 1111b is the distance between face image 1302 and face image 1303. Distance 1111c is the distance between face image 1303 and face image 1304. Distance 1111d is the distance between face image 1304 and face image 1305. Distance 1111e is the distance between face image 1305 and face image 1306.

Area 1311 is an area of a predetermined size below face image 1302 and includes an image of the chest of one living organism 102. Area 1312 is an area of a predetermined size below face image 1303 and includes an image of the chest of another living organism 102. Because areas 1311 and 1312 partially overlap, there is a risk that the signal generating unit 204 may erroneously generate a signal indicating the temporal change in pixel values due to the respiratory movement of multiple adjacent living organisms 102 as a time series signal 213 for one living organism 102.

In other words, if the distance between face image 1302 and face image 1303 is less than a predetermined distance, the signal generating unit 204 may erroneously generate a signal indicating the temporal change in pixel values due to the respiratory movements of multiple adjacent living organisms 102 as a time series signal 213 for one living organism 102. Therefore, when multiple face images are included in the moving image 211, the face images must be sufficiently far apart in order for the signal generating unit 204 to generate a time series signal 213 for each living organism 102.

Therefore, for example, if distance 1111b is the smallest value among distances 1111a to 1111e and distance 1111b is equal to or less than a predetermined distance, the respiratory rate calculation unit 205 terminates the process of calculating the respiratory rate 214. Furthermore, if distance 1111b is equal to or less than the predetermined distance, the output unit 208 outputs a message indicating an error.

As described above, when the moving image 211 contains multiple facial images, the measuring device 100 according to this embodiment can measure the respiratory rate 214 for each living organism 102 among multiple living organisms, with the facial images being sufficiently spaced apart. Therefore, even when the moving image 211 contains multiple facial images, the measuring device 100 according to this embodiment can output the respiratory rate for each living organism 102 with guaranteed reliability while reducing the measurement time.

(Modification of the third embodiment)
As a modified example of the measuring device 100 according to this embodiment, the measuring device 100 may include a measurement time setting unit 901 instead of the measurement time setting unit 209. In this case, the respiratory rate calculation unit 205 according to this modified example terminates the process of calculating the respiratory rate when the reliability 215 is lower than a threshold value. Furthermore, the output unit 208 according to this modified example outputs a message indicating an error when the reliability 215 is lower than a threshold value. As a result, when the moving image 211 includes images of multiple faces, the measuring device 100 according to this modified example does not extend the measurement time, thereby limiting the time required to measure the respiratory rate and reducing the burden on the multiple living organisms 102.

The processes performed in the above embodiments are not limited to the processing modes exemplified in each embodiment. The functional blocks described above may be implemented using either logic circuits (hardware) formed in an integrated circuit or the like, or software using a CPU. The processes performed in the above embodiments may be executed by multiple computers. For example, some of the processes executed by the functional blocks of the control unit 202 of the measuring device 100 may be executed by another computer, or all of the processes may be shared and executed by multiple computers.

This disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. Furthermore, new technical features can be created by combining the technical means disclosed in each embodiment.

100 Measurement device, 101 Imaging unit, 102 Living body, 201 Memory unit, 202 Control unit, 203 Region identification unit, 204 Signal generation unit, 205 Respiratory rate calculation unit, 206 Reliability calculation unit, 207 Determination unit, 208 Output unit, 209 Measurement time setting unit, 211 Moving image, 212 Region, 213 Time series signal, 215 Reliability, 301 Imaging unit, 401 Image, 402 Face image, 501 Time series signal, 502 Time series signal, 601 Time series signal, 602 Power spectrum, 801 Time series signal, 802 Power spectrum, 803 Time series signal, 804 Power spectrum, 805 Time series signal, 806 Power spectrum, 901 Measurement time setting unit, 1001 Distance calculation unit, 1101 Distance calculation unit, 1111 distance, 1111a-1111e distances, 1301-1306 face images, 1310 image, 1311 region, 1312 region, P611 power spectrum value, P612 power spectrum value, P811 power spectrum value, P812 power spectrum value, P813 power spectrum value, P814 power spectrum value, P815 power spectrum value, P816 power spectrum value, PEFQ maximum peak frequency

Claims (10)

an imaging unit that captures an image of a living body and acquires a moving image;
a respiratory rate calculation unit that calculates a respiratory rate by detecting an oscillation period of a respiratory signal included in a time change of a plurality of pixel values included in the moving image;
a reliability calculation unit that calculates the reliability of the respiratory rate based on a comparison result between the respiratory signal and a noise signal included in the time change;
an output unit that outputs the respiratory rate when the reliability is equal to or greater than a threshold;
a measurement time setting unit that sets a measurement time for imaging the living body;
Equipped with
the imaging unit images the living body during the measurement time to obtain the moving image;
The measurement time setting unit extends the measurement time when the reliability is lower than the threshold.
Measuring equipment. The measurement device according to claim 1 , wherein the reliability calculation unit calculates the reliability from a ratio between the magnitude of the respiratory signal and the magnitude of the noise signal. the plurality of pixel values are pixel values within a region including an image of a chest of the living body,
the time change is represented by a time series signal indicating a fluctuation in the plurality of pixel values or a fluctuation in a velocity vector determined from the plurality of pixel values;
The measurement device according to claim 1 , wherein the respiration rate calculation unit calculates the respiration rate from a maximum peak frequency in the time-series signal. the magnitude of the respiratory signal is the power spectrum value of the maximum peak frequency;
4. The measuring device according to claim 3, wherein the magnitude of the noise signal is a power spectrum value of the second or lower peak frequency in the time-series signal. the magnitude of the respiratory signal is a statistical value of power spectrum values within a predetermined range including the maximum peak frequency;
4. The measuring apparatus according to claim 3, wherein the magnitude of the noise signal is a statistical value of power spectrum values outside the predetermined range. 6. The measurement device according to claim 1 , wherein the measurement time setting unit sets at least one selected from the group consisting of a lower limit value and an upper limit value of the measurement time. the output unit outputs a message indicating an error when the reliability is lower than the threshold;
6. The measurement device according to claim 1, wherein the respiratory rate calculation unit terminates the process of calculating the respiratory rate when the reliability is lower than the threshold value. If the moving image includes a plurality of facial images, a distance calculation unit is further provided to calculate a distance between the plurality of facial images;
the output unit outputs a message indicating an error when the minimum distance is equal to or less than a predetermined distance;
The measurement device according to any one of claims 1 to 7 , wherein the respiration rate calculation unit terminates the process of calculating the respiration rate when the minimum value is equal to or less than the predetermined distance. The measuring device
a step of capturing an image of a living body to obtain a moving image;
calculating a respiratory rate by detecting an oscillation period of a respiratory signal included in a time change of a plurality of pixel values included in the moving image;
calculating a reliability of the respiratory rate from a comparison result between the respiratory signal and a noise signal included in the time change;
outputting the respiratory rate when the reliability is equal to or greater than a threshold;
setting a measurement time for imaging the living body;
Run
In the step of acquiring the moving image, the measurement device captures an image of the living body for the measurement time to acquire the moving image,
In the step of setting the measurement time, the measurement device extends the measurement time if the reliability is lower than the threshold.
How to measure respiratory rate. On the computer,
A function to capture images of a living body and acquire moving images;
a function of calculating a respiratory rate by detecting an oscillation period of a respiratory signal included in a time change of a plurality of pixel values included in the moving image;
a function of calculating the reliability of the respiratory rate from a comparison result between the respiratory signal and a noise signal included in the time change;
a function of outputting the respiratory rate when the reliability is equal to or greater than a threshold;
a function of setting a measurement time for imaging the living body;
Execute
In the function of acquiring the moving image, the living body is imaged for the measurement time to acquire the moving image ;
In the function of setting the measurement time, if the reliability is lower than the threshold, the measurement time is extended.
program.