JP7657092B2 - VIDEO ANALYSIS DEVICE, VIDEO ANALYSIS METHOD, AND PULSE WAVE DETECTION DEVICE - Google Patents

Description

The following disclosure relates to a video analysis device, a video analysis method, and a pulse wave detection device.

As an example of a device for detecting pulse waves of a living body, a pulse wave detection device that detects pulse waves based on the output value (pixel value) of an imaging element (light receiving element) is known. Patent Document 1 discloses a pulse wave measurement device that includes a pulse wave information acquisition unit that optically acquires pulse wave information from a part of the living body, and a pulse wave feature amount calculation unit that calculates pulse wave feature amounts based on the pulse wave information.

International Publication No. WO2015/098977

The above-mentioned conventional technology has the problem that it is not possible to remove noise contained in the frequency band of the pulse wave, making it impossible to obtain a pulse wave with good signal quality.

The objective of one aspect of the present disclosure is to obtain a pulse wave with good signal quality.

In order to solve the above-mentioned problems, a video analysis device according to one aspect of the present disclosure includes a region of interest detection unit that detects a region of interest of a living body based on a predetermined algorithm from a video of the living body captured by an imaging device having at least red, blue, and green color filters and an imaging unit, a time change detection unit that detects time series data of each pixel value of red pixels, green pixels, and blue pixels among a plurality of pixels included in the region of interest, and a pulse wave detection unit that detects a pulse wave of the living body based on the output from the time change detection unit, and the time change detection unit is configured to output the time series data of the pixel values of the green pixels as is, and to apply a noise reduction filter to at least one of the time series data of the pixel values of the red pixels and the time series data of the pixel values of the blue pixels, thereby outputting the time series data with noise reduced.

According to one aspect of the present disclosure, a pulse wave with good signal quality can be obtained.

1 is a block diagram showing a configuration of a main part of a pulse wave detecting device according to a first embodiment. 4A and 4B are diagrams illustrating an example of the arrangement of red, green, and blue color filters in a color filter portion. 4 is a diagram showing an example of a region of interest detected by a region of interest detection unit; FIG. 11A and 11B are diagrams illustrating an example of an interpolation process for missing pixel values in each color channel image. 11 is a diagram showing an example of time-series data of R pixel values, G pixel values, and B pixel values in a region of interest detected from the region of interest; FIG. 13 is a diagram showing frequency components of each pixel value after fast Fourier transform is applied to time series data of R pixel values, G pixel values, and B pixel values in a region of interest detected from the region of interest. FIG. 6 is a diagram showing an example of a pulse wave detected based on time series data of R pixel values, G pixel values, and B pixel values shown in FIG. 5 . FIG. 6 is a diagram showing an example of a pulse wave detected based on each of the time series data of R pixel values, G pixel values, and B pixel values shown in FIG. 5 after applying a noise reduction filter to the time series data of R pixel values and the time series data of B pixel values, respectively. FIG. 2 is a flowchart showing the flow of a series of processes executed by the video analysis device. 1A and 1B are diagrams illustrating a Bayer array of color filters and another method for generating a full-color image. FIG. 13 is a block diagram showing a configuration of a main part of a modified example of a pulse wave detecting device. FIG. 13 is a block diagram showing a configuration of a main part of a modified example of a pulse wave detecting device. 10 is a block diagram showing the configuration of a main part of a pulse wave detecting device according to a second embodiment. FIG.

[Embodiment 1]
A pulse wave detection device 1 according to the first embodiment will be described below. For convenience of explanation, the same reference numerals will be used in the following embodiments to designate components having the same functions as those described in the first embodiment, and the description thereof will not be repeated. For the sake of brevity, the description of matters similar to those in the publicly known technology will also be omitted as appropriate.

Please note that each configuration and each numerical value described in this specification is merely an example unless otherwise specified. Therefore, unless otherwise specified, the positional relationship of each member is not limited to the example in each figure. Also, please note that each drawing is intended to roughly explain the shape, structure, and positional relationship of each member, and is not necessarily drawn as it actually is. In this specification, the description "A to B" regarding two numbers A and B means "greater than or equal to A and less than or equal to B" unless otherwise specified.

(Overview of pulse wave detection device 1)
1 is a block diagram showing the configuration of the main parts of a pulse wave detection device 1. The pulse wave detection device 1 detects the pulse waves of a living organism 900 (e.g., a human). The pulse wave detection device 1 is an example of a non-contact type pulse wave detection device. Hereinafter, the pulse waves of a living organism H will be simply referred to as the pulse wave. The pulse wave detection device 1 includes a video analysis device 10 and an imaging device 50 (e.g., a camera). The video analysis device 10 and the imaging device 50 are connected so as to be able to communicate with each other.

The imaging device 50 captures a video (moving image) showing the living body H. Specifically, the imaging device 50 captures images (still images) that constitute the video for each frame period. In the first embodiment, the imaging device 50 is an RGB (Red, Green, Blue) camera. Therefore, the video in the first embodiment is an RGB color video. The imaging device 50 includes a color filter unit 51 and an imaging unit 52.

The color filter unit 51 includes at least a red color filter 511R, a green color filter 511G, and a blue color filter 511B. The color filter unit 51 may include other color filters, such as a yellow color filter or a near-infrared color filter. The imaging unit 52 includes a plurality of imaging elements 520 arranged in an array. The imaging elements 520 are known image sensors. The imaging elements 520 receive light (e.g., external light) that arrives from the living body 900 and passes through the color filter unit 51.

The imaging element 520 is also referred to as a pixel of the imaging device 50. For this reason, in this specification, the imaging element 520 is also referred to as an imaging pixel. W x H imaging pixels are arranged in the imaging section 52. W represents the number of imaging pixels in the horizontal direction (imaging pixels per row), and H represents the number of imaging pixels in the vertical direction (imaging pixels per column). The imaging pixels output, as pixel values (tone values), the signal values (e.g., current values) of electrical signals indicating the intensity (e.g., luminance) of the light they receive. More specifically, the imaging pixels output pixel values as analog values.

In the imaging device 50, the pixel values as analog values are converted to digital values by an AD converter (not shown). The imaging device 50 then outputs a digital image (an image having digital pixel values). As an example, the pixel values in the first embodiment are sampled as 8-bit digital values. Therefore, the pixel values in the first embodiment are in the range of 0 to 255.

Figure 2 is a diagram showing an example of the arrangement of the red color filter 511R, green color filter 511G, and blue color filter 511B in the color filter section 51. In the imaging device 50, the red color filter 511R, green color filter 511G, and blue color filter 511B are arranged in one-to-one correspondence with the imaging elements 520. One red color filter 511R is arranged so as to cover one imaging element 520. One green color filter 511G is arranged so as to cover another imaging element 520. One blue color filter 511B is arranged so as to cover yet another imaging element 520.

As shown in FIG. 2, in the first embodiment, a red color filter 511R, a green color filter 511G, and a blue color filter 511B are arranged in a Bayer pattern. In the color filter section 51 in the example of FIG. 2, color filter units 510 corresponding to four (two horizontally by two vertically) imaging pixels are defined.

One color filter unit 510 includes (i) one red color filter 511R, (ii) two green color filters 511G (green color filters 511G1 and 511G2), and (iii) one green color filter 511B. In the color filter section 51, a plurality of color filter units 510 are repeatedly arranged in the horizontal and vertical directions. In this manner, in the color filter section 51, the red color filter 511R, the green color filter 511G, and the blue color filter 511B are arranged in a mosaic pattern (more specifically, in a checkered pattern) with a number ratio of 1:2:1.

In this specification, an imaging pixel corresponding to the red color filter 511R is referred to as a red imaging pixel. One red imaging pixel receives light (red light) that has passed through the red color filter 511R, and outputs a red pixel value (R pixel value) that indicates the intensity of the red light. Also, an imaging pixel corresponding to the green color filter 511G is referred to as a green imaging pixel. One green imaging pixel receives light (green light) that has passed through the green color filter 511G, and outputs a green pixel value (G pixel value) that indicates the intensity of the green light. Also, an imaging pixel corresponding to the blue color filter 511B is referred to as a blue imaging pixel. One blue imaging pixel receives light (blue light) that has passed through the blue color filter 511B, and outputs a blue pixel value (B pixel value) that indicates the intensity of the blue light.

The imaging unit 52 generates, for each frame period of the video, (i) an R channel image IMGR (an image showing the distribution of R pixel values output by a plurality of red imaging pixels), (ii) a G channel image IMGRG (an image showing the distribution of G pixel values output by a plurality of green imaging pixels), and (iii) a B channel image IMGB (an image showing the distribution of B pixel values output by a plurality of blue imaging pixels). IMGR, IMGG, and IMGB all have a resolution W×H. The imaging unit 52 then synthesizes IMGR, IMGG, and IMGB to generate an image IMG as a mosaic-like RGB color image. In other words, the imaging unit 52 generates IMG as a RAW image.

As described above, the imaging unit 52 generates IMG as RAW images for each frame that constitutes the video. The imaging unit 52 then generates video (RAW video) by arranging each generated frame in the order of frame number. The imaging unit 52 supplies the generated video to the video analysis device 10.

The video analysis device 10 includes a region of interest detection unit 11, a time change detection unit 12, and a pulse wave detection unit 13. The time change detection unit 12 includes two noise reduction filters 21 and 22. The noise reduction filters 21 and 22 are both low-pass filters (noise reduction filters) that reduce noise. The time change detection unit 12 may also include other noise reduction filters, such as an average filter, a Gaussian filter, a median filter, a maximum value filter, or a minimum value filter.

The video analysis device 10 detects pulse waves by analyzing the video acquired from the imaging device 50. More specifically, the video analysis device 10 detects pulse waves by analyzing each frame of the video (each of the multiple IMGs).

The region of interest detection unit 11 detects a region of interest (ROI) from the IMG based on a predetermined algorithm (e.g., using a publicly known algorithm). The region of interest is a partial area of the IMG in which a predetermined part of the living body 900 is shown. FIG. 3 is a diagram showing an example of a region of interest detected by the region of interest detection unit 11. In the following description, the region of interest in the example of FIG. 3 is referred to as region of interest 910.

The IMG in the example of Figure 3 is an image showing the entire face of the living body 900. In the example of Figure 3, one cheek of the face is preset as a predetermined part. First, the region of interest detection unit 11 detects a face region (a region showing the face of the living body 900) from the IMG. Then, the region of interest detection unit 11 detects an area within the face region showing one cheek as the region of interest 910. The region of interest 910 in the example of Figure 3 is a rectangular region.

However, as will be clear to those skilled in the art, the predetermined area does not have to be limited to the cheek, but may be any area from which information regarding the blood vessels of the living body 900 can be detected. Other examples of the predetermined area include the forehead, the root of the nose, the neck, the fingertips, and the palm. The region of interest may be any image area that depicts a change in the color of the skin of the living body over time. Furthermore, the number of regions of interest is not necessarily limited to one. In this way, the IMG is not limited to the example of FIG. 3.

The time change detection unit 12 detects time series data for each pixel value of red pixels, green pixels, and blue pixels among the multiple pixels included in the region of interest 910. Time series data is a set of data in which each piece of data (pixel values, etc.) that changes over time and the time data at that time are arranged in a chronological order. The pulse wave detection unit 13 detects the pulse wave of the living body 900 based on each detected time series data.

As an example, the time change detection unit 12 calculates, for each frame, (i) the average value Rave of the R pixel values included in the region of interest 910, (ii) the average value Gave of the G pixel values included in the region of interest 910, and (iii) the average value Bave of the B pixel values included in the region of interest 910. That is, the time change detection unit 12 derives time series data for Rave, Gave, and Bave. Then, the pulse wave detection unit 13 may detect (derive) the pulse wave by analyzing the time series data using a known algorithm. As an example, the pulse wave detection unit 13 may detect the pulse wave using the method disclosed in International Publication WO2020/090348.

However, if pixel value saturation occurs within the region of interest 910, the accuracy of pulse wave detection by the pulse wave detection unit 13 may decrease. Such pixel value saturation may occur, for example, when the brightness of external light is high. Therefore, in the first embodiment, as described below, the pulse wave detection unit 13 detects pulse waves after reducing the influence of pixels in the region of interest 910 where pixel value saturation occurs (hereinafter referred to as saturated pixels). Note that the process of reducing the influence of saturated pixels may be performed by the time change detection unit 12.

In this specification, "saturation of pixel values" means that the pixel value is a predetermined maximum value that can be set in an image (more precisely, digital image data). As described above, in the first embodiment, pixel values are sampled as 8-bit digital values. Therefore, in the first embodiment, "saturation of pixel values" means that the pixel value is 255. In contrast, in the first embodiment, "non-saturation of pixel values" means that the pixel value is 0 to 254. From the above, a saturated pixel in the first embodiment is a pixel having a pixel value of 255. Therefore, in the first embodiment, the pixel value of 255 is also referred to as a saturated pixel value. Note that the pixel value in the above description refers to any one of the R pixel value, the G pixel value, and the B pixel value. Also, although the pixel value saturation is set to a pixel value of 255, this is not limited thereto, and a value close to 255 may be set as a threshold value (e.g., 240) and a pixel value equal to or greater than the threshold value may be set as a saturated pixel value.

Figure 4 is a diagram explaining a method of aligning missing information in each pixel when the color filters are arranged in a Bayer array. Below, an example of aligning missing information in a pixel that receives light passing through the blue color filter 511B will be explained. The blue pixel that receives light passing through the blue color filter 511B has blue information, but is missing green and red information. Therefore, the G pixel value of the blue pixel is obtained from each green pixel that receives light passing through the green color filter 511G, which is arranged on the top, bottom, left, and right of the blue pixel that receives light passing through the blue color filter 511B. Therefore, the R pixel value of the blue pixel is obtained from each red pixel that receives light passing through the red color filter 511R, which is arranged on the top right, top left, bottom right, and bottom left of the blue pixel that receives light passing through the blue color filter 511B. This allows three colors of information (pixel values) to be aligned in one pixel.

Figure 5 is a diagram showing an example of time series data of R pixel values, G pixel values, and B pixel values in the region of interest 910 detected from the region of interest 910. Reference numeral 1010 in Figure 5 indicates time series data of R pixel values, 1020 in Figure 5 indicates time series data of G pixel values, and 1030 in Figure 5 indicates time series data of B pixel values. In each graph in Figure 5, the vertical axis indicates the pixel value (brightness value) of each pixel, and the horizontal axis indicates the measurement frame (time). As shown in Figure 5, since the time series data of the G pixel values has little noise, a pulse wave component can be observed. On the other hand, since the time series data of the R pixel values and the time series data of the B pixel values have large noise, a pulse wave component cannot be observed.

As shown in Figure 5, the time series data of the G pixel values includes luminance changes in pixel values due to the movement of the living body. Therefore, it is difficult to detect a good pulse wave based only on the time series data of the G pixel values.

Figure 6 is a diagram showing the frequency components of each pixel value after a fast Fourier transform is applied to each time series data of the R pixel value, G pixel value, and B pixel value in the region of interest 910 detected from the region of interest 910. 1110 in Figure 6 shows the frequency components of the R pixel value after the fast Fourier transform is applied, 1020 in Figure 6 shows the frequency components of the G pixel value after the fast Fourier transform is applied, and 1030 in Figure 6 shows the frequency components of the B pixel value after the fast Fourier transform is applied. In each graph in Figure 6, the vertical axis shows the power spectrum, and the horizontal axis shows the frequency.

As shown in range 60 in FIG. 6 , in the frequency components of the G pixel values after the fast Fourier transform has been applied, a power spectrum peak is observed near the frequency of the pulse wave. On the other hand, in the frequency components of the R pixel values to which the fast Fourier transform has been applied, no power spectrum peak is observed near the frequency of the pulse wave. Similarly, in the frequency components of the B pixel values to which the fast Fourier transform has been applied, no power spectrum peak is observed near the frequency of the pulse wave.

Figure 7 shows an example of a pulse wave detected based on the time series data of the R pixel values, G pixel values, and B pixel values shown in Figure 5. As shown in this figure, the detected pulse wave contains noise. As a result, a pulse wave with poor signal quality is detected.

Figure 8 is a diagram showing an example of a pulse wave detected based on the time series data of the R pixel value, the G pixel value, and the B pixel value after applying the noise reduction filters 21 and 22 to the time series data of the R pixel value and the time series data of the B pixel value, respectively, among the time series data of the R pixel value, the G pixel value, and the B pixel value shown in Figure 5. As shown in this figure, noise has been removed from the detected pulse wave. Therefore, a pulse wave with good signal quality has been detected. The time series data of the R pixel value and the time series data of the B pixel value contain large noise as shown in Figure 5. However, these time series data also contain trends similar to the trends contained in the time series data of the G pixel value. Therefore, by applying the noise reduction filter 21 to the time series data of the R pixel value and further applying the noise reduction filter 22 to the time series data of the B pixel value, the trends contained in these time series data can be effectively removed.

The time series data of the pixel values of the red pixels and the time series data of the red pixels of the blue pixels contain almost no pulse wave components. Therefore, the noise reduction filters 21 and 22 may be filters that reduce noise in a band that includes the frequency band of the pulse wave. This makes it possible to reduce noise in a wide frequency band contained in the time series data of the R pixel values and the time series data of the B pixel values. In this way, by applying the noise reduction filters 21 and 22 to the time series data of the R pixel values and the time series data of the B pixel values, which contain almost no pulse wave but are mixed with noise, respectively, the effect of the change in brightness of the pixel values caused by the body movement of the living body 900 can be reduced, and a pulse wave with good signal quality can be detected.

9 is a flow chart showing a series of processes executed by the video analysis device 10. In step S1, a region of interest 910 of the living body 900 is detected from the image IMG of the living body 900 captured by the imaging device 50 based on a predetermined algorithm. In step S2, the time change detection unit 12 detects time series data of the pixel values of each of the red pixels, green pixels, and blue pixels among the multiple pixels included in the region of interest 910, and outputs them to the pulse wave detection unit 13. At that time, the time change detection unit 12 outputs the time series data of the pixel values of the green pixels as is to the pulse wave detection unit 13. Furthermore, the noise reduction filter 21 is applied to the time series data of the pixel values of the red pixels, and the noise reduction filter 22 is applied to the time series data of the pixel values of the blue pixels, thereby outputting each time series data with noise reduction to the pulse wave detection unit 13. In step S3, pulse wave detection unit 13 detects the pulse wave of living body 900 based on the output of time change detection unit 12, i.e., the time series data of R pixel values, G pixel values, and B pixel values. In step S4, video analysis device 10 determines whether or not the measurement of the pulse wave has ended. If the determination result is true (YES in step S4), video analysis device 10 ends the execution of the series of processes shown in FIG. 9. If the determination result is false (NO in step S4), region of interest detection unit 11 returns to step S1 and detects region of interest 910 again.

Figure 10 is a diagram for explaining a Bayer-arranged color filter 511 and another method for generating a full-color image. In this example, the video analysis device 10 obtains R, G, and B information (pixel values) of one pixel constituting a full-color image from a block of four pixels including one red pixel, two green pixels, and one blue pixel in the shaded portion shown in Figure 10. For example, for a block of four pixels included in an image captured by the imaging device 50 and a pixel of the full-color image corresponding to the block, the information of one red pixel present in the block is taken as the R information of the pixel of the full-color image. Furthermore, the average value of the information of two G pixels present in the block is taken as the G information of the pixel of the full-color image. Furthermore, the information of one B pixel present in the block is taken as the B information of the pixel of the full-color image. As a result, a full-color image with the number of pixels in the height direction being H÷2 and the number of pixels in the width direction being W÷2 is obtained from an image captured by the imaging device 50 in which the number of pixels in the height direction is H and the number of pixels in the width direction is W.

In this embodiment, as a condition for applying the noise reduction filters 21 and 22, a threshold may be set for each S/N ratio (signal to noise ratio) of the R pixel value and the B pixel value specific to the imaging device 50. In this case, the time change detection unit 12 determines whether to apply the noise reduction filter 21 to the time series data of the pixel value of the red pixel based on whether the S/N ratio of the R pixel value included in the region of interest 910 exceeds the threshold value set for the red pixel. Furthermore, the time change detection unit 12 determines whether to apply the noise reduction filter 22 to the time series data of the B pixel value based on whether the S/N ratio of the B pixel value included in the region of interest 910 exceeds the threshold value set for the blue pixel. More specifically, if the S/N ratio of the R pixel value included in the region of interest 910 exceeds the threshold value for the red pixel, the time change detection unit 12 determines to apply the noise reduction filter 21 to the time series data of the R pixel value, and if not, determines not to apply the noise reduction filter 21. Furthermore, if the S/N ratio of the B pixel values included in the region of interest 910 exceeds the threshold value for blue pixels, the time change detection unit 12 decides to apply the noise reduction filter 22 to the time series data of the B pixel values, and if it does not exceed the threshold value, decides not to apply the noise reduction filter 22.

Alternatively, as a condition for applying the noise reduction filters 21 and 22, a threshold may be set to the sum of the transmittance values included in a specific wavelength range for each of the spectral sensitivities of the red color filter 511R and the blue color filter 511B specific to the imaging device 50.

(Modification)
In the first embodiment, an example has been described in which the time change detection unit 12 applies the noise reduction filters 21 and 22 individually to both the time series data of R pixel values and the time series data of B pixel values. However, the present invention is not limited to this, and the time change detection unit 12 may be configured to apply only the corresponding noise reduction filter 21 or 22 individually to at least one of the time series data of R pixel values and the time series data of B pixel values. Such an example configuration will be described below.

Figure 11 is a block diagram showing the configuration of the main parts of a modified example of pulse wave detection device 1. As shown in this figure, video analysis device 10 constituting pulse wave detection device 1 may be configured to include only noise reduction filter 21, which corresponds to red pixels, out of noise reduction filters 21 and 22. In video analysis device 10 of this example, time change detection unit 12 applies noise reduction filter 21 to time series data of pixel values of red pixels, but does not apply noise reduction filter 22 to time series data of pixel values of blue pixels.

Therefore, similar to the first embodiment, the time change detection unit 12 outputs the time series data of the G pixel values to the pulse wave detection unit 13 as is, and also applies the noise reduction filter 21 to the time series data of the R pixel values to output the time series data of the R pixel values with noise reduced to the pulse wave detection unit 13. Furthermore, unlike the first embodiment, the time change detection unit 12 outputs the time series data of the B pixel values as is to the pulse wave detection unit 13. In this way, the pulse wave detection unit 13 detects a pulse wave based on the time series data of the B pixel values and the time series data of the G pixel values with no noise reduction, and the time series data of the R pixel values with noise reduced. As a result, a pulse wave with good signal quality can be detected.

Figure 12 is a block diagram showing the configuration of the main parts of a modified example of pulse wave detection device 1. As shown in this figure, video analysis device 10 constituting pulse wave detection device 1 may be configured to include only noise reduction filter 22, which corresponds to blue pixels, out of noise reduction filters 21 and 22. In video analysis device 10 of this example, time change detection unit 12 applies noise reduction filter 22 to time series data of pixel values of blue pixels, but does not apply noise reduction filter 21 to time series data of pixel values of red pixels.

Therefore, as in the first embodiment, the time change detection unit 12 outputs the time series data of the G pixel values to the pulse wave detection unit 13 as is, and also applies the noise reduction filter 22 to the time series data of the B pixel values to output the time series data of the B pixel values with noise reduced to the pulse wave detection unit 13. Furthermore, unlike the first embodiment, the time change detection unit 12 outputs the time series data of the R pixel values to the pulse wave detection unit 13 as is. As a result, the pulse wave detection unit 13 detects a pulse wave based on the time series data of the R pixel values and the time series data of the G pixel values with no noise reduced, and the time series data of the B pixel values with noise reduced. As a result, as in the first embodiment, a pulse wave with good signal quality can be detected.

[Embodiment 2]
Fig. 13 is a block diagram showing the configuration of the main parts of a pulse wave detection device 1A according to embodiment 2. As shown in this figure, the pulse wave detection device 1A includes a video analysis device 10A and an imaging device 50 (e.g., a camera). The video analysis device 10A and the imaging device 50 are connected so as to be able to communicate with each other. The configuration of the imaging device 50 is the same as that of embodiment 1, and therefore a detailed description thereof will be omitted.

The pulse wave detection unit 13A includes a pulse wave noise filter 31. The pulse wave noise filter 31 is a filter that removes noise in bands other than the frequency band of the pulse wave from the pulse wave detected by the pulse wave detection unit 13A. The pulse wave detection unit 13A removes noise from the pulse wave by applying the pulse wave noise filter 31 to the detected pulse wave.

The time series data of the pixel values of the red pixels and the time series data of the red pixels of the blue pixels contain almost no pulse wave components. Therefore, the noise reduction filters 21 and 22 may be filters that reduce noise in a band that includes the frequency band of the pulse wave. Furthermore, the noise reduction filters 21 and 22 may be filters that reduce noise at a lower frequency than the frequency of the noise removed by the pulse wave noise filter 31. By using these noise reduction filters 21 and 22 and the pulse wave noise filter 31, the signal quality of the pulse wave can be further improved.

In this embodiment, as in the first embodiment, a threshold can be set to determine whether or not to apply the noise reduction filters 21 and 22. Note that the pulse wave detection unit 13A always applies the pulse wave noise filter 31 to the detected pulse wave.

[Software implementation example]
The functions of pulse wave detection device 1 (hereinafter referred to as the "device") can be realized by a program for causing a computer to function as the device, and by a program for causing a computer to function as each control block of the device (in particular, each part included in video analysis device 10 or 10A).

In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing each of the functions described in each of the above embodiments.

The program may be recorded on one or more computer-readable recording media, not on a temporary basis. The recording media may or may not be included in the device. In the latter case, the program may be provided to the device via any wired or wireless transmission medium.

In addition, some or all of the functions of each of the control blocks can be realized by a logic circuit. For example, the scope of this disclosure also includes an integrated circuit in which a logic circuit that functions as each of the control blocks is formed. In addition, the functions of each of the control blocks can also be realized by, for example, a quantum computer.

The processes described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI may run on the control device, or on another device (such as an edge computer or a cloud server).

[Additional Notes]
One aspect of the present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. An embodiment obtained by appropriately combining the technical means disclosed in each of the different embodiments is also included in the technical scope of one aspect of the present disclosure. Furthermore, by combining the technical means disclosed in each of the embodiments, new technical features can be formed.

REFERENCE SIGNS LIST 1, 1A Pulse wave detection device 10, 10A Video analysis device 11 Region of interest detection section 12 Time change detection section 13, 13A Pulse wave detection section 21, 22 Noise reduction filter 31 Pulse wave noise filter 50 Imaging device 51 Color filter section 52 Imaging section 520 Imaging element 510 Color filter unit 511R Red color filter 511G Green color filter 511B Blue color filter 900 Living body 910 Region of interest IMG Image (one frame of an image showing a living body)

Claims (5)

a region of interest detection unit that detects a region of interest of a living body based on a predetermined algorithm from an image of the living body captured by an imaging device having at least red, blue, and green color filters and an imaging unit;
a time change detection unit that detects time series data of each pixel value of red pixels, green pixels, and blue pixels among a plurality of pixels included in the region of interest;
a pulse wave detection unit that detects a pulse wave of the living body based on an output from the time change detection unit,
The time change detection unit outputs the time series data of the pixel values of the green pixels as is, and also applies a noise reduction filter to at least one of the time series data of the pixel values of the red pixels and the time series data of the pixel values of the blue pixels, thereby outputting the time series data with noise reduced. the pulse wave detection unit applies a pulse wave noise filter to the detected pulse wave to reduce noise from the pulse wave;
2. The video analysis device according to claim 1, wherein the noise reduction filter is a filter that reduces noise having a frequency lower than a frequency of noise reduced by the pulse wave noise filter. The video analysis device according to claim 1 or 2, wherein the noise reduction filter is a filter that reduces noise in a frequency band that includes the frequency band of the pulse wave. A video analysis device according to any one of claims 1 to 3,
A pulse wave detection device comprising the above imaging device. a region of interest detection step of detecting a region of interest of a living body based on a predetermined algorithm from an image of the living body captured by an imaging device having at least red, blue, and green color filters and an imaging unit;
a time change detection step of detecting time series data of pixel values of red pixels, green pixels, and blue pixels among a plurality of pixels included in the region of interest;
a pulse wave detection step of detecting a pulse wave of the living body based on an output from the time change detection step,
In the time change detection step, the time series data of the pixel values of the green pixels is output as is, and a noise reduction filter is applied to at least one of the time series data of the pixel values of the red pixels and the time series data of the pixel values of the blue pixels, thereby outputting noise-reduced time series data.

Images