JP7110689B2 - MEAL DETECTION PROGRAM, MEAL DETECTION METHOD, AND MEAL DETECTION DEVICE - Google Patents

Description

The present invention relates to a meal detection program, a meal detection method, and a meal detection device.

One aspect of healthcare support is the utilization of records of various activities in daily life, such as meals, urination, and defecation. Among these behaviors, one of the technologies to detect meals is to compare the measured heart rate data with the learning model and identify the eating behavior period during the measurement period based on the correlation and similarity with the learning model. There is

WO2017/081829 WO2017/081787

Michael B. Sidery, Ian A. Macdonald “The effect of meal size on the cardiovascular responses to food ingestion” Department of Physiology and Pharmacology, University of Nottingham Medical School, Clfton Boulevard, Nottingham NG7 2UH, 23 July 1993

However, the above-described technology merely detects the characteristics of the heartbeat that appear when a certain amount of food is eaten at one time. Therefore, with the above technology, it is difficult to detect a meal in which a small amount of food is ingested over multiple times, such as a course meal. This is because when small meals are ingested over multiple meals, heart rate rises due to meals are less than when one meal is ingested all at once, partly because the amount of food eaten per meal is smaller. is.

In one aspect, an object of the present invention is to provide a meal detection program, a meal detection method, and a meal detection device that can detect a meal such as a course meal.

In one aspect, the meal detection program acquires heartbeat data of a subject, detects a peak whose amplitude exceeds a predetermined threshold in the acquired heartbeat data, and if the number of detected peaks is plural, the detection causes a computer to execute a process of determining whether or not the subject has ingested the course meal based on the amplitude or area of each peak obtained from each detected peak and the appearance interval of the peaks obtained between the detected peaks. .

Meals such as course meals can be detected.

FIG. 1 is a diagram illustrating an example of a functional configuration of each device included in a healthcare support system according to a first embodiment; FIG. 2 is a diagram showing an example of heartbeat data. FIG. 3 is a diagram showing an example of heartbeat data. FIG. 4 is a diagram showing an example of heartbeat data. FIG. 5 is a diagram showing an example of heartbeat data. FIG. 6 is a diagram showing an example of heartbeat data. FIG. 7 is a diagram showing an example of second window data. FIG. 8 is a diagram showing an example of a long-term trend. FIG. 9 is a diagram showing an example of a regression line. FIG. 10 is a diagram showing an example of short-term trends. FIG. 11 is a diagram showing an example of weak peaks. FIG. 12 is a diagram showing an example of weak peaks. FIG. 13 is a diagram showing an example of a healthcare support service. FIG. 14 is a flowchart illustrating procedures of healthcare support processing according to the first embodiment. 15 is a flowchart illustrating the procedure of the second detection process according to the first embodiment; FIG. FIG. 16 is a diagram showing an example of weak peaks. FIG. 17 is a diagram showing an example of weak peaks. FIG. 18 is a diagram showing an example of weak peaks. FIG. 19 is a diagram illustrating an example hardware configuration of a computer that executes the meal detection program according to the first and second embodiments.

A meal detection program, meal detection method, and meal detection device according to the present application will be described below with reference to the accompanying drawings. Note that this embodiment does not limit the disclosed technology. Further, each embodiment can be appropriately combined within a range that does not contradict the processing contents.

[System configuration]
FIG. 1 is a diagram illustrating an example of a functional configuration of each device included in a healthcare support system according to a first embodiment; A healthcare support system 1 shown in FIG. 1 provides various healthcare support services. Examples of such healthcare support services include a service that uses heartbeat data acquired by the wearable terminal 10 to record a subject's daily activities, such as meals, and a derivative service that utilizes the record. services, etc.

As shown in FIG. 1 , the healthcare support system 1 includes a wearable terminal 10 , an IoT (Internet of Things) device 20 and a server device 30 . Although FIG. 1 illustrates a case in which one wearable terminal 10 is connected to the IoT device 20 , the IoT device 20 may be connected to a plurality of wearable terminals 10 . Moreover, although FIG. 1 illustrates the case where one IoT device 20 is connected to the server device 30 , a plurality of IoT devices 20 may be connected to the server device 30 .

Among these, data is exchanged between the wearable terminal 10 and the IoT device 20 via short-range wireless communication such as BLE (Bluetooth (registered trademark) Low Energy), for example. This is merely an example, and the wearable terminal 10 and the IoT device 20 may be connected by wire. Also, the IoT device 20 and the server device 30 can be connected via any network, whether wired or wireless. Examples of such networks include mobile networks compatible with 3G (Generation), LTE (Long Term Evolution), 4G and 5G, local area networks (LANs), virtual private networks (VPNs) and other local communication networks. , and any network such as the Internet.

The wearable terminal 10 is a computer used by being worn by a subject who receives the above healthcare support service. For example, the wearable terminal 10 may be a bracelet-type, wristband-type, or ring-type gadget, or may be a head-mounted display or smart glasses worn on the head. The device may be of a type worn on the ankle as well as on the body such as.

For example, the wearable terminal 10 is equipped with a heartbeat sensor 11 that measures the heartbeat of the subject. The term "heart rate" as used herein refers to the number of times the heart beats to pump blood. For example, heart rate per unit time can be measured by employing a photoelectric pulse wave sensor that measures changes in blood flow corresponding to changes in heart rate as the heart rate sensor 11 . At this time, when the unit time is 1 minute, the heart rate is expressed in bpm (beats per minute) or the like. Also, when the unit time is 1 second, the heart rate is expressed in Hz. Alternatively, the heartbeat sensor 11 may be an electrocardiographic sensor that measures the electrical activity of the heart. In this case, the interval between R waves in the electrocardiographic waveform can be used instead of the heart rate. The heart rate time-series data measured by the heart rate sensor 11 in this manner is output to the IoT device 20 . Hereinafter, the time-series data of the heart rate measured by the heart rate sensor 11 may be referred to as "heart rate data".

Here, the case where the heart rate sensor 11 is mounted on the wearable terminal 10 is illustrated, but it is not necessarily mounted on the wearable terminal 10, and even if it is an environment sensor installed on the environment side where the subject lives I don't mind. For example, the heart rate can be detected from time-series changes in the brightness of an image in which a part of the subject's body is captured at a predetermined sampling frequency, or the Doppler frequency associated with heartbeat can be detected using an RF (Radio Frequency) motion sensor. By detecting the heart rate, the heart rate may be detected in a non-contact state with the body part of the subject.

The IoT device 20 is a device having, as one aspect, a function of connecting the wearable terminal 10 to a network. For example, the IoT device 20 corresponds to a mobile terminal device such as a smart phone, a mobile phone, a PHS (Personal Handyphone System) used by a target person, a slate terminal, a tablet terminal, or the like.

For example, the IoT device 20 transfers heartbeat data notified from the wearable terminal 10 via BLE communication or the like to the server device 30 via the network. Sensor events such as heartbeat data transferred from the wearable terminal 10 to the server apparatus 30 via the IoT device 20 can be transferred as MQTT (Message Queuing Telemetry Transport) messages, for example. At this time, the heartbeat data may be transmitted in real time each time the heartbeat is measured, or may be transmitted after accumulating over a predetermined period of time, such as 12 hours, 1 day, 1 week, or 1 month. It doesn't matter. The heartbeat data is transmitted in a state in which the user of the wearable terminal 10, who is the subject, can identify the heartbeat data on the side of the server device 30 that receives the heartbeat data. For example, the heartbeat data is transferred to the server device 30 together with user identification information or information such as the MAC (Media Access Control) address and individual identification number of the wearable terminal 10 associated with the user identification information.

The server device 30 is a computer that provides the healthcare support service described above. For example, the server device 30 can be implemented by installing a healthcare support program that realizes functions corresponding to the above healthcare support service as package software or online software on a desired computer. For example, the server device 30 may be implemented on-premises or rented as a server that provides the above healthcare support service, or may be implemented as a cloud that provides the above healthcare support service by outsourcing. .

[One aspect of the challenge]
As described above, the technology mentioned in the background art section above merely detects the characteristics of the heartbeat that are manifested by a meal in which a single meal is ingested all at once. In other words, the technologies mentioned in the Background Art section above assume only a meal in which a menu for one meal is provided at once, and do not assume a meal in which a menu for one meal is provided in multiple portions. For example, in meals such as course meals, so-called "plates" are served according to the items and order specified in the menu.

In this way, when multiple items are eaten in a fixed order, the amount of food per item in the entire course meal is the amount of meals in which one meal is eaten all at once, such as set meals and set menus. less than food. This is one of the reasons for suppressing changes in heart rate that occur when one plate of food in a menu such as a course meal is ingested. When heart rate changes are suppressed in this way, it is difficult to detect meal-like features in the heart rate waveform, making it difficult to detect a meal such as a course meal.

In the following, meals in which one meal of food is ingested all at once, such as meals such as set meals and set menus, are described as "regular meals", and one meal is divided into multiple meals and taken in multiple times. Meals such as course meals are sometimes referred to as 'course meals'.

2 and 3 are diagrams showing an example of heartbeat data. The vertical axis of the graphs shown in FIGS. 2 and 3 indicates heart rate (bpm), and the horizontal axis indicates time. In FIG. 2, a thin line indicates the heartbeat data h1 measured when a normal meal is taken, and a thick line indicates the heartbeat data H1 obtained by applying a predetermined filter, for example, a median filter to the heartbeat data h1. ing. Furthermore, in FIG. 2, as a reference, a meal period R11 from the start to the end of a normal meal is indicated by hatching. In addition, in FIG. 3, the heartbeat data h2 measured when the course meal was prepared is indicated by a thin line, and the heartbeat data H2 obtained by applying a median filter to the heartbeat data h2 is indicated by a thick line. Further, in FIG. 3, the meal period R21 for the first dish, the meal period R22 for the second dish, and the meal period R23 for the third dish in the course meal are hatched for reference.

As shown in FIG. 2, in the heart rate data H1, an increase in heart rate is observed due to normal eating during the meal period R11. In this case, the heart rate observed before a normal meal, for example, the lowest value of the heart rate observed in the waveform of the heart rate data H1, is taken as the baseline BL1, and the heart rate at the peak point p11 of the heart rate increase due to the normal meal. Subtract the baseline BL1 heart rate from the number. As a result, the peak amplitude P11 of the heart rate change due to normal eating is found to be approximately 30 bpm.

On the other hand, as shown in FIG. 3, in the heartbeat data H2, the first dish of the course meals is taken during the meal period R21, the second dish is taken during the meal period R22, and the third course is taken during the meal period R22. A dish is ingested during meal period R23. An increase in heart rate was also observed when the course meal was served in this way, but the increase in heart rate due to the intake of each plate of food was similar to the increase in heart rate due to normal meals. not wide enough. For example, by subtracting the heart rate of the baseline BL2 from the heart rate at the peak point p21 of heart rate increase due to the first dish of food, the peak amplitude P21 of the heart rate change due to the first dish of food is found to be approximately 10 bpm.

Comparing the peak amplitude P11 and the peak amplitude P21, there is a difference of about 3 times between when a normal meal is eaten and when the first plate of food is ingested. This is because in course meals, there is an interval between the meal periods in which each plate of food is eaten, so the amount of food consumed per plate is less than the amount consumed per meal of a normal meal. is one of the reasons. When a course meal is served in this way, features typical of a meal are less likely to appear as changes in the heart rate than when a normal meal is served.

For example, among the first peak that appears prior to the start of a meal and the second peak that appears subsequent to the first peak, the first peak that appears with an eating act, such as peristaltic movement of the esophagus, as the heart rate variability feature value. Take for example the case where the amplitude of is calculated. At this time, the amplitude P11 of the peak observed in the heartbeat data H1 shown in FIG. 2 is calculated as the amplitude of the first peak, and the amplitude P21 of the peak observed in the heartbeat data H2 shown in FIG. , the latter is only about one third of the former. Whether eating or not eating is determined by comparing such feature values with a threshold, or classifying eating or not eating by inputting feature values into a model generated by machine learning, It can be seen that course meals are harder to detect than normal meals.

[One aspect of the problem-solving approach]
Therefore, as part of the above-described healthcare support service, the server device 30 according to the present embodiment, when the heartbeat data includes a plurality of peaks, based on the feature amount calculated from each peak and the appearance interval of the peaks, determines whether or not the course meal has been ingested.

4 and 5 are diagrams showing an example of heartbeat data. 4 and 5 both show the heartbeat data H2 shown in FIG. Of these, in the heartbeat data H2 shown in FIG. 4, three peaks p21 to p23 included in the heartbeat data H2 are picked up and indicated by circles. Furthermore, in the heartbeat data H2 shown in FIG. 5, the feature values of the areas S1 to S3 are hatched for each of the three peaks p21 to p23. Furthermore, in FIG. 5, the interval from the end of appearance of peak p21 to the start of appearance of peak p22 is shown as peak appearance interval T1, and the interval from the end of appearance of peak p22 to the appearance of peak p23 is shown. is shown as the peak appearance interval T2.

In the heartbeat data H2 shown in FIG. 4, detection of a peak at which the heartbeat amplitude exceeds a predetermined threshold Th1 is performed. For example, the threshold Th1 is a value smaller than the heart rate increase observed during a normal meal, because the food intake per plate in the course meal is less than the intake per meal of a normal meal. For example, 8 bpm can be set. From this aspect, a peak at which the heart rate amplitude exceeds the above threshold Th1 may be referred to as a "weak peak". This results in weak peaks p21-p23 from the heartbeat data H2, as indicated by circles in FIG.

Subsequently, as shown in FIG. 5, the areas S1 to S3 are calculated for each of the three weak peaks p21 to p23 as feature amounts of the weak peaks p21 to p23. For example, in the section from the start of the weak peak p21 to the end of the weak peak p21, the area S1 of the figure surrounded by the waveform of the heartbeat data H2 and the baseline BL2 is calculated as the feature amount of the weak peak p21. Similarly, in the section from the start of the weak peak p22 to the end of the weak peak p22, the area S2 of the figure surrounded by the waveform of the heartbeat data H2 and the baseline BL2 is calculated as the feature amount of the weak peak p22. Furthermore, in the section from the start of the weak peak p23 to the end of the weak peak p23, the area S3 of the figure surrounded by the waveform of the heartbeat data H2 and the baseline BL2 is calculated as the feature amount of the weak peak p23.

Further, as shown in FIG. 5, the appearance interval T1 and the appearance interval T2 of weak peaks are calculated from the heartbeat data H2. The term "appearance interval" as used herein refers, as one aspect, to an interval from the end of the appearance of a certain weak peak to the start of the appearance of the next weak peak. With this definition, one appearance interval is calculated for two adjacent weak peaks, so if the number of weak peaks is N, the appearance intervals are (N−1). Since the heartbeat data H2 shown in FIG. 5 includes three weak peaks p21 to p23, 2 (=3-1) occurrence intervals are obtained. That is, the period from the time when the weak peak p21 ends to the time when the weak peak p22 starts appearing is calculated as the appearance interval T1. Along with this, the period from the time when the weak peak p22 appears to the end to the time when the weak peak p23 starts appearing is calculated as the appearance interval T2. In FIG. 5, the areas and appearance intervals of the weak peaks are given as examples of the feature values, but the amplitude of the weak peaks, the difference between the heart rate of the weak peak and the heart rate of the baseline, etc. It may be calculated as a feature amount of .

Then, it is determined whether or not the feature amount of each weak peak satisfies the first condition. By way of example only, it is determined whether or not the order regarding the magnitude of the feature amount of each weak peak corresponds to the order regarding the magnitude of the intake of each dish of the course meal. For example, in a course meal, there is an aspect that food with a larger intake is served later in the order. According to this aspect, if the number of dishes included in the menu is three, which is the same number as the number of weak peaks, the intake of each dish is the intake of the first dish < the intake of the second dish. <There is a high possibility that it will be in order of magnitude, such as the intake of the third dish. Therefore, the order of magnitude of the feature amounts of the weak peaks p21 to p23 is the order of the area S1 of the weak peak p21<the area S2 of the weak peak p22<the area S3 of the weak peak p23, that is, each dish of the course meal. It is determined whether or not there is correspondence in the order of intake.

Simultaneously or in parallel with this, it is determined whether or not the appearance interval of the weak peaks satisfies the second condition. By way of example only, it is determined whether or not the intervals at which the weak peaks appear correspond to the intervals at which each plate of food is served in the course meal. For example, in a course meal, the dishes corresponding to the order specified in the menu are served according to the order, and there is a certain interval between the serving of one dish and the serving of the next dish. That is, the next plate is served only after the person serving the course meal confirms that the immediately preceding plate has been eaten. On the other hand, it is less likely that the food on the previous plate will be left uneaten. From this, it is highly likely that the above intervals will converge within a certain period of time. Therefore, it is determined whether or not the appearance interval of the weak peak is within the food serving interval set by the lower limit value Tmin and the upper limit value Tmax.

Here, when the feature amount of each weak peak satisfies the first condition, there is a high possibility that the change in the elevation of heart rate of the subject corresponds to the change in intake of each dish provided according to the course meal menu. Furthermore, when the appearance interval of the weak peaks satisfies the second condition, there is a high possibility that the increase in heart rate of the subject corresponds to the interval at which each plate of food is served in the course meal. In other words, the increased heart rate is less likely to be caused by disturbances other than diet, such as exercise or mental factors.

For this reason, in this embodiment, when the feature amount of each weak peak satisfies the first condition and the appearance interval of the weak peaks satisfies the second condition, it is determined that the course meal has been ingested. This makes it possible to determine eating or non-meal by capturing the characteristics of heart rate changes that appear when a course meal is ingested. Therefore, it is possible to realize the detection of a course meal.

[Functional Configuration of Server Device 30]
Next, a functional configuration of the server device 30 according to this embodiment will be described. As shown in FIG. 1 , the server device 30 has an acquisition unit 31 , an application unit 32 , a first detection unit 33 , a second detection unit 34 and a provision unit 35 . In addition to the functional units shown in FIG. 1, the server device 30 may have various functional units of known computers, such as a communication interface for controlling communication with other devices.

Functional units such as the acquisition unit 31, the application unit 32, the first detection unit 33, the second detection unit 34, and the provision unit 35 shown in FIG. Virtually implemented by a processor. That is, the processor converts storage devices (not shown) such as a HDD (Hard Disk Drive), an optical disk, an SSD (Solid State Drive) into an OS (Operating System), as well as the above healthcare support program or a meal detection program which is a module thereof. Read programs such as In addition, the processor develops processes corresponding to the above functional units on a memory such as a RAM (Random Access Memory) by executing a healthcare support program and a meal detection program. As a result of executing the program in this manner, the functional units described above are virtually realized as processes. Here, a CPU and an MPU are illustrated as examples of processors, but the above functional units may be implemented by any processor, regardless of whether it is a general-purpose type or a specialized type. In addition, the functional unit or a part of the functional unit may be implemented by hardwired logic such as ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

The above functional units are provided with an operating environment in which storage devices such as HDDs, optical discs, and SSDs can be used as external storage devices or auxiliary storage devices. Allocated as a work area. For example, the memory corresponds to various semiconductor memory devices such as main storage devices such as RAM and flash memory. Also, the storage area accessible by the processor may not be implemented as a work area on the memory, and may be a swap area saved in an external storage device or an auxiliary storage device.

The acquisition unit 31 is a processing unit that acquires heartbeat data.

As one embodiment, the acquisition unit 31 outputs heartbeat data acquired from the heartbeat sensor 11 to the application unit 32 . The function of the acquisition unit 31 that acquires sensor events such as heartbeat data from the wearable terminal 10 can be implemented, for example, by executing a program for the MQTT server.

The application unit 32 is a processing unit that applies a predetermined filter to heartbeat data.

As one embodiment, the application unit 32 applies a median filter that extracts the median value to the heartbeat data acquired by the acquisition unit 31 . The heartbeat data to which the median filter has been applied in this way is output to the first detection unit 33 that detects normal meals and the second detection unit 34 that detects course meals.

Here, one of the reasons why the median filter is applied to the heartbeat data is that the heartbeat data contains steep noise. That is, the heart rate is observed as a discrete value between 42 bpm and 240 bpm on a computer, but has the property of continuously changing over time. Therefore, even if the heart rate rises or the heart rate that has risen falls, the heart rate at one point in time is unlikely to change to a far different value at the next point in time. For example, even if heart rate data such as 80 bpm, 120 bpm, and 81 bpm are observed in time series, in reality, the heart rate increases from 80 bpm to 120 bpm and the heart rate decreases from 120 bpm to 81 bpm. Change is hard to come by. Therefore, by applying a media filter to the heart rate data, the heart rate observations at each point in time are replaced with a moving median. For example, in the example above, an observation of 120 bpm would be replaced by the median of the three heart rates, 81 bpm. As a result, the heartbeat data containing steep noise is smoothed to the extent that it does not violate the nature of the heartbeat.

Although the median filter is used here as an example of the filter applied to the heartbeat data, the filter is not limited to this. It will be appreciated that other filters can be applied to the heart rate data, for example a low pass filter.

The first detection unit 33 is a processing unit that detects normal meals.

As one embodiment, the first detection unit 33 sets a first window having a predetermined time length, for example, 210 minutes, to the heartbeat data to which the median filter has been applied by the application unit 32 . Subsequently, the first detection unit 33 cuts out partial data corresponding to the section in which the first window is set. After the partial data corresponding to the first window is cut out in this way, the first detector 33 shifts the previously set first window by a predetermined shift width, for example, 5 minutes. After that, the first detection unit 33 cuts out partial data corresponding to the first window after the shift. Hereinafter, partial data obtained by extracting a portion corresponding to the first window from the heartbeat data may be referred to as "first window data".

For each first window data cut out in this way, the first detection unit 33 determines that a predetermined period, for example, 60 minutes from the start time of the first window data is before meals, and the first window data is started. A predetermined time after the time is defined as the "meal start time", and a feature value representing a change in heart rate corresponding to a normal meal is calculated.

Here, an example of a feature amount related to normal meals will be described. For example, changes in the heart rate that occur after the start of a normal meal include two peaks in which the heart rate rises (increases) and then falls (decreases) over time. That is, with the passage of time from the meal start time, the "first peak", which is the peak of the heart rate change that appears before the start of the normal meal, and the heart rate change that appears after the first peak. A "second peak" appears. In the following description, the predetermined region including the waveform of the first peak portion is referred to as "first peak region", and the predetermined region including the waveform of the second peak portion is referred to as "second peak region". ” may be stated.

Among these, the "first peak" is an increase in heart rate associated with eating, and is presumed to be an increase in heart rate due to, for example, mastication or peristalsis of the esophagus. Also, the "second peak" is presumed to be, for example, an increase in heart rate due to digestive activity in the digestive organs (gastrointestinal tract, etc.) for ingested food, ie, food. From this, it is possible to obtain knowledge that after a meal, the second peak appears after the appearance of the first peak, and that the second peak tends to last longer than the first peak.

Based on the tendencies of these first peaks and second peaks, the first detection unit 33 can, for example, calculate the following feature amount from the heartbeat data after application of the median filter. For example, the area of the first peak region a1, the area of the second peak region a2, and the like can be calculated as feature amounts. In addition, heart rate increase rate up to the first peak, heart rate increase rate up to the second peak, heart rate recovery rate from the first peak, and heart rate recovery rate from the second peak Velocity and the like can also be calculated as feature amounts. Further, the amplitude P1 of the peak of the waveform forming the first peak at which the heart rate takes the maximum value, and the amplitude P2 of the peak of the waveform forming the second peak at which the heart rate takes the maximum value are the feature values. can be calculated as In addition, when the meal start time is the starting point, the elapsed time t1 is the ending point when the peak amplitude P1 is measured, and the elapsed time t2 is the ending point when the heart rate recovers to a predetermined value after passing the peak amplitude P1. , the elapsed time t3 ending at the time when the peak amplitude P2 is measured, and the elapsed time t4 ending at the time when the heart rate recovers to a predetermined value after passing the peak amplitude P2 are calculated as feature quantities. can also In addition, the pre-meal heart rate, which is the lowest heart rate measured before the meal start time, and the increase in heart rate from the pre-meal heart rate to the heart rate at the meal start time in the section before the meal start time It is also possible to calculate the area of the region formed by the width as the feature amount.

After such a feature amount is calculated, the first detection unit 33 uses a model that outputs a certainty factor representing food-likeness with input of the feature amount vector obtained by vectorizing the above-described feature amount, and uses the first window A normal meal is detected by classifying the data, eg, having a normal meal or an atypical meal. For example, when the above model outputs values from −1 to +1, the first detection unit 33 inputs the feature amount vector calculated from the first window data for each first window data to the above model. . As a result, when the output value from the model is a predetermined value, for example, "0" or more, the first detection unit 33 classifies the first window data as a normal meal. On the other hand, when the output value from the meal estimation model is less than a predetermined value, for example, "0", the first detection unit 33 classifies the first window data as an unusual meal. In addition, in the above model, according to arbitrary algorithms such as support vector machines, boosting, and neural networks, partial data are labeled with the correct answer of positive examples “normal meals” or negative examples “abnormal meals”. It can be obtained by performing machine learning on the training data.

In this manner, the first detection unit 33 can classify the first window data for each first window data into a normal meal or an abnormal meal.

The second detection unit 34 is a processing unit that detects intake of a course meal. As shown in FIG. 1, the second detection section 34 has a detection section 34A, a setting section 34B, an extraction section 34C, a calculation section 34D and a determination section 34E.

The detection unit 34A is a processing unit that detects the above weak peak from the heartbeat data.

As one embodiment, the detection unit 34A detects the weak peak from the heartbeat data to which the median filter has been applied by the application unit 32 . For example, the detection unit 34A determines that the difference between the heart rate measured at the peak point detected from the heart rate data after application of the median filter and the heart rate measured at the start point where the heart rate rises to the peak point is A weak peak exceeding the above threshold Th1 is detected. Here, as an example of the above-mentioned "peak point", a convex zero-crossing point observed in the differential waveform of the heartbeat data is used. Further, as an example of the above-mentioned "starting point", a point at which the heart rate has completely decreased is used among the downward convex zero-crossing points observed before the peak point.

The setting unit 34B sets a second window for the heartbeat data.

As one embodiment, the setting unit 34B sets a second window having a predetermined time length, for example, 90 minutes, to the heartbeat data to which the median filter has been applied by the application unit 32 . The “second window” here is used differently from the first window for capturing the change from the second peak increase in heart rate that occurs with the intake of normal meals until the heart rate recovers to normal. That is, the second window is used, as one aspect, to capture multiple weak peaks that occur as multiple plates of food provided in a course meal are ingested. For example, the second window is set to the maximum length of time, for example, 90 minutes, which is expected to be required when a course meal is taken.

Subsequently, the setting unit 34B cuts out partial data corresponding to the section in which the second window is set. After the partial data corresponding to the second window is cut out in this way, the setting unit 34B shifts the previously set second window by a predetermined shift width, for example, 5 minutes. After that, the setting unit 34B cuts out the partial data corresponding to the second window after the shift. Hereinafter, the partial data obtained by extracting the portion corresponding to the second window from the heartbeat data may be referred to as "second window data".

The extraction unit 34C is a processing unit that extracts a long-term trend corresponding to changes in the heart rate that occur as the course meal is digested from the second window data.

Here, the above-mentioned long-term trend is extracted from the heartbeat data by estimating the effect of the long-term trend that appears in response to the digestion of the early dish, which is superimposed on the weak peak that appears in response to the intake of the final dish in the course meal. It is for removing.

That is, as the order of menus provided in the course meal progresses, the digestive activity of the food ingested before the order also progresses. In this way, in addition to the eating act of ingesting the food on the plate, for example, the peristaltic movement of the esophagus, the digestive activity of the food ingested in the order before that is performed in parallel, so In comparison, there is a tendency for the increase in heart rate to increase at the end of the menu. In the example of FIG. 3, the amplitude of the weak peak tends to increase in the order of the first, second, and third dishes, so the end of the menu is larger than the beginning of the menu. In this way, while the changes in the heart rate appearing with the digestion of the food in the early stage are superimposed on the weak peaks appearing corresponding to the late stage dishes, as one aspect, the feature values indexed from the late weak peaks, For example, amplitude and area may be overestimated. This is one of the reasons why it is difficult to detect a meal of a course meal by evaluating the characteristics of the weak peaks appearing corresponding to each plate of the course meal. From this, the long-term trend is extracted from the second window data from the aspect of removing the effect of heart rate variation of digestion superimposed on the weak peak.

At this time, the extraction unit 34C does not necessarily have to extract the long-term trend from all the second window data extracted from the heartbeat data by the setting unit 34B. For example, every time the setting unit 34B cuts out the second window data, the extraction unit 34C determines whether the second window data overlaps with the first window data classified as normal meals by the first detection unit 33. determine whether or not At this time, when the first window data and the second window data classified into the normal meal class overlap, the meal is already detected by the first detection unit 33, so it is not necessarily detected again as a course meal. It doesn't matter if you don't fix it. On the other hand, when the first window data and the second window data classified into the normal meal class do not overlap, the features of the course meal, which are difficult to evaluate with the feature amount of the normal meal calculated from the first window data, are overlooked. There remains the possibility that Therefore, the extraction unit 34C further determines whether or not the number of weak peaks included in the second window data is a predetermined number, for example, "3" or more. At this time, when the number of weak peaks included in the second window data is less than the predetermined number, the possibility that the food on the plurality of dishes has not been eaten increases. In this case, it can be identified that the course meal is not being served, and subsequent processing can be stopped. On the other hand, when the number of weak peaks included in the second window data is equal to or greater than the predetermined number, the possibility that multiple plates of food have been eaten increases. In this case, extraction of the long-term trend is started.

For example, the extraction unit 34C can extract a long-term trend by performing the following processing on the second window data. Specifically, the extraction unit 34C extracts a heartbeat waveform corresponding to a long-term trend by applying a predetermined filter, such as a low-pass filter, to the second window data. At this time, 0.001 Hz can be set as an example of a cutoff frequency in the low-pass filter from the aspect of removing the short-term trend component corresponding to the weak peak group. After that, the extraction unit 34C calculates, as a trend coefficient, the slope of the regression line obtained by applying linear regression analysis to the heartbeat waveform of the long-term trend extracted from the second window data. Then, the extraction unit 34C determines whether or not the trend coefficient of the long-term trend is a positive value, ie, whether or not the long-term trend is an upward-sloping straight line. At this time, when the trend coefficient of the long-term trend is a positive value, it is highly possible that digestion activity is observed along with the eating activity of the food from the second window data, so the detection of the meal of the course meal is continued. . On the other hand, if the trend coefficient of the long-term trend is not a positive value, there is a high possibility that the digestive activity cannot be observed from the second window data, so the detection of the meal of the course meal is suspended.

FIG. 6 is a diagram showing an example of heartbeat data. The vertical axis of the graph shown in FIG. 6 indicates heart rate (bpm), and the horizontal axis indicates time. FIG. 6 shows the heartbeat data H6 after application of the median filter. As shown in FIG. 6, eight weak peaks p61 to p68 whose difference in heart rate between the peak point and its starting point is equal to or greater than the threshold Th1 are detected from the heartbeat data H6. After weak peaks are detected from the heartbeat data H6 in this manner, a second window is set for the heartbeat data H6.

FIG. 7 is a diagram showing an example of second window data. In FIG. 7, along with the heartbeat data H6, the interval corresponding to the second window W2 set in the heartbeat data H6 is indicated by hatching. As shown in FIG. 7, the heartbeat data H6 is set with a second window of the maximum length of time that is expected to take when the course meal is taken, for example, about 90 minutes. As a result, the partial data corresponding to the second window W2, that is, the second window data is cut out from the heartbeat data H6.

FIG. 8 is a diagram showing an example of a long-term trend. In FIG. 8, partial data corresponding to the second window W2 of the heartbeat data H6 is extracted as the second window data 80 and shown. It is assumed that the second window data 80 shown in FIG. 8 do not overlap with the first window data classified as normal meals. As shown in FIG. 8, the second window data 80 includes three weak peaks p62-p64, so a long-term trend is extracted. That is, by applying a low-pass filter to the second window data 80, the long-term heartbeat waveform N1 is extracted from which short-term trend components corresponding to the three weak peaks p62 to p64 are removed. A linear regression analysis is applied to the heartbeat waveform N1 of the long-term trend extracted in this way.

FIG. 9 is a diagram showing an example of a regression line. FIG. 9 shows a regression line L1 obtained by applying linear regression analysis to the long-term trend heartbeat waveform N1 shown in FIG. As shown in FIG. 9, the regression line L1 of the long-term trend is an upward-sloping straight line, so the slope of the regression line L1, that is, the trend coefficient, is observed as a positive value. In this case, from the second window data 80, the digestive activity is observed along with the eating activity of the food, and the possibility that the course meal will be eaten in the section corresponding to the second window W2 increases. Therefore, the second window data 80 is processed to calculate the feature amount of the weak peak group.

The calculation unit 34D is a processing unit that calculates the feature amount of the weak peak group.

As one embodiment, the calculation unit 34D can remove the long-term trend from the second window data prior to calculating the feature amount of the weak peak group. For example, the calculation unit 34D subtracts the heartbeat waveform of the long-term trend extracted by the extraction unit 34C from the heartbeat waveform of the second window data cut out by setting the second window by the setting unit 34B, A short-term trend heartbeat waveform corresponding to the weak peak group is calculated. At this time, the calculation unit 34D adjusts the baseline of the heartbeat waveform of the short-term trend so that the heartbeat value at the starting point of the first weak peak in the weak peak group included in the second window data is zero. . For example, the calculation unit 34D subtracts the value of the heart rate at the starting point of the top weak peak from the heart rate at each time in the heart rate waveform of the short-term trend. The reason for adjusting the baseline in this way is that the heart rate value at the start point of the first weak peak may become negative by subtracting the heart rate waveform of the long-term trend from the heart rate waveform of the second window data. is. In this case, as one of the feature values of the weak peak group, when the area of the weak peak is calculated, the heart rate at the starting point of the leading weak peak falls to a negative value, so the area of the leading weak peak is A situation may arise in which the area is discounted compared to the areas of other weak peaks. To avoid this, the baseline of the short-term trend heartbeat waveform is adjusted.

After the baseline of the heartbeat waveform of the short-term trend is adjusted in this way, the calculation unit 34D calculates the start point and the weak peak at which the heartbeat starts to rise from the heartbeat waveform of the short-term trend after the baseline adjustment to the peak point of the weak peak. Identify the end point where the heart rate descent from the peak point of . FIG. 10 is a diagram showing an example of short-term trends. FIG. 10 shows a short-term heartbeat waveform D in which the baseline has been adjusted after the long-term heartbeat waveform N1 has been removed from the heartbeat waveform of the second window data 80 shown in FIG. As shown in FIG. 10, the calculation unit 34D determines the start point at which the heart rate starts to rise to the peak point of the weak peak and the end point at which the heart rate drop from the peak point of the weak peak ends for each of the weak peaks p62 to p64. Identify. For example, in the case of the weak peak p62, as indicated by the triangular mark, the starting point p62s is specified as the point before the peak point of the weak peak p62 and at which the heart rate completely drops. Furthermore, as indicated by the cross mark, the point at which the heart rate completely drops after the peak point of the weak peak p62 is identified as the end point p62e. Furthermore, similarly for the weak peak p63 and the weak peak p64, a start point p63s and an end point p63e, and a start point p64s and an end point p64e are specified.

After the start and end points of these weak peaks are identified, the calculation unit 34D calculates (1) the area of the weak peak, (2) the appearance interval of the weak peak, and (3) the feature amount of the weak peak group. Calculate the duration of the weak peak.

(1) Areas of weak peaks For example, in the example of weak peaks p62 to p64 shown in FIG. be. For example, in the section from the start point p62s of the weak peak p62 to the end of the end point p62e, the area S62 of the figure surrounded by the heartbeat waveform D of the short-term trend and the baseline BL3 is calculated as the feature amount of the weak peak p62. Similarly, in the interval from the start point p63s of the weak peak p63 to the end of the end point p63e, the area S63 of the figure surrounded by the heartbeat waveform D of the short-term trend and the baseline BL3 is calculated as the feature amount of the weak peak p63. be done. Furthermore, in the section from the start point p64s of the weak peak p64 to the end of the end point p64e, the area S64 of the figure surrounded by the heartbeat waveform D of the short-term trend and the baseline BL3 is calculated as the feature amount of the weak peak p64.

(2) Weak peak appearance intervals In addition, in the example of weak peaks p62 to p64 shown in FIG. 10, since three weak peaks p62 to p64 are included, 2 (=3-1) appearance intervals are obtained be done. That is, the period from the end point p62e of the weak peak p62 to the start point p63s of the weak peak p63 is calculated as the appearance interval T11. Along with this, the period from the end point p63e of the weak peak p63 to the start point p64s of the weak peak p64 is calculated as the appearance interval T12.

(3) Duration time of weak peaks Furthermore, in the example of weak peaks p62 to p64 shown in FIG. calculated as For example, in the case of the weak peak p62, the length of time from the start point p62s of the weak peak p62 to the end of the end point p62e of the weak peak p62 is calculated as the duration C62. In the case of the weak peak p63, the length of time from the start point p63s of the weak peak p63 to the end of the end point p63e of the weak peak p63 is calculated as the duration C63. Furthermore, in the case of the weak peak p64, the length of time from the start point p64s of the weak peak p64 to the end of the end point p64e of the weak peak p64 is calculated as the duration C64.

The determination unit 34E is a processing unit that determines whether or not the course meal has been ingested. As an example only, the determination unit 34E determines whether or not the feature amount of the weak peak group calculated by the calculation unit 34D satisfies a predetermined condition.

As one aspect, the determination unit 34E determines whether the area of each weak peak satisfies a first condition. For example, the determination unit 34E determines whether or not the order regarding the magnitude of the feature amount of each weak peak corresponds to the order regarding the magnitude of the intake of each dish of the course meal. In other words, in the course meal, there is an aspect that the amount of food consumed is larger as the turn is later, so the intake of each dish is the amount of intake of the first dish < the amount of intake of the second dish < the amount of the third dish There is a high possibility that it will be an order of size relationship such as intake. Based on this, the determination unit 34E determines whether or not the area of each weak peak increases in order from the leading weak peak. For example, in the example of three weak peaks p62 to p64 shown in FIG. It is determined whether or not the order of the area S63<the area S64 of the weak peak p64, that is, the order of the intake provided on each dish of the course meal.

As another aspect, the determination unit 34E determines whether or not the weak peak appearance interval satisfies the second condition. For example, the determination unit 34E determines whether or not the intervals at which the weak peaks appear correspond to the intervals at which each plate of food is served in the course meal. Taking the example of the three weak peaks p62 to p64 shown in FIG. It is determined whether it is within the interval.

As a further aspect, the determination unit 34E determines whether or not the duration of each weak peak satisfies the third condition. That is, in the course meal, it takes a certain amount of time for each plate to finish eating the food of the plate. In this way, the time required for ingestion of food per dish is sometimes referred to as "required intake time". This required time for eating should be shorter than the time required for a normal meal, because the amount of food consumed per plate of course meals is smaller than the amount of food consumed for the entire meal, such as set meals and set menus. can be set. Based on this, the determination unit 34E determines whether or not the duration of the weak peak is within the required intake time per dish. For example, in the example of three weak peaks p62 to p64 shown in FIG. It is determined whether or not it is within the required intake time per plate set by Dmin and the upper limit value Dmax.

Here, when the feature amount of each weak peak satisfies the first condition, there is a high possibility that the change in the elevation of heart rate of the subject corresponds to the change in intake of each dish provided according to the course meal menu. Also, if the appearance interval of the weak peaks satisfies the second condition, there is a high possibility that the intervals at which the target person's heart rate rises occur correspond to the intervals at which each plate of food is served in the course meal. In other words, the increased heart rate is less likely to be caused by disturbances other than diet, such as exercise or mental factors. Furthermore, when the duration of each weak peak satisfies the third condition, the possibility increases that the duration of the increase in heart rate of the subject corresponds to the period until the food of each dish in the course meal is finished being ingested.

FIG. 11 is a diagram showing an example of weak peaks. FIG. 11 shows, in chronological order, a group of weak peaks that appear due to factors other than meals, such as conversation and tension. As shown in FIG. 11, when a weak peak appears due to conversation or tension, the increase in heart rate that appears as a weak peak, such as the area and amplitude, the interval between weak peaks, and the duration of the weak peak, become irregular. Therefore, even if a plurality of weak peaks are detected from the heartbeat waveform D of the short-term trend, it is unlikely that all of the first to third conditions are met by chance. For this reason, condition determination using the first to third conditions prevents a weak peak group that appears due to factors other than meals, such as conversation and tension, from being erroneously detected as course meals. can.

From these aspects, the determination unit 34E determines that the feature amount of each weak peak satisfies the first condition, the appearance interval of the weak peak satisfies the second condition, and the duration of each weak peak satisfies the third condition. If the condition is satisfied, it is determined that the course meal was ingested in the second window data. In this case, the determination unit 34E calculates the meal time of the course meal. For example, in the example of the short-term trend heartbeat waveform D shown in FIG. Set to end time. Then, the determination unit 34E calculates the meal time of the course meal by calculating the difference between the meal start time and the meal end time, for example, "meal end time - meal start time".

Here, as an example of the order of the amount of intake provided in each dish of the course meal, the order of magnitude relationship such as the amount of intake of the first dish < the amount of intake of the second dish < the amount of intake of the third dish is illustrated. However, the order of intakes provided in each course meal is not necessarily limited to the above examples.

For example, in course meals, depending on the menu, the intake of each dish may not necessarily satisfy the above order. FIG. 12 is a diagram showing an example of weak peaks. FIG. 12 shows weak peaks that appear in time series when each dish is served in the order of appetizer, main dish, and dessert. For example, when an appetizer, main dish, and dessert are served, the order of intake is as follows: intake of dessert (third dish) < intake of appetizer (first dish) < intake of main (second dish) It becomes the order of size relation such as quantity. The order of these intakes also agrees with the order of magnitude of feature values of each weak peak. That is, as shown in FIG. 12, the areas of the weak peaks are in order of the weak peak appearing corresponding to the dessert, the weak peak appearing corresponding to the appetizer, and the weak peak appearing corresponding to the main dish. In this way, since the course menu is not uniform, the order of intakes to be served on each dish is set for each pattern of the course menu, and each weak point is set for each pattern of the course menu. It is also possible to determine whether or not the order regarding the magnitude of the feature amount of the peak corresponds to the order regarding the magnitude of the intake of each dish defined in the pattern. In this case, it may be determined that the first condition is satisfied when the order regarding the magnitude of the feature amount of each weak peak corresponds to the order of any pattern of the pattern of the course meal menu.

The providing unit 35 is a processing unit that provides the healthcare support service described above.

As one embodiment, the providing unit 35 records the meal time for the normal meal detected by the first detection unit 33 and the meal time for the meal of the course meal detected by the second detection unit 34. Generate and output a list of meal time periods over a predetermined period, such as one week, from the meal times recorded until then, or analyze eating habits or diets from the meal times recorded so far to output various advices. It should be noted that the processing corresponding to the healthcare support service mentioned here may be performed by a service or application executed on the back end, AI, or a computer outside the server device 30 . In addition, here, the case where the meal time is used for the above healthcare support service is exemplified, but the meal time does not necessarily have to be used for the above healthcare support service. Alternatively, at least one of these can be used for the above healthcare support service.

FIG. 13 is a diagram showing an example of a healthcare support service. FIG. 13 shows an output example when lifestyle monitoring is performed. Note that the display layout shown in FIG. 13 is merely an example, and needless to say, the display layout is not limited to the example shown in FIG.

As shown in FIG. 13, it is possible to display a timetable regarding meal times for each day included in one week. In such a timetable, the display form is distinguished between the meal time zone in which a normal meal is served and the meal time zone in which a course meal is served. This distinction of display form may be realized by changing the display color, or may be realized by changing the font or filling. This makes it possible to distinguish whether the meal event is a normal meal or a course meal. In addition to such a timetable, messages such as advice on lifestyle habits can also be displayed. For example, if the frequency at which dinner is held after 9:00 p.m. among meal events exceeds a predetermined frequency, for example, twice a week, as shown in FIG. ” can be displayed. In addition, if there is a day with no meal event between 6:00 am and 12:00 noon, a message such as ``Let's make it a habit to eat breakfast'' can be displayed.

[Process flow]
Next, the processing flow of the server device 30 according to this embodiment will be described. Here, (1) healthcare support processing executed by server device 30 will be described, and then (2) second detection processing executed as a subroutine of healthcare support processing will be described.

(1) Healthcare Assistance Processing FIG. 14 is a flowchart illustrating procedures of healthcare assistance processing according to the first embodiment. This process is started, for example, each time heartbeat data is acquired from the heartbeat sensor 11 mounted on the wearable terminal 10 .

As shown in FIG. 14, when heartbeat data is acquired by the acquisition unit 31 (step S101), the application unit 32 applies a median filter for extracting a median value to the heartbeat data acquired in step S101 (step S102). ). As a result, it is possible to obtain heartbeat data that has been smoothly filtered to such an extent that the heartbeat data containing steep noise does not contradict the nature of the heartbeat.

Subsequently, the first detection unit 33 executes a "first detection process" for detecting a normal meal from the heartbeat data to which the median filter has been applied in step S102 (step S103). Furthermore, the second detection unit 34 executes a "second detection process" for detecting a course meal from the heartbeat data to which the median filter is applied in step S102 (step S104).

After that, the providing unit 35 outputs an output corresponding to the above-mentioned healthcare support service, for example, a time table, based on the meal time for the normal meal detected in step S103 and the meal time for the course meal detected in step S104. and advice are generated and output (step S105), and the process ends.

(2) Second Detection Processing FIG. 15 is a flowchart illustrating the procedure of the second detection processing according to the first embodiment. This process corresponds, as one aspect, to the process of step S104 shown in FIG. As shown in FIG. 15, the detection unit 34A detects the heart rate measured at the peak point detected from the heart rate data to which the median filter is applied in step S102, and A weak peak whose difference from the measured heart rate exceeds the threshold Th1 is detected (step S201).

Subsequently, the setting unit 34B sets a second window having a predetermined time length, eg, 90 minutes, in the heartbeat data to which the median filter was applied in step S102 (step S202). As a result, the partial data of the section corresponding to the second window is cut out as the second window data from the heartbeat data to which the median filter has been applied.

Then, the extraction unit 34C duplicates the second window data extracted by setting the second window in step S202 and the first window data classified as normal meals in the first detection process of step S104. It is determined whether or not to do so (step S203).

At this time, if the first window data and the second window data classified into the normal meal class overlap (Yes in step S203), the meal is already detected by the first detection unit 33, so the course meal is not necessarily re-started. It does not matter if it is not re-detected as a meal. In this case, the process proceeds to step S213.

On the other hand, if the first window data and the second window data classified into the normal meal class do not overlap (No in step S203), the course meal that is difficult to evaluate with the feature amount of the normal meal calculated from the first window data. It is possible that features have been overlooked. Therefore, the extraction unit 34C further determines whether or not the number of weak peaks included in the second window data is a predetermined number, for example, "3" or more (step S204).

At this time, if the number of weak peaks included in the second window data is less than the predetermined number (No at step S204), it is highly likely that the food on multiple plates has not been eaten. In this case, it can be identified that the course meal is not being served. In this case, the process proceeds to step S213.

On the other hand, if the number of weak peaks included in the second window data is equal to or greater than the predetermined number (Yes in step S204), the possibility that multiple plates of food have been eaten increases. In this case, the extraction unit 34C extracts a heartbeat waveform corresponding to the long-term trend by applying a predetermined filter, such as a low-pass filter, to the second window data obtained in step S202 (step S205).

Subsequently, the extraction unit 34C determines whether the slope of the regression line obtained by applying the linear regression analysis to the heartbeat waveform of the long-term trend extracted in step S205 is a positive value, that is, whether the long-term trend is right shoulder. It is determined whether or not it is an upward straight line (step S206).

At this time, if the trend coefficient of the long-term trend is not a positive value (step S206 No), there is a high possibility that the digestive activity cannot be observed from the second window data. In this case, the process proceeds to step S213.

Here, if the trend coefficient of the long-term trend is a positive value (Yes in step S206), there is a high possibility that both the eating activity and the digestive activity of the food are observed from the second window data. is continued.

That is, the calculation unit 34D subtracts the heartbeat waveform of the long-term trend extracted in step S205 from the heartbeat waveform of the second window data obtained in step S202, thereby obtaining the heartbeat waveform of the short-term trend corresponding to the weak peak group. is calculated (step S207).

At this time, the calculation unit 34D adjusts the baseline of the heartbeat waveform of the short-term trend so that the heartbeat value at the starting point of the first weak peak in the weak peak group included in the second window data is zero. (Step S208).

After that, the calculation unit 34D identifies the start point at which the heart rate starts to rise from the heart rate waveform of the short-term trend after the baseline adjustment to the peak point of the weak peak and the end point at which the heart rate decline from the peak point of the weak peak ends. (Step S209). After that, the calculation unit 34D calculates the area of the weak peak, the appearance interval of the weak peak, and the duration of the weak peak as feature amounts of the weak peak group (Step S210).

Then, the determination unit 34E determines whether or not the feature amount of the weak peak group calculated in step S210 satisfies the first, second, and third conditions (step S211).

Here, when the feature amount of each weak peak group satisfies the first condition, the second condition, and the third condition (step S211 Yes), it is determined that the course meal was ingested in the second window data. In this case, the determining unit 34E sets the starting point of the first weak peak to the meal start time and the peak point of the trailing weak peak to the meal end time, and then calculates the difference between the meal start time and the meal end time. By calculating, the meal time of the course meal is calculated (step S212).

Then, the process proceeds to step S202 until the setting of the second window for the heartbeat data to which the median filter is applied is completed (step S213 No). That is, the setting unit 34B sets the previously set second window by shifting it by a predetermined shift width, and cuts out the partial data of the section corresponding to the shifted second window as the second window data. After that, the processing from step S203 to step S212 is executed.

After that, and when the setting of the second window for the heartbeat data to which the median filter has been applied is finished (step S213 Yes), the process is finished.

[One aspect of the effect]
As described above, when the heartbeat data includes a plurality of peaks, the server device 30 according to the present embodiment determines whether the course meal is ingested based on the feature amount calculated from each peak and the appearance interval of the peaks. determine whether or not This makes it possible to determine eating or non-meal by capturing the characteristics of heart rate changes that appear when a course meal is ingested. Therefore, according to the server device 30 of the present embodiment, it is possible to realize the detection of a course meal.

Although embodiments of the disclosed apparatus have been described so far, the present invention may be embodied in various forms other than the embodiments described above. Therefore, other embodiments included in the present invention will be described below.

[Course meal pattern]
In a course meal, depending on the menu, the intake amount of each dish may not necessarily satisfy the order given in the example 1 above. FIG. 16 is a diagram showing an example of weak peaks. FIG. 16 shows, in chronological order, weak peaks appearing in a French cuisine menu in which each dish is served in the order of appetizer, soup, main 1, main 2, and dessert. In the case of this menu, the order of the amount of intake is as follows: amount of dessert (fifth dish) < amount of appetizer (first dish) ≒ amount of soup (second dish) < main 1 (third dish) ) ≒ intake of main 2 (fourth dish). The order of these intakes, as shown in FIG. 16, also coincides with the order of magnitude of feature amounts of the weak peaks. That is, as shown in FIG. 16, the areas of the weak peaks are also in the order of the weak peaks corresponding to dessert, the weak peaks corresponding to appetizers and soup, and the weak peaks corresponding to main 1 and main 2. Become.

FIG. 17 is a diagram showing an example of weak peaks. FIG. 17 shows weak peaks appearing in a kaiseki cuisine menu in which each plate is served in the order of appetizer, oshigi, bowl, hassun, grilled dish, rice, and sweets in chronological order. In the case of this menu, the order of the amount of intake is as follows: amount of appetizer (first dish) ≒ amount of oshigi (second dish) < amount of sweets (seventh dish) < bowl (three dishes) second) ≒ intake of hassun (fourth dish) < intake of rice (sixth dish) < intake of pottery (fifth dish). The order of these intakes, as shown in FIG. 17, also coincides with the order of magnitude of feature amounts of the weak peaks. That is, as shown in FIG. 17, the areas of the weak peaks are also weak peaks corresponding to appetizers and oshinogi, weak peaks corresponding to sweets, weak peaks corresponding to bowls and hassun, rice The weak peaks appearing corresponding to , and the weak peaks appearing corresponding to pottery are in that order.

FIG. 18 is a diagram showing an example of weak peaks. FIG. 17 shows, in chronological order, weak peaks appearing in a lunch course menu in which each dish is served in the order of appetizer, main dish, and dessert. In the case of this menu, the order of the size relationship of the intake is the amount of dessert (third dish) < the amount of appetizer (first dish) < the amount of main (third dish). . The order of these intakes, as shown in FIG. 18, also coincides with the order of magnitude of feature amounts of the weak peaks. That is, as shown in FIG. 18, the areas of the weak peaks are also in the order of the weak peak appearing corresponding to the dessert, the weak peak appearing corresponding to the appetizer, and the weak peak appearing corresponding to the main dish.

In this way, since the course menu is not uniform, the order of intakes to be served on each dish is set for each pattern of the course menu, and each weak point is set for each pattern of the course menu. It is also possible to determine whether or not the order regarding the magnitude of the feature amount of the peak corresponds to the order regarding the magnitude of the intake of each dish defined in the pattern. In this case, it may be determined that the first condition is satisfied when the order regarding the magnitude of the feature amount of each weak peak corresponds to the order of any pattern of the pattern of the course meal menu.

[Standalone]
In the first embodiment, the healthcare support system 1 is constructed as a client-server system, but it does not necessarily have to be constructed as a client-server system. For example, the processing shown in FIGS. 14 and 15 executed by the functional units of the server device 30 may be executed stand-alone by the IoT device 20 connected to the wearable terminal 10 . Furthermore, the wearable terminal 10 and the IoT device 20 do not necessarily have to be configured as separate devices. 14 and 15, which are executed by the functional units of the server device 30, may be executed stand-alone by a computer equipped with the heart rate sensor 11. FIG.

Distributed and integrated
Also, each component of each illustrated device may not necessarily be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to the one shown in the figure, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Can be integrated and configured. For example, the acquisition unit 31, the application unit 32, the first detection unit 33, or the second detection unit 34 may be connected via a network as external devices of the server device 30. FIG. Further, different devices each have the acquisition unit 31, the application unit 32, the first detection unit 33, or the detection unit 34, and are connected to a network and cooperate with each other to realize the functions of the server device 30 described above. may

[Meal detection program]
Moreover, various processes described in the above embodiments can be realized by executing a prepared program on a computer such as a personal computer or a work station. Therefore, an example of a computer that executes a meal detection program having functions similar to those of the above embodiments will be described below with reference to FIG. 19 .

FIG. 19 is a diagram illustrating an example hardware configuration of a computer that executes the meal detection program according to the first and second embodiments. As shown in FIG. 19, the computer 100 has an operation section 110a, a speaker 110b, a camera 110c, a display 120, and a communication section . Furthermore, this computer 100 has a CPU 150 , a ROM 160 , an HDD 170 and a RAM 180 . Each part of these 110 to 180 is connected via a bus 140 .

As shown in FIG. 19, the HDD 170 has a meal detection program 170a that exhibits the same functions as the acquisition unit 31, the application unit 32, the first detection unit 33, and the second detection unit 34 shown in the first embodiment. remembered. The meal detection program 170a may be integrated or separated like the components of the acquisition unit 31, the application unit 32, the first detection unit 33, and the second detection unit 34 shown in FIG. That is, the HDD 170 does not necessarily store all the data shown in the first embodiment, and the HDD 170 only needs to store data used for processing.

Under such an environment, the CPU 150 reads the meal detection program 170 a from the HDD 170 and loads it into the RAM 180 . As a result, the meal detection program 170a functions as a meal detection process 180a, as shown in FIG. The meal detection process 180a deploys various data read from the HDD 170 in an area assigned to the meal detection process 180a among the storage areas of the RAM 180, and executes various processes using the deployed various data. For example, examples of processing executed by the meal detection process 180a include the processing shown in FIGS. 14 and 15. FIG. Note that the CPU 150 does not necessarily have to operate all the processing units described in the first embodiment, as long as the processing units corresponding to the processes to be executed are virtually realized.

Note that the meal detection program 170a described above does not necessarily have to be stored in the HDD 170 or the ROM 160 from the beginning. For example, the meal detection program 170a is stored in a “portable physical medium” such as a flexible disk inserted into the computer 100, so-called FD, CD-ROM, DVD disk, magneto-optical disk, IC card, or the like. Then, the computer 100 may acquire and execute the meal detection program 170a from these portable physical media. In addition, the meal detection program 170a is stored in another computer or server device connected to the computer 100 via a public line, the Internet, LAN, WAN, etc., and the computer 100 acquires the meal detection program 170a from these. You can also run

The following notes are further disclosed with respect to the embodiments including the above examples.

(Appendix 1) Acquiring the heart rate data of the subject,
Detecting a peak whose amplitude exceeds a predetermined threshold in the acquired heartbeat data,
When the number of detected peaks is plural, the course meal is consumed by the subject based on the amplitude or area of the peaks determined for each of the detected peaks and the appearance interval of the peaks determined between the detected peaks. determine whether or not
A meal detection program that lets a computer do the work.

(Supplementary note 2) In the determination process, the order regarding the magnitude of the amplitude or area of the peaks obtained for each of the detected peaks corresponds to the order regarding the magnitude of the intake of each item included in the menu of the course meal, and The meal detection program according to appendix 1, wherein the meal detection program determines whether or not the appearance intervals of the peaks found between the detected peaks are within the range of the intervals at which each item is provided in the course meal menu.

(Supplementary note 3) The meal detection program according to Supplementary note 1, wherein the judging process further determines whether or not the course meal has been ingested based on the peak duration time obtained for each of the detected peaks.

(Supplementary Note 4) The meal according to Supplementary Note 3, wherein the determining process determines whether or not the duration of the peak obtained for each of the detected peaks is within the range of the required intake time per item of the course meal. detection program.

(Appendix 5) In the determination process, if the slope of a straight line approximating the second heartbeat waveform in which the filter is applied to the first heartbeat waveform of the heartbeat data is a positive value, whether the course meal was ingested 1. The meal detection program according to appendix 1, which executes the determination of whether or not.

(Appendix 6) Calculate a third heartbeat waveform by subtracting the second heartbeat waveform to which the filter is applied to the heartbeat data from the first heartbeat waveform of the heartbeat data, and calculate the peak from the third heartbeat waveform The meal detection program according to appendix 1, which causes the computer to execute a process of calculating the amplitude or area and the appearance intervals of the peaks.

(Appendix 7) Acquiring the heartbeat data of the subject,
Detecting a peak whose amplitude exceeds a predetermined threshold in the acquired heartbeat data,
When the number of detected peaks is plural, the course meal is consumed by the subject based on the amplitude or area of the peaks determined for each of the detected peaks and the appearance interval of the peaks determined between the detected peaks. determine whether or not
A meal detection method in which the processing is computer-implemented.

(Supplementary note 8) In the determination process, the order regarding the magnitude of the amplitude or area of the peaks obtained for each of the detected peaks corresponds to the order regarding the magnitude of the intake of each item included in the menu of the course meal, and 8. The meal detection method according to appendix 7, wherein it is determined whether or not a peak appearance interval obtained between the detected peaks is within a range of intervals at which each item is provided in the course meal menu.

(Supplementary note 9) The meal detection method according to Supplementary note 7, wherein the determining process further determines whether or not the course meal has been ingested based on the peak duration time obtained for each of the detected peaks.

(Supplementary note 10) The meal according to Supplementary note 9, wherein the determination process determines whether or not the duration of the peak obtained for each of the detected peaks is within the range of the required intake time per item of the course meal. Detection method.

(Appendix 11) In the determination process, if the slope of a straight line that approximates the second heartbeat waveform obtained by applying a filter to the first heartbeat waveform of the heartbeat data has a positive value, is the course meal ingested? 8. The meal detection method according to appendix 7, wherein the determination of whether or not is performed.

(Appendix 12) Calculate a third heartbeat waveform by subtracting the second heartbeat waveform to which the filter is applied to the heartbeat data from the first heartbeat waveform of the heartbeat data, and calculate the third heartbeat waveform from the third heartbeat waveform, The meal detection method according to Supplementary Note 7, wherein the computer executes a process of calculating an amplitude or an area and an appearance interval of the peaks.

(Appendix 13) an acquisition unit that acquires heartbeat data of a subject;
a detection unit that detects a peak whose amplitude exceeds a predetermined threshold in the acquired heartbeat data;
When the number of detected peaks is plural, the course meal is consumed by the subject based on the amplitude or area of the peaks determined for each of the detected peaks and the appearance interval of the peaks determined between the detected peaks. A determination unit that determines whether or not
a meal detection device having a

(Supplementary Note 14) The determination unit determines that the order regarding the magnitude of the amplitude or area of the peaks obtained for each of the detected peaks corresponds to the order regarding the magnitude of the intake of each item included in the menu of the course meal, and the detection 14. The meal detection device according to appendix 13, wherein the meal detection device according to appendix 13 determines whether or not the appearance interval of the peaks found between the peaks obtained is within the range of intervals at which each item is provided in the course meal menu.

(Supplementary Note 15) The meal detection device according to Supplementary Note 13, wherein the determination unit determines whether or not the course meal has been ingested, further based on the peak duration time obtained for each of the detected peaks.

(Supplementary note 16) The meal detection according to Supplementary note 15, wherein the determination unit determines whether the duration of the peak obtained for each of the detected peaks is within the range of the required intake time per item of the course meal. Device.

(Supplementary Note 17) When the inclination of a straight line obtained by approximating the second heartbeat waveform obtained by applying a filter to the first heartbeat waveform of the heartbeat data has a positive value, the determination unit determines whether the course meal has been ingested. 14. The meal detection device according to appendix 13, which determines whether or not.

(Appendix 18) A third heartbeat waveform is calculated by subtracting the second heartbeat waveform to which the filter is applied to the heartbeat data from the first heartbeat waveform of the heartbeat data, and the peak of the heartbeat waveform is calculated from the third heartbeat waveform. 14. The meal detection device according to appendix 13, further comprising a calculator that calculates the amplitude or area and the appearance intervals of the peaks.

1 healthcare support system 10 wearable terminal 20 IoT device 30 server apparatus 31 acquisition unit 32 application unit 33 first detection unit 34 second detection unit 34A detection unit 34B setting unit 34C extraction unit 34D calculation unit 34E determination unit 35 provision unit

Claims (5)

Get the heart rate data of the subject,
Detecting a peak whose amplitude exceeds a predetermined threshold in the acquired heartbeat data,
The order regarding the magnitude of the amplitude or area of the peaks determined for each of the detected peaks corresponds to the order regarding the magnitude of the intake of each item included in the course meal menu, and the appearance interval of the peaks determined between the detected peaks. Determining that the course meal has been ingested by the subject when each item on the course meal menu is within the range of intervals provided,
A meal detection program that lets a computer do the work. 2. The meal detection program according to claim 1, wherein said determination processing determines that said course meal has been ingested when each of the peak durations obtained for each of said detected peaks is within a specific time . The determining process determines whether or not the course meal has been ingested when the slope of a straight line obtained by approximating the filtered second heartbeat waveform to the first heartbeat waveform of the heartbeat data has a positive value. 3. The meal detection program according to claim 1 or 2 , which executes Get the heart rate data of the subject,
Detecting a peak whose amplitude exceeds a predetermined threshold in the acquired heartbeat data,
The order regarding the magnitude of the amplitude or area of the peaks determined for each of the detected peaks corresponds to the order regarding the magnitude of the intake of each item included in the course meal menu, and the appearance interval of the peaks determined between the detected peaks. Determining that the course meal has been ingested by the subject when each item on the course meal menu is within the range of intervals provided,
A meal detection method in which the processing is computer-implemented. an acquisition unit that acquires heartbeat data of a subject;
a detection unit that detects a peak whose amplitude exceeds a predetermined threshold in the acquired heartbeat data;
The order regarding the magnitude of the amplitude or area of the peaks determined for each of the detected peaks corresponds to the order regarding the magnitude of the intake of each item included in the course meal menu, and the appearance interval of the peaks determined between the detected peaks. a determination unit that determines that the course meal has been ingested by the subject when each item on the course meal menu is provided within a range of intervals;
a meal detection device having a

Images