JP7705826B2 - Information processing device and information processing method - Google Patents

Description

The present invention relates to an information processing device and an information processing method.

Conventionally, systems are known that are used when a caregiver provides care to a person being assisted. Patent Document 1 discloses a method in which a sensor is placed in a living space, and information is generated about the condition of the resident living in the living space based on changes over time in the detection information acquired by the sensor.

Patent document 2 also discloses a method for evaluating the risk of agitation based on biometric information obtained from a sensor worn by the user and notifying the user of how to deal with the situation.

JP 2021-18760 A International Publication No. 2019/073927

To provide an information processing device and information processing method that appropriately supports a caregiver in providing care to a person being assisted.

One aspect of the present disclosure relates to an information processing device including an acquisition unit that acquires input data including sensing data acquired using a sensor, and a factor estimation unit that estimates, based on the input data, which of a plurality of factors including an environmental factor caused by the surrounding environment of the person being assisted, a first core factor caused by an executive dysfunction of the person being assisted, a second core factor caused by a disorientation of the person being assisted, and a psychological factor caused by the psychology of the person being assisted, is the main cause of the peripheral symptom state in which the person being assisted is in a state in which peripheral symptoms of dementia are observed,

Another aspect of the present disclosure relates to an information processing method that acquires input data including sensing data acquired using a sensor, and estimates, based on the input data, which of a number of factors, including an environmental factor caused by the surrounding environment of the person being assisted, a first core factor caused by an executive dysfunction of the person being assisted, a second core factor caused by a disorientation of the person being assisted, and a psychological factor caused by the psychology of the person being assisted, has caused a peripheral symptom state in which peripheral symptoms of dementia are observed in the person being assisted.

1 is a configuration example of an information processing device. FIG. 1 is a schematic diagram explaining symptoms and causes of dementia. 1 is a configuration example of an information processing system. 1 is a configuration example of a server system. 2 is a configuration example of a terminal device. This is an example of a sensing device that detects the start of movement. This is an example of a sensing device that detects the start of movement. 1 is an example of a food-related sensing device. 1 is an example of a sensing device for detecting incontinence. 1 is an example of a sensing device related to maintaining a sitting position. This is an example of a neural network. This is an example of input and output of a neural network. 13 is a flowchart illustrating a learning process. 13 is a flowchart illustrating an inference process including estimation of a main cause. This is an example of an automatic feeding device that supports meals for those requiring care. FIG. 13 is a diagram showing an example of connection between the choking detection device and the automatic feeding device. FIG. 13 is a diagram showing an example of a tab structure. FIG. 13 is a diagram showing an example of a tab structure. FIG. 13 is a diagram showing an example of a tab structure. FIG. 13 is a diagram showing an example of a tab structure. FIG. 13 is a diagram showing an example of spoon movement for adjusting the amount of food eaten. FIG. 13 is a diagram showing an example of spoon movement for adjusting the amount of food eaten. FIG. 13 is a diagram showing an example of spoon movement for adjusting the amount of food eaten. 13 is an example of a screen displayed on the choking detection device. FIG. 2 is a diagram illustrating an example of the configuration of a MAC frame. FIG. 2 is a diagram illustrating an example of a configuration of a frame body. FIG. 13 is a diagram illustrating an example of the relationship between a data type ID and contents.

The present embodiment will be described below with reference to the drawings. In the drawings, identical or equivalent elements are given the same reference numerals, and duplicated explanations will be omitted. Note that the present embodiment described below does not unduly limit the contents described in the claims. Furthermore, not all of the configurations described in the present embodiment are necessarily essential components of the present disclosure.

1. System Configuration Example Fig. 1 shows a configuration example of an information processing device 20 according to this embodiment. The information processing device 20 includes an acquisition unit 21 and a factor estimation unit 22. Details of each unit of the information processing device 20 will be described later. The information processing device 20 in this embodiment outputs information to support assistance to a caregiver who assists a person being assisted in, for example, a medical facility or a care facility.

The caregiver here may be a care worker at a care facility, or a nurse or assistant nurse at a medical facility such as a hospital. That is, in this embodiment, assistance includes various actions to support the person being assisted, and may include caregiving or medical actions such as injections. The person being assisted here is someone who receives assistance from the caregiver, and may be a resident of a care facility, or a patient who is hospitalized or visits a hospital. The person being assisted may also be, for example, a person who may be suffering from dementia.

The assistance in this embodiment may be provided at home. For example, the person being assisted in this embodiment may be a person requiring care who receives home care, or a patient receiving home medical care. The caregiver may be a family member of the person requiring care or the patient, or a visiting helper, etc.

Figure 2 is a schematic diagram explaining symptoms and factors in dementia. Various symptoms called peripheral symptoms are seen in dementia patients. The peripheral symptoms here include agitated behavior and delirium. Agitated behavior refers to a state in which the behavior is excessive and restless. Delirium is a mental function disorder accompanied by a decrease in attention and thinking ability. More specifically, the peripheral symptoms may include various symptoms such as delusion, depression, insomnia, excitement, wandering, hallucinations, etc. as shown in Figure 2. The peripheral symptoms are not limited to the example shown in Figure 2, and may include various symptoms not seen in a normal state such as anxiety, misidentification, hyperactivity, unclean behavior, verbal abuse, violence, etc. The peripheral symptoms include mental symptoms and behavioral symptoms, and are also called BPSD (Behavioral and Psychological Symptom of Dementia). In the above example, depression, delusion, hallucinations, etc. correspond to mental symptoms, and wandering, violence, etc. correspond to behavioral symptoms. Peripheral symptoms not only reduce the quality of life of the person receiving care, but also increase the burden of care on the caregiver.

In dementia, symptoms that are directly caused by a decline in brain function are called core symptoms. As shown in Figure 2, core symptoms include executive dysfunction, disorientation, memory impairment, aphasia, and apraxia.

Executive dysfunction refers to a disorder that makes it difficult to carry out things in an orderly manner. When someone has an executive dysfunction, for example, when performing a series of actions that combine multiple actions, it becomes difficult to carry out multiple actions efficiently, even if each action is possible.

Disorientation refers to a disorder that makes it difficult for a person to understand the situation they are in. When disorientated, for example, the person being assisted will not be able to understand what time it is, where they are, or what they are doing.

Memory impairment refers to a disorder in which it becomes difficult to remember new things or to forget things that have been remembered. Aphasia refers to a state in which a person is unable to understand words or express their thoughts in words. Apraxia refers to a state in which a person is unable to perform everyday actions.

When peripheral symptoms are observed in patients with dementia, these core symptoms may be the cause. However, as shown in Figure 2, peripheral symptoms are known to occur as a result of the interaction between the core symptoms and factors such as the environment and psychology of the person being cared for, making it difficult to identify the main cause.

Therefore, in the method of this embodiment, information related to at least one of the person being assisted who may have dementia and the caregiver who is assisting the person being assisted is obtained as input data, and a process is performed to estimate the main cause of the peripheral symptoms based on the input data.

As shown in FIG. 1, the information processing device 20 according to this embodiment includes an acquisition unit 21 and a factor estimation unit 22. The acquisition unit 21 acquires input data including sensing data, which is data acquired using a sensor. The factor estimation unit 22 estimates which of a plurality of factors is the main cause of the peripheral symptom state in which the assisted person, who is being assisted by the caregiver, is in a state in which peripheral symptoms of dementia are observed. Here, the multiple factors include environmental factors caused by the surrounding environment of the assisted person, a first core factor caused by the executive function disorder of the assisted person, a second core factor caused by the disorientation of the assisted person, and a psychological factor caused by the psychology of the assisted person. Note that the configuration of the information processing device 20 is not limited to FIG. 1, and various modifications such as adding other components or omitting some components are possible. The fact that modifications such as omission or addition of components are possible is also true for FIGS. 3 to 5, etc., which will be described later.

According to the method of this embodiment, when a peripheral symptom is observed in an assisted person, the main cause can be appropriately estimated. As described above, peripheral symptoms occur due to a combination of multiple interrelated factors, so it has not been easy for a caregiver to determine the main cause. However, the method of this embodiment makes it possible to appropriately identify the main cause. For example, it is possible to encourage the caregiver to take action according to the main cause, which makes it possible to improve the quality of life of an assisted person with dementia and reduce the burden on the caregiver. Note that in the method of this embodiment, the factor estimation unit 22 may estimate whether the assisted person is in a peripheral symptom state based on the input data. In this way, it becomes possible to determine whether or not peripheral symptoms are observed in the information processing device 20.

Below, an example of an information processing system 10 including an information processing device 20 will be described with reference to Figures 3 to 5.

Figure 3 is a diagram showing an example of the configuration of the information processing system 10. As shown in Figure 3, the information processing system 10 includes a server system 100, a terminal device 200, a management terminal device 300, and a sensing device 400. However, the configuration of the information processing system 10 is not limited to that shown in Figure 3. For example, in Figure 3, a bedside sensor 420, a detection device 430, and a choking detection device 460 are illustrated as examples of the sensing device 400, but as described below, other devices may be used as the sensing device 400. For example, the devices described below with reference to Figures 6 to 10 may be used as the sensing device 400. Note that in the following, when there is no need to distinguish between multiple sensing devices 400, they will simply be referred to as sensing device 400.

The information processing device 20 of this embodiment corresponds to, for example, the server system 100. However, the method of this embodiment is not limited to this, and the processing of the information processing device 20 may be executed by distributed processing using the server system 100 and other devices. For example, the information processing device 20 of this embodiment may include the server system 100 and a terminal device 200. Below, an example in which the information processing device 20 is the server system 100 will be described.

The server system 100 is connected to the terminal device 200, the management terminal device 300, and the sensing device 400, for example, via a network. The network here is, for example, a public communication network such as the Internet. However, the network is not limited to a public communication network and may be a LAN (Local Area Network) or the like. For example, the server system 100 may communicate in accordance with the IEEE 802.11 standard. However, various modifications are possible regarding the communication method between the devices.

The server system 100 may be one server, or may include multiple servers. For example, the server system 100 may include a database server and an application server. The database server stores various data, which will be described later with reference to FIG. 4. The application server performs the processes, which will be described later with reference to FIGS. 13-14, etc. Note that the multiple servers here may be physical servers or virtual servers. Furthermore, if a virtual server is used, the virtual server may be provided in one physical server, or may be distributed across multiple physical servers. As described above, the specific configuration of the server system 100 in this embodiment can be modified in various ways.

The terminal device 200 is, for example, a device used by a caregiver who provides care to a person being assisted. Here, the terminal device 200 is, for example, a mobile terminal device such as a smartphone or a tablet terminal. However, the terminal device 200 may be other devices such as a PC (Personal Computer), a headset, or a wearable device such as AR (Augmented Reality) glasses or MR (Mixed Reality) glasses. Furthermore, one caregiver may use multiple terminal devices 200. For example, the caregiver may use both a smartphone and a headset.

The management terminal device 300 is a device used to manage information about residents of care recipients in, for example, a care facility. The management terminal device 300 is, for example, a PC, but other devices may be used. The management terminal device 300 has, for example, care software installed, and manages the residents of care recipients and the schedules of caregivers (staff at the care facility). For example, the management terminal device 300 stores information about the attributes of the residents of care. The attributes here include age, sex, height, weight, medical history, medication history, etc.

The sensing device 400 has various sensors and acquires sensing data based on the sensors. The sensing data described below may be the sensor output itself, or may be information obtained by calculation based on the sensor output.

In the information processing system 10 shown in FIG. 3, for example, the management terminal device 300 transmits information about the person being assisted to the server system 100. The sensing device 400 transmits sensing data to the server system 100. The server system 100 performs processing to estimate the presence or absence of peripheral symptoms and the main cause when peripheral symptoms are observed, using data transmitted from the management terminal device 300 and the sensing device 400 as input data. The server system 100 transmits information to the terminal device 200 based on the estimated main cause. For example, the server system 100 may transmit information indicating the main cause, or may transmit information indicating a specific measure required based on the main cause. In addition, the destination of information transmitted by the server system 100 is not limited to the terminal device 200. For example, the server system 100 may transmit the estimation result of the main cause to the sensing device 400 or another controlled device (not shown in FIG. 3). The sensing device 400 or the controlled device executes processing to change, for example, the operation mode based on the transmitted main cause. Details of the processing will be described later.

Figure 4 is a block diagram showing a detailed configuration example of the server system 100. The server system 100 includes, for example, a processing unit 110, a storage unit 120, and a communication unit 130.

The processing unit 110 of this embodiment is configured by the following hardware. The hardware can include at least one of a circuit for processing digital signals and a circuit for processing analog signals. For example, the hardware can be configured by one or more circuit devices or one or more circuit elements mounted on a circuit board. The one or more circuit devices are, for example, an IC (Integrated Circuit), an FPGA (field-programmable gate array), etc. The one or more circuit elements are, for example, a resistor, a capacitor, etc.

The processing unit 110 may be realized by the following processor. The server system 100 of this embodiment includes a memory that stores information and a processor that operates based on the information stored in the memory. The information is, for example, a program and various data. The memory may be the storage unit 120 or another memory. The processor includes hardware. The processor can be various processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a DSP (Digital Signal Processor). The memory may be a semiconductor memory such as an SRAM (Static Random Access Memory), a DRAM (Dynamic Random Access Memory), or a flash memory, or may be a register, or may be a magnetic storage device such as a hard disk drive (HDD), or may be an optical storage device such as an optical disk device. For example, the memory stores instructions that can be read by a computer, and the function of the processing unit 110 is realized as a process by the processor executing the instructions. The instructions here may be instructions of an instruction set that constitutes a program, or may be instructions that instruct the hardware circuit of the processor to operate.

The processing unit 110 includes, for example, an acquisition unit 111, a cause estimation unit 112, a response decision unit 113, a communication processing unit 114, a learning unit 115, and a presentation processing unit 116.

The acquisition unit 111 acquires input data for identifying the main cause when a peripheral symptom is observed in an assisted person. For example, the acquisition unit 111 acquires input data from the management terminal device 300 or the sensing device 400 via the communication unit 130. Details of the input data will be described later.

The factor estimation unit 112 performs processing to estimate the main factor based on the input data. The server system 100 of this embodiment may store, for example, information that associates the input data with the main factor. The information that associates the input data with the main factor is, for example, the trained model 124. However, the information that associates the input data with the main factor may be table data, or may be an algorithm that determines the main factor based on the input data. The factor estimation unit 112 estimates the main factor based on the information that associates the input data with the main factor and the input data acquired by the acquisition unit 111.

The treatment decision unit 113 determines treatment information indicating a treatment recommended for the peripheral symptoms of the person being assisted based on the estimated main cause. The server system 100 of this embodiment may store, for example, a treatment table 125 that associates the main cause with the treatment information. The treatment decision unit 113 determines a treatment according to the main cause based on the estimated main cause and the treatment table 125.

The communication processing unit 114 controls communication using the communication unit 130. For example, the communication processing unit 114 executes processing to create data to be transmitted, such as a MAC frame in the data link layer. The communication processing unit 114 may also perform processing such as interpreting the frame structure of data received by the communication unit 130, extracting necessary data, and outputting it to a higher layer such as an application.

The learning unit 115 performs a process of obtaining the trained model 124 by performing machine learning based on training data. The machine learning here may use a neural network (hereinafter, abbreviated as NN), may use an SVM, or may be learning using other techniques. The learning unit 115 may also perform a process of updating the existing trained model 124 by performing machine learning based on training data for updating.

The presentation processing unit 116 performs processing to present the information requested in the server system 100 on another device such as the terminal device 200. For example, the presentation processing unit 116 may perform processing to transmit information indicating the main cause or information indicating a recommended action to the terminal device 200 via the communication unit 130. The presentation processing unit 116 may also transmit information specifying the presentation mode on the terminal device 200. The presentation here may be the display of an image or text, may be the output of a sound, or may be presentation using light emission, vibration, etc.

The memory unit 120 is a work area for the processing unit 110, and stores various information. The memory unit 120 can be realized by various types of memory, and the memory may be a semiconductor memory such as an SRAM, a DRAM, a ROM, or a flash memory, or may be a register, a magnetic storage device, or an optical storage device.

The memory unit 120 may store user information 121, device information 122, log data 123, a learned model 124, and a response table 125.

User information 121 is information that manages users of the information processing system 10, and includes information such as a user ID and a user name that uniquely identifies a user. The user here includes both the person being assisted and the person providing the assistance. The user here may also include related parties of the person being assisted. Related parties are people who have daily contact with the person being assisted, and may include family members, close friends, facility personnel, etc.

The device information 122 is information for managing various devices included in the information processing system 10, and includes a device ID that uniquely identifies the device, a device name, a device type ID that indicates the type of device, a vendor, etc. The device here may be the terminal device 200, the management terminal device 300, or the sensing device 400. The device information 122 may also include a user ID that identifies the user who uses the target device. The user of the terminal device 200 is, for example, a caregiver. The user of the sensing device 400 may be a person being assisted, a caregiver in charge of the person being assisted, or both. The device information 122 may also include information on the facility in which the sensing device 400 is used, etc.

The log data 123 is a log of the input data acquired by the acquisition unit 111. For example, the log data 123 is information in which time-series input data is associated with the acquisition timing of the input data. The log data 123 may also include information identifying the device from which the input data was acquired, the target person being assisted, etc.

The trained model 124 is information acquired by the learning process of the learning unit 115, and is a model that identifies the main cause of peripheral symptoms based on the input data. Details of the trained model 124 will be described later.

The action table 125 is information that associates the main cause with a recommended action. The specific associations will be described later.

The communication unit 130 is an interface for communicating via a network, and includes, for example, an antenna, an RF (radio frequency) circuit, and a baseband circuit when the server system 100 performs wireless communication. However, the server system 100 may also perform wired communication, in which case the communication unit 130 may include a communication interface such as an Ethernet connector and a control circuit for the communication interface. The communication unit 130 operates under the control of the communication processing unit 114. However, the communication unit 130 may also include a processor for communication control different from the communication processing unit 114. The communication unit 130 may perform communication according to a method defined in, for example, IEEE802.11 or IEEE802.3. However, the specific communication method can be modified in various ways.

Figure 5 is a block diagram showing a detailed configuration example of the terminal device 200. The terminal device 200 includes, for example, a processing unit 210, a storage unit 220, a communication unit 230, a display unit 240, and an operation unit 250. The configuration of the terminal device 200 is not limited to that shown in Figure 5, and modifications such as omitting some components and adding other components are possible. For example, the terminal device 200 may have various sensors according to the terminal device 200, such as a motion sensor such as an acceleration sensor or a gyro sensor, an image sensor, a pressure sensor, a GPS (Global Positioning System) sensor, etc. As described above, the terminal device 200 can use various types of devices and may have a device-specific configuration not shown in Figure 5.

The processing unit 210 is configured with hardware including at least one of a circuit for processing digital signals and a circuit for processing analog signals. The processing unit 210 may also be realized by a processor. The processor may be of various types, such as a CPU, a GPU, or a DSP. The processor executes instructions stored in the memory of the terminal device 200, whereby the functions of the processing unit 210 are realized as processing.

The memory unit 220 is the work area of the processing unit 210 and is realized by various types of memory such as SRAM, DRAM, and ROM.

The communication unit 230 is an interface for communicating over a network, and includes, for example, an antenna, an RF circuit, and a baseband circuit. The communication unit 230 communicates with the server system 100, for example, over a network. The communication unit 230 may perform wireless communication with the server system 100, for example, in compliance with the IEEE 802.11 standard.

The display unit 240 is an interface that displays various information and may be a liquid crystal display, an organic EL display, or another type of display. The operation unit 250 is an interface that accepts user operations. The operation unit 250 may be a button or the like provided on the terminal device 200. The display unit 240 and the operation unit 250 may also be a touch panel that is integrally configured.

The terminal device 200 may also include components not shown in FIG. 5, such as a light-emitting unit, a vibration unit, a sound input unit, and a sound output unit. The light-emitting unit is, for example, an LED (light emitting diode) and issues a notification by emitting light. The vibration unit is, for example, a motor and issues a notification by vibration. The sound input unit is, for example, a microphone. The sound output unit is, for example, a speaker and issues a notification by sound.

In addition, some or all of the processing performed by the information processing device 20 of this embodiment may be realized by a program. The processing performed by the information processing device 20 is, for example, processing performed by the processing unit 110 of the server system 100.

The program according to this embodiment can be stored in a non-transitory information storage device (information storage medium), which is, for example, a medium readable by a computer. The information storage device can be realized, for example, by an optical disk, a memory card, a HDD, or a semiconductor memory. The semiconductor memory is, for example, a ROM. The processing unit 110 and the like perform various processes of this embodiment based on the programs stored in the information storage device. That is, the information storage device stores programs for causing a computer to function as the processing unit 110 and the like. A computer is a device that includes an input device, a processing unit, a storage unit, and an output unit. Specifically, the program according to this embodiment is a program for causing a computer to execute each step described later using Figures 13 to 14 and the like.

The technique of this embodiment can also be applied to an information processing method including the following steps. The information processing method includes the steps of acquiring input data including sensing data acquired using a sensor, and estimating which of a plurality of factors has caused the person being assisted by the caregiver to enter a peripheral symptom state in which peripheral symptoms of dementia are observed. As described above, the plurality of factors include environmental factors caused by the surrounding environment of the person being assisted, a first core factor caused by the executive dysfunction of the person being assisted, a second core factor caused by disorientation of the person being assisted, and psychological factors caused by the psychology of the person being assisted.

2. Details of the Processing Next, the processing according to this embodiment will be described in detail. First, for each of the multiple factors, an example of data related to the factor will be described. The data related to the factor is used, for example, as part of the input data. Then, the processing in the server system 100 will be described in detail. Furthermore, taking meal assistance as an example, a specific example of a measure to add a device to be used will be described. Also, an example of a data structure when transmitting the estimated main factor to the terminal device 200 or another device will be described.

2.1 Examples of Data Related to Factors 2.1.1 Environmental Factors As described above with reference to FIG. 2, the environment surrounding the person receiving care is known as one of the factors of peripheral symptoms. Data related to environmental factors includes data that specifies the environment surrounding the person receiving care. Hereinafter, data related to environmental factors will be referred to as environmental information. Environmental information includes information on brightness, temperature, humidity, scent, sound, video, and season.

For example, the information representing brightness may be information about lighting equipment placed in the living environment of the person being assisted. The information about the lighting equipment may be information about brightness expressed in units of lumens or the like, or may be information about color temperature. It may also be information specifying the times during the day when the lighting is turned on. This information may be obtained automatically, for example, from the operation log of the lighting equipment, or may be input by a caregiver or the like.

The information representing the temperature and humidity may also be obtained using sensors such as a thermometer or a hygrometer. The information representing the temperature and humidity may also be information relating to an air conditioning device placed in the living environment of the person being assisted. If the air conditioning device includes a temperature sensor or a humidity sensor, the information representing the temperature and humidity may be obtained based on the output of the sensor. The information representing the temperature and humidity may also be information such as the set temperature of the air conditioning device, the operating mode (cooling, heating, dehumidification, etc.), and the operating time. This information may be obtained automatically from, for example, the operation log of the air conditioning device, or may be input by a caregiver, etc.

The information representing the scent is, for example, information about the scent present around the person being assisted, and includes information such as the type of scent and the duration for which the scent is felt. For example, if flowers or air fresheners are placed in the room of the person being assisted, the type of scent may be identified based on the variety of flowers, the manufacturer or product model number of the air freshener, etc. Information about detergent or fabric softener may be used as information representing the scent of the clothes of the person being assisted. The information representing the scent may also include information representing the body odor of the person being assisted or the caregiver. The information representing the scent may be obtained automatically based on the output of an odor sensor, or may be input by a caregiver, etc.

The information representing the sound includes information representing the type of music output from a music player or the like, and the time period during which the music is output. The type of music may be information representing a genre such as pop or classical, for example. The information representing the music may be automatically acquired based on the operation log of the music player, or may be input by a caregiver or the like. The information representing the sound may also be information acquired using a sound collection device such as a microphone. For example, information such as the frequency, volume, and duration of the sound may be used as information representing the sound.

The information representing the video includes, for example, information representing the type of video displayed using a display device such as a television, the time period during which the video is output, etc. The type of video may be information representing a genre such as movie, drama, news, etc., or may include more specific genres such as action movies or comedy movies. The information representing the video may be automatically obtained based on the operation log of the television, or may be input by a caregiver, etc.

The information representing the season is, for example, information that specifies whether the current season is spring, summer, autumn, or winter. For example, the information representing the season may be two-bit data that identifies the four states of spring, summer, autumn, and winter. The information representing the season may also include information on whether or not it is a time of seasonal change, or may include information on a specific date. The information representing the season may be automatically obtained using calendar software or the like that runs on the management terminal device 300, or may be input by a caregiver, etc.

2.1.2 First core factor (executive dysfunction)
As described above, executive dysfunction, apraxia, etc. are known as core symptoms, and executive dysfunction and apraxia represent a state in which it becomes difficult to perform daily activities. Therefore, the data related to the first core factor may be data representing the ability to perform daily activities. The data representing the ability here may be an index value of ADL (Activities of Daily Living). Furthermore, ADL here may be iADL (Instrumental Activities of Daily Living).

For example, a person receiving care who has executive dysfunction or apraxia will be unable to perform daily activities such as initiating movement, eating, and excretion. Initiating movement here includes, for example, standing up and getting up. In this embodiment, sensing data from a sensing device 400 that detects initiating movement, a sensing device 400 that detects eating, a sensing device 400 that detects incontinence, and the like may be output to the server system 100 as data related to the first core factor. Specific examples of each sensing device 400 will be described below.

<Getting started>
FIG. 6 is a diagram showing an example of an imaging device 410 that is a sensing device 400 that detects the start of movement, and an example of an output image IM1 of the imaging device 410. The imaging device 410 has an image sensor that outputs an imaged image as sensing data. The imaging device 410 may be placed in a place where many people gather and are active, such as a living room or hall of a nursing home. In the example of FIG. 6, the imaging device 410 is placed on top of a television device. The location where the imaging device 410 is placed is not limited to this, and the imaging device 410 may be placed in other places such as a living room. For example, a bedside sensor 420 and a detection device 430, which will be described later with reference to FIG. 7, detect the start of movement in bed, but the imaging device 410 placed in the living room may be used instead of these sensing devices 400.

The imaging device 410 may perform a process of detecting the start of movement of a person based on the captured image. The imaging device 410, for example, operates according to an application installed in the imaging device 410 to obtain the captured image as input data, and executes a process of detecting a person from the captured image and a process of determining whether the detected person has started to move. For example, the imaging device 410 outputs the captured images in time series to the server system 100 as sensing data. Alternatively, the imaging device 410 may determine the presence or absence of movement by executing the process described below based on the captured images in time series, and output the determination result as sensing data to the server system 100. The imaging device 410 may also determine the presence or absence of movement, and when the start of movement is detected, output the captured images for a predetermined period before and after the start of movement to the server system 100 as sensing data. In this case, information that is particularly useful for determining the ability to stand up can be output as sensing data. In addition, various modifications of the specific examples of sensing data are possible.

For example, the imaging device 410 performs face recognition processing to recognize the face of a person based on a captured image. For example, the imaging device 410 may store a face image of a person to be detected, and perform face recognition processing based on a matching process using the face image as a template. Various methods of face recognition processing are known, and these can be widely applied in this embodiment. For example, when the movement of the detected face area continues to be below a given threshold for a certain period of time, the imaging device 410 sets the position of the face area in that state as a reference position. Then, the imaging device 410 may set a detection area at a position a predetermined distance away from the reference position, and determine that a movement has occurred when the face area reaches the detection area. For example, when a standing-up action is performed, it is expected that the position of the face will move relatively upward, so the detection area may be set at a position a predetermined distance above the reference position. In this case, when the position of the face area on the image moves upward by more than a predetermined distance from the reference position, a movement is detected. Note that the detection area here is, for example, a line-shaped area, but an area of another shape may be set.

The imaging device 410 can also identify the target recipient by face recognition processing. Therefore, the imaging device 410 may perform movement detection for a specific recipient and omit movement detection for other recipients. The specific recipient here may be, for example, a recipient suspected of having dementia, or a recipient whose ADL index value is below a predetermined level. For example, the imaging device 410 may perform movement detection for a recipient who is at high risk of falling when standing up.

Movement detection processing is not limited to the above method. For example, the image capture device 410 may perform skeletal tracking processing based on the captured image. Note that various image-based skeletal tracking methods are known, such as "Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields" (https://arxiv.org/pdf/1611.08050.pdf) and OpenPose disclosed by Zhe Cao et al., and these methods can be widely applied in this embodiment.

OpenPose also discloses a method for performing skeletal tracking on each of multiple people captured in an image and displaying the results. In the example of Figure 6, the image sensor outputs a captured image including three people receiving care, and the imaging device 410 determines whether each of the three people receiving care has started to move.

For example, a person receiving care whose ability has declined and who finds it difficult to get up and down may fall even if they simply try to stand up. Therefore, the imaging device 410 may use skeletal tracking to determine whether the person is in a standing up position. For example, if the imaging device 410 determines that the person has leaned forward from a sitting position with their hands on their knees or the seat of a chair, it may determine that the person is in a standing up position and notify the caregiver of the risk of falling. For example, if the imaging device 410 detects from the skeletal tracking results that the distance between the positions of the hands and knees is less than a predetermined value, or that the shoulders have moved downward by more than a predetermined value, it may determine that the person receiving care is in a standing up position.

Alternatively, the imaging device 410 may divide the data to be processed into windows of several seconds each, and determine that a posture change such as standing up has occurred when a specific position such as the head or neck has moved within each window by more than a predetermined threshold. The parts targeted for movement detection may be other than the head or neck. The direction of movement may be vertical, horizontal, or diagonal. The threshold used for detection may be changed depending on the part to be detected. Alternatively, the imaging device 410 may obtain an area that includes feature points detected by skeletal tracking of a stationary person receiving care, and determine that a movement such as standing up has occurred when a predetermined number of feature points or more fall outside the area. In addition, various modifications are possible for the method of movement detection processing using the imaging device 410.

IM1 in FIG. 6 is an example of an output image output using the imaging device 410. Below, an example in which the imaging device 410 generates the output image will be described, but the server system 100 may also generate the output image IM1 based on sensing data.

For example, the imaging device 410 may superimpose some display object on the captured image. In the example of FIG. 6, an object including an "!" mark is displayed in association with the assisted person whose movement has been detected. In this way, it is possible to clearly notify the caregiver of the assisted person whose movement has been detected. For example, the imaging device 410 may transmit the output image IM1 to the server system 100 as part of the sensing data. The server system 100 outputs the output image IM1 to the terminal device 200 used by the caregiver. However, the output of the imaging device 410 may be information identifying the assisted person whose movement has been detected (for example, the ID of the assisted person), and the specific form may be variously modified. For example, although an example of notifying the assisted person whose movement has been detected is shown here, information for stopping the movement of the assisted person may be output. For example, the imaging device 410 may identify the assisted person whose movement has been detected and output audio data, video data, etc. of the assisted person's family, etc. Dementia patients, in particular, tend to be slow to respond when called out to, but they often remember the faces and voices of family members, making stopping their movements an effective method. By stopping the movements of the person being assisted in this way, it is possible to buy time before the caregiver can intervene.

Figure 7 is a diagram illustrating an example of a bedside sensor 420 and a detection device 430 placed at the bottom of a bed 610. The bedside sensor 420 and the detection device 430 are, for example, sheet-like or plate-like devices provided between the bottom of the bed 610 and the mattress 620, as shown in Figure 7.

The bedside sensor 420 includes a pressure sensor that outputs a pressure value as sensing data, and is disposed on the side of the bottom that the caregiver uses to get on and off the bed. In the example of FIG. 7, the caregiver gets on and off the bed using the front side of the bed 610. In this case, as shown in FIG. 7, a fence for preventing falls may be disposed on the front side of the bed 610, and the bedside sensor 420 may be disposed in a position where the fence is not provided. In this way, the user who gets on and off the bed 610 first sits on the bedside sensor 420. The bedside sensor 420 may output time-series pressure data as sensing data to the server system 100. Alternatively, the bedside sensor 420 may execute the process described below to determine whether or not the user has started to move, and output the determination result to the server system 100 as sensing data.

The bedside sensor 420, for example, operates according to an application installed on the bedside sensor 420 to obtain pressure values as input data and execute a process to determine the movement of the person being assisted on the bed 610 from the pressure values.

For example, when the person being assisted stands up from the bed 610, it is assumed that the person being assisted moves from lying on the bed to sitting on the side of the bed (hereinafter referred to as "sitting on the edge of the bed"), and then places hands on the knees or the bed bottom to apply force to perform the standing up motion. The pressure values detected by the bedside sensor 420 increase in the order of lying, sitting on the edge of the bed, and standing up motion. For example, the bedside sensor 420 may determine that the start of movement has been detected when it detects a change from sitting on the edge of the bed to standing up motion based on a comparison process between the pressure value and a given threshold value. Alternatively, from the viewpoint of detecting the standing up motion at an earlier stage, the bedside sensor 420 may determine that the start of movement has been detected when it detects a change from lying on the bed to sitting on the edge of the bed based on a comparison process between the pressure value and a given threshold value.

Alternatively, if the standing up motion continues, the buttocks of the person being assisted rise from the bottom surface, causing a large decrease in the pressure value output from the pressure sensor. Therefore, the bedside sensor 420 may determine that the standing up motion has been performed when the pressure value increases to or exceeds a first threshold value and then decreases to or below a second threshold value that is smaller than the first threshold value, based on the time-series change in the pressure value. In addition, various modifications can be made to the specific processing content for determining whether or not the person has started to move.

The detection device 430 shown in FIG. 7 is a sensing device 400 that senses information related to the sleep of the person being assisted. The detection device 430 includes a pressure sensor that outputs a pressure value.

When the user gets in bed, the detection device 430 detects the user's body vibration (body movement, vibration) through the mattress 620. Based on the body vibration detected by the detection device 430, information on the respiration rate, heart rate, activity level, posture, awake/asleep, and getting out of bed/being in bed can be obtained. The detection device 430 may also determine whether the sleep is non-REM or REM sleep, and the depth of sleep. For example, the periodicity of the body movement may be analyzed, and the respiration rate and heart rate may be calculated from the peak frequency. The periodicity may be analyzed by, for example, Fourier transform. The respiration rate is the number of breaths per unit time. The heart rate is the number of heartbeats per unit time. The unit time is, for example, one minute. Body vibration may also be detected per sampling unit time, and the number of detected body vibrations may be calculated as the amount of activity. When the user gets out of bed, the detected pressure value decreases compared to when the user is in bed, so it is possible to determine whether the user is out of bed or in bed based on the pressure value and changes over time.

For example, the detection device 430 may output the output of a pressure sensor as sensing data to the server system 100. Alternatively, the detection device 430 may output the above-mentioned information regarding the respiration rate, heart rate, activity level, posture, awake/asleep, and getting out/being in bed as sensing data to the server system 100.

For example, the detection device 430 determines that a movement has occurred when the person being assisted transitions from a state in bed to a state out of bed. Furthermore, from the viewpoint of detecting signs of movement at an earlier stage, the detection device 430 may determine that a movement has occurred when the person being assisted transitions from a sleeping state to an awake state. The detection device 430 may output information regarding awake/asleep and out/in bed as sensing data to the server system 100, or may output the detection result of the start of movement to the server system 100 as sensing data.

<Meals>
FIG. 8 is a diagram illustrating a choking detection device 460 that is a sensing device 400 used in a meal situation. As shown in FIG. 8, the choking detection device 460 includes a throat microphone 461 that is attached around the neck of the person being assisted, and a terminal device 462 having a camera. Note that other devices having a camera may be used instead of the terminal device 462. The throat microphone 461 outputs audio data generated by the person being assisted swallowing or coughing. The camera of the terminal device 462 outputs an image capturing an image of the state of the person being assisted eating. The terminal device 462 is, for example, a smartphone or a tablet PC that is placed on a table where the person being assisted eats. The throat microphone 461 is connected to the terminal device 462 using Bluetooth (registered trademark) or the like, and the terminal device 462 is connected to the server system 100 via a network. However, both the throat microphone 461 and the terminal device 462 may be directly connected to the server system 100, and the specific connection mode can be variously modified.

The choking detection device 460 outputs the time-series audio data detected by the throat microphone 461 and the time-series captured images taken by the camera of the terminal device 462 to the server system 100 as sensing data. Alternatively, the choking detection device 460 may obtain various information related to the meal by performing the processing described below based on the audio data and captured images, and output the information to the server system 100 as sensing data.

The swallowing choking detection device 460 determines whether the person being assisted has choked or swallowed based on the audio data from the throat microphone 461. A device that detects swallowing using a microphone worn around the neck is described, for example, in U.S. patent application Ser. No. 16/276,768, filed Feb. 15, 2019, entitled "Swallowing action measurement device and swallowing action support system." This patent application is incorporated by reference in its entirety into this specification. The processor can detect the number of choking incidents, the duration of the choking (time of occurrence, duration, etc.), and whether or not the person swallowed based on the audio data.

The camera of the terminal device 462 can detect the mouth and eyes of the person being assisted, as well as chopsticks and spoons used by the person being assisted, by capturing an image of the person being assisted from the front, as shown in FIG. 8, for example. There are various known methods for detecting these facial parts and objects based on image processing, and in this embodiment, a wide range of known methods can be applied.

For example, the choking detection device 460 can determine, based on the image captured by the camera, whether the person being assisted has their mouth open, whether food is coming out of their mouth, and whether they are chewing their food. The choking detection device 460 can also determine, based on the image captured by the camera, whether the person being assisted has their eyes open. The choking detection device 460 can also determine, based on the image captured by the camera, whether chopsticks, a spoon, etc. are near tableware, whether the person being assisted is able to hold them, and whether food is being spilled.

In the method of this embodiment, the swallowing and choking status of the person being assisted is estimated based on this information. For example, the choking detection device 460 may obtain information about the meal based on the detection results of choking and swallowing and the determination results of whether the person being assisted is opening or closing their mouth.

For example, the swallowing choking detection device 460 may determine whether choking occurs frequently based on the number of times or duration of choking, and output the determination result. For example, the swallowing choking detection device 460 may determine that choking occurs frequently when the number of times choking per unit time exceeds a threshold value. In this way, the situation regarding choking can be automatically determined.

The choking detection device 460 may also determine the swallowing time from when the assisted person opens their mouth to when they swallow, based on the swallowing detection result and the result of determining whether the assisted person's mouth is open or closed. In this way, for example, if the number of swallows has decreased, it is possible to determine specific situations, such as whether the action of putting food in the mouth has not been performed, or whether food has been put in the mouth but has not been swallowed. For example, the choking detection device 460 may start counting up a timer when the mouth transitions from a closed state to an open state based on the captured image of the terminal device 462, and stop counting the timer when swallowing is detected by the throat microphone 461. The time when the timer stops represents the swallowing time. In this way, it is possible to accurately determine whether there is a high risk of aspiration during meals and whether the caregiver should take some kind of action.

The choking detection device 460 may also determine the pace of eating based on the swallowing time. The choking detection device 460 may also determine whether the swallowing time is long or not based on the change in the swallowing time during one meal (for example, the increase or ratio compared to the swallowing time at the beginning). Alternatively, the processor may obtain the average swallowing time for the same person receiving care for each of multiple meals, and determine whether the swallowing time has become longer or not based on the change in the average swallowing time.

In addition, by using the results of determining whether the mouth is open or closed based on the captured image from the terminal device 462, it can be determined whether the mouth will no longer open even when the caregiver approaches a spoon or the like. In this way, if the person being assisted is reluctant to open their mouth and the swallowing time becomes long, it can be inferred that food has remained in the mouth and become blocked. In addition, by using the captured image to determine whether food is coming out of the mouth and the mouth recognition results to determine whether the person being assisted is chewing food, it can be determined whether the person being assisted is no longer able to chew food. For example, if the number of chews is normal but the swallowing time is long, it is inferred that the person is no longer able to chew food. In addition, if the captured image is used to determine that the eyes are closed, it can be determined whether the person being assisted is becoming sleepy.

In addition, by performing recognition processing of chopsticks, spoons, etc. using the captured image, it may be determined whether the person is playing with food, unable to hold a bowl, doing nothing, etc. For example, if an object such as a spoon is overlapping the hand of the person being assisted, but the time it takes for the person to bring the object to their mouth is equal to or exceeds a predetermined threshold, it is determined that the person is unable to hold a bowl or is playing with food. In addition, if an object such as a spoon is not overlapping the hand of the person being assisted, and the time the person being assisted keeps their gaze directed toward the food is equal to or exceeds a predetermined threshold, it is determined that the person is doing nothing but looking at the food.

<Incontinence>
9 shows an example of an incontinence detection device 450, which is an example of the sensing device 400 used to detect incontinence. The incontinence detection device 450 is placed on a mattress 620 of a bed 610, for example.

The incontinence detection device 450 shown in FIG. 9 includes, for example, an odor sensor, and detects whether the person being assisted has incontinence. The odor sensor here may be of a semiconductor type, a quartz crystal oscillator type, or of another type. For example, the incontinence detection device 450 determines that incontinence has occurred when the numerical data representing odor output by the odor sensor is equal to or greater than a given threshold value.

The incontinence detection device 450 may output the output data of the odor sensor to the server system 100 as sensing data, or may determine the presence or absence of incontinence based on the output data and output the determination result to the server system 100 as sensing data. The incontinence detection device 450 may also output the rate of incontinence as sensing data. The rate of incontinence here may be the ratio of the number of days on which incontinence was detected to the total number of days in the target period, or the ratio of the number of excretions determined to be incontinence to the total number of excretions in a unit period such as one day, or it may be some other ratio.

Furthermore, the incontinence detection device 450 is not limited to one that includes an odor sensor. For example, the incontinence detection device 450 may include a capacitance sensor and detect the presence or absence of incontinence based on a change in capacitance due to the presence or absence of liquid (urine). The incontinence detection device 450 outputs at least one of the output data of the capacitance sensor and the incontinence detection result (which may be the presence or absence of incontinence or the incontinence rate) to the server system 100 as sensing data.

In the above example, as shown in FIG. 9, for example, incontinence is detected using an incontinence detection device 450 placed on a mattress 620. Therefore, when the person being assisted is not wearing a diaper, urine reaches the vicinity of the incontinence detection device 450, making it possible to properly detect odor and changes in capacitance due to incontinence. On the other hand, when the person being assisted is wearing a diaper, if the incontinence is severe enough to cause leakage from the diaper, there will be a large change in odor and capacitance, but if the amount of incontinence is an amount that can be absorbed in the diaper, the change in odor and capacitance will be small, and it may be difficult to detect incontinence in the diaper.

Therefore, in this embodiment, the incontinence detection device 450 may change the content of the incontinence judgment depending on the condition of the person being assisted, specifically, whether the person being assisted is wearing a diaper or not. For example, when the person being assisted is not wearing a diaper, the incontinence detection device 450 sets a first threshold value as a threshold value for judging the presence or absence of incontinence, and when the person being assisted is wearing a diaper, sets a second threshold value smaller than the first threshold value as a threshold value for judging the presence or absence of incontinence. In this way, the sensitivity is increased when a diaper is worn, making it possible to detect incontinence with high accuracy. In addition, the judgment regarding incontinence may be performed in the server system 100, and for example, the processing unit 110 may make a judgment using multiple threshold values that differ depending on whether a diaper is worn or not.

The incontinence detection device 450 shown in FIG. 9 is a device that detects incontinence in the bed 610, but the sensing device 400 used to detect incontinence is not limited to this. For example, the sensing device 400 may be a portable device that includes a capacitance sensor and is attached to the underwear or diaper of the person being assisted. In this way, it becomes possible to obtain sensing data related to incontinence even in places other than the bed 610.

In addition, in this embodiment, the sensing device 400 for incontinence may be both an incontinence detection device 450 placed on the bed and a portable device attached to the clothing of the person being assisted. The incontinence detection device 450 and the server system 100 may determine the presence or absence of incontinence and the incontinence rate by combining the sensing data from these sensing devices 400. Alternatively, the sensing device 400 to be used may be switched depending on the situation, such as when the person being assisted is not wearing a diaper, the incontinence detection device 450 placed on the bed is used, and when the person being assisted is wearing a diaper, the incontinence detection device 450 and the portable device are used together.

<Other Modifications>
Furthermore, the ADL of the user may be estimated based on the sitting ability and walking ability. The sitting ability refers to the ability to maintain a sitting state. The walking ability refers to the ability to walk without falling or the like.

10 is an example of a sensing device 400 that outputs sensing data for estimating the sitting ability or walking ability, and is a diagram showing a seat sensor 440 arranged on the seat of a wheelchair 630, for example. The seat sensor 440 includes a pressure sensor that outputs a pressure value, and determines whether the posture of the person being assisted when sitting on the wheelchair 630 (hereinafter also referred to as the sitting posture) is one of a plurality of postures including normal, forward shift, side shift, etc., based on the pressure value. Forward shift refers to a state in which the user's center of gravity is shifted forward more than usual, and side shift refers to a state in which the user's center of gravity is shifted to either the left or right more than usual. Both forward shift and side shift correspond to a state in which the risk of falling from the seat is relatively high. The seat sensor 440 may be a sensor device arranged on a normal chair, or may be a sensor device that determines the posture of a user sitting on a bed or the like. The seat sensor 440 may also perform a fall possibility determination to determine whether or not the person being assisted is likely to fall from the seat.

In the example of FIG. 10, four pressure sensors Se1 to Se4 are arranged on the underside of a cushion 441 that is placed on the seat of the wheelchair 630. Pressure sensor Se1 is a sensor that is arranged in the front, pressure sensor Se2 is a sensor that is arranged in the rear, pressure sensor Se3 is a sensor that is arranged on the right, and pressure sensor Se4 is a sensor that is arranged on the left. Note that front, back, left and right here refer to the directions as seen by the person being assisted when he or she is sitting in the wheelchair 630.

As shown in FIG. 10, the pressure sensors Se1 to Se4 are connected to a control box 442. The control box 442 includes a processor that controls the pressure sensors Se1 to Se4 and a memory that serves as the work area for the processor. The processor detects pressure values by operating the pressure sensors Se1 to Se4.

A person being assisted sitting in a wheelchair 630 may feel pain in the buttocks and may shift the position of the buttocks. For example, forward shift is when the buttocks shift further forward than usual, and lateral shift is when the buttocks shift to the left or right. Forward and lateral shifts may also occur simultaneously, causing the center of gravity to shift diagonally. By using a pressure sensor placed on the cushion 441 as shown in FIG. 10, the change in the position of the buttocks can be appropriately detected, making it possible to accurately detect forward and lateral shifts.

For example, the initial state is the timing when the person being assisted transfers to the wheelchair 630 and assumes a normal posture. In the initial state, the person being assisted sits deep on the seat of the wheelchair 630, so it is assumed that the value of the rear pressure sensor Se2 is relatively large. On the other hand, when forward slippage occurs, the position of the buttocks moves forward, so the value of the front pressure sensor Se1 becomes large. For example, the processor of the control box 442 may determine that forward slippage has occurred when the value of the pressure sensor Se1 has increased by a predetermined amount compared to the initial state. Also, instead of using the value of the pressure sensor Se1 alone, processing may be performed using the relationship between the values of the pressure sensors Se2 and Se1. For example, the difference in voltage values that are the outputs of the pressure sensors Se2 and Se1 may be used, or the ratio of the voltage values may be used, or the rate of change of the difference or ratio relative to the initial state may be used. Also, when the value of the pressure sensor Se1 exceeds a certain threshold, it is determined that the person being assisted is in the wheelchair 630, and it may be determined that forward slippage has occurred based only on the change in the value of the pressure sensor Se2 without comparing it with the pressure sensor Se1.

Similarly, when a lateral shift occurs, the position of the buttocks moves to the left or right, so if there is a left shift, the value of pressure sensor Se4 increases, and if there is a right shift, the value of pressure sensor Se3 increases. Thus, the processor may determine that a left shift has occurred when the value of pressure sensor Se4 has increased by a predetermined amount or more compared to the initial state, and may determine that a right shift has occurred when the value of pressure sensor Se3 has increased by a predetermined amount or more compared to the initial state. Alternatively, the processor may determine right and left shifts using the relationship between the values of pressure sensors Se4 and Se3. As with the example of forward shift, the difference in voltage values that are the outputs of pressure sensors Se4 and Se3 may be used, or the ratio of the voltage values may be used, or the rate of change of the difference or ratio from the initial state may be used.

The seat sensor 440 may output the pressure values output from the pressure sensors Se1 to Se4 to the server system 100 as sensing data, or may output the results of the determination of forward or lateral slippage, the determination of the possibility of falling, etc. to the server system 100 as sensing data. The control box 442 may also include a light-emitting unit or the like, and the light-emitting unit may be used to notify the caregiver. In this way, the caregiver can be notified of changes in the seating position in the wheelchair 630 or the like in an easily understandable manner, making it possible to prevent the person being assisted from falling.

The sensing device 400 of this embodiment may also include a device that outputs sensing data for estimating walking ability. The sensing device 400 here is, for example, a foot pressure sensor or an acceleration sensor. The foot pressure sensor may include, for example, pressure sensors provided at multiple locations on the insole. The sensing data of the foot pressure sensor may be a time series change in the pressure value itself, or a time series change in the center of gravity position. The acceleration sensor may be included in a smartphone or the like carried by the person being assisted, or may be included in a wearable device (for example, a wristwatch-type device, or a device attached to clothing or skin) worn by the person being assisted. The sensing data of the acceleration sensor may be a time series change in the acceleration value itself, or information regarding the period and amplitude of the acceleration value. For example, if the person being assisted has high walking ability, parameters such as the center of gravity position, the period and amplitude of the acceleration are stable, but as the walking ability decreases, the variation of these parameters increases due to foot entanglement, unsteadiness, falls, etc. Therefore, the sensing data of the foot pressure sensor and the acceleration sensor becomes information indicating the walking ability.

A device that determines the dementia level based on the ratio of the number of long presses to the number of presses when operating an electronic device is described, for example, in PCT/JP2018/038075, entitled "Dementia Determination System." This patent application is incorporated by reference in its entirety into this specification. The sensing device 400 of this embodiment may include a device that outputs the ratio of long presses of the device as sensing data, similar to this method.

2.1.3 Second core factor (disorientation)
As mentioned above, disorientation is known as a core symptom. Disorientation refers to a state in which it becomes difficult to recognize one's situation (time, place, etc.). For example, a person receiving care who is disoriented is more likely to wander around at night. Also, a person receiving care who is disoriented is more likely to become incontinent because they are unable to find the location of the toilet. Also, a person receiving care who is disoriented is likely to move to a place other than where they should be because they are unable to find the location. For example, they may enter a room or bed assigned to a person receiving care other than themselves, or defecate in a place other than the toilet.

Therefore, in this embodiment, information including at least one of wandering information regarding wandering of an assisted person and incontinence information regarding incontinence of an assisted person may be transmitted to the server system 100 as data related to the second core factor. Note that wandering here may be wandering at night. Wandering may also be moving to a place other than where you should be, regardless of whether it is day or night. Also, as described above, incontinence may occur by moving to a place other than where you should be, so it is not prevented that part of the wandering information overlaps with the incontinence information.

The detection device 430 shown in FIG. 7 can determine the state of being in bed/out of bed as described above. Therefore, in this embodiment, the detection device 430 may be used as a sensing device 400 for obtaining wandering information. For example, the detection device 430 may transmit at least one of the output of the pressure sensor and the result of being in bed/out of bed as sensing data to the server system 100. Alternatively, by associating with time information, the timing of nighttime bed leaving, the duration of bed leaving, the number of nighttime bed leavings per unit time, etc. may be obtained. These nighttime bed leavings can be used as wandering information representing nighttime wandering. The detection device 430 may transmit wandering information regarding nighttime bed leaving as sensing data to the server system 100. It is also possible to detect bed leaving using the bedside sensor 420 in FIG. 7. Therefore, the bedside sensor 420 can be used as a device for obtaining wandering information.

Also, an RFID (radio frequency identifier) reader and an IC tag may be used as a device for acquiring wandering information. For example, an RFID reader is installed in advance in a room, bed, dining room, toilet, etc. in the nursing facility, and an IC tag is installed on the clothing of the person being assisted or on a terminal device carried by the person being assisted. In this way, it is possible to obtain the history of where the person being assisted has moved to, such as a room, dining room, toilet, etc., as wandering information based on the reading result of the RFID reader. For example, the RFID reader may output the reading result to the server system 100 as wandering information. Alternatively, the RFID reader or another device connected to the RFID reader may determine whether the person being assisted has entered someone else's room or bed, and output the determination result as wandering information. For example, the frequency with which the person being assisted has entered someone else's room or bed (for example, the number of times per month) may be used as wandering information.

Furthermore, when a person defecates outside the toilet, the location is often fixed to a certain extent. Therefore, an RFID reader may be installed in advance in a location that the person being assisted may mistake for a toilet, and the results of reading the location and the determination of whether the person has moved to that location may be output as wandering information. For example, the frequency with which the IC tag of the person being assisted is read by the RFID reader (e.g., the number of times per month) may be used as wandering information or incontinence information.

The incontinence information may also be obtained using the incontinence detection device 450 described above with reference to FIG. 9 or a capacitance sensor attached to the clothing of the person being assisted. As with the example of the first core factor, the incontinence information here may be at least one of output data from a capacitance sensor or the like and the incontinence detection result (which may be the presence or absence of incontinence, or the incontinence rate). The incontinence information may also include the output of the detection device 430. For example, the incontinence information may be information including the number of times the person left the bed or the time spent leaving the bed before and after the incontinence was recorded, or a time series change in these. In this way, it becomes possible to distinguish whether the person lost control of the incontinence in bed without realizing that they needed to urinate, or whether the person left the bed but lost control because they did not know where the toilet was.

The incontinence information may also include information about the location where the incontinence occurred. The information about the location may be obtained, for example, using the RFID reader and IC tag described above, or may be obtained by other methods. By including the location information in the incontinence information, it becomes possible to distinguish whether the incontinence occurred on the way to the toilet or in bed. If the incontinence occurred on the way to the toilet, the person is likely to be aware of the need to urinate and the location where the person should defecate, and the severity is relatively low. On the other hand, if the incontinence occurs in bed, the person may not be aware of the need to urinate or may not be aware that the current location is not the toilet, and the severity is relatively high. By including the location of the incontinence in the incontinence information, it is possible to distinguish between these cases. For example, if a given person receiving care changes from a state where he or she is prone to incontinence on the way to the toilet to a state where he or she is prone to incontinence in bed, it can be determined that the condition of the person receiving care may be worsening.

As mentioned above, PCT/JP2018/038075 describes a device for determining the level of dementia. In this embodiment, the sensing device 400 that outputs sensing data related to the second core factor may include a device similar to this method.

2.1.4 Psychological factors When dementia patients show symptoms such as difficulty in daily activities, inability to understand what is said, or inability to remember, being scolded by family members or other related parties may worsen the symptoms. In addition, if a person is scolded by a related party, there are cases where the person tries to hide the fact that they were unable to do something well for fear of being scolded again. In other words, not only at the time of the scolding, but also for a period thereafter, the person being assisted may have psychological anxiety, which may worsen the symptoms.

In this way, the psychological state of the person being assisted is closely related to the state of those involved who are in daily contact with the person being assisted. Therefore, in this embodiment, biometric information of those involved with the person being assisted may be acquired as data related to the psychological factors of the person being assisted. For example, those involved who become enraged by the peripheral symptoms of the person being assisted may experience an increase in heart rate and activity level, and a decrease in sleep time due to stress. Therefore, based on the biometric information of those involved with the person being assisted, it is possible to estimate the presence or absence of those involved who have an influence on the psychology of the person being assisted.

Here, the bioinformation is, for example, the above-mentioned information such as the heart rate, the amount of activity, and the sleep time. The bioinformation may also include other information that indicates the bioactivity of the person concerned. The bioinformation of the person concerned may be detected, for example, by using the detection device 430 described above with reference to FIG. 7. In addition, a method of using a wearable device such as a wristwatch type device that includes various sensors such as an acceleration sensor and a photoelectric sensor as a device for detecting bioinformation such as the heart rate is known, and the bioinformation of the person concerned may be obtained using these methods. For example, these devices may output the sensor output as bioinformation to the server system 100, or may transmit the heart rate, etc. calculated based on the sensor output to the server system 100 as bioinformation. In addition, the determination of whether the person concerned has become enraged may be made based on whether the bioinformation has changed significantly compared to a normal state. In other words, the bioinformation here may be data that indicates a time-series change.

2.2 Details of Processing in the Server System Next, an example of processing executed in the server system 100 will be described. Below, a learning process for creating the trained model 124, an inference process for estimating the main factors using the trained model 124, and an update process for updating the trained model 124 will be described. Note that the method of this embodiment is not limited to using the trained model 124, and the main factors may be identified by other processing using input data.

2.2.1 Learning process In this embodiment, machine learning using a neural network (NN) may be performed. However, the machine learning is not limited to NN, and other methods such as a support vector machine (SVM) or a k-means method may be used, or methods developed from these may be used. In addition, although supervised learning is exemplified below, other machine learning such as unsupervised learning may also be used.

Figure 11 shows an example of the basic structure of a neural network. Each circle in Figure 11 is called a node or neuron. In the example of Figure 11, the neural network has an input layer, two or more intermediate layers, and an output layer. The input layer is I, the intermediate layers are H1 and Hn, and the output layer is O. In the example of Figure 11, the input layer has two nodes, the intermediate layers have five nodes, and the output layer has one node. However, the number of intermediate layers and the number of nodes in each layer can be modified in various ways. Figure 11 shows an example in which each node in a given layer is connected to all nodes in the next layer, but this configuration can also be modified in various ways.

The input layer accepts input values and outputs them to the intermediate layer H1. In the example of FIG. 11, the input layer I accepts two types of input values. Each node in the input layer may perform some processing on the input value and output the processed value.

In a NN, weights are set between two connected nodes. W1 in Figure 11 is the weight between the input layer I and the first hidden layer H1. W1 represents the set of weights between a given node included in the input layer and a given node included in the first hidden layer. For example, W1 in Figure 11 is information that includes 10 weights.

At each node in the first hidden layer H1, the outputs of the nodes in the input layer I connected to that node are weighted and added using weight W1, and a bias is then added. Furthermore, at each node, the output of that node is found by applying an activation function, which is a nonlinear function, to the sum result. The activation function may be a ReLU function, a sigmoid function, or another function. The activation function may also be a softmax function that normalizes the sum of multiple outputs to 1. For example, the softmax function may be used as the activation function at the last node.

The same is true for subsequent layers. That is, in a given layer, the output of the previous layer is weighted and added using the weight W, a bias is added, and then the activation function is applied to determine the output to the next layer. The output of the output layer is the output of the NN.

As can be seen from the above explanation, in order to obtain the desired output data from input data using a NN, it is necessary to set appropriate weights and biases. In learning, training data is prepared that associates given input data with ground truth data that represents the correct output data for that input data. The NN learning process is a process that determines the most likely weights based on the training data. Note that various learning methods, such as backpropagation, are known for the NN learning process. In this embodiment, these learning methods can be widely applied, so a detailed explanation is omitted.

The structure of the NN in this embodiment is not limited to the example in FIG. 11. For example, the NN in this embodiment may be a Recurrent Neural Network (RNN). An RNN is an NN in which an input at a certain point in time affects the output thereafter, and is suitable for processing time series data. For example, the NN may be a Long Short Term Memory (LSTM).

Figure 12 is a diagram illustrating the input data and output data of the NN in this embodiment. As shown in Figure 12, the input data includes biometric information of the person being assisted, biometric information of people related to the person being assisted, information from electronic medical records and nursing software, sensing data, and environmental information. The input data may also include other data not shown in Figure 12.

The biological information of the person being assisted is information that indicates the state of the biological activity of the person being assisted, and includes information such as heart rate, activity level, and sleep time. The biological information may also include various information such as body temperature, blood pressure, blood oxygen saturation, electrocardiogram, and brain wave measurement results. This information can be acquired using various devices, as in the example of acquiring biological information of related parties. Since biological information is closely related to the condition of the person being assisted, there is a possibility that the biological information will differ between states in which peripheral symptoms are likely to occur and states in which they are unlikely to occur. Therefore, it is considered useful to use biological information when making inferences regarding peripheral symptoms.

The biometric information of related persons is as described above. That is, in this embodiment, the input data used to estimate the main cause may include biometric information of related persons of the person being assisted. And, as described above, the biometric information of related persons is data related to psychological factors. In this way, information related to psychological factors can be included in the input data, so that it becomes possible to appropriately estimate the contribution of psychological factors to peripheral symptoms.

Here, an example has been described in which biometric information of related parties is used to determine whether or not the main cause of peripheral symptoms is a psychological factor. However, a method of using biometric information of related parties who are different from the person being assisted to estimate the psychological state of the person being assisted can also be used in other situations. For example, the method of this embodiment can be applied to an information processing device including an acquisition unit (e.g., corresponding to acquisition unit 111) that acquires biometric information of related parties of the person being assisted, and an estimation unit that estimates the psychological state of the person being assisted based on the biometric information of the related parties. As described above, the psychological state of the person being assisted is strongly influenced by the relationship with the related parties. Therefore, by deliberately using information of the related parties rather than information of the person being assisted himself, it becomes possible to appropriately estimate the psychological state of the person being assisted.

Electronic medical record information is provided by the hospital or other institution where the person being assisted was examined, and includes information such as medical history and medication history. Nursing care software information is information accumulated by software used to manage the person being assisted and the caregiver at the nursing care facility. Nursing care software information includes various information that indicates the life history of the person being assisted within the nursing care facility, such as information indicating the attributes of the person being assisted, such as the age of the person being assisted, the occurrence history of the person's peripheral symptoms (e.g., agitated behavior), food content and intake, the caregiver in charge, and participation status in activities held at the nursing care facility.

For example, because peripheral symptoms may be more likely to occur when suffering from a certain illness or taking a certain medication, it is useful to use information from electronic medical records when making inferences about peripheral symptoms. In addition, the life of a person receiving care in a nursing facility may change depending on the presence or absence of peripheral symptoms and how easily they occur, so information from nursing software is also related to peripheral symptoms. Therefore, it is useful to use information from nursing software when making inferences about peripheral symptoms.

The sensing data is the output of the sensing device 400. Here, the sensing device 400 may include an imaging device 410 that detects the start of movement, a bedside sensor 420 and a detection device 430, a choking detection device 460 that performs processing related to meals, etc. The sensing device 400 may also include an incontinence detection device 450, a seat sensor 440, an acceleration sensor, a foot pressure sensor, etc. The sensing data of these sensing devices 400 is information that represents the abilities of the person being assisted in daily activities, such as the ability to stand up, swallow, sit, and walk. As described above, due to executive dysfunction and apraxia, it becomes difficult for the person being assisted to perform daily activities well. In other words, these sensing data are information related to the first core factor.

In other words, the input data in this embodiment may include ability information that indicates the ability of the person being assisted in daily activities as data related to the first core factor. In this way, it becomes possible to include data related to the first core factor in the input data, making it possible to accurately determine the contribution of the first core factor to peripheral symptoms.

As described above, the ability information here is sensing data that represents the sensing results of at least one of the following: starting to move, eating, maintaining a sitting position, and walking of the person being assisted. In this way, it is possible to obtain information that represents the degree to which movements in daily life are performed normally as ability information. Since it is possible to obtain information that is highly related to executive dysfunction and apraxia, it becomes possible to accurately determine the degree of contribution of the first core factor to peripheral symptoms.

The sensing device 400 that outputs the sensing data may also include a detection device 430, an RFID reader and IC tag, an incontinence detection device 450, and the like. The detection device 430, the RFID reader and IC tag output wandering information as described above. The incontinence detection device 450 outputs incontinence information as described above. As described above, due to disorientation, it becomes difficult for the person being assisted to recognize the situation they are in, leading to increased wandering and incontinence. In other words, these sensing data are information related to the second core factor.

That is, the input data in this embodiment may include at least one of incontinence information regarding incontinence of the caregiver and wandering information regarding wandering of the assisted person as data related to the second core factor. In this way, it becomes possible to include data related to the second core factor in the input data, making it possible to accurately determine the contribution of the second core factor to the peripheral symptoms.

The input data may also include environmental information. As described above, the environmental information includes information on brightness, temperature, humidity, scent, sound, video, and season. In this way, data related to environmental factors can be included in the input data, making it possible to accurately determine the contribution of environmental factors to peripheral symptoms.

As described above, the NN may be an RNN, and the input data in this embodiment may be time-series data. In other words, each data included in the input data described above is not limited to data at a single timing, and may be time-series data.

As shown in FIG. 12, the output data of the NN may include the contribution of environmental factors to peripheral symptoms, the contribution of the first core factor, the contribution of the second core factor, and the contribution of psychological factors. The contribution here may be information indicating the likelihood that the peripheral symptoms have been caused by the target factor.

For example, the NN may output five pieces of data: a peripheral symptom score indicating the likelihood that a peripheral symptom has occurred; an environmental factor score indicating the likelihood that a peripheral symptom has occurred due to an environmental factor; a first core factor score indicating the likelihood that a peripheral symptom has occurred due to a first core factor; a second core factor score indicating the likelihood that a peripheral symptom has occurred due to a second core factor; and a psychological factor score indicating the likelihood that a peripheral symptom has occurred due to a psychological factor. For example, each score may be information normalized to a value between 0 and 1. In this case, the peripheral symptom score indicates the probability that a peripheral symptom has occurred. Furthermore, each factor score indicates the degree of contribution of the corresponding factor to the peripheral symptom. In this way, it becomes possible to determine whether a peripheral symptom has occurred and the contribution of each factor when a peripheral symptom has occurred based on the output data.

However, as shown in FIG. 12, the peripheral symptom score may not be directly output. For example, the sum of the environmental factor score, the first core factor score, the second core factor score, and the psychological factor score may be used as the peripheral symptom score. In this case, too, it is possible to determine whether or not peripheral symptoms have occurred, and the contribution of each factor when peripheral symptoms have occurred, based on the output data of the NN.

Note that, although an example of a NN that both determines whether peripheral symptoms have occurred and determines the degree of contribution of each factor has been given here, this is not limiting. For example, the determination of whether peripheral symptoms have occurred and the determination of the degree of contribution of each factor may each be performed using different NNs.

The NN may output information that identifies more detailed factors, rather than simply identifying one of the environmental factors, the first core factor, the second core factor, and the psychological factor as the main factor. For example, the environmental factors may be divided into more detailed factors such as brightness factors, temperature factors, humidity factors, and seasonal factors. The first core factors can be divided into movement factors, eating factors, sitting position factors, walking factors, etc. The second core factors can be divided into nighttime wandering factors, incontinence factors, entering inappropriate places factors, etc. In addition, various modifications of the NN configuration are possible.

Figure 13 is a flowchart explaining the learning process in which the learning unit 115 creates the learned model 124. First, in step S101, the learning unit 115 acquires learning input data. An example of the input data is as described above with reference to Figure 12, and data of a similar configuration can be used for the learning input data.

Next, in step S102, the learning unit 115 acquires correct answer data. The correct answer data here includes at least information that identifies the main cause of the peripheral symptoms. In the above example, the correct answer data may also include information that identifies the occurrence or non-occurrence of the peripheral symptoms. The correct answer data here is data input by an expert who has knowledge of the symptoms and causes of dementia, such as a doctor or an experienced caregiver. For example, an expert who examines and assists the person being assisted estimates the occurrence or non-occurrence of the peripheral symptoms and the main cause, and inputs the estimation result. For example, the learning unit 115 displays an annotation screen on the PC of the expert connected to the server system 100, and prompts the input of the estimation result on the annotation screen. Then, the learning unit 115 associates the input annotation result with the input data related to the target person being assisted as correct answer data. As a result, training data in which the correct answer data is associated with the learning input data is acquired.

In step S103, the learning unit 115 performs machine learning based on the training data. Specifically, the learning unit 115 inputs learning input data to the NN and obtains output data by performing forward calculations using the weights at that stage. The learning unit 115 calculates an objective function based on the output data and the correct answer data. The objective function here is, for example, an error function based on the difference between the output data and the correct answer data, or a cross entropy function based on the distribution of the output data and the distribution of the correct answer data. The learning unit 115 updates the weights based on the objective function. The above-mentioned backpropagation method and the like are known as weight update methods, and these methods can be widely applied in this embodiment as well.

In step S104, the learning unit 115 determines whether to end the learning process. For example, the learning unit 115 ends the learning process when a given condition is satisfied. For example, the training data may be divided into learning data and validation data. The learning unit 115 may end the learning process when the accuracy rate based on the validation data exceeds a given threshold. After the learning process is completed, the NN including the weights at that stage is stored in the storage unit 120 as the trained model 124. Note that the learning process is not limited to being executed by the server system 100, and may be executed by an external device. The server system 100 may acquire the trained model from the external device.

2.2.2 Inference process The factor estimation unit 112 may perform a process of finding the main factor based on the trained model 124, which takes the input data as input and outputs a peripheral symptom score indicating the likelihood that the person being assisted is in a peripheral symptom state and each of the factor scores of a plurality of factors. As described above, the peripheral symptom score is not limited to being directly output, and may be found as the sum of a plurality of factor scores, etc. Specifically, the factor estimation unit 112 finds the main factor by using the trained model 124 acquired by the above-mentioned learning process. In this way, machine learning can be applied to the estimation of the main factor, making it possible to find the main factor with high accuracy.

Figure 14 is a flowchart explaining the inference process for inferring the main cause of peripheral symptoms based on input data. The factor estimation unit 112 may, for example, periodically execute the inference process targeting all assisted persons using the information processing system 10. Alternatively, the factor estimation unit 112 may execute the inference process targeting assisted persons suffering from dementia or suspected of having dementia. Furthermore, when a caregiver or related person determines that an assisted person is showing peripheral symptoms, the factor estimation unit 112 may execute the inference process in response to an operational input by the caregiver or the like as a trigger. In addition, various modifications are possible regarding the target and timing for executing the inference process.

First, in step S201, the acquisition unit 111 acquires input data. The input data is as described above with reference to FIG. 12. For example, the acquisition unit 111 acquires information on electronic medical records and nursing care software from the management terminal device 300, and acquires sensing data from the sensing device 400. The environmental information is stored, for example, in the nursing care software, and is transmitted from the management terminal device 300 to the acquisition unit 111. However, the environmental information may be transmitted to the acquisition unit 111 directly from a device that detects the environmental information. The biometric information of the person being assisted and related parties is transmitted to the acquisition unit 111 from a device that detects the biometric information. However, the biometric information may be transmitted to the acquisition unit 111 via another device, such as a terminal device used by the related party or the management terminal device 300.

In step S202, the factor estimation unit 112 estimates the main cause of the peripheral symptoms based on the input data. For example, the factor estimation unit 112 may obtain factor scores representing the contribution to the peripheral symptom state for each of the environmental factors, the first core factors, the second core factors, and the psychological factors based on the input data, which is time-series data. The factor scores here correspond to the scores described above with reference to FIG. 12, for example. For example, the factor estimation unit 112 reads out the trained model 124, which is an RNN or an LSTM, from the storage unit 120, and obtains the peripheral symptom score and the factor score for each factor by inputting the input data, which is time-series data, into the trained model 124. When a peripheral symptom occurs, it is not only the most recent event that affects the person being assisted, but past events and their accumulation may affect the person being assisted. For example, there may be cases where the second core factor is suspected when only a single timing is observed, but when observed over a longer span, it is determined that the contribution of the first core factor was large. In this regard, by using the input data that is time-series data, it becomes possible to estimate the main cause with high accuracy. In step S202, the factor estimation unit 112 may obtain a peripheral symptom score that indicates the likelihood that a peripheral symptom has occurred. The peripheral symptom score may be one of the outputs of the NN as described above, may be the sum of factor scores related to multiple factors, or may be obtained from other outputs.

In step S203, the factor estimation unit 112 determines whether the peripheral symptom score is equal to or greater than a given threshold. If the peripheral symptom score is less than the threshold (S203: No), it is estimated that no peripheral symptoms are occurring. Therefore, the processes in steps S204-S206 are not performed, and the inference process shown in FIG. 14 ends.

If the peripheral symptom score is equal to or greater than the threshold (S203: Yes), there is a high probability that a peripheral symptom is occurring. Therefore, in step S204, the factor estimation unit 112 estimates the main factor. For example, the factor estimation unit 112 estimates the main factor based on the degree of fluctuation of the factor score. For example, the factor estimation unit 112 calculates the degree of increase in the factor score during a specified period. The degree of increase here may be the ratio of the factor score at the end of the period to the factor score at the start of the period. The degree of increase may also be the ratio of the maximum value of the factor score to the minimum value during the period. In addition, various modifications of the method of calculating the degree of increase are possible. The factor estimation unit 112 determines that the factor with the largest degree of increase in the factor score is the main factor.

Also, in step S204, if the rate of increase in the factor scores of all of the environmental factors, first core factors, second core factors, and psychological factors is below a predetermined threshold, the factor estimation unit 112 may determine that the main factor is a psychological factor. As described above, it is possible to use biometric information of related parties as information related to psychological factors, but there may be people or events other than the related parties whose biometric information is being measured that place a burden on the psychology of the person being assisted. In this case, even if it is actually a psychological factor, the factor score of the psychological factor may not change. Therefore, if the rate of increase in all of the factor scores is small, the factor estimation unit 112 may determine that the main factor is a psychological factor.

Note that the factor scores of each factor in this embodiment may vary from person to person. For example, because the ability to perform daily actions is related not only to cognitive function but also to motor function, the level of ability represented by the ability information also changes depending on the motor function of the person being assisted. Therefore, even if there are assisted persons in which executive dysfunction contributes to peripheral symptoms to the same extent, some assisted persons will have a high factor score value for the first core factor and others will have a low factor score value depending on the level of motor function. The same is true for other factors, and the magnitude relationship between the factor scores of the environmental factors, first core factors, second core factors, and psychological factors may vary from person to person being assisted, and if the values are simply compared (for example, if the factor with the largest factor score is estimated to be the main factor), the estimation accuracy may not be sufficient. In this regard, by using the degree of fluctuation in the factor score, a judgment is made based on the past value of the assisted person in question. As a result, the influence of individual differences can be suppressed, so the main factor can be accurately estimated. However, in this embodiment, it is not prevented from determining the main factor using the magnitude of the factor score itself.

As described above in the learning process, the NN is not limited to estimating the factor scores of the environmental factors, the first core factors, the second core factors, and the psychological factors, and may obtain factor scores of factors obtained by dividing each factor more finely. That is, the factor estimation unit 112 may estimate a main factor from among a plurality of sub-factors when a factor obtained by dividing at least one of the environmental factors, the first core factors, the second core factors, and the psychological factors is a sub-factor. Alternatively, the factor estimation unit 112 may estimate a main factor from among the four factors using the trained model 124, and then separately execute a process of estimating the sub-factor with the highest contribution degree included in the main factor. For example, the factor estimation unit 112 estimates the sub-factor with the highest contribution degree according to which data of the input data related to the main factor has the largest fluctuation. For example, if the main factor is the first core factor, the factor estimation unit 112 may determine that the rising factor is dominant if the fluctuation in the pressure value during rising is large, and may determine that the meal factor is dominant if the fluctuation in the various information measured during a meal is large.

The information processing device 20 of this embodiment may further include a countermeasure decision unit that determines how to deal with the peripheral symptom state according to the main cause estimated by the factor estimation unit 22. For example, as shown in FIG. 4, the server system 100 includes a countermeasure decision unit 113 that determines how to deal with the peripheral symptom state according to the main cause estimated by the factor estimation unit 112. In this way, not only is the main cause estimated, but a countermeasure according to the main cause can be determined, making it possible to reduce the burden on the caregiver. However, the notification content may be changed depending on the caregiver to be notified, such as notifying professionals such as caregivers only of the main cause and notifying non-professionals such as the family of the person being assisted of the countermeasure.

For example, in step S205, the treatment decision unit 113 performs a process of requesting a recommended treatment from the caregiver or related parties based on the estimated main cause. For example, the storage unit 120 may store a treatment table 125 as shown in FIG. 4. The treatment table 125 is information in which the main cause is associated with a recommended treatment. The treatment decision unit 113 requests a treatment based on the estimated main cause and the treatment table 125. Hereinafter, information representing the treatment requested by the treatment decision unit 113 will be referred to as treatment information.

For example, if it is determined that the main cause is an environmental factor, the treatment decision unit 113 outputs treatment information indicating a treatment to align the environment of the person being assisted with a reference environment. The reference environment here is an environment that is favorable for the person being assisted, for example, the environment before a major environmental change occurred. For example, if a person being assisted who had been spending time at home enters a facility, etc., and then develops peripheral symptoms that are estimated to be primarily caused by environmental factors, the reference environment corresponds to the home environment of the person being assisted. Alternatively, if there is a change in the room or rearrangement in the nursing facility, the environment before the change may be used as the reference environment. Furthermore, a process may be performed to estimate an environment in which the target person being assisted is less likely to develop peripheral symptoms, based on a log of the occurrence or frequency of peripheral symptoms of the person being assisted. In this case, the environment in which peripheral symptoms are less likely to occur is used as the reference environment.

For example, information specifying the reference environment (hereinafter, referred to as reference environment information) may be obtained based on log data of home appliances such as lighting equipment, air conditioners, music players, and televisions in the reference environment. These home appliances may be controlled by an application such as a smartphone, and the reference environment information may be obtained by acquiring setting information of each appliance using the application. In this case, an application having a function of sharing information about the home environment with other appliances such as the server system 100 may be provided. The reference environment information may be obtained by performing measurements using a photoelectric sensor, an odor sensor, a microphone, etc. In addition, although information such as brightness, temperature, humidity, scent, sound, and video has been exemplified as the reference environment information in the above, the reference environment information is not limited to this. For example, the type of assistance equipment used and information on the operation history may be used as the reference environment information. In this way, it is possible to match the equipment used to assist the person being assisted and its operation settings to the reference environment. The assistance equipment here may be a nursing bed whose height and bottom angle can be changed, a reclining wheelchair whose back angle can be changed, a walker, or other equipment. Operational settings may include the settings for height, bottom angle, back angle, etc., as mentioned above.

The response decision unit 113, for example, acquires reference environmental information, and determines a recommended response to be a response that adjusts the environment of the person being assisted to the reference environment. Note that, although an example in which the response decision unit 113 acquires reference environmental information has been shown here, this is not limiting. For example, the response decision unit 113 may output information indicating that acquisition of reference environmental information is recommended, and the reference environmental information may be acquired by another device or the like that has acquired the information.

As described above, the input data in this embodiment may include information for identifying the season as data related to environmental factors. The factor estimation unit 112 may then determine whether the main factor is a seasonal factor that is caused by the season among environmental factors. If such a determination is possible, the treatment decision unit 113 may take different measures when the main factor is estimated to be a seasonal factor and when it is estimated to be an environmental factor other than a seasonal factor.

For example, when an environmental factor other than a seasonal factor is the main factor, the treatment decision unit 113 recommends a treatment to match the reference environment as described above. On the other hand, there are cases where dementia patients are more likely to develop peripheral symptoms when a certain season arrives or when the seasons change. In the case of peripheral symptoms caused by seasonal factors, the suppression effect of the peripheral symptoms may be low even if the home environment is adapted. Also, if the peripheral symptoms are caused by seasonal factors, the peripheral symptoms may subside naturally after the target season. Therefore, when it is determined that the factor is seasonal, the treatment decision unit 113 may recommend a treatment to suppress an active response. In this way, it is possible to suppress a treatment that is less necessary. Also, if it is known that the factor is seasonal, it is sufficient to get through that season, so the period of heavy burden can be estimated to some extent. Therefore, it is possible to reduce the psychological burden on the caregiver.

Furthermore, when it is estimated that the main cause is the first core cause, the treatment decision unit 113 may perform a process of deciding that the recommended treatment is to use an assistance device that supports daily movements. As described above, when the first core cause is the main cause, the peripheral symptoms are caused by the inability to perform daily movements. By introducing an assistance device that supports daily movements, it becomes easier for the person being assisted to perform daily movements. Note that the use of an assistance device here is not limited to the introduction of a new device, and may also include changing the operating mode of an existing device.

The countermeasure decision unit 113 may recommend a countermeasure of communicating with the person being assisted or the caregiver before the introduction of the assistance device. For example, a process of hearing from the person being assisted or the caregiver as to whether or not support for the movement is necessary may be performed using a communication robot or a chat robot. If the answer given in the hearing is that support for the movement is necessary, the countermeasure decision unit 113 recommends the introduction of the assistance device. For example, the countermeasure decision unit 113 may perform a process of identifying a specific device type, vendor, model number, etc. Alternatively, the countermeasure decision unit 113 may perform a process of identifying an EC (Electric Commerce) site where the assistance device can be purchased. The countermeasure decision unit 113 may output information indicating that such a process is recommended, and the actual process may be executed by another device that receives the information.

Furthermore, when it is estimated that the main cause is the second core cause, the treatment decision unit 113 may perform a process of deciding that the recommended treatment is to intervene against the risk of the person being assisted falling. When disorientation is a major factor, the person being assisted cannot grasp the situation he or she is in, which increases the risk of falling. Therefore, by intervening against the risk of falling, the occurrence of incidents due to falls can be reduced.

For example, the countermeasure decision unit 113 may output a countermeasure to use a portable airbag that can be attached to the waist of the person being assisted. In this way, even if the caregiver is not near the person being assisted or there is a shortage of manpower, injuries due to falls can be prevented. The countermeasure decision unit 113 may also recommend a countermeasure to perform a movement detection process using the imaging device 410, bedside sensor 420, acceleration sensor, etc. described above with reference to FIG. 6, etc. For example, when a movement is detected, the caregiver is notified of this using the output image IM1 of FIG. 6, etc. In this way, the caregiver can be notified of the presence of a person being assisted who is likely to fall, so that the risk of falling can be reduced. The movement may also be detected using the detection device 430 of FIG. 7. For example, the detection device 430 determines that the person being assisted has started to move when the person has moved from a bed-in state to a bed-out state, or from a sleep state to an awake state. For example, by presenting the detection result of the movement to the caregiver, the level of the risk of falling can be presented, so that the need for intervention can be presented to the caregiver in an easy-to-understand manner.

Furthermore, when it is estimated that the main cause is a psychological factor, the countermeasure decision unit 113 may perform a process of determining that the recommended countermeasure is to identify related parties who have a high degree of influence on the psychology of the person being assisted. As described above, related parties who influence the psychology of the person being assisted are highly likely to show changes in their biometric information. However, related parties do not always live with the person being assisted, and the biometric information may change for reasons unrelated to the person being assisted. In other words, even if a related party's biometric information changes, this does not necessarily mean that the related party is affecting the psychology of the person being assisted.

Therefore, the treatment decision unit 113 may recommend a treatment that determines the strength of the involvement of the relevant person. For example, the person being assisted and the relevant person may each carry a communication device, and a process may be performed to determine the degree of involvement of the relevant person based on beacon signals sent and received by the communication device. For example, the longer a person is located within a specified distance from the person being assisted, the higher the level of involvement in the person being assisted is determined to be. Note that the beacon signal here may be a Bluetooth (registered trademark) advertisement packet, an SSID broadcast in IEEE802.11, or a beacon signal of another communication method. If a relevant person has a specified level of involvement or higher and has a large change in biological information, it is determined that the relevant person is highly likely to be a cause of the peripheral symptoms.

In this case, the treatment decision unit 113 may recommend a treatment of having a conversation with the person being assisted using a communication robot or a chat robot. For example, in the conversation, questions about the factors from among the environmental factors, the first core factors, and the second core factors may be asked to the person being assisted in the order of the increase rate of the factor score. For example, in a case where the person being assisted is scolded by the relevant parties because he or she is unable to do something well, it is useful to identify the specific situation of what behavior caused the relevant parties to become angry. In this regard, since the higher the increase rate of the factor score, the more likely it is to be a cause of peripheral symptoms, by asking questions in that order, it is possible to efficiently identify the reason why the relevant parties scold the assistant (i.e., the reason why the person being assisted feels stressed). In this case, the conversation may be converted into text and a text mining process may be performed on the text to identify the situation in which the person being assisted feels psychological stress.

When a situation in which the person being assisted feels psychologically stressed is identified, the treatment decision unit 113 may further recommend a treatment of presenting a summary of the conversation to the relevant parties. This may be a text summary or a summary of audio recordings of the conversation. In this way, the relevant parties can be notified of the situation and the specific words and actions that caused stress to the person being assisted, thereby preventing a similar situation from recurring.

The information processing device 20 of this embodiment may include a presentation processing unit that presents at least one of the main cause estimated by the factor estimation unit 22 and the countermeasure determined by the countermeasure determination unit. For example, as shown in FIG. 4, the server system 100 includes a presentation processing unit 116 that presents at least one of the main cause estimated by the factor estimation unit 112 and the countermeasure determined by the countermeasure determination unit 113.

For example, in step S206, the presentation processing unit 116 performs processing to present the main cause and measures. For example, the presentation processing unit 116 presents the main cause and measures to the terminal device 200 used by the caregiver via the communication processing unit 114 and the communication unit 130. The presentation here may be the display of an image or text, or may be the output of sound using a headset speaker.

Note that the countermeasures are presented by presenting the equipment and operations required to execute the countermeasures, and the specific operations may be left to the caregiver. However, the method of this embodiment is not limited to this, and the server system 100 may automatically control the controlled device based on the countermeasures determined by the countermeasure determination unit 113. For example, the control of a nursing bed or a reclining wheelchair to match the reference environment may be automatically performed based on instructions from the server system 100. In addition, when a conversation with a person being assisted is conducted using a communication robot or the like, the operation to start the communication robot and start the conversation may be automatically performed based on a control signal from the server system 100. In addition, various modifications of the specific method of executing the recommended countermeasures are possible.

Also, in FIG. 12, various input data is input to the NN, and output data including factor scores corresponding to a plurality of factors is output based on the input data. However, since inputting many parameters to the NN increases the processing load, the factor estimation unit 112 may divide the processing. For example, the factor estimation unit 112 performs a first judgment to determine the main factor based on the first input data, and if the main factor is not estimated in the first judgment, performs a second judgment to determine the main factor based on the second input data including data not included in the first input data. For example, the factor estimation unit 112 first performs a process to obtain the factor scores of the environmental factor and the first core factor by using input data related to the environmental factor and the first core factor as the first input data, and if the degree of fluctuation of either is equal to or greater than a predetermined value, the factor is estimated as the main factor and the estimation process of the main factor is terminated. Then, the factor estimation unit 112 may obtain the factor scores of the second core factor and the psychological factor by using input data related to the second core factor and the psychological factor as the second input data when the degree of fluctuation of both the factor scores of the environmental factor and the first core factor are less than a threshold value. In this case, in the first stage of processing, it is possible to omit inputting the wandering information, which is data related to the second core factor, and the biometric information of the relevant person, which is data related to the psychological factor. Also, in the second stage of processing, it is possible to omit inputting the environmental information related to the environmental factor, and the ability information, which is data related to the first core factor. As a result, it is possible to reduce the processing load on the factor estimation unit 112.

The second input data may also be data that includes the first input data. For example, the first input data may be a portion of the input data shown in FIG. 12, and the second input data may be all of the input data shown in FIG. 12. In this way, the first judgment, which has a relatively low load, can be attempted first. If the main cause is estimated in the first judgment, the second judgment becomes unnecessary, making it possible to reduce the processing load.

In addition, when data related to any one of the environmental factors, the first core factors, the second core factors, and the psychological factors, and not related to any of the other three, is defined as unique input data, the second input data may include at least one unique input data that is not included in the first input data. For example, in the example described above using FIG. 12, information about incontinence is not unique input data because it is related to both the first core factor and the second core factor. On the other hand, information such as brightness, temperature, and humidity included in the environmental information is less related to the first core factor, the second core factor, and the psychological factors, and is therefore unique input data.

In this way, data obtained by excluding at least one unique input data from the second input data can be used as the first input data. For example, the first judgment may be processed mainly on data whose relevance is not biased toward a specific factor, such as biometric information of the person being assisted, electronic medical records, and information from nursing software. In this way, even if the number of types of data included in the first input data is reduced, it is possible to prevent judgments that are biased toward a specific factor (whereby the factor score of a specific factor tends to be high). In addition, since the second input data includes unique input data, it is possible to accurately determine the factor score of each factor. As described above, by making the presence or absence and number of unique input data different between the first input data and the second input data, it is possible to combine two judgments with different characteristics.

Although the method of estimating the main factor based on the trained model 124 has been described above, the method of this embodiment is not limited to this. For example, the factor estimation unit 112 may obtain the degree of variability over time for each of the multiple input data. For example, the factor estimation unit 112 may identify the input data with the largest degree of variability and determine the factor corresponding to the input data as the main factor. The correspondence between each input data and the factor is as described above. The factor estimation unit 112 may obtain the evaluation value of each factor by obtaining the degree of variability of the multiple input data. For example, the evaluation value may be numerical data that increases as the degree of variability of the input data corresponding to the target factor increases. For example, when multiple input data are associated with a given factor, the evaluation value of the given factor is calculated based on the degree of variability of each of the multiple input data. The factor estimation unit 112 may estimate that the factor with the largest evaluation value among the multiple factors is the main factor. In addition, various modifications are possible regarding the details of the process of estimating the main factor.

2.2.3 Update process In the present embodiment, an update process of the trained model 124 may be performed. For example, when a process of presenting factor information representing a main cause is performed, the learning unit 115 may perform a process of updating the trained model 124 based on the factor information, the first peripheral symptom score which is the peripheral symptom score before the presentation of the factor information, and the second peripheral symptom score which is the peripheral symptom score after the presentation of the factor information. Here, the first peripheral symptom score is not limited to the peripheral symptom score at a single timing, and may be a statistic for a predetermined period before the presentation of the factor information. For example, the first peripheral symptom score may be an average value, a maximum value, or a minimum value for a predetermined period before the presentation of the factor information. The first peripheral symptom score may be the number of times the peripheral symptom score exceeds a threshold corresponding to the presence of a peripheral symptom. The second peripheral symptom score is similar to the first peripheral symptom score, except that the target period is the period after the presentation of the factor information.

In this way, it becomes possible to compare the degree of occurrence of peripheral symptoms before and after the presentation of the main cause. For example, if it is determined that the degree of occurrence of peripheral symptoms is decreasing based on the first peripheral symptom score and the second peripheral symptom score, it indicates that the presented main cause is appropriate and that the condition of the person being assisted has improved by taking measures based on the main cause. In this case, the trained model 124 is updated so that the same main cause is more likely to be selected in similar situations. For example, the learning unit 115 performs a weight update process by treating the training data that corresponds the input data to the estimated result of the main cause as positive training data.

On the other hand, if it is determined that the occurrence of the peripheral symptoms is increasing based on the first peripheral symptom score and the second peripheral symptom score, the presentation of the main cause has not led to an improvement in the peripheral symptoms. In other words, the estimation result of the main cause may have been incorrect. In this case, the trained model 124 is updated so that the same main cause is less likely to be selected in a similar situation. For example, the learning unit 115 performs a weight update process by using training data that associates the input data with the estimation result of the main cause as negative training data.

In addition, when the process of presenting the cause information and the countermeasure information representing the countermeasure is performed, the learning unit 115 may perform a process of updating the learned model based on the cause information, the countermeasure information, the first peripheral symptom score, and the second peripheral symptom score. In this case, it becomes possible to update the learned model 124 not only from the perspective of presenting the main cause, but also from the perspective of whether the presentation of the countermeasure contributed to the improvement of the peripheral symptoms.

2.3 Specific example of meal assistance A specific example of measures to be taken based on the estimation results of the main factors will be described using meal assistance as an example. For example, consider a case where a nursing facility has previously left it up to the care recipient to eat their own meals and has used the choking detection device 460 for monitoring meals.

In this case, since the peripheral symptom score has exceeded the threshold value since a certain point, the factor estimation unit 112 may execute a process to identify the main factor. The sensing data of the choking detection device 460 may be used as input data when calculating the peripheral symptom score from the beginning. Alternatively, the sensing data of the choking detection device 460, which is unique input data, may be added to the input data when the peripheral symptom score calculated without using the sensing data of the choking detection device 460 exceeds the threshold, which is a trigger. In any case, the factor estimation unit 112 estimates that the first core factor is the main factor based on the input data, and more specifically, determines that the contribution of "looking at the meal without doing anything" is high.

In this case, the countermeasure determination unit 113 determines the use of an assistance device that supports eating as the recommended countermeasure, and the presentation processing unit 116 presents the countermeasure to the caregiver's terminal device 200. The presentation processing unit 116 may first output a voice message such as "Do you need assistance with eating?" on the terminal device 462 of the choking detection device 460, and present the above countermeasure to the caregiver's terminal device 200 when the person being assisted responds "Yes" or nods.

The assistance device here may be an automatic feeder 470 that automatically delivers food to the mouth of the person being assisted, as shown in FIG. 15. The automatic feeder 470 includes a container 472 that includes multiple plates for holding food, and an arm 471 that scoops up a predetermined amount of food from the container 472 and delivers it to the mouth of the person being assisted. The container 472 may be provided with a tab 473. The tab 473 will be described later with reference to FIGS. 17A to 17G. The arm 471 includes, for example, multiple frames connected by joints, and an end effector provided at the tip. The end effector here may be, for example, a hand that holds a spoon, or the spoon itself. The automatic feeder 470 may also include a scale 474 that measures weight, and a camera 475. In the example of FIG. 15, the scale 474 is disposed adjacent to the container 472, but is not limited thereto. For example, the scale may be built into the spoon, or the scale may be provided on the arm 471, and the weight may be measured from the change in the weight of the spoon. In the example of FIG. 15, the camera 475 is disposed near the tip of the arm 471 and captures, for example, the contents of the spoon or the face of the person being assisted. For example, a device for automatically feeding a patient is described in U.S. Patent Application No. 15/094800, filed on April 8, 2016, entitled "APPARATUS AND METHOD FOR FOOD CAPTURE." This patent application is incorporated herein by reference in its entirety. The automatic feeding device 470 in this embodiment may be, for example, the feeding device disclosed in U.S. Patent Application No. 15/094800, or a device having a similar mechanism.

FIG. 16 is a diagram illustrating an example of a connection between the choking detection device 460 and the automatic feeder 470. For example, the automatic feeder 470 is connected to the terminal device 462 of the choking detection device 460. As described above, the terminal device 462 is connected to the throat microphone 461. These connections may be wired connections using a cable such as a USB (Universal Serial Bus) cable, or wireless connections using WiFi (registered trademark), Bluetooth, etc.

Note that various specific modes of measures to recommend the use of the automatic supply device 470 are conceivable. For example, the presentation processing unit 116 may present information recommending the use of the automatic supply device 470 as the countermeasure information representing the recommended countermeasure. In this case, the decision on whether to use the automatic supply device 470, the arrangement of the automatic supply device 470 when it is used, and the connection shown in FIG. 16 are performed by the caregiver or the like. The presentation processing unit 116 may also perform a process of displaying the connection method of the automatic supply device 470 and the choking detection device 460 on the terminal device 200 as the countermeasure information representing the recommended countermeasure. In this way, it is possible to smoothly introduce the automatic supply device 470. In addition, the countermeasure decision unit 113 may first confirm the need for support by communicating with at least one of the assisted person and the caregiver using a communication robot or the like. If a response is received that support is necessary, the countermeasure decision unit 113 may output countermeasure information to the effect that the automatic supply device 470 is automatically transported and the automatic supply device 470 is automatically connected to the terminal device 462. The device that performs the automatic transport and automatic connection is not limited to the server system 100, but may be another device that receives the handling information. In this case, the connection between the automatic supply device 470 and the terminal device 462 may be a wireless connection such as WiFi. In this way, the arrangement and connection of the automatic supply device 470 can be automated, which can reduce the burden on the caregiver, etc.

The automatic feeder 470 may be taught before actual use. Teaching is a process in which, for example, a caregiver actually moves the arm 471, causing the automatic feeder 470 to memorize the movement of the arm 471 up to the point where the spoon is brought to the mouth of the person being assisted. Note that teaching is a well-known technique in the field of robotics, so a detailed explanation will be omitted.

After connection and teaching are completed, meal assistance is performed in cooperation with the choking detection device 460 and the automatic feeder 470. For example, the automatic feeder 470 may determine the amount of food to put on the spoon at one time and the pace at which the spoon is brought to the mouth based on the determination result of the choking detection device 460. For example, the choking detection device 460 may learn these parameters in advance based on the tacit knowledge of an expert. The choking detection device 460 (or, more narrowly, the terminal device 462) may store application software that executes a process corresponding to the tacit knowledge and transmit the application software to the automatic feeder 470. In this way, at least a part of the process (process corresponding to tacit knowledge) in the choking detection device 460 described below can be executed in the automatic feeder 470.

For example, the automatic feeder 470 puts a serving of food on the spoon and calculates the amount of food based on the scale 474 and image processing. The terminal device 462 determines, based on the tacit knowledge described above, whether the amount of food put on the spoon is appropriate for the person being assisted.

If it is determined that the amount is appropriate, the automatic feeder 470 brings the spoon to the mouth of the person being assisted based on the teaching results. If it is determined that the mouth of the person being assisted is open based on the captured image, the arm 471 is controlled to place the spoon into the mouth. Note that the captured image here may be captured by the camera of the terminal device 462. Alternatively, the captured image may be captured by a camera 475 installed near the tip of the arm 471.

The terminal device 462 judges whether swallowing has been detected based on the audio data from the throat microphone 461. If swallowing has been detected, it judges whether there is any mouth choking. Mouth choking means that food remains in the mouth without being swallowed. For example, it is known that the more food is swallowed, the louder the swallowing sound becomes. Therefore, the terminal device 462 can estimate the volume of the swallowing sound when there is no mouth choking based on the amount of food taken up on the spoon. If the swallowing sound detected by the throat microphone 461 is quieter than the expected swallowing sound, the terminal device 462 judges that there is mouth choking and waits until swallowing occurs again. Furthermore, if the terminal device 462 judges that the swallowing sound is sufficiently loud and there is little mouth choking, it allows the automatic feeder 470 to operate for the next bite. This causes the automatic feeder 470 to repeat the above operation again. By repeating this loop process, it becomes possible to properly support a person receiving care who is no longer able to eat properly.

In the above loop process, the choking detection device 460 may determine whether a situation has occurred in which the mouth has stopped opening, the time until swallowing has become longer, the patient has become sleepy, etc. For example, the terminal device 462 determines whether the mouth has stopped opening based on an image captured by a camera. The terminal device 462 also determines the time from opening the mouth to swallowing based on the captured image and the audio data from the throat microphone 461. The terminal device 462 also determines that the patient is sleepy when a tilt in posture is detected based on the seat sensor 440. When a meal is eaten in bed, the detection device 430 may be used to determine whether the patient is sleepy.

If it is determined that the patient has stopped opening his/her mouth, is taking longer to swallow, or is becoming drowsy, the terminal device 462 prompts the person being assisted to take action by outputting audio data. After outputting the audio data, the terminal device 462 determines whether the situation has improved, and if so, returns to the loop process described above. If the situation has not improved, the terminal device 462 may output audio data again. Alternatively, the terminal device 462 may request the caregiver to intervene.

As described above, there is an appropriate amount of food for a person receiving assistance when it comes to meal assistance, and this amount may vary depending on the person receiving assistance. Therefore, when using the automatic feeding device 470, it is desirable to be able to finely adjust the amount of food put into the spoon. Therefore, in this embodiment, the height of the tab 473 provided on the container 472 may be adjusted. In this way, fine adjustment of the amount of food is possible. In particular, when the amount scooped with the spoon is too much, it becomes easy to reduce it to an appropriate amount. A specific example of the tab 473 will be described below. Note that the tab 473 here is a member that has one end connected to the periphery of the plate part in which the food is placed, and the other end inclined in the direction toward the center of the plate part. By providing the tab 473, it is possible to prevent food from spilling out of the plate when scooping food with the spoon. Also, by providing the tab 473, the tab 473 pushes the food toward the back of the spoon, making it easier to scoop food.

Figures 17A to 17G are diagrams showing an example of the configuration of the tab 473 in this embodiment. Figure 17A is a diagram of the plate portion of the container 472 and the tab 473 observed from vertically above. As described above, one end of the tab 473 is connected to the periphery of the plate portion in which the food is placed, and the other end is inclined toward the center of the plate portion.

Figures 17B to 17D are diagrams illustrating the cross-sectional structure of tab 473 at three locations, A-A, B-B, and C-C in Figure 17A, respectively. Figure 17B shows the cross-sectional structure at A-A, where tab 473 has surface 473a that slopes toward the center of the dish portion, and member 473b that is connected to surface 473a and has a surface oriented along the horizontal direction. For example, member 473b is a member that has a roughly triangular cross-sectional shape as shown in Figure 17B.

Similarly, FIG. 17C is an example of a cross-sectional structure taken along line B-B, and FIG. 17D is an example of a cross-sectional structure taken along line C-C. As in FIG. 17A, tab 473 has inclined surface 473a and member 473b having a substantially triangular cross-sectional shape. However, as can be seen from FIGS. 17B to 17D, the height of member 473b may vary depending on the position on tab 473. For example, when the height to member 473b in FIG. 17B is L1, the height to member 473b in FIG. 17C is L2, and the height to member 473b in FIG. 17D is L3, L1<L2<L3.

For example, when the automatic feeder 470 moves the spoon along the trajectory shown by the arrow in FIG. 17E, the food in the spoon collides with the member 473b at a relatively low position (FIG. 17B). As a result, when the tip of the spoon is pulled upward along the tab 473, a relatively large amount of food falls onto the dish portion, and a small amount of food remains on the spoon.

On the other hand, when the automatic feeder 470 moves the spoon along the trajectory shown by the arrow in Figure 17G, the food in the spoon collides with member 473b at a relatively high position (Figure 17D). As a result, when the tip of the spoon is pulled upward along tab 473, the degree of contact between the food and tab 473 becomes relatively small, so less food falls onto the plate and more food remains on the spoon.

Also, when the automatic feeder 470 moves the spoon along the trajectory shown by the arrow in Figure 17F, an amount of food remaining on the spoon will be intermediate between the cases in Figures 17E and 17G.

In this way, by providing tabs 473 with different heights depending on the position, the amount of food per mouthful can be easily adjusted by adjusting the trajectory of the spoon, as shown in Figures 17E to 17G. For example, if the amount of food scooped into the spoon on the trajectory of Figure 17G is determined to be too much for one mouthful as a result of measuring it with a scale 474 or image processing, the amount can be easily reduced by changing the trajectory to Figures 17F or 17E. In this case, the action of scooping again after returning all the food already scooped into the spoon to the container 472 may be performed, but this is not essential. For example, by the simple action of moving the spoon along a new trajectory while keeping the food in the spoon, an appropriate amount of food will collide with the tab 473 and fall into the container 472, so it is also possible to fine-tune the direction of reducing the amount. As a result, it becomes possible to automatically supply an appropriate amount of food to the person being assisted. Note that here, an example in which tabs 473 with different heights depending on the position have been described, but the method of this embodiment is not limited to this. For example, a movable tab 473 may be provided, and the amount of food per mouthful may be adjusted by adjusting the angle or height of the tab 473.

When the automatic feeding device 470 is used, the operation log of the automatic feeding device 470 and the operation log of the terminal device 462 that controls the automatic feeding device 470 may be automatically recorded. In this way, various information can be automatically collected, such as the pace at which meals were eaten, what situations occurred during the meal of the person being assisted, how the situation was responded to, and whether the response was effective.

The amount of food consumed may also be automatically recorded. For example, in many cases, a barcoded note is provided with the meal in a nursing facility. Therefore, when the above-mentioned loop process is completed, a process of reading the barcode using a camera may be performed. The camera here may be the camera 475 provided on the arm 471, or the camera of the terminal device 462. This allows the target assisted person to be identified. As described above, the amount of food consumed is calculated as the total amount of food scooped up with a spoon. Since it is known which dish in the container 472 contains which food, the content of the food consumed can also be identified based on the operation log of the arm 471. It may also be possible to determine which dish the food was consumed on using an RFID or the like, and the process of identifying the content of the food consumed can be modified in various ways. Therefore, in this embodiment, a process of storing the amount of food consumed and the content of the food consumed in association with the assisted person corresponding to the barcode may be performed. In this embodiment, it may be possible to select the order in which the multiple dishes are eaten. For example, the automatic feeder 470 may be controlled to rotate through each dish in order, or to feed a specific dish intensively. When automatically recording food intake, the order in which food is consumed may also be recorded.

In addition, when assisting with meals, there may be cases where a small number of caregivers (one person in the narrow sense) assist multiple people being assisted at the same time. In this case, each person may be provided with a choking detection device 460, and information regarding swallowing and choking may be monitored. As described above, an automatic feeder 470 may be used for people being assisted who are having difficulty eating. In this case, there is no prohibition on the choking detection devices 460 and automatic feeders 470 used by multiple people being assisted operating independently. However, for multiple people being assisted by the same caregiver, multiple choking detection devices 460 and automatic feeders 470 may operate in conjunction with each other.

For example, assume that N swallowing choking detection devices 460 and N automatic feeders 470 are used by N persons receiving care (N is an integer of 2 or greater). In this case, one of the N swallowing choking detection devices 460 may operate as a master, and the other N-1 may operate as slaves. For example, the slave swallowing choking detection device 460 may transmit the detection results of swallowing or choking to the master swallowing choking detection device 460. The master terminal device 462 may then perform a process of displaying the output of all N swallowing choking detection devices 460 at once.

18 is an example of a screen displayed on the display unit of the master terminal device 462. In the example of FIG. 18, information on six people being assisted is displayed, and the information on each person includes the audio waveform output from the throat microphone 461 and the result of the choking judgment as to whether the person being assisted is normal, has a warning, or is abnormal. The display manner of the judgment result is arbitrary, but for example, as shown in FIG. 18, the judgment result may be displayed using the lighting/unlighting of three buttons corresponding to normal, warning, and abnormal, respectively. Normal, for example, indicates a state in which choking has not been detected. Caution, for example, indicates that choking has been detected but the degree of danger is low. Abnormal, indicates that choking with a high degree of danger has been detected. In this way, it is possible to display information on multiple people being assisted in an easy-to-understand manner. In particular, in the example of FIG. 18, the amount of information per person being assisted is limited, so the caregiver can easily determine whether or not intervention is necessary.

It may also be possible to control N automatic supply devices 470 using the master terminal device 462. For example, as shown in FIG. 18, a stop button for emergency stopping the automatic supply device 470 may be displayed in the display area for each assisted person. When the stop button is selected, the master terminal device 462 stops the operation of the automatic supply device 470 used by the assisted person. For example, when the stop button of an assisted person using the master terminal device 462 is pressed, the terminal device 462 directly outputs a control signal to stop the automatic supply device 470. When the stop button of an assisted person using a terminal device 462 other than the master is pressed, the master terminal device 462 transmits information instructing the slave terminal device 462 used by the assisted person to stop, and the slave terminal device 462 controls the target automatic supply device 470 to stop. The master choking detection device 460 (terminal device 462) and the slave choking detection device 460 (terminal device 462) may be directly connected or may be connected via the server system 100.

In addition, rather than one of the multiple choking detection devices 460 operating as a master, the caregiver's terminal device 200 may control the multiple choking detection devices 460. For example, the screen shown in FIG. 18 may be displayed on the display unit 240 of the caregiver's terminal device 200. In addition, when any of the stop buttons is selected, the terminal device 200 may output a signal to the target terminal device 462, either directly or via the server system 100, instructing the target terminal device 462 to stop the automatic supply device 470.

As can be seen from the above explanation, the processing content of the choking detection device 460 changes depending on whether or not the automatic supply device 470 is used. In other words, the operation mode of the choking detection device 460 changes when it is determined that the primary cause is the first core cause. In this manner, in this embodiment, the operation mode of the device used for assistance may be changed based on the estimation result of the primary cause.

For example, the information processing device 20 (e.g., the server system 100) includes a communication unit 130 that is used to assist the person being assisted and communicates with a device that operates in one of a number of operation modes. The device here may be, for example, the sensing device 400 or another device. The communication unit 130 may then transmit information indicating the main cause estimated by the cause estimation unit 112 to the device as information for determining in which of a number of operation modes the device will operate. In this way, the operation mode of each device can be automatically changed based on the estimation result of the main cause. As a result, at least a portion of the response to peripheral symptoms can be automated, further reducing the burden on the caregiver.

For example, the device here may include a first device that operates in multiple operation modes including a first mode that performs normal processing to detect at least one of swallowing and choking during a meal by the person being assisted, and a second mode that performs processing to automatically bring food to the mouth of the person being assisted in addition to the normal processing. The first device here is, for example, a choking detection device 460. When it is estimated that the main cause is the first core cause, the communication unit of the information processing device 20 (for example, the communication unit 130 of the server system 100) transmits information indicating the main cause to the first device as information instructing a transition from the first mode to the second mode. When the first device receives information indicating that the main cause is the first core cause from the information processing device 20, it changes the operation mode from the first mode to the second mode. In this way, it is possible to realize not only monitoring of choking but also automation of meals. Furthermore, since the change of operation mode is automated, it is possible to reduce the burden on the caregiver.

2.4 Data transmission from server system to assistance device As described above with the choking detection device 460 as an example, the sensing device 400 in this embodiment includes multiple operation modes and may operate in any of the operation modes. For example, the sensing device 400 may include multiple applications each of which executes a process corresponding to the tacit knowledge of an expert, and the operation mode may be changed by controlling the activation/inactivation of each application. In other words, the tacit knowledge to be used is switched by switching the operation mode.

For example, each application may be a trained model generated using machine learning. Here, machine learning refers to learning based on training data in which input data (sensing data) is associated with correct answer data corresponding to the input data. The correct answer data is data input by an expert, and is, for example, information that represents the expert's judgment result in providing care. In this way, it is possible to appropriately digitize the tacit knowledge of an expert. Since the tacit knowledge of an expert is digitized, even a less skilled caregiver can provide care similar to that of an expert. For example, the processes in each sensing device 400 described above with reference to Figures 6 to 10 may each correspond to tacit knowledge. In addition, a device capable of performing multiple processes corresponding to tacit knowledge may be a device different from the sensing device 400.

For example, the switching of tacit knowledge may be performed based on the peripheral symptom score or the estimation result of the main cause as described above. In this way, the tacit knowledge to be used can be switched according to the dementia state of the person being assisted, so that assistance appropriate for dealing with peripheral symptoms and preventing their occurrence can be provided.

The switching of tacit knowledge may be performed using the ability information of the person being assisted. The ability information here is information that indicates the activity ability of the person being assisted, and is, for example, information obtained as a result of the sensing device 400 performing processing using some kind of tacit knowledge. The ability information may be the same information as the input data related to the first core factor, or may be different information. The ability information may be, for example, an index value that indicates the degree of ADL, or may be information related to the level of risk that may occur in assistance. The risks here include various risks such as the risk of falling over, the risk of falling from a wheelchair or bed, the risk of aspiration related to aspiration pneumonia, and the risk of bedsores. For example, when a specific risk is low, there is little need to use tacit knowledge to respond to that risk. For example, when the risk of aspiration is low, there is little need to use tacit knowledge to determine the above-mentioned dangerous choking. Conversely, when a specific risk is high, it is desirable to actively use tacit knowledge to respond to that risk. Therefore, by considering the ability information, it is possible to appropriately switch tacit knowledge.

Scene information that identifies the scene of assistance for the person being assisted may also be used to switch between tacit knowledge. The scene information may be information that identifies the type of assistance to be performed, such as meal assistance, excretion assistance, or movement/transfer assistance. The scene information may also be information about the assistants providing assistance to the person being assisted, such as the number of assistants providing assistance to the person being assisted and their level of expertise. The scene information may also be information about the person being assisted, such as the attributes of the person being assisted. In this way, it becomes possible to use tacit knowledge that is appropriate for the type of assistance, the situation of the assistant (whether they can respond to risks with ease, etc.), and the attributes of the person being assisted.

In addition, device type information indicating the type of sensing device 400 used in combination may be used to switch between tacit knowledge. The device type here indicates a rough classification such as wheelchair, bed, etc., and may be information that does not distinguish between vendors. In this way, the tacit knowledge to be used can be determined taking into account the sensing devices 400 that are used in combination. For example, it becomes possible to use tacit knowledge that is suitable for cooperation between multiple sensing devices 400.

Considering these points, it is desirable for the server system 100 of this embodiment to notify each sensing device 400 of capability information, scene information, device type information, and the estimation result of the main cause as information for identifying the operation mode. Note that, although an example of notifying the estimation result of the main cause is described below, the peripheral symptom score may also be notified. Furthermore, both the peripheral symptom score and the estimation result of the main cause may also be notified.

However, the sensing devices 400 described above may be products of various vendors. In that case, if the data structure of the data transmitting the above-mentioned capability information, etc. differs for each vendor, the processing load on the server system 100 will be large and the versatility of communication control will be reduced. Therefore, the server system 100 of this embodiment may unify the data structure regardless of the vendor, model number, type, etc. of the sensing device 400. For example, the server system 100 standardizes the bit allocation of the MAC frame used in the data link layer for communication with the sensing device 400, regardless of the sensing device 400 of the communication destination.

Figure 19 is an example of a MAC frame format. Note that the format example shown in Figure 19 can also be applied to management frames and control frames, but the following explanation uses a data frame as an example. As shown in Figure 19, a MAC frame includes a MAC header, a frame body, and a trailer.

The MAC header includes the following fields: Frame Control, Duration ID, Address 1, Address 2, Address 3, Sequence Control, Address 4, QoS Control, and HT Control. Some of these fields may be omitted.

Frame Control includes a type field to determine whether the target MAC frame is a data frame, management frame, or control frame. Frame Control may also include a subtype field to specify a more specific type. Duration/ID stores information indicating the planned period for which radio waves will be used. The planned period for which radio waves will be used can be rephrased as the time required to transmit a frame. Duration/ID is used for RTS (Request To Send)/CTS (Clear to Send), etc.

Address 1 through Address 4 store information that indicates the addresses of the receiving and sending devices. For example, Address 1 corresponds to the receiving address, and Address 2 corresponds to the sending address. Address 3 and Address 4 store data according to the frame's purpose.

Sequence Control corresponds to the sequence number of the data to be transmitted. QoS Control stores information used for QoS control. QoS control refers to control that takes into account the priority of a frame when transmitting. HT Control is a field used, for example, in management frames.

The structure of the frame body will be described later with reference to Figures 20A and 20B. The trailer is, for example, a Frame Check Sequence (FCS). The FCS is information used to detect errors in the frame, and is, for example, a checksum code. The FSC is, for example, a Cyclic Redundancy Code (CRC).

Figures 20A and 20B are diagrams illustrating an example of bit allocation in the frame body of a data frame transmitted by the server system 100. As shown in Figure 20A, the frame body may include the following fields: User ADL, scene flag, device type ID, primary factor, data type ID, Instruction length, and contents.

User ADL is a field that stores the ability information of the person being assisted. For example, the ability information is numerical data that indicates which level the ability of the person being assisted belongs to when the degree of ability is divided into a predetermined number of levels. For example, if there are eight levels or less, User ADL is a three-bit field. If there are nine levels or more, User ADL may be a four-bit or more field. Since the number of bits required is known according to the definition of the ability information, User ADL may be a fixed-length field. Furthermore, the ability information in this embodiment is not limited to the index value of ADL, and may be more detailed information such as information indicating how to stand up, ability to maintain a sitting position, ability to swallow, and ability to walk. Therefore, User ADL may be a field having a number of bits that can express each of these abilities. User ADL may also include an ID that identifies the person being assisted.

The scene flag is a field that stores scene information. For example, the scene information may include a bit that indicates whether the number of caregivers is greater than or equal to a predetermined number. When the bit is a first value (e.g., 0), the number of caregivers is greater than or equal to a predetermined number, and when the bit is a second value (e.g., 1), the number is less than the predetermined number. The scene information may also include a bit that specifies the type of care. For example, when the type of care is identified as four types, namely, meal care, excretion care, moving/transferring care, and others, the scene information includes two bits that specify the type of care. For example, when the two bits are 00, it indicates meal care, when they are 01, it indicates excretion care, when they are 10, it indicates moving/transferring care, and when they are 11, it indicates others. The scene information is not limited to these, and other information may be used. Therefore, the specific number of bits and meaning of the scene flag can be modified in various ways. However, since the type of scene information to be used is known, and the number of bits required to express the scene information is also known, the scene flag may be a fixed-length field.

The device type ID is a field that stores device type information. The device type information may be different for each sensing device 400 described above with reference to Figures 6 to 10. Alternatively, if the sensing device 400 corresponds to the risk of falling, the same device type ID may be assigned to the imaging device 410 in Figure 6 and the bedside sensor 420 in Figure 7. Alternatively, the device type ID may be assigned based on the type of sensor that the sensing device 400 has. Since the number of device type IDs to be targeted is known, the device type ID may be a fixed-length field.

The primary factor is a field that represents the estimation result of the primary factor. For example, if the primary factor is selected from environmental factors, first core factors, second core factors, and psychological factors, the primary factor may be two-bit data that identifies these four. However, as described above, the factor estimation unit 112 may estimate more specific factors that subdivide the above four factors, and the data structure of the primary factor can be modified in various ways.

Although a primary factor is exemplified here, a field that stores information indicating the presence or absence of peripheral symptoms may be used instead of the primary factor field. For example, a peripheral symptom score may be stored in this field.

The data type ID, instruction length, and contents shown in FIG. 20A are fields used to send a control signal to the controlled device. The controlled device here may be the terminal device 200 used by the caregiver, the sensing device 400, or another device used for the response represented by the response information. The data type ID is a field that stores information indicating the type of instruction output to the controlled device. The instructions here may include four types: "notification (alarm)", "movement/transport", "control", and "recommendation, etc." In this case, the data type ID is a 2-bit fixed-length field that can identify the four instructions.

"Notification" is information used when the controlled device notifies the processing result of the sensing device 400. For example, "notification" may be an instruction used when a fall risk is detected by a device that judges the risk of falling and the terminal device 200 notifies the user of the risk. "Movement/transportation" is an instruction to move a movable controlled device such as a reclining wheelchair or a walker. For example, the instruction to move/transport may be used to control the moving of a walker or the like closer to the target person being assisted so that the person being assisted can be caught when a fall risk is detected. "Control" broadly includes control other than "movement/transportation" that operates the controlled device, and includes changing the angle of the back part of a reclining wheelchair or changing the bottom angle of a nursing bed. "Recommendation, etc." includes, for example, recommendations such as the purchase of a product used to improve the quality of care. For example, the recommendation may represent an instruction to output a decision to recommend the use of a cushion on the bed 610 or wheelchair 630 to the controlled device, which is the terminal device 200 of the caregiver. "Recommendations, etc." may also include news distribution, etc. For example, "recommendations, etc." may be used to introduce popular sensing devices 400.

The contents is a field that stores information that specifies the specific content of the instructions. For example, the contents field stores information such as the notification content in "notification", the current position and target position in "movement/transport", the controlled area and drive amount in "control", and the recommended product in "recommendation". The length of the contents field varies depending on the data type ID and the specific content of the instructions. Therefore, as shown in FIG. 20A, the frame body may include an instruction length field before the contents field that stores the length of the contents field. Since the maximum length of the contents field is considered to be known, the instruction length may be, for example, a fixed-length field.

Although the present embodiment has been described in detail above, those skilled in the art will easily understand that many modifications are possible that do not substantially deviate from the novel matters and effects of the present embodiment. Therefore, all such modifications are intended to be included in the scope of the present disclosure. For example, a term described at least once in the specification or drawings together with a different term having a broader or similar meaning may be replaced with that different term anywhere in the specification or drawings. All combinations of the present embodiment and modifications are also included in the scope of the present disclosure. Furthermore, the configurations and operations of the information processing system, information processing device, server system, terminal device, sensing device, etc. are not limited to those described in the present embodiment, and various modifications are possible.

10...information processing system, 20...information processing device, 21...acquisition unit, 22...factor estimation unit, 100...server system, 110...processing unit, 111...acquisition unit, 112...factor estimation unit, 113...handling decision unit, 114...communication processing unit, 115...learning unit, 116...presentation processing unit, 120...storage unit, 121...user information, 122...device information, 123...log data, 124...learned model, 125...handling table, 130...communication unit, 200...terminal device, 210...processing unit, 220...storage unit, 230...communication unit, 240...display unit, 250...operation unit, 300...management terminal device, 400...sensing device, 410...imaging device, 420...bedside sensor, 430...detection device, 440...seat sensor, 441...cushion, 442...control box, 450...incontinence detection device, 460...choking detection device, 461...throat microphone, 462...terminal device, 470...automatic supply device, 471...arm, 472...container, 473...tab, 473a...surface, 473b...component, 474...scale, 475...camera, 510...reclining wheelchair, 520...nursing bed, 610...bed, 620...mattress, 630...wheelchair, IM1...output image, Se1 to Se4...pressure sensor

Claims (20)

an acquisition unit that acquires input data including sensing data acquired using a sensor in a sensing device having a plurality of operation modes ;
a factor estimation unit that estimates, based on the input data, which of a plurality of factors is the main factor that caused the person being assisted by a caregiver to enter a peripheral symptom state in which peripheral symptoms of dementia are observed, including an environmental factor caused by the surrounding environment of the person being assisted, a first core factor caused by an executive function disorder of the person being assisted, a second core factor caused by a disorientation of the person being assisted, and a psychological factor caused by the psychology of the person being assisted;
A communication unit that communicates with the sensing device;
Including,
The factor estimation unit is
a trained model generated by a learning process based on training data in which correct answer data for identifying the main cause of the peripheral symptom is associated with the input data, the trained model taking the input data as an input and outputting factor scores representing the contribution of each of the environmental factors, the first core factors, the second core factors, and the psychological factors to the peripheral symptom state, estimating the main cause based on the trained model;
The communication unit is
An information processing device that outputs an estimation result of the main cause to the sensing device, thereby switching the sensing device to operate in one of the plurality of operation modes . In claim 1,
The factor estimation unit is
An information processing device that, based on the input data which is time series data, obtains a degree of fluctuation in the factor scores by comparing the factor scores at two different times for each of the environmental factors, the first core factors, the second core factors, and the psychological factors, and estimates the main factors based on the degree of fluctuation in the factor scores. In claim 1,
The information processing device further includes a measure determination unit that determines a measure to be taken for the peripheral symptom state based on information in which the main cause estimated by the factor estimation unit and a recommended measure is associated with the main cause . In claim 3,
The handling determining unit,
An information processing device that, when it is estimated that the main cause is a psychological factor, performs processing to determine that the measure is to identify related parties who have a high degree of influence on the psychology of the person being assisted. In claim 3,
The input data is
The data on the environmental factors includes information for identifying a season,
The factor estimation unit is
determining whether the behavioral and mental symptoms are more likely to occur in a particular season than in other seasons, thereby determining whether the main factor is a seasonal factor caused by the particular season among the environmental factors;
The handling determining unit,
The information processing device differentiates the countermeasure when the main factor is estimated to be a seasonal factor from the countermeasure when the main factor is estimated to be an environmental factor other than the seasonal factor. In any one of claims 1 to 5 ,
The input data is
An information processing device that includes ability information representing the ability of the person being assisted in daily activities as data related to the first core factor. In any one of claims 1 to 5 ,
The input data is
An information processing device that includes wandering information indicating the wandering of the person being assisted as data relating to the second core factor. In any one of claims 1 to 5 ,
The input data is
An information processing device that includes biometric information of people related to the person being assisted as data related to the psychological factors. In claim 2,
The factor estimation unit is
When the input data includes a plurality of sensing data of different types,
performing a first determination for determining the main cause based on first input data that is a part of the plurality of sensing data ;
If the primary cause is not estimated in the first judgment, an information processing device performs a second judgment to determine the primary cause based on second input data including data among the plurality of sensing data that is not included in the first input data. In claim 9 ,
a degree of association between each of the plurality of sensing data and the environmental factor, the first core factor, the second core factor, and the psychological factor is set; and
When data among the plurality of sensing data is determined to have a degree of association biased toward any one of the environmental factors, the first core factors, the second core factors, and the psychological factors by a predetermined amount or more, the data is set as unique input data,
The second input data includes at least one unique input data that is not included in the first input data. In any one of claims 1 to 5 ,
The trained model is generated by a learning process based on training data in which the input data is associated with correct answer data including information identifying the main cause of the peripheral symptom and information indicating whether the person being assisted is in the peripheral symptom state;
The factor estimation unit is
An information processing device that estimates whether the person being assisted is in the peripheral symptom state based on the input data and the trained model . In claim 11 ,
The communication unit is
A device is used to assist the person being assisted, and communicates with the device operating in any one of a plurality of operation modes ;
An information processing device that transmits at least one of information indicating whether or not the peripheral symptom state estimated by the factor estimation unit is present and information indicating the main cause to the device as information for determining in which of the multiple operating modes the device will operate. In claim 12 ,
The device comprises:
a first device that operates in a plurality of operation modes including a first mode that performs a normal process to detect at least one of swallowing and choking during a meal by the person being assisted, and a second mode that performs a process to automatically carry food to the mouth of the person being assisted in addition to the normal process;
The communication unit is
an information processing apparatus that, when it is estimated that the main cause is the first core cause, transmits information representing the main cause to the first device as information instructing a transition from the first mode to the second mode. In claim 2,
The trained model is generated by a learning process based on training data in which the input data is associated with correct answer data including information identifying the main cause of the peripheral symptom and information indicating whether the person being assisted is in the peripheral symptom state;
The factor estimation unit is
An information processing device that uses the input data as input and determines the main factors based on the learned model that outputs a peripheral symptom score that indicates the likelihood that the person being assisted is in the peripheral symptom state, and the factor scores for each of the multiple factors. In claim 14 ,
An information processing device including a learning unit that, when a presentation process of factor information representing the main cause is performed, performs a process of updating the learned model based on the factor information, a first peripheral symptom score which is the peripheral symptom score before the presentation of the factor information, and a second peripheral symptom score which is the peripheral symptom score after the presentation of the factor information. In claim 15 ,
The method further includes a countermeasure determination unit that determines a countermeasure for the peripheral symptom state based on information in which the main cause estimated by the factor estimation unit and a countermeasure recommended for the main cause are associated with each other ,
The learning unit is
When the presentation process of the cause information and the handling information representing the handling is performed , an information processing device performs a process of updating the learned model by determining whether the peripheral symptom state of the person being assisted has improved after presentation of the handling information, based on the first peripheral symptom score and the second peripheral symptom score. 6. The information processing device according to claim 1,
The communication unit is
An information processing device that transmits a data frame including a MAC (Media Access Control) header, a frame body, and a trailer in a data link layer for communication with the sensing device, wherein the frame body includes a fixed-length area including a first area for storing an estimation result of the main cause. 6. The information processing device according to claim 1,
The communication unit is
An information processing device that switches in which of the multiple operating modes the sensing device operates by outputting to the sensing device ability information indicating the activity ability of the person being assisted, scene information identifying the assistance scene of the person being assisted, and device type information indicating the type of device to be used in conjunction with the sensing device. 20. The information processing device according to claim 18,
The communication unit is
An information processing device that transmits a data frame including a Media Access Control (MAC) header, a frame body, and a trailer in a data link layer for communication with the sensing device, wherein the frame body includes a fixed-length area including a first area for storing an estimation result of the main cause, a second area for storing the capability information, a third area for storing the scene information, and a fourth area for storing the device type information. An information processing device,
acquiring input data including sensing data acquired using a sensor in a sensing device having a plurality of operation modes ;
Based on the input data, estimate which of a number of factors is the main cause of the person being assisted by a caregiver becoming in a peripheral symptom state in which peripheral symptoms of dementia are observed, including an environmental factor caused by the surrounding environment of the person being assisted, a first core factor caused by an executive function disorder of the person being assisted, a second core factor caused by a disorientation of the person being assisted, and a psychological factor caused by the psychology of the person being assisted ;
outputting the estimation result of the main cause to the sensing device, thereby switching the operation mode of the sensing device to one of the plurality of operation modes;
In estimating the main factors
a trained model generated by a learning process based on training data in which correct answer data for identifying the main cause of the peripheral symptom is associated with the input data, the trained model taking the input data as an input and outputting factor scores representing the contribution of each of the environmental factors, the first core factors, the second core factors, and the psychological factors to the peripheral symptom state, and estimating the main cause based on the trained model;
Information processing methods.