JP7797178B2 - Life support system and life support method - Google Patents

Description

The present invention relates to a life assistance system and a life assistance method.

By measuring the physical activity and health status of elderly people receiving care during their daily lives, appropriate interventions such as rehabilitation can be implemented at the appropriate time, for example, when they become unwell and have difficulty moving their legs. However, elderly people may feel burdened or uncomfortable when wearing wearable sensors on their bodies to directly measure their activity.
Therefore, a method has been proposed for detecting health conditions while reducing the burden on the subject by placing a device that measures physical activity in the living space where the subject is present.

A common method is to install cameras or sensors on the ceiling or walls of the room where the subject lives to capture images and distance images, and capture the movements of the subject as they are active indoors.
However, the method of installing cameras and sensors indoors has the following problems.
- Only people who are in the room where the camera is installed and within the camera's field of view can be measured.
- From a privacy standpoint, there is a reluctance to install cameras in private living spaces that are not public spaces.
- Measuring physical activity throughout the home requires installing cameras and sensors in each room, which is costly.

A method has been proposed in which a mobile object such as a robot is equipped with a camera that can be moved, allowing the physical activity of people to be captured regardless of location. The mobile object uses non-contact sensors to detect the presence of people, predicts the person's next action, estimates the range of that action, and identifies obstacles. The mobile object then sets a travel route that avoids the recognized obstacles, thereby avoiding collisions with people or walls.

Patent Document 1 describes a mobile training support device in which a running body having a driving means is provided with a detection means for detecting the posture of a user undergoing training such as walking.
Patent document 2 describes a robotic measuring instrument that is equipped with a three-dimensional spatial information sensor on its running section that takes images using optical means, and that continuously measures the movement of the entire body of the person being measured, including the limbs, by estimating the correspondence between the collected three-dimensional spatial coordinates and each part of the person being measured from the two-dimensional images through computational processing.
Patent Document 3 describes an autonomous robot that has an imaging unit that captures images of its surroundings and maintains a natural sense of distance from an object in accordance with the size of the captured object.

JP 2016-73630 A JP 2016-80671 A International Publication No. 2018/047802

Family members and care managers who care for elderly people and other measurement subjects often do not want to constantly monitor the elderly person's daily life, but rather would like to be alerted only at appropriate times (when their health condition deteriorates or is likely to deteriorate) so that they can intervene appropriately, such as by visiting the hospital. In other words, there is a need for a well-balanced monitoring service that reduces the burden of caregiving by minimizing alerts under normal circumstances, but supports intervention by issuing alerts when necessary.

Furthermore, unlike a medical checkup at a hospital, when measuring subjects in their living space, the subjects are engaged in various daily activities such as sleeping, eating, etc. Therefore, there are situations where it may be inappropriate to immediately issue an alert indicating a decline in the subject's physical ability just because the subject's walking speed has decreased compared to yesterday.
For example, when comparing walking speeds at the same time of 10:00 a.m., if today's walking speed during a break is slower than yesterday's walking speed while cleaning, this is simply because daily activities were different, and in this case an alert should not be issued.

The present invention was made in consideration of these circumstances, and its main objective is to improve the accuracy of alerts that notify of abnormalities in a subject when monitoring the subject in a living space.

In order to solve the above problems, the life assistance system of the present invention has the following features.
The present invention provides a mobile object that moves to the vicinity of a resident, calculates a feature of the physical activity of the resident and the living behavior of the resident from a measurement result including location information of the resident obtained from a measurement sensor provided in the mobile object, and transmits the calculation result to a server;
the server compares, for each received living behavior of the resident, past physical activities of the same living behavior with the current physical activity to determine a change from a feature amount of the past physical activity to a feature amount of the current physical activity, and transmits an alert to an information terminal in response to the change ;
The moving body is
a distance sensor that uses electromagnetic waves or sound waves emitted from the moving object to determine a distance to the resident and a moving speed of the resident along the direction of irradiation of the electromagnetic waves or sound waves as a feature of the physical activity;
The measuring sensor of the moving body is characterized by having an image sensor that determines the resident's movement speed perpendicular to the direction of irradiation of the electromagnetic waves or sound waves as a characteristic quantity of the physical activity based on the change in the resident's position over time as seen in an image of the resident .
Other means will be described later.

This invention improves the accuracy of alerts that notify of abnormalities in a subject when monitoring the subject in a living space.

1 is a schematic diagram of a life assistance system according to a first embodiment. FIG. 2 is a plan view of the house of FIG. 1 according to the first embodiment. FIG. 2 is a diagram showing a display screen of the information terminal according to the first embodiment. FIG. 2 is a schematic diagram of a moving body of the life assistance system according to the first embodiment. FIG. 1 is a block diagram of a life assistance system according to a first embodiment. 10 is a flowchart illustrating an example of a process performed by a moving body according to the first embodiment. FIG. 10 is an explanatory diagram illustrating an example of feature amounts related to the first embodiment. 8 is an example of a flowchart showing a process of the server following the process of FIG. 7 according to the first embodiment. FIG. 10 is a plan view of a house according to a second embodiment. FIG. 10 is a plan view of the same house as FIG. 9 according to the second embodiment, but with a different charging station location. 10 is an example of a flowchart of a moving body according to the second embodiment. FIG. 10 is a schematic diagram of a life assistance system according to a third embodiment. 11 is a schedule graph showing the daily activities of a resident according to Example 3. FIG. 11 is a block diagram illustrating an example of a life assistance system according to a third embodiment. 10 is an example of a flowchart of a life assistance system according to a third embodiment. 16 is an example of a flowchart of the server following the process of FIG. 15 according to the third embodiment.

Each embodiment of the present invention will be described in detail below with reference to the drawings.

1 is a schematic diagram of a life assistance system 1 according to a first embodiment. In the first embodiment, an example will be described in which a resident 11 is monitored while a mobile object 2 is moving.
The life assistance system 1 includes a mobile object 2 and a server 3 .
The mobile object 2 moves close enough to the resident 11 so that it can observe the resident 11, and calculates the physical activity characteristics 110 (Figure 7) of the resident 11 and the daily activities of the resident 11 from the measurement results including the location information of the resident 11 obtained from the measurement sensors (image sensor 21, distance sensor 22) equipped on the mobile object 2, and transmits the calculation results to the server 3.
For each received daily living activity of the resident 11, the server 3 calculates the change from the past physical activity characteristic 110 to the current physical activity characteristic 110, and transmits an alert or the physical activity characteristic 110 to the information terminal 5 in accordance with the change.
The daily activities include, for example, sleeping, going out, resting, eating, and doing laundry, as will be described later in Fig. 13. The physical activity feature 110 includes, for example, the activity amount, energy (energy consumption), sharpness of movement, time (time required), and number of times for the resident 11, as will be described later in the radar chart 52 of Fig. 3.

The mobile object 2 and the server 3 are connected wirelessly or by wire so as to be able to communicate with each other.
1, the server 3 is installed outside the residence 10. On the other hand, as long as the server 3 can communicate with the mobile object 2, the installation location of the server 3 may be within the operating company of the life support service, the operating company of the cloud service, or within the residence 10.
The mobile object 2 is installed in a house 10 in which a resident 11 resides, and can move freely and autonomously within the house 10 by driving the drive wheels 23. Autonomous movement means that the mobile object 2 moves along a movement path that is autonomously calculated by a computer such as the mobile object 2 or the server 3, rather than being directly commanded by a human such as the resident 11 with an operation signal such as moving forward or turning left.

The mobile object 2 is equipped with an image sensor 21 and a distance sensor 22. The image sensor 21 acquires a visible light image of the resident 11.
The distance sensor 22 can measure the distance 221 between the mobile object 2 and the resident 11 by emitting radio waves 220 such as microwaves or millimeter waves from the mobile object 2 to the outside. The distance sensor 22 may emit electromagnetic waves (radio waves 220, laser light, infrared light, etc.) or sound waves (ultrasound waves) to measure the distance to the target object.
This allows the resident 11 to have his or her physical activity measured without having to wear anything.

The information terminal 5 receives an alert about the physical activity of the resident 11 and the feature amount 110 of the physical activity from the server 3, and displays the alert and the feature amount 110 of the physical activity to the user of the information terminal 5 as follows.
・Resident 11 himself/herself (for example, elderly people, children living alone to attend school, preschoolers, elementary, junior high, and high school students, etc.)
Family members (resident family members 5A) who watch over the resident 11. The resident family members 5A are, for example, adult children of the resident 11 who live together with or separately from the resident 11.
・Resident 11's care manager ・A person related to the nursing care facility that has a contract with Resident 11

Therefore, the information terminal 5 is configured as a device that can receive the following information in the server 3, for example.
- A smartphone that communicates with the server 3 and displays the internal information of the server 3 - A terminal such as a tablet or PC - An image display device such as a television or monitor - A device in a monitoring system that watches over residents - A device in a system that manages medical and nursing care information, etc.

This allows the user of the information terminal 5 to detect changes or abnormalities in the physical activity or daily behavior of the resident 11. Furthermore, by sending alert information to the information terminal 5 of a system that oversees medical and nursing care information, the physical information of the resident 11 and related health information can be shared with a wide range of stakeholders, such as medical professionals and nursing care workers. In this case, in addition to detecting the amount of change and issuing an alert, necessary information from the physical activity feature amount 110 transmitted by the mobile object 2 may also be transmitted to the information terminal 5 or other systems.

FIG. 2 is a plan view of the house 10 of FIG.
The map data 301 of the house 10 shown in this floor plan may be registered in advance in the mobile body 2, or may be created by measuring the shape of the interior of the house 10 using the image sensor 21 and distance sensor 22 while the mobile body 2 travels within the house.
This map data 301 is used not only to set a route for the moving object 2 so that it does not collide with walls or furniture, but also to identify the actions of the resident 11 from their location within the house 10, such as when the resident 11 is eating in the kitchen.

FIG. 3 is a diagram showing the display screen of the information terminal 5. As shown in FIG.
The information terminal 5 receives and displays information stored in the server 3, thereby informing the resident's family 5A and others of the state of the resident 11.
The display screen of the information terminal 5 displays time series data 51 that displays characteristic quantities measured within the house 10, such as the activity level of the resident 11, in a time series graph, and a radar chart 52 for comparing the balance between multiple characteristic quantities measured from the resident 11.
Since the server 3 stores information that has been transmitted by the mobile body 2 up to now, the server 3 may display the results of a comparison between past information and current information on the information terminal 5, such as time series data 51.

On the other hand, the server 3 may display, as detailed data at a certain time, a total of seven (heptagonal) feature amounts such as the amount of activity (maximum) and energy (total) as shown in a radar chart 52 in FIG. 3 . For this purpose, the mobile object 2 calculates, for example, the following information and transmits the calculation results to the server 3:
The shape of the room in the house 10 The position of the moving object 2 in the house 10 The distance 221 between the moving object 2 and the resident 11
Position and posture of the resident 11 Skeletal model of the resident 11 Physical activity feature quantities 110 (details in FIG. 7 ) extracted from the skeletal model of the resident 11, such as walking speed, movement speed of limbs, movement speed of specific points on the body, and angle of limbs relative to the trunk Physical activity index (sharpness of movement, amount of activity, etc.) calculated using a specific formula from the physical activity feature quantities 110 of the resident 11
Location of the resident 11 Living activities of the resident 11 (times spent sleeping, going out, etc., and number of meals, etc.; see FIG. 13 for details)

FIG. 4 is a schematic diagram of a moving body of the life assistance system 1. As shown in FIG.
The moving body 2 is equipped with an image sensor 21 , a distance sensor 22 , drive wheels 23 , and a control board 24 .
4, the distance sensor 22 is mounted on the front surface of the main body of the moving body 2 and is configured to measure the distance ahead. On the other hand, the position of the distance sensor 22 is not limited to the front surface, but may be mounted on the top surface or in all directions, or multiple distance sensors may be mounted.
4, the image sensor 21 is mounted on the top surface of the main body of the moving object 2. However, the position of the image sensor 21 is not limited to the top surface, and it may be mounted on a side surface, or multiple sensors may be mounted. The angle of view that can be photographed is arbitrary, but the wider the angle, the wider the range of image information that can be acquired with one image sensor 21.

It is desirable to use a mobile object 2 designed for use within a house 10 as described below.
- The size of a typical robot vacuum cleaner (long diameter φ250-350mm) that can move freely around the house
- Weight is up to 5 kg - Does not generate noise - Operates wirelessly - Wheels and legs that do not damage the floors of an ordinary home On the other hand, outdoor mobile bodies that weigh more than 30 kg and mobile bodies for buildings, offices, and facilities that are larger than φ500 mm are not suitable for following occupants 11 who move frequently, but these mobile bodies may also be used.

Furthermore, the mobile object 2 may be equipped with sensors other than the image sensor 21 and the distance sensor 22, and may have at least one of the following functions: cleaning function, air purification function, pet function, security function, sound recognition function, and monitoring function.
For example, the mobile object 2 may be provided with functions other than movement and sensing, such as a household robot such as (Action 1) or (Action 2) below.
(Operation 1) The moving object 2 is operated as a robot vacuum cleaner. A robot vacuum cleaner is equipped with a fan, brushes, etc. inside the main body, and can collect and clean dust accumulated on the floor. Furthermore, if the robot vacuum cleaner is equipped with a dust sensor that can detect fine particles, it can detect the amount of dust and clean with an appropriate airflow.

(Operation 2) The moving object 2 is operated as a pet robot. The appearance of the pet robot is made to resemble a pet that people like, such as a small dog. If the pet robot is equipped with an audio sensor to acquire sound information, it can move in the direction of the sound in response to a voice calling the pet.
In this way, if a robot vacuum cleaner or pet robot has the configuration of the moving body 2, physical activity can be measured as needed while the robot vacuum cleaner or pet robot is normally operated according to its original functions and roles.

Furthermore, if the mobile object 2 has other functions such as a robot vacuum cleaner or a pet robot, it can measure physical activity as needed while performing its original functions of cleaning the house 10 and being loved by the resident 11. Therefore, the mobile object 2 can obtain physical activity and related health information without interfering with the resident 11's life or invading it more than necessary.
Furthermore, small mobile objects 2 such as robot vacuum cleaners and pet robots are much smaller than the camera-equipped mobile objects that have been common in the past, and so can be moved freely within the house 10, eliminating the need to prepare a large space for measuring physical activity.

In general, from the perspective of protecting privacy, it is often not desirable to have devices that collect images and sounds, such as cameras and microphones, permanently installed in the home 10 where the resident 11 lives. Therefore, by measuring the resident's physical activity using a mobile object 2 equipped with an image sensor 21 and a distance sensor 22, sensors such as the image sensor 21 only approach when necessary, so that the physical activity of the resident 11 is measured without causing discomfort to the resident 11, and the health condition of the resident 11 can be understood from the physical activity of the resident 11.

Furthermore, by using the mobile object 2, which is used as a home appliance or the like in the house 10, to monitor the resident 11, the following effects can be obtained.
- There is no need to prepare a separate environment to measure the behavior of subjects, such as a large space in a public facility hall or a senior care facility where elderly people receive mobility training.
A general-purpose mobile robot measuring instrument has difficulty moving over steps and other obstacles within the house 10. On the other hand, the mobile body 2, which is designed in advance to move within the house 10, can move smoothly within the house 10 and easily maintain an appropriate distance to measure (photograph) the entire body of the resident 11.
By using the mobile object 2 that has previously coexisted in the living space, the psychological burden on the resident 11 of being monitored can be reduced.

FIG. 5 is a block diagram of the life assistance system 1.
The mobile object 2 is equipped with an image sensor 21 , a distance sensor 22 , and drive wheels 23 , and detects the state of the occupant 11 using the image sensor 21 and the distance sensor 22 while moving on the drive wheels 23 .
The control board 24 in the moving body 2 includes a control unit 41 , a communication unit 42 , and a storage unit 43 .

The control unit 41 performs the following control using information from the image sensor 21 and the distance sensor 22.
The drive wheels 23 are driven to move the moving body 2.
The control unit 41 estimates the shape of the rooms in the house 10 and the position of the mobile object 2 itself within the house 10. Then, the control unit 41 stores the results of detecting the shape of the rooms in the house 10 in the memory unit 43 as the map data 301 in FIG. 2 .
If it is not the first time the mobile object 2 has been moved into the room, the mobile object 2 moves in accordance with the shape of the room by retrieving the map data 301 learned up to that point from the memory unit 43 and using it.
Calculates the feature quantities expressing the physical activity of the resident 11 and the daily activities of the resident 11, as explained in the radar chart 52 of FIG. 3, and transmits the results to the server 3 via the communication unit 42.

The server 3 includes a control unit 31 , a communication unit 32 , and a storage unit 33 .
The communication unit 32 can exchange information by communicating with the mobile object 2 or with an external device (e.g., information terminal 5) outside the life assistance system 1. The communication unit 32 receives information transmitted from the mobile object 2, and the control unit 31 adds time information to the information received via the communication unit 32 and stores the information in the storage unit 33.
The control unit 31 retrieves the information stored in the memory unit 33, processes it together with the information received from the mobile object 2 via the communication unit 32, and outputs the processing result (e.g., time series data 51 in Figure 3) to the outside (e.g., information terminal 5) via the communication unit 32.

The mobile object 2 and the server 3 are each configured as a computer having a CPU, RAM, ROM, HDD, communication I/F, input/output I/F, and media I/F.
The communication I/F is connected to an external communication device. The input/output I/F is connected to an input/output device. The media I/F reads and writes data from a recording medium. Furthermore, the CPU controls each processing unit by executing a program (also called an application or its abbreviation, "app") loaded into RAM. This program can also be distributed via a communication line or recorded on a recording medium such as a CD-ROM and distributed.

FIG. 6 is an example of a flowchart showing the processing of the moving body 2.
The mobile object 2 checks whether the memory unit 43 contains the map data 301 of the interior of the house 10 shown in Fig. 2 (S111). If the map data 301 is not present, the control unit 41 creates map data 301 of the interior of the house while the mobile object 2 travels within the house 10 (S113) and stores the created map data 301 in the memory unit 43. In the process of generating the map data 301, the shapes of the walls, furniture, etc. within the house 10 are measured using the image sensor 21 and distance sensor 22.
The mobile object 2 travels freely within the house 10 using the map data 301 created in S113 or the map data 301 previously stored in the storage unit 43 (S112).

Next, the mobile object 2 determines whether it is in a state where it can start measurement, such as when it detects the resident 11 (S114). If the resident 11 is in a state that the resident 11 dislikes, such as when he or she is changing clothes, the mobile object 2 may receive a voice or gesture notification from the resident 11, whose physical activity feature 110 it is measuring, that measurement is not possible, and may stop measuring the physical activity feature 110 (No in S114), and return to S112.
In addition, if multiple residents 11 (such as family members living with the elderly person or caregivers caring for the elderly person) are detected in S114, the detected resident 11 may be identified from a facial image or the like to determine whether or not the resident 11 is a person to be monitored, and measurement may be started if a person to be monitored is detected (Yes in S114).

In this way, when it becomes possible to start measurement (Yes in S114), the moving object 2 acquires the distance 221 between the resident 11 and the moving object 2 and the own position of the moving object 2 (S121). Here, the method of detecting the resident 11 may be to recognize the shape of a person using the image sensor 21, or to detect an object that does not fit the shape of the room using the distance sensor 22.
If the distance 221 in S121 is shorter than the specified value, the moving body 2 is too close to the resident 11 (Yes in S122), so the control unit 41 drives the drive wheels 23 to move the moving body 2 so that the distance 221 is longer than the specified value (S126).

Here, the specified value in S126 is the maximum distance at which the image sensor 21 can acquire the feature amount of the resident 11 (described later), and if the distance 221 is shorter than the specified value, the feature amount cannot be acquired. Since the specified value differs depending on the installation position of the image sensor 21, the angle of view that can be captured, and the feature amount to be extracted, it is necessary to determine the specified value in advance according to the conditions of the image sensor 21 and the feature amount.
In Figure 4, a wide-angle image sensor 21 is installed on the mobile body 2, and can acquire an image of the resident 11 in front of it. Furthermore, since the physical activity feature 110 is calculated using the speed of the limbs as described below, a specified value is set as the distance at which the image sensor 21 can capture an image of the entire body of the resident 11, including the tips of the limbs, and this value is stored in advance in the memory unit 43.
As a result, the mobile object 2 moves to a position where the entire body of the resident 11 to be measured is within the imaging range of the image sensor 21, and then captures an image of the resident 11 from that position.

When the moving object 2 moves to a position where the distance 221 is greater than the specified value (No in S122 or after execution of S126), the moving object 2 acquires the following information (S123).
The image sensor 21 acquires a photographed image of the resident 11. The distance sensor 22 acquires the distance 221 between the resident 11 and the mobile object 2. The map data 301 acquires the self-position of the mobile object 2.

Thereafter, the control unit 41 of the mobile object 2 calculates the feature amount of the resident 11 as described with reference to the radar chart 52 in FIG. 3 from the image, distance, and self-position acquired in S123 (S124).
7 is an explanatory diagram showing an example of the feature amount calculated in S 124. The control unit 41 calculates the feature amount 110 of the physical activity of the resident 11, such as a skeleton model extracted from the captured image.
The mobile object 2 transmits the feature values calculated in S124 to the server 3 via the communication unit 42 (S125). In this way, compared to the case where the image itself acquired in S123 is directly transmitted to the server 3, the amount of data communicated between the mobile object 2 and the server 3 can be reduced by transmitting feature values extracted (compressed) from the image.

FIG. 8 is an example of a flowchart showing the process of the server 3 subsequent to the process of FIG.
The server 3 receives the feature information of S125 from the mobile object 2 via the communication unit 32 (S201), and the control unit 31 adds time information to the received feature information and stores it in the storage unit 33. The control unit 31 also compares the latest feature information with past feature information already stored in the storage unit 33, and calculates the amount of change from the past feature information (S202).
In this way, since the server 3 performs the data storage process of S201 and the data comparison process of S202 instead of the mobile object 2, there is no need to provide high processing performance or large data storage capacity within the mobile object 2. Therefore, compared to when all processing is performed by the mobile object 2, the performance of the mobile object 2 can be minimized, thereby making it possible to make the mobile object 2 smaller and less expensive.

If the amount of change calculated in S202 is greater than the specified value (Yes in S203), it is determined that the amount of change from the past is large, and alert information is sent to the external information terminal 5 via the communication unit 32 (S204).
The specified value in S203 is set to a value that is a certain amount lower than the past feature amount, as exemplified below.
- When the walking speed, which is one of the currently acquired features, is more than 20% (= specified value) lower than the walking speed measured several months ago. - When the value of the physical activity index calculated from the walking speed and limb movement speed is lower than the specified value. The alert information of S204 is, for example, an alert that the physical activity of resident 11 is decreasing, or an alert that an abnormality has occurred in resident 11.

FIG. 9 is a plan view of the house 10.
In Example 2, the mobile object 2 is fixed and used like a surveillance camera. The following description will focus on the differences from Example 1. In Example 2, an example will be described in which the mobile object 2 is a robot vacuum cleaner. The mobile object 2 moves autonomously when necessary, and when not moving, waits and charges at the charging station 12 within the house 10.

In the plan view of the house 10, the horizontal direction of the plane in FIG. 9 is the X axis, the vertical direction of the plane is the Y axis, and the height direction from the ceiling to the floor is the Z axis.
The charging station 12 is installed at a distance equal to or greater than a specified value in a direction substantially perpendicular to the resident movement path 111 along which the resident 11 moves (Y-axis direction).
Here, the resident movement trajectory 111 is the movement trajectory of an area in which the resident 11 moves in one direction (the X-axis direction in Figure 9), such as near an opening or closing door such as the front door of the house 10, or in a corridor within a room.
In a large space, the resident 11 moves freely and not in a specific direction, so the direction of movement is not fixed, but in a corridor or an opening/closing door, the direction of movement is fixed to one direction, namely the direction of the corridor or the direction of passing through the opening/closing door. Therefore, it is assumed that the resident 11 moves along the resident movement trajectory 111.

This arrangement of the mobile object 2 allows the radio waves 220 to reach the resident movement trajectory 111 and measure the entire body of the resident 11. Therefore, the mobile object 2 waiting at the charging station 12 continues to transmit radio waves 220 toward the resident movement trajectory 111 (in the Y-axis direction). The mobile object 2 then measures physical activity data from the resident 11 using images captured by the image sensor 21 mounted on the mobile object 2 and the radio waves 220 emitted from the distance sensor 22.

The movement of the occupant 11 in the X-axis direction and the Z-axis direction along the occupant movement trajectory 111 is measured from the images captured by the image sensor 21 .
The movement of the resident 11 in a direction perpendicular to the resident movement trajectory 111 (Y-axis direction) is determined from the change in the distance between the mobile object 2 and the resident 11 detected by the radio waves 220 of the distance sensor 22 .
The control board 24 can measure the movement speed of the occupant 11 with high accuracy by dividing the movement distance of the occupant 11 in each direction (X, Y, Z) by the movement time.

Fig. 10 is a plan view of the same house 10 as Fig. 9. The position of the charging station 12 is different between Fig. 9 and Fig. 10.
The charging station 12 is installed on the mobile object 2 at a certain distance on an extension line in the same direction (X-axis direction) as the resident movement trajectory 111 along which the resident 11 moves.
The movement of the occupant 11 in the direction perpendicular to the occupant movement trajectory 111 (Y-axis direction) and in the Z-axis direction is measured from the images captured by the image sensor 21.
The movement of the resident 11 in the X-axis direction along the resident movement trajectory 111 is determined from the change in the distance between the mobile object 2 and the resident 11 detected by the radio waves 220 of the distance sensor 22 .
The control board 24 can measure the movement speed of the resident 11 with high accuracy by dividing the movement distance in each direction (X, Y, Z) of the resident 11 by the movement time. Also, because there is a certain distance between the moving object 2 and the resident movement trajectory 111, it is possible to measure the entire body of the resident 11 and measure physical activity without the moving object 2 moving away from the resident 11 to create a distance.

As explained above in Figures 9 and 10, the image sensor 21 determines the movement speed of the resident 11 perpendicular to the direction of irradiation of the radio waves 220 as a feature 110 of physical activity based on the change over time in the position of the resident 11 shown in the image captured of the resident 11.
The distance sensor 22 uses radio waves 220 emitted from the mobile object 2 to determine the distance to the resident 11 and the moving speed of the resident 11 along the direction of the radio waves 220 as the feature amount 110 of physical activity.

FIG. 11 is an example of a flowchart for the moving body 2.
If the mobile object 2 is not moving autonomously, it charges at the charging station 12 (S131). The mobile object 2, which is stationary at the charging station 12, determines to start measurement when it receives a measurement instruction or detects that the resident 11 is moving along the resident movement trajectory 111 (Yes in S114B). In this case, the mobile object 2 acquires an image of the resident 11 with the image sensor 21 and acquires the distance to the resident 11 with the distance sensor 22 (S123B). Note that because the mobile object 2 is not moving, there are no errors such as blurring of the captured image, and the physical activity of the resident 11 can be measured more easily than in the first embodiment.

Here, the measurement instruction may be any method, such as the following examples.
(Method 1) When it is estimated from the resident's daily life that the resident 11 moves along the resident movement trajectory 111, a measurement instruction is received from the server 3 at the time of movement. For example, when the charging station 12 is installed in a position where the hallway leading to the entrance is visible as shown in FIGS. 9 and 10 , the movement time of the resident 11 is the time when the resident returns home or leaves the house 10.
(Method 2) If it is expected that the resident 11 will soon return home to the house 10 based on location data from the smartphone he or she has, a measurement instruction is received from the server 3 at the expected time of return.
(Method 3) Receive measurement instructions from a door opening/closing sensor or another sensor installed at the entrance.

Then, the control board 24 of the mobile object 2 calculates the feature amount 110 of the physical activity of the resident 11 from the information acquired in S123B (S124), and transmits the calculated information to the server 3 (S125). Meanwhile, because the location of the charging station 12 is known, the process of the mobile object 2 acquiring its own location can be omitted in S123B, unlike S123 in Fig. 6 .
As described above, in the second embodiment, the mobile object 2 measures the physical activity of the resident 11 without moving while installed at the charging station 12. This makes it possible to measure the physical activity and health condition of the resident 11 in the same way as in the first embodiment, and also makes it easier to measure the physical activity.

FIG. 12 is a schematic diagram of a life assistance system 1 according to a third embodiment.
In Example 3, a non-contact sensor 6 and a home appliance 7 that are installed in a house 10 and that calculate a feature amount 110 of physical activity are added to the configuration of Example 1 or Example 2. Note that it is not necessary to have both the non-contact sensor 6 and the home appliance 7; either one is sufficient. The non-contact sensor 6 and the home appliance 7 can each be connected to the server 3.

The non-contact sensor 6 detects the position and posture of the resident 11 by emitting radio waves 60 such as millimeter waves or microwaves to the resident 11, and transmits the detection results to the server 3. The non-contact sensor 6 is installed, for example, on a wall or ceiling surface inside the house 10, and therefore can measure physical activity more precisely than when measured by the mobile object 2. The number of non-contact sensors 6 is not limited to one as illustrated in FIG. 12 , and multiple non-contact sensors 6 may be provided inside the house 10.
Instead of the non-contact sensor 6, the following sensor may be used.
・Humanity sensor that detects whether or not an occupant is present by emitting infrared light ・Door opening/closing sensor that detects whether a door is open or closed ・Vibration sensor that detects vibrations

Furthermore, by installing the non-contact sensor 6 at a higher position on the wall of the house 10 or on the ceiling, physical information of the resident 11 can be obtained without the view being obstructed by furniture or other installed objects. On the other hand, if the resident 11 is located in a blind spot from the installation position of the non-contact sensor 6, it may not be possible to obtain the physical information of the resident 11. Therefore, by having the mobile object 2 obtain the physical information of the resident 11 in a blind spot, more physical information of the resident 11 can be obtained by both the mobile object 2 and the non-contact sensor 6.

The home appliance 7 can indirectly acquire physical activity of the resident 11 by transmitting operation information of the home appliance when the resident 11 uses (operates) the home appliance to the server 3. Note that although a washing machine is shown in Fig. 12, the type and number of home appliances are arbitrary.
For example, instead of a washing machine, a home appliance 7 such as a refrigerator, a conventional vacuum cleaner held by the resident 11, a robot vacuum cleaner, or a cooking appliance may be used. The operation information of the home appliance 7 may include, for example, whether the door is open or closed and the temperature inside the refrigerator, the washing mode and washing time of the washing machine, and the cooking menu and temperature inside the cooking appliance.

FIG. 13 is a schedule graph showing the daily activities of the resident 11.
The vertical axis of the schedule graph 112 represents the day of the week (Monday to Sunday), and the horizontal axis represents the daily activities of the resident 11 from morning to night on that day. Daily activities are classified into, for example, sleeping, going out, resting, eating, making a choice, etc., and the resident 11 performs one of the daily activities at each time.
The server 3 estimates daily living activities at each time, such as the schedule graph 112, based on information acquired from at least one of the mobile object 2 and the home appliance 7. For example, the server 3 estimates the current daily living activity (eating) of the resident 11 from information acquired from the home appliance 7 (e.g., that the resident 11 is currently operating a cooking appliance).

From such estimated daily living activities at each time, the server 3 obtains what daily living activities are performed each day for each day of the week, and creates a schedule graph 112 .
The schedule graph 112 is transmitted from the server 3 to the information terminal 5 and is viewed by the resident family 5A, so that the resident family 5A can understand the lifestyle habits of the resident 11 without directly monitoring the resident 11.

The server 3 identifies the living activity as exemplified below.
- Obtain "button operation information" and "door opening/closing information" from the washing machine and combine this with hot water supply to estimate whether the resident 11 is taking a bath.
- Obtain "door opening/closing information" from the refrigerator, and also obtain "button operation information" and "door opening/closing information" from the cooking appliance, and combine both to estimate the meals of the resident 11.
- Obtain "operation" and "movement information" from the vacuum cleaner and estimate the cleaning status of the resident 11.

FIG. 14 is an example of a block diagram of the life assistance system 1.
The life assistance system 1 of the third embodiment includes a mobile object 2, a server 3, a non-contact sensor 6, and a home appliance 7. The configurations of the mobile object 2 and the server 3 are the same as those of the first embodiment (FIG. 5).

The non-contact sensor 6 includes a detection unit 61 , a control unit 62 , and a communication unit 63 .
The detection unit 61 acquires physical information of the resident 11 by irradiating the resident 11 with radio waves 60. Alternatively, the detection unit 61 acquires physical information of the resident 11 by irradiating the resident 11 with light (laser) from LiDAR (Light Detection And Ranging) instead of the radio waves 60 and performing a 3D scan.
The physical information of the resident 11 is, for example, the resident's position and posture obtained as point cloud data (distance data), and the resident's activity data obtained from the time change of the point cloud data (such as the speed of the point cloud).
The control unit 62 extracts feature quantities indicative of the physical activity of the resident 11 from the results acquired by the detection unit 61, and transmits the extracted feature quantity results to the server 3 via the communication unit 63. As with the information acquired by the mobile object 2 in Example 1, the feature quantities include the resident's position, movement speed, posture, and movement speed of the limbs, as well as various numerical values of physical activity calculated from this information.

The home appliance 7 includes a detection unit 71 , a control unit 72 , and a communication unit 73 .
The detection unit 71 acquires resident information 70 such as operation information when the resident 11 operates the home appliance and environmental information of the house 10. The environmental information of the house 10 is, for example, information about the environment of the home appliance 7, such as the temperature, humidity, and air pressure around and inside the home appliance 7, the water pressure of the tap water, and the voltage of the grid power.
Similar to the control unit 62, the control unit 72 extracts feature quantities indicative of the physical activity of the resident 11 from the results obtained by the detection unit 71, and transmits the extracted feature quantity results to the server 3 via the communication unit 73. The feature quantities transmitted here include operation information of the home appliances, and therefore, information can be obtained such as when and what home appliance was operated, when which home appliance was operated, under what conditions, and in what mode.

FIG. 15 is an example of a flowchart of the life assistance system 1 (the moving object 2, the non-contact sensor 6, and the home appliance 7) according to the third embodiment.
When control is started, the mobile object 2 moves freely within the house (S112) in the same manner as in the first embodiment (FIG. 6). Meanwhile, the non-contact sensor 6 and the home appliance 7 enter a standby state to wait for an operation (S141).

When the non-contact sensor 6 detects the resident 11, or when the resident 11 operates the home appliance 7 (Yes in S142), the non-contact sensor 6 acquires information about the resident 11, or the home appliance 7 acquires operation information (S143).
If the non-contact sensor 6 detects the resident 11, the position of the resident 11 is known, and if the home appliance 7 is operated, the position of the resident 11 is known from the position of the home appliance 7, and therefore, detection information of the resident 11 is transmitted to the mobile object 2 based on the position (S143B). In the transmission process of S143B, the information may be transmitted directly from the non-contact sensor 6 or the home appliance 7 to the mobile object 2, or the information may be transmitted via the server 3.

The mobile object 2 travels to the location of the resident 11 notified in S143B (S112B) and starts measuring the physical information of the resident 11 (Yes in S114). The subsequent processes (S123 to S125) executed by the mobile object 2 are the same as those in the first embodiment (FIG. 6).
Furthermore, the non-contact sensor 6 and the home appliance 7 calculate the necessary features from the physical information of the resident 11 acquired in S143 (S144, similar to S124 in Figure 6), and transmit the calculation results to the server 3 (S145, similar to S125 in Figure 6).

FIG. 16 is an example of a flowchart of the server 3 following the process of FIG.
The server 3 receives the information notified in S125 from the mobile object 2 (S201), and receives the information notified in S145 from the non-contact sensor 6 and the home appliance 7 (S201B). Examples of the received information are shown below.
- Information on the presence of the resident 11 - Activity amount of the resident 11 (mainly received from the non-contact sensor 6)
- Posture of the resident 11 - Position of the resident 11 (position of center of gravity, position of hands and feet) In the case of an electrical appliance 7, whether or not the resident 11 is present at the location of the electrical appliance.
- Speed of the resident 11 (walking speed, movement speed of limbs, movement speed of center of gravity)
- Values calculated from the collection of information listed above - Detection of the behavior of the resident 11 (location, what they are doing)

The server 3 synthesizes (combines) the information of S201 and the information of S201B to calculate the characteristic amount of the resident 11 as exemplified below (S201C).
Energy consumption of resident 11 (= walking speed + limb movement speed)
- Sharpness of resident 11's behavior (obtained by frequency analysis of walking speed)
- Daily activities of the resident 11 (eating, resting, etc.) Furthermore, if the daily activities of the resident 11 are specific daily activities, by combining image information of the mobile object 2, it is possible to measure physical activity during certain daily activities that would be difficult to obtain using only the mobile object 2. For example, it is possible to measure the posture of the resident 11 while cooking, or the walking speed of the resident 11 when going back and forth between the bedroom and the toilet at night.
- Amount of activity and movement speed for each daily activity of the resident 11 - Amount of activity and movement speed for each location of the resident 11 (obtained for each of the entrance, kitchen, living room, bedroom, hallway, and toilet)

When combining the information from S201 and the information from S201B, if there is no overlap between the two pieces of information, one of the pieces of information is adopted, and if there is overlap between the two pieces of information, the more accurate piece of information is adopted. For example, if location information for a resident 11 at the same time is received from both the mobile object 2 and the non-contact sensor 6, the information source closest to the resident 11 should be adopted.

The server 3 saves the current information synthesized in S201C and compares the current information with past information to calculate the amount of change in the feature of the resident 11 (S202). In the comparison process of S202, the accuracy of the comparison is improved by comparing physical activities of the same daily activities (for example, physical activity during current laundry and physical activity during past laundry). Examples of changes or amounts of change compared in S202 are as follows, allowing for more precise acquisition of health information of the resident 11.
- Amount of change in walking speed and walking posture - Amount of change in walking speed, walking posture, and physical activity index for a specific daily activity - A change in daily activity at the same time on the current day and in the past For example, if a person had a meal at 7:00 AM on the past day but was sleeping at 7:00 AM on the current day, the difference between eating and sleeping will be recognized as an abnormality (poor health) of the resident 11.

The server 3 determines whether the amount of change calculated in S202 is greater than a specified value (predetermined threshold) (S203). Alternatively, the server 3 may determine in S203 whether a change was detected in S202.
If S203 returns Yes (amount of change > predetermined threshold, or there is a change), the server 3 notifies the resident family 5A of a possible abnormality in the resident 11 by issuing an alert to the information terminal 5 (S204). The alert is information indicating a change, such as, for example, that a certain behavior is usually performed at a certain time on a certain day of the week, but that behavior was not detected today.

Furthermore, the server 3 may instruct the mobile object 2 to check the location of the abnormality, such as where the resident 11 has fallen, and the mobile object 2 may notify the resident 11 with a voice message such as "Are you OK?". This allows the mobile object 2 to receive a detailed voice response from the resident 11, such as "I think I sprained my ankle," and notify the resident's family 5A of this as a more detailed alert.

The life support system 1 of Examples 1 to 3 described above includes an image sensor 21 that acquires images, a distance sensor 22 that acquires the distance to an object, a mobile body 2 that estimates its own position indoors, and a server 3 that can communicate with the mobile body 2.
The mobile body 2 then processes the physical activity of the resident 11 acquired by the on-board sensor to obtain physical activity characteristics 110, which are then transmitted to the server 3, and the server 3 stores the physical activity characteristics 110.

Furthermore, the server 3 detects the feature 110 of the physical activity of the resident 11 to be measured, and if there is a large change from the previously saved feature 110 of the physical activity, it notifies the information terminal 5 of this fact as an alert. This alert can improve accuracy by being the result of comparing the feature 110 of the same daily activity between the past and present.
This allows the physical activity and health status of the resident 11 in the house 10 to be measured, and if any changes occur in that status, the resident 11 and those involved can be notified of the change, thereby encouraging appropriate intervention and maintaining and improving the health status of the resident 11.

It should be noted that the present invention is not limited to the above-described embodiment, and includes various modifications. For example, the above-described embodiment has been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to an embodiment having all of the described configurations.
Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment.
Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations. Furthermore, the above-mentioned configurations, functions, processing units, processing means, etc. may be partly or entirely realized in hardware, for example, by designing them as integrated circuits.
Furthermore, each of the above-mentioned configurations and functions may be realized by software, with a processor interpreting and executing a program that realizes each function.

Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a storage medium such as an IC (Integrated Circuit) card, an SD card, or a DVD (Digital Versatile Disc).The cloud can also be used.
In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected.
Furthermore, the communication means connecting the devices is not limited to a wireless LAN, but may be changed to a wired LAN or other communication means.

REFERENCE SIGNS LIST 1 Life support system 2 Mobile object 3 Server 5 Information terminal 5A Resident family 6 Non-contact sensor 7 Home appliance 10 House 11 Resident 12 Charging station 21 Image sensor (measurement sensor)
22 Distance sensor (measurement sensor)
23 Drive wheel 24 Control board 31 Control unit 32 Communication unit 33 Memory unit 41 Control unit 42 Communication unit 43 Memory unit 51 Time series data 52 Radar chart 60 Radio wave 61 Detection unit 62 Control unit 63 Communication unit 70 Resident information 71 Detection unit 72 Control unit 73 Communication unit

Claims (8)

a mobile object that moves to the vicinity of a resident, calculates a feature of the physical activity of the resident and the living behavior of the resident from a measurement result including location information of the resident obtained from a measurement sensor provided in the mobile object, and transmits the calculation result to a server;
the server compares, for each received living behavior of the resident, past physical activities of the same living behavior with the current physical activity to determine a change from a feature amount of the past physical activity to a feature amount of the current physical activity, and transmits an alert to an information terminal in response to the change ;
The moving body is
a distance sensor that uses electromagnetic waves or sound waves emitted from the moving object to determine a distance to the resident and a moving speed of the resident along the direction of irradiation of the electromagnetic waves or sound waves as a feature of the physical activity;
an image sensor that determines, as a feature of the physical activity, the moving speed of the resident perpendicular to the irradiation direction of the electromagnetic waves or sound waves, based on a time change in the position of the resident shown in an image of the resident, and The moving body moves to a position where the whole body of the resident to be measured is within the imaging range of the image sensor, and then the resident is photographed from that position.
The life assistance system according to claim 1 . 2. The life assistance system according to claim 1, wherein the mobile object stops measuring the characteristic amount of the physical activity when the mobile object is notified by voice or gesture that measurement is impossible from the resident who is measuring the characteristic amount of the physical activity. 2. The life assistance system according to claim 1, wherein the mobile body calculates, as a characteristic of the physical activity of the resident, at least one of a set of the resident's presence information, the resident's position, the resident's posture, the movement speed of the resident's center of gravity, the movement speed of the resident's limbs, and the resident's activity amount, and a value calculated from the set. The life support system according to claim 1, wherein the information terminal is a terminal used by at least one of the resident, the resident's family, the resident's care manager, and a person associated with a nursing care facility under contract with the resident, and receives and displays, in addition to an alert, the characteristic amount of the resident's physical activity. The life assistance system according to claim 1 , wherein the mobile object further has at least one function selected from the group consisting of a cleaning function, an air purification function, a petting function, a security function, a sound recognition function, and a monitoring function. The life assistance system according to claim 1 , wherein the server receives the feature of the physical activity calculated from at least one of a non-contact sensor installed in a home and a home appliance installed in the home. The life assistance system includes a mobile object and a server,
The moving body is
a distance sensor that uses electromagnetic waves or sound waves emitted from the moving object to determine a distance to the resident and a moving speed of the resident along the direction of irradiation of the electromagnetic waves or sound waves as a feature of physical activity;
an image sensor that determines a moving speed of the resident perpendicular to the direction of irradiation of the electromagnetic waves or sound waves as a feature of the physical activity based on a time change in the position of the resident shown in an image of the resident; and
The mobile object moves to the vicinity of the resident, calculates a feature of the physical activity of the resident and a living behavior of the resident from a measurement result including location information of the resident obtained from the measurement sensor provided in the mobile object, and transmits the calculation result to the server;
The server, for each received daily living activity of the resident, compares the past physical activity of the resident with the current physical activity of the resident for the same daily living activity, thereby determining changes from the feature amount of the past physical activity to the feature amount of the current physical activity, and transmits an alert to an information terminal in response to the changes.