JP7487735B2 - METHOD, PROGRAM AND INFORMATION PROCESSING APPARATUS FOR DISPLAYING INFORMATION - Google Patents

Description

This disclosure relates to displaying information about a monitored subject.

Various technologies have been proposed for systems that display information about monitored subjects. For example, Japanese Patent Application Laid-Open No. 2017-202058 (Patent Document 1) discloses, in FIG. 6 and other figures, an information processing device that displays a "status window" for displaying information about multiple monitored subjects, and a pop-up window on top of the status window that displays information about a monitored subject whose status change has been detected. The pop-up window is displayed in a position that does not overlap with the status icon displayed in the status window, or in a position that partially overlaps with the status icon to the extent that the status icon is visible.

JP 2017-202058 A

However, conventional technology has not considered displaying a status icon of a monitored target whose status has changed after a pop-up window is displayed. If the status of a monitored target changes after a pop-up window is displayed, there is a possibility that the status icon of that monitored target may not be visible due to the pop-up window that is already displayed. In this sense, conventional technology cannot be said to be able to reliably display changes in the status of multiple monitored targets when displaying detailed information of a target whose status has changed.

This disclosure has been devised in light of the current situation, and its purpose is to provide technology for reliably displaying changes in the status of multiple monitored targets while displaying detailed information about some of the monitored targets.

According to one aspect of the present disclosure, there is provided a method for displaying information of a plurality of monitored targets, the method comprising the steps of: displaying on a display a first area representing information of each of the plurality of monitored targets; in response to an operation on the first area, displaying on the display a second area together with the first area representing additional information of the monitored targets corresponding to the operation; and in response to a change in the state of at least some of the plurality of monitored targets, updating the display on the display so that a change in the first area corresponding to the change is visible, regardless of the display of the second area.

The step of updating the display may include displaying a third area on the display representing information about a monitored subject whose status has changed among the multiple monitored subjects.

The third region may be located outside the first region and the second region.
The third region may be located within the second region.

The step of updating the display may include moving the second area so that it does not overlap any part of the first area corresponding to the monitored subject whose status had changed before the second area was displayed.

The step of updating the display may include moving the second area so that it does not overlap a portion of the first area corresponding to a monitored subject whose status had changed before the second area was displayed.

The step of updating the display may include reducing the first area.

The step of displaying the second region may include displaying the second region so that at least a portion of the first region overlaps with the second region. The step of updating the display may include reducing the first region when the second region cannot be displayed in a portion of the first region other than a portion corresponding to a monitoring target whose state has changed before and after the second region is displayed.

The step of displaying the second region may include displaying the second region outside the first region.

The method may further include transmitting at least a portion of the additional information in the second area to the mobile device.

The method may further include accepting input of a comment for the second area. The step of transmitting at least a portion of the additional information in the second area to the mobile device may include transmitting the comment to the mobile device.

The method may further include a step of accepting input of information specifying the mobile terminal as the destination to which the additional information is to be sent.

The method may further comprise the step of removing the second region from the display. The step of removing the second region may involve magnifying the first region.

The step of removing the second area from the display may be performed under at least one of the conditions that at least a portion of the additional information in the second area has been transmitted to the mobile terminal and that a given time has elapsed since the display of the second area began.

The second area may display more detailed information about the target of monitoring than the first area.
According to another aspect of the present disclosure, there is provided a program that, when executed by one or more processors, causes the one or more processors to perform the method described above.

According to yet another aspect of the present disclosure, there is provided an information processing device comprising a memory storing the above program and one or more processors executing the program stored in the memory.

According to the present disclosure, in response to a change in the state of at least some of the multiple monitored targets, the display on the display is updated so that the changed portion in the first area is visible regardless of the display of the second area. While the changes in the state of the multiple monitored targets are reliably displayed, detailed information of some of the monitored targets is displayed in the second area.

FIG. 1 is a diagram illustrating an example of the configuration of a monitoring system. FIG. 13 is a diagram for explaining an example of a detection range of a sensor in a room. FIG. 2 is a diagram showing an example of the hardware configuration of the equipment in each room 10. FIG. 2 illustrates an example of a hardware configuration of a client terminal 200. FIG. 2 is a diagram illustrating an example of a hardware configuration of a mobile terminal 300. FIG. 4 illustrates an example of a hardware configuration of a management server 400. 13 is a flowchart of an example of a process for displaying information about a resident. FIG. 13 is a diagram showing an example of a list display of information on a plurality of residents 20. FIG. 13 is a diagram showing an example of a display form of a pop-up window. FIG. 11 is a diagram illustrating an example of updating a list display. FIG. 13 is a diagram showing an example of a screen after updating in step S60. FIG. 12 is a diagram showing an example of a screen after updating the screen of FIG. 11 . 10 is a diagram showing an example of a display mode when a change occurs in the list display from the state shown in FIG. 9 . FIG. 14 is a diagram showing an example of a screen after updating the screen of FIG. 13 . FIG. 13 is a diagram showing an example of a list display screen. FIG. 16 is a diagram showing an example of a screen updated from the state shown in FIG. 15 . 10 is a diagram showing an example of a display mode when a change occurs in the list display from the state shown in FIG. 9 . FIG. 18 is a diagram showing an example of a state after the display in FIG. 17 is updated. FIG. 16 is a diagram showing an example of a screen after updating from the state shown in FIG. 15 .

Below, one embodiment of the monitoring system will be described with reference to the drawings. In the following description, identical parts and components are given the same symbols. Their names and functions are also the same. Therefore, their description will not be repeated.

[1. Configuration of the monitoring system]
FIG. 1 is a diagram showing a schematic diagram of an example of the configuration of a monitoring system. The monitoring system manages information on two or more patients or residents in a facility such as a hospital or a welfare facility for the elderly. In this specification, the monitoring system is described as a system for managing information on multiple residents (targets of monitoring) in a welfare facility for the elderly, as an example. Note that the facility to which the monitoring system is applied may be another type of facility such as a hospital. The target of monitoring may be changed to a patient in the hospital, for example.

The monitoring system includes sensor boxes 100A, 100B, etc. installed in each resident's room. The monitoring system includes a management server 400 that manages the output from each of the sensor boxes 100A, 100B, etc. The management server 400 transfers the output from the sensor boxes 100A, 100B, etc. to two or more client terminals (client terminals 200A, 200B, etc.). Each of the two or more client terminals 200A, 200B, etc. displays the transferred output. In one example, the display is controlled using a web browser installed in each of the two or more client terminals 200A, 200B, etc. Each of the directors 820A, 820B, etc. looks at the display on each of the client terminals 200A, 200B, etc. and decides whether or not to dispatch staff to each room. The director may decide which staff to assign to which room.

The monitoring system includes mobile terminals 300A, 300B, etc. carried by each of staff members 850A, 850B, etc. When the director inputs information indicating that a staff member needs to rush to the room, each of client terminals 200A, 200B, etc. transmits a notification to at least one of mobile terminals 300A, 300B, etc.

When the mobile terminals 300A, 300B, etc. receive the notification, they each display the information contained in the notification. Each of the staff members 850A, 850B, etc. rushes to the room or the like and takes the necessary action, following the display on the mobile terminal they carry with them.

In this specification, when referring to common characteristics among the sensor boxes 100A, 100B..., client terminals 200A, 200B..., mobile terminals 300A, 300B..., staff members 850A, 850B..., and directors 820A, 820B..., the terms "sensor box 100," "client terminal 200," "mobile terminal 300," "staff member 850," and "director 820" are used.

[2. Acquiring information about the person being watched]
2 is a diagram for explaining an example of a detection range of a sensor in a room. In the example of FIG. 2, the sensor box 100 is installed on the ceiling CL of the room 10.

2, range AR generally represents the detection range of a sensor in sensor box 100. If the sensor is a Doppler sensor, the Doppler sensor detects behavior occurring within range AR. If the sensor is a camera, the camera captures images within range AR.

In the example of FIG. 2, a bed 30 is installed in a room 10, and a sensor in a sensor box 100 generates an output according to the behavior of a resident 20. The resident 20 can operate a care call handset 500. The operation contents of the care call handset 500 in each room 10 may be transmitted to the sensor box 100.

[3. Hardware configuration (equipment in the room)]
3 is a diagram showing an example of a hardware configuration of the equipment in each room 10. In each room, a sensor box 100, a care call handset 500, a door sensor 510, a toilet sensor 520, an odor sensor 530, and a vital sensor 540 are installed.

The sensor box 100 comprises a control device 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a communication interface 104, a camera 105, a Doppler sensor 106, a wireless communication device 107, and a memory device 120.

The control device 101 controls the sensor box 100. The control device 101 is, for example, configured with at least one integrated circuit. The integrated circuit is, for example, configured with at least one CPU (Central Processing Unit), at least one ASIC (Application Specific Integrated Circuit), at least one FPGA (Field Programmable Gate Array), or a combination of these.

An antenna (not shown) and the like are connected to the communication interface 104. The sensor box 100 exchanges data with an external communication device via the antenna. An example of an external communication device is the management server 400.

In one implementation, the camera 105 is a near-infrared camera. The near-infrared camera includes an IR (Infrared) projector that projects near-infrared light. By using the near-infrared camera, an image showing the interior of the room 10 can be captured even at night. In another implementation, the camera 105 is a surveillance camera that receives only visible light. In still another implementation, a 3D sensor or a thermography camera may be used as the camera 105. The camera 105 may be configured integrally with the housing of the sensor box 100, or may be configured separately.

The Doppler sensor 106 is, for example, a microwave Doppler sensor that emits and receives radio waves to detect the behavior (movement) of objects in the room 10. This allows the detection of biometric information of the resident 20 in the room 10. In one example, the Doppler sensor 106 emits microwaves in the 24 GHz band toward the bed 30 in each room 10 and receives the waves reflected by the resident 20. The reflected waves are Doppler shifted due to the movements of the resident 20. The Doppler sensor 106 can detect the breathing condition and heart rate of the resident 20 from the reflected waves.

The wireless communication device 107 receives signals from the care call handset 500, door sensor 510, toilet sensor 520, odor sensor 530, and vital sensor 540, and transmits the signals to the control device 101. For example, the care call handset 500 has a care call button 501, and when the care call button 501 is operated, it transmits a signal indicating that operation has occurred to the wireless communication device 107. The door sensor 510, toilet sensor 520, odor sensor 530, and vital sensor 540 each transmit their respective detection outputs to the wireless communication device 107.

The storage device 120 is, for example, a storage medium such as a hard disk or an external storage device. The storage device 120 stores the programs executed by the control device 101 and various data used to execute the programs. The various data may include behavioral information of the resident 20.

At least one of the above programs and data may be stored in a storage device other than the storage device 120 (for example, a memory area of the control device 101 (for example, a cache memory, etc.), ROM 102, RAM 103, an external device (for example, the management server 400, etc.)) as long as the storage device is accessible by the control device 101.

[4. Tenant behavior information]
The above behavioral information will be explained. The behavioral information is, for example, information that indicates that the resident 20 has performed a predetermined behavior. In one example, the predetermined behavior includes four behaviors: "getting up" that indicates that the resident 20 has woken up, "getting out of bed" that indicates that the resident 20 has left the bed, "falling down" that indicates that the resident 20 has fallen down from the bed, and "falling down" that indicates that the resident 20 has collapsed.

In one embodiment, the control device 101 generates behavioral information of the resident 20 associated with each room 10 based on images captured by a camera 105 installed in each room 10. The control device 101, for example, detects the head of the resident 20 from the images, and detects whether the resident 20 has "gotten up," "got out of bed," "fallen over," or "falled" based on the change over time in the size of the detected head of the resident 20. A specific example of generating behavioral information is described in more detail below.

First, the area in the living room 10 where the bed 30 is located, the first threshold value Th1, the second threshold value Th2, and the third threshold value Th3 are stored in the memory device 120. The first threshold value Th1 distinguishes the size of the resident's head between when the resident is in a recumbent position and when the resident is in a sitting position within the area where the bed 30 is located. The second threshold value Th2 distinguishes whether the resident is in a standing position or not based on the size of the resident's head within the living room 10 excluding the area where the bed 30 is located. The third threshold value Th3 distinguishes whether the resident is in a recumbent position or not based on the size of the resident's head within the living room RM excluding the area where the bed 30 is located.

The control device 101 extracts a moving object region from the target image as the area of the resident 20, for example by background subtraction or frame subtraction. The control device 101 further extracts the head region of the resident 20 from the extracted moving object region, for example by a circular or elliptical Hough transform, by pattern matching using a pre-prepared head model, and by using a threshold derived by a neural network trained for head detection. The control device 101 detects "getting up," "getting out of bed," "falling over," and "falling off" from the position and size of the extracted head.

The control device 101 may determine that the action "getting up" has occurred when it detects that the position of the head extracted as described above is within the location area of the bed 30 and that the size of the head extracted as described above has changed from the size of a lying position to the size of a sitting position by using the first threshold value Th1.

The control device 101 may determine that the action "getting out of bed" has occurred when the position of the head extracted as described above moves from within the area where the bed 30 is located to outside the area where the bed 30 is located, and when it detects that the size of the head extracted as described above has changed from a certain size to the size of a standing position by using the second threshold value Th2.

The control device 101 may determine that the behavior "fall" has occurred when the position of the head extracted as described above moves from within the area where the bed 30 is located to outside the area where the bed 30 is located, and when the size of the head extracted as described above changes over time from a certain size to the size of a lying position by using a third threshold value Th3.

The control device 101 may determine that the action "fall" has occurred if it detects that the position of the head extracted as described above is located within the room 10 excluding the area where the bed 30 is located, and that the size of the extracted head has changed from a certain size to the size of a lying position by using a third threshold value Th3.

As described above, in one specific example, the control device 101 of the sensor box 100 generates behavioral information of the resident 20. Note that in the monitoring system, other elements may generate behavioral information of the resident 20 using images of the inside of the room 10.

[5. Hardware Configuration (Client Terminal)]
4 is a diagram showing an example of a hardware configuration of the client terminal 200. The client terminal 200 includes a control device 201, a ROM 202, a RAM 203, a communication interface 204, a display interface 205, an operation interface 207, and a storage device 220.

The control device 201 controls the client terminal 200. The control device 201 is, for example, configured with at least one integrated circuit. The integrated circuit is, for example, configured with at least one CPU, at least one ASIC, at least one FPGA, or a combination of these.

An antenna (not shown) and the like are connected to the communication interface 204. The client terminal 200 exchanges data with an external communication device via the antenna. The external communication device includes, for example, the sensor box 100.

The display interface 205 is connected to the display 206 and sends an image signal to the display 206 to display an image in accordance with instructions from the control device 201 or the like.

The operation interface 207 is, for example, a USB (Universal Serial Bus) terminal, and is connected to the input device 209. The operation interface 207 receives a signal indicating a user operation from the input device 209. The input device 209 is, for example, a mouse, a keyboard, a touch panel, or any other device capable of receiving an input operation from a user.

The storage device 220 is, for example, a storage medium such as a hard disk or an external storage device. In one implementation, the storage device 220 stores a program executed by the control device 201, but the program may be stored in another storage device accessible to the control device 201.

[6. Hardware Configuration (Mobile Terminal)]
5 is a diagram showing an example of a hardware configuration of the mobile terminal 300. The mobile terminal 300 includes a control device 301, a RAM 303, a communication interface 304, a display 305, an input device 306, and an internal memory 320.

The control device 301 controls the mobile terminal 300. The control device 301 is, for example, configured with at least one integrated circuit. The integrated circuit is, for example, configured with at least one CPU, at least one ASIC, at least one FPGA, or a combination thereof.

An antenna (not shown) and the like are connected to the communication interface 304. The mobile terminal 300 exchanges data with external communication devices via the antenna and the access point AP (FIG. 1). The external communication devices include, for example, the sensor box 100 and the client terminal 200.

The display 305 is realized, for example, by an organic EL (Electro Luminescence) display. The input device 306 is realized, for example, by a touch sensor overlaid on the display 305. The touch sensor accepts various operations on the mobile terminal 300 by touch operations and outputs the contents of the operations to the control device 301.

The built-in memory 320 is, for example, a storage medium such as an eMMC (Embedded MultiMediaCard). As an example, the built-in memory 320 stores a program executed by the control device 301, but the program may be stored in a storage device other than the built-in memory 320 as long as the storage device is accessible by the control device 301.

[7. Hardware Configuration (Management Server)]
6 is a diagram showing an example of a hardware configuration of the management server 400. The management server 400 includes a control device 401, a ROM 402, a RAM 403, a communication interface 404, and a storage device 420.

The control device 401 controls the management server 400. The control device 401 is, for example, configured with at least one integrated circuit. The integrated circuit is, for example, configured with at least one CPU, at least one ASIC, at least one FPGA, or a combination thereof.

An antenna (not shown) and the like are connected to the communication interface 404. The management server 400 exchanges data with external communication devices via the antenna. External communication devices include, for example, the sensor box 100 and the client terminal 200.

The storage device 420 is, for example, a storage medium such as a hard disk or an external storage device. In one implementation, the storage device 420 stores a program executed by the control device 401, but the program may be stored in another storage device accessible to the control device 401.

8. Processing Flow
FIG. 7 is a flowchart of an example of a process for displaying information of residents. In one embodiment, the display of information of each resident 20 is controlled according to the process of FIG. 7. The displayed information is based on the biological information and/or care calls of each resident 20. The biological information includes at least one of vital information, behavioral information, and excretion information. The process of FIG. 7 is realized, for example, by the CPU of the control device 201 executing a given program. The program may be stored in the recording device 220, or may be stored in an external storage device accessible to the CPU of the control device 201.

Referring to Figure 7, in step S10, the control device 201 displays a list of information for each of the multiple residents 20. Figure 8 is a diagram showing an example of a list display of information for multiple residents 20.

In FIG. 8, the display 206 displays a list display screen 800. The list display screen 800 includes a matrix 810. Each square in the matrix 810 represents the status of each resident 20 with a color. In FIG. 8, the matrix 810 includes a total of 100 squares arranged in 10 columns and 10 rows. In one example, each square is associated with a resident 20. The control device 201 displays each square in a color that corresponds to the status of the associated resident 20.

In one example, the status of resident 20 includes "normal", "awake", "get out of bed", "fall", "slip", "abnormal vital signs", and "care call made". "Wake up", "get out of bed", "fall", and "slip" are generated as behavioral information. "Abnormal vital signs" indicates that the detection output of vital signs sensor 540 has indicated an abnormal value. "Care call made" indicates that care call handset 500 has been operated. Another status of resident 20 that is expected to be displayed is "excretion made". "Excretion made" indicates, for example, that the value detected by odor sensor 530 has indicated an abnormal value corresponding to excretion.

"Normal" indicates a state in which the above behavioral information or abnormal values have not been detected. When the state is "normal", the squares are displayed in white, and when the state is other than "normal", the squares are displayed in a color other than white. A priority may be assigned to each of the two or more states. The types of states to be displayed and the priority for each state may be determined in advance, or may be set appropriately by a director, etc.

Returning to FIG. 7, in step S20, the control device 201 determines whether the status of any of the residents 20 to be displayed in the list has changed. If the control device 201 determines that the status of none of the residents 20 has changed (NO in step S20), it advances control to step S60, and if the control device 201 determines that the status of at least one resident 20 has changed (YES in step S20), it advances control to step S30.

In step S60, the control device 201 determines whether or not an operation has been performed on a square in the matrix 810. An "operation on a square" is, for example, a click operation or a touch operation on a square on the display 206. If the control device 201 determines that an operation has been performed on a square (YES in step S60), the control proceeds to step S70, and if not (NO in step S60), the control proceeds to step S80.

In step S70, the control device 201 displays a pop-up window corresponding to the square that was the target of the operation in step S60. Figure 9 is a diagram showing an example of the display mode of the pop-up window.

In the list display screen 900 of Figure 9, a pop-up window 910 is displayed above the matrix 810. The pop-up window 910 displays detailed information about the resident 20 associated with the square being operated on out of the 100 squares of the matrix 810.

The pop-up window 910 includes a field 911 and a field 912. The field 911 shows the number of the room 10 and the name of the resident 20. The field 912 shows the image of the camera 105 installed in the room 10.

The pop-up window 910 includes a button 914, a comment field 915, buttons 916 and 917, and a field 919. The button 914 is operated to stop the change in display according to the information from the sensor box 100 for a certain period of time. The control device 201 changes the color of the square corresponding to each resident according to the behavioral information or vital abnormality of each resident or the care call from each resident input from the sensor box 100. By operating the button 914, the control device 201 identifies the resident corresponding to the pop-up window 910 including the button 914, and stops using the input from the sensor box 100 corresponding to the resident to change the display of the display 206 for a certain period of time. During the certain period, the output of information from the sensor box 100 to the management server 400 and the client terminal 200 may be continued or stopped.

The behavioral information comment field 915 accepts input of comments. Button 916 is operated to notify the mobile terminal 300 of the information in the pop-up window 910. Button 917 is operated when it is determined that no action is required for the resident 20 corresponding to the pop-up window 910. Field 919 displays icons representing behavioral information generated for the resident and/or care calls from the resident. By checking the type of icon displayed in field 919, the director can easily understand why the squares have been colored for the resident.

Column 918 represents resident information about resident 20 that has been stored in advance in management server 400 or the like. Resident information includes, for example, information regarding the need for care (can walk on their own, needs assistance walking, etc.), information regarding the priority of care, or information regarding medication (take high blood pressure medication after breakfast, take cold medicine after every meal, etc.). The director can refer to the resident information to determine whether or not to notify staff and/or to determine the content of comments to enter.

Returning to FIG. 7, in step S80, the control device 201 determines whether or not an operation has been performed on the pop-up window 910. More specifically, after displaying the pop-up window 910 in step S70, the control device 201 returns control to step S20, and then in step S80, determines whether or not an operation has been performed on the pop-up window 910 displayed in step S70.

If the control device 201 determines that an operation has been performed on the pop-up window 910 (YES at step S80), it proceeds to step S90; if not (NO at step S80), it returns control to step S20.

In step S90, the control device 201 accepts input of a comment in the comment field 915 of the pop-up window 910. An example of the content of the comment is instructions to staff. Another example is communication of information regarding the resident 20 who is the subject of the notification.

In step S100, the control device 201 accepts the designation of the mobile terminal 300 to which notifications are to be sent. For example, a staff member 850 who can assist the resident 20 uses the mobile terminal 300 carried by the staff member 850 to register information indicating that the staff member 850 can assist the client terminal 200. The control device 201 may display the IDs of two or more registered mobile terminals 300 in the pop-up window 910, for example as a pull-down menu. The director designates one or more mobile terminals 300 from the pull-down menu as the destination of notifications of the information in the pop-up window 910.

In step S110, the control device 201 notifies the mobile terminal 300 of (at least a part of) the information in the pop-up window 910. The control device 201 may perform the control of step S110 on the condition that an operation on the button 916 is accepted. In one implementation example, the notification includes information specifying the room shown in the pop-up window. The notification may also include the comment accepted in step S90.

In step S110, the notification is sent to the mobile terminal 300 specified in step S100. The specification of the mobile terminal 300 in step S100 may be omitted. In this case, the notification is sent to a predetermined mobile terminal 300.

In step S120, the control device 201 erases the display of the pop-up window 910 that was the target of the operation in step S80 from the display 206. Thereafter, control is returned to step S20. In one example, when the control device 201 receives a click operation on the square 811 in Fig. 8, it displays the pop-up window 910 (Fig. 9), and when it receives an operation on the button 916, it erases the display of the pop-up window 910 from the display 206. At the same time that the display of the pop-up window 910 is erased, the coloring in the square 811 (Fig. 8) may be removed.

In addition, the control device 201 may erase the display of the pop-up window 910 from the display 206 in response to the operation of the button 917. In other words, when the button 917 is operated, the controls of steps S90 to S110 may be omitted, and the pop-up window 910 may be erased in step S120.

The control device 201 may also erase the display of the pop-up window 910 from the display 206 after a given time has elapsed since the start of displaying the pop-up window 910. In other words, if a given time has elapsed since the start of displaying the pop-up window 910, the controls of steps S90 to S110 may be omitted, and the pop-up window 910 may be erased in step S120.

In step S30, the control device 201 updates the list display (matrix 810). In one example, when the status of a certain resident 20 changes from "normal" to "awake," the matrix 810 is updated so that the square corresponding to that resident 20 is colored.

Figure 10 is a diagram illustrating an example of updating the list display. In the list display screen 1000 of Figure 10, in addition to square 811, squares 812 to 814 are colored in the matrix 810. The color of each square is determined according to the status of the resident to which each square corresponds. As shown in squares 813 and 814, the colored squares may include an icon representing the status corresponding to the color assigned to each square. For example, the square corresponding to the "lit" status may include an icon representing the "lit" status.

Returning to FIG. 7, in step S40, the control device 201 determines whether or not a pop-up window is displayed above the updated square. If the control device 201 determines that a pop-up window is displayed above the updated square (YES in step S40), the control proceeds to step S50; if not (NO in step S40), the control proceeds to step S60.

When the list display changes from Figure 8 to Figure 10, no pop-up window is displayed above the updated squares (squares 812-814).

On the other hand, when the squares 812 to 814 are updated from the state shown in FIG. 9, the updated squares (squares 812 to 814) are covered by the pop-up window 910. In other words, the pop-up window 910 is displayed above the updated squares.

In step S50, the control device 201 updates the screen so that the information to be displayed in the updated squares is displayed on the display 206. Control is then returned to step S20.

11 is a diagram showing an example of a screen after updating in step S60. The list display screen 1100 in FIG. 11 shows an updated screen when the squares 812 to 814 are updated from the state shown in FIG. 9. The list display screen 1100 includes a column 1110. The column 1110 shows the character string "3 remaining: Room_102_Yamada_Falls/Room_202_Yamamoto_Wakes_Falls/Room_217_Yamanaka_Wakes_Falls". "3 remaining" shows the number of squares (squares 812 to 814) that have been updated in the area covered by the pop-up window 910. "Room_102_Yamada_Falls", "Room_202_Yamamoto_Wakes_Falls", and "Room_217_Yamanaka_Wakes_Falls" each show information on the resident to whom the squares 812 to 814 correspond.

In the example of FIG. 11, the text in field 1110 is displayed in a position that does not overlap with matrix 810 and pop-up window 910. Matrix 810 is an example of a first area that represents information about each of multiple monitored targets. Pop-up window 910 is an example of a second area that represents additional information about a monitored target that corresponds to a square that has been operated on in matrix 810. Field 1110 is an example of a third area that represents information about a monitored target whose state has changed after pop-up window 910 is displayed.

In one implementation, the second area outputs more detailed information about the resident than the first area. In the example of FIG. 11, each square of the matrix 810 represents only the status of each resident by its color. In the matrix 810, the status of each resident may be represented by the display pattern of each square (on, blinking, off, etc.) rather than by color or in combination with color. Meanwhile, the pop-up window 910 displays an image captured by the camera 105 in a field 912 as detailed information about the resident. Vital information about the resident (body temperature detected by the vital sensor 540, heart rate detected by the Doppler sensor 106, etc.) may be displayed in a field 918 as detailed information about the resident.

By displaying column 1110 outside matrix 810 and pop-up window 910, it is possible to display information that should be displayed in matrix 810 for a resident whose status has changed since pop-up window 910 was displayed, regardless of whether pop-up window 910 is displayed. In other words, when a part of matrix 810 that should be changed due to a change in the resident's status is not visible due to the display of pop-up window 910, the change is displayed in column 1110.

To view additional information about a resident in column 1110 (for example, Yamada-san in Room_102), the director moves pop-up window 910 to display the square corresponding to that resident in matrix 810 and clicks on that square. This causes a new pop-up window for that resident to be displayed. When an instruction to notify mobile terminal 300 of the newly displayed pop-up window is received, control device 201 sends a notification about that resident (for example, Yamada-san in Room_102) to mobile terminal 300 and then erases the newly displayed pop-up window. This updates the display on display 206 as shown in FIG. 12.

12 is a diagram showing an example of a screen after updating the screen of FIG. 11. Compared to FIG. 11, the display of the list display screen 1200 of FIG. 12 has been updated in the field 1110. The field 1110 of FIG. 12 displays the character string "2 items remaining: Room_202_Yamamoto_wake_up_fall/Room_217_Yamanaka_wake_up". That is, in FIG. 12, the number of remaining items is one less than in FIG. 11, and does not include information on Yamada in Room_102. This is based on the fact that a notification has been sent to the mobile terminal 300 regarding Yamada in Room_102, as described above. The control device 201 can update the display of the third area in response to the sending of a notification to the mobile terminal 300. The display of the third area may be updated in response to the "No action required" button (button 917 in FIG. 9) on the pop-up window. That is, if the "No action required" button is clicked in the pop-up window of a resident whose status has changed, the third area may be updated to erase the information of that resident.

[9. Modification (1)]
A first modified example of how to handle a change in the list display after the pop-up window is displayed will be described with reference to Fig. 13 and Fig. 14. Fig. 13 is a diagram showing an example of a display mode when a change occurs in the list display from the state shown in Fig. 9.

In this modified example, the third region is displayed within a pop-up window. More specifically, when the display 206 is displaying the pop-up window 910 as shown in Fig. 9, the control device 201 changes the display color of the squares 812-814 covered by the pop-up window 910 in step S40, and then displays the list display screen 1300 of Fig. 13 in step S50.

Compared to list display screen 1100 of Fig. 11, list display screen 1300 includes a pop-up window 1310. Pop-up window 1310 includes a column 1311 below buttons 916 and 917. In the example of Fig. 13, control device 201 displays in column 1311 the information that was displayed in column 1110 in the example of Fig. 11. That is, when there is a change in the status of a resident corresponding to a square located in the part of matrix 810 covered by pop-up window 1310, the change in the status of that resident is displayed in column 1311.

In the example of FIG. 13, column 1311 is an example of the third area. Column 1311 is located within pop-up window 1310. Because pop-up window 1310 is displayed in accordance with the director's operation, it is highly likely that the director is paying attention to pop-up window 1310. By displaying column 1311 within an area (pop-up window 1310) that is highly likely to be the director's focus, the example of FIG. 13 can more reliably allow the director to recognize changes in the squares located in the area covered by pop-up window 1310.

Figure 14 is a diagram showing an example of a screen after updating the screen of Figure 13. After displaying the screen of Figure 13, the control device 201 displays the screen of Figure 14 when a notification is sent to the mobile terminal 300 regarding the status of a resident (for example, Yamada-san in Room_102).

11 and 12, when a notification is sent to the mobile terminal 300 about the status of a resident (Yamada in Room_102), the control device 201 deletes the information about that resident from column 1311. Similarly, in column 1311 of Figure 14, the resident (Yamada in Room_102) has been deleted from column 1311 of Figure 13.

[10. Modification (2)]
A second modified example of how to respond to a change in the list display after a pop-up window is displayed will be described with reference to Fig. 15 and Fig. 16. Fig. 15 is a diagram showing an example of a list display screen. A list display screen 1500 in Fig. 15 includes a pop-up window 1510.

In this variant, the pop-up window that is already displayed is moved so as not to overlap the square in the list display corresponding to the resident whose status has changed. More specifically, when the control device 201 is displaying the list display screen 1500 of Figure 15 and the display color of the square covered by the pop-up window 1510 is changed in step S40, the control device 201 updates the display on the display 206 as shown in Figure 16 in step S50.

Figure 16 is a diagram showing an example of a screen updated from the state shown in Figure 15. List display screen 1600 in Figure 16 includes pop-up window 1510. In Figure 16, dashed frame 1601 represents the position of pop-up window 1510 in Figure 15. Arrow 1602 represents the direction of movement of the pop-up window.

Compared to FIG. 15, the matrix 810 in FIG. 16 displays the squares 815 and 816 in further color. Before the movement (FIG. 15), the square 816 was covered by the pop-up window 1510 (FIG. 15). As the pop-up window 1510 has moved in accordance with the arrow 1602, the square 816 is visible on the list display screen 1600 in FIG. 16.

Pop-up window 1510 is positioned so that it does not overlap not only square 816, but also squares 811-814 that were displayed before pop-up window 1510 was displayed. This more reliably prevents the display of a pop-up window for one resident from interfering with the display of the status of other residents on the list display screen.

[11. Modification (3)]
A third modified example of how to handle a change in the list display after the pop-up window is displayed will be described with reference to Figures 17 and 18. Figure 17 is a diagram showing an example of a display state when a change occurs in the list display from the state shown in Figure 9.

In this modified example, as shown in Figure 17, the matrix 810 in the list display screen 1700 is reduced to cause a pop-up display field 1720 to appear. A pop-up window 1721 is displayed in the pop-up display field 1720. More specifically, when the display 206 is displaying the pop-up window 910 as shown in Figure 9, the control device 201 changes the display color of the squares covered by the pop-up window 910 (squares 812 to 814) in step S40, and then updates the display on the display 206 as shown in Figure 17 in step S50.

Multiple pop-up windows can be displayed in the pop-up display field 1720. Figure 18 shows an example of the state after the display in Figure 17 is updated.

When an operation is performed on any of the squares 811 to 817 in Fig. 17, the control device 201 displays a pop-up window 1821 corresponding to the operated square in the pop-up display area 1720, as shown in Fig. 18. This allows the director to view multiple pop-up windows simultaneously, and thus to decide which staff member should be assigned to which resident while simultaneously viewing the detailed status of multiple residents.

In this variation, the pop-up windows are displayed outside of the matrix 810. This allows the director to concentrate on reviewing the resident information displayed in each pop-up window without confusing it with other resident information.

[12. Modification (4)]
15, 17, and 19, a fourth modification of how to deal with changes in the list display after a pop-up window is displayed will be described. In this modification, the pop-up window is displayed on the list display screen as long as it can be displayed without overlapping with the colored squares on the list display screen. Thereafter, when it can no longer be displayed on the list display screen so as not to overlap with the colored squares, the matrix on the list display screen is reduced and the pop-up window is displayed in the pop-up display field.

More specifically, when the squares 811 to 814 are colored on the list display screen, the control device 201 displays the pop-up window 1510 as shown in FIG. 15 in step S50. At this time, at least a portion of the pop-up window 1510 is displayed within the list display screen 1500 so as to overlap with the matrix 810. In this state, the pop-up window 1510 does not overlap any of the colored squares 811 to 814. Therefore, control is returned from step S40 to step S20 without proceeding to step S50.

Thereafter, when the number of colored squares increases on the list display screen, the control device 201, in step S50, reduces the matrix 810 of the list display screen as shown in FIG. 17, and displays a pop-up window 1721 in the pop-up display field 1720.

Even if the number of colored squares on the list display screen does not increase, the control device 201 may set a pop-up display column when the number of displayed pop-up windows increases. Figure 19 is a diagram showing an example of a screen after updating from the state shown in Figure 15. In Figure 19, squares 811 to 814 are colored on the list display screen 1710, and two pop-up windows 1721, 1821 are displayed in the pop-up display column 1720.

When an operation is performed on any of the squares in the state of Fig. 15, the control device 201 updates the display on the display 206 as shown in Fig. 19. More specifically, if the control device 201 determines in step S50 that it is not possible to add a pop-up window to the list display screen 1500 of Fig. 15 so as not to overlap any of the squares 811 to 814, it displays two pop-up windows in the pop-up display column as shown in Fig. 19.

In this variant, shrinking of the matrix 810 can be avoided as much as possible. On the other hand, if necessary, the matrix 810 is shrunk so that one or more pop-up windows are displayed so as not to overlap with the colored squares in the matrix 810.

When it is no longer necessary to display the pop-up window display field 1720, the control device 201 may erase the display of the pop-up window display field 1720 from the display 206. For example, when there is no longer a pop-up window displayed in the pop-up window display field 1720 due to erasing of the pop-up window (step S120), it is no longer necessary to display the pop-up window display field 1720.

The control device 201 may enlarge the display of the matrix 810 in association with the elimination of the pop-up window display field 1720. For example, when the number of pop-up windows increases on the display 206, the matrix 810 may be reduced as shown in FIG. 17, etc., and when the number of pop-up windows decreases, the matrix 810 may be enlarged as shown in FIG. 8, etc. In other words, the control device 201 may enlarge the matrix 810 in association with the elimination of the pop-up window.

[13. Other features of the monitoring system]
1 is not limited to the above, and it is expected that appropriate modifications may be made. For example, each of the sensor boxes 100A, 100B, etc. may directly output data to each of the client terminals 200A, 200B, etc., without going through the management server 400. The number of client terminals may be one.

In the monitoring system, various "programs" may be provided not as standalone programs but as part of any program. Various "programs" may be realized by any multiple programs working together. In one example, when the control device 101 of the sensor box 100 executes a first program to realize the function described in this specification, the first program may use some modules of the second program. Even if the first program does not include some modules for realizing the function, if it mainly contributes to the realization of the function, it does not deviate from the purpose of being a program for realizing the function. In addition, the functions described in this specification may be realized in part or in whole by dedicated hardware. Furthermore, the device providing the functions described in this specification may provide some or all of its functions from an external device as a so-called cloud service.

The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is indicated by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. Furthermore, the inventions described in the embodiments and each modified example are intended to be implemented, as far as possible, either alone or in combination.

10 Room, 20 Resident, 30 Bed, 100, 100A, 100B Sensor box, 101, 201, 301, 401 Control device, 200, 200A, 200B Client terminal, 300, 300A, 300B Mobile terminal, 400 Management server, 500 Care call handset, 501 Care call button, 800, 900, 1000, 1100, 1200, 1300, 1500, 1600, 1710 List display screen, 810 Matrix, 811, 812, 814, 815, 816, 817 Grid, 820, 820A, 820B Director, 850, 850A, 850B Staff, 910, 1310, 1510, 1721, 1821 Pop-up window, 911, 912, 918, 919, 1110, 1311 columns, 914, 916, 917 buttons, 915 comment column, 1601 frame, 1602 arrow, 1720 popup display column.

Claims (34)

A method for displaying information on a plurality of monitoring targets, comprising:
Displaying a first area on a display, the first area representing information of each of a plurality of monitoring targets;
displaying, on the display, a second area representing additional information of a target being watched over corresponding to an operation on the first area together with the first area in response to the operation;
In response to a change in state of at least some of the multiple monitoring targets, updating the display on the display so that a change in the first area corresponding to the change is recognized regardless of the display of the second area,
the second region is displayed so as to overlap at least a portion of the first region;
The method, characterized in that the step of updating the display includes updating the display so that a change in the state of the monitored object corresponding to at least the portion of the first area is recognized. The step of updating the display on the display includes displaying, on the display, a third area representing information of a monitoring target whose status has changed among the plurality of monitoring targets.
The method of claim 1. The method of claim 2, wherein the third region is located outside the first region and the second region. The method of claim 2, wherein the third region is located within the second region. The method according to any one of claims 1 to 4, wherein the step of updating the display includes moving the second region so that it does not overlap the portion of the first region that has been modified in response to the change. The method of claim 5, wherein the step of updating the display includes moving the second area so that it does not overlap with a portion of the first area corresponding to a target whose state had changed before the second area was displayed. The method according to any one of claims 1 to 6, wherein the step of updating the display includes reducing the first region. the step of displaying the second region includes displaying the second region such that at least a portion of the first region and the second region overlap with each other;
The step of updating the display of the display includes reducing the first area when the second area cannot be displayed in a portion of the first area other than a portion corresponding to a watching target whose state has changed before and after the second area is displayed.
The method according to claim 7. The method according to any one of claims 1 to 8, wherein the step of displaying the second region includes displaying the second region outside the first region. The method according to any one of claims 1 to 9, further comprising a step of transmitting at least a portion of the additional information in the second region to a mobile terminal. The method further includes a step of accepting an input of a comment for the second area,
The method of claim 10 , wherein transmitting at least a portion of the additional information in the second region to a mobile device includes transmitting the comment to the mobile device. The method according to claim 10 or 11, further comprising a step of accepting input of information specifying a mobile terminal as a destination of the additional information. removing the second region from the display;
the step of removing the second region involves enlarging the first region.
The method according to any one of claims 1 to 12. The method of claim 13, wherein the step of removing the display of the second region from the display is performed on at least one of the conditions that at least a portion of the additional information in the second region has been transmitted to the mobile device and that a given time has elapsed since the display of the second region began. The method according to any one of claims 1 to 14, wherein the second area displays more detailed information about the target of monitoring than the first area. The method according to any one of claims 1 to 15, wherein the additional information includes a camera image of the monitored subject. The method according to any one of claims 1 to 16, wherein the change in state includes a change to getting up or getting out of bed. The method according to any one of claims 1 to 17, wherein the first area displays information based on biological information of each of the multiple monitored subjects. The method according to any one of claims 1 to 18, wherein a plurality of squares are arranged in a matrix in the first area, and information about the target being monitored is displayed in each of the plurality of squares. The method according to any one of claims 1 to 19, wherein in the first area, the color of the area corresponding to the target of monitoring changes in response to a change in the state of the target of monitoring. the second region is a pop-up window;
A method according to any preceding claim, wherein the step of updating the display comprises displaying a pop-up window separate from the pop-up window. The method of claim 21 , wherein the separate pop-up window includes information about the watched target whose state has changed. The method according to claim 21 or 22, wherein the separate pop-up window includes an image of the watched target whose state has changed. A program that, when executed by one or more processors, causes the one or more processors to carry out the method according to any one of claims 1 to 23 . A memory storing the program according to claim 24;
and one or more processors that execute the programs stored in the memory. A method for displaying information on a plurality of monitoring targets, comprising:
Displaying a first area on a display, the first area representing information of each of a plurality of monitoring targets;
displaying, on the display, a second area representing additional information of a target being watched over corresponding to an operation on the first area together with the first area in response to the operation;
In response to a change in state of at least some of the multiple monitoring targets, updating the display on the display so that a change in the first area corresponding to the change is visually recognized regardless of the display of the second area,
The step of updating the display on the display includes displaying, on the display, a third area representing information of a monitoring target whose status has changed among the plurality of monitoring targets;
The method of claim 1, wherein the third region is located within the second region. A method for displaying information on a plurality of monitoring targets, comprising:
Displaying a first area on a display, the first area representing information of each of a plurality of monitoring targets;
displaying, on the display, a second area representing additional information of a target being watched over corresponding to an operation on the first area together with the first area in response to the operation;
In response to a change in state of at least some of the multiple monitoring targets, updating the display on the display so that a change in the first area corresponding to the change is visually recognized regardless of the display of the second area,
The method of claim 1, wherein updating the display includes moving the second region so that it does not overlap the portion of the first region that has been modified in response to the change. 28. The method of claim 27, wherein the step of updating the display includes moving the second region so that it does not overlap any portion of the first region corresponding to a monitored object whose state had changed before the second region was displayed. A method for displaying information on a plurality of monitoring targets, comprising:
Displaying a first area on a display, the first area representing information of each of a plurality of monitoring targets;
displaying, on the display, a second area representing additional information of a target being watched over corresponding to an operation on the first area together with the first area in response to the operation;
In response to a change in state of at least some of the multiple monitoring targets, updating the display on the display so that a change in the first area corresponding to the change is visually recognized regardless of the display of the second area,
The method, wherein updating the display includes shrinking the first region. the step of displaying the second region includes displaying the second region such that at least a portion of the first region and the second region overlap with each other;
The step of updating the display of the display includes reducing the first area when the second area cannot be displayed in a portion of the first area other than a portion corresponding to a watching target whose state has changed before and after the second area is displayed.
30. The method of claim 29 . A method for displaying information on a plurality of monitoring targets, comprising:
Displaying a first area on a display, the first area representing information of each of a plurality of monitoring targets;
displaying, on the display, a second area representing additional information of a target being watched over corresponding to an operation on the first area together with the first area in response to the operation;
In response to a change in state of at least some of the multiple monitoring targets, updating the display on the display so that a change in the first area corresponding to the change is visually recognized regardless of the display of the second area,
The method further comprising transmitting at least a portion of the additional information in the second region to a mobile device. The method further includes a step of accepting an input of a comment for the second area,
32. The method of claim 31 , wherein transmitting at least a portion of the additional information in the second region to a mobile device includes transmitting the comment to the mobile device. The method according to claim 31 or 32 , further comprising the step of accepting input of information designating a mobile terminal as a destination of the additional information. A method for displaying information on a plurality of monitoring targets, comprising:
Displaying a first area on a display, the first area representing information of each of a plurality of monitoring targets;
displaying, on the display, a second area representing additional information of a target being watched over corresponding to an operation on the first area together with the first area in response to the operation;
updating the display of the display in response to a change in state of at least some of the multiple monitoring targets so that a change in the first area corresponding to the change is visible regardless of the display of the second area;
removing the second region from the display;
A method, wherein the step of removing the display of the second region involves enlarging the first region.

Images