JP7688848B2 - Display control device and monitoring system - Google Patents

Description

The present invention relates to a display control device and a monitoring system.
This application claims priority based on Japanese Patent Application No. 2021-167872, filed on October 13, 2021, the contents of which are incorporated herein by reference.

Technology relating to remote monitoring devices has been disclosed in the past (see, for example, Patent Document 1).

Japanese Patent Application Publication No. 2017-11698

One embodiment of the present invention is a display control device comprising: a receiving unit that receives event information regarding events occurring at a monitored location; and a presentation unit that presents a history of the event information received by the receiving unit.

FIG. 1 is a diagram illustrating an example of a device configuration of a monitoring system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a functional configuration of the monitoring system according to the present embodiment. FIG. 4 is a diagram illustrating an example of setting information of the monitoring system of the present embodiment. FIG. 4 is a diagram illustrating an example of an initial setting screen according to the present embodiment. FIG. 4 is a diagram illustrating an example of an initial setting screen according to the present embodiment. FIG. 4 is a diagram showing an example of an operation screen of an emergency call function according to the present embodiment. 11A to 11C are diagrams illustrating an example of a display for each monitoring mode in the present embodiment. 11A to 11C are diagrams illustrating an example of a display for each monitoring mode in the present embodiment. FIG. 2 is a diagram showing an example of communication in an emergency response function of the present embodiment. FIG. 13 is a diagram illustrating an example of a pop-up display at the time of an emergency response according to the present embodiment. FIG. 2 is a diagram showing an example of a live view display according to the present embodiment. FIG. 13 is a diagram showing an example of a correspondence selection screen according to the present embodiment. FIG. 13 is a diagram showing an example of a display screen of a response history according to the present embodiment. FIG. 11 is a diagram showing another example of communication in the emergency response function of the present embodiment. FIG. 4 is a diagram illustrating an example of an event history display according to the present embodiment. FIG. 13 is a diagram illustrating an example of a selection screen for events to be presented according to the present embodiment. FIG. 2 is a diagram showing an example of communication of a conversation function of the present embodiment. FIG. 11 is a diagram showing an example of communication of a live view function in a private mode according to the present embodiment. FIG. 11 is a diagram showing an example of communication of a live view function in a public mode according to the present embodiment. 1A and 1B are diagrams illustrating an example of the operation of the monitoring system in a public mode and a private mode according to the present embodiment. FIG. 4 is a diagram illustrating an example of communication in a voltage drop notification function of the present embodiment. FIG. 13 is a diagram illustrating an example of a voltage drop notification screen according to the embodiment. 11 is a diagram showing an example of communication in a remote control device search function of the present embodiment. FIG. 10 is a diagram showing an example of communication in a remote control device out-of-service notification function of the embodiment; FIG.

[Surveillance system configuration]
FIG. 1 is a diagram showing an example of a device configuration of a monitoring system 1 according to the present embodiment.
The monitoring system 1 includes a terminal device 10, a server device 20, a communication device 30, and a remote control device 40. The monitoring system 1 provides a monitoring service that enables a monitor to monitor the status of a person to be monitored.

In the following description, the person to be monitored is also referred to as the person being monitored or the person being monitored. The monitoring service provided by the monitoring system 1 is also referred to as the monitoring service. Moreover, the monitor is also referred to as the monitor, the user of the monitoring service, or the user.
That is, the monitoring system 1 provides a monitoring service that enables a watcher to watch over the state and behavior of a person being watched over.
As an example, the person being watched over is a family member of the person being watched over who does not live with the person being watched over. The monitoring system 1 provides a function that enables the person being watched over to remotely monitor the situation of the person being watched over.

The terminal device 10 is a device used by a watcher, that is, a user. The terminal device 10 is, for example, a smartphone, a tablet, a personal computer, or the like, and performs information communication with the server device 20 via the network N1.
The server device 20 is a computer device operated by a monitoring service provider. The server device 20 performs various settings in the monitoring system 1, and holds and provides information acquired by the monitoring system 1.
The communication device 30 includes an imaging unit 350 (imaging device) such as a camera, and captures images of the monitoring target area A. The communication device 30 communicates information with the server device 20 via the network N2.
The remote control device 40 is a device used by the person being watched over. The remote control device 40 communicates information with the communication device 30 by short-range wireless communication, infrared communication, or the like.

The monitoring system 1 may include a plurality of terminal devices 10, communication devices 30, and remote control devices 40.
As an example, the monitoring system 1 can be configured to set multiple watchers, with each watcher using one terminal device 10. For example, if the person watching has brothers or sisters, the monitoring system can be configured so that the multiple watchers (brothers or sisters) each watch over the parent who is the person being watched over.
Furthermore, the monitoring system 1 can be configured to install the communication device 30 in each room of a building and monitor the state and behavior of the person being watched in each room. For example, if a building has multiple rooms, the monitoring system 1 can be configured to monitor the state and behavior of the person being watched in each room, with a first room (e.g., a living room) as a monitored area A1 and a second room (e.g., a bedroom) as a monitored area A2.
Furthermore, the monitoring system 1 can be configured to allow multiple people being watched over who live together in a building to each use the remote control device 40. For example, if the people being watched over are the parents of the person being watched over, the monitoring system 1 can be configured so that each of the father and mother uses the remote control device 40.
Furthermore, the monitoring system 1 can monitor not only one building but multiple buildings. For example, if the person watching over has a spouse, the monitoring system 1 can be configured to monitor the state and behavior of the person watching over in each building, with a first building where the person watching over's parents live as a monitoring target area A1 and a second building where the person watching over's spouse's parents live as a monitoring target area A2.

In the following description, when it is necessary to distinguish between the multiple terminal devices 10, the first terminal device 10 will be referred to as terminal device 10-1, the second terminal device 10 as terminal device 10-2, . . .
Furthermore, when it is necessary to distinguish between the multiple communication devices 30, the first communication device 30 will be referred to as communication device 30-1, the second communication device 30 as communication device 30-2, . . .
Next, the functional configuration of each device in the monitoring system 1 will be described with reference to FIG.

[Functional configuration of surveillance system]
FIG. 2 is a diagram showing an example of a functional configuration of the monitoring system 1 of the present embodiment.

[Functional configuration of terminal device]
The terminal device 10 includes a terminal operation detection unit 110 , a presentation unit 120 , a terminal control unit 130 , a terminal communication unit 140 (receiving unit), a terminal sound collection unit 150 , and a terminal sound output unit 160 .
The terminal operation detection unit 110 includes an operation device such as a touch panel or an operation switch, and detects an operation on the operation device by a user of the terminal device 10. When the terminal operation detection unit 110 detects an operation, it generates operation information indicating the detected operation and outputs the operation information to the terminal control unit 130.
The presentation unit 120 includes, for example, a liquid crystal display, and presents an image under the control of the terminal control unit 130 .
The terminal control unit 130 includes, for example, a microcomputer, and controls each part of the terminal device 10 .
The terminal communication unit 140 communicates information with the server device 20 under the control of the terminal control unit 130 .

The terminal sound collection unit 150 includes, for example, a microphone, and collects sounds around the terminal device 10. The terminal sound collection unit 150 outputs information about the collected sounds to the terminal control unit 130.
In addition, the terminal device 10 may operate each function of the terminal control unit 130 by the collected sound (for example, by performing voice recognition). In this case, the terminal sound collection unit 150 can be said to be an example of the terminal operation detection unit 110 that operates the terminal device 10 by sound.
The terminal sound output unit 160 includes, for example, a speaker, and emits a sound based on the control of the terminal control unit 130. Note that the terminal device 10 may alert the user by the emitted sound (for example, by sounding an alarm). In this case, the terminal sound output unit 160 can be said to be an example of the presentation unit 120 that presents information by sound.

[Functional configuration of the server device]
The server device 20 includes a server communication unit 210 and a server storage unit 220 .
The server communication unit 210 communicates information with the terminal device 10 and with the communication device 30 .
The server storage unit 220 includes a storage device such as a semiconductor memory, and stores various types of information. An example of the information stored in the server storage unit 220 will be described with reference to FIG.

FIG. 3 is a diagram showing an example of setting information of the monitoring system 1 of this embodiment.
As described above, the terminal device 10 can set multiple monitoring destinations as monitoring targets in the monitoring system 1. In this example, among the multiple monitoring destinations a to c, the terminal device 10-1 and the terminal device 10-2 are set as monitoring targets for the monitoring destinations a to c.
In addition, in the terminal device 10-3, the watching destinations a to b are set as the watching targets, but the watching destination c is not set as the watching target.
Furthermore, when multiple terminal devices 10 are monitoring one monitoring destination, it is possible to set some of these multiple terminal devices 10 to function as a main monitoring device and some to function as sub monitoring devices. For example, for monitoring destination a, terminal device 10-1 functions as the main monitoring device, and terminal devices 10-2 and 10-3 function as sub monitoring devices. In the monitoring system 1, the main monitoring device and the sub monitoring devices can provide different functions.
The server storage unit 220 stores setting information as shown in Fig. 3. This setting information is referenced by the terminal device 10 or the communication device 30 as necessary.

Here, the user of the main monitoring device and the user of the sub monitoring device may be given different operation rights. For example, the user of the main monitoring device may be given the right to view real-time video regardless of the consent of the person being watched, and the user of the sub monitoring device may not be able to view real-time video without the consent of the person being watched.
Further, although the operation authority has been described here as having two levels, main and sub, the present invention is not limited to this, and the operation authority may be set in three or more levels.
That is, each of the multiple users is given one of at least two levels of operation authority, and the presenting unit 120 presents the content based on the operation authority given to the user.

This operation authority may be given for each monitored area A (i.e., each location to be monitored).
That is, there are multiple locations to be monitored, and the user is given operating authority for each monitored location, and the presentation unit 120 of the terminal device 10 presents content based on the operating authority given to the user for each monitored location.

[Functional configuration of communication device]
Returning to Figure 2, the communication device 30 has the functions of a first communication unit 310, a second communication unit 320, a communication device control unit 330, a measurement unit 340, an imaging unit 350, a communication device sound collection unit 360, a communication device sound generation unit 370, an operation detection unit 380, and a communication device memory unit 390.

First communication unit 310 communicates information with remote control device 40 under the control of communication device control unit 330 .
The second communication unit 320 communicates information with the server device 20 under the control of the communication device control unit 330 .

As described above, the second communication unit 320 communicates with the terminal device 10 via the server communication unit 210 of the server device 20. However, in the following description, the explanation of the server device 20 mediating the communication between the terminal device 10 and the communication device 30 may be omitted, and it may be described, for example, that the second communication unit 320 transmits the first operation information to the terminal device 10.

The measurement unit 340 measures various conditions in the monitored area A. For example, the measurement unit 340 may include a temperature sensor and a humidity sensor to measure the temperature and humidity of the monitored area A. The measurement unit 340 may also include an infrared sensor to measure the movement of people in the monitored area A.
The imaging unit 350 includes, for example, a camera, and captures an image of the monitoring area A based on the control of the communication device control unit 330 .

The measurement unit 340 and the imaging unit 350 are collectively referred to as a detection unit. That is, the communication device 30 detects various conditions in the monitored area A by the detection unit.
In addition to the imaging unit 350, the detection unit may have a judgment unit (not shown) that judges the state of the monitored subject based on the video captured by the imaging unit 350. The judgment unit, for example, performs posture analysis of the monitored subject based on the results of machine learning and judges whether or not the monitored subject has experienced an abnormality, such as a fall. The judgment unit outputs the judgment result to the communication device control unit 330.

The communication device control unit 330, for example, has a microcomputer and controls each part of the communication device 30, including the first communication unit 310, the second communication unit 320, and the imaging unit 350.

The communication device sound pickup section 360 includes, for example, a microphone, and picks up sounds around the communication device 30. The communication device sound pickup section 360 outputs information about the picked up sounds to the communication device control section 330.
The communication device sound output unit 370 includes, for example, a speaker, and emits sounds based on the control of the communication device control unit 330 .
The operation detection unit 380 detects an operation on an operation unit (not shown), such as a switch, included in the communication device 30. As an example, the communication device 30 includes an operation mode changeover switch (not shown). The operation detection unit 380 detects the setting state of the operation mode changeover switch and outputs the setting state to the communication device control unit 330.

The communication device storage unit 390 includes a storage device such as a semiconductor memory, and stores various types of information. As an example, the communication device storage unit 390 stores information on images (e.g., moving images) captured by the imaging unit 350 and information on sounds captured by the communication device sound collection unit 360.

[Functional configuration of remote control device]
Remote control device 40 includes an emergency button 410 , an OK button 420 , an NG button 480 , a remote control communication section 430 , a remote control control section 440 , a remote control sound collection section 450 , a remote control sound generation section 460 , and a voltage detection section 470 .
The OK button 420 is an operation button indicating agreement, approval, or a positive answer, and is also called a “Yes button.” The NG button 480 is an operation button indicating disagreement, rejection, or a negative answer, and is also called a “No button.”

The emergency button 410 is a button that is operated by the person being watched over if any abnormality occurs to the person being watched over.
The OK button 420 is a button that is operated by the person being watched over to approve a consent request made to the person being watched over by the monitoring system 1 .

The remote control communication unit 430 communicates information with the communication device 30 based on the control of the remote control control unit 440.

The remote control control unit 440 includes, for example, a microcomputer, and controls each unit of the remote control device 40. For example, when the remote control control unit 440 detects an operation on the emergency button 410, it generates first operation information, which is information indicating that the emergency button 410 has been operated. The remote control control unit 440 transmits the generated first operation information to the communication device 30 via the remote control communication unit 430. Similarly, when the remote control control unit 440 detects an operation on the OK button 420, it generates second operation information, which is information indicating that the OK button 420 has been operated. The remote control control unit 440 transmits the generated second operation information to the communication device 30 via the remote control communication unit 430.

Remote control sound collection section 450 includes, for example, a microphone, and collects sounds around remote control device 40. Remote control sound collection section 450 outputs information on the collected sounds to remote control control section 440.
Remote control sound generating section 460 includes, for example, a speaker, and emits sounds based on the control of remote control control section 440 .
Voltage detection unit 470 detects the voltage value of the operating power supply of remote control device 40. Voltage detection unit 470 outputs the detected voltage value of the operating power supply to remote control control unit 440. When the voltage value detected by voltage detection unit 470 is equal to or lower than a predetermined value, remote control control unit 440 transmits voltage drop information to communication device 30 via remote control communication unit 430.

[Explanation of each function]
Next, the functions provided by the monitoring system 1 and their operations will be described with reference to the drawings.
In the following description, application software (hereinafter also simply referred to as an app) provided by the monitoring system 1 is installed in the terminal device 10. The terminal control unit 130 controls each unit of the terminal device 10 based on the installed app. A screen defined by the app is displayed on the presentation unit 120 of the terminal device 10. After the app is installed in the terminal device 10, an initial setting screen is displayed on the presentation unit 120 when the app is launched for the first time.

[Initial Settings]
4A and 4B are diagrams showing an example of an initial setting screen in this embodiment. Fig. 4A shows an example of a notification setting screen. The monitoring system 1 provides a function for initially setting a situation related to the person being watched that should be notified to the terminal device 10. An example of a situation to be notified is shown below.

Emergency call: When the emergency button 410 is operated.
Fall: When the person being watched falls within monitored area A.
Temperature and humidity abnormality: When the temperature or humidity of monitored area A is outside the specified range.
- Safety confirmation abnormality: When the person being monitored does not respond when you call out to them.
Camera abnormality: When an abnormality such as a breakdown occurs in the imaging unit 350.
- Deviation from routine: When the behavior of the person being monitored is different from usual.

The user operates the notification setting screen displayed on the presentation unit 120 of the terminal device 10 and selects a situation to be notified of from among multiple types of situations. The terminal device 10 transmits the selected notification setting to the server device 20. The server device 20 stores the received notification setting in the server storage unit 220. The monitoring system 1 notifies the terminal device 10 when the set situation occurs.
In this example, the notification setting is described as being stored in the server storage unit 220, but is not limited to this. The notification setting may be stored in any of the terminal device 10, the server device 20, and the communication device 30 as long as any of the devices can determine whether or not a certain situation should be notified to the user when the situation occurs. Various pieces of information described below may also be stored in any of the devices as long as the functions using the information can be satisfied.

Figure 4B shows an example of a contact setting screen. The monitoring system 1 provides an emergency call function that contacts pre-defined contacts with minimal operations (e.g., tapping an icon displayed on the terminal device 10) in the event of an emergency.

Fig. 4C shows an example of an operation screen of the emergency call function. When the above-mentioned situation to be notified occurs, the operation screen (emergency response screen) of the emergency call function shown in Fig. 4C is displayed on the presentation unit 120 of the terminal device 10. As an example, when three emergency call destinations are set, emergency call destination icons P11 to P13 are displayed on the presentation unit 120. The contacts for this emergency call function can be set on the contact setting screen shown in Fig. 4B.
The user sets the contacts by operating the contact setting screen displayed on the presentation unit 120 of the terminal device 10. The terminal device 10 stores the set contacts in a storage unit (not shown) of the terminal device 10.

[Monitoring mode settings]
5A and 5B are diagrams showing examples of displays for each monitoring mode of this embodiment.
The monitoring system 1 provides a plurality of monitoring modes, including a private mode and a public mode. FIG. 5A shows an example of a screen displayed on the terminal device 10 in the private mode. In the private mode, an image of the monitored subject is not displayed, and an image P21 showing the status of the monitored subject in text is displayed. FIG. 5B shows an example of a screen displayed on the terminal device 10 in the public mode. In the public mode, an image P22 captured by the imaging unit 350 of the communication device 30 of the monitored area A is displayed. At this time, if there is a monitored subject in the monitored area A, a moving image (real-time image) showing the current status of the monitored subject is displayed in the image P22. Note that, for convenience of explanation, in FIG. 5B, the moving image of the monitored subject is expressed by a line drawing. In this embodiment, the terminal device 10 displays a moving image of the monitored subject that is actually captured by the imaging unit 350, rather than a line drawing.
The private mode and the public mode can be switched by an operation mode switching switch (not shown) of the communication device 30 described above.
By making it possible to switch monitoring modes, the monitoring system 1 can provide a monitoring function that allows the degree of privacy protection and the level of monitoring function to be selected depending on the relationship between the person being watched and the watcher and the situation of the person being watched.
Next, each function after the initial setting will be described.

[Monitoring function (1) Emergency response function]
FIG. 6 is a diagram showing an example of communication in the emergency response function of this embodiment.
(Step S101) When some abnormality occurs to the person being watched over, the person being watched over operates the emergency button 410 of the remote control device 40. When the remote control device 40 detects that the emergency button 410 has been operated, it transmits first operation information to the communication device 30. The first operation information is information indicating that the emergency button 410 has been operated.

(Step S<b>102 ) When the communication device control section 330 receives information indicating that the emergency button 410 has been operated from the remote control device 40 , the communication device control section 330 transmits first operation information from the second communication section 320 to the terminal device 10 .
In the following description, the information that the communication device control unit 330 transmits to the terminal device 10 when the emergency button 410 is operated is also referred to as "first operation information." This expression includes a case where the communication device control unit 330 transfers the first operation information received from the remote control device 40 directly to the terminal device 10, and a case where the communication device control unit 330 transmits information generated in response to receiving the first operation information from the remote control device 40 to the terminal device 10.

Here, when the communication device 30 is equipped with the imaging unit 350, when the communication device control unit 330 receives the first operation information from the remote control device 40, it transmits the image captured by the imaging unit 350 and the first operation information from the second communication unit 320 to the terminal device 10.
Furthermore, communication device 30 may include communication device sound collection unit 360. In this case, when communication device control unit 330 receives first operation information from remote control device 40, it transmits information on the sound collected by communication device sound collection unit 360 and the first operation information to terminal device 10.
Furthermore, the communication device control unit 330 may transmit information on the sound picked up by the communication device sound collection unit 360 without transmitting the image captured by the imaging unit 350. Furthermore, the communication device control unit 330 may transmit both the image captured by the imaging unit 350 and the information on the sound picked up by the communication device sound collection unit 360 to the terminal device 10.
In addition, when the communication device 30 transmits the image captured by the imaging unit 350 and the sound collected by the communication device sound collection unit 360 immediately after the image capture and sound collection, i.e., in real time, these are also referred to as “live view images and sound”.

In the following, the communication device 30 is described as having an imaging unit 350 and a communication device sound collection unit 360, and transmitting live view images and audio to the terminal device 10.

In addition, sounds in the monitored area A can also be picked up by the remote control sound pickup unit 450 provided in the remote control device 40 operated by the person being watched over. In this case, the sound picked up by the remote control sound pickup unit 450 is transmitted from the remote control communication unit 430 and acquired by the communication device 30. Here, after the emergency button 410 is operated, communication between the remote control device 40 and the communication device 30 may be interrupted for some reason. By providing the communication device sound pickup unit 360 to the communication device 30, even if communication between the communication device 30 and the remote control device 40 is not possible, the communication device 30 can pick up surrounding sounds and provide live view images and sounds to the terminal device 10.

(Step S103) When the terminal device 10 receives the first operation information and the live view image and audio from the communication device 30, it displays an image on the presentation unit 120 indicating that the emergency button 410 has been operated.

Figure 7 is a diagram showing an example of a pop-up display during an emergency response in this embodiment. When the terminal device 10 receives the first operation information from the communication device 30, it displays a pop-up image P31 as shown in Figure 7. According to the monitoring system 1 configured in this manner, it is possible to notify the user that the person being watched has operated the emergency button 410, even when the user is not operating the terminal device 10 or is performing an operation other than the monitoring system 1 app.

FIG. 8 is a diagram showing an example of a live view display according to this embodiment.
(Step S104) When the pop-up image P31 displayed in step S103 is operated (for example, tapped), the terminal device 10 displays a live view image P41, a response selection icon P42, and a conversation icon P43.
For ease of explanation, the moving image of the person being watched over in the live view image P41 is represented by a line drawing in Fig. 8. In this embodiment, the moving image of the person being watched over that is actually captured by the imaging unit 350 is displayed on the terminal device 10 instead of a line drawing.

FIG. 9 is a diagram showing an example of the correspondence selection screen according to this embodiment.
(Step S105) When the response selection icon P42 is operated, the terminal device 10 displays a response selection screen P51 shown in FIG.

Here, the first operation information is an example of event information indicating an event in which the emergency button 410 has been operated. The response selection screen P51 includes icons indicating response contents according to the type of event.
In other words, the event information includes the type of event, and the presentation unit 120 presents content according to the type of event included in the received event information.

When the terminal device 10 receives the first operation information, that is, when the type of the event is “operation of the emergency button 410”, the terminal device 10 may, for example,
The user rushes to the location of the person being watched over (icon P511)
・Report to public institutions (fire and ambulance, police, health center, city hall, etc.) (icon P512)
Contact a predefined contact (icon P513)
An action selection screen P51 including various icons such as an icon P514 for launching a telephone application is displayed.
That is, the response includes at least one of moving the user to the location to be monitored, notifying public institutions, and notifying other users. When any icon on the response selection screen P51 is operated, the terminal device 10 performs an action corresponding to the operated icon (e.g., calling a specified contact). Furthermore, the terminal device 10 transmits the selected response content to the server device 20.

In other words, when the device communication unit 140 receives event information, the presentation unit 120 displays a pop-up icon indicating the event information, and presents content according to the type of event in response to a user operation after the icon is displayed.

As described above, if the communication device 30 has a judgment unit (not shown) and this judgment unit judges whether or not an abnormality has occurred based on the results of the posture analysis of the person being monitored, the judgment result by the judgment unit that "the person being monitored has fallen" can also be considered an example of event information.

6, when the server device 20 receives the response content from the terminal device 10, the server device 20 stores the received response content as a response history in the server storage unit 220 together with time information transmitted by the terminal device 10. In addition, the server device 20 transmits to the communication device 30 information indicating that the user has responded to the operation of the emergency button 410.
(Step S107) The communication device 30 releases the emergency state and returns to normal operation.

[Information sharing with other terminal devices]
(Step S108) The other terminal devices 10 (for example, the terminal device 10-2 and the terminal device 10-3) acquire from the server device 20 the response history stored in step S106.

Here, the monitoring system 1 provides a function that allows other terminal devices 10 to share the selection result of the response content selected in step S105. For example, if the terminal device 10 that selected the response content in step S105 is terminal device 10-1, the other terminal devices 10 (for example, terminal device 10-2 and terminal device 10-3) can obtain the selection result (i.e., the response content of the user of terminal device 10-1 in response to the operation of the emergency button 410).

That is, the event information is transmitted to multiple users, and the presentation unit 120 presents the status of a response to the event information by any one of the multiple users.

Figure 10 is a diagram showing an example of a response history display screen in this embodiment. The terminal device 10 displays a response history screen P61 obtained from the server device 20. This response history screen P61 includes the name of the user who responded, the time of the response, and the content of the response.

In other words, the presentation unit 120 of the terminal device 10 presents a screen (response selection screen P51) for selecting multiple responses, and presents the content of the response selected by any one of the multiple users.

As described above, a plurality of destination terminal devices 10 can be set for one communication device 30. In this case, when the communication device 30 receives event information (e.g., the first operation information) from the remote control device 40, the communication device 30 transmits the event information to each of the plurality of destination terminal devices 10.
At this time, the fact that a user of any one of the terminal devices 10 has taken some kind of action in response to the event information (for example, rushing to the location of the person being watched over in response to the operation of the emergency button 410) is shared among the users of the multiple terminal devices 10, allowing each user to determine whether or not further action is required. The monitoring system 1 configured in this manner can provide users with a sense of security.

[Video display when emergency button is operated]
The monitoring system 1 may provide a function for displaying images captured by the imaging section 350 in the time period before and after the operation of the emergency button 410 on the video display screen P62 shown in FIG.

FIG. 11 is a diagram showing another example of communication in the emergency response function of the present embodiment.
(Step S101) As described with reference to Fig. 6, when some abnormality occurs in the person being watched over, the person being watched over operates the emergency button 410 of the remote control device 40. When the remote control device 40 detects that the emergency button 410 has been operated, it transmits first operation information to the communication device 30.

(Step S201) The communication device 30 of this embodiment constantly captures images using the imaging unit 350 and collects sounds using the communication device sound collection unit 360, regardless of whether the emergency button 410 is operated or not.
The communication device control unit 330 stores the captured images and collected sounds in the communication device storage unit 390. Depending on the capacity of the communication device storage unit 390, the communication device control unit 330 may erase information captured and collected before a predetermined period of time and store new images and sounds. Here, the predetermined period of time is, for example, 12 hours, 24 hours, or 7 days.
(Step S202) When the operation of the emergency button 410 is detected in step S101, the communication device 30 stores in the communication device storage unit 390, based on a predetermined recording time, images and sounds from the most recent period among the past images and sounds stored in the communication device storage unit 390, and images and sounds captured and collected after the operation of the emergency button 410 is detected. For example, when the predetermined recording time is 10 minutes, the communication device 30 stores in the communication device storage unit 390 images and sounds from 5 minutes before and 5 minutes after the operation of the emergency button 410 is detected.

(Step S203) The terminal device 10 requests, from the communication device 30, images and audio before and after the operation of the emergency button 410 is detected. For example, the user taps the play button image on the video display screen P62 shown in Fig. 10. When the terminal device 10 detects a tap on the play button image on the video display screen P62, it requests, from the communication device 30, images and audio before and after the operation of the emergency button 410 is detected.
(Step S204) When the communication device 30 receives a request for images and sounds before and after the operation of the emergency button 410 was detected, it transmits to the terminal device 10 the images and sounds from the time period before and after the operation of the emergency button 410 was detected, which were saved in step S202.
(Step S205) When the terminal device 10 receives from the communication device 30 images and sounds from the time period before and after the operation of the emergency button 410 is detected, the terminal device 10 reproduces the received images and sounds.

The information indicating the operation of the emergency button 410 (first operation information) is an example of event information. The event information includes information indicating a video of a time range including the occurrence time of the event, and the presentation unit 120 presents the video. Here, the time range including the occurrence time of the event is, for example, 10 minutes, 5 minutes before and 5 minutes after the operation of the emergency button 410 described above. Note that the time width and the ratio of before and after shown here are examples, and may be longer or shorter than 10 minutes, and the time ratio before and after may be an equal ratio or an unequal ratio in which the before (or after) is longer. In addition, the information indicating the video may be the video file itself, or may be link information indicating the storage location of the video.

The monitoring system 1 configured in this manner can present the situation before the operation of the emergency button 410, that is, the situation during the time period when the cause that led to the operation of the emergency button 410 occurred, on the terminal device 10. This allows the user to understand the cause of the abnormality that occurred in the person being watched over, and enables more appropriate response.

[Display emergency button operation history]
12 is a diagram showing an example of an event history display according to the present embodiment. As described above, the terminal device 10 includes the presentation unit 120 and the terminal communication unit 140.
The terminal communication unit 140 receives event information regarding an event that has occurred in a place to be monitored. The event information includes, as an example, first operation information indicating that the emergency button 410 has been operated.
The presentation unit 120 presents a history of event information received by the device communication unit 140. As an example, the presentation unit 120 of the terminal device 10 displays, on an event history screen P71, events that occurred in the past among the events to be notified, together with the date and time of the events.
The monitoring system 1 configured in this manner is not only able to notify the user of the occurrence timing of an event (for example, operation of the emergency button 410), but is also able to present the user with events that have occurred in the past.
Furthermore, the terminal device 10 may have a function of excluding (i.e., deleting) an event selected by the user from the event history to be displayed.
The terminal device 10 may also display the occurrence timing of events in a calendar format, or display detailed information about events that occurred on a date selected from the calendar.

Here, the event information includes the time at which the event occurred, and the presentation unit 120 presents the history of the event information based on the time at which the event occurred.
Here, the presentation unit 120 can also be configured to present a history of event information based on the arrival time of the event information at the terminal device 10. However, since the event information is transmitted from the remote control device 40 to the terminal device 10 via the communication device 30 and the server device 20, a discrepancy may occur between the arrival time of the event information and the occurrence time of the event depending on the communication conditions, etc. Furthermore, for example, when multiple events occur, the order in which the events occur and the order in which the event information is received may be interchanged.
According to the monitoring system 1 configured to present the history of event information based on the time of event occurrence as in this embodiment, the event occurrence history can be presented more accurately regardless of the communication situation.

As described above, a communication device 30 equipped with an imaging unit 350 is installed in the monitored area A. Event information (in the above example, information indicating that the emergency button 410 has been operated; i.e., first operation information) transmitted to the terminal device 10 includes information on the image captured by the imaging unit 350 (e.g., a thumbnail image or a video). The presentation unit 120 presents this image information.

Furthermore, the terminal device 10 may mark an unopened event P72 with respect to an event whose contents have not been confirmed by the user, that is, an unopened event, among the event histories shown on the event history screen P71.
That is, the presentation unit 120 may present the fact that the event information has not been viewed by the user.

In addition, for an unopened event, if a predetermined time has passed since the event occurred and the event has remained unopened, the terminal device 10 may notify the user that the predetermined time has passed by changing the color of the unopened display P72, displaying a pop-up image, sounding a sound, vibrating, or the like.
That is, the presentation unit 120 presents that a predetermined time has elapsed since the event information was not viewed by the user.

In addition, the terminal device 10 may change the color or shape of the unopened display P72 depending on the type of event. In other words, the presentation unit 120 may present the event in different ways depending on the type of event indicated by the event information.

Figure 13 is a diagram showing an example of a selection screen for events to be presented in this embodiment. The terminal device 10 may narrow down notifications when an event occurs based on a user's setting operation on the event selection screen P81. The presentation unit 120 selects the event information to be presented based on at least one of the time of occurrence of the event and the type of the event.

The monitoring system 1 configured as described above makes it easier for users to grasp the situation when an event occurs, enabling users to respond to the event more appropriately.

[Monitoring function (2) Conversation function using a remote control device]
The monitoring system 1 provides a conversation function using the remote control device 40. For example, when the user taps the conversation icon P43 shown in Fig. 8 described above, a conversation can be held between the user and the person being watched over.

As described above, the remote control device 40 includes a remote control sound collection section 450. When a conversation request is received from the terminal device 10, the communication device control section 330 transmits information on the sound collected by the remote control sound collection section 450 to the terminal device 10 based on the second operation information. Here, the second operation information is information indicating that the OK button 420 has been operated. In other words, when the remote control device 40 detects an operation on the OK button 420, it transmits the second operation information.
More specific steps of the conversation function will be described below.

FIG. 14 is a diagram showing an example of communication of the conversation function of the present embodiment.
(Step S301) When detecting a tap on the conversation icon P43, the terminal device 10 transmits a conversation request to the communication device 30.
(Step S302) When the communication device 30 receives a conversation request from the terminal device 10, it issues a conversation request sound from the communication device sound output unit 370. The person being watched over hears the conversation request sound, knows that there has been a request for conversation, and decides whether or not to accept the conversation. If the person being watched over accepts the conversation, he or she operates the OK button 420 on the remote control device 40.
(Step S303) When remote control device 40 detects an operation on OK button 420, it notifies terminal device 10 of information indicating that OK button 420 has been operated via communication device 30 and server device 20. Here, the information indicating that OK button 420 has been operated is also referred to as second operation information. That is, when remote control device 40 detects an operation on OK button 420, it transmits the second operation information.

(Step S304) When the terminal device 10, the server device 20, the communication device 30, and the remote control device 40 detect that the conversation request has been accepted in step S303, they start a two-way conversation. Here, the two-way conversation function is a function that enables a two-way conversation between the user and the person being watched over by making the sound collected by the terminal sound collection unit 150 of the terminal device 10 sound from the communication device sound collection unit 370 of the communication device 30 and making the sound collected by the remote control sound collection unit 450 of the remote control device 40 sound from the terminal sound collection unit 160 of the terminal device 10. In the two-way conversation function, the monitoring system 1 may display an image of the person being watched over during a conversation captured by the imaging unit 350 on the terminal device 10.
(Step S305) The communication device 30 captures an image of the monitoring target area A using the imaging unit 350.
(Step S306) Remote control device 40 picks up the voice of the person being watched over by remote control sound collection unit 450.
(Step S307) Remote control device 40 transmits the picked up voice to communication device 30 as voice information.
(Step S308) The communication device 30 transmits to the terminal device 10 the image captured in step S305 and the audio information received from the remote control device 40 in step S307.
(Step S309) The terminal device 10 presents the image and sound received from the communication device 30 to the user.
(Step S310) In parallel with steps S305 to S309, the terminal device 10 transmits the voice picked up by the terminal sound pickup unit 150 to the communication device 30 as voice information.
(Step S311) The communication device 30 receives voice information from the terminal device 10 and produces sound from the sound output unit 370 of the communication device.

According to the monitoring system 1 configured as described above, the voice of the person being watched over is picked up using the remote control sound pickup unit 450 of the remote control device 40 carried by the person being watched over, so the voice can be picked up more clearly than, for example, when the voice is picked up using the communication device sound pickup unit 360 of the communication device 30. Also, according to the monitoring system 1, the voice is pronounced using the communication device sound pickup unit 370 of the communication device 30, so the voice can be pronounced louder or with better sound quality than, for example, when the remote control sound pickup unit 460 of the remote control device 40 is used. Therefore, according to the monitoring system 1, the quality of the conversation between the user and the person being watched over can be improved.

As described above, the remote control device 40 has an OK button 420. This OK button 420 is a different operation button from the emergency button 410. The remote control device 40 transmits second operation information indicating the operation of the OK button 420. The communication device control unit 330 of the communication device 30 controls the type of information to be transmitted to the terminal device 10 based on receiving the second operation information from the remote control device 40. In the case of the above example, when the OK button 420 is operated, the communication device control unit 330 controls the type of information to be transmitted to the terminal device 10 by transmitting an image and sound to the terminal device 10.

[Monitoring function (3) Live View function]
The monitoring system 1 provides a live view function that presents, in real time, current images and sounds of the monitoring target area A to the terminal device 10. In the live view function, the terminal device 10 displays, for example, the live view image P41 shown in Fig. 8 described above.
In the conversation function described above, remote control sound collection section 450 of remote control device 40 collects sound, but in the live view function, communication device sound collection section 360 of communication device 30 collects sound, which is different.
Furthermore, the live view function operates differently in private mode and in public mode. In private mode, a permission procedure is required from the person being watched. An example of the procedure is described below.

<Live View function: Private mode>
FIG. 15 is a diagram showing an example of communication of the live view function in the private mode of the present embodiment.
(Step S401) When the terminal device 10 detects a user operation requesting a live view function (for example, selection of the “Live” tab or the “Current Status” tab), the terminal device 10 transmits a live view request to the communication device 30.
(Step S402) When the communication device 30 receives a live view request from the terminal device 10, it issues a live view request sound from the communication device sound output unit 370. The person being watched over hears the live view request sound, knows that a live view has been requested, and decides whether or not to permit the live view. If the person being watched over permits the live view, he or she operates the OK button 420 on the remote control device 40.
(Step S403) When remote control device 40 detects an operation on OK button 420, it notifies terminal device 10 of information indicating that OK button 420 has been operated via communication device 30 and server device 20. Here, the information indicating that OK button 420 has been operated is also referred to as second operation information. That is, when remote control device 40 detects an operation on OK button 420, it transmits the second operation information.
In the above-described conversation function, remote control device 40 also transmits the second operation information.
Depending on the requested function, the meaning of the operation of the OK button 420, that is, the meaning of the second operation information, changes.

(Step S404) The communication device 30 captures an image of the monitored area A using the imaging unit 350. The communication device 30 also collects sounds in the monitored area A using the communication device sound collection unit 360.
(Step S405) The communication device 30 transmits the image and audio information to the terminal device 10.
(Step S406) The terminal device 10 presents the image and sound received from the communication device 30 to the user.

<Live View function: Public mode>
16 is a diagram showing an example of communication of the live view function in the public mode of this embodiment. In the public mode, the communication device 30 constantly captures images of the monitored area A by the imaging unit 350.

(Step S501) When the terminal device 10 detects a user operation requesting a live view function (for example, selection of the “Live” tab or the “Current Status” tab), the terminal device 10 transmits a live view request to the communication device 30.
(Step S502) When the communication device 30 receives a live view request from the terminal device 10, it issues a live view request sound from the communication device sound output unit 370. The person being watched over hears the live view request sound, knows that a live view has been requested, and decides whether or not to permit the live view. If the person being watched over permits the live view, he or she operates the OK button 420 on the remote control device 40.
(Step S503) When remote control device 40 detects an operation on OK button 420, it notifies terminal device 10 via communication device 30 and server device 20 of information indicating that OK button 420 has been operated.

(Step S504) The communication device 30 picks up sounds in the monitoring area A using the communication device sound pickup unit 360.
(Step S<b>505 ) The communication device 30 transmits the image and audio information to the terminal device 10 .
(Step S506) The terminal device 10 presents the image and sound received from the communication device 30 to the user.

The live view request is also called an image transmission request. That is, when an image transmission request is received from the terminal device 10, the communication device control unit 330 of the communication device 30 transmits the image captured by the imaging unit 350 to the terminal device 10 based on the second operation information.

Here, the differences in operation of the monitoring system 1 between public mode and private mode are explained with reference to the figure.

Fig. 17 is a diagram showing an example of the operation of the surveillance system 1 in the public mode and the private mode of this embodiment. In Fig. 17, "OFF" indicates a state in which the button is not operated, and "ON" indicates a state in which the button is operated.
Regarding images, in the public mode, regardless of the operation of the emergency button 410 and the OK button 420, the image captured by the imaging unit 350 is presented to the terminal device 10. On the other hand, in the private mode, when the emergency button 410 is operated, or when the OK button 420 is operated in response to a conversation request or a live view request (i.e., when the person being watched over gives permission), the image captured by the imaging unit 350 is presented to the terminal device 10. That is, in the private mode, when neither the emergency button 410 nor the OK button 420 is operated (when they are OFF), the image is not presented.
Regarding audio, in both the public mode and the private mode, when the emergency button 410 is operated or when the OK button 420 is operated in response to a conversation request or a live view request (i.e., when the person being watched over gives permission), the collected audio is presented on the terminal device 10. In other words, regarding audio, in both the public mode and the private mode, when neither the emergency button 410 nor the OK button 420 is operated (when both are OFF), the audio is not presented.

Regarding audio, in both public and private modes, when the emergency button 410 is operated, the sound picked up by the communication device sound pickup unit 360 of the communication device 30 is presented. Also, in both public and private modes, when the OK button 420 is operated (i.e., when the person being watched over gives permission), in the case of a conversation request, the sound picked up by the remote control sound pickup unit 450 is presented, and in the case of a live view request, the sound picked up by the communication device sound pickup unit 360 is presented.

As described above, the meaning of the information indicating the operation of the OK button 420, that is, the second operation information, changes depending on the content of the request from the user and the operation mode.
The second operation information is information indicating an operation that permits at least one of information about the picked-up sound and the captured image to be provided to the terminal device 10 in a private mode in which the image captured by the imaging unit 350 is not made public.

In addition, the second operation information is information indicating an operation that allows information about the picked-up sound to be provided to the terminal device 10 in a public mode in which the image captured by the imaging unit 350 is made public.

[Monitoring function (4) Voltage drop notification function for remote control device]
FIG. 18 is a diagram showing an example of communication in the voltage drop notification function of the present embodiment.
(Step S601) Voltage detection unit 470 of remote control device 40 detects the power supply voltage of remote control device 40. If the detected power supply voltage is equal to or lower than a predetermined value, remote control control unit 440 transmits a voltage drop notification to terminal device 10. That is, when communication device control unit 330 receives voltage drop information from remote control device 40, it transmits information indicating that the power supply voltage of remote control device 40 has dropped to terminal device 10.
(Step S602) The terminal device 10 receives the voltage drop notification.
(Step S603) The terminal device 10 notifies the user of the voltage drop.

19 is a diagram showing an example of a voltage drop notification screen according to the present embodiment. In step S603, the terminal device 10 displays a voltage drop notification screen P91 shown in FIG.
When the power supply voltage of remote control device 40 is restored by replacing or charging the battery, remote control device 40 may transmit a voltage recovery notification to communication device 30 and terminal device 10. Communication device 30 and terminal device 10 may manage the period from when they receive a voltage drop notification to when they receive a voltage recovery notification as the power supply voltage drop period of remote control device 40.

[Monitoring function (5) Remote control device search function]
The monitoring system 1 provides a remote control device search function that, when the remote control device 40 is lost, causes the remote control device 40 to ring to assist in searching for the remote control device 40.

FIG. 20 is a diagram showing an example of communication in the remote control device search function of this embodiment.
(Step S701) The communication device 30 includes a remote control device search switch (not shown). The communication device 30 detects an operation of the remote control device search switch. The remote control device search switch is an example of an operation receiving unit that receives an operation to search for the remote control device 40.
(Step S702) When the operation of the remote control device search switch is detected, the communication device 30 transmits a remote control device search instruction to the remote control device 40.
(Step S703) When remote control device 40 receives a remote control device search instruction from communication device 30, it emits a sound from remote control sound unit 460 to notify the person being watched over or the user of its location.

In addition, the remote control device 40 may be equipped with a vibration unit (not shown) or a light-emitting unit (not shown), and when a remote control device search instruction is received from the communication device 30, it may notify its location by vibration or light.

[Monitoring function (6) Remote control device out of range notification function]
The monitoring system 1 provides a remote control device out-of-service notification function that notifies the terminal device 10 when the remote control device 40 is out of service area with the communication device 30 .

FIG. 21 is a diagram showing an example of communication in the remote control device out-of-service notification function of this embodiment.
(Step S801) The communication device 30 determines whether or not communication with the remote control device 40 is possible. The communication device 30 may perform the determination of whether or not communication is possible periodically, or may perform the determination in response to a request from the terminal device 10. When the communication device 30 is unable to communicate with the remote control device 40, it determines that the remote control device 40 is out of communication range, and transmits remote control out-of-range information to the terminal device 10.
In addition, by combining with the voltage drop notification function of the remote control device described above, communication device 30 may determine that remote control device 40 is outside the communication range when the power supply voltage of remote control device 40 has not dropped and communication with remote control device 40 is not possible.

(Step S802) The terminal device 10 receives remote control out-of-range information from the communication device 30.
(Step S803) Terminal device 10 notifies the user that remote control device 40 is out of the communication range of communication device 30.

As described above, the monitoring system 1 can provide a monitoring system that is highly convenient for both monitors and non-monitors.

Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and appropriate modifications can be made without departing from the spirit of the present invention. The configurations described in each of the above-mentioned embodiments may be combined.

In addition, each unit of each device in the above embodiments may be realized by dedicated hardware, or may be realized by a memory and a microprocessor.

In addition, each part of each device may be composed of a memory and a CPU (central processing unit), and the functions of each part of each device may be realized by loading a program for realizing the function of the part into memory and executing the program.

In addition, a program for realizing the functions of each part of each device may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed to perform processing by each part of the control unit.
It should be noted that the term "computer system" herein includes hardware such as the OS and peripheral devices.

Furthermore, if the "computer system" utilizes a WWW system, it also includes the home page providing environment (or display environment).
In addition, "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Furthermore, "computer-readable recording medium" also includes devices that dynamically store a program for a short period of time, such as communication lines when transmitting a program via a network such as the Internet or a communication line such as a telephone line, and devices that store a program for a certain period of time, such as volatile memory inside a computer system that serves as a server or client in such cases. Furthermore, the above-mentioned program may be one that realizes part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

1...Monitoring system, 10...Terminal device, 110...Terminal operation detection unit, 120...Presentation unit, 130...Terminal control unit, 140...Terminal communication unit, 150...Terminal sound collection unit, 160...Terminal sound generation unit, 30...Communication device, 310...First communication unit, 320...Second communication unit, 330...Communication device control unit, 340...Measurement unit, 350...Imaging unit, 360...Communication device sound collection unit, 370...Communication device sound generation unit, 380...Operation detection unit, 390...Communication device storage unit, 40...Remote control device, 410...Emergency button, 420...OK button, 430...Remote control communication unit, 440...Remote control unit, 450...Remote control sound collection unit, 460...Remote control sound generation unit, 470...Voltage detection unit, 480...NG button

Claims (16)

A receiver for receiving event information relating to an event occurring at a monitored location;
a presentation unit that presents a history of the event information received by the receiving unit;
Equipped with
There are a plurality of locations to be monitored, and user authority can be set for each of the locations to be monitored;
The presentation unit presents content based on the authority set for the user for each of the locations to be monitored.
Display control device. The event information includes a time when the event occurred,
The display control device according to claim 1 , wherein the presentation unit presents the history of the event information based on an occurrence time of the event. an imaging device is installed at a location to be monitored, and the event information includes information on an image captured by the imaging device;
The display control device according to claim 1 , wherein the presentation unit presents information about the image. the event information includes information indicating a video of a time range including an occurrence time of the event,
The display control device according to claim 3 , wherein the presentation unit presents the video. The display control device according to claim 2 , wherein the presentation unit selects the event information to be presented based on at least one of an occurrence time of the event and a type of the event. The display control device according to claim 1 , wherein the presenting unit presents a message that the event information has not been viewed by a user. The display control device according to claim 1 , wherein the presenting unit presents a notification that a predetermined time has elapsed since the event information was not viewed by a user. The presentation unit presents the event in a different manner depending on the type of the event indicated by the event information.
The display control device according to claim 7. The event information is transmitted to a plurality of users,
The display control device according to claim 1 , wherein the presentation unit presents a status of a response to the event information by any one of the plurality of users. The presentation unit presents a screen for selecting a plurality of the responses,
The display control device according to claim 9 , further comprising: a display control unit configured to display a display image of the display screen of the display device; The display control device according to claim 10 , wherein the response includes at least one of moving the user to a location to be monitored, notifying a public institution, and notifying other users. Each of the users is given one of at least two levels of operation authority;
The display control device according to claim 1 , wherein the presentation unit presents content based on an operation authority given to a user. The event information includes a type of the event,
The display control device according to claim 1 , wherein the presentation unit presents content according to a type of the event included in the received event information. 3. The display control device according to claim 1, wherein the presentation unit, when the receiving unit receives the event information, pops up an icon indicating the event information, and presents content according to a type of the event in response to an operation by the user after the icon is displayed. A detection unit that detects a monitoring target;
a control unit that determines an occurrence of an event based on a detection result of the detection unit, and when it is determined that the event has occurred, outputs event information indicating the occurrence of the event;
a receiving unit that receives the event information output from the control unit;
a presentation unit that presents a history of the event information received by the receiving unit ;
There are a plurality of monitoring targets, and user authority can be set for each of the monitoring targets.
The presentation unit presents content based on the authority set for the user for each of the monitoring targets.
Surveillance system. The detection unit has an imaging unit that images the monitoring target, and a determination unit that determines a state of the monitoring target based on an image captured by the imaging unit,
The monitoring system according to claim 15 , wherein the control unit determines the occurrence of the event based on the state determined by the determination unit.

Images