JP7821965B2 - Surveillance equipment and surveillance system - Google Patents

Description

The present invention relates to a monitoring device and monitoring system that detects events that require notification based on camera images of a monitored area within a facility and instructs a notification device to perform a specified notification operation.

At railway stations, passengers sometimes fall from the platform onto the tracks, and it is important to detect such accidents early and quickly implement the necessary measures, such as operating an emergency stop or conducting rescue operations.

There is a known technology that uses camera images of the platform and tracks to detect a person who has fallen from the platform onto the tracks and notify station staff that a person has fallen (see Patent Document 1). This technology also detects workers performing work on the tracks and identifies whether the person on the tracks is a worker or someone who has fallen.

Japanese Patent Application Laid-Open No. 2003-276605

Now, when a person falls from the platform onto the tracks, the response of staff will vary depending on the condition of the person who fell. For example, if the person is unable to move on the tracks, it is a high urgency and staff need to rush to the scene. On the other hand, if the person who fell immediately manages to climb back onto the platform on their own, it is less urgent and there may be no need to issue an alarm in the first place.

On the other hand, conventional technology simply uses background subtraction to determine whether an object has newly appeared on the tracks, and distinguishes between the person who fell and the worker, thereby uniformly notifying station staff that a person has fallen. This poses the problem of staff being unable to immediately grasp the situation of the person who fell.

The primary objective of the present invention is to provide a monitoring device and monitoring system that allows staff to instantly grasp the situation of a person who enters a detection area and quickly take appropriate action according to the person's situation.

The monitoring device of the present invention is a monitoring device equipped with a processor that acquires camera images taken by cameras installed in each specified monitoring area within a facility, detects events that should be reported for each camera, and instructs an alarm device to perform a specified alarm operation.The processor is configured to set, in advance, for each camera, information regarding the posture of a person entering a detection area set within the monitoring area, as well as at least one of the person's entry status into the detection area, status of stay in the detection area, and movement status, as detection conditions for events that should be reported, based on the operation of an administrator from a management terminal.When a person entering the detection area is detected from the camera image, the processor determines whether the detected person matches the detection conditions for the event that should be reported, and if it determines that the detected person matches the detection conditions, it controls the alarm operation of multiple different types of alarm devices in accordance with a specified alarm procedure corresponding to the event that should be reported.

The surveillance system of the present invention is configured to include the surveillance device, multiple cameras that capture images of the surveillance area, and multiple alarm devices that perform predetermined alarm operations in response to instructions from the surveillance device.

According to the present invention, an event for which a notification should be issued can be detected in a manner that allows the person's status to be identified based on the detection conditions for the event, which include information on the person's posture as well as at least one of the person's entry status into the detection area, the person's stay status in the detection area, and the person's movement status. Furthermore, notifications are issued by multiple different types of notification devices according to notification procedures corresponding to the detected event. This allows an attendant to immediately grasp the status of a person who has entered the detection area and quickly take appropriate action according to the person's status.

1 is a diagram illustrating the overall configuration of a monitoring system according to an embodiment of the present invention. An explanatory diagram showing an overview of the processing performed by the monitoring server Block diagram showing the general configuration of the monitoring server An explanatory diagram showing the settings screen displayed on the management terminal FIG. 10 is an explanatory diagram showing a notification screen displayed on a mobile terminal; FIG. 10 is an explanatory diagram showing a notification screen displayed on a fixed terminal in a facility. An explanatory diagram showing a notification screen displayed on a fixed terminal in the control center. Flow diagram showing the processing steps performed by the monitoring server

The first invention made to solve the above problem is a monitoring device equipped with a processor that acquires camera images taken by cameras installed in each specified monitoring area within a facility, detects events that should be reported for each camera, and instructs an alarm device to perform a specified alarm operation.The processor is configured to set, in advance, for each camera, information regarding the posture of a person entering a detection area set within the monitoring area, as well as at least one of the person's entry status into the detection area, status of stay in the detection area, and movement status, as detection conditions for events that should be reported, based on operation by an administrator from a management terminal.When a person entering the detection area is detected from the camera image, it determines whether the detected person matches the detection conditions for the event that should be reported, and if it determines that the detection conditions are met, it controls the alarm operation of multiple different types of alarm devices in accordance with a specified alarm procedure corresponding to the event that should be reported.

This allows events to be detected based on the detection conditions for an event that requires a notification, including not only the person's posture but also information on at least one of the person's entry status into the detection area, their stay status in the detection area, and their movement status, so that the person's status can be identified. Furthermore, notifications are issued by multiple different types of notification devices according to notification procedures corresponding to the detected events. This allows staff to immediately grasp the status of a person who has entered the detection area and quickly take appropriate action according to the person's status.

This allows for accurate detection of events that require an alert. Information regarding the entry status may be, for example, the direction from which the target person entered the detection area, information regarding the stay status may be, for example, the amount of time the target person stayed in the detection area, and information regarding the movement status may be, for example, whether or not the target person moved.

In a second aspect of the present invention, the processor is configured to recognize any one of a standing position, a crouching position, a face-down position, and a face-up position as a posture of a person included in the detection conditions .

This allows for accurate detection of events that require an alert based on the posture of the target person.

In addition, a third invention is configured such that the processor recognizes the posture of a person included in the detection conditions by using an image recognition model constructed by machine learning using the camera images as learning data.

This allows the posture of a person to be recognized with high accuracy using an image recognition model (classifier). Note that the process of detecting a person from a camera image may also be performed using an image recognition model. Furthermore, the process of detecting a person from a camera image and the process of recognizing the posture of a person may be performed together using an image recognition model.

In a fourth aspect of the present invention, the processor causes the notification device to display an image that allows the occurrence status of an event for each of the cameras to be identified.

This allows facility staff to instantly grasp the status of an event occurring on each camera. In this case, for example, a notification screen that notifies facility staff that a specific event has occurred may display a status mark for each camera that changes form (color, etc.) depending on the status of the event, as an image that makes it possible to identify the status of the event occurring on each camera.

In a fifth aspect of the present invention, the processor causes the notification device to display an image that allows the occurrence status of an event for each facility to be identified.

This allows staff in the control department who oversee the facilities to immediately understand the status of events occurring at each facility. In this case, for example, on a notification screen that notifies staff in the control department that a specific event has occurred, the status of the event occurring at each facility can be displayed as an image that makes it possible to identify the status of the event, and the operation unit (e.g., tab) for selecting the facility can be displayed in a way that changes form (e.g., color) depending on the status of the event.

In addition, in a sixth invention, the processor is configured to control the notification operations of multiple different types of notification devices, including a mobile terminal device carried by an attendant, a fixed terminal device installed in the facility, and a fixed terminal device installed in a department that oversees multiple facilities.

As a result, if the detected event is highly urgent, a notification can be sent via a mobile terminal device, allowing staff to rush to the scene and take appropriate action quickly. Furthermore, if the detected event is less urgent, a notification can be sent via a fixed terminal device, reducing unnecessary notifications.

A seventh invention is a surveillance system comprising the surveillance device, a camera that photographs the specified surveillance area , and a plurality of alarm devices that perform specified alarm operations in response to instructions from the surveillance device.

As with the first invention, this allows staff to instantly grasp the situation of a person who has entered the detection area and quickly take appropriate action according to the person's situation.

Embodiments of the present invention will be described below with reference to the drawings.

Figure 1 is a diagram showing the overall configuration of a monitoring system according to this embodiment.

This monitoring system detects events that require an alert, such as a person falling from the platform onto the tracks at a railway station (facility) or a person entering the tracks without permission (an intruder), and notifies staff of the incident.

This monitoring system comprises a camera 1, a recorder 2 (recording device), a monitoring server 3 (monitoring device), a management terminal 4, a mobile terminal 5 (alarm device, mobile terminal device), an in-facility fixed terminal 6 (alarm device, fixed terminal device), a command center fixed terminal 7 (alarm device, fixed terminal device), and a speaker 8. The camera 1, recorder 2, monitoring server 3, management terminal 4, mobile terminal 5, in-facility fixed terminal 6, command center fixed terminal 7, and speaker 8 are interconnected via a network.

Camera 1 is installed in each designated monitoring area within the station. Camera 1 captures images of the platform and the tracks along the platform as its monitoring area. Camera 1 transmits camera images of the monitoring area to recorder 2 and monitoring server 3 via the network.

Recorder 2 acquires and stores camera images from camera 1.

The monitoring server 3 is installed in an equipment room, data center, or other location within the station. The monitoring server 3 acquires camera images from the camera 1 and detects events that warrant an alert, such as a person falling from the platform onto the tracks (a fallen person) or a person entering the tracks without permission (an intruder). When the monitoring server 3 detects an event that warrants an alert, it notifies staff of the event. Specifically, it instructs the mobile terminal 5, the in-facility fixed terminal 6, the control center fixed terminal 7, and the speaker 8 to perform a specified alert operation. Note that in the example shown in Figure 1, a monitoring server 3 is provided for each station (facility), but the functions of the monitoring server 3 may also be implemented by a cloud computer.

The management terminal 4 is operated by the administrator and is configured as a PC or the like. The management terminal 4 displays a settings screen (see Figure 4) on which the administrator performs various setting operations.

The mobile terminal 5 is a mobile terminal such as a smartphone or tablet. The mobile terminal 5 is carried by a staff member who rushes to the scene and takes the necessary measures. In response to instructions from the monitoring server 3, the mobile terminal 5 displays a notification screen (see Figure 5) and outputs an alarm sound or vibration to notify the staff member that an event requiring a notification has occurred.

The in-facility fixed terminal 6 is installed in an office or the like within the station premises. The in-facility fixed terminal 6 can be implemented by installing a notification application on a PC. In response to instructions from the monitoring server 3, the in-facility fixed terminal 6 displays a notification screen (see Figure 6) as a notification operation to notify station staff that a specific event has occurred.

The control center fixed terminal 7 is installed in a control center (management center). The control center is a department that oversees multiple stations (facilities) within the line section under its jurisdiction. The control center fixed terminal 7 can be implemented by installing a notification application on a PC. In response to instructions from the monitoring server 3, the control center fixed terminal 7 displays a notification screen (see Figure 7) as a notification operation to notify the control center staff that a specific event has occurred.

The speaker 8 is installed near the camera 1. In response to instructions from the monitoring server 3, the speaker 8 outputs sounds such as an alarm or spoken voice as a notification to notify station staff that a specific event has occurred.

In this embodiment, an example has been described in which the facility to be monitored is a train station, but the facility to be monitored is not limited to a train station. The facility to be monitored may also be, for example, an airport or a retail store.

Next, we will explain the overview of the processing performed by the monitoring server 3. Figure 2 is an explanatory diagram showing the overview of the processing performed by the monitoring server 3.

In this embodiment, a detection area is set on the camera image, and the occurrence of an event that requires an alert (alert event) is detected based on the person's entry into this detection area. At this time, if predetermined detection conditions are met regarding the person who has entered the detection area, it is determined that an event that requires an alert has occurred, and an alert is issued.

Here, if a person falls from the platform onto the tracks and is unable to move, the situation is highly urgent and staff need to rush to the scene. On the other hand, if a person also falls onto the tracks but quickly climbs back up onto the platform on their own, the situation is less urgent and there may be no need to issue an alert in the first place. Also, if a worker is working on the tracks, there is no need for staff to rush to the scene, but it is desirable to issue an alert to alert staff in the office within the station. Also, in the case of an intruder entering the tracks without permission, staff need to rush to the scene, but the urgency is lower than in the case of a person who has fallen.

In this embodiment, the posture of the target person is set as a detection condition. Specifically, the detection condition is set to be a posture other than standing, i.e., crouching, lying face down, or lying on one's back. Also, the detection condition is set to be that the target person stays within the detection area for a predetermined period of time or longer. Also, the detection condition is set to be that the target person does not move.

In the example shown in Figure 2 (A-1), the target person remains in a posture other than standing within the detection area for a predetermined period of time or more, and does not move. This satisfies the detection conditions related to posture, duration of stay, and movement, and it is determined that an event requiring an alert has occurred, and an alert is issued. On the other hand, in the example shown in Figure 2 (A-2), the target person changes from a posture other than standing to a standing posture, but does not remain in a posture other than standing for a predetermined period of time or more. This does not satisfy the detection conditions related to posture, duration of stay, and movement, and it is not determined that an event requiring an alert has occurred, and an alert is not issued.

Furthermore, the nature of the event that occurs will differ depending on the direction from which the person entered the detection area, i.e., depending on where the person was immediately before entering the detection area. For example, if a person falls from a platform onto the tracks, they will cross the boundary between the platform and the tracks and enter the tracks, which form the detection area.

In this embodiment, the detection condition is set to the expected direction of a person's entry into the detection area when a specified event occurs. Specifically, as shown in Figures 2 (B-1) and (B-2), a judgment line is set on the boundary line of the detection area (here, the vertical line on the left) that a person is expected to cross when a specified event occurs, and only when a person crosses the judgment line when entering the detection area is it determined that a specified event has occurred and an alert is issued.

In the example shown in Figure 2 (B-1), the target person crossed the judgment line to enter the detection area, so the detection conditions for the direction of entry were met, it was determined that an event requiring an alert had occurred, and an alert was issued. On the other hand, in the example shown in Figure 2 (B-2), the target person entered the detection area without crossing the judgment line, so the detection conditions for the direction of entry were not met, it was not determined that an event requiring an alert had occurred, and an alert was not issued.

Next, we will explain the general configuration of the monitoring server 3. Figure 3 is a block diagram showing the general configuration of the monitoring server 3.

The monitoring server 3 includes a communication unit 11, a memory 12, and a processor 13.

The communication unit 11 communicates with the camera 1, recorder 2, management terminal 4, mobile terminal 5, in-facility fixed terminal 6, command center fixed terminal 7, and speaker 8 via the network.

Memory 12 stores programs executed by processor 13, etc.

The processor 13 performs various processes by executing programs stored in the memory 12. In this embodiment, the processor 13 performs image acquisition processing, person detection processing, entry determination processing, posture determination processing, entry direction determination processing, stay determination processing, movement determination processing, event detection processing, alert procedure selection processing, and notification processing.

In the image acquisition process, the processor 13 acquires camera images received from each camera 1 via the communication unit 11. Camera 1 transmits camera images of the monitoring area captured at a predetermined frame rate (e.g., 5 fps).

In the person detection process, processor 13 detects people from camera images. This person detection process can be performed using an image recognition model (machine learning model) constructed using machine learning such as deep learning, using camera images as training data. Here, processor 13 also performs a process of tracking people, that is, a process of matching the same people detected from camera images (frames) at each time (person tracking process).

The entry determination process determines whether a target person has entered the detection area based on the position of the target person detected in the person detection process.

In the posture determination process, the processor 13 recognizes the posture of the target person (standing, crouching, prone, supine, etc.) based on the person image extracted from the camera image (posture recognition process), and determines whether the recognized posture matches the detection conditions set for each event for which an alert should be issued. This posture recognition process can be performed using an image recognition model (machine learning model) constructed using machine learning such as deep learning, using the camera images as learning data. Note that the person detection process and posture detection process may also be performed at the same time using the same image recognition model.

In the approach direction determination process, processor 13 identifies the determination line that the target person crossed when entering the detection area as the target person's approach direction into the detection area based on the tracking information of the target person acquired in the person detection process, and determines whether the determination line matches the detection conditions set for each event that should be reported.

In the stay determination process, processor 13 obtains the time the target person has been staying in the detection area (stay time), i.e., the elapsed time from the time entry into the detection area was detected to the current time, and determines whether the target person's stay time matches the detection conditions set for each event for which an alert should be issued.

In the movement determination process, the processor 13 determines whether the movement of the target person matches the detection conditions for an event that should trigger an alert. In this movement determination process, if the target person's position remains unchanged for a predetermined period of time or longer, it is determined that there is no movement. Also, if both the target person's position and posture remain unchanged for a predetermined period of time or longer, it is determined that there is no movement. It may also be determined that there is no movement if the change in the target person's position per predetermined period of time is extremely small (for example, less than 30 cm).

In the event detection process, the processor 13 detects the occurrence of an event that warrants an alert (such as a person falling, an intruder, or a worker) based on the results of the posture determination process, the approach direction determination process, the stay determination process, and the movement determination process.

In the alert procedure selection process, the processor 13 selects an alert procedure corresponding to the event detected in the event detection process. In this embodiment, an alert procedure is set for each event that should be alerted.

In the notification process, the processor 13 instructs the mobile terminal 5, the in-facility fixed terminal 6, and the command center fixed terminal 7 to perform a notification operation to notify staff that a specified event has occurred. At this time, the processor 13 causes at least one of the mobile terminal 5, the in-facility fixed terminal 6, and the command center fixed terminal 7 to perform the notification operation in sequence, according to the notification procedure selected in the notification procedure selection process. As a notification operation, the mobile terminal 5, the in-facility fixed terminal 6, and the command center fixed terminal 7 display a specified notification screen (see Figures 5, 6, and 7).

In addition, in the notification process, the processor 13 instructs the speaker 8 to perform a notification operation to notify staff that a specified event has occurred. The speaker 8 outputs a sound such as an alarm or spoken voice as the notification operation. At this time, the following notification procedure may be selected as appropriate: a notification procedure in which only speakers 8 installed near the location where the event occurred perform the notification operation; a notification procedure in which all speakers 8 installed on the platform facing the location where the event occurred perform the notification operation; or a notification procedure in which all speakers 8 installed within the station (facility) premises perform the notification operation.

Next, we will explain the settings screen 21 displayed on the management terminal 4. Figure 4 is an explanatory diagram showing the settings screen 21 displayed on the management terminal 4.

A settings screen 21 is displayed on the management terminal 4. This settings screen 21 is displayed for each camera 1, allowing the administrator to perform operations related to various settings for each camera 1.

The setting screen 21 has a camera image display section 22. The camera image display section 22 displays a camera image 31.

The camera image display unit 22 also displays the detection area 32. Specifically, a frame image representing the range of the detection area 32 is displayed in a predetermined color (e.g., blue) on the camera image 31. This detection area 32 is the area that is the target for detecting an event that requires an alert, and it is determined whether a predetermined event has occurred regarding a person who enters this detection area 32. The camera image display unit 22 also displays the names (L1 to L5) of the determination lines that make up the boundary line of the detection area 32.

A mask area 33 is also displayed on the camera image display unit 22. Specifically, a frame image of the mask area 33 is displayed in a predetermined color (e.g., red) on the camera image 31. This mask area 33 is an area in which human detection is not performed, and is set in places where there is little chance of a human being being present, such as on a roof, or on a wall surface where there is a high chance of a human being being mistakenly detected, such as when a poster is posted.

The settings screen 21 also has an input field 23 for person detection. This input field 23 for person detection has an input field 41 for the range of the detection area, an input field 42 for the range of the mask area, and an input field 43 for the threshold value for the person detection process.

In the input field 41 for the range of the detection area, the administrator can specify the range of the detection area as a polygon. Specifically, the administrator can input the positions of multiple vertices of the polygonal detection area by using a pointing device to specify them on the camera image 31 on the camera image display unit 22. The administrator can also use the keyboard to directly input the coordinate values of each vertex into the input unit 44 (input box).

In the input field 42 for the range of the mask area, the administrator can specify the range of the mask area as a polygon, just like the detection area. Specifically, the administrator can input the positions of multiple vertices of the polygonal detection area by using a pointing device to specify them on the camera image 31 on the camera image display unit 22. The administrator can also directly input the coordinate values of each vertex into the input unit 45 using a keyboard.

In the input field 43 for the threshold value for person detection processing, the administrator can use the keyboard to directly input the threshold value for person detection processing into the input unit 46. The threshold value for person detection processing is used to determine a person based on the score (a numerical value indicating the likelihood of being a person) obtained in the person detection processing, and a score equal to or greater than the threshold value is determined to be a person.

The settings screen 21 also has an input field 24 for detection conditions. This input field 24 for detection conditions includes an input field 51 for the person's posture, an input field 52 for the person's approach direction, an input field 53 for the person's stay time, and an input field 54 for the person's movement.

In this detection condition input field 24, the administrator can set multiple detection conditions for events that should be reported by combining detection conditions related to the person's posture, entry direction, stay time, and movement. In the example shown in Figure 4, detection conditions are set for two events, #1 and #2. Event #1 is the case of a person falling, and event #2 is the case of a worker.

In the posture input field 51, the administrator can specify standing, crouching, prone, and supine as the person's posture. The administrator can also specify crouching, prone, and supine as a group of postures other than standing. In the example shown in FIG. 4, the administrator can select the person's posture using a pull-down menu in the input unit 55. A multiple selection button 56 is also provided, and when the administrator operates this multiple selection button 56, a selection screen is displayed where multiple postures can be selected using check boxes.

In the example shown in Figure 4, event #1 is the case of a person who has fallen. In this case, unless the person is able to immediately climb back up onto the platform on their own, the person will be in a position other than standing and will not be moving, so "other than standing" is selected as the posture detection condition. On the other hand, event #2 is the case of a worker. In this case, the person will be working in a standing position, so "standing" is selected. Note that if there is a possibility that the worker will be working while crouching, "standing" and "crouching" are selected.

The detection conditions for a person's posture differ depending on the location being monitored. For example, in locations such as roads and escalators where a posture other than standing is unusual, the detection condition for a person's posture should be "other than standing," as in the case of a person falling off train tracks. On the other hand, there are also locations where a standing posture is unusual, such as waiting rooms with chairs, and in such locations the detection condition for a person's posture should be "standing."

In the entry direction input field 52, the administrator can specify the judgment line that the target person will cross when entering the detection area as the direction of entry of the target person into the detection area. In the example shown in Figure 4, the administrator can select the judgment line using a pull-down menu in the input unit 57. In addition, a multiple selection button 58 is provided, and when the administrator operates this multiple selection button 58, a selection screen is displayed where multiple judgment lines can be selected using check boxes.

In the example shown in Figure 4, the range of the detection area 32 is specified as a pentagon on the camera image 31 of the camera image display unit 22, and five judgment lines, L1 to L5, are set on the boundary lines of the detection area 32. Event #1 is the case of a person falling, and in this case, the person crosses the judgment line L3, which is located on the boundary line between the platform and the tracks in the camera image, to enter the detection area, but does not cross any other judgment lines to enter the detection area, so "L3" is selected. On the other hand, event #2 is the case of a worker, and in this case, the person could potentially cross all judgment lines to enter the detection area, so "all" is selected.

In the input field 53 for stay time, the administrator can specify a threshold value for determining the length of time the target person stays in the detection area (stay time). If the person stays in the detection area for only a short time, there is no need to issue an alert, so a threshold value for determining stay time is set, and the detection condition is that the target person's stay time is equal to or greater than the threshold value. In the example shown in Figure 4, the administrator can select the threshold value for determining stay time using a pull-down menu in the input unit 59. Note that it is also possible to directly enter values into the input unit 59 using a keyboard, etc.

Here, in the example shown in Figure 4, event #1 is the case of a person falling, and event #2 is the case of a worker, and in both cases "10 seconds or more" is selected.

In the movement input field 54, the administrator can specify whether or not the target person is moving as a detection condition. In the example shown in Figure 4, the administrator can select whether or not the target person is moving using a pull-down menu on the input unit 60.

In the example shown in Figure 4, event #1 is a case of a person falling off a platform. In this case, the person will remain motionless and in a standing position unless they are able to immediately climb back up onto the platform on their own, so "none" is selected as the movement detection condition. On the other hand, event #2 is a case of a worker. In this case, the person may move or stand still while working, so "none/yes" is selected.

The settings screen 21 also has an input field 25 for the alarm procedure. In this input field 25 for the alarm procedure, the administrator can specify the priority terminal, i.e., the terminal (alarm device) that will be given priority for alarm generation, as the alarm generation procedure. If the mobile terminal 5 is selected as the priority terminal, alarm generation will be performed in the following order: mobile terminal 5, in-facility fixed terminal 6, and command center fixed terminal 7. If the in-facility fixed terminal 6 is selected as the priority terminal, alarm generation by the mobile terminal 5 is omitted, and alarm generation will be performed in the following order: in-facility fixed terminal 6, and command center fixed terminal 7.

In this embodiment, the terminals are ranked in order of proximity to the site, i.e., mobile terminals 5, fixed terminals within the facility 6, and fixed terminals in the command center 7, and alerts are issued in stages, starting with the terminal with the highest priority. If the terminal that issued the alert does not respond with confirmation or completion of the response within a specified time, the terminal with the next highest priority will issue an alert. Note that the administrator may also be able to specify the priority of the terminals.

In the example shown in Figure 4, event #1 is the case of a person falling, and in this case, station staff need to rush to the scene and take the necessary measures as soon as possible, so mobile terminal 5 is selected as the priority terminal. The same is true for the case of an intruder. On the other hand, event #2 is the case of a worker, and in this case, station staff do not need to go to the scene, but it is desirable to alert staff in the office within the station, so in-facility fixed terminal 6 is selected as the priority terminal.

The settings screen 21 also has a registration button 26. After the administrator has completed input into the person detection input field 23, the detection condition input field 24, and the alarm procedure input field 25, they operate the registration button 26. This registers the input information as setting information.

Next, we will explain the notification screen 71 displayed on the mobile terminal 5. Figure 5 is an explanatory diagram showing the notification screen 71 displayed on the mobile terminal 5.

In response to instructions from the monitoring server 3, the mobile terminal 5 displays a notification screen 71 as a notification operation to notify station staff that a specific event has occurred.

The notification screen 71 is provided with a notification section 72 and a camera image display section 73. The notification section 72 displays text informing the staff of the details of the detected event. The camera image display section 73 displays the camera image at the time the event occurred. In addition, a frame image 77 surrounding the target person is displayed on the camera image in the camera image display section 73. The shape (color, etc.) of the frame image 77 may change depending on the detected event.

The camera images displayed on the camera image display unit 73 are recorded on the recorder 2, and are acquired from the recorder 2 by the monitoring server 3 and provided to the mobile terminal 5, or are acquired directly from the recorder 2 by the mobile terminal 5.

The example shown in Figure 5(A) is a case where an alert should be issued when a person falls, and the notification unit 72 displays text indicating that a person has fallen and their location, while the camera image display unit 73 displays a camera image of the person who has fallen. The example shown in Figure 5(B) is a case where an alert should be issued when an intruder has occurred, and the notification unit 72 displays text indicating that a intruder has occurred and their location, while the camera image display unit 73 displays a camera image of the intruder.

The notification screen 71 also has a confirmation button 75 and a response completion button 76. When an attendant looks at the notification screen and confirms that a specific event has occurred, they operate the confirmation button 75. When an attendant has completed necessary response measures, such as rescue, they operate the response completion button 76. Here, when a notification is sent via the notification screen 71 to each of the mobile devices 5 carried by multiple attendants, if one attendant carries out necessary response measures, such as rescue, and operates the response completion button 76, display control may be performed, such as graying out (disabling) the confirmation button 75 and response completion button 76 on the notification screen 71 on the mobile devices 5 carried by the other attendants, so that they can recognize that no response measures are required.

Next, we will explain the notification screen 81 displayed on the in-facility fixed terminal 6. Figure 6 is an explanatory diagram showing the notification screen 81 displayed on the in-facility fixed terminal 6.

In response to instructions from the monitoring server 3, the in-facility fixed terminal 6 displays a notification screen 81 as a notification operation to notify station staff that a specific event has occurred.

The notification screen 81 displays a status mark 82 and camera name 83 for each camera 1. The status mark changes color depending on the status level (warning, caution, normal) for each camera 1. The status level indicates the urgency (importance) of the current situation in the detection area, and is set depending on the occurrence of an event in the detection area of each camera 1.

In the example shown in Figure 6, in the detection area of camera 1 #1, a person has fallen, which is an event that requires a report. In this case, staff must rush to the scene, and because the urgency is high, the status mark 82 is displayed in a predetermined warning-level color (e.g., red). In the detection area of camera 1 #2, an intruder has occurred, which is an event that requires a report. In this case, staff must rush to the scene, but because the urgency is lower than in the case of a person falling, the status mark 82 is displayed in a predetermined caution-level color (e.g., orange). In the detection areas of the other cameras 1, no events that require a report have occurred, so the status mark 82 is displayed in a predetermined normal-level color (e.g., green). This allows station staff to immediately grasp the status of events occurring in the detection areas of each camera 1.

The alarm screen 81 also has a first camera image display section 84 and a second camera image display section 85. The first camera image display section 84 displays a camera image taken at the time an event requiring an alert occurred. The second camera image display section 85 displays camera images taken for a predetermined period of time (e.g., 10 seconds) before and after the event requiring an alert occurred. The first camera image display section 84 displays the camera image as a still image. Meanwhile, the second camera image display section 85 displays the camera image as a video when the staff member operates the operation section 86. This allows the staff member to check the situation at the time the event requiring an alert occurred on the first camera image display section 84. The staff member can also check the situation before and after the event requiring an alert occurred on the second camera image display section 85. A frame image 89 surrounding the target person is displayed on the camera image displayed on the camera image display sections 84 and 85. The shape (color, etc.) of the frame image 89 may change depending on the detected event.

The camera images displayed on the first camera image display unit 84 and the second camera image display unit 85 are recorded on the recorder 2, and are acquired from the recorder 2 by the monitoring server 3 and provided to the in-facility fixed terminal 6, or are acquired directly from the recorder 2 by the in-facility fixed terminal 6.

The notification screen 81 also has a log display section 87. The log display section 87 displays a list of events for which a report has been issued in the past. Specifically, the date and time of the report for each event, the name of the camera 1, and the current status are displayed. The log display section 87 also displays a status mark 88 for each event. Like the status mark 82, the color of this status mark 88 changes depending on the status level (warning, caution, normal). When one of the events displayed in the log display section 35 is selected, the camera image related to the selected event is displayed in the first camera image display section 84 and the second camera image display section 85.

This allows an attendant to select an event for which an alert was previously issued and check the status of that event using camera images. Alternatively, an attendant may enter search criteria on a log search input screen (not shown), select an event from the logs that match those search criteria, and check the status of that event using camera images. Furthermore, filtering may be set so that only the selected status level is displayed, depending on the operation to select the desired status level to be displayed.

Next, we will explain the notification screen 91 displayed on the fixed command center terminal 7. Figure 7 is an explanatory diagram showing the notification screen 91 displayed on the fixed command center terminal 7.

In response to instructions from the monitoring server 3, the fixed command center terminal 7 displays a notification screen 91 as a notification operation to notify the command center staff that a specific event has occurred.

The notification screen 91 has multiple station selection tabs 92 (facility selection section) for each station (facility) included in the line section under the control of the control center. When the manager operates the station selection tab 92, the notification screen 91 changes to correspond to the corresponding station.

The station selection tab 92 changes color depending on the status level of each camera 1 within the corresponding station. Specifically, the station selection tab 92 is displayed in a color corresponding to the highest status level (urgency) of the status levels (warning, caution, normal) of each camera 1 within the station. This allows control center staff to instantly grasp the alert status for each station (facility).

The notification screen 91 also has a log display section 87, similar to the notification screen displayed on the in-facility fixed terminal 6 (see Figure 6). However, in this case, the log display section 87 displays the name of the facility in addition to the date and time of the event, the name of the camera 1, and the current situation.

Otherwise, the notification screen is the same as that displayed on the in-facility fixed terminal 6 (see Figure 6).

Next, we will explain the processing steps performed by the monitoring server 3. Figure 8 is a flow diagram showing the processing steps performed by the monitoring server 3.

In the monitoring server 3, the processor 13 first acquires the camera image received from the camera 1 via the communication unit 11 (image acquisition process) (ST101).

Next, the processor 13 detects people from the camera image (person detection process) (ST102).

Next, the processor 13 determines whether the target person has entered the detection area based on the position of the target person detected in the person detection process (entry determination process) (ST103).

If the target person enters the detection area (Yes in ST103), a loop (ST104 to ST110) is entered in which a determination is made as to whether the detection conditions set for each event for which an alert should be issued are met, and the process of detecting the event for which an alert should be issued is repeated for all events for which an alert should be issued.

Here, processor 13 first recognizes the posture of the target person based on the person image extracted from the camera image (posture recognition processing), and determines whether the posture of the target person matches the detection conditions for the event for which an alert should be issued (posture determination processing) (ST105).

Here, if the posture of the target person matches the detection conditions (Yes in ST105), the processor 13 then determines whether the target person's entry direction into the detection area, specifically, the determination line crossed by the target person when entering the detection area, matches the detection conditions for an event that should be reported (entry direction determination process) (ST106).

Here, if the determination line crossed by the target person when entering the detection area matches the detection conditions (Yes in ST106), the processor 13 then determines whether the target person's stay time matches the detection conditions for an event that should be reported (stay determination process) (ST107).

Here, if the target person's stay time matches the detection conditions (Yes in ST107), the processor 13 then determines whether the target person's movement matches the detection conditions for an event that should be reported (movement determination process) (ST108).

Here, if the target person's movement matches the detection conditions (Yes in ST108), it is determined that an event requiring notification has occurred (event detection process) (ST109). If the processing of the loop (ST104 to ST110) has not been completed for all events requiring notification, the process returns to ST105 and proceeds to determination of the next event requiring notification.

On the other hand, if the processing of the loop (ST104 to ST110) has been completed for all events that require an alert, processor 13 then selects an alert procedure corresponding to the detected event (alert procedure selection process) (ST111). Then, processor 13 instructs mobile terminal 5, in-facility fixed terminal 6, and command center fixed terminal 7 to perform an alert operation to notify staff that a specified event has occurred (alert process) (ST112) in accordance with the selected alert procedure.

As mentioned above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can also be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. Furthermore, it is possible to combine the components described in the above embodiments to create new embodiments.

The monitoring device and monitoring system of the present invention have the effect of allowing staff to immediately grasp the situation of a person who has entered a detection area and quickly take appropriate action based on that person's situation. They are also useful as monitoring devices and monitoring systems that detect events that require notification based on camera images of the monitored area within a facility and instruct a notification device to perform a specified notification operation.

1 Camera 2 Recorder 3 Surveillance server (surveillance device)
4 Management terminal 5 Mobile terminal (alert device, mobile terminal device)
6. Fixed terminals within the facility (alarm devices, fixed terminal devices)
7. Fixed terminals at the control center (alarm devices, fixed terminal devices)
13. Processor

Claims (7)

A monitoring device comprising a processor that acquires camera images taken by cameras installed in each predetermined monitoring area within a facility, detects an event that should be reported for each camera, and instructs a notification device to perform a predetermined notification operation,
The processor:
In advance, based on an operation of an administrator from a management terminal, information on at least one of the posture of a person entering a detection area set within the monitoring area, the person's entry status into the detection area, the person's stay status in the detection area, and the person's movement status is set for each of the cameras as a detection condition for an event to be reported;
When a person entering the detection area is detected from the camera image, it is determined whether the detected person satisfies the detection condition for an event for which an alert should be issued;
When it is determined that the detection conditions are met, the monitoring device controls the alarm operations of the plurality of different types of alarm devices in accordance with a predetermined alarm procedure corresponding to the event for which an alarm should be issued. The processor:
2. The monitoring device according to claim 1, wherein the detection conditions include recognizing any one of a standing position, a crouching position, a face-down position, and a face-up position. The processor:
The monitoring device according to claim 1, wherein the posture of a person included in the detection conditions is recognized using an image recognition model constructed by machine learning using the camera images as learning data. The processor:
2. The monitoring device according to claim 1, wherein the notification device displays an image that allows the occurrence of an event for each of the cameras to be identified. The processor:
2. The monitoring device according to claim 1, wherein the notification device displays an image that allows identification of the occurrence status of an event at each facility. The processor:
The monitoring device described in claim 1, characterized in that the multiple different types of alarm devices are mobile terminal devices carried by staff, fixed terminal devices installed in the facility, and fixed terminal devices installed in a department that oversees multiple facilities. A monitoring device according to any one of claims 1 to 6;
a camera that photographs the predetermined monitoring area ;
a plurality of alarm devices that perform predetermined alarm operations in response to instructions from the monitoring device;
A surveillance system equipped with