JP7606434B2 - VIDEO MONITORING DEVICE, VIDEO MONITORING SYSTEM, AND VIDEO MONITORING METHOD - Google Patents

Description

The present invention relates to a video monitoring device, a video monitoring system, and a video monitoring method, and is suitable for use in a video monitoring device, a video monitoring system, and a video monitoring method that monitors abnormalities in target equipment using video captured by a surveillance camera.

Conventionally, when an abnormality is detected by an abnormality detection sensor such as a fire detector and a camera associated with the sensor, a technology is known in which the image captured by the camera is displayed on a terminal operated by a user and a report is sent to a pre-set destination (for example, Patent Document 1).

JP 2013-089092 A

However, the anomaly reporting system described in Patent Document 1 only allows users to view video when an anomaly occurs, which is problematic for building managers who monitor various facilities in a building collectively. In addition, the anomaly reporting system described in Patent Document 1 requires the installation of anomaly detection sensors and cameras on each of the various facilities in the building to be monitored, which increases implementation costs.

The present invention has been made in consideration of the above points, and aims to propose a video monitoring device, a video monitoring system, and a video monitoring method that can assist a user in taking action in response to a detected abnormality.

In order to solve such problems, the present invention provides a video monitoring device for monitoring abnormalities in a monitored facility using video from a surveillance camera that captures the facility, the video monitoring device comprising: a camera video acquisition unit that acquires video data of the facility captured by the surveillance camera; a camera video analysis unit that detects abnormalities in the facility from the video of the video data and identifies the contents of the detected abnormality; a storage device that stores customer information that associates information on a customer who has concluded a maintenance contract for monitoring the facility, the facility that is the monitored subject of the maintenance contract, and the surveillance camera that captures the facility, and response action information that links one or more response actions that can be selected by the customer when the abnormality is detected to one or more abnormality contents of the abnormality; and a storage device that stores a response action information that corresponds to the abnormality contents identified by the camera video analysis unit based on the response action information stored in the storage device. and a visualization processing unit that generates display data for displaying a monitoring screen on a customer operation terminal operated by the customer, the monitoring screen displaying the video of the video data acquired by the camera video acquisition unit, and uses the display data to display the monitoring screen on the customer operation terminal, wherein when the camera video analysis unit detects an abnormality in the equipment, the visualization processing unit generates display data for abnormality situations including a display of the video in which the abnormality was detected and a list of the one or more response actions acquired by the response action selection unit in response to the detection of the abnormality, and uses the display data for abnormality to display the monitoring screen on the customer operation terminal in a manner that allows the customer to determine which response action to execute from the list of the one or more response actions .

In addition, in order to solve the above problem, the present invention provides a video monitoring system for monitoring abnormalities in a monitored facility, the system comprising: a surveillance camera for capturing images of the facility; a camera video recording device for recording video data of the facility captured by the surveillance camera; a customer operation terminal operated by a customer who has concluded a maintenance contract for monitoring the facility; and a video monitoring device for detecting an abnormality in the facility using the video data of the facility, and presenting to the customer operation terminal a list of next response actions selectable by the customer when the abnormality is detected. The video monitoring device comprises a camera video acquisition unit for acquiring the video data of the facility from the camera video recording device; a camera video analysis unit for detecting an abnormality in the facility from the video of the video data and specifying the content of the detected abnormality; A video monitoring system is provided that has: a storage device that stores customer information associating information regarding a customer who has entered into a maintenance contract, the equipment that is the target of monitoring under the maintenance contract, and the surveillance camera that captures the equipment, and response action information in which the response action for the abnormality is linked to one or more for each anomaly content of the anomaly; a response action selection unit that acquires one or more of the response actions associated with the anomaly content identified by the camera video analysis unit based on the response action information stored in the storage device; and a visualization processing unit that generates display data for displaying on a customer operation terminal operated by the customer a monitoring screen that displays the image of the video data acquired by the camera video acquisition unit, and uses the display data to display the monitoring screen on the customer operation terminal. In this video monitoring system, when the camera image analysis unit detects an abnormality in the equipment, the visualization processing unit generates display data for abnormal situations, including a display of the image in which the abnormality was detected and a list of the one or more response actions obtained by the response action selection unit in response to the detection of the abnormality, and uses the display data for abnormal situations to display the monitoring screen on the customer operation terminal in a manner that allows the customer to perform an operation to decide which response action to perform from the list of one or more response actions, and the customer operation terminal determines which response action to perform from the list of response actions presented on the monitoring screen in response to operation by the customer .

In addition, in order to solve the above problem, the present invention provides a video monitoring method using a video monitoring device that monitors equipment for abnormalities using video from a surveillance camera that captures the equipment, the video monitoring device storing in a storage device customer information that associates information on a customer who has concluded a maintenance contract for monitoring the equipment, the equipment that is the subject of monitoring under the maintenance contract, and the surveillance camera that captures the equipment, and response action information that links one or more response actions that can be selected by the customer when an abnormality in the equipment is detected for each abnormality content of the abnormality, the video monitoring device includes a camera video acquisition step in which the video monitoring device acquires video data of the equipment captured by the surveillance camera, a camera video analysis step in which the video monitoring device detects an abnormality in the equipment from the video of the video data and identifies the abnormality content of the detected abnormality, and the video monitoring device performs the steps of: and a visualization processing step in which the video monitoring device generates display data for displaying a monitoring screen displaying the video of the video data acquired in the camera video acquisition step on a customer operation terminal operated by the customer , and uses the display data to display the monitoring screen on the customer operation terminal, wherein when an abnormality of the equipment is detected in the camera video analysis step, the video monitoring device in the visualization processing step generates display data for abnormality situations including a display of the video in which the abnormality was detected and a list of the one or more response actions acquired in the response action selection step in response to the detection of the abnormality, and uses the display data for abnormality to display the monitoring screen on the customer operation terminal in a manner in which the customer can determine which response action to execute from the list of one or more response actions.

The present invention can assist the user in taking action in response to a detected abnormality. Problems, configurations, and effects other than those described above will become clear from the description of the embodiments described below.

1 is a block diagram showing an example of the configuration of a video monitoring system 100 according to an embodiment of the present invention. FIG. 4 is a diagram showing an example of customer information. FIG. 11 is a diagram illustrating an example of response action information. FIG. 2 is a diagram showing an example of a processing flow of the entire processing in the video monitoring system 100. FIG. 13 is a diagram showing an image of data reference etc. in video data analysis processing. FIG. 13 is a diagram showing an image of data reference etc. in the response action acquisition process. 11 is a diagram for explaining an example of a screen configuration of a monitoring screen. FIG. FIG. 13 is a diagram showing an example of a monitoring screen display when an abnormality occurs. FIG. 13 is a diagram showing another example of the monitoring screen when an abnormality occurs;

In the following description, an "interface apparatus" may refer to one or more interface devices. The one or more interface devices may be at least one of the following:
One or more I/O (Input/Output) interface devices. The I/O interface devices are interface devices to at least one of the I/O devices and a remote display computer. The I/O interface device to the display computer may be a communications interface device. The at least one I/O device may be a user interface device, for example, either an input device such as a keyboard or a pointing device, or an output device such as a display device.
One or more communication interface devices. The one or more communication interface devices may be one or more homogeneous communication interface devices (e.g., one or more NICs (Network Interface Cards)), or two or more heterogeneous communication interface devices (e.g., a NIC and an HBA (Host Bus Adapter)).

In the following description, "memory" refers to one or more memory devices, typically a primary storage device. At least one of the memory devices in the memory may be a volatile memory device or a non-volatile memory device.

In the following description, a "persistent storage device" refers to one or more persistent storage devices. A persistent storage device is typically a non-volatile storage device (e.g., an auxiliary storage device), and more specifically, for example, a hard disk drive (HDD) or a solid state drive (SSD).

In the following description, a "storage device" refers to at least one of the above-mentioned memory and persistent storage devices.

In the following description, a "processor" refers to one or more processor devices. The at least one processor device is typically a microprocessor device such as a CPU (Central Processing Unit), but may also be another type of processor device such as a GPU (Graphics Processing Unit). The at least one processor device may be a single-core or multi-core. The at least one processor device may also be a processor core. Alternatively, the at least one processor device may be a broader processor device such as a hardware circuit (e.g., an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)) that performs some or all of the processing.

In the following description, functions may be explained using expressions such as "kkk unit", but the functions may be realized by one or more computer programs being executed by a processor, or by one or more hardware devices (e.g., FPGAs or ASICs), or by a combination of these. The description of each function is an example, and multiple functions may be combined into one function, or one function may be divided into multiple functions.

In the following explanation, expressions such as "xxx DB" may be used to describe information that produces an output for an input, but the information may be data of any structure (for example, structured data or unstructured data), or a learning model such as a neural network that produces an output for an input. Therefore, a "xxx DB" can be called "xxx information." In the following explanation, the configuration of each DB is an example, and one DB may be divided into two or more DBs, or two or more DBs may be realized in whole or in part by one DB. Note that "DB" is an abbreviation for database.

The following describes the embodiments of the present invention in detail with reference to the drawings. In this specification, when describing elements of the same type (elements having the same function) without distinction, the common part (part excluding the suffix or branch number) of the reference number including the suffix or branch number is used, and when describing elements of the same type with distinction, the reference number including the suffix or branch number may be used. For example, when describing a surveillance camera without distinction, it is written as "surveillance camera 22", whereas when describing each surveillance camera 22 with distinction, it may be written as "surveillance camera 22a", "surveillance camera 22b", "surveillance camera 22c", etc. In addition, the repeated description of elements having the same function as described above will be omitted. In addition, the description of the general components that make up an elevator, specifically, the main rope, counterweight, car, hoist, security camera, and video recording device will be omitted.

(1) Configuration Fig. 1 is a block diagram showing an example of the configuration of a video monitoring system 100 according to one embodiment of the present invention. As shown in Fig. 1, the video monitoring system 100 includes a video monitoring device 1, a camera video recording device 21 and a surveillance camera 22 in a building 2, and a customer operation terminal 3. The video monitoring device 1 is communicatively connected to the camera video recording device 21 via a communication network 4, and communicatively connected to the customer operation terminal 3 via a communication network 5.

In this embodiment, the equipment 23 in the building 2 (specifically, the rooftop 23a, elevator 23b, fire equipment 23c, automatic doors 23d, and parking lot 23e) are monitored, and surveillance cameras 22a to 22e are installed to capture the equipment 23. The images captured by each surveillance camera 22 are stored in a camera image recording device 21. The camera image recording device 21 may be, for example, a general computer or storage device, and may have at least a storage function for recording image data and a communication function for transmitting the recorded image data to the video monitoring device 1. Note that the equipment 23 shown in FIG. 1 is merely an example, and is not limited to this. For example, the equipment 23 to be monitored may be an outside passageway, a garbage collection area, etc.

The building 2 in which the surveillance camera 22 that monitors the monitored facility 23 is installed is not limited to one, but may be multiple. In this embodiment, the surveillance camera 22 is treated as one management unit, but the management unit is not limited to this. For example, a group of buildings (structures) 2 may be the management unit. The monitored facility 23 is not limited to facilities within a building, but facilities in structures other than buildings, such as factories and stations, may also be monitored, and these structures may be the management units.

The customer operation terminal 3 is an information processing terminal owned or used by a customer, and includes a display device, an input device, and an interface device. The input device is, for example, a keyboard and a mouse, but may also be, for example, a touch panel integrated with the display device. Specific examples of the customer operation terminal 3 are not particularly limited, but include, for example, a personal computer, a tablet, or a smartphone. The customer operation terminal 3 is not limited to one, and may be multiple. Note that in this embodiment, the "customer" refers to a person or organization (contractor) that has concluded a contract regarding the maintenance and management of the building 2 (hereinafter, the maintenance contract).

The communication network 4 connecting the video monitoring device 1 and the camera video recording device 21, and the communication network 5 connecting the video monitoring device 1 and the customer operation terminal 3 may be the same type of communication network or different types of communication networks. Each of the communication networks 4 and 5 may be, for example, the Internet, a mobile communication network, a wired LAN (Local Area Network), a wireless LAN, or a dedicated line, or a combination of these. Furthermore, the communication networks 4 and 5 may be a single communication network (for example, the Internet).

The video monitoring device 1 may be an information processing device including an interface device (not shown), a storage device 12, and a processor connected to them, or may be a device realized on the device (e.g., a cloud platform). The interface device performs input and output of data to the video monitoring device 1. Specifically, the video monitoring device 1 may be provided with a keyboard and a display, or an operation terminal (not shown) used by an operator or salesperson may be used as the interface device for the video monitoring device 1.

In the video monitoring device 1, the processor is, for example, a CPU (Central Processing Unit). The storage device 12 includes a memory, for example, a ROM (Read Only Memory) and a RAM (Random Access Memory). In the video monitoring device 1, the CPU reads and executes program code stored in the ROM, and the software and hardware work together to realize the functions of each processing unit (customer information input unit 11, camera image acquisition unit 13, visualization processing unit 14, camera image analysis unit 15, and response action selection unit 16) described below. In addition to the above-mentioned ROM and RAM, the storage device 12 may also be realized by including a persistent storage device such as an HDD (Hard Disk Drive). The communication unit 17 is realized by an interface device having a communication function.

As shown in FIG. 1, the video monitoring device 1 includes a customer information input unit 11, a storage device 12, a camera image acquisition unit 13, a visualization processing unit 14, a camera image analysis unit 15, a response action selection unit 16, and a communication unit 17.

The customer information input unit 11 has a function of inputting information (customer information) about customers who have concluded a maintenance contract for video surveillance of the equipment 23 and the target of surveillance into the customer information DB 12a of the storage device 12. The customer information can include information indicating the person receiving the maintenance service, the contractor, a building ID for identifying the building 2, and the building address. Specific examples of customer information are shown in FIG. 2, which will be described later. In this embodiment, information about the equipment 23 to be monitored in the building 2 and information about the surveillance camera 22 monitoring the equipment 23 are associated with each other and registered in the customer information.

The storage device 12 has a function of storing specific information. In this embodiment, the storage device 12 is configured with different DBs (databases) depending on the type of information to be stored. Specifically, the storage device 12 has a customer information DB 12a, a camera image management DB 12b, anomaly record DB 12c, and a response action DB 12d.

The customer information DB 12a stores information (customer information) about customers who have signed a maintenance contract for video surveillance of the equipment 23 and the target of surveillance. As described above, the customer information is input from the customer information input unit 11, and a specific example is shown in FIG. 2, which will be described later. The camera video management DB 12b stores information (camera video information) about the video data captured by each surveillance camera 22. The camera video information is acquired from the camera video recording device 21 and stored in the camera video management DB 12b. The abnormality record DB 12c stores information (abnormality information) about abnormalities detected by analyzing the camera video information for each surveillance camera 22. The camera video information is analyzed by the camera video analysis unit 15. The response action DB 12d stores information (response action information) about response actions that are options for responding to detected abnormalities. A specific example of response action information is shown in FIG. 3, which will be described later.

The camera image acquisition unit 13 connects to the camera image recording device 21, which records the image data of each surveillance camera 22, via the communication network 4 at a predetermined interval (for example, once every 5 minutes) based on each camera (surveillance camera 22) shown in the customer information stored in the customer information DB 12a. The camera image acquisition unit 13 then periodically acquires surveillance images (image data) of the equipment 23 from each surveillance camera 22 from the camera image recording device 21, and registers the camera image information based on the acquired image data in the camera image management DB 12b. The camera image acquisition unit 13 may be configured to be constantly connected to the camera image recording device 21. The camera image acquisition unit 13 may also register the image data acquired from the camera image recording device 21 as camera image information in the camera image management DB 12b as it is, but preferably adds predetermined information related to the image data (for example, information about the image data itself, various information registered in the customer information about the shooting environment of the image data, etc.) and registers it as camera image information in the camera image management DB 12b.

The visualization processing unit 14 has a function to process data to be output to the customer operation terminal 3, and a function to generate screen data to be output to the customer operation terminal 3. Specifically, for example, the visualization processing unit 14 can generate a screen requesting input of a user name and password, a monitoring screen for the monitored equipment 23 in the building 2, a response selection screen when an abnormality is detected, and the like, and provide them to the customer operation terminal 3.

The camera video analysis unit 15 has a function of detecting anomalies from the camera video information of the surveillance camera 22 acquired from the camera video recording device 21, and identifying the type of the detected anomaly (anomaly type). In analyzing the camera video information (i.e., detecting the anomaly and identifying the anomaly type as described above), the camera video analysis unit 15 obtains an analysis result by referring to the customer information DB 12a and the camera video management DB 12b and inputting the necessary data into a predetermined algorithm for detecting anomalies. The camera video analysis unit 15 then registers the obtained analysis result in the anomaly record DB 12c as anomaly information. The camera video analysis unit 15 also has a function of outputting the obtained analysis result to the visualization processing unit 14.

The response action selection unit 16 has a function of receiving the analysis results (detection of anomaly, identification of anomaly type) by the camera image analysis unit 15 and acquiring a recommended response action from among the response actions associated with the identified anomaly type. Specifically, the response action selection unit 16 refers to the anomaly record DB 12c and the response action DB 12d, and selects and acquires information on an action (response action) corresponding to the anomaly recorded in the anomaly information from the response action information. The response action selection unit 16 also has a function of outputting information on the acquired response action to the visualization processing unit 14.

The communication unit 17 functions as an input/output unit that allows each part of the video monitoring device 1 to input and output data between the camera video recording device 21, the customer operation terminal 3, or the operation terminal of a maintenance staff or sales staff (not shown), etc.

(2) Various Information Various information stored in the storage device 12 will be described in detail below, focusing on the customer information and the response action information that are set before the video surveillance is performed.

First, we will explain customer information. Customer information is information about customers who have concluded a maintenance contract, and is registered in the customer information DB 12a by the customer information input unit 11 in response to input operations by sales personnel who conduct sales activities and maintenance personnel who perform maintenance work.

Figure 2 is a diagram showing an example of customer information. The customer information 121 shown in Figure 2 is a specific example of customer information registered in the customer information DB 12a. When a video surveillance maintenance contract is concluded with a customer on a building 2 basis, the customer information 121 is formed from a customer record that compiles information on the customer and the monitored object for each building 2 with which the video surveillance maintenance contract is concluded.

Specifically, in the customer information 121 of FIG. 2, the customer record is composed of the following items: customer 1211, customer ID 1212, building ID 1213, building address 1214, building facilities 1215, building facilities ID 1216, camera 1217, and camera ID 1218. Note that the items shown in FIG. 2 are merely examples, and the customer information according to this embodiment may be such that the customer record is formed including items other than those exemplified in FIG. 2.

In customer 1211, the name of the customer who has concluded a maintenance contract is registered, and in customer ID 1212, an identifier (customer ID) assigned to uniquely identify the customer is registered. In building ID 1213, an identifier (building ID) assigned to uniquely identify building 2 in which the equipment subject to video monitoring under the maintenance contract is installed is registered, and in building address 1214, the address of the building 2 is registered. In building equipment 1215, the name of the equipment subject to video monitoring (equipment 23) is registered, and in building equipment ID 1216, an identifier (building equipment ID) assigned to identify the equipment 23 is registered. In camera 1217, the name of the surveillance camera 22 that monitors the equipment subject to video monitoring (equipment 23) is registered, and in camera ID 1218, an identifier (camera ID) assigned to identify the surveillance camera 22 is registered.

Although not shown in FIG. 1, in this embodiment, a user DB may be stored in the storage device 12. In this embodiment, a "user" refers to a person belonging to a customer (e.g., an employee of the customer's company). The user DB may have a user record for each user, and it is desirable for each user record to include an individual user ID, password, and customer ID. The user ID and password are authentication information for identifying users who can access the video monitoring device 1.

Here, the usage of the user ID and password will be explained. The customer operation terminal 3 is connected to the video monitoring device 1 through the communication network 5 by the user's operation. In response to this connection, the visualization processing unit 14 requests the user to input the user ID and password. That is, the visualization processing unit 14 outputs information prompting the user to input the user ID and password to the customer operation terminal 3. Then, only when the combination of the user ID and password registered in the user DB is input from the customer operation terminal 3, the visualization processing unit 14 can output (for example, display) various information acquired from the customer information DB 12a, the camera video management DB 12b, the abnormality record DB 12c, and the response action DB 12d to the customer operation terminal 3. Note that the output data created by the visualization processing unit 14 is not limited to a specific data format, and specifically, for example, an image format, a video format, an HTML format, an XML format, or a CSV format can be adopted.

In addition, this embodiment mainly describes the video data from the surveillance camera 22, the abnormalities (type and content of abnormality) detected by analyzing the video data, and the actions (response actions) taken in response to the abnormality, but the video monitoring device 1 according to the present invention may display, instead of or in addition to this information, responses to inquiries related to building facilities, scheduled inspection dates, or inspection results. By configuring in this way, the video monitoring device 1 according to the present invention can handle various types of contract information related to building facilities.

Next, the response action information will be described. The response action information is information regarding recommended actions (response actions) for abnormalities detected in the equipment (equipment 23) subject to video monitoring under the maintenance contract, and one or more response actions are set in advance depending on the nature of the abnormality. The response action information only needs to be registered in the response action DB 12d before video monitoring is performed by the video monitoring device 1, and various methods can be adopted for the registration method. Specifically, for example, the response action information may be registered in the response action DB 12d in advance by a system administrator, or response action information for representative abnormality contents may be registered in the response action DB 12d by the manufacturer when the video monitoring device 1 is manufactured or sold. In addition, the response action information may be appropriately customized by the customer (for example, the owner or management company of building 2).

Figure 3 is a diagram showing an example of response action information. The response action information 124 shown in Figure 3 is a specific example of response action information registered in the response action DB 12d, and the response action information 124 is formed from response action records in which one or more response actions to be taken next when an abnormality is detected are linked for each combination of the equipment 23 of the building 2 for which a maintenance contract for video monitoring by the surveillance camera 22 has been concluded and the abnormality content.

Specifically, in the response action information 124 of FIG. 3, the response action record is composed of the following items: building equipment 1241, building equipment ID 1242, abnormality type 1243, abnormality content 1244, abnormality content priority 1245, action number 1246, action content 1247, and contact information 1248. Note that the items shown in FIG. 3 are merely examples, and the response action information according to this embodiment may be one in which a response action record is formed including items other than those exemplified in FIG. 3. Also, in FIG. 3, a response action record is shown for the "fire equipment" of the building equipment ID "S001" and the "elevator" of the building equipment ID "E001", but it can be considered that in the actual response action information 124, a response action record is registered for all the equipment shown in the building equipment 1215 in the customer information 121 of FIG. 2.

Building equipment 1241 and building equipment ID 1242 correspond to building equipment 1215 and building equipment ID 1216 in customer information 121 shown in FIG. 2. Anomaly type 1243 indicates the type of anomaly that can be detected by video data analysis by camera video analysis unit 15. In the case of FIG. 3, "smoke/fire" belonging to fire anomalies and "abnormal behavior" belonging to human behavior anomalies are exemplified.

In the abnormality content 1244, an abnormality content indicating a specific abnormality event is registered. This abnormality content can be regarded as a detailed classification of the abnormality type 1243. In the abnormality content priority 1245, a priority corresponding to each abnormality content registered in the abnormality content 1244 is registered. The priority corresponding to each abnormality content is set in advance, but may be changed by an administrator or the like during operation. In addition, the abnormality content priority 1245 may be set for each abnormality type 1243, but considering the case where abnormalities belonging to multiple abnormality types occur simultaneously, it is preferable to set the priority for all abnormality contents 1244 belonging to all abnormality types 1243. In the example shown in FIG. 3, three types of abnormality contents belonging to "smoke/fire" are registered: "smoke detection", which means an abnormality in which smoke is detected from the video data, "fire detection", which means an abnormality in which a flame is detected from the video data, and "operation without smoke/fire", which means an abnormality in which neither smoke nor flame is detected from the video data but smoke or fire is estimated from other situations in the video (for example, blinking of a lamp, etc.). Furthermore, three types of abnormality contents belonging to "abnormal behavior" are registered: "abnormal behavior (unruly) detection", which means that an abnormality is detected in the video data where a person is violent; "abnormal behavior (staying) detection", which means that an abnormality is detected in the video data where a person is staying still; and "abnormal behavior (other) detection", which means that an abnormality in human behavior other than violence or staying still is detected in the video data. For the above six types of abnormality contents 1244, priorities of "1" to "6" are set in the abnormality content priority 1245, with the smaller the priority value, the higher the priority. For example, of the above six types of abnormality contents 1244, "ignition detection" is the most dangerous abnormality, and therefore the highest priority "1" is set.

In the action content 1247, one or more response actions recommended when the abnormality content 1244 is detected are registered, and in the contact information 1248, a contact to be contacted when the response action is performed is registered. In addition, in the action number 1246, the order in which each response action is recommended is registered for one or more response actions (action content 1247) recommended for each abnormality content 1244. In the case of FIG. 3, for example, when the detected abnormality is "smoke detection", "call a fire engine" is registered in the action content 1247 as the response action linked to the "first" recommended action number 1246, and in the contact information 1248, "119", which is the telephone number of the fire department, is registered as the contact for the response action. In addition, for "smoke detection", "announce over the public address system" is registered in the action content 1247 as the response action linked to the "second" recommended action number 1246, but since a contact is not required for the response action, no specific contact is registered in the contact information 1248. In this embodiment, as described above, multiple response actions can be registered for one anomaly, and a recommended order for these response actions can be set, making it easy to understand what action the customer should take if an anomaly is detected.

In the example of FIG. 3, since the abnormality type 1243 differs for each building facility 1241, the abnormality content 1244 belonging to the abnormality type 1243 and the action content 1247 corresponding to the abnormality content 1244 also differ for each building facility 1241. However, the response action information 124 according to this embodiment may register common or overlapping abnormality types 1243 even for different building facilities 1241. Furthermore, in the response action information 124 (response action record) according to this embodiment, it is sufficient that at least the action content 1247 is individually defined for each abnormality content 1244.

In addition, the response action information 124 (response action DB 12d) is registered in advance by, for example, a system administrator, but if the user is an employee of a management company, it may be necessary to register contact information that the maintenance company cannot anticipate, such as contacting the building owner when an abnormality is identified. Therefore, as a variation of this embodiment, the response action information 124 may be configured so that the user can change or newly register the action number 1246, action content 1247, or contact information 1248, etc.

As described above, abnormality information DB 12c stores abnormality information about abnormalities detected from video data. In this abnormality information, the data items related to the detected abnormality use a data format corresponding to response action information 124 (specifically, abnormality type 1243, abnormality content 1244, etc.).

(3) Processing In the following, we will explain the processing in which the video monitoring system 100 of this embodiment provides a monitoring screen to the customer operation terminal 3 using video data from the surveillance camera 22, monitors the occurrence of anomalies from the video data from the surveillance camera 22, and, when an abnormality is detected, notifies the customer operation terminal 3 to support appropriate response actions to the abnormality (e.g., reporting the abnormality).

Figure 4 is a diagram showing an example of a processing flow of the overall processing in the video monitoring system 100. Figure 4 mainly shows the processing related to displaying information on the monitoring screen and operations on the monitoring screen, and omits an explanation of basic operations such as logging in and logging out of the system (video monitoring device 1).

According to FIG. 4, first, the camera video recording device 21 constantly acquires and records video data of the equipment 23 captured by the surveillance camera 22 installed in the building 2 (step S11). Then, the camera video acquisition unit 13 of the video monitoring device 1 constantly or periodically accesses the camera video recording device 21 to acquire the video data of the surveillance camera 22, and registers the camera video information based on the video data in the camera video management DB 12b (step S21). Note that in step S21, the camera video acquisition unit 13 may also acquire various information required to generate camera video information based on the video data from the camera video recording device 21 and the customer information DB 12a.

Next, when a predetermined condition is met in the video monitoring device 1, the camera video analysis unit 15 starts a video data analysis process for each video data. The above-mentioned predetermined condition may be, for example, a timing of a predetermined cycle after the monitoring screen is launched in the customer operation terminal 3, or a timing of a predetermined cycle regardless of the function of the monitoring screen. In the former case, abnormality detection is performed only when the monitoring screen is launched in the customer operation terminal 3, so that the processing load in the video monitoring device 1 can be reduced. On the other hand, in the latter case, video data analysis is performed periodically on all video data acquired from the camera video recording device 21 regardless of the operation status in the customer operation terminal 3, so that although the processing load is high, the occurrence of an abnormality can be detected quickly and reliably. In addition, predetermined conditions other than those mentioned above may be defined.

In the video data analysis process, the camera video analysis unit 15 selects one unanalyzed video data from among the video data to be analyzed registered in the camera video management DB 12b, and obtains the camera video information and customer information related to the selected video data from the camera video management DB 12b and the customer information DB 12a (step S22). For example, when the monitoring screen is activated on the customer operation terminal 3, the customer ID and building ID entered by the customer when the screen was activated are transmitted to the video monitoring device 1, and therefore the video data of the building facility 1215 linked to these IDs in the customer information 121 (i.e., the video data captured by the camera 1217) becomes the video data to be analyzed.

Next, the camera image analysis unit 15 uses the information acquired in step S22 to perform image analysis to determine whether there are any abnormalities in the image of the video data to be analyzed (step S23). The image analysis in step S23 detects abnormalities corresponding to the abnormality type 1243 and the abnormality content 1244 in the response action information 124 (see FIG. 3).

The camera video analysis unit 15 then determines whether or not an abnormality was detected in the image analysis of step S23 (step S24). If no abnormality was detected in step S24 (NO in step S24), the image (still image or video image) of the video data in which no abnormality was detected is sent to the visualization processing unit 14, and the process proceeds to step S28, which will be described later. If an abnormality was detected in step S24 (YES in step S24), the camera video analysis unit 15 identifies the type and content of the detected abnormality, generates abnormality information from the analysis results including the identified information, and registers the generated abnormality information in the abnormality record DB 12c (step S25).

Note that Figure 5, which will be described later, shows an example of data referencing in the video data analysis process in steps S22 to S25.

After the processing of step S25, the response action selection unit 16 performs a response action acquisition process to acquire the response action associated with the detected abnormality.

In the response action acquisition process, the response action selection unit 16 refers to the anomaly information registered in the anomaly record DB 12c in step S25, and refers to the response action DB 12d using the anomaly type and the anomaly content as keys to acquire all response action records that match the anomaly type 1243 and the anomaly content 1244 (step S26). If the video monitoring device 1 has a pop-up notification function that notifies the detection of an anomaly, the response action selection unit 16 may notify the visualization processing unit 14 of the detection of the anomaly at the timing of step S26, and the visualization processing unit 14 may transmit a pop-up notification to the customer operation terminal 3 via the communication unit 17. The timing of the pop-up notification may be any of steps S27 to S29 described below, but by adopting an early timing such as step S26, the detection of an anomaly can be promptly notified to the customer.

Next, the response action selection unit 16 refers to the abnormality content priority 1245 of the response action record acquired in step S26, and if an abnormality has been detected in the video data analysis of other video data that has been performed so far, rearranges the information on the multiple abnormality detections based on the abnormality content priority 1245 (step S27). The process of step S27 is a process for determining which abnormality should be notified preferentially (for example, displayed at the top of the monitoring screen) when abnormalities are detected in multiple pieces of equipment 23 at the same time. Therefore, if no abnormality has been detected in the video data analysis of other video data that has been performed so far, the process of step S27 may be skipped. After the process of step S27 is completed, the response action selection unit 16 transmits the information of one or more response action records after rearrangement to the visualization processing unit 14, and proceeds to step S28.

Note that Figure 6, which will be described later, shows an example of data referencing and other processing in the response action acquisition process in steps S26 and S27.

In step S28, a predetermined processing unit of the video monitoring device 1 (for example, it may be the camera video analysis unit 15 or an overall control unit (not shown)) determines whether any unanalyzed video data remains. If unanalyzed video data exists (YES in step S28), the process returns to step S22 and repeats the above-mentioned process. If video data analysis has been completed for all video data to be analyzed (NO in step S28), the process proceeds to step S29. Here, "all video data to be analyzed" may be replaced with video data from the surveillance cameras 22 for all equipment 23 in the building 2.

In step S29, the visualization processing unit 14 generates display data for the monitoring screen using various information received from the camera image analysis unit 15 and the response action selection unit 16 in the processing of steps S22 to S28. The display data generated here includes not only display data related to the equipment 23 in which an abnormality was detected, but also display data related to the equipment 23 in which no abnormality was detected. Then, the visualization processing unit 14 transmits the display data to the customer operation terminal 3, whereby the customer operation terminal 3 can display the received display data on the activated monitoring screen (corresponding to step S33 described later).

In the process flow of FIG. 4, the visualization processing unit 14 generates display data after the video data analysis process (and corresponding action acquisition process) is completed for all video data to be analyzed. However, as a modified example, display data may be generated each time the video data analysis process (and corresponding action acquisition process) is completed for each video data, and the display data may be transmitted to the customer operation terminal 3 at any stage.

When the processing of step S29 is completed, in principle, the video monitoring device 1 ends the series of processes. However, if the execution of a specific response action is decided from the customer operation terminal 3, the video monitoring device 1 may execute that response action (step S30). For example, when a response action that can be executed by a response action processing unit (not shown) of the video monitoring device 1 (specifically, "announce via the in-house broadcast") is set, the response action processing unit executes that response action in response to an operation on the monitoring screen of the customer operation terminal 3 (step S35 described below).

Next, the processing in the customer operation terminal 3 will be described with reference to FIG. 4. Note that the processing flow shown in FIG. 4 is merely one example of processing in the customer operation terminal 3, and is not limited to this. For example, the customer can operate the customer operation terminal 3 to launch the monitoring screen before receiving a pop-up notification of abnormality detection.

As shown in FIG. 4, when the customer operation terminal 3 receives a pop-up notification from the visualization processing unit 14 of the video monitoring device 1 in step S26, the customer operation terminal 3 issues a pop-up notification by sound, display, vibration, or the like, and notifies the customer of the detection of the abnormality (step S31). The function of the pop-up notification is a generally known technology, so a detailed description is omitted.

Next, in response to the pop-up notification in step S31, or by the customer's will, the customer performs a predetermined operation, causing the customer operation terminal 3 to launch the monitoring screen (step S32). When the monitoring screen is launched, the customer is requested to input a customer ID (user ID) assigned to the customer and the building ID of the building 2 that contains the equipment 23 to be monitored, and these customer ID and building ID are transmitted to the video monitoring device 1 via the communication network 5. When the monitoring screen is launched, authentication may also be performed using the user ID and password described above.

After the monitoring screen is launched, the customer operation terminal 3 displays the monitoring screen using the display data received from the visualization processing unit 14 of the video monitoring device 1 (step S33).

Then, the customer checks whether or not an abnormality has been detected by looking at the monitoring screen displayed in step S33 (step S34). If no abnormality has been detected, the monitoring screen displays "Normal" (NO in step S34), and the customer does not need to take any special action; the process returns to step S33, and the display of the monitoring screen (including updating of the display content) continues. At this time, the customer may, for example, close the monitoring screen. On the other hand, if an abnormality has been detected, the monitoring screen displays "Abnormal" (YES in step S34), and in this case the customer is requested to perform the operation of step S35.

In addition, Figure 7, which will be described later, shows examples of the "normal" and "abnormal" monitoring screen configurations.

When "abnormal" is displayed on the monitoring screen, in step S35, the customer checks the video in which the abnormality is detected and presses the response action display button (response action display button 3022 in FIG. 7 described later) to select the response action to be executed from the response actions displayed in a list (response action list 3203 in FIG. 7 described later).

Next, the customer performs a predetermined operation to execute the response action determined in step S35, and the customer operation terminal 3 or the video monitoring device 1 executes the response action (step S30 or step S36). The predetermined operation to execute the response action corresponds to, for example, pressing the "contact" button for the response action displayed on the monitoring screen. It can be considered that which of the customer operation terminal 3 or the video monitoring device 1 executes the response action is set in advance in the response action information (response action DB 12d). Specifically, for example, in the case of a response action in which a specific telephone number is registered in the contact, it is more efficient for the customer operation terminal 3 to make a call directly in step S36. Also, in the case of a response action in which a specific telephone number is not registered in the contact and the response action is to make an announcement in the building 2, it may be more efficient for the video monitoring device 1 to take the initiative in executing the response action in step S30.

After the response action has been taken, the customer can terminate operations on the monitoring screen, but it is preferable that the information continue to be updated and displayed on the monitoring screen until the abnormality is actually resolved.

It is possible to adopt various configurations for the execution of the response action, as follows. For example, when contacting a contact person according to the response action, the video in which the abnormality is detected may be repeatedly played on the monitoring screen in order to make it easier to communicate the abnormal situation. As another example, as a response action in the event of an abnormality, the customer operation terminal 3 may be directly or indirectly connected to the in-house broadcasting equipment of the building 2 or an intercom in the elevator car, so that questions can be made to users of the building 2. As another example, as a response action in the event of an abnormality, the customer operation terminal 3 may be directly or indirectly connected to a signage in the building 2 or an LCD display in the elevator car, so that the video of the abnormality can be distributed, making it easier to communicate information to security guards and emergency personnel who rush to the scene. As another example, as a response action in the event of an abnormality, a response action such as remote control of equipment (such as disaster prevention equipment) in the building 2 may be provided.

In addition, if no abnormality is detected, the monitoring screen may be updated periodically, and each time an update is performed, the video data may be analyzed and the video may be displayed on the monitoring screen.

In addition, regarding whether an abnormality has been detected and notifications when an abnormality is detected, historical information (logs) may be saved and accumulated in the storage device 12 of the video monitoring device 1. With such a configuration, the accumulated logs can be used for post-mortem verification of detected abnormalities, etc.

In addition, the processing flow in FIG. 4 shows a procedure in which a monitoring screen is launched on the customer operation terminal 3 after a pop-up notification is issued when an abnormality is detected, but the processing flow when an abnormality is detected in this embodiment is not limited to the above procedure. For example, when an abnormality is detected while the monitoring screen is being displayed, the customer operation terminal 3 may be notified of the detection of the abnormality separately by a predetermined means such as a pop-up notification. Also, for example, when an abnormality is detected while the monitoring screen is being displayed, the display of the monitoring screen may be switched to a monitoring screen for abnormality, which will be described later in FIG. 8 and FIG. 9, without a pop-up notification.

In the above description, as an example, the analysis of the camera video (video data) in step S22 is performed based on the state of the customer operation terminal 3, such as when the monitoring screen is started up, but the trigger for analyzing the video data in this embodiment is not limited to this. For example, the camera video analysis unit 15 may analyze the video data every time the camera video acquisition unit 13 acquires video data from the camera video recording device 21.

As described above, the overall processing in the video monitoring system 100 has been explained with reference to FIG. 4. In the following, as a detailed explanation, the video data analysis processing and corresponding action acquisition processing in the video monitoring device 1, and the display of the monitoring screen in the customer operation terminal 3 will be explained with reference to the drawings.

(3-1) Video Data Analysis Processing Fig. 5 is a diagram showing an image of data reference in video data analysis processing. As shown inside the camera video analysis unit 15 in Fig. 5, the data reference (reference, acquisition, transmission, recording) shown in Fig. 5 corresponds to the processing in steps S22 to S25 in Fig. 4.

The data flow in the video data analysis process is explained with reference to Figure 5.

First, when a request to display a monitoring screen is made in response to operation of a customer's customer operation terminal 3, the visualization processing unit 14 receives the request via the communication unit 17 and transmits to the camera image analysis unit 15 the customer ID (user ID) of the customer operating the customer operation terminal 3 and the building ID of the building 2 to be monitored, both of which are included in the request.

Next, the camera video analysis unit 15 refers to the customer information DB 12a using the received customer ID and building ID as keys to obtain the camera ID 1218 of the corresponding customer record. Furthermore, the camera video analysis unit 15 refers to the camera video management DB 12b using the acquired camera ID 1218 and the above-mentioned customer ID and building ID as keys to obtain the "video recording date and time (start)" and "video recording date and time (end)" indicating the recording date and time of the target video data, and the "video data" which is the main body of the target video data, as information necessary for analyzing the video data.

Next, the camera image analysis unit 15 performs image analysis (video data analysis) on the acquired video data to check for any abnormalities. The camera image analysis unit 15 is equipped with a program that includes an algorithm for detecting abnormalities in camera images (still images or video images) using machine learning or the like, and the abnormality detection algorithm is capable of detecting, for example, equipment failures such as automatic doors not closing, smoke, fire, abnormal behavior of building users (being violent or staying still), suspicious behavior (mischief such as graffiti), etc.

When the camera image analysis unit 15 detects equipment failure such as an automatic door not closing, smoke, fire, abnormal behavior of building users (unruly or staying still), suspicious behavior (mischief such as graffiti), etc. through the video data analysis, it identifies the type and content of the abnormality, generates abnormality information from the analysis results including these, and registers it in the abnormality record DB 12c. A specific data configuration example of the abnormality information is as shown in FIG. 5, and includes the abnormality type and content, various information acquired from the customer information DB 12a and the camera image management DB 12b in step S22, the date and time of the abnormality detection, video data including video of the abnormal state, etc. Note that the "video data" of the abnormality information registered in the abnormality record DB 12c does not need to be the entire video data that has been subjected to the image analysis. For example, it is preferable to generate video data showing video for 5 seconds (total 10 seconds) before and after the date and time of the abnormality detection as the reference point, and use this as the "video data" of the abnormality information. In this way, by treating a portion of the original video data that has undergone image analysis as the "video data" for the abnormality information, it is possible to prevent an increase in the data volume of the abnormality information while preserving the video showing the detected abnormality.

(3-2) Response Action Acquisition Processing Fig. 6 is a diagram showing an image of data reference, etc. in the response action acquisition processing. The data reference, etc. shown in Fig. 6 (reference, acquisition, transmission) correspond to the processing in steps S26 to S27 in Fig. 4.

The data flow in the response action acquisition process will be explained with reference to Figure 6.

After the camera image analysis unit 15 analyzes the video data and stores the analysis results as anomaly information in the anomaly record DB 12c, the response action selection unit 16 refers to the anomaly record DB 12c to obtain the anomaly type and anomaly content of the detected anomaly. Next, the response action selection unit 16 refers to the response action DB 12d using the obtained anomaly type and anomaly content as keys to obtain the action number 1246, action content 1247, and contact information 1248 of the corresponding response action record.

Then, the response action selection unit 16 combines the abnormality information acquired from the abnormality record DB 12c (which includes part of the customer information as shown in FIG. 5) and the response action information acquired from the response action DB 12d, and transmits the combined information to the visualization processing unit 14. The visualization processing unit 14 then generates display data for the monitoring screen using the received information, and outputs the display data to the customer operation terminal 3 via the communication unit 17 and the communication network 5, thereby displaying information on the detected abnormality on the monitoring screen of the customer operation terminal 3. On the other hand, if no abnormality is detected, the visualization processing unit 14 generates display data for the monitoring screen based on the information acquired from the camera image management DB 12b by the camera image analysis unit 15 described in FIG. 5, and normal information is displayed on the monitoring screen of the customer operation terminal 3. In this way, the customer (user) can check the camera image of the surveillance camera 22 that has captured the equipment 23 through the monitoring screen, whether during normal times or during abnormal times, and can check the response action to the abnormality, etc., as described below, during abnormal times.

(3-3) Monitoring Screen Fig. 7 is a diagram for explaining an example of the screen configuration of the monitoring screen. The monitoring screen 300 displayed on the customer operation terminal 3 is, for example, a GUI (Graphical User Interface), and displays, based on the display data generated by the visualization processing unit 14, the video of the equipment 23 captured by the surveillance camera 22, abnormalities detected from the video of the equipment 23, and corresponding actions, etc.

As shown in FIG. 7, the monitoring screen 300 has different screen configurations for a normal state (when no abnormality is detected) (301) and an abnormal state (when one or more abnormalities are detected) (302).

The monitoring screen 301 under normal conditions is composed of a building information display section 3011 that displays information about the building 2 in which the monitored equipment 23 is installed, a message display section 3012 that displays a message indicating that no abnormalities have been detected, and a camera image display section 3013 that displays the name of each monitored equipment 23 and camera images (still images or video images) taken of the equipment 23.

The abnormality monitoring screen 302 includes, in addition to a building information display field 3011 and a camera image display field 3013 similar to those of the normal monitoring screen 301, an abnormality detection information display field 3021 that displays information about a detected abnormality, a response action display button 3022 for displaying a list of recommended response actions (actions to be taken next) for a detected abnormality, and a response action list 3023 that displays a list of response actions when the response action display button 3022 is pressed and can accept an operation to decide to carry out a response action.

Here, the abnormality detection information display field 3021 includes at least the name of the equipment where the abnormality was detected, the video at the time of the abnormality, the date and time of the abnormality, and the abnormality content. The abnormality detection information display field 3021 is preferably displayed in a more emphasized manner than in normal times so that the abnormality can be intuitively recognized as not being in a normal state. In this case, the abnormality detection information display field 3021 is displayed in a different manner from the message display field 3012 displayed in normal times, and specifically, the display frame of the video in which the abnormality was detected is displayed in red (e.g., green or black in normal times) or flashes (no flashing in normal times). The color of the text displayed in the abnormality detection information display field 3021 may be a different color from that in normal times. In addition, in the monitoring screen 302 in abnormal times, as shown in FIG. 7, it is preferable that the display mode of the abnormality detection information display field 3021, which displays the camera video of the equipment in which the abnormality was detected, is also displayed higher (in other words, in a more emphasized display manner) than the camera video display field 3013, which displays the camera video of the equipment in which the abnormality was not detected.

The response actions that can be displayed in the response action list 3023 are the action contents 1247 of the response action information linked to the detected abnormality content, and the number of responses may be one or more. In the response action list 3023, similar to the display mode of the abnormality detection information display field 3021 described above, the text color may be changed to red to indicate that an abnormality has occurred. In addition to the configuration exemplified in FIG. 7, the monitoring screen 302 during an abnormality may be provided with a button for acquiring and switching between the image at the time of the abnormality occurrence and the current image.

Each of the display areas 3011 to 3013 and 3021 to 3023 on the above-mentioned monitoring screens 301 and 302 is realized, for example, by a GUI component.

By configuring the monitoring screen 300 as described above, it is possible to display that no abnormality has been detected under normal circumstances, and when an abnormality occurs, it is possible to intuitively and clearly display the detected abnormality and the corresponding response action associated with the abnormality, thereby supporting the management work of customers such as building managers.

Figure 8 is a diagram showing an example of the monitoring screen display when an abnormality occurs. Figure 8 shows monitoring screen 303 when an abnormality of "smoke detection" is detected in the firefighting equipment, and monitoring screen 304 when an abnormality of "abnormal behavior (violence)" is detected in the elevator, and the screen configuration of these monitoring screens 303, 304 conforms to the screen configuration of monitoring screen 302 explained in Figure 7. Note that in Figure 8 and Figure 9 described below, an explanation is given of monitoring screen 300 that displays camera images of the monitored target for a customer record in customer information 121 in Figure 2 where customer 1211 is "XX Management Association", customer ID 1212 is "001", and building ID 1213 is "Building A".

The monitoring screen 303 is an example of the display when an abnormality in "smoke detection" is detected in firefighting equipment.

On the monitoring screen 303, the building information display field 3031 outputs the building ID "Building A" and the building address "○○1-2-3, □□ City, Ibaraki Prefecture" obtained from the customer information DB 12a (the specific values are listed in the building ID 1213 and building address 1214 in Figure 2).

In the abnormality detection information display field 3032, the building equipment "firefighting equipment" (specific values are listed in the customer information 121 in FIG. 2) acquired from the abnormality record DB 12c, the video data, the abnormality detection date and time, and the abnormality content "smoke detected" are output. As described above with respect to the abnormality monitoring screen 302 shown in FIG. 7, it is preferable that the abnormality detection information display field 3032 has a display mode different from that in normal times so that the abnormality can be intuitively recognized. Specifically, for example, when the display frame of the camera image of other equipment (such as "automatic door" in the figure) in which no abnormality has been detected is green, the display frame of the camera image displayed in the abnormality detection information display field 3032 can be colored red.

In the response action list 3033, the values of the action number 1246, the action content 1247, and the contact 1248 in the response action record corresponding to the combination of the abnormality type "smoke/fire" and the abnormality content "smoke detected" of the detected abnormality are output. Specifically, in the response action list 3033, "call a fire engine" is displayed as the action content corresponding to the action number "1", and the telephone number "119" is displayed as the contact. In addition, "announce via in-house broadcast" is displayed as the action content corresponding to the action number "2", and a button labeled "make broadcast" is displayed instead of the contact. In this case, if a smartphone is used as the customer operation terminal 3, the customer can make a call to the displayed contact or make an in-house broadcast by tapping "119" or "make broadcast" displayed in the response action list 3033.

Monitoring screen 304 is an example of a display when an "abnormal behavior" is detected in an elevator. The screen configuration of monitoring screen 304 (e.g., building information display field 3041, abnormality detection information display field 3042, response behavior list 3043) is the same as monitoring screen 303, and the acquisition of various information output to monitoring screen 304 can be considered to be similar to the explanation for monitoring screen 303, so a detailed explanation will be omitted.

Figure 9 is a diagram showing another example of the monitoring screen display when an abnormality occurs. The monitoring screen 305 shown in Figure 9 is an example of the monitoring screen 300 display when an abnormality is detected in multiple pieces of equipment 23 to be monitored.

Specifically, in the case of the monitoring screen 305 in FIG. 9, an abnormality of "fire detection" in the firefighting equipment and an abnormality of "smoke detection" in the automatic door are detected. In this example, in the response action information 124 registered in the response action DB 12d, the abnormality content priority 1245 of the fire detection in the firefighting equipment is higher than the abnormality content priority 1245 of the smoke detection in the automatic door.

At this time, on the monitoring screen 305 of FIG. 9, a first abnormality detection information display field 3052 is arranged below the building information display field 3051, and a second abnormality detection information display field 3054 is arranged below that. The first abnormality detection information display field 3052 displays information about the abnormality with the highest abnormality content priority among the abnormalities currently detected, and the second abnormality detection information display field 3054 displays information about the abnormality with the next highest abnormality content priority. That is, the first abnormality detection information display field 3052 displays the detection of a fire in a firefighting facility, and the second abnormality detection information display field 3054 displays the detection of a smoke in an automatic door. In addition, a normality information display field 3055 showing images of facilities 23 (elevators, parking lot, rooftop) in which no abnormality has been detected is arranged below the second abnormality detection information display field 3054.

In the monitoring screen 305 displayed as above, for example, when the response action display button corresponding to the first abnormality detection information display field 3052 is pressed, a list of response actions linked to the detection of fire in the firefighting equipment is displayed in the response action list 3053, and the customer can select and execute a response action from among the response actions of "calling a fire engine" or "announcing over the public address system." Similarly, when the response action display button corresponding to the second abnormality detection information display field 3054 is pressed, a list of response actions linked to the detection of smoke in the automatic door is displayed in the response action list, and the customer can select a response action to execute.

When multiple abnormalities are detected, as in the monitoring screen 305, the abnormality with the highest importance is displayed at the top (in a prominent location) based on the abnormality priority set in the response action information 124, allowing the customer to intuitively recognize the abnormality that should be dealt with as a priority, and to efficiently take action to respond to the abnormality.

As described above, the video monitoring system 100 (video monitoring device 1) according to this embodiment can determine an abnormal state by analyzing the video from the surveillance camera 22 associated with the monitored facility 23 without the need for an abnormality detection sensor, and the next action to be taken according to the determination result, i.e., information on the response action, is displayed on the customer operation terminal 3 of the user (customer) along with the video of the detected abnormality, thereby supporting the user's response action to the detected abnormality while keeping implementation costs down. In addition, by providing the customer operation terminal 3 with not only the video from the surveillance camera 22 but also an intuitive and easy-to-understand display screen, the burden of management work on building managers and the like can be reduced.

The present invention is not limited to the above-described embodiment and modifications, but includes various modifications. For example, the above-described embodiment has been described in detail to explain the present invention in an easy-to-understand manner, and is not necessarily limited to having all of the configurations described. In addition, it is possible to add, delete, or replace part of the configuration of the embodiment with other configurations.

Furthermore, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in hardware, in part or in whole, for example by designing them as integrated circuits. Furthermore, the above-mentioned configurations, functions, etc. may be realized in software, by a processor interpreting and executing a program that realizes each function. Information on the programs, tables, files, etc. that realize each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

In addition, the control lines and information lines shown in the drawings are those that are considered necessary for explanation, and do not necessarily show all control lines and information lines on the product. In reality, it can be assumed that almost all components are interconnected.

1. Video monitoring device 2. Building
3 Customer operation terminal 4, 5 Communication network 11 Customer information input unit 12 Storage device 12a Customer information DB
12b Camera image management DB
12c Abnormality record DB
12d Response Action DB
REFERENCE SIGNS LIST 13 Camera image acquisition unit 14 Visualization processing unit 15 Camera image analysis unit 16 Response action selection unit 17 Communication unit 21 Camera image recording device 22 Surveillance camera 23 Equipment 100 Video monitoring system 121 Customer information 124 Response action information 300 Monitoring screen

Claims (10)

A video monitoring device that monitors abnormalities in a monitored facility using video from a surveillance camera that captures the facility,
a camera image acquisition unit that acquires image data of the facility captured by the surveillance camera;
A camera image analysis unit that detects an abnormality in the equipment from the image of the video data and identifies the content of the detected abnormality;
a storage device that stores customer information that associates a customer who has concluded a maintenance contract for monitoring the facility, the facility that is the subject of monitoring under the maintenance contract, and information about the monitoring camera that captures the facility, and response action information that associates one or more response actions that can be selected by the customer when the abnormality is detected with each of the abnormality contents;
a response action selection unit that acquires one or more of the response actions associated with the abnormality content identified by the camera image analysis unit based on the response action information stored in the storage device;
a visualization processing unit that generates display data for displaying a monitoring screen displaying the image of the image data acquired by the camera image acquisition unit on a customer operation terminal operated by the customer, and displays the monitoring screen on the customer operation terminal using the display data;
Equipped with
a display unit that displays the monitoring screen on the customer operation terminal using the display data for abnormalities when the camera image analysis unit detects an abnormality in the equipment, the display data for abnormalities including a display of the image in which the abnormality was detected and a list of the one or more response actions acquired by the response action selection unit in response to the detection of the abnormality, and the customer is able to determine which response action to perform from the list of the one or more response actions. In the response action information, a plurality of response actions can be registered by assigning different action priorities as the response action corresponding to one of the abnormality contents,
When the plurality of response actions are associated in the response action information with the abnormality content identified from the abnormality detected by the camera image analysis unit,
The response action selection unit acquires all of the plurality of corresponding response actions,
The video monitoring device according to claim 1, characterized in that the visualization processing unit generates display data for the abnormality so as to present the list of the plurality of response actions acquired by the response action selection unit on the monitoring screen in a display mode according to the action priority. 2. The video monitoring device according to claim 1, wherein, when the camera video analysis unit detects the abnormality, the visualization processing unit executes a predetermined output process to notify the customer operation terminal of the occurrence of the abnormality. The video monitoring device according to claim 1, characterized in that, when a request for launching the monitoring screen is made from the customer operation terminal, the visualization processing unit allows the display of the monitoring screen on the customer operation terminal only if authentication using the customer information is met. When there are a plurality of pieces of equipment that are to be monitored under the maintenance contract, the visualization processing unit generates the display data so as to display images of each of the plurality of pieces of equipment on the monitoring screen;
When the camera image analysis unit detects an abnormality in any one of the plurality of pieces of equipment,
The video monitoring device according to claim 1, characterized in that the visualization processing unit generates display data for the abnormality so that video of equipment in which an abnormality has been detected is displayed in a more emphasized manner on the monitoring screen than video of equipment in which an abnormality has not been detected. When the camera image analysis unit detects an abnormality in two or more pieces of equipment among the plurality of pieces of equipment, the camera image analysis unit identifies the abnormality content for each of the detected abnormalities,
the response action selection unit acquires, for each of the detected abnormalities, one or more response actions associated with respective abnormality contents based on the response action information;
The video monitoring device according to claim 5, characterized in that the visualization processing unit generates display data for the abnormality so that, for each detected abnormality, a video in which the abnormality was detected and a list of the one or more response actions to the abnormality are displayed on the monitoring screen. An abnormality priority is assigned to each of the plurality of abnormality contents registered in the response action information,
When the camera image analysis unit detects an abnormality in two or more pieces of equipment among the plurality of pieces of equipment and identifies different abnormality contents for each abnormality,
the response action selection unit acquires, for each of the detected abnormalities, one or more response actions associated with respective abnormality contents based on the response action information;
The video monitoring device according to claim 5, characterized in that the visualization processing unit generates display data for when an abnormality occurs, so that, for each detected abnormality, a video in which the abnormality was detected and a list of the one or more response actions for the abnormality are displayed on the monitoring screen in a display order according to the abnormality priority assigned to the abnormality content of each abnormality. The video monitoring device according to claim 1, further comprising a response action processing unit that executes a specific response action when an operation for deciding to execute the specific response action is performed on the monitoring screen displayed on the customer operation terminal using the display data for abnormal situations. A video monitoring system for monitoring abnormalities in a monitored facility, comprising:
A surveillance camera that photographs the facility;
a camera image recording device that records image data of the facility captured by the surveillance camera;
a customer operation terminal operated by a customer who has concluded a maintenance contract for monitoring the facility;
a video monitoring device that detects an abnormality in the equipment using video data of the equipment, and presents, when the abnormality is detected, a list of next response actions that can be selected by the customer on the customer operation terminal;
Equipped with
The video monitoring device comprises:
a camera image acquisition unit that acquires image data of the facility from the camera image recording device;
A camera image analysis unit that detects an abnormality in the equipment from the image of the video data and identifies the content of the detected abnormality;
a storage device that stores customer information in which a customer who has concluded the maintenance contract, the equipment that is a monitoring target of the maintenance contract, and information on the monitoring camera that captures the equipment are associated with each other, and response action information in which the response action to the abnormality is associated with one or more of the abnormality contents of the abnormality;
a response action selection unit that acquires one or more of the response actions associated with the abnormality content identified by the camera image analysis unit based on the response action information stored in the storage device;
a visualization processing unit that generates display data for displaying a monitoring screen, which displays the image of the image data acquired by the camera image acquisition unit, on a customer operation terminal operated by the customer, and displays the monitoring screen on the customer operation terminal using the display data;
having
When the camera image analysis unit detects an abnormality in the equipment,
the visualization processing unit generates display data for an abnormality including a display of the video in which the abnormality is detected and a list of the one or more response actions acquired by the response action selection unit in response to the detection of the abnormality, and displays the monitoring screen on the customer operation terminal using the display data for an abnormality in a manner that allows the customer to perform an operation of deciding which response action to execute from the list of the one or more response actions ;
The customer operation terminal determines a response action to be executed from the list of response actions presented on the monitoring screen in response to an operation by the customer . A video monitoring method using a video monitoring device that monitors an abnormality in a monitored facility using video from a surveillance camera that captures the facility, comprising:
The video monitoring device stores in a storage device customer information that associates information regarding a customer who has concluded a maintenance contract for monitoring the facility, the facility that is the subject of monitoring under the maintenance contract, and the monitoring camera that captures the facility, and response action information that associates one or more response actions that can be selected by the customer when an abnormality is detected in the facility with each type of the abnormality,
a camera image acquisition step in which the image monitoring device acquires image data of the facility captured by the surveillance camera;
A camera image analysis step in which the video monitoring device detects an abnormality in the equipment from the video of the video data and identifies the content of the detected abnormality;
a response action selection step in which the video monitoring device acquires the one or more response actions associated with the abnormality content identified in the camera video analysis step based on the response action information stored in the storage device;
a visualization processing step in which the video monitoring device generates display data for displaying a monitoring screen displaying the video of the video data acquired in the camera video acquisition step on a customer operation terminal operated by the customer, and displays the monitoring screen on the customer operation terminal using the display data;
Equipped with
A video monitoring method characterized in that, when an abnormality in the equipment is detected in the camera image analysis step, in the visualization processing step, the video monitoring device generates display data for abnormal situations, including a display of the image in which the abnormality was detected and a list of the one or more response actions obtained in the response action selection step in response to the detection of the abnormality, and uses the display data for abnormal situations to display the monitoring screen on the customer operation terminal in a manner that allows the customer to determine which response action to execute from the list of one or more response actions .

Images