JP7724709B2 - Image processing device, monitoring system and program - Google Patents

Description

This disclosure relates to a video processing device, a surveillance system, and a program that superimposes information onto video.

Technology for extracting people from various types of video, including surveillance footage from surveillance cameras, is known. Patent Document 1 discloses technology for identifying a person extracted from video captured by a video sensor using sensor data indicating the reception strength of radio waves received by a mobile terminal from a wireless signal generator. With the technology described in Patent Document 1, if the person extracted from video captured by the video sensor is the person holding the mobile terminal, the person's identification information is superimposed on the video. As a result, a monitor monitoring the video can determine whether the person in the video is an outsider based on whether or not the identification information is displayed.

JP 2018-61114 A

However, not all people carrying mobile devices are necessarily permitted to enter all areas. For example, in areas with strict security management, such as power plants, the same person's access may vary depending on the time of day, or even depending on the area. For this reason, a monitor who monitors by checking video footage cannot adequately determine whether or not to allow entry simply by checking a person's identification information using the technology described in Patent Document 1. Therefore, there is a need for the generation of video that can more appropriately assist in determining whether or not to allow entry into a monitored area.

The present disclosure has been made in light of the above, and aims to provide an image processing device that can generate images that can more appropriately assist in determining whether or not to allow entry into a monitored area.

To solve the above-mentioned problems and achieve the objectives, the video processing device disclosed herein includes a video data receiving unit that receives video data showing video captured of a monitored area of a power plant; a location information receiving unit that receives location information including the location of a terminal device in the monitored area and device identification information that identifies the terminal device; and a registration information storage unit that stores, for each device identification information, registration information including the device identification information and personal identification information that identifies a person carrying a terminal device corresponding to the device identification information. The video processing device further includes a video generation unit that generates location information video that shows the location included in the location information as the person's position and obtains registration information corresponding to the device identification information included in the location information from the registration information storage unit; and a permission plan storage unit that stores, for each time period, a permission plan that includes information indicating the personal identification information of people who are permitted to enter the monitored area. The video processing device further includes a determination unit that uses the personal identification information included in the registration information acquired by the video generation unit and the permission plan to determine whether the person corresponding to the personal identification information is permitted to enter the monitored area and identifies the personal identification information of the person determined to be prohibited from entering the monitored area, and a video superimposition unit that superimposes the location information video with the video corresponding to the video data, outputs the superimposed video as a monitoring video, and generates the monitoring video so that the position of the person corresponding to the personal identification information identified by the determination unit is highlighted.

The video processing device disclosed herein has the effect of more appropriately assisting in determining whether or not to allow entry into a monitored area.

FIG. 1 is a diagram illustrating a configuration example of a monitoring system according to a first embodiment. FIG. 1 is a diagram schematically illustrating the arrangement of a signal receiving device according to a first embodiment. 1 is a flowchart showing an example of a video processing procedure in the video processing device according to the first embodiment. FIG. 1 is a diagram showing an example of registration information according to the first embodiment; FIG. 1 is a diagram showing an example of a work plan according to the first embodiment; FIG. 1 is a diagram schematically illustrating a camera image output from a video data processing unit according to the first embodiment. FIG. 10 is a diagram schematically illustrating an example of a position information image output from the image generating unit according to the first embodiment; FIG. 1 is a diagram schematically illustrating an example of a superimposed image generated by a video superimposing unit according to the first embodiment; FIG. 1 is a diagram schematically illustrating an example of a monitoring image displayed on the image display device according to the first embodiment; FIG. 1 is a diagram showing an example of the configuration of a computer system that realizes a video processing device according to a first embodiment. FIG. 10 is a diagram illustrating a configuration example of a monitoring system according to a second embodiment. FIG. 10 is a diagram illustrating a configuration example of a monitoring system according to a third embodiment. FIG. 10 is a diagram illustrating a configuration example of a monitoring system according to a fourth embodiment. FIG. 10 is a diagram for explaining synchronization processing according to the fourth embodiment. FIG. 13 is a diagram illustrating a configuration example of a monitoring system according to a fifth embodiment. FIG. 20 is a diagram showing an example of trajectory information according to the fifth embodiment; FIG. 13 shows an example of a superimposed image in the fifth embodiment.

The video processing device, monitoring system, and program according to the embodiments are described in detail below with reference to the accompanying drawings.

Embodiment 1.
1 is a diagram showing an example of the configuration of a monitoring system according to a first embodiment. A monitoring system 7 of this embodiment includes a video processing device 1 that generates monitoring video used for monitoring a monitoring area, which is an area to be monitored, and a video display device 2 that displays the monitoring video. The video processing device 1 is a device that generates monitoring video by superimposing a video acquired by a video acquisition device 3 and a position information video generated based on position information acquired by a position information generation device 6.

Monitored areas are areas where access is controlled, such as power plants and other plants, production facilities, areas where work is performed that involves operations that could have serious consequences, and areas where confidential information is stored. The following describes an example in which the monitored area is an area within a power plant, but monitored areas are not limited to this.

The image capture device 3 is, for example, a camera or an image sensor that captures images of the monitoring area, which is the area to be monitored. While one image capture device 3 is shown in FIG. 1, there may be multiple image capture devices 3. Furthermore, the monitoring area to be monitored by the monitoring system 7 may be one or multiple. If there are multiple monitoring areas, one or more image capture devices 3 are provided in each monitoring area.

In this embodiment, workers performing work or management at a power plant wear signal transmission devices 5 to detect their location. The signal transmission devices 5 are attached to the workers' helmets, for example, but the location where the signal transmission devices 5 are attached is not limited to this. The signal transmission devices 5 are transmitters also known as beacons, and transmit, as wireless signals, signals containing beacon identification information that is identification information unique to the signal transmission device 5. While FIG. 1 illustrates only one signal reception device 4 and omits the illustration of other signal reception devices 4, multiple signal reception devices 4 are disposed within the monitoring area. Each signal reception device 4 is disposed at multiple different locations within the monitoring area, and adds its own identification information to received information that includes information indicating the reception strength of the wireless signal received from the signal transmission device 5 and the beacon identification information contained in the received wireless signal, and transmits the received information to the position information generation device 6. The communication path between the signal reception device 4 and the position information generation device 6 may be wired, wireless, or a combination of wired and wireless.

The location information generating device 6 calculates the location of the signal transmitting device 5 that is the source of the signal corresponding to the received information based on the reception strength of the received information received from each signal receiving device 4, generates location information including the calculated location and the beacon identification information included in the received information, and transmits the generated location information to the video processing device 1.

Figure 2 is a diagram schematically illustrating the placement of signal receiving devices 4 in this embodiment. In the example shown in Figure 2, five signal receiving devices 4 are provided within a monitoring area 8, and a worker 9 wearing a signal transmitting device 5 is present within the monitoring area 8. Wireless signals transmitted from the signal transmitting device 5 are received by multiple signal receiving devices 4 within the monitoring area 8. In addition, in the example shown in Figure 2, two image capturing devices 3 are provided in the monitoring area 8. Figure 2 is a diagram for explaining the concept of position detection in this embodiment, and the number and placement positions of signal receiving devices 4 and image capturing devices 3 are not limited to the example shown in Figure 2.

Note that, while an example is described here in which each signal receiving device 4 calculates the position of the signal transmitting device 5 using the reception strength of the wireless signal received from the signal transmitting device 5, the position information generating device 6 may also calculate the position using the angle of incidence of the signal. Furthermore, the worker may also wear an acceleration sensor, and the position information generating device 6 may also calculate the worker's position using movement information obtained from the acceleration sensor. Alternatively, multiple transmitters may be installed in the monitoring area, and the worker may wear a wireless communication device as a terminal device, and the wireless communication device may transmit the reception strength of signals received from the multiple transmitters to the position information generating device 6 along with device identification information, which is identification information for the wireless communication device. In this case, the position information generating device 6 calculates the position of each wireless communication device using the reception strength received by the wireless communication device from each transmitter.

Furthermore, if the area is outdoors or elsewhere where GPS (Global Positioning System) positioning is possible, instead of using the signal receiving device 4, signal transmitting device 5, and position information generating device 6, the worker may wear a wireless communication device equipped with a GPS receiver that performs GPS positioning as a terminal device, and transmit the position calculated by the GPS receiver and device identification information, which is identification information for the wireless communication device, to the video processing device 1 as position information. The GPS receiver may also perform high-precision positioning using augmentation information.

The above-described method for detecting the worker's position is one example, and the method for detecting the worker's position is not limited to the above example and any method may be used. In other words, the signal transmission device 5 is an example of a terminal device carried by a human worker, and the beacon identification information is an example of device identification information that identifies the terminal device. In this embodiment, it is sufficient to calculate the position of the terminal device carried by the worker, and there are no particular restrictions on the method for calculating the position of the terminal device, and any method may be used.

The monitoring system 7 may also include a signal receiving device 4, a signal transmitting device 5, and a location information generating device 6. Furthermore, the monitoring system 7 may also include an image capturing device 3.

As shown in FIG. 1, the video processing device 1 includes a video data receiving unit 12, a video data processing unit 13, a position information receiving unit 14, a coordinate calculation unit 15, a video generation unit 16, a video superimposition unit 17, a registration information storage unit 18, a management information receiving unit 19, a work plan storage unit 20, a determination unit 21, and an alarm unit 22.

The video data receiving unit 12 receives video data showing video captured of the monitoring area of the power plant. Specifically, the video data receiving unit 12 receives video data, which is data of video captured by the video capture device 3, from the video capture device 3 and outputs the received video data to the video data processing unit 13. The video data processing unit 13 performs video processing such as distortion correction and color correction on the video data, and outputs the processed video data to the video superimposition unit 17 as camera video. If multiple video capture devices 3 are installed in one monitoring area, the video data processing unit 13 may perform synthesis processing to generate a single camera video using the video data from these multiple video capture devices. The synthesis processing may be overhead video synthesis processing. Note that if the video data received from the video capture device 3 is encoded, the video data processing unit 13 also performs decoding processing. The video data processing unit 13 may also extract an area corresponding to a person from the video corresponding to the video data. For example, the video data processing unit 13 may extract a person appearing in the video represented by the video data, and add an area corresponding to the extracted person as a bounding box to the video data to generate a camera image.

The position information receiving unit 14 receives position information including the position of the signal transmitting device 5, which is a terminal device in the monitoring area, and beacon identification information, which is device identification information for the signal transmitting device 5. In detail, the position information receiving unit 14 receives position information from the position information generating device 6 and outputs the received position information to the coordinate calculation unit 15. The coordinate calculation unit 15 calculates the coordinates in the camera image for each position included in the position information. In other words, the coordinate calculation unit 15 converts the positions included in the position information into coordinate values in a coordinate system that matches the camera image, and outputs the coordinate values obtained by the conversion and the beacon identification information included in the position information to the image generating unit 16.

The management information receiving unit 19 receives, as management information, registration information related to workers and work plans from the management device 10, which manages work at the power plant. The management information receiving unit 19 stores the received registration information in the registration information storage unit 18 and stores the work plans in the work plan storage unit 20. The registration information is information related to the worker corresponding to each beacon identification information, and includes device identification information and worker identification information that identifies the worker carrying the signal transmission device 5 corresponding to the device identification information. Details of the registration information will be described later. The work plan is information that indicates the workers who will be working in the monitored area and the work content for each time period. The work plan is an example of a permission plan that includes information indicating the personal identification information of people who are permitted to enter the monitored area for each time period. Details of the work plan will be described later.

The image generation unit 16 generates a location information image, which is an image showing the location included in the location information as the person's location, and acquires registration information corresponding to the beacon identification information included in the location information from the registration information storage unit 18. Specifically, the image generation unit 16 uses the coordinate values received from the coordinate calculation unit 15 to generate a location information image in which the worker's position (person's position), which is the position of the coordinate values, is indicated by a mark such as a figure or letter. The image generation unit 16 also extracts registration information corresponding to the beacon identification information received from the coordinate calculation unit 15 from the registration information stored in the registration information storage unit 18, associates the extracted registration information with the worker, adds the registration information associated with the worker to the location information image, and outputs the location information image with the added registration information to the image superimposition unit 17. In this way, the image generation unit 16 associates the registration information corresponding to the beacon identification information with the worker's position corresponding to the beacon identification information, and outputs the associated registration information to the image superimposition unit 17 together with the location information image. Note that, although an example of adding registration information to the position information video is described here, the video generation unit 16 may not add the registration information to the position information video, but may instead output the registration information to the video superimposition unit 17 in association with each worker separately from the position information video.

The image superimposition unit 17 outputs the worker identification information in the registration information corresponding to each task added to the position information image to the determination unit 21. The image superimposition unit 17 also generates a superimposed image by superimposing the camera image and the position information image. The image superimposition unit 17 also associates people in the camera image with worker positions in the position information image. For example, the image superimposition unit 17 determines the worker position corresponding to the coordinate value with the shortest distance from the representative point of the bounding box as the worker position corresponding to the bounding box. The representative point of the bounding box may be the center of the bounding box, the center of the feet in the bounding box, or the center of the head in the bounding box. The image data processing unit 13 may also detect the position of the helmet, and determine the center of the helmet as the representative point of the bounding box. Furthermore, if there is a bounding box that is not associated with the worker's position, the video superimposition unit 17 determines that the person is not holding a signal transmission device 5 and notifies the determination unit 21 that there is a person who is not holding a signal transmission device 5.

The determination unit 21 uses the worker identification information and work plan included in the registration information acquired by the video generation unit 16 to determine whether the worker corresponding to the worker identification information is permitted to enter the monitored area, and identifies the worker identification information of the worker determined to be prohibited from entering the monitored area. Specifically, the determination unit 21 references the work plan stored in the work plan storage unit 20 and determines whether the worker corresponding to the worker identification information notified by the video superimposition unit 17 is currently permitted to enter the monitored area. If the workers corresponding to the worker identification information in the monitored video include a worker who is not permitted to enter the monitored area, the determination unit 21 determines that an abnormality has occurred, notifies the video superimposition unit 17 of the worker identification information corresponding to the worker who is not permitted to enter, and instructs the alarm unit 22 to issue an alarm. Furthermore, if the determination unit 21 is notified by the video superimposition unit 17 that a person not carrying a signal transmission device 5 is present, the determination unit 21 determines that an abnormality has occurred and instructs the alarm unit 22 to issue an alarm.

The video overlay unit 17 generates a surveillance video in which the worker position corresponding to the worker identification information identified by the determination unit 21 is highlighted. Specifically, if the video overlay unit 17 detects at least one of a bounding box corresponding to the worker identification information notified by the determination unit 21 and a bounding box not associated with a worker position, the video overlay unit 17 modifies the superimposed video to highlight the bounding box, and uses the modified superimposed video as the surveillance video. Highlighting may be achieved, for example, by changing the color to a specific color, blinking, changing at least a portion of the shape, or adding graphics or text, but is not limited to these, as long as it can be distinguished from other bounding boxes. The method of highlighting may be different for the bounding box corresponding to the worker identification information notified by the determination unit 21 and the bounding box not associated with a worker position. Furthermore, the video superimposition unit 17 adds information indicating that the person is not holding a signal transmission device 5 to bounding boxes not associated with the worker position, and adds information indicating that the worker is not permitted to enter to bounding boxes corresponding to worker identification information notified by the determination unit 21. The video superimposition unit 17 outputs the surveillance video to the video display device 2, causing the video display device 2 to display the surveillance video. The video superimposition unit 17 also outputs registration information associated with the worker to the video display device 2 along with the surveillance video. If the determination unit 21 does not notify the video superimposition unit 17 of worker identification information, the video superimposition unit 17 outputs the above-mentioned superimposed video as surveillance video to the video display device 2.

When the determination unit 21 instructs the alarm unit 22 to issue an alarm, it issues an alarm to warn that an unauthorized person is present in the monitored area. The alarm may be a sound such as a voice or warning tone, or may be the illumination of a lamp, or a combination of these. Also, although the alarm unit 22 is provided within the video processing device 1 in FIG. 1, the alarm unit 22 may also be provided outside the video processing device 1.

Next, the operation of this embodiment will be described. Figure 3 is a flowchart showing an example of the video processing procedure in the video processing device 1 of this embodiment. As shown in Figure 3, the video processing device 1 receives video data (step S1) and processes the video data (step S2). In detail, in step S1, the video data receiving unit 12 receives video data from the video acquisition device 3 and outputs the video data to the video data processing unit 13. In step S2, the video data processing unit 13 performs video processing on the video data and outputs the processed video data to the video superimposition unit 17 as camera video.

The video processing device 1 also receives position information (step S3), calculates coordinates (step S4), and generates position information video (step S5). Specifically, in step S3, the position information receiving unit 14 receives position information from the position information generating device 6 and outputs the received position information to the coordinate calculation unit 15. In step S4, the coordinate calculation unit 15 converts each position in the position information to a coordinate value in the camera video and outputs the converted coordinate value together with the beacon identification information in the position information to the video generation unit 16. In step S5, the video generation unit 16 generates position information video in which a mark is displayed at the worker position corresponding to the coordinate value received from the coordinate calculation unit 15. The video generation unit 16 then extracts registration information corresponding to the beacon identification information received from the coordinate calculation unit 15 from the registration information stored in the registration information storage unit 18, adds the extracted registration information to the position information video, and outputs the position information video with the registration information added to the video superimposition unit 17.

Figure 4 is a diagram showing an example of registration information in this embodiment. The registration information storage unit 18 stores, for example, the registration information shown in Figure 4. In the example shown in Figure 4, the registration information includes beacon identification information, worker name, affiliated company, department, position, and assigned work. Figure 4 is just one example, and the registration information may include beacon identification information and worker identification information, which is personal identification information that identifies the person corresponding to the signal transmission device 5 that corresponds to the beacon identification information. The worker name is an example of worker identification information for the person carrying the signal transmission device 5, but the worker identification information is not limited to the worker name and may be a combination of the worker name and at least one of the affiliated company, department, and position. Furthermore, if the worker has been assigned unique identification information separate from the worker name, that identification information may be used.

Returning to the explanation of Figure 3, after steps S2 and S5, the video processing device 1 performs video overlay processing (step S6). In detail, the video overlay unit 17 generates a superimposed video by overlaying the camera video and the position information video, and outputs the worker name corresponding to each worker position in the position information video to the determination unit 21. Furthermore, if there is a bounding box that is not associated with a worker position, the video overlay unit 17 notifies the determination unit 21 of this.

The video processing device 1 determines whether or not an abnormality has occurred (step S7). In particular, the determination unit 21 determines that an abnormality has occurred when the video superimposition unit 17 notifies the determination unit 21 that a bounding box not associated with a worker's position exists. The determination unit 21 also references the work plan stored in the work plan memory unit 20 and determines that an abnormality has occurred when a worker corresponding to the worker name notified by the video superimposition unit 17 is not currently permitted to enter the monitored area. The determination unit 21 determines that no abnormality has occurred when the video superimposition unit 17 has not notified the determination unit 21 that a bounding box not associated with a worker's position exists and all of the workers corresponding to the worker names notified by the video superimposition unit 17 are currently permitted to enter the monitored area.

Figure 5 is a diagram showing an example of a work plan according to this embodiment. As shown in Figure 5, the work plan includes information indicating, for example, work (work content) and the worker in charge, for each time period. In the example shown in Figure 5, the worker's name is used as worker identification information for identifying the worker, i.e., personal identification information for identifying the person. However, the worker identification information is not limited to the worker's name and may be identification information uniquely assigned separately from the worker's name. Furthermore, beacon identification information corresponding to the worker may also be used as the worker identification information. Furthermore, as described above, the worker identification information may be a combination of the worker's name and other information, such as a combination of the worker's name and the company and department to which they belong. If there are multiple monitored areas, the work plan storage unit 20 stores a work plan for each monitored area. Furthermore, a work plan may be created so that multiple tasks are performed at the same time. Figure 5 is merely an example, and the contents of the work plan are not limited to the example shown in Figure 5.

The determination unit 21 is monitoring area R1, and if the current time is 9:00 and the names of the workers notified by the video superimposition unit 17 are A, B, and D, then a contractor with worker name A and a worker with worker name B are scheduled to work in monitoring area R1 and are permitted to enter, but worker name D is not scheduled to work at 9:00. In this case, the determination unit 21 determines that an abnormality has occurred.

Returning to the explanation of Figure 3, if an abnormality is detected (step S7: Yes), the video processing device 1 issues an alarm (step S8), performs highlighting processing (step S9), and outputs the video (monitoring video) (step S10). Specifically, in step S8, the determination unit 21 causes the alarm unit 22 to issue an alarm. In addition, in step S9, the video superimposition unit 17 generates the monitoring video so that bounding boxes determined to be abnormal, i.e., bounding boxes not associated with worker positions and bounding boxes corresponding to workers determined by the determination unit 21 to not currently have permission to enter, are highlighted. While an example of highlighting bounding boxes is described here, this is not limiting, and highlighting may also be achieved by displaying graphics or text within or around the bounding boxes. In step S10, the video superimposition unit 17 outputs the monitoring video together with the registration information to the video display device 2.

If no abnormality is detected (No in step S7), the video processing device 1 proceeds to step S10. In other words, if no abnormality is detected, the video superimposition unit 17 outputs the superimposed video, which is not subjected to highlighting processing, as the monitoring video to the video display device 2.

Note that the processing shown in Figure 3 is for one frame of video data, and the processing shown in Figure 3 is performed sequentially for each frame. Also, here, it is assumed that the imaging cycle of the video data and the generation cycle of the position information in the position information generating device 6 are the same.

Through the above processing, the video processing device 1 can issue an alert when at least one of a worker not wearing a signal transmission device 5 and a worker not permitted to enter the monitored area according to the work plan is captured in camera footage. Furthermore, when at least one of a worker not wearing a signal transmission device 5 and a worker not permitted to enter the monitored area according to the work plan is captured in camera footage, the video display device 2 highlights the bounding box, which is the area corresponding to the corresponding person. This allows the monitor to easily identify people not permitted to enter according to the work plan.

Figure 6 is a diagram that schematically illustrates camera footage output from the video data processing unit 13 of this embodiment. In the example shown in Figure 6, two people appear in the camera footage, and through processing in the video data processing unit 13, the areas corresponding to these people are detected as bounding boxes 201 and 202, respectively.

Figure 7 is a diagram showing an example of a position information image output from the image generation unit 16 of this embodiment. In the example shown in Figure 7, worker positions 203 and 204 corresponding to the positions indicated by the position information are shown. In addition, each black circle has the corresponding beacon identification information 001 and 002 added to it.

Figure 8 is a diagram showing a schematic example of a superimposed image generated by the image superimposition unit 17 of this embodiment. The superimposed image shown in Figure 8 is an image obtained by superimposing the camera image shown in Figure 6 and the location information image shown in Figure 7.

Figure 9 is a schematic diagram illustrating an example of a surveillance image displayed on the video display device 2 of this embodiment. Figure 9 shows an example of surveillance image in which the determination unit 21 determines that the person in the upper left is a worker who is not currently authorized to enter the restricted area. As shown in Figure 9, the bounding box corresponding to the worker is displayed in a different form from the other bounding boxes, and the words "No Entry Authorization" are associated with the bounding box, indicating that the worker is not authorized to enter the restricted area. Although a bounding box is attached to the surveillance image in the example shown in Figure 9, it is not necessary for a bounding box to be attached to the surveillance image. In addition, in the example shown in Figure 9, the registered information for the person on the right is displayed within a rectangle. For example, when the corresponding person in the surveillance image is selected, the registered information may be displayed as shown in Figure 9. This allows the worker to obtain information about the worker he or she wishes to check. The work time extracted from the work plan may also be displayed together with the registered information shown in Figure 9. The images shown in Figures 6 to 9 are merely examples, and the displayed shapes, text, etc. are not limited to these.

Figure 9 shows an example in which, when a person in the surveillance video is selected while the surveillance video is displayed on the video display device 2, registration information is displayed as shown in Figure 9. However, the method of displaying registration information is not limited to the example shown in Figure 9. For example, registration information may be displayed as text for each worker in an area separate from the surveillance video. In this case, identification information such as beacon identification information or worker name that identifies the worker is displayed for each person in the surveillance video, and registration information is displayed for each identification information, allowing the monitor to check the registration information of the worker in the surveillance video. Note that when the surveillance video and registration information are displayed simultaneously in separate areas, the video superimposition unit 17 may generate a video in which the surveillance video and a video in which the registration information is displayed as text are side-by-side and output this to the video display device 2. Alternatively, the video display device 2 may have a function for generating display data that displays the registration information as text, and use the generated display data to display the registration information and surveillance video side-by-side.

Furthermore, as shown in FIG. 9, when the registration information corresponding to a person designated by a user is displayed on the video display device 2, the generation of an image including the registration information corresponding to the selected person may be performed, for example, by the video display device 2, the video superimposition unit 17, or an image processing unit (not shown) provided downstream of the video superimposition unit 17 in the video processing device 1. Note that if the selected person is not a worker, i.e., if there is no registration information corresponding to the selected person, the registration information is not displayed. When the video display device 2 generates an image including the registration information, an input means (not shown) of the video display device 2 or an input means connected to the video display device 2 receives a person selection result from a user, such as a monitor, and the video display device 2 superimposes the registration information corresponding to the selection result on the position of the designated person on the surveillance video, and displays the surveillance video with the superimposed registration information. The input means may be a mouse, a keyboard, or a touch panel integrated with the display screen of the video display device 2. The selection result may be indicated by the worker's identification information, such as beacon identification information, or by the person's position in the surveillance video. When the video superimposition unit 17 or an image processing unit (not shown) generates video including registration information, after the surveillance video is displayed on the video display device 2, an input means of the video display device 2 (not shown) or an input means connected to the video display device 2 receives a person selection result from a user such as a security officer and notifies the video processing device 1. The video superimposition unit 17 or an image processing unit (not shown) superimposes the registration information corresponding to the selection result notified from the video display device 2 onto the surveillance video and outputs the surveillance video with the registration information superimposed to the video display device 2. As a result, the video display device 2 generates surveillance video with the registration information superimposed. In other words, the video display device 2 displays the registration information along with the surveillance video.

Next, the hardware configuration of the video processing device 1 of this embodiment will be described. The video processing device 1 of this embodiment functions as the video processing device 1 by executing a video processing program, which is a computer program that describes the processing to be performed by the video processing device 1, on the computer system. Figure 10 is a diagram showing an example configuration of a computer system that realizes the video processing device 1 of this embodiment. As shown in Figure 10, this computer system includes a control unit 101, an input unit 102, a memory unit 103, a display unit 104, a communication unit 105, and an output unit 106, which are connected via a system bus 107.

In FIG. 10, the control unit 101 is a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), and executes a program describing the processing of the video processing device 1 of this embodiment. The input unit 102 is composed of, for example, a keyboard, a mouse, etc., and is used by the user of the computer system to input various information. The memory unit 103 includes various types of memory, such as RAM (Random Access Memory) and ROM (Read Only Memory), and a storage device, such as a hard disk, and stores programs to be executed by the control unit 101, necessary data obtained during processing, etc. The memory unit 103 is also used as a temporary storage area for programs. The display unit 104 is composed of a display, LCD (Liquid Crystal Display Panel), etc., and displays various screens to the user of the computer system. The communication unit 105 is a receiver and transmitter that performs communication processing. The output unit 106 is a printer, speaker, etc. Note that FIG. 10 is an example, and the configuration of the computer system is not limited to the example in FIG. 10.

Here, we will explain an example of the operation of the computer system until the program of this embodiment is ready to be executed. In a computer system configured as described above, for example, a computer program is installed into storage unit 103 from a CD-ROM or DVD-ROM inserted in a CD (Compact Disc)-ROM drive or DVD (Digital Versatile Disc)-ROM drive (not shown). When the program is executed, the program is read from storage unit 103 and stored in the main memory area of storage unit 103. In this state, control unit 101 executes processing as video processing device 1 of this embodiment in accordance with the program stored in storage unit 103.

In the above explanation, the program describing the processing in the video processing device 1 is provided on a CD-ROM or DVD-ROM as a recording medium. However, this is not limiting. Depending on the configuration of the computer system and the capacity of the program provided, for example, a program provided via a transmission medium such as the Internet via the communication unit 105 may also be used.

The program of this embodiment causes a computer system to execute, for example, the following steps: receive video data showing video captured of a monitored area of a power plant; receive location information including the location of a terminal device in the monitored area and device identification information that identifies the terminal device; and generate location information video that shows the person's location indicated by the location information along with corresponding additional information. Furthermore, the program of this embodiment causes the computer system to execute the following steps: store, for each device identification information, registration information including the device identification information and person identification information that identifies the person carrying the terminal device corresponding to the device identification information in the registration information storage unit 18; generate location information video that shows the location included in the location information as the person's location, and acquire from the registration information storage unit 18 the registration information corresponding to the device identification information included in the location information; and store, in the permission plan storage unit, a permission plan that includes information indicating the person identification information of people who are permitted to enter the monitored area for each time period. Furthermore, the program of this embodiment causes the computer system to execute the steps of using the personal identification information included in the acquired registration information and the permission plan to determine whether the person corresponding to the personal identification information is permitted to enter the monitored area, and identifying the personal identification information of the person determined to be prohibited from entering the monitored area; and superimposing the location information image with an image corresponding to the image data, outputting the superimposed image as a monitored image, and generating the monitored image so that the position of the person corresponding to the identified personal identification information is highlighted.

The video data processing unit 13, coordinate calculation unit 15, video generation unit 16, video superimposition unit 17, and determination unit 21 shown in FIG. 1 are realized by the control unit 101 shown in FIG. 10 executing a computer program stored in the memory unit 103 shown in FIG. 10. The memory unit 103 shown in FIG. 10 is also used to realize the video data processing unit 13, coordinate calculation unit 15, video generation unit 16, video superimposition unit 17, and determination unit 21 shown in FIG. 1. The registration information memory unit 18 and work plan memory unit 20 shown in FIG. 1 are part of the memory unit 103 shown in FIG. 10. The video data receiving unit 12, location information receiving unit 14, and management information receiving unit 19 shown in FIG. 1 are realized by the communication unit 105 shown in FIG. 10. The alarm unit 22 is realized, for example, by the output unit 106 shown in FIG. 10. The video processing device 1 may also be realized by multiple computer systems. For example, the video processing device 1 may be realized by a cloud computer system. Furthermore, at least some of the video data processing unit 13, coordinate calculation unit 15, video generation unit 16, video superimposition unit 17, and determination unit 21 may be implemented using dedicated hardware. For example, some of the video data processing unit 13 and video superimposition unit 17 may be implemented using dedicated hardware for video processing.

As described above, in this embodiment, an alarm is issued when at least one of a worker not wearing a signal transmission device 5 and a worker not permitted to enter the monitored area according to the work plan is captured in camera footage. Furthermore, in this embodiment, when at least one of a worker not wearing a signal transmission device 5 and a worker not permitted to enter the monitored area according to the work plan is captured in camera footage in which the camera footage and the location information image are superimposed, the corresponding area can be highlighted. This allows this embodiment to more appropriately assist in determining whether or not to enter the monitored area compared to when the work plan, which is information indicating whether or not entry is permitted according to time periods, is not taken into account.

Embodiment 2.
11 is a diagram showing an example of the configuration of a monitoring system according to the second embodiment. A monitoring system 7a according to this embodiment is similar to the monitoring system 7 according to the first embodiment, except that it includes a video processing device 1a instead of the video processing device 1. The video processing device 1a is similar to the video processing device 1 according to the first embodiment, except that a delay processing unit 11 is added. Components having the same functions as those in the first embodiment are assigned the same reference numerals as those in the first embodiment, and redundant explanations will be omitted. Below, differences from the first embodiment will be mainly explained.

In this embodiment, as in embodiment 1, camera images and location information images are superimposed, but there is generally a time lag between the camera images and the location information images. In particular, since the processing in the location information generating device 6 calculates the position using the reception strength of multiple signal receiving devices 4, the position detection process takes time, and processing delays may occur.

In this embodiment, the delay processing unit 11 delays the video data output from the video data processing unit 13 and outputs it to the video superimposition unit 17. In other words, the delay processing unit 11 delays the video data output from the video data processing unit 13 by a predetermined delay time and outputs it to the video superimposition unit 17.

For example, during an adjustment period, such as when the monitoring system 7a starts operating, the delay time is changed while generating superimposed images, and the adjustment worker checks whether there is any positional misalignment between the camera image and the position information image to determine the appropriate delay time, which is then input into the image processing device 1a using input means (not shown).

Alternatively, the number of signal receiving devices 4 may be changed, and appropriate delay time values may be determined for each number of signal receiving devices 4 during the adjustment period described above. These results may then be used to obtain information for calculating the delay time. For example, a table correlating delay time with the number of signal receiving devices 4 may be generated based on the results described above, and the generated table may be stored in the delay processing unit 11. The delay processing unit 11 may then determine the delay time corresponding to the number of signal receiving devices 4 based on the stored table. Alternatively, a formula for calculating the delay time from the number of signal receiving devices 4 may be determined using the results of the adjustment period described above, and the delay processing unit 11 may then use the number of signal receiving devices 4 to calculate the delay time according to the formula to determine the delay time. The number of signal receiving devices 4 may be input to the video processing device 1a, for example, using input means not shown.

Alternatively, a table or formula for calculating the delay time using the number of signal receiving devices 4 and the number of signal transmitting devices 5 for which the position information generating device 6 has generated position information may be generated in the same manner as in the above example, and the delay processing unit 11 may calculate the delay time according to the number of signal receiving devices 4 and the number of signal transmitting devices 5 for which the position information generating device 6 has generated position information. For example, a delay time according to the result of multiplying the number of signal receiving devices 4 by the number of signal transmitting devices 5 for which the position information generating device 6 has generated position information may be determined using a table or formula. The number of signal transmitting devices 5 for which the position information generating device 6 has generated position information may be obtained by the video processing device 1a from the signal transmitting devices 5, or an average value may be adjusted based on the work plan and input into the video processing device 1a using input means (not shown).

Alternatively, a trained model may be generated by machine learning using a training dataset in which the number of signal receiving devices 4 and the number of signal transmitting devices 5 for which the position information generating device 6 has generated position information are used as input data, and the corresponding delay time obtained during the adjustment period is used as correct answer data. In this case, the delay processing unit 11 can calculate the delay time by inputting the number of signal receiving devices 4 and the number of signal transmitting devices 5 for which the position information generating device 6 has generated position information into the trained model.

Furthermore, after the monitoring system 7a begins operation, if a monitor checks the monitoring video and determines that there is a misalignment between the location information video and the camera video, the monitor can use an input means (not shown) to instruct the video processing device 1a to readjust, and the delay processing unit 11 of the video processing device 1a may change the delay time by a certain amount when the readjustment instruction is received. The amount by which the delay is to be changed may also be input by the monitor.

Other than as described above, the operation of this embodiment is the same as that of embodiment 1. Like the video processing device 1 of embodiment 1, the video processing device 1a of this embodiment is also realized by a computer system.

As described above, in this embodiment, the video processing device 1a applies a delay to the camera image in order to correct for the positional shift of a person between the two images due to a delay in the positional information image relative to the camera image. As a result, this embodiment not only achieves the same effects as in embodiment 1, but also reduces the positional shift between the positional information image and the camera image, improving the accuracy of identifying people.

Embodiment 3.
12 is a diagram showing an example of the configuration of a monitoring system according to the third embodiment. A monitoring system 7b according to this embodiment is similar to the monitoring system 7 according to the first embodiment, except that it includes a video processing device 1b instead of the video processing device 1. The video processing device 1b is similar to the video processing device 1 according to the first embodiment, except that a synchronization processing unit 23 is added. Components having the same functions as those in the first embodiment are assigned the same reference numerals as those in the first embodiment, and redundant explanations will be omitted. Below, differences from the first embodiment will be mainly explained.

As described in the second embodiment, a time lag typically occurs between the camera image and the location information image due to processing delays and the like. In this embodiment, a first timestamp indicating the time of image capture by the image acquisition device 3 is added to the image data, and a second timestamp indicating the time the signal receiving device 4 received the wireless signal from the signal transmitting device 5 is added to the location information. The synchronization processing unit 23 synchronizes the camera image and the location information image using these first and second timestamps. That is, the synchronization processing unit 23 synchronizes the camera image and the location information image based on the time the sensors, the image acquisition device 3 and the signal receiving device 4, captured the physical phenomenon that forms the basis of the camera image and the location information image, i.e., the time at which light or radio waves were detected, and outputs the synchronized camera image and location information image to the image superimposition unit 17. The signal receiving device 4 adds a timestamp indicating the time the wireless signal was received from the signal transmitting device 5 to the received information and transmits it to the location information generating device 6. The location information generating device 6 then adds this timestamp to the location information as a second stamp and transmits it to the image processing device 1b.

In detail, a first timestamp is added to the video data output from the video data processing unit 13, and a second timestamp is added to the location information video output from the video generation unit 16. The synchronization processing unit 23 temporarily holds multiple frames of camera video and location information video, and outputs, for example, the set of video data and location information video with the smallest difference between the first and second timestamps to the video superimposition unit 17. Note that because the time from when the signal receiving device 4 receives a wireless signal to when it transmits it to the location information generating device 6 is considered to be short compared to the processing time, the second timestamp may be information indicating the time when the location information generating device 6 received the reception information from the signal receiving device 4. In other words, the second timestamp is the time when the signal used to calculate the position indicated by the location information was detected.

Other than as described above, the operation of this embodiment is the same as that of embodiment 1. Like the video processing device 1 of embodiment 1, the video processing device 1b of this embodiment is also realized by a computer system.

In this way, in this embodiment, the camera image and the location information image are synchronized using the time when the image acquisition device 3 and the signal receiving device 4, which are sensors, capture the physical phenomena that form the basis of the camera image and the location information image. This achieves the same effects as in embodiment 1, while also reducing positional deviation between the location information image and the camera image, improving the accuracy of identifying people.

Embodiment 4.
13 is a diagram showing an example of the configuration of a monitoring system according to a fourth embodiment. A monitoring system 7c according to this embodiment is similar to the monitoring system 7 according to the first embodiment, except that it includes a video processing device 1c instead of the video processing device 1. The video processing device 1c is similar to the video processing device 1 according to the first embodiment, except that it includes a video data processing device 13a, a video generation unit 16a, and a synchronization processing unit 23a instead of the video data processing unit 13 and the video generation unit 16. Components having the same functions as those according to the first embodiment are assigned the same reference numerals as those according to the first embodiment, and redundant explanations will be omitted. Below, differences from the first embodiment will be mainly explained.

As described in embodiments 2 and 3, a time lag generally occurs between the camera image and the position information image due to processing delays, etc. In this embodiment, the synchronization processing unit 23a compares a bounding box in the camera image with the worker's position in the position information image and synchronizes the camera image and the position information image using the comparison result. That is, the synchronization processing unit 23a associates a bounding box, which is an area corresponding to a person, with the worker's position in the position information image, calculates the amount of time lag between the camera image and the position information image based on the distance between a representative point in the associated bounding box and the worker's position, and uses the amount of lag to synchronize the camera image and the position information image and output them to the image superimposition unit 17. The synchronization processing unit 23a also causes at least one of the image data processing unit 13a and the image generation unit 16a to perform a correction to correct the position lag using the comparison result.

Figure 14 is a diagram for explaining the synchronization processing of this embodiment. In Figure 14, the camera video for each frame is shown on the left, and the position information video for each frame is shown on the right. Figure 14 shows camera video and position information video from frame n (n is an integer greater than or equal to 0) + 0 to frame n + 5. If we assume that the video processing device 1c performs processing on a frame-by-frame basis, i.e., the time for one frame is defined as the frame time, then the camera video and position information video received by the synchronization processing unit 23a within one frame time are video with the same frame number.

As shown in Figure 14, there is a discrepancy between the position of the bounding box 301 in the camera image and the worker position 302 in the position information image for the same frame number. Note that in Figure 14, the rectangle without the reference number 301 in the camera image is also a bounding box, and the black circle without the reference number 302 in the position information image is also the worker position.

The synchronization processing unit 23a associates a worker position with each bounding box. The video data processing unit 13a outputs the representative point of the bounding box as position coordinates to the synchronization processing unit 23a. As in embodiment 1, the representative point of the bounding box may be, for example, the center of the bounding box, the position of the feet, the position of the head, etc., but is not limited to these. For example, for each bounding box, the synchronization processing unit 23a determines the worker position with the smallest distance between the representative point of the bounding box and the worker position in the position information video as the worker position corresponding to that bounding box. The synchronization processing unit 23a then sequentially shifts the frame numbers of the camera video and the position information video to calculate the shift amount of the frame number that minimizes the sum or RMS (Root Mean Square) of the distance between the representative point of the bounding box and the corresponding worker position for all worker positions.

For example, for the camera video of frame n+0, the above sum or RMS is calculated using position information video from frame n+0 to frame n+10, and the frame number of the position information video that minimizes the sum or RMS is calculated. The difference between the calculated frame number and n+0, the frame number of frame n+0, is used as the frame number deviation. The synchronization processing unit 23a may perform similar processing by changing the frame number of the camera video, and use the average of the deviations calculated for multiple frames of the camera video as the final deviation. In the example shown in Figure 14, the frame number deviation is calculated as 1, and the camera video of frame n+0 corresponds to the position information video of frame n+1.

The synchronization processing unit 23a uses the calculated amount of deviation to synchronize the camera video and the position information video and output them to the video superimposition unit 17. That is, the synchronization processing unit 23a delays either the camera video or the position information video by the calculated amount of deviation and outputs them to the video superimposition unit 17. Furthermore, using the synchronized camera video and position information video, if the distance between the bounding box representative point and the corresponding worker position is equal to or greater than a threshold, the synchronization processing unit 23a calculates the difference in coordinate values between the bounding box representative point and the corresponding worker position, and outputs position correction information indicating the difference in coordinate values to the video generation unit 16a. For example, if the horizontal direction in the video is the X axis and the vertical direction is the Y axis, the synchronization processing unit 23a outputs the difference in X and Y coordinate values for each worker position to the video generation unit 16a. Furthermore, the synchronization processing unit 23a may calculate the difference in coordinate values between the bounding box representative point and the corresponding worker position using multiple frames of camera video and the synchronized position information video. For example, the synchronization processing unit 23a may output position correction information indicating the average value of the difference in coordinate values between the representative point of the bounding box and the corresponding worker position over multiple frames to the image generation unit 16a.

The image generation unit 16a uses the position correction information received from the synchronization processing unit 23a to correct the worker's position in the position information image and outputs the corrected position information image to the synchronization processing unit 23a. In other words, the image generation unit 16a cancels out the difference indicated by the position correction information, changes the worker's position so that the worker's position overlaps with the representative point of the bounding box, and outputs the position information image after the worker's position has been changed to the synchronization processing unit 23a.

For example, during an adjustment period, such as the initial operation of the monitoring system 7c, the amount of deviation required to synchronize the camera image and the location information image may be calculated, and the calculated amount of deviation may be used. In this case, if a monitor or other person checks the monitoring image during operation and determines that the amount of deviation is inappropriate, they may use an input means (not shown) to instruct readjustment, and the amount of deviation may be calculated again.

Other than as described above, the operation of this embodiment is the same as that of embodiment 1. Like the video processing device 1 of embodiment 1, the video processing device 1c of this embodiment is also realized by a computer system.

In this way, in this embodiment, the camera image and the position information image are synchronized using a bounding box indicating the position of the person in the camera image and the worker position in the position information image. This achieves the same effects as in embodiment 1, while also reducing positional deviation between the position information image and the camera image, improving the accuracy of identifying people. Furthermore, the image generator 16a corrects the positional deviation of the worker position using the bounding box indicating the position of the person in the camera image and the worker position in the position information image, thereby further reducing positional deviation.

Embodiment 5.
15 is a diagram showing an example of the configuration of a monitoring system according to a fifth embodiment. A monitoring system 7d according to this embodiment is similar to the monitoring system 7 according to the first embodiment, except that it includes a video processing device 1d instead of the video processing device 1. The video processing device 1d is similar to the video processing device 1 according to the first embodiment, except that it includes a video data processing device 13b, a video generation unit 16a, and a synchronization processing unit 23b instead of the video data processing unit 13 and the video generation unit 16. Components having the same functions as those according to the first embodiment are assigned the same reference numerals as those according to the first embodiment, and redundant explanations will be omitted. Differences from the first embodiment will be mainly explained below.

As described in embodiments 2, 3, and 4, a time lag generally occurs between the camera image and the position information image due to processing delays, etc. In this embodiment, the image data processing unit 13b generates trajectory information indicating the trajectory of the person's movement, and the synchronization processing unit 23b determines that a position lag has occurred due to a delay in the position information image if the worker's position is on the trajectory indicated by the trajectory information. In order to align the worker's position with the position in the camera image, the synchronization processing unit 23b outputs the difference between the worker's position and the person's position in the camera image to the image generation unit 16a as position correction information.

In detail, the video data processing unit 13b outputs the trajectory information to the synchronization processing unit 23b along with the camera image. The trajectory information indicates the movement history of an area estimated to be the same person across multiple frames in the video data. A detailed description of the method for determining whether or not the person is the same person will be omitted, as it can use a general method for identifying people in image and video processing. For example, when a new area is extracted as a person, the video data processing unit 13b sequentially identifies areas estimated to be the same person in subsequent frames. For example, the video data processing unit 13b calculates the trajectory information by sequentially recording the coordinate values of the representative point of the bounding box corresponding to the area extracted as a person, for each person. As in embodiment 1, the representative point of the bounding box may be, for example, the center of the bounding box, the position of the feet, the position of the head, etc., but is not limited to these. The trajectory information may be the trajectory since the person was newly extracted, or the trajectory of each person from a certain period of time ago.

In this embodiment, when a worker's position in the position information video is on the trajectory indicated by the trajectory information or within a distance from the trajectory within a threshold value, the synchronization processing unit 23b determines that the worker's actual position corresponding to that worker's position is the position of the corresponding person in the camera video. The synchronization processing unit 23b then generates position correction information indicating the difference between the worker's position and the actual position, and outputs the position correction information to the video generation unit 16a. The video generation unit 16a corrects the worker's position using the position correction information, as in embodiment 4. The synchronization processing unit 23b outputs the corrected position information video and camera video to the video superimposition unit 17.

Figure 16 is a diagram showing an example of trajectory information in this embodiment. The example shown in Figure 16 shows an example of camera footage of two people corresponding to bounding boxes 401 and 402, with trajectories 403 and 404 indicated by trajectory information corresponding to bounding boxes 401 and 402, respectively, superimposed on the camera footage.

Figure 17 is a diagram showing an example of a superimposed image in this embodiment. The example shown in Figure 17 shows an image in which camera image before position correction and position information image are superimposed. The upper diagram in Figure 17 shows the image before position correction, and the lower diagram in Figure 17 shows the image after position correction. In the example shown in Figure 17, position information image is superimposed on the camera image on which the trajectory information shown in Figure 16 is superimposed. Worker positions 405 and 406 shown in Figure 17 are located on trajectories 403 and 404, respectively. Therefore, synchronization processing unit 23b determines that the current position of the person corresponding to worker position 405 is the position of bounding box 401, and the current position of the person corresponding to worker position 406 is the position of bounding box 402. For example, synchronization processing unit 23b calculates the difference between the worker position and the representative point of the corresponding bounding box as position correction information, and outputs the position correction information to image generation unit 16a. As a result, image generation unit 16a corrects the worker position. In Figure 17, the representative points of the bounding boxes are set to the feet, so worker positions 405 and 406 are corrected to the feet of the corresponding bounding boxes 401 and 402. Note that in Figure 17, trajectories 403 and 404 are superimposed on the superimposed image for the sake of explanation, but the superimposed image superimposed by the image superimposition unit 17 may or may not include trajectories 403 and 404.

Other than as described above, the operation of this embodiment is the same as that of embodiment 1. The video processing device 1d of this embodiment is also realized by a computer system, just like the video processing device 1 of embodiment 1.

In this way, in this embodiment, the trajectory of the person in the camera image is used to correct the worker's position in the position information image. This achieves the same effects as in embodiment 1, while also reducing positional discrepancies between the position information image and the camera image, improving the accuracy of identifying the person.

The configurations shown in the above embodiments are merely examples, and may be combined with other known technologies, or different embodiments may be combined with each other. Parts of the configuration may also be omitted or modified without departing from the spirit of the invention.

1, 1a, 1b, 1c, 1d: Image processing device; 2: Image display device; 3: Image acquisition device; 4: Signal receiving device; 5: Signal transmitting device; 6: Position information generating device; 7, 7a, 7b, 7c, 7d: Monitoring system; 8: Monitoring area; 9: Worker; 10: Management device; 11: Delay processing unit; 12: Image data receiving unit; 13, 13a, 13b: Image data processing unit; 14: Position information receiving unit; 15: Coordinate calculation unit; 16, 16a: Image generating unit; 17: Image superimposition unit; 18: Registration information storage unit; 19: Management information receiving unit; 20: Work plan storage unit; 21: Determination unit; 22: Alarm unit; 23, 23a, 23b: Synchronization processing unit.

Claims (14)

a video data receiving unit that receives video data representing a video image of a monitoring area in a power plant;
a location information receiving unit that receives location information including a location of the terminal device in the monitoring area and device identification information that is identification information of the terminal device;
a registration information storage unit that stores, for each piece of device identification information, registration information including device identification information and person identification information that is identification information of a person carrying the terminal device corresponding to the device identification information;
an image generating unit that generates a position information image that shows the position included in the position information as a person position, and acquires the registration information corresponding to the device identification information included in the position information from the registration information storage unit;
a permission plan storage unit that stores a permission plan including information indicating personal identification information of persons who are permitted to enter the monitoring area for each time period;
a determination unit that uses personal identification information included in the registration information acquired by the image generation unit and the permission plan to determine whether or not a person corresponding to the personal identification information is permitted to enter the monitoring area, and identifies the personal identification information of a person who is determined to be prohibited from entering the monitoring area;
an image superimposing unit that superimposes the position information image on an image corresponding to the image data, outputs the superimposed image as a surveillance image, and generates the surveillance image so that the person position corresponding to the person identification information specified by the determination unit is highlighted;
A video processing device comprising: The image processing device of claim 1, characterized in that the determination unit issues an alarm when a person determined to be prohibited from entering the monitored area is present. the image generation unit outputs the registration information corresponding to the device identification information to an image superimposition unit together with the position information image;
3. The video processing device according to claim 1, wherein the video superimposing unit outputs the registration information together with the monitoring video. a delay processing unit that delays the video corresponding to the video data by a predetermined delay time and outputs the delayed video to the video superimposing unit;
4. The video processing device according to claim 1, further comprising: a synchronization processing unit that synchronizes the video corresponding to the video data with the position information video using a first timestamp indicating an image capture time in a video capture device that captures the video corresponding to the video data and a second timestamp indicating a time when a signal used to calculate a position indicated by the position information was detected, and outputs the synchronized video corresponding to the video data and the position information video to the video superimposition unit;
4. The video processing device according to claim 1, further comprising: a video data processing unit that extracts an area corresponding to a person from a video corresponding to the video data;
a coordinate calculation unit that calculates coordinate values of each of the positions included in the position information in an image corresponding to the image data;
a synchronization processing unit that associates the area with the person position in the position information image generated based on the coordinate values calculated by the coordinate calculation unit, calculates a time lag between the image corresponding to the image data and the position information image based on a distance between a representative point in the associated area and the person position, and uses the time lag to synchronize the image corresponding to the image data with the position information image and output the synchronized image to the image superimposition unit;
4. The video processing device according to claim 1, further comprising: the synchronization processing unit generates position correction information indicating a difference in coordinate values between the associated area and the person position after synchronization;
The image processing device according to claim 6 , wherein the image generating unit corrects the person position using the position correction information, and generates the position information image indicating the person position after correction. a video data processing unit that extracts an area corresponding to a person from a video corresponding to the video data and generates trajectory information that indicates a trajectory of the area for each person;
a coordinate calculation unit that calculates coordinate values in an image corresponding to the image data of each of the positions included in the position information;
a synchronization processing unit that, when it is determined that the person position in the position information video generated based on the coordinate values calculated by the coordinate calculation unit is on the trajectory indicated by the trajectory information , generates position correction information that indicates a difference between the person position and a representative point in the area that corresponds to the trajectory information;
Equipped with
4. The image processing device according to claim 1, wherein the image generation unit corrects the person position using the position correction information and generates the position information image showing the person position after correction. 9. The image processing device according to claim 6, wherein the image superimposing unit generates the monitoring image so that, when there is no person position corresponding to the area, the area is highlighted. a video processing device that generates a monitoring video of a monitoring area in a power plant;
a video display device that displays the monitoring video;
Equipped with
The video processing device includes:
a video data receiving unit that receives video data representing a video image of the monitoring area;
a location information receiving unit that receives location information including a location of the terminal device in the monitoring area and device identification information that is identification information of the terminal device;
a registration information storage unit that stores, for each piece of device identification information, registration information including device identification information and person identification information that is identification information of a person carrying the terminal device corresponding to the device identification information;
an image generating unit that generates a position information image that shows the position included in the position information as a person position, and acquires the registration information corresponding to the device identification information included in the position information from the registration information storage unit;
a permission plan storage unit that stores a permission plan including information indicating personal identification information of persons who are permitted to enter the monitoring area for each time period;
a determination unit that uses personal identification information included in the registration information acquired by the image generation unit and the permission plan to determine whether or not a person corresponding to the personal identification information is permitted to enter the monitoring area, and identifies the personal identification information of a person who is determined to be prohibited from entering the monitoring area;
an image superimposing unit that superimposes the position information image and an image corresponding to the image data, outputs the superimposed image to the image display device as a surveillance image, and generates the surveillance image so that the person position corresponding to the person identification information specified by the determination unit is highlighted;
A monitoring system comprising: the image generation unit outputs the registration information corresponding to the device identification information to an image superimposition unit together with the position information image;
the video superimposing unit outputs the registration information together with the monitoring video to the video display device;
11. The monitoring system according to claim 10, wherein the video display device displays the registration information together with the monitoring video. the terminal device that transmits a signal including the device identification information as a wireless signal;
a plurality of signal receiving devices that calculate the reception strength of the wireless signal transmitted from the terminal device and transmit reception information including the reception strength and the corresponding device identification information;
a location information generating device that receives the reception information from the plurality of signal receiving devices and generates the location information using the reception information;
12. The monitoring system according to claim 10, further comprising: a delay processing unit that delays the video corresponding to the video data by a predetermined delay time and outputs the delayed video to the video superimposing unit;
Equipped with
The monitoring system according to claim 12, wherein the delay time is determined based on the number of the signal receiving devices. In the computer system,
receiving video data representing a captured image of a monitored area in a power plant;
receiving location information including a location of a terminal device in the monitoring area and device identification information that is identification information of the terminal device;
generating a position information image that shows the person position indicated by the position information together with corresponding additional information;
storing registration information including the device identification information and personal identification information that is identification information of a person carrying the terminal device corresponding to the device identification information in a registration information storage unit for each of the device identification information;
generating a position information image that shows the position included in the position information as a person position, and acquiring the registration information corresponding to the device identification information included in the position information from the registration information storage unit;
storing a permission plan including information indicating personal identification information of persons permitted to enter the monitoring area for each time period in a permission plan storage unit;
a step of determining whether or not a person corresponding to the personal identification information is permitted to enter the monitoring area using the personal identification information included in the acquired registration information and the permission plan, and identifying the personal identification information of a person who is determined to be prohibited from entering the monitoring area;
a step of superimposing the position information image on an image corresponding to the image data, outputting the superimposed image as a surveillance image, and generating the surveillance image so that the person position corresponding to the specified person identification information is highlighted;
A program characterized by executing the following.