JP7725379B2 - surveillance system - Google Patents

Description

The present invention relates to a surveillance system that uses multiple LiDAR (Light Detection and Ranging/remote sensing devices) and mobile surveillance cameras, and in particular to a surveillance system that allows for easy setting of parameters required for system construction.

[Prior Art]
Conventional surveillance systems required manual registration of parameters for location information from sensors such as LiDAR and surveillance cameras into the system during installation, which required time before the system could begin operation.

Furthermore, while there are systems that link various sensors with surveillance cameras, there are no systems that combine and display images of objects detected by LiDAR or other sensors with images from surveillance cameras.

[Related Technology]
As a related prior art, there is Japanese Patent No. 6823211 entitled "Method for identifying the location of an autonomous vehicle including multiple sensors" (Patent Document 1).
Patent Document 1 discloses a method for determining the position of a ground vehicle using a plurality of sensors for autonomous driving of an autonomous vehicle.

Patent No. 6823211

As such, with conventional surveillance systems, location information from sensors such as LiDAR and surveillance cameras had to be manually registered during installation, which required a significant amount of adjustment work before the system could begin operation, resulting in time and cost involved from installation to the start of system operation.

Furthermore, there is no system that can combine and display images of objects detected by LiDAR or other sensors with images from surveillance cameras, so the objects cannot be overlaid on the surveillance images, which poses the problem of not being able to immediately identify the detected objects.

Note that Patent Document 1 does not describe a configuration for automating the registration of location information from LiDAR and other devices and the location information from surveillance cameras, or for overlaying surveillance images of surveillance objects detected by LiDAR and other devices onto the surveillance footage from the surveillance cameras.

The present invention was made in consideration of the above-mentioned circumstances, and aims to provide a surveillance system that automatically collects position information in the local coordinate system of LiDAR, etc. and the local coordinate system of surveillance cameras, and expands it into a global coordinate system for the entire monitored space, thereby automating the registration of position information for LiDAR, etc. and surveillance cameras during installation work, thereby reducing and shortening the adjustment work required before the system can be put into operation.

The present invention also aims to provide a surveillance system that overlays monitored objects detected by LiDAR or other sensors onto the surveillance footage from a surveillance camera, making it easy to identify the detected objects.

The present invention, which solves the problems of the above-mentioned conventional examples, is a surveillance system in which a plurality of remote sensing devices, a plurality of surveillance cameras, and an integrated management server that performs surveillance-related processing are connected via a wireless network, wherein the remote sensing devices have built-in GPS to acquire location information of the devices, detect the direction and distance of the detected objects to be monitored and the surveillance cameras, and generate local coordinate systems indicating the positions of the detected objects and the surveillance cameras from the direction and distance of the detected objects and the surveillance cameras, with the position of the remote sensing device as the origin, and transmit the location information, the direction and distance of the detected objects and the surveillance cameras, and the local coordinate system to the integrated management server, and the integrated management server generates a global coordinate system by combining the multiple pieces of location information transmitted from the multiple remote sensing devices, and each remote sensing device acquires its own local coordinate system and combines the local coordinate systems so that the positions of the detected objects and the surveillance cameras that overlap in each of the acquired local coordinate systems overlap, and then expands the combined local coordinate system into the global coordinate system.

The present invention is characterized in that in the above-mentioned monitoring system, if the integrated management server attempts to superimpose the positions of overlapping detected objects and monitoring cameras in each acquired local coordinate system but the positions are still misaligned, the integrated management server calculates the average value of the distances from the remote sensing devices detected by the remote sensing devices in each acquired local coordinate system , corrects the amount of positional misalignment using the difference between the average value and the detected distance, and expands the synthesized local coordinate system into a global coordinate system.

The present invention is characterized in that in the above-mentioned monitoring system, the integrated management server periodically repeats the process of expanding the synthesized local coordinate system into a global coordinate system.

The present invention is characterized in that in the above-mentioned surveillance system, the integrated management server controls the surveillance camera to point toward the position coordinates of the detected object shown in the global coordinate system.

The present invention is characterized in that in the above-mentioned surveillance system, a terminal device that displays images from the surveillance camera is connected to the wireless network, and the integrated management server forms a rectangle in a global coordinate system that includes all of the point cloud data of the detected object detected by the remote sensing device, and when the surveillance camera is pointed at the center coordinates of the rectangle, the terminal device overlays and displays a rectangle corresponding to the detected object on the image.

In the above, the present invention is characterized in that when an object is detected by a remote sensing device, the integrated management server issues a warning to a terminal device, and when a correct or false report is input from the terminal device in response to the report, it accumulates data from the remote sensing device in the case of a correct report, performs machine learning using the data, and uses the results of the machine learning to detect the object .

According to the present invention, there is provided a surveillance system in which a plurality of remote sensing devices, a plurality of surveillance cameras, and an integrated management server that performs surveillance-related processing are connected via a wireless network, wherein the remote sensing devices have built-in GPS to acquire location information of the devices, detect the direction and distance of the detected objects and the surveillance cameras to be monitored, and generate local coordinate systems indicating the positions of the detected objects and the surveillance cameras from the direction and distance of the detected objects and the surveillance cameras, with the position of the remote sensing device as the origin, and transmit the location information, the direction and distance of the detected objects and the surveillance cameras, and the local coordinate system to the integrated management server, and the integrated management server synthesizes the plurality of pieces of location information transmitted from the plurality of remote sensing devices to generate a global coordinate system, and each remote sensing device acquires its own local coordinate system and synthesizes the local coordinate systems so that the positions of the detected objects and surveillance cameras that are overlapping in each of the acquired local coordinate systems overlap, and the synthesized local coordinate system is expanded into the global coordinate system, thereby providing the effect of identifying the positions of the plurality of surveillance cameras and automating the registration of the surveillance camera's location information.

FIG. 1 is a schematic diagram of the system. FIG. 2 is a flow chart showing the outline of the processing of the present system. FIG. 1 is an image diagram of a global coordinate system. FIG. 1 is an image diagram of a local coordinate system. This is an image of the global coordinate system when a suspicious person is detected. FIG. 10 is an image of a detection image when a suspicious person is detected. FIG. 10 is a process image diagram for overlaying a detection frame on an image. FIG. 2 is an image diagram of the XY plane of the global coordinate system. FIG. 2 is an image diagram of the XZ plane of the global coordinate system.

An embodiment of the present invention will be described with reference to the drawings.
[Outline of the embodiment]
A surveillance system (this system) according to an embodiment of the present invention generates a global coordinate system from latitude/longitude information from multiple remote sensing devices (LiDAR), synthesizes the local coordinate systems generated by each LiDAR so that detected objects, including the detected surveillance cameras, overlap, and then expands the synthesized local coordinate system into a generated global coordinate system, identifies the positions of the multiple surveillance cameras based on the global coordinate system, and controls the surveillance cameras.This system can automate the registration of surveillance camera position information and also automate the control of the surveillance cameras in response to detected objects.

In addition, this system forms a rectangle containing point cloud data of the object detected by LiDAR in a global coordinate system, and when the surveillance camera is pointed at the center coordinates of the rectangle, a rectangle corresponding to the detected object is overlaid on the image displayed on the terminal device that displays the surveillance camera image, making it easier to identify the detected object.

[This system: Figure 1]
The configuration of this system will be described with reference to Fig. 1. Fig. 1 is a schematic diagram of the system.
As shown in Figure 1, this system basically comprises multiple remote sensing devices (LiDAR) 1 that serve as sensors, multiple mobile surveillance cameras (monitoring cameras) 2, a wireless network 3, an integrated management server 4, and an operation terminal 5.

The LiDAR 1 and the surveillance camera 2 are wirelessly connected to a wireless network 3.
The integrated management server 4 and the operation terminal 5 are connected to the wireless network 3 via a wired connection.

Here, the prerequisites for this system will be explained.
The LiDAR 1 is designed to be installed horizontally relative to the ground in the monitored space.
The ground is assumed to be level and without slope.
In addition, the individual coordinate system formed by sensing by LiDAR 1 is referred to as the "local coordinate system," and the coordinate system integrated by the integrated management server 4 as the entire monitored space is referred to as the "global coordinate system."

[Parts of this system]
Next, each part of this system will be specifically described.
[LiDAR1]
The LiDAR 1 is a remote sensing device equipped with a laser emitter and an optical sensor. The optical sensor receives the reflected light of the laser light emitted (emitted) from the laser emitter to detect the direction and distance of the monitored object (detected object) and transmits this information to the integrated management server 4. The integrated management server 4 then acquires movement trajectory information, movement speed information, and point cloud density information based on the direction and distance of the detected monitored object, as well as time information. The direction and distance of the monitored object are used to generate a local coordinate system for each LiDAR 1.

In addition, the LiDARs 1 are arranged so that their detection ranges overlap each other.
In addition, the LiDAR 1 has a built-in GPS (Global Positioning System) function and transmits location information (latitude/longitude information) of the latitude/longitude of the installation location to other LiDARs 1 and the integrated management server 4 via the wireless network 3.
The integrated management server 4 will generate a global coordinate system based on the latitude/longitude information of all the LiDARs 1 collected.

Furthermore, LiDAR1 calculates the position information (local position information) of adjacent LiDAR1 in local coordinates from the direction of the laser light emitted from adjacent LiDAR1 and the latitude/longitude information of adjacent LiDAR1. The specific calculation method will be described later.

[Surveillance camera 2]
The surveillance camera 2 is a movable surveillance camera that receives pan/tilt/zoom control values from the integrated management server 4 and directs itself toward a detected object according to these control values. The control of directionality toward a detected object will be described later.

[Wireless Network 3]
The wireless network 3 is a communication network that realizes wireless communication such as the fifth generation (5G).
The LiDAR 1 and the surveillance camera 2 are wirelessly connected to the wireless network 3, and the integrated management server 4 and the operation terminal 5 are wired connected to the wireless network 3.

[Integrated Management Server 4]
The integrated management server 4 of this system is basically configured to perform global coordinate system generation processing, local coordinate system integration processing, global coordinate system expansion (reflection/conversion) processing, detection processing, alarm processing, learning processing, etc.
These processes may be distributed among a plurality of processing devices.

The integrated management server 4 comprises a control unit, a storage unit, and an interface unit, and the processing programs stored in the storage unit run on the control unit to realize the functions of each of the above processes.
The storage unit also stores data on the local coordinate system and the global coordinate system, as well as data related to detection and data for controlling the surveillance camera 2.
The interface unit is used to connect the integrated management server 4 to the wireless network 3 .

[Global coordinate system generation process]
The integrated management server 4 generates a global coordinate system for the entire monitored space from the latitude/longitude information of all the LiDARs 1. The specific generation of the global coordinate system will be described later.

[Local coordinate system integration processing]
The integrated management server 4 also acquires the coordinates of adjacent LiDARs 1, obstacles, and the surveillance camera 2 obtained by sensing from each LiDAR 1, acquires a local coordinate system in which these coordinates are plotted, and further synthesizes the local coordinate system. The specific synthesis of the local coordinate system will be described later.

[Global coordinate system expansion processing]
The integrated management server 4 then expands (reflects) the combined local coordinate system into a global coordinate system. Since the position coordinates of the surveillance camera 2 are specified in the global coordinate system, the position information of the surveillance camera 2 can be automatically registered in the integrated management server 4. The specific generation and expansion of the global coordinate system will be described later.

[Detection process]
In addition, when a monitored object is detected by LiDAR 1, the integrated management server 4 receives information on the direction and distance of the detected object from LiDAR 1, identifies the coordinates (position) in the local coordinate system, and then expands it into the global coordinate system to identify the coordinates (position) in the global coordinate system, and controls the surveillance camera 2 to point in the direction of those coordinates.

[Alarm processing]
Furthermore, the integrated management server 4 displays the image captured by the surveillance camera 2 whose directionality has been controlled on the display unit of the operation terminal 5, and issues an alert to draw attention.
Then, the monitor checks the content of the alert on the operation terminal 5 and inputs "true" or "false", and the input result is sent to the integrated management server 4.

[Learning process]
Furthermore, the integrated management server 4 stores the sensing data of the LiDAR 1 in the "true report" inputted to the operation terminal 5 in a memory unit, and uses the data as training data to train an AI (artificial intelligence) on the machine learning, and performs detection processing using the trained AI model. This makes it possible to improve the accuracy of detection as a "true report."

[Operation terminal 5]
The operation terminal 5 is a terminal device that displays the image captured by the pointing-controlled surveillance camera 2 on the display unit under control of the integrated management server 4, issues an alarm from the speaker, and, when the monitor inputs a "true report" or "false report" from the input unit, transmits the information of "true report" or "false report" to the integrated management server 4.

[Processing overview of this system: Figure 2]
Next, an outline of the processing in this system will be described with reference to Fig. 2. Fig. 2 is a flow chart showing the outline of the processing in this system.
As shown in FIG. 2, the integrated management server 4 of this system generates a global coordinate system from the latitude/longitude information of the LiDAR 1 connected to the wireless network 3 (S1).

Next, the integrated management server 4 acquires each local coordinate system from the measurement results obtained by sensing each LiDAR 1 (S2).
Furthermore, the integrated management server 4 combines the respective local coordinate systems (S3), and corrects errors during this combination.
Then, the synthesized local coordinate system is expanded (reflected/transformed) into the generated global coordinate system (S4).

Furthermore, when a new object is detected by the LiDAR 1, the integrated management server 4 performs a detection process (S5).
Specifically, when LiDAR1 detects a suspicious event (suspicious person or object, etc.), it transmits the detected coordinates of the suspicious event in the local coordinate system to the integrated management server 4, and the integrated management server 4 receives the detected coordinates in the local coordinate system.

Next, the integrated management server 4 controls the monitoring cameras 2 in response to suspicious events (S6).
Specifically, the integrated management server 4 calculates the distance and direction from the position coordinates of the surveillance camera 2 in the global coordinate system and the detection coordinates in the local coordinate system, and controls the surveillance camera 2 to point towards the suspicious event.

The integrated management server 4 then issues a warning to the operation terminal 5, and causes the operation terminal 5 to display and output an image captured by the surveillance camera 2 whose direction is controlled (S7).
Here, on the operation terminal 5, the monitor registers the alert event as a "true report" if it is a suspicious event, or as a "false report" if it is a plant or tree, etc., and this information is sent to the integrated management server 4.

Furthermore, the integrated management server 4 links the movement trajectory information, movement speed information, and point cloud density information for each alert and stores them in the memory unit, and in the case of a ``true alert,'' stores this information in the memory unit as training data for machine learning (S8).
This training data will then be subjected to machine learning using AI to detect only suspicious events, reduce false alarms, and improve detection accuracy.
When the above process is completed, the process returns to step S2 and is repeated periodically.

[Global coordinate system: Figure 3]
Next, the global coordinate system generated by the integrated management server 4 will be described with reference to Fig. 3. Fig. 3 is an image diagram of the global coordinate system.
The integrated management server 4 calculates the average latitude/longitude of all LiDARs 1 connected to the wireless network 3 from the latitude/longitude information transmitted from the LiDARs 1, and generates a global coordinate system with the coordinate point of that average value as the origin (center point).

In Figure 3, the center point of the global coordinate system is determined from the latitude/longitude information of LiDAR 1a-1d, a mesh of the global coordinate system is formed, and the positions of LiDAR 1a-1d are set on this mesh. The size of the mesh is arbitrary, and although the position of surveillance camera 2 is shown in Figure 3, this is displayed after synthesis of the local coordinate system.

[Local coordinate system composition]
If the laser irradiation range of LiDAR1 is 180° in front, the direction of LiDAR1's local coordinate system is recognized from the laser irradiation direction of adjacent LiDAR1s detected by LiDAR1, and the local coordinate systems are synthesized so that the position coordinates of the overlapping detected LiDAR1s and obstacles overlap, and then expanded (converted) into the generated global coordinate system. At this time, the surveillance camera 2 is also detected in the same way as the obstacle, and its position information is also set in the local coordinate system and then further set in the global coordinate system. In other words, the position information of the surveillance camera 2 is automatically acquired.
The synthesis of the local coordinate systems will be explained with reference to FIG.

[Local coordinate system: Figure 4]
Next, the local coordinate system formed by each LiDAR 1 will be described with reference to Fig. 4. Fig. 4 is an image diagram of the local coordinate system.
FIG. 4(a) shows the local coordinate system resulting from sensing by LiDAR 1a, and FIG. 4(b) shows the local coordinate system resulting from sensing by LiDAR 1b.

Comparing Figures 4(a) and (b), it can be seen that the position of the obstacle 6 and the position of the other LiDAR 1 that sensed it are misaligned (do not match). This is due to a detection error occurring in each LiDAR 1. Therefore, the integrated management server 4 corrects the local coordinate system and combines them.

[Error correction]
Specifically, the average value of the distance from the LiDAR 1 to the obstacles 6 detected simultaneously in each local coordinate system (two local coordinate systems in Figure 4) is calculated.
The difference between this average value and the detected distance is used as the deviation (error) between the two coordinate systems, and when the local coordinate systems are combined to generate a global coordinate system, the deviation is corrected and combined.

The integrated management server 4 registers a global coordinate system including the LiDAR 1, the surveillance camera 2, and the obstacles 6 as background data.
The integrated management server 4 periodically performs background subtraction processing on the background data and issues a warning when it detects a change. If the position coordinates of a fixed object such as an obstacle 6 change over time, it considers that the orientation of the LiDAR 1 has shifted due to vibration or aging, causing a change in the laser irradiation angle, and issues an alarm.

[Global coordinate system for suspicious person detection: Figure 5]
FIG. 5 is an image diagram of the global coordinate system when a suspicious person is detected.
5 is depicted on a plane, but in reality it is a three-dimensional coordinate system, so the suspicious person detection coordinates 7 have a Z coordinate (height information). The integrated management server 4 calculates the center coordinates of a rectangle that encompasses all of the point cloud data detected by the LiDAR 1, and controls the surveillance camera 2 to point it toward the center coordinates.

One technique for pointing the surveillance camera 2 toward a suspicious person involves multiple sensors detecting the suspicious person, a control device pre-registering multiple preset positions, predicting the suspicious person's movements based on the position coordinates output from the multiple sensors, and calculating the direction of the suspicious person's face from the direction of movement of the suspicious person, and tracking the suspicious person by changing the viewing angle of the surveillance camera relative to the position of the suspicious person detected by the sensors (see Patent Publication No. 2009-273006, "Monitoring Device and Monitoring System").
Other techniques than those mentioned above are described in Japanese Patent No. 5161045, "Monitoring System and Monitoring Method."

[Detection image of suspicious person: Figure 6]
6 is a conceptual diagram of a detection image when a suspicious person is detected. Here, a detection image 22 of the suspicious person is set as a rectangle within a detection image 21 detected by the monitoring camera 2.
The range displayed as an image within a plane perpendicular to the ground to which the center coordinate belongs is the angle of view of the movable camera relative to the distance to the center coordinate.

The minimum point of this angle of view is the coordinate point at the bottom left of the angle of view, and the maximum point is the coordinate point at the top right of the angle of view.
The integrated management server 4 calculates the size and position of the rectangle on the entire screen from the bottom left and top right coordinate points of the rectangle and the bottom left and top right coordinate points of the field of view, and notifies the operation terminal 5 of this information.
The operation terminal 5 overlays the rectangle on the video based on the position information of the rectangle.

[Overlay display: Figures 7 to 9]
Next, the overlay display will be described with reference to Figures 7 to 9. Figure 7 is a processing image diagram for overlaying a detection frame on an image, Figure 8 is an image diagram of the XY plane of the global coordinate system, and Figure 9 is an image diagram of the XZ plane of the global coordinate system.
To simplify the explanation, the position of the surveillance camera 2 is set to the origin of the global coordinate system, and the Z coordinate of the center of the detection frame is set to "0" (zero). The surveillance camera 2 automatically orients itself according to the position (distance and direction) of the object to be detected (suspicious person), so the object to be detected (suspicious person) is captured at the center of the detection screen.

7, a suspicious person and a detection frame 22 are shown in a detection image 21 detected by the surveillance camera 2. Also shown are a point 211 at the bottom left of the detection image 21, a point 212 at the top right of the detection image 21, a point 221 at the bottom left of the detection frame 22, a point 222 at the top right of the detection frame 22, a point 213 at the left end of the horizontal angle of view, a point 214 at the right end of the horizontal angle of view, a point 215 at the bottom end of the vertical angle of view, and a point 216 at the top end of the vertical angle of view.
Furthermore, in the XY plane of the global coordinate system, Figure 8 shows the horizontal angle of view (horizontal angle: θ _H ) 23 of the surveillance camera 2, and in the XZ plane of the global coordinate system, Figure 9 shows the vertical angle of view (horizontal angle: θ _V ) 24 of the surveillance camera 2.

If the X coordinate of the suspicious person is L, the coordinates (X, Y, Z) of each of the points 213 to 216 are as follows:
Coordinates of 213 (L, -Ltan(θ _H /2), 0)
Coordinates of 214 (L, Ltan(θ _H /2), 0)
Coordinates of 215 (L, 0, -Ltan(θ _V /2))
Coordinates of 216 (L, 0, Ltan(θ _V /2))

From the above, the coordinates of 211 and 212 are as follows:
Coordinates of 211 (L, -Ltan(θ _H /2), -Ltan(θ _V /2))
Coordinates of 212 (L, Ltan(θ _H /2), Ltan(θ _V /2))

From the above, the coordinates corresponding to the points at each edge of the detected image can be determined.
Next, the coordinates corresponding to points 221 and 222 in the detection frame 22 are obtained.
The point cloud coordinates of the suspicious person detected by LiDAR1 are ( _X1 , _Y1 , _Z1 ), ( _X2 , _Y2 , _Z2 ), ( _X3 , _Y3 , _Z3 ), ... ( _Xn , _Yn , _Zn ), and the maximum absolute values of X, Y, and Z in the point cloud are _Xmax , _Ymax , and _Zmax .

If the detection frame 22 is a rectangle on the YZ plane (X is fixed) and is a rectangle 20% larger than the suspicious person, the coordinates (X, Y, Z) of 221 and 222 are as follows:
Coordinates of 221 (L, -1.2Y _max , -1.2Z _max )
Coordinates of 222 (L, 1.2Y _max , 1.2Z _max )

In the above explanation, the detection frame is provided with a 20% margin for the detection target, but the amount of margin is determined depending on the system based on the detection and capture capabilities of LiDAR1 and the movement speed of the detection target.
From the above, the coordinate values of each point in the detection image 21 are obtained, and a detection frame 22 encompassing the object to be detected (suspicious person) can be overlaid and displayed in the image according to the coordinate values.

According to this system, when installing the LiDAR 1 and surveillance camera 2, there is no need to manually perform the settings and adjustments required for detection, which shortens the construction period and reduces construction costs.
In addition, by performing machine learning during operation, it is possible to improve detection accuracy over the operation period.
Furthermore, by overlaying the object detected by LiDAR 1 onto the surveillance image on the display unit of the operation terminal 5, it becomes possible to immediately grasp the detected object when an alarm is issued.

[Effects of the embodiment]
According to this system, the integrated management server 4 generates a global coordinate system from the latitude/longitude information of multiple LiDARs 1, synthesizes the local coordinate systems generated by each LiDAR 1 so that detected objects including the detected surveillance cameras 2 overlap, and then expands the synthesized local coordinate system into the generated global coordinate system, identifies the positions of the multiple surveillance cameras 2 based on the global coordinate system, and controls the orientation of the surveillance cameras 2, thereby automating the registration of surveillance camera position information and automating the control of the surveillance cameras in response to detected objects.

This invention automatically collects location information from local coordinate systems such as LiDAR and surveillance cameras, and expands it into a global coordinate system for the entire monitored space. This automates the registration of location information for LiDAR and surveillance cameras during installation work, making it ideal for surveillance systems that can reduce and shorten the adjustment work required before the system is operational.

1 (1a to 1d)... LiDAR, 2 (2A, 2B)... Surveillance camera, 3... Wireless network, 4... Integrated management server, 5... Operation terminal, 6... Obstacle, 7... Detected image, 21... Detected image, 22... Detection frame, 23... Horizontal angle of view (horizontal angle: θ _H ), 24... Vertical angle of view (horizontal angle: θ _V ), 211... Bottom left point of detected image, 212... Top right point of detected image, 213... Left end point of horizontal angle of view, 214... Right end point of horizontal angle of view, 215... Bottom end point of vertical angle of view, 216... Top end point of vertical angle of view, 221... Bottom left point of detection frame, 222... Top right point of detection frame

Claims (6)

A surveillance system in which a plurality of remote sensing devices, a plurality of surveillance cameras, and an integrated management server that performs surveillance-related processing are connected via a wireless network,
The remote sensing device has a built-in GPS to acquire location information of the device, detects the direction and distance of the detected object to be monitored and the surveillance camera, generates a local coordinate system indicating the positions of the detected object and the surveillance camera, with the position of the remote sensing device as the origin, from the direction and distance of the detected object and the surveillance camera, and transmits the location information, the direction and distance of the detected object and the surveillance camera, and the local coordinate system to the integrated management server,
The integrated management server generates a global coordinate system by synthesizing multiple pieces of location information transmitted from the multiple remote sensing devices, each of the remote sensing devices acquires its own local coordinate system, synthesizes the local coordinate systems so that the positions of the detected objects and the surveillance cameras that are detected in the acquired local coordinate systems overlap, and expands the synthesized local coordinate system into the global coordinate system. The monitoring system described in claim 1, characterized in that if the positions of the overlapping detected objects and the surveillance cameras are misaligned even when the positions are superimposed on each acquired local coordinate system, the integrated management server calculates an average value of the distances from the remote sensing devices detected by the remote sensing devices in each acquired local coordinate system , corrects the amount of positional misalignment using the difference between the average value and the detected distance, and expands the synthesized local coordinate system into a global coordinate system. A monitoring system as described in claim 1 or 2, characterized in that the integrated management server periodically repeats the process of expanding the synthesized local coordinate system into a global coordinate system. A surveillance system according to any one of claims 1 to 3, characterized in that the integrated management server controls the surveillance camera to point toward the position coordinates of the detected object indicated in the global coordinate system. A terminal device that displays images from the surveillance camera is connected to the wireless network.
A monitoring system as described in any one of claims 1 to 4 , characterized in that the integrated management server forms a rectangle in a global coordinate system that includes all point cloud data of detected objects detected by a remote sensing device, and when a monitoring camera is pointed at the center coordinates of the rectangle, causes the terminal device to overlay the rectangle corresponding to the detected object on the image. The monitoring system described in claim 5 , characterized in that when an object is detected by a remote sensing device, the integrated management server issues a warning to the terminal device , and when a correct or false report is input from the terminal device in response to the report, the integrated management server accumulates data from the remote sensing device in the case of the correct report , performs machine learning using the data , and uses the results of the machine learning to detect the object.