JP7620524B2 - Surveillance system and method - Google Patents

Description

The present invention relates to a surveillance system that uses cameras mounted on each car of a train.

In railway vehicle operations, one-man operation by a single driver is adopted for routes with few passengers and short train configurations. As the number of railway employees is also on the decline due to demographic trends resulting from a decline in the working population, each railway operator is adopting a policy of increasing the number of routes and train configurations on which one-man operation is applied. However, increasing the number of train configurations increases the burden on a single driver to perform safety checks, and there are concerns that accidents will become more likely to occur due to oversight.

Conventionally, to support one-man operation, systems have been used that use cameras and monitors or mirrors installed on the platforms of each station to check the status of passengers getting on and off. There are also on-board camera systems in which cameras and monitors are installed on the train itself, and these have become more widespread in recent years as they do not require maintenance of infrastructure equipment. For example, Patent Document 1 discloses a monitoring system that allows the status of a door in which an abnormality has been detected in closing to be confirmed on video on an in-car monitor.

JP 2019-92136 A

In recent years, surveillance systems have been developed that use cameras attached to each carriage of a train to capture images of the area around the train (mainly near the doors) and display the camera images on a display device in the driver's cab, allowing the driver to check the images of passengers getting on and off the train. However, because the driver must visually check the camera images, there is a possibility that someone exhibiting abnormal behavior may be overlooked. For example, it is not easy to spot a person moving toward a carriage while the train is moving (including when it starts moving) just by looking at the camera images. In particular, the more carriages in a train's formation, the greater the driver's monitoring burden becomes, making it more difficult to spot such a person.

The present invention was made in consideration of the above-mentioned conventional circumstances, and aims to enable the driver to easily recognize an object moving in the direction of the train while it is moving.

In order to achieve the above object, a monitoring system according to one aspect of the present invention is configured as follows.
In other words, the surveillance system of the present invention comprises a camera attached to each car of a train and an analysis device that analyzes the camera's image, and the analysis device is configured to detect abnormal behavior, which is an object moving toward the train in the camera's image taken while the train is moving, and, upon detection of an abnormal behavior, output abnormality detection information indicating the detection of an abnormal behavior.

In the monitoring system according to the present invention, the analysis device can be configured to detect, as an abnormally behaving object, an object moving in the direction of the train and whose distance from the train is less than a threshold value.

In addition, in the monitoring system according to the present invention, the analysis device can be configured to use multiple thresholds, with the threshold values set to be larger as the train speed increases.

The surveillance system according to the present invention may further include a display device that displays the image from the camera, and the display device may be configured to display information indicating the abnormal behavioral entity by superimposing it on the image from the camera when the abnormal behavioral entity is detected by the analysis device.

In addition, in the monitoring system according to the present invention, the display device stops displaying the camera image when the train starts moving, but if the analysis device detects an abnormally behaving subject, the display device can be configured to cancel the stopping of the camera image display.

In addition, in the surveillance system according to the present invention, the cameras can be mounted on the side of each car of the train, facing in the direction of train travel or in the opposite direction.

In addition, in the monitoring system according to the present invention, the camera may include a camera mounted on the front side of the leading car in the direction of train travel, capturing images of the area ahead in the direction of train travel.

Moreover, a monitoring method according to another aspect of the present invention is configured as follows.
In other words, the monitoring method of the present invention is a monitoring method that uses cameras attached to each car of a train and an analysis device that analyzes the camera footage, and is configured to detect abnormal behavior objects, which are objects moving toward the train in the camera footage taken while the train is moving, and to output abnormality detection information indicating the detection of an abnormal behavior object in response to the detection of an abnormal behavior object.

According to the present invention, when an object is moving in the direction of a moving train, the driver can easily recognize the object.

1 is a diagram illustrating an example of the configuration of a monitoring system according to an embodiment of the present invention. FIG. 13 is a diagram showing an example of a camera image of a person behaving normally; FIG. 13 is a diagram showing another example of a camera image capturing a person behaving normally. FIG. 13 is a diagram showing an example of a camera image capturing a person behaving abnormally; FIG. 13 is a diagram showing an example of a flow relating to detection of an abnormal behavior entity. 13 is a diagram showing an example of a camera image displayed when an abnormally behaving object is detected; FIG. FIG. 11 is a diagram illustrating an example of the configuration of a monitoring system according to another embodiment of the present invention.

An embodiment of the present invention will be described with reference to the drawings.
Fig. 1 shows an example of the configuration of a monitoring system according to an embodiment of the present invention. Here, a train consisting of four cars is taken as an example for explanation, but the number of cars in a train can be any number. In Fig. 1, each of cars 10-1 to 10-4 of the train has a camera 11(A) facing the train's travel direction and a camera 11(B) facing the opposite direction on both sides of the car body, and also has a relay device (e.g., a switching hub) 22 for constructing an in-train network. The camera 11(A) is installed on the rear side of the car, and the camera 11(B) is installed on the front side of the car.

Two display devices 13(A), 13(B) are installed in the driver's cabs of cars 10-1, 10-4 at both ends of the train. Camera images from each car are displayed on the display devices 13 while the train is stopped at the station. This allows the train driver to check images of passengers getting on and off the train on the spot. In this example, it is assumed that only the display device 13 in the front car in the direction of train travel (e.g., car 10-1) is in operation, but the display device 13 in the last car 10-4 may also be in operation.

In addition, the driver's cabs in the cars 10-1 and 10-4 at both ends of the train are also equipped with a control device 14 that controls the display on/off of the display device 13 based on car information 15 obtained from the car side or a higher-level system. One example of the car information 15 is speed information indicating the train's running speed. The control device 14 controls the display device 13 so that the display is on while the train is stopped at the station, and the display is turned off when the train starts running. This allows the driver to concentrate on driving the train, since the camera image is not displayed on the display device 13 while the train is running. In this example, the display device 13 continues to display not only when the train is completely stopped, but also when the train is traveling at a speed that is not considered to be running. Then, when the train's speed reaches a predetermined value (for example, 5 km per hour), the display device 13 stops displaying.

Furthermore, in the monitoring system of this example, each of the vehicles 10-1 to 10-4 is equipped with an analysis device 16 that analyzes the camera footage of that vehicle. The analysis device 16 analyzes the camera footage to detect abnormal behavior, which is an object moving toward the train in the footage captured by the camera 11 while the train is running (including when the train starts running). An example of an abnormal behavior object is typically a person moving toward a moving train, but it can also include objects other than people (e.g., a suitcase). When an abnormal behavior object is detected, the analysis device 16 notifies the driver by outputting abnormality detection information indicating the detection of an abnormal behavior object (details will be described later).

The process of detecting abnormal behavior by the analysis device 16 will now be described. The camera footage is made up of multiple still images (hereafter referred to as "frames") per second, and the analysis device 16 detects abnormal behavior by comparing previous and subsequent frames. The analysis device 16 detects abnormal behavior by noting that the vector of movement of an abnormal behavior object is different from that of a normal person on a station platform. Here, the analysis device 16 compares previous and subsequent frames and determines that an object moving laterally (particularly in the direction toward the train car) is an abnormal behavior object. A detailed explanation will now be given with reference to Figures 2 to 4.

<Pattern 1 (normal behavior)>
Fig. 2 shows an example of a camera image of a person behaving normally. Note that each of the images in the first to third frames shown in Fig. 2 was taken at an arbitrary timing after the train started to move, and the same is true for Figs. 3 and 4 described later. In the image of each frame, reference numeral 21 indicates a train, reference numeral 22 indicates a braille block, reference numerals 23 and 24 indicate a person on the platform, and reference numeral 25 indicates an object (a trash can) installed on the platform.

Figure 2 shows an example of an image in which the train 21 is traveling upward (in the direction of the white arrow) while the people 23, 24 and the installation 25 are stationary. In Figure 2, the positions of the people 23, 24 and the installation 25 in the image move downward, opposite to the direction of travel of the train 21, as the frames progress. This type of object movement is normal and does not pose any particular problems.

<Pattern 2 (normal behavior)>
Fig. 3 shows another example of a camera image of a person behaving normally. Fig. 3 shows an example of an image in which the train 21 runs upward, the person 23 moves upward, the person 24 moves downward, and the installation 25 is stationary. In Fig. 3, the positions of the person 24 and the installation 25 in the image move downward, opposite to the traveling direction of the train 21, as the frames progress. Also, the position of the person 23 in the image is approximately the same in each frame (i.e., the traveling speed of the train 21 and the walking speed of the person 23 are approximately the same). This case is also normal behavior, and does not pose any particular problem.

<Pattern 3 (Abnormal Behavior)>
Fig. 4 shows an example of a camera image capturing a person behaving abnormally. Fig. 4 shows an example of an image in which a train 21 is traveling upward, a person 23 is moving toward the left where the train is, a person 24 is moving downward, and an installation 25 is stationary. In Fig. 4, the positions of the person 24 and the installation 25 in the image are moving downward, opposite to the direction of travel of the train 21, as the frames progress. Meanwhile, the position of the person 23 in the image is moving toward the left where the train 21 is as the frames progress. In this case, the movement vector of the person 23 is different from the normal behavior of patterns 1 and 2, and the person is detected as an abnormally behaving person.

The analysis device 16 detects the above-mentioned abnormal behavior object (person 23 in pattern 3 shown in FIG. 4) by comparing previous and next frames of the camera image. That is, the analysis device 16, for example, detects the feature amount of the object contained in the image of each frame, tracks the object movement between frames based on the identity or similarity of the feature amount, calculates the amount of movement in the lateral direction (to the left where the train is in this example) between frames, and detects an object whose movement amount is equal to or exceeds a predetermined value as an abnormal behavior object. Note that the position of the train in the image may be set as advance information, or may be recognized from the camera image by deep learning or the like, and the method is not important.

Here, the analysis device 16 may analyze the image from camera 11(A) facing in the direction of train travel, or may analyze the image from camera 11(B) facing in the opposite direction. However, camera 11(B) facing in the opposite direction to the direction of train travel is less likely to get dirty and has better visibility of the image, so is more preferable as the analysis object than camera 11(A) facing in the direction of train travel. Note that images from both cameras 11(A) and 11(B) may be analyzed.

The analysis device 16 analyzes the camera images from the cameras on both sides of the vehicle, on the side where the doors open at the station (i.e., the platform side). Which side of the vehicle the platform is on can be easily determined, for example, by setting this information in advance for each station. The determination may also be made using a platform detection device, or it may be made from camera images using deep learning or the like, and the method is not important.

Here, the analysis device 16 of this example does not detect abnormal behavior based only on the direction of movement of the object in the camera image, but also detects the abnormal behavior by taking into account the train speed and the distance between the object and the train. The train speed is incorporated into the detection conditions for abnormal behavior because the train speed affects safety. In other words, even if an object is moving in the direction of the train, there is little danger if the train is stopped or traveling slowly, whereas the danger increases as the train speed increases. In this example, the conditions are set for train speed, divided into three stages: "stopped," "slow," and "medium/high speed."

The distance between the object and the train is also incorporated into the detection conditions for abnormal behavior because the distance between the object and the train also affects safety. In other words, if the object and the train are more than a certain distance away, there is no problem even if the train is moving, but if the distance is close, the risk increases accordingly. If the train is "stopped," there is no particular problem even if the object is moving toward the train or if the distance between the object and the train is close. If the train speed is "low," there is little problem even if the distance between the object and the train is relatively close, so a small value is set as the threshold for the distance between the object and the train. On the other hand, if the train speed is "medium/high," the risk increases, so the threshold for the distance between the object and the train is set to a higher value than for "low."

5 shows an example of a process flow for detecting an abnormally behaving object by the analysis device 16. Here, as a threshold value for the distance between an object and a train, Acm is applied when the train is moving at a "low speed," and Bcm (where B>A) is applied when the train is moving at a "medium/high speed."
The analysis device 16 judges the speed of the train (step S11). If it is judged to be "stopped" in step S11, nothing in particular is done. Here, "stopped" may include not only a state in which the train is completely stopped, but also a state in which the train is traveling at a speed that is not recognized as being in motion (for example, less than 5 km/h).

If "low speed" is determined in step S11, the analysis device 16 determines the direction of movement of the object in the video (step S12), and further determines the distance between the object and the train (step S13). If the direction of movement of the object is determined to be vertical in step S12, or if the distance is determined to be A cm or more in step S13, the object is determined not to be an abnormally acting object. On the other hand, if the direction of movement of the object is determined to be horizontal (approaching the train) in step S12 and the distance is determined to be less than A cm in step S13, the object is determined to be an abnormally acting object, and abnormality detection information is output to notify the driver.

Even if the train speed is judged to be "medium/high speed" in step S11, the analysis device 16 judges the direction of movement of the object in the video (step S14) and further judges the distance between the object and the train (step S15). If the direction of movement of the object is judged to be vertical in step S14, or if the distance is judged to be B cm or more in step S15, the object is judged not to be an abnormally acting object. On the other hand, if the direction of movement of the object is judged to be horizontal (approaching the train) in step S14 and the distance is judged to be less than B cm in step S15, the object is judged to be an abnormally acting object and abnormality detection information is output to notify the driver.

Next, a method for notifying the driver of the detection of an abnormal behavior object will be described. The abnormality detection information output by the analysis device 16 when it detects an abnormal behavior object is transmitted to, for example, the display device 13. This abnormality detection information includes identification information (e.g., camera number) of the camera image in which the abnormal behavior object was detected, the position and size of the abnormal behavior object in the camera image, etc. Based on the abnormality detection information received from the analysis device 16, the display device 13 displays information indicating the abnormal behavior object superimposed on the camera image in which the abnormal behavior object was detected.

Figure 6 shows an example of the camera image displayed when an abnormal behavior entity is detected. In the example of Figure 6, two people 23, 24 are shown in the camera image 31 displayed on the display device 13, and person 23, who has been determined to be an abnormal behavior entity, is surrounded by a frame 32. The frame 32 is displayed in an accent color (e.g., red) that is easy for the driver to recognize. Therefore, the driver can confirm the abnormal behavior entity moving toward the running train with just one glance at the display of the display device 13. In this example, the frame 32 is displayed as information indicating the abnormal behavior entity, but the information indicating the abnormal behavior entity may be displayed in other ways, such as by displaying a specified mark near the abnormal behavior entity, person 23.

The display device 13 may display only the camera image in which the abnormal behavior entity was detected, or may also display the camera image in the opposite shooting direction. In addition, when the display device 13 is configured to display the camera images of each vehicle in a divided manner, the camera image in which the abnormal behavior entity was detected may be surrounded by a highlighting color such as a red frame or enlarged more than the other camera images to make it stand out more than the other camera images. In addition, when two display devices 13(A) and 13(B) are used as in this example, for example, the camera image in which the abnormal behavior entity was detected may be displayed on the entire surface of the display device 13(A), and the other display device 13(B) may be used to display the divided camera images of each vehicle.

As described above, the display device 13 is controlled so that the display of the camera image stops when the train starts moving. Therefore, if this continues, it will not be possible to check the condition of the abnormal behavior object on the display device 13. Therefore, in this example, when the display device 13 receives abnormality detection information, that is, when an abnormal behavior object is detected by the analysis device 16, the stopping of the display of the camera image is canceled. As a result, the display of the camera image continues when an abnormal behavior object is detected, so that the abnormal behavior object can be checked promptly. The display of the camera image while the train is moving continues until the driver performs a stop operation.

The abnormality detection information is not limited to information instructing the output of a video as described above, and may be information in other forms. As one example, the abnormality detection information may be information instructing a speaker (not shown) installed in the driver's cab to output a specified voice message. As another example, the abnormality detection information may be information instructing a warning lamp (not shown) installed in the driver's cab to light up or flash. Furthermore, information instructing an emergency stop of the train may be used as the abnormality detection information.

As described above, the monitoring system of this example includes cameras 11 attached to each car of the train and an analysis device 16 that analyzes the camera footage. The analysis device 16 is configured to detect abnormal behavior, which is an object moving in the direction of the train in the camera footage taken while the train is traveling, and to output abnormality detection information indicating the detection of an abnormal behavior object in response to the detection of an abnormal behavior object. Therefore, when an object moving in the direction of the train is present while the train is traveling, the driver can easily recognize the object as an abnormal behavior object. This makes it possible to prevent accidents in which an object comes into contact with a moving train.

In addition, the analysis device 16 of this example is configured to detect all objects moving in the direction of the train as abnormally behaving objects, but rather to detect objects as abnormally behaving objects if their distance from the train is less than a threshold value. In other words, objects moving in the direction of the train at a distance from the train are less dangerous, so by not detecting them as abnormally behaving objects, the output of unnecessary abnormality detection information can be suppressed.

The analysis device 16 in this example is also configured to use threshold A as a threshold for determining the distance from the train when the train is moving at "low speed," and threshold B, which is larger than threshold A, when the train is moving at "medium/high speed." In this way, by using multiple thresholds in which the faster the train speed is, the larger the set value, the more abnormally behaving an object can be detected, taking into account the difference in safety distance according to the train speed. Note that in the above explanation, the speed of a moving train is classified into two stages, "low speed" and "medium/high speed," and thresholds are set, but the speed of the train may be classified into three or more stages, and the threshold settings may be subdivided.

The monitoring system of this example also includes a display device 13 that displays camera images, and the display device 13 is configured to display information indicating an abnormal behavior object by superimposing it on the camera image when an abnormal behavior object is detected by the analysis device 16. Therefore, the driver can recognize the presence of an abnormal behavior object simply by looking at the display on the display device 13.

In addition, the display device 13 in this example is configured so that the display of the camera image is stopped when the train starts moving, but if the analysis device 16 detects an abnormal behavioral entity, the stopping of the display of the camera image is canceled. Therefore, the display of the camera image, which would normally be stopped when the train starts moving, is continued when an abnormal behavioral entity is detected, making it possible to quickly confirm the abnormal behavioral entity.

In the above description, the analysis device 16 mounted on each car of the train analyzes the camera footage of that car, but it is also possible to mount a single analysis device 16 on the train and have the camera footage of all cars analyzed by the single analysis device 16. Also, in the above description, the display device 13 has a function for controlling the display content, but the display content may be controlled by a device separate from the display device 13 (e.g., the control device 14).

Figure 7 shows an example of the configuration of a monitoring system according to another embodiment of the present invention. The monitoring system shown in Figure 7 has the same basic configuration as the monitoring system shown in Figure 1. The difference from the monitoring system shown in Figure 1 is that the monitoring system shown in Figure 7 has a front camera 18 on the front side of the cars 10-1 and 10-4 at both ends of the train, which captures the view ahead (front) in the direction of train travel.

In this embodiment, the analysis device 16 mounted on the leading car 10-1 in the direction of train travel detects an abnormal behavior object by analyzing the camera image captured by the forward camera 18 when the train enters the station platform. The method of detecting an abnormal behavior object by the analysis device 16 is the same as in the above-mentioned embodiment. When an abnormal behavior object is detected, the analysis device 16 outputs abnormality detection information indicating the detection of an abnormal behavior object. The display device 13 has its display turned OFF while the train is running, but is controlled to turn ON the camera image display when abnormality detection information is received from the analysis device 16. This allows the driver to quickly recognize an abnormal behavior object even when the train enters the platform, making it possible to prevent accidents in which an object comes into contact with the train while entering the platform.

Here, the entrance of a train to the platform can be detected by various methods. For example, the entrance of a train to the platform can be detected based on information indicating the position of the train, such as kilometre information or GPS (Global Positioning System) location information. Also, for example, the entrance of a train to the platform can be detected by analyzing camera images captured by the forward camera 18. Also, for example, the entrance of a train to the platform can be detected when the train's speed drops below a predetermined speed.

Although the embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and their modifications are included in the scope and gist of the invention described in this specification, etc., and are included in the scope of the invention described in the claims and their equivalents.

The present invention can be provided not only as the devices described above or as systems composed of these devices, but also as methods executed by these devices, programs for implementing the functions of these devices using a processor, and storage media for storing such programs in a computer-readable format.

The present invention can be used in a surveillance system that uses cameras mounted on each car of a train.

10-1 to 10-4: vehicle, 11(A), 11(B): camera, 12: relay device, 13(A), 13(B): display device, 14: control device, 15: vehicle information, 16: analysis device, 18: front camera

Claims (6)

Cameras installed in each carriage of the train,
and an analysis device that analyzes the image of the camera,
the analysis device detects features of an object included in each frame of the image captured by the camera while the train is traveling, tracks the movement of the object between frames based on identity or similarity of the features, calculates the amount of movement of the object in the train direction between frames, identifies an object whose amount of movement in the train direction is equal to or greater than a predetermined value, detects the object as an abnormally acting object if the distance between the object and the train is less than a threshold, and, in response to the detection of the abnormally acting object, outputs abnormality detection information indicating the detection of the abnormally acting object;
The analysis device has a plurality of thresholds set to larger values as the train speed increases, and the abnormal behavior is detected using the threshold corresponding to the train speed, characterized by the monitoring system. 2. The monitoring system according to claim 1 ,
Further, a display device is provided for displaying an image from the camera.
A surveillance system characterized in that, when the abnormal behavior entity is detected by the analysis device, the display device displays information indicating the abnormal behavior entity by superimposing it on the image from the camera. 3. The monitoring system according to claim 2 ,
A surveillance system characterized in that the display device stops displaying the image from the camera when the train starts moving, but if the analysis device detects an abnormal behavioral entity, the stopping of the display of the image from the camera is canceled. 4. The monitoring system according to claim 1,
A surveillance system characterized in that the cameras are mounted on the sides of each car of the train, facing in the direction of train travel or in the opposite direction. 5. The monitoring system according to claim 4 ,
The surveillance system is characterized in that the cameras include a camera mounted on the front side of the leading car in the direction of train travel, which captures the area ahead in the direction of train travel. A surveillance method using cameras attached to each carriage of a train and an analysis device that analyzes images from the cameras, comprising:
the analysis device detects features of an object included in each frame of the image captured by the camera while the train is traveling, tracks the movement of the object between frames based on identity or similarity of the features, calculates the amount of movement of the object in the train direction between frames, identifies an object whose amount of movement in the train direction is equal to or greater than a predetermined value, detects the object as an abnormally acting object if the distance between the object and the train is less than a threshold, and, in response to the detection of the abnormally acting object, outputs abnormality detection information indicating the detection of the abnormally acting object;
The analysis device has a plurality of thresholds set to larger values as the train speed increases, and the abnormal behavior object is detected using the threshold corresponding to the train speed, characterized by this monitoring method.

Images