JP7514769B2 - Display control device, display control method, and monitoring system - Google Patents

Description

The present invention relates to a display control device, a display control method, and a monitoring system.

Conventionally, there is known technology that allows frames of the same time from video uploaded from multiple different vehicles traveling on a road to be displayed and checked on the same screen (see, for example, Patent Document 1).

JP 2018-7198 A

However, with conventional technology, it is difficult to easily obtain a frame that was taken at a specific location at a specific time. For example, with conventional technology, in order to obtain a frame that was taken at a specific location at a specific time, it is necessary to seek the composite frame for each time, which is not easy.

The present invention has been made in consideration of the above, and aims to provide a display control device and a display control method that can easily obtain a frame captured at a specific location at a specific time.

In order to solve the above-mentioned problems and achieve the object, the display control device according to the present invention has a collection unit, a linking unit, and a display control unit. The collection unit collects a plurality of images taken by a plurality of vehicles and location information of the vehicles. The linking unit links each of the plurality of images to each of the location information. The display control unit displays the images and the location information on the screen in association with each other according to the linking by the linking unit.

The present invention makes it easy to obtain frames captured at a specific location at a specific time.

FIG. 1 is a diagram illustrating an example of the configuration of a monitoring system according to an embodiment. FIG. 2 is a diagram showing an example of a UI. FIG. 3 is a diagram for explaining how images are captured at each time. FIG. 4 is a diagram for explaining the change of the vehicle to be photographed. FIG. 5 is a flowchart showing a procedure of a process executed by the display control device according to the embodiment.

Below, embodiments of the display control device, display control method, and monitoring system disclosed in the present application will be described in detail with reference to the attached drawings. Note that the present invention is not limited to the embodiments described below.

First, a monitoring system including a display control device according to an embodiment will be described with reference to FIG. 1. FIG. 1 is a diagram showing an example of the configuration of a monitoring system according to an embodiment.

The monitoring system 1 has a display control device 10 and an in-vehicle device 20. The display control device 10 and the in-vehicle device 20 can communicate data, for example, via the Internet. The display control device 10 functions as a center for the in-vehicle device 20.

The in-vehicle device 20 is mounted on a vehicle. The in-vehicle device 20 may be a drive recorder, a smartphone, or the like. The in-vehicle device 20 has a function of taking images and a function of acquiring location information.

The in-vehicle device 20 is mounted on a vehicle. The number of in-vehicle devices 20 is not limited to that shown in the figure.

The display control device 10 collects images and position information from the in-vehicle device 20. The display control device 10 also displays a UI (User Interface) generated based on the collected information on the screen.

Figure 2 shows an example of a UI. As shown in Figure 2, the UI includes a coordinate plane with time on the vertical axis and location on the horizontal axis.

Images collected by the display control device 10 are arranged on the coordinate plane. For example, the images are specific frames of a video (video) captured by the in-vehicle device 20.

Hereinafter, images placed on the coordinate plane may be referred to as thumbnails. In addition, the spacing between thumbnails on the vertical and horizontal axes can be set in advance as a thinning interval.

The coordinate plane can be moved vertically or horizontally by scrolling the screen on which the UI is displayed. This makes it possible to display thumbnails from the past and thumbnails from different locations on the screen.

The process for obtaining the UI described in FIG. 2 will be described below along with details of each part of the display control device 10.

The display control device 10 has a collection unit 101, a linking unit 102, a matching unit 103, a selection unit 104, and a display control unit 105.

The collection unit 101 collects multiple images taken by multiple vehicles and vehicle position information. The collection unit 101 passes the collected collected data 151 to the linking unit 102.

For example, the collection unit 101 collects frames of video images captured by a vehicle and coordinates indicating the vehicle's position. For example, the coordinates indicating the vehicle's position are obtained by GPS.

The linking unit 102 links each of the multiple images to each of the location information. For example, the linking unit 102 links a frame to location information based on the shooting date and time of the frame and the acquisition date and time of the location information.

By this processing by the linking unit 102, frames are obtained, each of which is assigned with location information and time. The linking unit 102 passes each frame, to which location information and time have been assigned, to the matching unit 103.

The matching unit 103 matches the location information and images with positions on the road using map matching.

The selection unit 104 selects images that correspond to the specified position on the road based on the matching results by the matching unit 103.

For example, the selection unit 104 selects frames that correspond to a position on a specified road. This allows the user to obtain a desired frame by specifying information about the road, without directly specifying location information.

For example, the selection unit 104 can accept the designation of a location or range. The designated location and range are assumed to be included in the setting information 152. The selection unit 104 obtains the designated location and range from the setting information 152.

When a user observes an image (frame) of an accident scene, the user may specify the location where the accident occurred. In this case, the selection unit 104 can select a frame that corresponds to the specified location based on the road information matched by map matching.

In addition, when a user wants to observe frames around a certain point for the purpose of investigating traffic congestion, etc., it is conceivable to specify a range, for example, within 10 m in front of or behind a given point.

The user may also specify a range, such as the area around a specific parking area on a highway, from a specific parking area to a specific interchange, or only the uphill direction of the highway.

Even in such a case, the selection unit 104 can select all frames that correspond to the specified range based on the road information matched by map matching.

The display control unit 105 associates the image and the location information according to the linking performed by the linking unit 102 and displays them on the screen.

In this way, by displaying the results of linking images with location information, it is possible to easily obtain frames taken at a specific location at a specific time. Note that the time is assumed to be associated with the image or location information.

For example, the display control unit 105 displays a screen in which the image is arranged on a coordinate plane having a first axis corresponding to the date and time the image was taken, and a second axis corresponding to the position on the road matched by the matching unit 103, as shown in FIG. 2.

In this way, the display control device 10 provides a UI like that shown in FIG. 2, allowing the user to intuitively and comprehensively grasp the information.

The relationship between the vertical and horizontal axes is not limited to that shown in FIG. 2. In FIG. 2, the first axis corresponds to the vertical axis, and the second axis corresponds to the horizontal axis. Alternatively, the first axis may correspond to the horizontal axis, and the second axis may correspond to the vertical axis.

The display control unit 105 displays the images selected by the selection unit 104. In this way, the display control device 10 can display images according to conditions specified by the user, allowing the user to monitor the situation at a specific location on the road in real time.

The display control unit 105 displays a screen in which the images are arranged on a first axis of a coordinate plane at a preset interval. The first axis corresponds to the vertical axis (elapsed time) in FIG. 2. The interval may be set to a specific value, such as an interval of 3 seconds.

The display control unit 105 also displays a screen in which the images are arranged at a preset interval along a second axis of the coordinate plane. The second axis corresponds to the horizontal axis (location of the road) in FIG. 2. The interval may also be set to a specific value, such as 10 m intervals.

By setting the image display interval (thinning interval) for each axis, users can monitor images at the desired range and scale.

The display interval is assumed to be included in advance in the setting information 152 as a thinning time interval and a thinning distance interval. The display control unit 105 can obtain information regarding the display interval from the setting information 152.

As shown in FIG. 1, the in-vehicle device 20 has a camera 201, an acquisition unit 202, and a transmission unit 203.

The camera 201 captures images of the vehicle's surroundings. The camera 201 may capture images at all times, or may capture images at a location instructed by the display control device 10. The acquisition unit 202 acquires location information of the current location at each time using a GPS or the like.

The transmission unit 203 transmits the image captured by the camera 201 and the location information acquired by the acquisition unit 202 to the display control device 10.

Here, the capturing of images by a vehicle equipped with the in-vehicle device 20 will be described with reference to FIG. 3. FIG. 3 is a diagram for explaining the capturing of images at each time.

3, it is assumed that a collision accident occurs at a location L1 at a time _t1 . At this time, the location L1 is captured in the image capturing range of the vehicle V1.

As time passes, at time _t2 ( _t2 > _t1 ), the vehicle V1 has passed near the location L1. Meanwhile, at time _t2 , the vehicle V2 located behind the vehicle V1 captures the location L1 in its shooting range.

2 is captured by an in-vehicle device of a vehicle V1 at time _t1 , and an image F2 is captured by an in-vehicle device of a vehicle V2 at time _t2 .

In this way, the UI provided by the display control device 10 makes it possible to display images of the same location taken by multiple vehicles side-by-side over time.

This allows users, for example, to investigate the relationship between a collision and traffic congestion by viewing images from the location where the collision occurred to the location where the traffic congestion is occurring.

As shown in Figure 2, the vehicle being photographed may change even at the same location. In the example of Figure 2, the image of vehicle V4 is swapped with the image of vehicle V5.

Figure 4 is a diagram explaining the change of vehicles to be photographed. In Figure 2, the roads running above and below Figure 4 are the roads to be photographed.

As shown in FIG. 4, at a T-junction on the road being photographed, vehicle V4 proceeds onto a different road from the road being photographed, and vehicle V5 instead enters the road being photographed.

In this way, even if the vehicle being photographed is changed midway, the display control device 10 can continue to display the image of the road being photographed.

Figure 5 is a flowchart showing the procedure of the process executed by the display control device according to the embodiment. As shown in Figure 5, first, the collection unit 101 collects images and position information from the in-vehicle device (step S101).

Next, the linking unit 102 links the acquired information (step S102). Then, the matching unit 103 performs map matching based on the location information (step S103).

Next, the selection unit 104 selects frames to be displayed based on the road setting information (step S104). Then, the display control unit 105 places the selected frames on the coordinate plane based on the information regarding thinning out (step S105).

Furthermore, the display control unit 105 outputs a UI that displays the coordinate plane on which the frames are arranged (step S106). Note that the UI output by the display control unit 105 is displayed on a display or the like.

As described above, the display control device 10 has a collection unit 101, a linking unit 102, and a display control unit 105. The collection unit 101 collects a plurality of images taken by a plurality of vehicles and vehicle position information. The linking unit 102 links each of the plurality of images to each of the position information. The display control unit 105 displays the images and the position information on the screen in association with each other according to the linking by the linking unit 102.

In this way, by linking images with location information and displaying the results, it becomes easy to obtain frames taken at a specific location at a specific time.

Conventionally, roadside road surveillance cameras have been used on the roadside to investigate the cause of congestion on expressways and to see if it can be resolved. However, the conventional method has the problem that it is not possible to monitor areas outside the surveillance camera's range.

In this embodiment, images captured by cameras mounted on multiple vehicles traveling on the road are used as a substitute for surveillance camera images, making it possible to perform surveillance as if the surveillance camera were placed at a specific location.

The functions of the display control device 10 are realized by a computer having, for example, a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), flash memory, input/output ports, etc., and various circuits.

The display control device 10 may also acquire the above-mentioned programs and various information via other computers or portable recording media connected via a wired or wireless network.

The computer's CPU functions as a collection unit 101, a linking unit 102, a matching unit 103, a selection unit 104, and a display control unit 105, for example, by reading and executing a program stored in the ROM.

Further advantages and modifications may readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof.

REFERENCE SIGNS LIST 1 Surveillance system 10 Display control device 20 Vehicle-mounted device 101 Collection unit 102 Linking unit 103 Matching unit 104 Selection unit 105 Display control unit 151 Collected data 152 Setting information 201 Camera 202 Acquisition unit 203 Transmission unit

Claims (8)

Collecting images taken by a plurality of vehicles together with time and location information of the images;
A display control device that arranges the collected images of multiple vehicles based on time and location, arranges images obtained from the multiple vehicles at the same date and time, and displays the corresponding changes in location for each of the multiple vehicles . The display control device collects frames of a moving image captured by the vehicle and coordinates indicating a position of the vehicle;
The display control device according to claim 1 , wherein the frame and the position information are linked based on a photographing date and time of the frame and an acquisition date and time of the position information. the display control device matches the location information and the image with a location on a road by map matching ;
Based on the matching results, images that correspond to the specified location on the road are selected ,
2. The display control device according to claim 1, which displays the selected image. The display control device according to claim 3, wherein the display control device displays a screen on which the image is arranged on a coordinate plane having a first axis corresponding to the date and time the image was taken and a second axis corresponding to the position on the matched road. The display control device according to claim 4 , wherein the display control device displays a screen on which the images are arranged with respect to the first axis of the coordinate plane at preset intervals. The display control device according to claim 4 , wherein the display control device displays a screen on which the images are arranged with respect to the second axis of the coordinate plane at preset intervals. A display control method executed by a display control device, comprising:
Collecting images taken by a plurality of vehicles together with time and location information of the images;
A display control method in which the collected images of multiple vehicles are arranged based on time and location, images obtained from the multiple vehicles at the same date and time are displayed, and the corresponding changes in location are displayed for each of the multiple vehicles . A monitoring system having an in-vehicle device mounted in each of a plurality of vehicles and a display control device,
The in-vehicle device includes:
A camera for capturing an image of the surroundings of the vehicle ;
Get location information ,
transmitting the image and the position information to the display control device ;
The display control device includes:
Collecting the images transmitted by the vehicle-mounted device together with the time and location information of the images;
A surveillance system that arranges the collected images of multiple vehicles based on time and location, arranges images taken from the multiple vehicles at the same date and time, and displays the corresponding changes in location for each of the multiple vehicles .

Images