JP7748620B2 - Photo location registration system - Google Patents

Description

The present invention relates to a photography location registration system.

It is known that images captured by vehicles are collected by information centers and used for various purposes. Patent Document 1 describes having an on-board camera capture images at a location and in a direction specified by the information center.

Japanese Patent Application Laid-Open No. 2003-315069

In the case of an in-vehicle camera with a fixed shooting direction, if a shooting direction other than the direction in which shooting is possible at the specified shooting location is specified, there is a problem in that the intended image cannot be obtained.
The present invention has been made in consideration of the above-mentioned problems, and aims to provide a technique for preventing mistakes in specifying the shooting direction at a shooting location.

To achieve the above objective, the photography location registration system includes a reception unit that receives the photography location and the photography direction at the photography location; a direction identification unit that identifies the direction in which the road section including the photography location extends; a guidance unit that notifies the user, if there is a difference between the photography direction and the direction in which the road section including the photography location extends; and a registration unit that registers the photography direction re-specified in accordance with the guidance, in association with the photography location.

In other words, the photography location registration system notifies the user if there is a difference between the photography direction at a specified photography location and the direction in which the road section including that photography location extends. Therefore, the photography location registration system can give the user an opportunity to reconsider whether the specified photography direction is as intended. If the difference indicates that it is not as intended, the photography direction can be re-specified. The photography location registration system registers the re-specified photography direction in association with the photography location. Therefore, the photography location registration system can reduce the possibility that the user will make a mistake in specifying the photography direction at a photography location.

FIG. 1 is a block diagram showing the configuration of a photography location registration system. FIG. 10 is a diagram showing an example of a shooting location registration screen. 10 is a flowchart of a photography location registration process.

Here, the embodiments of the present invention will be described in the following order.
(1) Configuration of the photography location registration system:
(2) Shooting location registration process:
(3) Other embodiments:

(1) Configuration of the photography location registration system:
FIG. 1 is a block diagram showing the configuration of a photography location registration system 10 according to the present invention. In this embodiment, the photography location registration system 10 is a system for a road maintenance manager to register photography locations for use in a drive recorder to capture images of road repair-needing areas. The captured images are shared with road repair workers and used to identify repair-needing areas near the photography location and to check their status. In this embodiment, the photography location registration system 10 is configured as a server computer and is capable of communicating with both an in-vehicle system 50 of a vehicle equipped with a drive recorder and an administrator terminal 60 used by a road repair manager. There may be multiple vehicles equipped with the in-vehicle system 50. There may also be multiple administrator terminals 60.

When the administrator operates the administrator terminal 60 to register a photography location and other information in the photography location registration system 10, the photography location registration system 10 distributes photography instruction information including the photography location and other information to the in-vehicle system 50. The in-vehicle system 50 is configured to take photographs based on the photography instruction information when the vehicle passes through the photography location.

The in-vehicle system 50 includes a communication unit 50a, a positioning unit 50b, a control unit 50c, and a camera 50d. The communication unit 50a is a communication circuit for communicating with other devices such as the photography location registration system 10. The positioning unit 50b includes a GNSS receiver, a vehicle speed sensor, and a gyro sensor. The GNSS receiver is a device that receives GNSS (Global Navigation Satellite System) signals, receives radio waves from navigation satellites, and outputs signals for calculating the vehicle's current location. The vehicle speed sensor outputs a signal corresponding to the rotational speed of the vehicle's wheels. The gyro sensor detects the angular acceleration of the vehicle when it turns in a horizontal plane, and outputs a signal corresponding to the vehicle's orientation.

The control unit 50c is equipped with a CPU, RAM, ROM, etc., and can execute programs stored on a recording medium (not shown) or ROM. The control unit 50c acquires the current location of the vehicle based on the signal output from the GNSS receiver and the self-contained navigation trajectory, which is a trajectory of the position estimated based on the signals output from the vehicle speed sensor and gyro sensor. The control unit 50c periodically acquires the current location of the vehicle. The control unit 50c generates a driving trajectory of the vehicle by associating the acquired current location of the vehicle with the time. The control unit 50c associates the driving trajectory with the ID of the in-vehicle system 50 and transmits it to the photography location registration system 10 at a predetermined timing (for example, when the vehicle stops or when the ACC is turned off).

Camera 50d is attached to the vehicle so that its imaging range includes the front side in the vehicle's direction of travel. In this embodiment, camera 50d does not have the function of automatically changing the imaging direction under the control of control unit 50c, and camera 50d is fixed to the vehicle so that its imaging direction is forward in the vehicle's direction of travel. Photographing instruction information distributed from the photography location registration system 10 is recorded on a recording medium (not shown). The photography instruction information includes information indicating the photography location registered by the user (administrator) of the photography location registration system 10 and the photography direction at that photography location. When the vehicle passes the photography location indicated in the photography instruction information, control unit 50c determines whether the difference between the photography direction indicated in the photography instruction information and the direction of travel is within a predetermined threshold, and if it is within the threshold, causes camera 50d to capture an image. Control unit 20 associates image data indicating the captured image with the photography location, photography direction, and photography time, and transmits the image data to photography location registration system 10. In this embodiment, if the difference between the shooting direction indicated by the shooting instruction information and the vehicle's direction of travel is not within a threshold, the control unit 50c will not perform shooting. Therefore, if the direction in which the road extends (which is approximately parallel to the direction in which the vehicle will travel on that road) and the shooting direction at the shooting point on that road do not match, no image will be obtained at that shooting point.

The administrator terminal 60 is composed of a PC, tablet, smartphone, etc. The administrator terminal 60 has a communication unit 60a, a user I/F unit 60b, and a control unit 60c equipped with a CPU, RAM, ROM, etc., and can execute programs stored in a recording medium or ROM (not shown) using the control unit 60c. The communication unit 60a has a circuit for communicating with other devices, and the control unit 20 can communicate with the photography location registration system 10 via the communication unit 60a. The user I/F unit 60b inputs user (administrator) operations via input devices such as a keyboard and mouse, and provides the user with various information via output devices such as a display.

The administrator displays the photography location registration interface provided by the photography location registration system 10 on the display of the user I/F unit 60b, and uses a mouse or other device to register the photography location and the photography direction at that photography location. This information is sent from the administrator terminal 60 to the photography location registration system 10 and saved as photography instruction information 30b on the recording medium 30 of the photography location registration system 10.

The photography location registration system 10 includes a control unit 20, a recording medium 30, and a communication unit 41. The communication unit 41 includes a communication circuit for communicating with other devices, allowing the photography location registration system 10 to communicate with the in-vehicle system 50 and the photography location registration system 10 itself. The control unit 20 includes a CPU, RAM, ROM, etc., and can execute programs stored in the ROM or recording medium 30. When the control unit 20 receives a driving trajectory transmitted from the in-vehicle system 50, it associates it with an ID that identifies the in-vehicle system 50 and stores it on the recording medium 30 (driving trajectory 30a). The driving trajectory 30a is a collection of data that associates the vehicle's position with the date and time the vehicle was located at that position. The driving trajectory 30a is the driving trajectory of the vehicle equipped with the in-vehicle system 50, and in this embodiment, it is used to identify the direction in which a road section extends.

Furthermore, when the control unit 20 receives information such as the shooting location and shooting direction from the administrator terminal 60, it stores this information as shooting instruction information 30b on the recording medium 30. The shooting instruction information 30b includes information indicating the shooting location and the shooting direction at that shooting location. The control unit 20 distributes the shooting instruction information 30b to the in-vehicle system 50. When image data captured by the in-vehicle system 50 based on the shooting instruction information 30b is sent to the shooting location registration system 10, the control unit 20 stores the image data 30c on the recording medium 30 in association with the shooting instruction information 30b.

By executing the photography location registration program 21, the control unit 20 provides the administrator terminal 60 with a registration interface for obtaining images taken at a specified photography location in a specified photography direction. The photography location registration program 21 includes a reception unit 21a, a direction identification unit 21b, a guidance unit 21c, and a registration unit 21d. Using the function of the reception unit 21a, the control unit 20 accepts the specification of the photography location and the photography direction at the photography location. In other words, the control unit 20 displays the registration interface on the display of the administrator terminal 60 and transmits control data to the administrator terminal 60 to accept operations by the administrator.

Figure 2 shows an example of a registration interface. The registration interface includes a map. The control unit 20 obtains map information for the area subject to road maintenance management from a map server (not shown), and displays it on the display of the administrator terminal 60 as a map within the registration interface. The administrator can use a mouse or other device on the administrator terminal 60 to select any position on the road within the map as the starting point for photography. In the example of Figure 2, an icon 100 on the road indicates the position and direction of the photography point selected by the administrator. In the example of Figure 2, the arrow on the icon 100 points upward in the figure. In other words, the icon 100 indicates that photography will be taken at the photography point indicated by the position of the icon 100 when a vehicle travels along the road on which the icon 100 is set in a direction from bottom to top in the figure.

The registration interface also includes a direction specification section 102 for specifying the shooting direction at the shooting location specified by the administrator, a survey item selection section 101 for the shooting location, and a registration button 103. The administrator specifies the shooting location by placing the icon 100 at the desired shooting location on a road on the map. The administrator can then rotate the direction of the arrow on the icon 100 (i.e., the shooting direction) by sliding the direction specification section 102 left or right on the map. For example, the arrow on the icon 100 can be rotated 360 degrees clockwise by moving the direction specification section 102 from the left end to the right end of the slider. The administrator can also select the desired item from the pull-down list in the survey item selection section 101. After specifying the survey items and shooting direction for the shooting location, the administrator selects the registration button 103, and the shooting location, shooting direction, and survey items are sent from the administrator terminal 60 to the shooting location registration system 10. The control unit 20 accepts the shooting location and shooting direction sent from the administrator terminal 60 as the administrator's designation.

As described above, the camera 50d of the in-vehicle system 50 is mounted on the vehicle facing forward in the direction of travel of the vehicle. If the administrator unintentionally specifies a shooting direction that differs by more than the threshold value from the direction in which the road section in which the shooting point is set extends, the in-vehicle system 50 will not be able to capture images in the specified shooting direction at the specified shooting point. Therefore, the control unit 20 is configured to determine whether the shooting direction at the specified shooting point matches the direction in which the road section including the shooting point extends.

Using the function of the direction identification unit 21b, the control unit 20 identifies the direction in which the road section including the shooting location extends. That is, the control unit 20 first identifies the direction in which the road section including the shooting location extends to determine whether the shooting direction at the specified shooting location matches the direction in which the road section including the shooting location extends. In this embodiment, map information is not recorded on the recording medium 30 of the shooting location registration system 10, and the direction in which the road section including the shooting location extends is identified without using map information. Specifically, the control unit 20 identifies the direction in which the road section including the shooting location extends based on the vehicle's traveling direction indicated by the traveling trajectory of the vehicle that passed near the shooting location. Therefore, the control unit 20 references the traveling trajectory 30a and identifies the traveling trajectory that passes within a predetermined distance of the specified shooting location. The control unit 20 identifies the road section of the traveling trajectory that is closest to the specified shooting location and identifies the direction in which the road section extends. Specifically, the control unit 20 identifies the line segment that is closest to the specified shooting location among the line segments connecting the points that make up the traveling trajectory. The control unit 20 then identifies the direction in which the line segment extends. In this embodiment, the direction in which the line segment extends indicates the direction in which the road section extends (for example, extending north to south) and the estimated direction in which travel is possible on that road section (available travel direction). With regard to the latter, for example, if multiple line segments closest to the image capture point are identified from multiple driving trajectories, and the available travel directions indicated by the multiple line segments include some that are approximately the same and others that differ by approximately 180 degrees, it can be estimated that the image capture point was specified on a road section that is passable in both directions.

Using the function of the guidance unit 21c, the control unit 20 notifies the administrator of the difference between the shooting direction and the direction of the road section including the shooting location when there is a difference. In this embodiment, the control unit 20 determines whether the difference between the specified shooting direction and the direction of the road section including the shooting location is within a threshold. If the difference between the two is greater than the threshold, the control unit 20 notifies the administrator that there is a difference between the two. In addition, when providing this guidance, the control unit 20 suggests setting the direction of the road section including the shooting location as the shooting direction. Specifically, the control unit 20 displays a message asking whether the direction identified by the direction identification unit 21b should be set as the shooting direction, along with buttons for selecting "Accept" and "Reject." In this embodiment, for example, if only one possible travel direction has been estimated from the driving trajectory, it is determined whether the difference between the specified shooting direction and that possible travel direction is greater than a threshold. If the difference is greater than the threshold, a button to accept the possible travel direction as the shooting direction and a button to reject the decision (selecting the originally specified direction) are displayed. For example, if the driving trajectory indicates that the road section is passable in both directions, even if the specified shooting direction roughly matches one of the possible travel directions, the difference from the other possible travel direction will be greater than the threshold, and the control unit 20 will determine that the difference is not within the threshold. For road sections that are passable in both directions, the control unit 20 may be configured to present both possible travel directions and allow the user to select either the two possible travel directions or the originally specified direction. Alternatively, in another embodiment, if the difference between the initially specified shooting direction and one of the two possible travel directions is within the threshold, the control unit 20 may be configured to determine that there is no difference between the originally specified shooting direction and the direction of the road section.

Using the function of the registration unit 21d, the control unit 20 registers the shooting direction re-specified in response to the guidance, associating it with the shooting location. In other words, if the manager selects acceptance, the control unit 20 receives notification of this from the manager terminal 60 and automatically re-specifies the shooting direction originally specified manually by the manager to the direction specified by the direction specification unit 21b. The control unit 20 then associates the re-specified shooting direction with the shooting location and survey item, and registers it in the recording medium 30 as shooting instruction information 30b. Note that if the manager does not select acceptance, the control unit 20 receives notification of this from the manager terminal 60 and registers the shooting direction originally specified by the manager as shooting instruction information 30b in the recording medium 30, associating it with the shooting location and survey item.

As described above, according to this embodiment, the photography location registration system 10 can notify the user if there is a difference between the photography direction at the specified photography location and the direction in which the road section including the photography location extends. Therefore, the photography location registration system 10 can give the user an opportunity to reconsider whether the specified photography direction is as intended. Furthermore, in this embodiment, if the direction in which the road section including the photography location extends is proposed as the photography direction and accepted, the newly specified photography direction can be automatically registered in association with the photography location, allowing the user to easily re-specify the photography direction. If the photography direction is re-specified because the difference is not as intended, the photography location registration system registers the newly specified photography direction in association with the photography location. Therefore, the photography location registration system can reduce the possibility that the user will make a mistake in specifying the photography direction at the photography location.

(2) Shooting location registration process:
Next, the photography location registration process executed by the control unit 20 will be described with reference to FIG. 3. The photography location registration process is executed when a photography location registration interface is displayed on the display of the administrator terminal 60 in response to an administrator's operation on the administrator terminal 60. When the photography location registration process starts, the control unit 20 uses the function of the reception unit 21a to receive the photography location and photography direction specified by the user (step S100). That is, when the user operates a mouse or the like to specify the photography location, photography direction, and survey items and selects the registration button 103, the control unit 20 receives the specified photography location and photography direction.

Next, the control unit 20 uses the function of the direction identification unit 21b to determine whether or not there is a driving trajectory near the shooting location (step S105). That is, the control unit 20 determines, by referring to the driving trajectory 30a, whether or not there is a driving trajectory that passes within a predetermined distance of the shooting location accepted in step S100.

If it is determined in step S105 that a driving trajectory exists near the shooting point, the control unit 20 uses the function of the direction identification unit 21b to identify the nearest road section from the driving trajectory (step S110), and identifies the direction in which the road section extends from the vehicle's direction of travel indicated by the driving trajectory (step S115). That is, the control unit 20 identifies the line segment that is closest to the specified shooting point among the line segments connecting the sequence of points that make up the driving trajectory. Then, the control unit 20 identifies the direction in which the line segment extends.

Next, the control unit 20, using the function of the guidance unit 21c, determines whether the shooting direction and the extension direction of the road section match (step S120). That is, the control unit 20 determines whether the difference between the shooting direction acquired in step S100 and the extension direction of the road section acquired in step S115 is within a threshold value.

If it is determined in step S120 that the two do not match, the control unit 20, using the function of the guidance unit 21c, suggests to the user that the direction in which the road section extends be set as the shooting direction (step S125). That is, the control unit 20 displays, on the display of the administrator terminal 60, a message asking whether the direction identified in step S115 should be set as the shooting direction, along with buttons for selecting "Accept" and "Reject."

The control unit 20 then uses the function of the registration unit 21d to determine whether the user has accepted the proposal (step S130). That is, if the "accept" button is selected, the control unit 20 assumes that the user has accepted the proposal; if the "reject" button is selected, the control unit 20 assumes that the user has not accepted the proposal. If it is determined in step S130 that the proposal has been accepted, the control unit 20 uses the function of the registration unit 21d to set the direction in which the road section extends as the shooting direction and register it in association with the shooting location (step S135). That is, the control unit 20 automatically re-specifies the shooting direction that was initially manually specified by the administrator to the direction identified in step S115. The control unit 20 then associates the re-specified shooting direction with the shooting location and survey item, and registers it as shooting instruction information 30b in the recording medium 30.

If it is not determined in step S130 that consent has been given, or if it is determined in step S120 that the two match, or if it is not determined in step S105 that a driving trajectory exists near the shooting location, the control unit 20 uses the function of the registration unit 21d to register the user-specified shooting direction in association with the shooting location (step S140).

In other words, if it is determined in step S105 that no driving trajectory exists near the specified shooting location, in this embodiment, the direction of the road section including the specified shooting location cannot be identified based on the past vehicle driving trajectory. Therefore, the control unit 20 adopts the shooting direction specified by the user as is, associates it with the shooting location and investigation item, and registers it as shooting instruction information 30b. If it is determined in step S120 that the two match, there is no difference between the two, so the control unit 20 adopts the shooting direction specified by the user as is, associates it with the shooting location and investigation item, and registers it as shooting instruction information 30b. If it is not determined in step S130 that the user accepted the suggestion, it is assumed that the user intentionally specified a direction that differs from the direction of the road section, and the control unit 20 adopts the shooting direction specified by the user as is, associates it with the shooting location and investigation item, and registers it as shooting instruction information 30b. In this case, for example, an operator may manually operate a camera such as a smartphone at the specified shooting location to capture an image in the shooting direction.

(3) Other embodiments:
The above embodiment is one example for implementing the present invention, and various other embodiments can be adopted. At least some of the reception unit 21a, direction identification unit 21b, guidance unit 21c, and registration unit 21d that make up the photography location registration system may be separated into multiple devices. For example, the functions of the direction identification unit 21b and guidance unit 21c may be realized by the administrator terminal 60. Of course, some of the components of the above embodiment may be omitted, and the order of processing may be changed or omitted.

In addition, although in the above embodiment the processing in the flowchart of Figure 3 is executed by the photography location registration system 10, this processing may also be executed by the administrator terminal 60. In that case, the photography location registration system 10 acquires the photography location and photography direction determined in steps S135 and S140 from the administrator terminal 60 and registers them as photography instruction information 30b on the recording medium 30. In addition, the driving trajectory 30a referenced in the processing from steps S105 to S115 is sent to the administrator terminal 60, and processing is performed at the administrator terminal 60 based on the driving trajectory 30a.

The direction in which the road section including the shooting location extends may be identified based on map information. The map information includes node data indicating the positions of nodes set on the road along which the vehicle is traveling, shape interpolation point data indicating the positions of shape interpolation points used to identify the shape of the road between nodes, link data indicating the connections between nodes, and facility data indicating the positions of facilities on and around the road. The link data also includes information indicating the possible travel directions (two-way, one-way, etc.). When a point on a road within the map is selected, the link including that point may be identified based on the link data and shape interpolation point data. The direction in which the road extends and the possible travel directions near the specified shooting location may also be identified based on the link data and shape interpolation point data.

Furthermore, the guidance unit may be configured to notify the user of a difference between the shooting direction and the direction in which the road section including the shooting location extends if such a difference exists. If the user is notified of the difference but the difference is not what they intended, the guidance unit may be configured to allow the user to manually re-specify the shooting direction.

The shooting location may be a roadway, a bicycle path, or a sidewalk. A sidewalk may be a pedestrian road separated from the roadway by a curb or other such object and with a difference in elevation from the roadway, or a pedestrian road with no difference in elevation from the roadway. The subject of the shooting may be, for example, a hole (sinkhole), cracks, deteriorated road paint, etc. Note that the subject of the shooting does not have to be the road surface, and may also be, for example, a structure such as a sign, guardrail, or curb.

The camera 50d of the in-vehicle system 50 may be attached to the vehicle so as to capture images of the area behind the vehicle in the direction of travel. The positioning unit 50b of the in-vehicle system 50 may also be a GNSS positioning unit built into a camera or a drive recorder.

The photography location registration system can be used in a variety of situations, with onboard cameras capturing images in the direction of the road section that includes the photography location, and can be used for purposes other than road repair. For example, this photography location registration system can be used to register locations where onboard cameras can capture images of road congestion, or locations where onboard cameras can capture images of frozen road surfaces, snow accumulation, and flooding.

Furthermore, the techniques of the present invention can also be applied as programs or methods. Furthermore, the above-mentioned systems, programs, and methods may be realized as stand-alone devices or may be realized using components shared with various parts of the vehicle, and include a variety of configurations. They can also be modified as appropriate, such as being partly software and partly hardware. Furthermore, the invention can also be realized as a recording medium for a program that controls the system. Of course, the recording medium for the program may be a magnetic recording medium or a semiconductor memory, and any recording medium developed in the future can be considered in exactly the same way.

10...Photography location registration system, 20...Control unit, 21...Photography location registration program, 21a...Reception unit, 21b...Direction identification unit, 21c...Guidance unit, 21d...Registration unit, 30...Recording medium, 30a...Driving trajectory, 30b...Photography instruction information, 30c...Image data, 41...Communication unit, 50...In-vehicle system, 50a...Communication unit, 50b...Positioning unit, 50c...Control unit, 50d...Camera, 60...Administrator terminal, 60a...Communication unit, 60b...User I/F unit, 60c...Control unit, 100...Icon (Photography location), 101...Survey item selection unit, 102...Direction designation unit, 103...Registration button

Claims (3)

a reception unit that receives a designation of a photographing location and a photographing direction at the photographing location;
a direction specifying unit that specifies the direction in which a road section including the photographing point extends;
a guidance unit that, when there is a difference between the photographing direction and the direction in which the road section including the photographing point extends, notifies the user that there is a difference;
a registration unit that registers the photographing direction re-specified in response to the guidance in association with the photographing location;
A photography location registration system comprising: the direction identification unit identifies the direction in which the road section including the image capturing point extends based on a traveling direction of a vehicle indicated by a travel locus of the vehicle that has passed near the image capturing point;
2. The photography location registration system according to claim 1. The guidance unit suggests that the direction in which a road section including the photography point extends be set as the photography direction.
3. The photography location registration system according to claim 1.