JP7444905B2 - Asset data creation system and program - Google Patents

Description

The present invention relates to an asset data creation system and an asset data creation program.

In recent years, major automobile manufacturers around the world, including Japan, have been developing autonomous driving systems. In order to put this automated driving system into practical use, there is an urgent need to develop data on the road environment and road infrastructure equipment (so-called road asset data).

In particular, road signs provide meaningful and important information for autonomous and semi-autonomous driving. For this reason, there is a growing movement to create a database of the locations and types of signs. Therefore, a system has been proposed that performs image recognition of signs from road images and manually corrects the recognition results to create a sign database.

As such a system, Patent Document 1 states, ``(Claim 7) When the photographing position of the road image is displayed on the map as a symbol or character on the photographing camera trajectory, and the position is selected with an indicating device, ``Display road images at a plurality of adjacent photographing positions from that position on a sub-screen'' and ``(Paragraph 0038) Specify a sign icon displayed on the map and follow the corresponding sign candidate''.

Japanese Patent Application Publication No. 2008-287379

In the technique disclosed in Patent Document 1, the sign database is interactively corrected and finalized based on the sign recognition results, so that it is possible to reduce the number of manual steps and time.
However, since the number of signs that must be checked manually is enormous, there is a strong demand for further improvements in the interactive interface.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a technique for improving work efficiency when interactively modifying or finalizing a marker database.

In order to solve the above problems, one of the representative asset data creation systems of the present invention includes a sign recognition section, a sign position detection section, a data storage section, a video display section, a map display section, and a control section.

The sign recognition unit acquires video data photographed by a moving body with a photographing function and position information of the mobile body, performs image recognition of a sign from the video data, and outputs a sign type.

The sign position detection unit calculates the installation position of the sign based on the video data in which the sign is image-recognized and the position information of the moving object at the time when the sign is photographed.
The data storage unit stores the installation position of the sign and the sign type as a sign database.
The video display section generates a video display area for displaying the video data.
The map display section generates a map display area that displays the installation position of the sign on a map.

The control unit outputs the map display area and the video display area, receives an operation input, and modifies the sign database in the data storage unit according to the operation input. Furthermore, upon receiving the operation input for selecting and specifying the installation position of the sign in the map display area, the control unit displays the video data in which the specified installation position is photographed in the video display area. .

The present invention provides techniques for improving work efficiency when interactively modifying or finalizing a tag database.

Problems, configurations, and effects other than those described above will be made clear by the following description of the embodiments.

FIG. 1 is a block diagram showing the configuration of an asset data creation system according to the first embodiment. FIG. 2 is a flowchart illustrating the automated processing of video data. FIG. 3 is a diagram showing a data structure (tree view structure) in which data is managed by the data storage unit. FIG. 4 is a flowchart illustrating the operation screen display routine. FIG. 5 is an explanatory diagram showing an example of the operation screen. FIG. 6 is a flowchart (1/2) illustrating the operation input processing routine. FIG. 7 is a flowchart (2/2) illustrating the operation input processing routine. FIG. 8 is a diagram illustrating the cooperative operation between the map display area and the video display area. FIG. 9 is a diagram illustrating (continuous) skip playback of a detection section. FIG. 10 is a diagram illustrating correction of a travel route and batch correction of sign icons. FIG. 11 is a diagram illustrating other operating functions. FIG. 12 is a diagram illustrating other operation functions.

Embodiments of the invention will be described below with reference to the drawings.

<Configuration of asset data creation system>
FIG. 1 is a block diagram showing the configuration of an asset data creation system 100 according to the first embodiment.

In the figure, the asset data creation system 100 includes a sign recognition section 110, a sign position detection section 120, a data storage section 130, a map display section 140, a video display section 150, a control section 160, and a representative image extraction section 170.

The sign recognition unit 110 receives video data, which is a peripheral image of the moving object 200, from a moving object 200 (vehicle, etc.) including an imaging device 210 and a GPS terminal 220, and position information (latitude and longitude, direction of the vehicle body) of the moving object 200. and collect data. Such data collection may be performed in real time via a communication network, or may be performed as a batch process via the recording device 230 in between. Note that the position information of the mobile object 200 may be collected by the marker position detection unit 120.
The sign recognition unit 110 uses image recognition to detect a sign appearing in a video frame of the acquired video data. Note that position detection means other than the GPS terminal 220 may be used. Furthermore, the position information (latitude and longitude and direction of the vehicle body) of the mobile object 200 may be obtained from various sensors, a driving control device, an autonomous driving system, an external traffic monitoring center, an artificial satellite, a drone, etc. of the mobile object 200. . Furthermore, the orientation of the vehicle body may be determined based on the position of a celestial body such as the sun that can be seen from the mobile object 200, in addition to a direction sensor such as a geomagnetic sensor.

The sign position detection unit 120 calculates the installation position of the sign based on the video data in which the sign is image-recognized and the position information of the moving body 200 at the time when the sign is photographed. This calculation method will be described later.

The data storage unit 130 stores a sign database including installation positions, sign types, etc. for signs to be image recognized in a storage medium 130A built into the system. For each individual marker in this marker database, there is provided a "confirmed flag" area that indicates whether or not the marker has been corrected or confirmed by an operator in a set/reset state.
The map display unit 140 generates a map display area that displays the installation positions of the signs on a map.
The video display unit 150 generates a video display area for displaying video data.

The control unit 160 outputs an operation screen including a map display area and a video display area to the client terminal 300 via the network NW, and receives operation input from the client terminal 300.
The representative image extraction unit 170 extracts a "representative image in which a sign is captured" from the video data.

Such an asset data creation system 100 uses a CPU (Central CPU) as hardware.
It is configured as a computer system equipped with l Processing Unit) and memory. When this hardware executes an asset data creation program stored in a computer-readable medium, various functions of the system are realized. Part or all of this hardware may be a dedicated device, machine learning machine, DSP (Digital Signal Processor), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), PLD (programmable logic device), etc. May be substituted. Furthermore, various functions may be provided as cloud services by configuring a cloud system by centralizing or distributing some or all of the hardware and programs on servers on a network.

<About automated processing of video data>
Next, a description will be given of the processing automatically performed by the asset data creation system 100 on the collected video data.

FIG. 2 is a flowchart illustrating automated processing of video data. The automated processing here may be performed without delay (for example, in a pipeline manner) at the time when the moving object 200 acquires the video data. Alternatively, the processing may be performed all at once as a batch process after the video data is acquired.
The steps will be explained below in accordance with the step numbers shown in the figure.

Step S101: The sign recognition unit 110 acquires (video data, position information) collected at the mobile object 200. These data are temporarily stored in the data storage unit 130.

Step S102: The sign recognition unit 110 performs image recognition of the sign (including the main sign and the auxiliary sign) in the image of the video data, and outputs it as a sign type.

For example, the sign recognition unit 110 uses a convolutional neural network for this image recognition. An example of this convolutional neural network is that a scanning range of a predetermined vertical and horizontal size is sequentially cut out from a video frame of video data (such as a video frame that has been pre-processed such as pixel number conversion, distortion correction, gradation correction, etc.) and is input. It includes an input layer, an intermediate layer including multiple layers such as a convolution layer, a pooling layer, and a combination layer, and an output layer that outputs "likelihood for each label type." For this convolutional neural network, machine learning (error backpropagation method, etc.) has been repeatedly performed in advance using a learning data set consisting of training sign images and teacher values indicating correct answers for the sign type.

Furthermore, for auxiliary signs containing characters and symbols, the sign recognition unit 110 also uses character and symbol recognition processing such as OCR and neural networks to decode the characters and symbols.

Step S103: The sign position detection unit 120 specifically calculates the installation position (latitude and longitude) of the sign through the following calculation process.

(1) The sign position detection unit 120 determines the shooting distance D to the sign with the moving object 200 as the origin O. This photographing distance D is calculated based on the size of the sign in the image data, the known actual size of the sign, and the photographing angle of view of the imaging device 210. Further, if the imaging device 210 mounted on the moving object 200 is capable of outputting depth data of the object field, the photographing distance D to the sign may be determined from the depth data.

(2) The sign position detection unit 120 detects the sign with the mobile object 200 as the origin O based on the position of the recognized sign on the image and information on the photographing direction determined based on the orientation (azimuth) of the vehicle body of the mobile object 200. Find the direction toward as the spatial angle α.

(3) The sign position detection unit 120 offsets the position coordinates (latitude and longitude) of the moving object 200 at the time of photographing the sign by the polar coordinates (shooting distance D, spatial angle α) of the sign with the moving object 200 as the origin O. By performing calculations, the installation position (latitude and longitude) of the sign is calculated.

(4) If there are signs with substantially the same sign type and installation position between adjacent video frames, the sign position detection unit 120 performs name matching processing (installation position, sign type) of the multiple signs. and integrate them into one sign (installation location, sign type). During integration, priority is given to signs with a higher likelihood of image recognition, and integration is performed through weighted addition of installation locations.

In addition, regarding small signs installed on the side of the roadway, a plurality of different signs are often installed side by side on the same installation support. Therefore, in the subsequent processing, errors in the installation positions of small signs that are recognized as the same installation support will be treated as the same installation position.

Furthermore, regarding large signs installed above a roadway consisting of multiple lanes, a plurality of large signs are often arranged side by side on the same installation bar. Therefore, in the subsequent processing, errors in the installation positions of large signs that are recognized as the same installation bar will be treated as the same installation position.

Step S104: FIG. 3 is a diagram showing a data structure (tree view structure) managed by the data storage unit 130.
As shown in the figure, the data storage unit 130 searches for main signs and auxiliary signs (groups) that exist at the same installation position, and groups them hierarchically.

Further, the data storage unit 130 searches for auxiliary signs (groups) that are vertically sandwiched between the main signs at the same installation position, and hierarchically groups the searched auxiliary signs (groups) with the main sign above.

Step S105: The representative image extraction unit 170 extracts the "video frame in which the sign is image-recognized" from the video data, and sets it as a representative image of the sign.

At this time, if there are multiple candidates for the representative image, the representative image extracting unit 170 selects a video frame that has a high likelihood in image recognition of the sign (see step S102), and sets it as the representative image of the sign.

Step S106: The data storage unit 130 stores the signs (installation position, sign type, representative image) as a sign database in the data structure shown in FIG. Since the confirmed flags of these signs have not been modified or confirmed by the operator, they are initially set to a reset state.

Through the above processing, the asset data creation system 100 completes the video data automation processing.

<About the display routine of the operation screen>
Next, the operation screen that the asset data creation system 100 outputs to the client terminal 300 will be described.

FIG. 4 is a flowchart illustrating the operation screen display routine. The display routine is executed by the control unit 160 when the client terminal 300 starts working or updates the display.
FIG. 5 is an explanatory diagram showing an example of the operation screen.
This will be explained below using FIGS. 4 and 5.

Step S201: The control unit 160 reads the sign database from the data storage unit 130 and acquires data (installation position, sign type, representative image) for each sign. Furthermore, the control unit 160 also acquires the generation sources of these data (video data, position information) from the data storage unit 130.

Step S202: The video display section 150 generates a video display area 320 for playing and displaying video data. In this video display area 320, a sign whose image has been recognized is displayed with a rectangular frame indicating "image recognized" (see FIGS. 3, 8, and 9).

Step S203: The video display unit 150 creates a time axis display 321 indicating the playback period of video data and a playback operation unit 322 at the bottom of the video display area 320.

This time axis display 321 includes a slider for indicating the playback position of the video data. The video display section 150 interlocks the position of the slider on the time axis display 321 according to the playback position of the video data displayed in the video display area 320. Further, when receiving a slider movement operation, the video display unit 150 changes the playback position of the video data according to the slider movement position.

Step S204: The video display unit 150 displays the next section on the time axis display 321 in distinguishable colors.

(1) Traveling section (displayed in green color): Corresponds to a video section photographed during a period in which the position information of the moving object 200 is moving. Or it corresponds to a video section in which the background of the video data is moving.

(2) Stop section (displayed in gray color): Corresponds to a video section photographed during a period in which the position information of the moving object 200 is stopped. Or it corresponds to a video section in which the background of video data is static.

(3) Detection section (displayed in red color): corresponds to a video section in which the sign is image-recognized by the sign recognition unit 110.

(4) Caution section (yellow color-coded display): Corresponds to a video section in which a sign-like object is image-recognized by the sign recognition unit 110.

If these sections overlap on the same time axis display 321, the video display section 150 performs color-coded display according to the priority order of the detection section, caution section, driving section, and stop section.

Step S205: The map display unit 140 acquires the position information of the mobile object 200 that captured the video data being played back in the video display area 320. Data of a surrounding map of the position information of this moving object is acquired from a map database (not shown) or the like and displayed in the map display area 310.

Subsequently, the map display unit 140 converts the installation position of the sign read from the sign database into map coordinates of the map display area 310, and displays a sign icon 313 in a superimposed manner at the position of the converted map coordinates. At this time, for a sign whose confirmed flag is in a reset state, a yellow pin indicating that the flag has not yet been confirmed is used in the sign icon 313. Further, for a sign whose confirmed flag is set, a green pin indicating confirmed is used in the sign icon 313.

Step S206: The map display unit 140 calculates the trajectory of the moving route 312 based on the position information of the mobile object 200, and displays the trajectory in a superimposed manner on the map display area 310. Further, the map display unit 140 displays a moving object icon 311 in which the position of the moving object 200 changes in synchronization with video reproduction in the video display area 320 in a superimposed manner on the map display area 310.

Step S207: The control unit 160 generates an input area 330 for modifying and editing the sign type of the sign, and a thumbnail area 340 for selecting video data and representative images.

Step S208: The control unit 160 generates an operation screen (see FIG. 5) including the above-described map display area 310, video display area 320, input area 330, and thumbnail area 340, and outputs it to the client terminal 300. The client terminal 300 displays an operation screen on a monitor screen.

With the above operations, the operation screen display routine is completed.

<Operation input processing routine>
Next, a processing routine for inputting an operation to the operation screen will be described.
6 and 7 are flowcharts illustrating the operation input processing routine.
The following will explain the steps according to the step numbers shown in FIGS. 6 and 7.

Step S301: The control unit 160 outputs an operation screen (see FIG. 5) to the client terminal 300, and starts accepting operation input from the client terminal 300.

Step S302: The control unit 160 determines whether the received operation input is an operation input for clicking on the sign icon 313 (sign installation position).
When the sign icon 313 is designated by clicking, the control unit 160 advances the operation to step S303.
In other cases, the control unit 160 moves the operation to step S311.

Step S303: As shown in FIG. 8, the map display unit 140 changes the click-designated sign icon 313 to a red pin (selected state), indicating that the corresponding sign is subject to correction work in the map display area. show.

Step S304: Upon acquiring the installation position of the specified sign icon 313, the control unit 160 displays a representative image of the corresponding sign in the video display area 320 as "video data captured at this installation position."

Step S305: The operator determines whether the corresponding sign is clearly visible by observing the representative image. If the sign is inappropriate as a representative image, such as because it is too small, the operator manually selects an appropriate representative image by moving the video data in the video display area 320 back and forth frame by frame.

When the control unit 160 receives such a representative image changing operation, the control unit 160 advances the operation to step S306. In other cases, the control unit 160 moves the operation to step S307.

Step S306: The control unit 160 changes the representative image for the corresponding sign according to the representative image changing operation.

Step S307: The control unit 160 changes the installation position and sign type of the corresponding sign according to the operation input to the map display area 310, video display area 320, and input area 330.

For example, when receiving an operation to move the sign icon 313 in the map display area 310, the control unit 160 changes the installation position of the corresponding sign based on the position on the map after the movement of the sign icon 313.
For example, when receiving an input operation in the input area 330, the control unit 160 changes the sign type of the corresponding sign based on the input content in the input area 330.

Step S311: When the control unit 160 receives an operation input for “continuous skip playback” in the video display area 320 (such as double-clicking the skip button on the playback operation unit 322), the control unit 160 advances the operation to step S312. In other cases, the control unit 160 moves the operation to step S313.

Step S312: The video display unit 150 sequentially extracts from the video data only the detection sections in which the sign has been image-recognized, and performs continuous skip playback (see FIG. 9).

Step S313: When the control unit 160 receives an operation input for “skip playback” in the video display area 320 (such as clicking a skip button on the playback operation unit 322), the control unit 160 advances the operation to step S314. In other cases, the control unit 160 moves the operation to step S315.

Step S314: The video display unit 150 performs skip playback by skipping from the current playback position to the next detection section (see FIG. 9).

Step S315: When the control unit 160 receives "a movement operation for a point on the movement route" in the map display area 310, the control unit 160 advances the operation to step S316. In other cases, the control unit 160 moves the operation to step S318.

Step S316: When the control unit 160 receives a movement operation for one point on the movement route, the control unit 160 smoothly corrects the shape of the movement route by spreading the movement operation to a plurality of surrounding points as shown in FIG.

Step S317: The sign position detection unit 120 calculates and collectively corrects the installation positions of the signs as shown in FIG. 10 according to each piece of position information of the moving body 200 that is displaced due to the correction of the movement route. The calculation method here is the same as the calculation method in step S103, so repeated explanation here will be omitted.

Step S318: If the operator finds an "unrecognized sign" on the video display area 320 that is not provided with a rectangular frame indicating "image recognized," the operator draws a rectangular frame for the "unrecognized sign." do. Furthermore, the operator inputs the type of sign for "unrecognized sign" into the input area 330.

On the other hand, the sign position detection unit 120 calculates the installation position of the sign based on the video frame in which the rectangular frame is drawn and the position information of the video frame at the time the sign was photographed. The calculation method here is the same as the calculation method in step S103, so repeated explanation will be omitted.
The map display unit 140 displays a sign icon 313 superimposed on the installation position of the new sign in the map display area 310.

Additionally, if the worker finds an "object mistakenly recognized as a sign" with a rectangular frame indicating "image recognized" attached on the video display area 320, the operator can delete the rectangular frame. . The map display unit 140 deletes the corresponding sign icon 313 from the map display area 310 in conjunction with the rectangular frame deletion operation. The data storage unit 130 deletes the data of the corresponding marker from the marker database in conjunction with the deletion operation of the rectangular frame.

Note that instead of deleting the rectangular frame, the operator may delete the corresponding sign icon 313 from the map display area 310. In conjunction with the deletion operation of the sign icon 313, the video display unit 150 deletes the corresponding rectangular frame from the video display area 320. Further, in conjunction with the deletion operation of the sign icon 313, the data storage unit 130 deletes the data of the corresponding sign from the sign database.

Step S319: The map display unit 140 changes the sign icon 313 that has been corrected or confirmed by the operator to a green pin (the color of the confirmed state).

Step S320: The data storage unit 130 sets a confirmed flag for the signs that have been corrected or confirmed by the operator up to this point and stores them in the sign database. After this storage operation, the control unit 160 returns the operation to step S301 and repeats the operation routine of the operation screen.

<Effects of Examples>
Example 1 has the following remarkable effects.

(1) Normally, when determining whether or not the sign type is correct for the sign icon 313 in the map display area 310, the worker must repeatedly play back the video data and search for the video data in which the corresponding sign was photographed. Must be. The work of reproducing and searching for video data as described above takes time and thus reduces work efficiency.

However, in the first embodiment, when an operation input for selecting and specifying the sign icon 313 (sign installation position) in the map display area 310 is received, video data captured at the specified installation position is automatically displayed in the video display area 320. will be displayed.

Therefore, the operator can immediately determine whether or not the sign type of the sign icon 313 is correct based on the automatically displayed "video data of the installation position." Therefore, it becomes possible for the worker to significantly save the work of reproducing and searching for video data, and it becomes possible to improve work efficiency.

(2) Example 1 differs greatly from Patent Document 1 in the following points.

According to claim 7 of Patent Document 1, when a photographing position (that is, the position of a moving object) is selected on a map, video data photographed from that photographing position is displayed. In this case, since the shooting direction is not specified, there is no guarantee that the displayed video data will show "the direction of the sign that the worker wants to see."

In contrast, in the first embodiment, the installation position of the sign is selected on the map, and video data of the installation position is displayed. Therefore, the displayed video data always shows the direction of the installation position that the worker wants to confirm (that is, the direction of the sign). Therefore, the first embodiment is excellent in that the operator can confirm the mark more reliably.

Note that Patent Document 1 includes a description up to "(Paragraph 0038) Specify a sign icon displayed on a map and follow the corresponding sign candidate," but the selection display of video data related to this operation is Regarding this, no specific disclosure is found in Patent Document 1.

(3) In the first embodiment, the detection section of the video data in which the image recognition has been performed is displayed in such a manner that the detection section of the video data in which the marker has been image-recognized is identifiably displayed on the time axis display indicating the playback period of the video data. Therefore, when searching for a sign from video data, the operator only has to play back the video data by focusing on the detection section. In this respect as well, the working efficiency of the first embodiment is increased.

(4) Embodiment 1 has a function of automatically extracting and reproducing only the detection section displayed on the time axis display from the video data. Therefore, the effort of manually selecting and reproducing the detection section is further eliminated. In this respect as well, the working efficiency of the first embodiment is increased.

(5) In the first embodiment, "an image in which a sign is image-recognized" is automatically extracted from the video data and saved as a representative image of the sign. Therefore, there is no need to manually extract representative images of signs from video data. In this respect as well, the working efficiency of the first embodiment is increased.

(6) In the embodiment, a video with a high likelihood of image recognition of a sign is selected from among "videos in which a sign is recognized as an image" and used as a representative video. For images that have a high likelihood of image recognition, it is possible to obtain a representative image in which the sign is clearly visible, such as a sign facing the front.

(7) In the first embodiment, when an operation input regarding the installation position of a sign is received in the map display area, a representative image of the sign is output to the video display area, and an operation to change the representative image is accepted. Therefore, even if video data is selected manually, the representative image can be changed using the automatically selected representative image as a starting point, which increases work efficiency.

(8) In the first embodiment, by performing a movement operation on one point on the movement path, the movement path is corrected by affecting multiple points on the movement path. The installation position of the sign can be corrected according to the movement of multiple points on this movement route. Therefore, it becomes possible to collectively correct errors in the installation positions of signs based on errors in travel routes. In this respect as well, the working efficiency of the first embodiment is increased.

(9) In the first embodiment, the sign icon 313 before being confirmed and the sign icon 313 after being corrected or confirmed are displayed in a distinguishable manner in the map display area. Therefore, the operator can easily distinguish the sign icon 313 before it is finalized.

(10) In the first embodiment, the auxiliary sign that is vertically sandwiched between the main sign is hierarchically grouped with the main sign above it. Therefore, when a plurality of main signs exist at the same installation position, it is possible to automatically determine which main sign the auxiliary sign should be hierarchically classified into.

(11) In the first embodiment, due to the synergistic effects of the above-mentioned (1) to (10), it becomes possible to significantly reduce the number of manual steps and time required for correction work of the marker database.

<About additional information>

Note that in the first embodiment described above, a convolutional neural network was used for image recognition of a sign. However, the present invention is not limited thereto. For example, such learning models include decision tree learning, association rule learning, neural networks, genetic programming, inductive logic programming, support vector machines, clustering, Bayesian networks, reinforcement learning, representation learning, principal component analysis, and extreme Learning machines and other learning models may also be employed.

Furthermore, image recognition of the sign may be performed by applying subject recognition technology such as face detection.

Furthermore, in the first embodiment, a system configuration in which a plurality of client terminals 300 are provided via the network NW has been described. However, the present invention is not limited thereto. For example, the asset data creation system 100 may be configured by a single computer.

In the first embodiment, the mobile object 200 and the asset data creation system 100 are configured separately. However, the present invention is not limited thereto. For example, part or all of the asset data creation system 100 may be mounted on the mobile body 200.

Furthermore, in the first embodiment, a case has been described in which the mobile object 200 is a road vehicle and road signs are handled. However, the present invention is not limited thereto. For example, it is also possible to handle railway signs by using the mobile object 200 as a railway vehicle. Furthermore, for example, it is also possible to treat sidewalk signs, facility signs, etc. as the moving object 200 as a pedestrian or bicycle.

Note that the present invention is not limited to the above-described embodiments, and includes various modifications. For example, the embodiments described above are described in detail to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to having all the configurations described.
Furthermore, the present invention is not limited to the operating functions described in the first embodiment, and other operating functions (see, for example, FIGS. 11 and 12) may be added as appropriate.

100... Asset data creation system, 110... Sign recognition section, 120... Sign position detection section, 130... Data storage section, 130A... Storage medium, 140... Map display section, 150... Video display section, 160... Control section, 170... Representative image extraction unit, 200... Moving object, 210... Imaging device, 220... GPS terminal, 310... Map display area, 312... Movement route, 313... Sign icon, 320... Video display area, 321... Time axis display, 322... Playback operation section, 330...input area, 340...thumbnail area

Claims (9)

a sign recognition unit that performs image recognition of the sign from video data captured by an imaging device installed on the moving object and outputs the sign type;
a sign position detection unit that calculates the installation position of the sign based on the video data in which the sign is image-recognized and the position information of the moving body at the time when the sign is photographed;
a data storage unit that stores the installation position of the sign and the sign type as a sign database;
a video display unit that generates a video display area for displaying the video data;
a map display unit that generates a map display area that displays the installation position of the sign on a map;
a control unit that outputs the map display area and the video display area, receives an operation input, and corrects the sign database of the data storage unit according to the operation input,
When the control unit receives the operation input for selecting and specifying the installation position of the sign in the map display area, the control unit displays the video data of the sign at the specified installation position in the video display area. death,
The video display section creates a time axis display showing a reproduction period of the video data, and displays a section of the video data in which the sign has been image-recognized by the sign recognition section (hereinafter referred to as a "detection section") and the sign recognition section. An asset data creation system characterized in that the section of the video data in which the sign-like object is image-recognized is displayed in distinguishable colors on the time axis display. The asset data creation system according to claim 1,
The asset data creation system is characterized in that the video display section has a function of automatically extracting and reproducing only the detection section displayed on the time axis display from the video data. The asset data creation system according to any one of claims 1 to 2,
An asset data creation system comprising: a representative image extraction unit that extracts "an image in which the sign is recognized as an image" from the video data and stores it in the sign database as a representative image of the sign. The asset data creation system according to claim 3,
The asset data creation system is characterized in that the representative image extraction unit selects a video that has a high likelihood of image recognition of the sign from among "videos in which the sign has been image recognized" and uses it as the representative video. . The asset data creation system according to any one of claims 3 to 4,
When the control unit receives the operation input for the installation position of the sign in the map display area, the control unit outputs the representative image of the sign to the video display area, and receives an operation to change the representative image. Asset data creation system. The asset data creation system according to any one of claims 1 to 5,
The map display section displays a travel route of the mobile object in the map display area,
When the control unit receives a movement operation for one point on the movement route in the map display area, the control unit spreads the movement operation to a plurality of points on the movement route to smoothly modify the shape of the movement route;
The asset data creation system is characterized in that the sign position detection unit corrects the installation position of the sign based on each of the position information of the moving object that is displaced based on the correction of the movement route. The asset data creation system according to any one of claims 1 to 6,
The asset data creation system is characterized in that the map display unit displays, in the map display area, a display of the sign before confirmation and a display of the sign that has been confirmed in a distinguishable manner. The asset data creation system according to any one of claims 1 to 7,
The sign recognition unit image-recognizes the main sign and the auxiliary sign from the video data,
The data storage unit searches for the auxiliary sign that is vertically sandwiched between the main sign at the same installation position, groups the searched auxiliary sign with the main sign above, and stores the group. Asset data creation system. An asset data creation program that causes a computer system to function as the asset data creation system according to any one of claims 1 to 8.

Images