JP7657175B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

This disclosure relates to an information processing device, an information processing method, and an information processing program.

Conventionally, there is a driving support device that supports a vehicle traveling on a road. The driving support device acquires the position (autonomous position) of the vehicle based on the vehicle's traveling direction and traveling speed. The driving support device also selects some of a plurality of artificial satellites based on celestial information, and acquires a position (heteronomic position) measured based on satellite signals transmitted from the selected artificial satellites. Here, the celestial information is incorporated into road map information, and includes information indicating the azimuth and elevation angles at which features are located as viewed from the road. The driving support device corrects accumulated errors occurring in the autonomous position based on the heteronomic position.

JP 2006-267004 A

High-precision maps, as an example of road maps, may be created using heteronomous positioning. The accuracy of heteronomous positioning may vary depending on the reception strength of satellite signals and the number of satellites used for positioning. For this reason, it is necessary to estimate the reliability of identifying the position of the vehicle (measurement vehicle) used to create the road map.

As described above, the driving support device described in Patent Document 1 is a technology that corrects the accumulated error of the autonomous position based on the heteronomous position. In this way, since the driving support device aims to correct the autonomous position, if the accuracy of the heteronomous position used when creating a road map, etc. is relatively poor, no technical processing can be added and a road map cannot be created more accurately. In other words, the driving support device described in Patent Document 1 cannot be used, for example, when creating a road map.

The present disclosure provides an information processing device, an information processing method, and an information processing program that can estimate the reliability of a position.

An information processing device according to one embodiment includes an acquisition unit that acquires image information based on an image of the outside captured by the vehicle and position information related to the vehicle's traveling position, and an estimation unit that estimates the reliability of the traveling position according to the scenery recorded in the image information based on a trained model based on learning the scenery recorded in the image and the label assigned to the scenery.

According to one embodiment, it is possible to estimate the reliability of the driving position according to the scenery recorded in the image information based on the trained model.

FIG. 1 is a diagram illustrating an information processing device according to an embodiment. FIG. 1 is a block diagram illustrating an information processing device according to an embodiment. FIG. 1 is a diagram for explaining an example of a road including an underpass section. FIG. 11 is a diagram illustrating an example of correspondence information. 1 is a flowchart illustrating an information processing method according to an embodiment.

One embodiment is described below.

[Overview of information processing device 1]
First, an overview of an information processing device 1 according to an embodiment will be described.
FIG. 1 is a diagram illustrating an information processing device 1 according to an embodiment.

Introduction: For example, in generating high-precision map information, the positional accuracy of the travel trajectory of a vehicle (measurement vehicle) equipped with a measuring instrument is an important factor. The travel position of the vehicle may be measured using, for example, GNSS or the like, and in areas where the reception environment of satellite signals is unstable, such as urban areas and mountainous areas, it is required to effectively and efficiently correct the travel trajectory (travel position).
However, when the driving track (driving position) of a vehicle or the like needs to be corrected, the map creator may not be able to understand the degree and range of deviation of the driving track (driving position). In addition, depending on the season in which the vehicle is traveling, the state of vegetation around the road may differ, and depending on the timing of the vehicle's travel, a large vehicle may travel parallel to the vehicle and cause an obstacle to positioning. In light of such a situation, the information processing device 1 corrects the driving position after narrowing down the range in which the accuracy of the driving position is expected to deteriorate, rather than correcting the driving position by trial and error on the part of the map creator.

That is, the information processing device 1 of the present disclosure may be configured, for example, as an estimation device that estimates the reliability of the traveling position of a moving object based on position information. The information processing device 1 may be configured, for example, as a correction device that corrects the traveling position of a moving object based on position information. The information processing device 1 may be configured, for example, as a creation device that creates a road map based on the traveling position. The information processing device 1 is not limited to the device of the above example, and may constitute various devices.
The information processing device 1 may be a computer such as a server, a desktop, a laptop, a tablet, or a smartphone.

The information processing device 1 acquires, for example, image information and position information.
The image information may be generated by, for example, capturing an image of an external landscape from a moving object. The image information may be generated by, for example, capturing an image of the moving object in the direction of travel by the imaging unit 101 disposed in the moving object, and recording a landscape including an area above the moving object. Alternatively, the image information may be generated by, for example, capturing an image of the area above the moving object by the imaging unit 101, and recording a landscape including an area above the moving object.
The position information may be, for example, information about a traveling position of a mobile object. As an example, the position information may be information about a position acquired when a mobile object moves from a departure point to a destination.

A moving body (such as a vehicle 100, for example) may perform positioning and acquire location information based on receiving, for example, satellite signals (radio signals) using GNSS (artificial satellites 111) or radio signals emitted from an external antenna 112 (such as multiple antennas at a radio signal station, and an antenna using Bluetooth (registered trademark)) outside the moving body. In addition, in the case of an external antenna 112 whose position can be identified, such as an access point that can emit radio signals over a relatively short distance as a specific example, the moving body may estimate the position of the external antenna 112 (such as an access point) as the position of the moving body itself (moving body).

The moving body may also estimate its own position (moving body) based on measurements made by an on-board inertial sensor 103 mounted on the moving body. The on-board inertial sensor 103 may be, for example, an inertial measurement unit (IMU) including various physical quantity sensors such as a gyro sensor and an acceleration sensor.

In the present disclosure, location information obtained using radio signals may be referred to as radio location information, etc. Similarly, location information obtained using the measurement results of the inertial sensor 103 may be referred to as self-location information, etc.

A moving body (for example, the vehicle 100, etc.) may transmit, for example, position information and image information to the information processing device 1 or the server 200, etc. In this case, the moving body may transmit, for example, the position information and the image information in association with each other to the outside. Here, the association may be, for example, in a manner that allows identification of the position information and the image information acquired during one trip when the moving body moves from the departure point to the destination.
The server 200 may, for example, accumulate position information and image information, and transmit the position information and image information to the information processing device 1 .
In addition, the mobile body and server 200 are not limited to transmitting location information and image information to the information processing device 1 via a communication line, etc., but may store the location information and image information in an external memory and input the location information and image information to the information processing device 1 via the external memory.

The moving body may be, for example, an automobile, a motorcycle, a light vehicle, or a mobile terminal such as a smartphone, a tablet, a smartwatch, or a laptop. In the following, a "vehicle" will be described as an example of the above-mentioned moving body. In addition, the "vehicle" in this disclosure may be synonymous with the above-mentioned moving body or may be a concept that includes a moving body.

The information processing device 1 estimates the reliability of the location information based on, for example, image information and a trained model. The trained model may be, for example, a model generated based on learning a scene recorded in an image and a label assigned to the scene. That is, the information processing device 1 may input image information into the trained model and estimate the reliability of the location information based on the scene recorded in the image information.

The information processing device 1 may correct the position information according to the reliability obtained based on the above-described estimation, for example.
Furthermore, the information processing device 1 may create a road map, for example, based on the position information and the corrected position information.

[Details of information processing device 1]
Next, the information processing device 1 according to an embodiment will be described in detail.
FIG. 2 is a block diagram for explaining the information processing device 1 according to an embodiment.
FIG. 3 is a diagram for explaining an example of a road including an underpass section.

The information processing device 1 includes, for example, a communication unit 21, a storage unit 22, a display unit 23, and a control unit 11. The communication unit 21, the storage unit 22, and the display unit 23 may be an embodiment of the output unit. The control unit 11 includes, for example, an acquisition unit 12, an estimation unit 13 (first estimation unit 131 and second estimation unit 132), a correction unit 14, a creation unit 15, and an output control unit 16. The control unit 11 may be configured, for example, by a calculation processing device of the information processing device 1. The control unit 11 (for example, a calculation processing device, etc.) may realize the functions of each unit (for example, the acquisition unit 12, the estimation unit 13 (first estimation unit 131 and second estimation unit 132), the correction unit 14, the creation unit 15, and the output control unit 16, etc.) by, for example, appropriately reading and executing various programs stored in the storage unit 22, etc.

The communication unit 21 is, for example, a communication interface capable of transmitting and receiving various information to and from devices (external devices) external to the information processing device 1. Here, examples of the external devices may be the vehicle 100, the server 200, and a user terminal (not shown), etc.

The storage unit 22 may store, for example, various information and programs. Examples of the storage unit 22 may be a memory, a solid state drive, a hard disk drive, etc. The storage unit 22 may be, for example, a storage area and a server on the cloud.

The display unit 23 is, for example, a display capable of displaying various characters, symbols, images, etc.

The acquisition unit 12 acquires image information based on an image of the outside captured by the vehicle 100, and position information related to the traveling position of the vehicle 100. The acquisition unit 12 may acquire image information and position information from the outside via the communication unit 21, for example. In this case, the outside may be, for example, the vehicle 100 and the server 200. Alternatively, for example, when the image information and position information are stored in an external memory (not shown), the acquisition unit 12 may acquire the image information and position information from the external memory when the external memory is inserted into an interface (not shown) of the information processing device 1.

The image based on the image information may be, for example, a video or a still image. If the image information is a still image, multiple still images may be generated by capturing images at predetermined timings in the vehicle 100. At predetermined timings may be, for example, at predetermined times.

In this case, the acquisition unit 12 may acquire radio wave location information and self-location information as the location information. As in the above example, the radio wave location information may be location information based on receiving a radio wave signal by the vehicle-mounted receiver 102 (see FIG. 1) of the vehicle 100. The radio wave signal may be, for example, a satellite signal using GNSS, or a signal emitted from an external antenna 112 (for example, multiple antennas such as a radio signal station, a Bluetooth antenna, and an access point). Similarly, as in the above example, the self-location information may be location information acquired based on the measurement results of the vehicle-mounted inertial sensor 103.

The estimation unit 13 may, for example, classify a scene (landscape) based on image information (driving image) captured by the vehicle 100, and estimate the accuracy of the radio wave position information of the shooting position (good or bad reception environment of the radio wave signal). That is, the estimation unit 13 estimates the reliability of the driving position corresponding to the scenery recorded in the image information based on a learned model based on learning the scenery recorded in the image and the label assigned to the scenery. The estimation unit 13 may estimate the reliability of the driving position based on the radio wave position information.

As an example, when the vehicle 100 runs in a city of buildings, inside a tunnel, etc., the reception conditions of radio signals (satellite signals) using GNSS become relatively worse, and it is considered that the accuracy of the position acquired by positioning becomes relatively worse. As a specific example, as shown in Fig. 3, when the vehicle 100 runs on a road 300, when the vehicle 100 runs in a section 301 under an elevated road, the reception conditions of radio signals (satellite signals) using GNSS become worse than when the vehicle 100 runs in a section 302 that is not under an elevated road. It is considered that the reliability of the running position 303 of the vehicle 100 is relatively lower in the section 301 under an elevated road than in the section 302 that is not under an elevated road.
Similarly, as an example, when the vehicle 100 travels along the sea on flat ground, the reception conditions of radio signals (satellite signals) using the GNSS are relatively good, and the accuracy of the position acquired by positioning is considered to be relatively good. In such a case, the reliability of the traveling position of the vehicle 100 is considered to be relatively high.
Therefore, the estimation unit 13 may estimate the reliability of the driving position of the vehicle 100 based on the radio wave location information in accordance with the scenery recorded in the image information based on the image information acquired by the acquisition unit 12 and the learned model.

The trained model may be generated by learning in a learning unit (or a learning unit arranged outside the control unit 11) (not shown) as a function of the control unit 11 of the information processing device 1. Alternatively, the trained model may be generated by learning in a learning device (not shown) or the like that is external to the information processing device 1. When the trained model is generated outside the information processing device 1, the control unit 11 may acquire the trained model from the outside via the communication unit 21 or via an external memory or the like (not shown). The trained model may be stored in, for example, the memory unit 22 or the like.
The estimation unit 13 may acquire the learned model described above.

When classifying a scene (landscape), the estimation unit 13 may, for example, use image recognition (as an example, the deep learning model EfficientNet, etc.) to add annotations such as city streets, mountainous areas, underpasses, tree cover, large vehicles running side by side, etc., to classify the scene (landscape) recorded in the image information acquired by the acquisition unit 12. The estimation unit 13 may, for example, set the position reliability of the scene (landscape) according to the annotations added to each image. As a specific example, the estimation unit 13 may set the position reliability of the estimated scene (landscape) with respect to the running trajectory (running position) of the vehicle 100, and specify a range in which the numerical value of the reliability is equal to or less than a threshold value. In this case, the estimation unit 13 may, for example, visualize the specified range (for example, a section in which the reliability of the running position is relatively low, etc.).

The above-mentioned estimation unit 13 may include, for example, a first estimation unit 131 and a second estimation unit 132 .
The first estimation unit 131 may estimate, for example, by image recognition, the scenery (classification content of the scenery) of the image information acquired by the acquisition unit 12. In this case, the first estimation unit 131 may estimate the classification content of the scenery recorded in the image information acquired by the acquisition unit 12 based on a trained model that has learned the scenery and the classification content of the scenery as a label.
During the learning stage when generating a trained model, one or more labels may be assigned to a single scene.
The classification content of the landscape as a label may be, for example, a feature amount for estimating the landscape, or a feature amount for classifying the landscape. That is, the first estimation unit 131 may estimate the landscape recorded in the image information based on, for example, a trained model that has learned the landscape and the feature amount for estimating the landscape. Alternatively, the first estimation unit 131 may estimate the classification of the landscape recorded in the image information based on a trained model that has learned the landscape and the feature amount for classifying the landscape.

The classification of the scenery estimated by the first estimation unit 131 may be, for example, a classification of the surrounding conditions of the position where the vehicle 100 is traveling, such as whether the scenery recorded in the image information is a mountainous area, a seaside area, whether the area above the vehicle 100 is covered with trees (tree obstruction), whether it is under an overpass, in a city with buildings, or whether a large vehicle is running alongside the vehicle 100 (the vehicle itself). The first estimation unit 131 inputs the image information acquired by the acquisition unit 12 into the learned model and estimates the scenery (classification of the scenery) recorded in the image information. In this case, the first estimation unit 131 may estimate, for example, multiple scenery (classification of the scenery). As an example, the first estimation unit 131 may estimate that the scenery (classification of the scenery) recorded in the image information is a seaside and mountainous area, a mountainous area and obstructed by trees, or an overpass and a city with buildings. Note that the first estimation unit 131 may, for example, estimate three or more landscapes (landscape classifications) rather than estimating two landscapes (landscape classifications) as described above.

For example, when the acquisition unit 12 acquires a video as image information, the first estimation unit 131 (estimation unit 13) may estimate the classification of the scenery recorded in the frames based on all frames constituting the video or a portion of the frames constituting the video and the learned model. When a portion of the frames is used, the first estimation unit 131 (estimation unit 13) may, for example, extract frames from the video at predetermined timings and estimate the classification of the scenery based on the extracted frames and the learned model. The predetermined timings may be, for example, at predetermined times. When a portion of the frames is used, the first estimation unit 131 (estimation unit 13) can reduce the processing load of estimation compared to when all frames constituting the video are used.

For example, even when multiple still images are acquired as image information by the acquisition unit 12, the first estimation unit 131 (estimation unit 13) may estimate the classification content of the scenery recorded in the frame based on all of the multiple still images or some of the multiple still images and the learned model, as in the case of the video described above.

Figure 4 is a diagram illustrating an example of correspondence information.

The second estimation unit 132 may estimate the reliability of the driving position for each image based on the image information, for example. That is, the second estimation unit 132 estimates the reliability of the driving position based on the position information acquired by the acquisition unit 12 based on the classification content of the scenery estimated by the first estimation unit 131. For example, if the scenery (scenery classification) estimated by the first estimation unit 131 is a scene in which the reception conditions of the radio signal are relatively poor, or if there is a possibility that the reception conditions of the radio signal will become relatively poor, the second estimation unit 132 may estimate that the reliability of the driving position based on the radio position information acquired based on the radio signal is relatively low. Alternatively, the second estimation unit 132 may estimate that the reliability of the driving position based on the radio position information acquired based on the radio signal is relatively high, for example, if the scenery (scenery classification) estimated by the first estimation unit 131 is a scene in which the reception conditions of the radio signal are relatively good, or if there is a possibility that the reception conditions of the radio signal will become relatively good.

In this case, the second estimation unit 132 may estimate the reliability of the driving position based on the position information acquired by the acquisition unit 12, based on correspondence information that associates the classification content of the scenery with the reliability of the driving position. The correspondence information may be a relationship between a scenery (classification of the scenery) and the reliability of the driving position when the scenery (classification of the scenery) is obtained.
As shown in an example in Fig. 4, the correspondence information may be information in which the classification content of the scenery is associated with a score indicating the reliability of the classification content. The second estimation unit 132 may estimate a score indicating the reliability of the traveling position based on the correspondence information.

For example, if the scenery (scenery classification) estimated by the first estimation unit 131 is one in which the reception conditions for the radio signal are relatively poor, or if there is a possibility that the reception conditions for the radio signal will become relatively poor, a relatively low value (a value indicating a relatively low reliability) may be associated with that scenery (scenery classification).
Alternatively, for example, a relatively high value (a value indicating a relatively low reliability) may be associated with a landscape (landscape classification) estimated by the first estimation unit 131 when the landscape (landscape classification) is one in which the reception conditions for the radio signal are relatively poor, or when there is a possibility that the reception conditions for the radio signal will become relatively poor.

In other words, a relatively high value (a value indicating a relatively high reliability) may be associated with a landscape (landscape classification) estimated by the first estimation unit 131 when the landscape (landscape classification) is one in which the reception conditions for the radio signal are relatively better, or when there is a possibility that the reception conditions for the radio signal will become relatively better.
Alternatively, for example, a relatively low value (a value indicating a relatively high reliability) may be associated with a landscape (landscape classification) estimated by the first estimation unit 131 when the reception conditions for the radio signal are relatively good, or when there is a possibility that the reception conditions for the radio signal will become relatively good.

Here, the estimation unit 13 may estimate the reliability of a landscape recorded in image information based on a numerical value output from a trained model as a possibility for each of a plurality of landscape classification contents.
As a specific example, when the vehicle 100 is traveling in a mountainous area and the estimation unit 13 uses the learned model to estimate that the scenery recorded in the image information while the vehicle 100 is traveling is a mountainous area with a 100% probability, the estimation unit 13 may use the score recorded in the corresponding information (for example, a score corresponding to a mountainous area (70 points)) as the reliability score of the traveling position (70 points).
As a specific example, when the vehicle 100 is traveling in a mountainous area and the scenery recorded in the image information during the traveling of the vehicle 100 is estimated to be a mountainous area with a probability of 60% using the trained model, the estimation unit 13 may adjust the score recorded in the corresponding information based on the probability of 60%, and set the adjusted score as the score of the reliability of the traveling position. One example of this adjustment may be an adjustment such as multiplying the probability output from the trained model by the score of the classification content of the scenery estimated using the trained model (the score of the corresponding information). In this example, the estimation unit 13 may multiply the probability (60%) by the score (70 points) to calculate 42 points as the score of the reliability of the traveling position.

Note that the estimation unit 13 may estimate the reliability not only as in the above-described example, but also as follows, for example.
That is, the estimation unit 13 may estimate the reliability of the driving position corresponding to the scenery recorded in the image information based on a trained model that has learned the scenery and the reliability of the driving position as a label. The trained model is generated by, for example, learning the scenery and the reliability of the driving position of the vehicle 100 based on the position information (radio wave position information) at the time of the scenery.
During this learning, as in the case of the second estimation unit 132 described above, for example, when the reception conditions for the radio signal are relatively poor in a certain landscape, or when there is a possibility that the reception conditions for the radio signal will become relatively poor, it may be learned that the reliability of the driving position based on the radio position information obtained on the basis of the radio signal is relatively low.
Similarly, during this learning, for example, if the reception conditions for the radio signal are relatively good in a certain landscape, or if there is a possibility that the reception conditions for the radio signal will become relatively good, it may be learned that the reliability of the driving position based on the radio position information obtained based on the radio signal is relatively high.
In this case, the reliability may be expressed, for example, by a score.

The trained model may be generated by learning in a learning unit or a learning device (not shown) or the like, as in the above-described case. The control unit 11 may acquire the trained model by various methods, as in the above-described case. The trained model may be stored in, for example, the storage unit 22 or the like.
The estimation unit 13 may acquire the learned model described above.

For example, when the estimation unit 13 identifies a range in which the numerical value of the position reliability is below a threshold as described above, the correction unit 14 may give a higher priority to the self-position information (e.g., position information using autonomous navigation, etc.) than the radio wave position information in the identified range, or change the weight of the self-position information (as an example, give a higher weight to the self-position information than the radio wave position information) to correct the driving trajectory (driving position). If the position reliability is relatively high, the correction unit 14 may maintain the driving position based on the radio wave position information. If the position reliability is relatively low, the correction unit 14 may correct the driving position based on the radio wave position information.
In this case, the correction unit 14 may correct the traveling trajectory (traveling position) by using, for example, various correction algorithms for the traveling trajectory (traveling position). The correction unit 14 may perform the correction so that the error between the traveling trajectory (traveling position) obtained by integrating the measurement results (for example, the traveling speed of the vehicle 100) of the inertial sensor 103 and the traveling trajectory (traveling position) based on the radio wave position information is minimized. Alternatively, the correction unit 14 may correct the traveling trajectory (traveling position) based on the radio wave position information by using, for example, image tracking and VSLAM (Visual Simultaneous Localization and Mapping) (self-location estimation).

That is, the correction unit 14 may correct the traveling position based on the position information acquired by the acquisition unit 12, based on the reliability of the traveling position estimated by the estimation unit 13. In this case, when the reliability is relatively low, the correction unit 14 may correct the traveling position with the relatively low reliability. The correction unit 14 may, for example, set the position information to be corrected as radio wave position information.
As an example, the correction unit 14 may correct the driving position based on the radio wave position information when the reliability corresponds to a case where the scenery (classification of scenery) corresponds to at least one of under an elevated road, in a city surrounded by buildings, in a mountainous area, and obscured by trees, etc.

The correction unit 14 may correct the traveling position based on the radio wave position information according to the reliability corresponding to various landscapes (landscape classifications), without being limited to the above example.
That is, the correction unit 14 may correct the driving position based on the position information acquired by the acquisition unit 12, based on the relationship between the score of the reliability of the driving position estimated by the second estimation unit 132 and a threshold value, for example. The correction unit 14 may correct the driving position based on the radio wave position information in accordance with a comparison between the score indicating the reliability and a threshold value.

As an example, when a lower score is estimated (a lower score is assigned) when the reliability is low in the estimation unit 13, the correction unit 14 may correct the driving position based on the radio wave position information when the score becomes equal to or below a threshold value or when the score becomes less than the threshold value.
Or, as an example, when a higher score is estimated (a score is assigned) when the reliability is low in the estimation unit 13, the correction unit 14 may correct the driving position based on the radio wave position information when the score becomes equal to or exceeds a threshold value or when the score exceeds the threshold value.

When the estimation unit 13 estimates that the reliability of the driving position is relatively low, the correction unit 14 may correct the driving position with relatively low reliability by using the driving position based on the self-position information. That is, the correction unit 14 may correct, for example, the driving position based on the radio wave position information by using the driving position based on the self-position information. In this case, the correction unit 14 may correct the driving position estimated to have a relatively low reliability or a driving position within a predetermined section including a driving position estimated to have a low reliability by using the driving position based on the self-position information.

As an example, the above-mentioned "traveling position estimated to have a relatively low reliability" may be a traveling position under an overpass, inside a tunnel, or the like.
As another example, the above-mentioned "driving position within a specified section including a driving position estimated to have low reliability" may be a plurality of driving positions within a section including a driving position (first position) that is under an overpass or inside a tunnel, a driving position up to a specified distance before the start point of that driving position (first position), and a driving position up to a specified distance after the end point of that driving position (first position).
As another example, the above-mentioned "driving position in a predetermined section including a driving position estimated to have low reliability" may be a driving position (second position) that is under an overpass or in a tunnel, and multiple driving positions in a link of a road network that includes the driving position (second position). The road network may be, for example, a road map that is represented by nodes that are characteristic points of roads and links that connect adjacent nodes. In this case, the correction unit 14 may use the radio wave position information and the road network to specify which position on the road network the driving position based on the radio wave position information corresponds to, thereby specifying the link that includes the driving position (second position) and multiple driving positions in the link.

For example, the correction unit 14 may, as a correction of the traveling position, replace a traveling position (third position) estimated to have a relatively low reliability, or a traveling position (based on radio wave position information) within a predetermined section including the traveling position (third position) estimated to have a low reliability, with a traveling position based on the self-position information corresponding to the traveling position (third position) or within the traveling section. For example, the correction unit 14 may replace the third position with a traveling position based on the self-position information that is closest to the third position. Alternatively, the correction unit 14 may replace the third position with a traveling position based on the self-position information that corresponds to the closest time from the time when the third position was acquired in the vehicle 100 (for example, the positioning time and the radio signal reception time, etc.).
In this case, the vehicle 100 may include, for example, information on the time when the radio wave position information is acquired (for example, the positioning time and the reception time of the radio wave signal, etc.) in the radio wave position information. Similarly, the vehicle 100 may include, for example, information on the time when the vehicle 100 acquires its own position information (for example, the measurement time of the inertial measurement unit, etc.) in the self-position information.

The creation unit 15 may use the driving trajectory (driving position) corrected by the correction unit 14 to update the point cloud information and the road surface orthoimage, or to estimate the position of a three-dimensional object, etc.

The creation unit 15 may create a road map based on the driving position of the vehicle 100 corrected by the correction unit 14. In this case, the creation unit 15 may update, for example, the point cloud information and road surface orthoimage information used to create a road map or the like so that they correspond to the corrected driving position.

The point cloud information may be, for example, data generated by measurements using a 3D laser scanner mounted on the vehicle 100 or the like. The vehicle 100 acquires the above-mentioned image information and position information when performing measurements using the 3D laser scanner or the like. The information processing device 1 is able to update the measurement position when obtaining the point cloud information so that it corresponds to the corrected driving position.

The road surface orthoimage information may be information on an orthoimage of the road surface, for example. The road surface orthoimage information includes, for example, position information on the position of the road. The information processing device 1 becomes able to update the position recorded in the road surface orthoimage information so that it corresponds to the corrected driving position.

The creation unit 15 creates a road map based on, for example, at least one of the point cloud information and the road surface orthoimage. As an example, the creation unit 15 can create a road map using various information including road position information and feature information (shape, size, and location information of the feature). In addition, the creation unit 15 may create a road map by various methods using the driving position corrected by the correction unit 14 (for example, the trajectory of the driving position), etc., without being limited to the above example.

The output control unit 16 may control the output unit to output at least one selected from, for example, the reliability estimation result by the estimation unit 13, the driving position correction result by the correction unit 14, and a group (first group) of road maps created by the creation unit 15. Here, the output unit may be, for example, the communication unit 21, the storage unit 22, the display unit 23, etc.
That is, the output control unit 16 may control the communication unit 21 to transmit at least one piece of information selected from the above-mentioned first group to an external device, etc. Here, the external device may be, for example, the vehicle 100, a server, a user terminal, etc. The user terminal is, for example, a terminal used by a user of the information processing device 1, and may be, for example, a desktop, a laptop, a tablet, a smartphone, etc.
The output control unit 16 may control the storage unit 22 to store at least one piece of information selected from the above-mentioned first group, for example.
The output control unit 16 may control the display unit 23 to display at least one selected from the above-mentioned first group, for example.

[Information processing method]
Next, an information processing method according to an embodiment will be described.
FIG. 5 is a flowchart illustrating an information processing method according to an embodiment.

In step ST101, the acquisition unit 12 acquires image information based on an image of the outside captured by the vehicle 100, and position information related to the traveling position of the vehicle 100.
The image information may be, for example, information obtained by capturing an image of the moving body in the direction of travel, thereby recording a landscape including the area above the moving body. Alternatively, the image information may be, for example, information obtained by capturing an image of the area above the moving body, thereby recording a landscape including the area above the moving body.
The position information may include, for example, radio wave position information and self-position information. The radio wave position information may be position information based on receiving a radio wave signal by the in-vehicle receiver 102 of the vehicle 100. The radio wave signal may be, for example, a satellite signal using GNSS, or a signal radiated from the external antenna 112. The self-position information may be position information acquired based on the measurement result of the inertial sensor 103.

In step ST102, the estimation unit 13 estimates the reliability of the driving position corresponding to the scenery recorded in the image information acquired in step ST101, based on a trained model based on learning the scenery recorded in the image and the label assigned to the scenery. The estimation unit 13 may estimate the reliability of the driving position based on the radio wave position information.
In this case, the estimation unit 13 (first estimation unit 131) may estimate the classification content of a landscape recorded in the image information acquired by the acquisition unit 12, for example, based on a learned model that has learned the landscape and the classification content of the landscape as a label.

In step ST103, the estimation unit 13 (second estimation unit 132) estimates the reliability of the driving position based on the position information acquired in step ST101 based on the classification content of the scenery estimated in step ST102. In this case, the estimation unit 13 (second estimation unit 132) may estimate the reliability of the driving position based on the position information acquired in step ST101 based on, for example, correspondence information that associates the classification content of the scenery with the reliability of the driving position. The correspondence information may be information that associates the classification content of the scenery with a score indicating the reliability of the classification content. The estimation unit 13 (second estimation unit 132) may estimate the score indicating the reliability of the driving position based on the correspondence information.

Alternatively, as a processing step instead of the above-mentioned steps ST102 and ST103, the estimation unit 13 may estimate the reliability of the driving position according to the scenery recorded in the image information, for example, based on a trained model that has learned the scenery and the reliability of the driving position as a label.

In step ST104, the correction unit 14 may correct the driving position based on the position information acquired in step ST101, based on the reliability of the driving position estimated in step ST103, etc. In this case, when the reliability is relatively low, the correction unit 14 may correct the driving position with a relatively low reliability. As an example, the correction unit 14 may correct the driving position based on the position information acquired in step ST101, based on the relationship between the score of the reliability of the driving position estimated in step ST103, etc. and a threshold value. For example, when the reliability of the driving position estimated in step ST103, etc. is relatively low, the correction unit 14 may correct the driving position with a relatively low reliability by using the driving position based on the self-position information.

In step ST105, the creation unit 15 creates a road map based on the traveling position of the vehicle 100 corrected in step ST104.
[Modification]
In the above-described embodiment, an example has been described in which the control unit 11 includes the acquisition unit 12, the estimation unit 13 (the first estimation unit 131 and the second estimation unit 132), the correction unit 14, the creation unit 15, the output control unit 16, and the like. However, the present disclosure is not limited to this example. That is, the control unit 11 may be configured to include any one of the acquisition unit 12, the estimation unit 13 (the first estimation unit 131 and the second estimation unit 132), the correction unit 14, the creation unit 15, and the output control unit 16.
As an example, the control unit 11 of the information processing device 1 may be configured to include a correction unit 14, a creation unit 15, an output control unit 16, etc. In this case, a computer (on-board device) (e.g., an arithmetic processing device such as an on-board control unit) (not shown) mounted on the vehicle 100 may be configured to include an acquisition unit 12, an estimation unit 13, etc.
Alternatively, as an example, the control unit 11 of the information processing device 1 may be configured to include a second estimation unit 132, a correction unit 14, a creation unit 15, an output control unit 16, etc. In this case, a computer (on-board device) (e.g., a calculation processing device such as an on-board control unit) (not shown) mounted on the vehicle 100 may be configured to include an acquisition unit 12 and a first estimation unit 131.

This allows the vehicle 100 to estimate the classification of the scenery recorded in the image information or the reliability of the location information using an on-board device without sending image information to the information processing device 1. Even in this case, the information processing device 1 obtains location information (e.g., radio location information and self-location information, etc.) from the vehicle 100. That is, by sending the classification of the scenery (determination information that determines the scene at the position where the vehicle 100 is traveling) or reliability information regarding the reliability of the traveling position to the information processing device 1, the vehicle 100 can reduce communication volume, etc., compared to sending image information to the information processing device 1.

The information processing device 1 can estimate the reliability of the driving position based on, for example, the classification content of the scenery acquired from the vehicle 100. Alternatively, the information processing device 1 can correct the driving position based on, for example, the reliability of the driving position acquired from the vehicle 100.

Each of the above-described units of the information processing device 1 may be realized as a function of a processing device of a computer, etc. That is, the acquisition unit 12, the estimation unit 13 (the first estimation unit 131 and the second estimation unit 132), the correction unit 14, the creation unit 15, and the output control unit 16 (the control unit 11) of the information processing device 1 may be realized as an acquisition function, an estimation function (the first estimation function and the second estimation function), a correction function, a creation function, and an output control function (control function) by the processing device of a computer, etc.
The information processing program can cause a computer to realize each of the above-mentioned functions. The information processing program may be recorded in a non-transitory computer-readable recording medium, such as a memory, a solid-state drive, a hard disk drive, or an optical disk. The recording medium may be referred to as a non-transitory computer-readable medium, for example.
As described above, each part of the information processing device 1 may be realized by a computer arithmetic processing device or the like. The arithmetic processing device or the like is configured by, for example, an integrated circuit or the like. Therefore, each part of the information processing device 1 may be realized as a circuit constituting the arithmetic processing device or the like. That is, the acquisition unit 12, the estimation unit 13 (the first estimation unit 131 and the second estimation unit 132), the correction unit 14, the creation unit 15, and the output control unit 16 (the control unit 11) of the information processing device 1 may be realized as an acquisition circuit, an estimation circuit (the first estimation circuit and the second estimation circuit), a correction circuit, a creation circuit, and an output control circuit (a control circuit) constituting the arithmetic processing device or the like of a computer.
The communication unit 21, the storage unit 22, and the display unit 23 (output unit) of the information processing device 1 may be realized as a communication function, a storage function, and a display function (output function) including the functions of an arithmetic processing device, etc. The communication unit 21, the storage unit 22, and the display unit 23 (output unit) of the information processing device 1 may be realized as a communication circuit, a storage circuit, and a display circuit (output circuit) by being configured, for example, by an integrated circuit, etc. The communication unit 21, the storage unit 22, and the display unit 23 (output unit) of the information processing device 1 may be realized as a communication device, a storage device, and a display device (output device) by being configured, for example, by being configured by a plurality of devices.

The information processing device 1 can include one or any combination of the above-mentioned multiple units.
In this disclosure, the term "information" is used, but the term "information" can be replaced with "data" and the term "data" can be replaced with "information."

[Aspects and Effects of the Present Embodiment]
Next, one aspect of this embodiment and the effects of each aspect will be described. Note that each aspect described below is an example at the time of filing, and this embodiment is not limited to the aspects described below. In other words, this embodiment is not limited to each aspect described below, and may be realized by appropriately combining each of the above-mentioned parts. Also, the effects described below are examples, and the effects of each aspect are not limited to those described below. Also, each aspect may, for example, achieve at least one of the effects described below.

(Aspect 1)
An information processing device of one embodiment includes an acquisition unit that acquires image information based on an image of the outside captured by the vehicle and position information regarding the vehicle's driving position, and an estimation unit that estimates the reliability of the driving position corresponding to the scenery recorded in the image information based on a learned model based on learning the scenery recorded in the image and a label assigned to the scenery.
This enables the information processing device to estimate the reliability of the traveling position based on the position information (radio wave position information), for example.
In addition, since the information processing device estimates the reliability of the traveling position based on the image information, it is possible to perform a more accurate estimation. Here, for example, when performing positioning using GNSS or the like, even if the reception strength of the satellite signal is not reduced, when a relatively small number of satellite signals are received, the accuracy of the positioning is relatively reduced. In addition, for example, when a vehicle receives a satellite signal to perform positioning, it may be difficult to determine whether the received satellite signal is a reliable signal for positioning. That is, for example, when a vehicle receives a multipath of a satellite signal, it may be difficult to determine whether it is a multipath, which satellite is transmitted from which satellite (for example, whether the satellite signal is transmitted from a satellite near the zenith, etc.), and whether the satellite signal is reliable for positioning. The above-mentioned concerns regarding the satellite signal can be said to be similar to Bluetooth signals, etc. In this regard, in this embodiment, the information processing device can estimate the reliability of the traveling position more accurately by using image information.
Also, information on the situation of, for example, shielding by trees at a relatively high elevation angle and shielding by buildings at a relatively low elevation angle cannot be obtained from a road map, etc. In contrast, the information processing device can obtain the situation of shielding by trees and buildings, etc. by using image information, and can obtain a more accurate reliability of the driving position according to the road on which the vehicle is traveling and the surrounding conditions.

(Aspect 2)
In one embodiment of the information processing device, the estimation unit includes a first estimation unit that estimates the classification content of a landscape recorded in image information acquired by the acquisition unit based on a learned model that has learned the landscape and the classification content of the landscape as a label, and a second estimation unit that estimates the reliability of a driving position based on the position information acquired by the acquisition unit based on the classification content of the landscape estimated by the first estimation unit.
This allows the information processing device to use the image information to more accurately estimate the reliability of the traveling position.

(Aspect 3)
In one embodiment of the information processing device, the second estimation unit may estimate the reliability of the driving position based on the position information acquired by the acquisition unit, based on correspondence information that associates the classification content of the scenery with the reliability of the driving position.
This allows the information processing device to use the image information to more accurately estimate the reliability of the traveling position.

(Aspect 4)
In one embodiment of an information processing device, the second estimation unit may estimate a score indicating the reliability of the driving position based on corresponding information that associates the classification content of the scenery with a score indicating the reliability of the classification content.
This allows the information processing device to use the image information to more accurately estimate the reliability of the traveling position.

(Aspect 5)
In one aspect of the information processing device, the estimation unit may estimate the reliability of the driving position corresponding to the scenery recorded in the image information based on a learned model that has learned the scenery and the reliability of the driving position as a label.
This allows the information processing device to use the image information to more accurately estimate the reliability of the traveling position.

(Aspect 6)
An information processing device of one embodiment may include a correction unit that corrects the traveling position based on the position information acquired by the acquisition unit based on the relationship between a reliability score of the traveling position estimated by the second estimation unit and a threshold value.
This allows the information processing device to correct the driving position as necessary depending on the magnitude relationship between the reliability score and the threshold value, thereby obtaining a more accurate driving position (for example, a trajectory of the driving position, etc.).

(Aspect 7)
The information processing device of one aspect may include a correction unit that corrects a traveling position based on the position information acquired by the acquisition unit, based on a reliability of the traveling position estimated by the estimation unit.
When the information processing device obtains a relatively more accurate driving position (a driving position based on more accurate radio position information) (for example, when more accurate positioning can be performed), it can maintain higher position accuracy for driving positions with higher reliability by correcting only driving positions with low reliability or driving positions with low reliability.
Furthermore, when correcting a plurality of driving positions (or all driving positions within the predetermined section) within a predetermined section including a driving position with low reliability, the information processing device may not need to estimate reliability at all driving positions based on the radio wave position information. That is, the information processing device can reduce the processing load at the time of estimation by estimating reliability at any of all driving positions based on the radio wave position information, for example, compared to estimating reliability at all driving positions. In this case, the information processing device (estimation unit) may estimate reliability at, for example, every predetermined timing. The predetermined timing may be, for example, every predetermined time, or every predetermined travel distance based on the radio wave position information, etc.
Furthermore, the information processing device can obtain a more accurate running position (for example, a trajectory of the running position) by correcting the running position of the correction target.

(Aspect 8)
In the information processing device of the aspect, when the reliability is relatively low, the correction unit may correct the traveling position having the relatively low reliability.
This allows the information processing device to correct a driving position with a relatively low reliability and maintain a driving position with a relatively high reliability as a more accurate driving position based on radio wave position information.

(Aspect 9)
In one embodiment of an information processing device, the acquisition unit acquires, as position information, radio wave position information based on receiving a radio wave signal by the vehicle's onboard receiver and self-position information based on measurement by the vehicle's onboard inertial sensor, the estimation unit estimates the reliability of the driving position based on the radio wave position information, and the correction unit, when the estimation unit estimates that the reliability of the driving position is relatively low, corrects the driving position with relatively low reliability using the driving position based on the self-position information.
This enables the information processing device to correct the traveling position based on radio wave position information with a relatively low reliability to the traveling position based on self-location information with a relatively high reliability.

(Aspect 10)
The information processing device according to one aspect may further include a creating unit that creates a road map based on the vehicle travel position corrected by the correcting unit.
This allows the information processing device to create a more accurate road map by using the corrected driving position, etc.

(Aspect 11)
In one aspect of the information processing method, a computer executes an acquisition step of acquiring image information based on an image of the outside captured by the vehicle and position information regarding the vehicle's traveling position, and an estimation step of estimating the reliability of the traveling position corresponding to the scenery recorded in the image information based on a learned model based on learning the scenery recorded in the image and the labels assigned to the scenery.
As a result, the information processing method may achieve the same effect as the information processing device of the above-described aspect.

(Aspect 12)
An information processing program of one embodiment provides a computer with an acquisition function that acquires image information based on an image of the outside captured by the vehicle and position information regarding the vehicle's driving position, and an estimation function that estimates the reliability of the driving position corresponding to the scenery recorded in the image information based on a learned model based on learning the scenery recorded in the image and the labels assigned to the scenery.
As a result, the information processing program may achieve the same effect as the information processing device of the above-described aspect.

REFERENCE SIGNS LIST 1 Information processing device 11 Control unit 12 Acquisition unit 13 Estimation unit 131 First estimation unit 132 Second estimation unit 14 Correction unit 15 Creation unit 16 Output control unit 21 Communication unit 22 Memory unit 23 Display unit 100 Vehicle 101 Imaging unit 102 Vehicle-mounted receiver 103 Vehicle-mounted inertial sensor 111 Artificial satellite (GNSS)
112 External antenna 200 Server 300 Road 301 Underpass section 302 Non-underpass section 303 Travel position

Claims (11)

an acquisition unit that acquires image information based on an image of the outside captured by the vehicle and position information related to a traveling position of the vehicle obtained by using a Global Navigation Satellite (GNSS) ;
and an estimation unit that estimates the reliability of the driving position according to the scenery recorded in the image information based on a trained model based on learning the scenery recorded in the image and a label that specifies the quality of the reception environment of the GNSS radio signal that is assigned to the scenery ,
The estimation unit is
a first estimation unit that estimates a classification content of a landscape recorded in image information acquired by the acquisition unit based on the trained model that has learned the landscape and a classification content that classifies, as the label, a quality of a reception environment of a GNSS radio wave signal according to the landscape;
a second estimation unit that estimates a reliability of a traveling position based on the position information acquired by the acquisition unit, based on the classification content of the scenery estimated by the first estimation unit;
Equipped
Information processing device. The second estimation unit estimates the reliability of the traveling position based on the position information acquired by the acquisition unit, based on correspondence information that associates the classification content of the scenery with the reliability of the traveling position.
The information processing device according to claim 1 . The second estimation unit estimates a score indicating the reliability of the traveling position based on information in which a classification content of the scenery is associated with a score indicating the reliability of the classification content as the correspondence information.
The information processing device according to claim 2 . The information processing device according to claim 1 , wherein the estimation unit estimates the reliability of the driving position according to a scenery recorded in the image information based on a trained model that has learned the scenery and the reliability of the driving position as the label . a correction unit that corrects the traveling position based on the position information acquired by the acquisition unit based on a relationship between a score of the reliability of the traveling position estimated by the second estimation unit and a threshold value;
The information processing device according to any one of claims 1 to 4 . A correction unit corrects a traveling position based on the position information acquired by the acquisition unit based on a reliability of the traveling position estimated by the estimation unit.
The information processing device according to any one of claims 1 to 4 . The correction unit corrects the traveling position having a relatively low reliability when the reliability is relatively low.
7. The information processing device according to claim 5 or 6 . The acquisition unit acquires, as the location information, radio wave location information based on receiving a radio wave signal using GNSS with an in-vehicle receiver of the vehicle, and self-location information based on measurement with an inertial sensor of the vehicle,
The estimation unit estimates a reliability of a traveling position based on radio wave position information,
When the estimation unit estimates that the reliability of the traveling position is relatively low, the correction unit corrects the traveling position having the relatively low reliability by using a traveling position based on self-position information.
8. The information processing device according to claim 6 or 7 . A creation unit that creates a road map based on the vehicle travel position corrected by the correction unit.
The information processing device according to any one of claims 5 to 8 . The computer
an acquisition step of acquiring image information based on an image of the outside taken by the vehicle and position information related to a traveling position of the vehicle obtained by using a Global Navigation Satellite (GNSS) ;
and an estimation step of estimating the reliability of the driving position according to the scenery recorded in the image information based on a trained model based on learning the scenery recorded in the image and the label that specifies the quality of the reception environment of the GNSS radio signal that is assigned to the scenery , and
The estimation step includes:
a first estimation step of estimating a classification content of a scenery recorded in image information acquired by the acquisition step based on the trained model that has learned the scenery and a classification content that classifies, as the label, the quality of a reception environment of a GNSS radio wave signal according to the scenery;
a second estimation step of estimating a reliability of a traveling position based on the position information acquired in the acquisition step, based on the classification content of the scenery estimated in the first estimation step;
Run
Information processing methods. On the computer,
An acquisition function for acquiring image information based on an image of the outside of the vehicle and position information related to the traveling position of the vehicle obtained by using GNSS ;
and an estimation function for estimating the reliability of the driving position according to the scenery recorded in the image information based on a trained model based on learning the scenery recorded in the image and the label that specifies the quality of the reception environment of the GNSS radio signal that is assigned to the scenery ,
The estimated function is
A first estimation function that estimates the classification content of a landscape recorded in image information acquired by the acquisition function based on the trained model that has learned the landscape and the classification content that classifies the reception environment of a GNSS radio signal according to the landscape as the label; and
a second estimation function that estimates a reliability of a traveling position based on the position information acquired by the acquisition function, based on the classification content of the scenery estimated by the first estimation function;
Make it happen
Information processing program.

Images