JP7616189B2 - Mobile object control system and mobile object control method - Google Patents

Description

This disclosure relates to technology for controlling a moving object that performs localization processing (self-location estimation processing).

Patent document 1 discloses a self-location estimation device that estimates the self-location of a moving object. The self-location estimation device calculates the self-location estimation accuracy based on the result of the self-location estimation process.

JP 2021-117893 A

Localization processing is known for estimating the position of a moving object. If the accuracy of the localization processing decreases, it is possible to, for example, evacuate the moving object to a safe place. However, because the accuracy of the position information of the moving object has already decreased, control for evacuating the moving object to a safe place is not necessarily executed with high accuracy.

One objective of the present disclosure is to provide technology that can improve the safety of moving objects that perform localization processing.

A first aspect of the present disclosure relates to a mobile object control system for controlling a mobile object.
The mobile control system is
one or more storage devices for storing feature map information indicating the positions of features;
The system includes one or more processors that acquire sensor detection information detected by an external sensor mounted on the moving body and execute a localization process that estimates the position of the moving body based on the sensor detection information and feature map information.
Features present within a first range around the moving object can be recognized based on the sensor detection information and are used for localization processing.
The second range is a range detectable by the external sensor that is farther than the first range.
The expected detection information is sensor detection information that is expected to be detected regarding a feature present in the second range.
The one or more processors may further comprise:
Obtaining expected detection information based on the feature map information;
Acquire actual detection information, which is sensor detection information that is actually detected from the expected detection information;
Based on the actual detection information, estimate whether the accuracy of the future localization process satisfies an acceptable condition;
If the permissible conditions are not met, the moving body is slowed down or stopped.

A second aspect of the present disclosure relates to a moving object control method for controlling a moving object.
The moving object control method includes:
obtaining feature map information indicating the location of the feature;
Acquiring sensor detection information detected by an external sensor mounted on a moving object;
and executing a localization process for estimating the position of the moving object based on the sensor detection information and the feature map information.
Features present within a first range around the moving object can be recognized based on the sensor detection information and are used for localization processing.
The second range is a range detectable by the external sensor that is farther than the first range.
The expected detection information is sensor detection information that is expected to be detected regarding a feature present in the second range.
The moving object control method further includes:
Obtaining expected detection information based on the feature map information;
acquiring actual detection information which is sensor detection information that is actually detected among the expected detection information;
Estimating whether the accuracy of a future localization process satisfies an acceptable condition based on the actual detection information; and
and slowing or stopping the moving body if the permissible conditions are not met.

According to the present disclosure, the accuracy of future localization processing is evaluated in advance. If it is found that the accuracy of future localization processing does not satisfy the acceptable conditions, safety control is executed to slow down or stop the moving body. By executing safety control before the accuracy of the localization processing decreases, it is possible to improve the safety of the moving body.

1 is a block diagram showing an overview of a vehicle control system according to an embodiment; FIG. 1 is a conceptual diagram for explaining a problem associated with localization processing. FIG. 11 is a conceptual diagram for explaining a pre-evaluation process according to the embodiment. FIG. 11 is a conceptual diagram for explaining an example of a pre-evaluation process according to the embodiment. FIG. 11 is a conceptual diagram for explaining another example of the pre-evaluation process according to the embodiment. 1 is a flowchart summarizing a process related to a pre-evaluation process according to an embodiment;

The present disclosure relates to the control of a moving body. Examples of a moving body include a vehicle and a robot. The moving body may be capable of autonomous driving. The vehicle may be an autonomous vehicle. As an example, in the following description, a case where the moving body is a vehicle will be considered. When generalizing, "vehicle" in the following description should be read as "moving body".

1. Overview of the Vehicle Control System 1-1. Configuration Example Fig. 1 is a block diagram showing an overview of a vehicle control system 10 according to the present embodiment. The vehicle control system 10 controls a vehicle 1. The vehicle control system 10 includes an internal sensor 20, an external sensor 30, a Global Navigation Satellite System (GNSS) sensor 40, a driving device 50, and a control device 100.

The internal sensor 20 is mounted on the vehicle 1 and detects the driving state of the vehicle 1. Examples of the internal sensor 20 include a vehicle speed sensor (wheel speed sensor), a steering angle sensor, an acceleration sensor, a yaw rate sensor, etc.

The external sensor 30 is mounted on the vehicle 1 and detects the situation around the vehicle 1. Examples of the external sensor 30 include a LIDAR (Laser Imaging Detection and Ranging), a camera, a radar, etc.

The traveling device 50 includes a steering device, a drive device, and a braking device. The steering device steers the wheels. The drive device generates a driving force. The braking device generates a braking force.

The control device 100 is a computer that controls the vehicle 1. The control device 100 includes one or more processors 110 (hereinafter simply referred to as processor 110) and one or more storage devices 120 (hereinafter simply referred to as storage device 120). The processor 110 executes various processes. For example, the processor 110 includes a CPU (Central Processing Unit). The storage device 120 stores various information. Examples of the storage device 120 include a volatile memory, a non-volatile memory, a HDD (Hard Disk Drive), and an SSD (Solid State Drive). Typically, the control device 100 is mounted on the vehicle 1. Alternatively, a part of the control device 100 may be disposed in an external device outside the vehicle 1 and control the vehicle 1 remotely.

The storage device 120 stores a vehicle control program 210, vehicle driving information 220, sensor detection information 230, map information 240, localization result information 250, etc.

The vehicle control program 210 is a computer program for controlling the vehicle 1. The processor 110 executes the vehicle control program 210 to realize various processes by the control device 100. The vehicle control program 210 may be recorded on a computer-readable recording medium.

The vehicle driving information 220 is information detected by the internal sensor 20 and indicates the driving state of the vehicle 1. Examples of the driving state of the vehicle 1 include the vehicle speed, steering angle, acceleration, yaw rate, etc.

The sensor detection information 230 is information detected by the external sensor 30. For example, the sensor detection information 230 includes point cloud information obtained by a LIDAR. As another example, the sensor detection information 230 may include images captured by a camera.

Furthermore, the sensor detection information 230 includes object information regarding objects around the vehicle 1. Examples of objects around the vehicle 1 include pedestrians, other vehicles, obstacles, feature objects FE, etc. The feature objects FE are objects (landmarks) used in the localization process described below. Examples of feature objects FE include white lines, curbs, poles, telephone poles, signs, signs, building corners, etc. The feature objects FE can also be called "feature amounts." The object information indicates the relative position and relative speed of an object with respect to the vehicle 1. For example, an object can be recognized based on point cloud information obtained by LIDAR, and the relative position and relative speed of the object can be obtained. As another example, an object can be recognized and identified by analyzing an image obtained by a camera, and the relative position of the object can be calculated.

Map information 240 includes a general navigation map. Map information 240 may also indicate lane layouts and road shapes. Furthermore, map information 240 includes "feature object map information MAP-FE." Feature object map information MAP-FE indicates the position (absolute position) of feature object FE in an absolute coordinate system. This feature object map information MAP-FE is used in the localization process described below.

1-2. Localization process (self-position estimation process)
The processor 110 executes a localization process to estimate the position (absolute position) of the vehicle 1. The position of the vehicle 1 will hereinafter be simply referred to as the “vehicle position.” The initial value of the vehicle position is obtained by, for example, the GNSS sensor 40.

The processor 110 acquires the vehicle driving information 220, calculates the amount of movement (displacement) of the vehicle 1 based on the steering angle and vehicle speed of the vehicle 1, and estimates the approximate vehicle position based on the amount of movement. The method of estimating the vehicle position based on the steering angle and vehicle speed is also called "dead reckoning."

The processor 110 also acquires sensor detection information 230 and recognizes (extracts) feature objects FE around the vehicle 1 based on the sensor detection information 230. The object information indicates the relative position of the recognized feature object FE with respect to the vehicle 1. The absolute position of the recognized feature object FE is calculated by combining the vehicle position obtained by dead reckoning with the relative position of the recognized feature object FE. Meanwhile, the absolute position of the feature object FE is registered in the feature object map information MAP-FE. The processor 110 corrects the vehicle position so that the absolute position of the recognized feature object FE and the absolute position of the feature object FE obtained from the feature object map information MAP-FE match as closely as possible. In other words, the processor 110 corrects the vehicle position by comparing the recognition result of the feature object FE by the external sensor 30 with the feature object map information MAP-FE.

By repeatedly performing position estimation using dead reckoning and the above-mentioned position correction, the processor 110 can continuously obtain a highly accurate vehicle position.

Furthermore, the processor 110 may calculate the reliability of the estimated vehicle position, i.e., the reliability of the localization process. Various methods for calculating the reliability have been proposed. For example, a probability density distribution is defined for the vehicle position estimated by the localization process. The smaller the variance of the probability density distribution, the higher the reliability of the localization process. Conversely, the larger the variance of the probability density distribution, the lower the reliability of the localization process.

The localization result information 250 indicates the result of the localization process. Specifically, the localization result information 250 indicates the vehicle position estimated by the localization process. The localization result information 250 may further indicate the reliability of the localization process.

1-3. Vehicle Driving Control The processor 110 executes vehicle driving control for controlling the driving of the vehicle 1. The vehicle driving control includes steering control, acceleration control, and deceleration control. The processor 110 executes vehicle driving control by controlling the driving device 50 (steering device, drive device, braking device).

The processor 110 also executes driving assistance control to assist the driving of the vehicle 1. The driving assistance control automatically performs at least one of steering, acceleration, and deceleration without the need for driving operation by the driver. Examples of such driving assistance control include automatic driving control, risk avoidance control, and lane keeping control. In the driving assistance control, the processor 110 generates a target route (target trajectory) for the vehicle 1 based on the vehicle driving information 220, the sensor detection information 230 (object information), the map information 240, and the localization result information 250 (vehicle position information). For example, the target route is generated so as to follow the center position of the lane. As another example, the target route is generated so as to avoid an obstacle in front of the vehicle 1. Then, the processor 110 executes vehicle driving control so that the vehicle 1 follows the target route.

To ensure the accuracy of driving assistance control, the accuracy of the vehicle position, i.e., the accuracy of the localization process, is important.

2. Description of the Issues Fig. 2 is a conceptual diagram for explaining issues related to localization processing. When the vehicle 1 is at point PA, there are feature objects FE1 to FE4 around the vehicle 1. These feature objects FE1 to FE4 are visible to the external sensor 30 and are recognized based on the sensor detection information 230. The number of recognized feature objects FE1 to FE4 is sufficient for localization processing, and the localization processing is performed with high accuracy based on them.

When the vehicle 1 is at point PB, there are features FE5 to FE8 around the vehicle 1. However, these features FE5 to FE8 are blocked by an obstacle 3 (e.g. a parked vehicle, a newly installed object) and are not visible to the external sensor 30. In other words, the features FE5 to FE8 are not recognized based on the sensor detection information 230. As the number of recognized features FE is insufficient, the accuracy of the localization process decreases. Similarly, if a feature FE registered in the feature map information MAP-FE has been removed and disappeared, the number of recognized features FE will be insufficient and the accuracy of the localization process will decrease.

When the accuracy of the localization process is degraded, it is possible to perform vehicle driving control to evacuate vehicle 1 to a safe place (e.g., the shoulder of the road), for example. However, since the accuracy of the position information of vehicle 1 has already decreased, vehicle driving control to evacuate vehicle 1 to a safe place is not necessarily performed with high accuracy. This is undesirable from the viewpoint of the safety of vehicle 1.

3. Pre-evaluation process According to this embodiment, the processor 110 executes a "pre-evaluation process" that evaluates in advance the accuracy of the localization process in the near future. If the result of the pre-evaluation process indicates that the accuracy of the localization process in the near future does not satisfy the allowable condition, the processor 110 executes a "safety control" that decelerates or stops the vehicle 1 early. The safety control may evacuate the vehicle 1 to a safe place such as a road shoulder and stop it. It is possible to improve the safety of the vehicle 1 by detecting a decrease in the accuracy of the localization process in the future and executing the safety control before the accuracy of the localization process decreases.

The pre-evaluation process according to this embodiment is explained in more detail below.

Figure 3 is a conceptual diagram for explaining the range detectable by the external sensor 30. The detectable range of the external sensor 30 can also be referred to as the visual field range or measurement range. Here, as an example, consider a case where the external sensor 30 is a LIDAR. Distance d2 is the maximum measurement distance of the LIDAR. The range from the vehicle 1 to the distance d2 is the detectable range (measurement range) of the LIDAR. The sensor detection information 230 includes point cloud information obtained by the LIDAR.

The density of the laser light emitted from the LIDAR decreases as the distance from the LIDAR increases. Therefore, the number of point clouds obtained from a certain feature FE (e.g., a signboard) decreases as the distance between the vehicle 1 and the feature FE increases. When the number of point clouds is small, it is possible to see that the laser light is being reflected by something, but that something is not recognized (extracted, clustered) as a single object. In other words, when the number of point clouds is small, the feature FE is not recognized based on the point cloud. As the vehicle 1 approaches the feature FE, the number of point clouds obtained from the feature FE increases. When the number of point clouds reaches a certain value Nth or more, the feature FE is recognized (extracted) with sufficient accuracy based on the point cloud. The distance d1 is the distance between the vehicle 1 and the feature FE, and is the distance at which the number of point clouds becomes Nth. The distance d1 may be different for each type of feature FE. In any case, the distance d1 is smaller than the distance d2 (maximum measurement distance).

If the external sensor 30 is a camera, the "point cloud" in the above description is replaced with "pixels."

The first range R1 is a range up to a distance d1 from the vehicle 1. Features FE present within the first range R1 around the vehicle 1 can be recognized based on the sensor detection information 230. The recognized features FE are then used in the above-mentioned localization process.

The second range R2 is a range of the external sensor 30 that is detectable other than the first range R1. In other words, the second range R2 is a range of the external sensor 30 that is detectable farther than the first range R1. Although some sensor detection information 230 may be obtained for a feature FE that exists in the second range R2, the feature FE is not recognized based on the obtained sensor detection information 230.

The "range definition information RNG" that defines the first range R1 and the second range R2 is prepared in advance and stored in the storage device 120. Different range definition information RNG may be prepared for each type of feature FE. Alternatively, an average range definition information RNG may be prepared for various feature FE.

The "expected detection information EXP" is sensor detection information 230 (e.g., point cloud information) that is expected to be detected for a feature FE that exists in the second range R2. The position of the feature FE is registered in the feature map information MAP-FE. If the feature FE is not occluded by another object, and if the feature FE is not removed, it is expected that some sensor detection information 230 will be obtained for the position of the feature FE indicated in the feature map information MAP-FE. Thus, the processor 110 can obtain the expected detection information EXP based on the vehicle position, the range definition information RNG, and the feature map information MAP-FE.

To efficiently take into account features FE related to the localization process, only the second range R2 in the vicinity of the target route of the vehicle 1 may be considered. The target route of the vehicle 1 is generated in the driving assistance control described above. In this case, the processor 110 can obtain the expected detection information EXP based on the vehicle position, the target route, the range definition information RNG, and the feature map information MAP-FE.

The "actual detection information ACT" is the sensor detection information 230 that is actually detected from the expected detection information EXP related to the second range R2. The processor 110 acquires the actual detection information ACT based on the expected detection information EXP and the actually detected sensor detection information 230. The processor 110 then estimates (determines) whether the accuracy of future localization processing satisfies the acceptable conditions based on at least the actual detection information ACT.

Figure 4 shows an example of a case where the tolerance conditions are met. Four features FE5 to FE8 are present in the second range R2. It is expected that sensor detection information 230 will be obtained from the feature map information MAP-FE regarding the positions of the features FE5 to FE8. The actual detection information ACT indicates that the sensor detection information 230 has actually been obtained from the positions of the features FE5 to FE8. In the future, when the vehicle 1 approaches the vicinity of the features FE5 to FE8, the features FE5 to FE8 will be recognized based on the sensor detection information 230. Because a sufficient number of features FE are available, the processor 110 estimates (determines) that the accuracy of future localization processing will meet the tolerance conditions.

On the other hand, FIG. 5 shows an example of a case where the tolerance conditions are not met. Features FE5 and FE7 are blocked by an obstacle 3 (e.g., a parked vehicle) and are not visible to the external sensor 30. Also, features FE6 and FE8 have been removed and are also not visible to the external sensor 30. Therefore, the actual detection information ACT indicates that sensor detection information 230 has not been obtained from the positions of features FE5 to FE8. In the future, when the vehicle 1 approaches the vicinity of features FE5 to FE8, features FE5 to FE8 will not be recognized based on sensor detection information 230. Because there are insufficient available features FE, the processor 110 estimates (determines) that the accuracy of future localization processing will not meet the tolerance conditions.

If it is estimated that the accuracy of future localization processing will not satisfy the tolerance conditions, the processor 110 interrupts the driving assistance control and executes safety control early. The safety control includes slowing down or stopping the vehicle 1. The safety control may be to stop the vehicle 1 by evacuating it to a safe place such as the shoulder of the road. The processor 110 executes the safety control before the accuracy of the localization processing decreases, that is, when the estimation accuracy of the vehicle position is high. For example, the processor 110 starts the safety control before the vehicle 1 reaches the second range R2. This ensures the safety of the vehicle 1.

To determine whether the accuracy of future localization processes meets acceptable conditions, the processor 110 may perform the following process:

Based on the expected detection information EXP, the processor 110 obtains the expected number f of features FE that are expected to be recognized when the current second range R2 becomes the first range R1 in the future. In other words, based on the expected detection information EXP, the processor 110 obtains the expected number f of features FE that are available for future localization processing.

Furthermore, based on the actual detection information ACT, the processor 110 estimates the number n of features FE that can be recognized when the current second range R2 becomes the first range R1 in the future. In other words, based on the actual detection information ACT, the processor 110 estimates the number n of features FE that can be used in future localization processing. For example, if the number Na of points obtained from the position of the expected feature FE and its vicinity is equal to or greater than a predetermined threshold, the feature FE is determined to be recognizable. As another example, the expected number Ne of points is calculated based on the size of the expected feature FE, and if the ratio of the number Na of points to the expected number Ne of points is equal to or greater than a predetermined threshold, the feature FE is determined to be recognizable. Note that the estimated number n of features FE is equal to or less than the expected number f described above.

There are various possible examples of cases where the permissible conditions are not met, i.e., safety control is executed, as shown below.

[First example] The number of feature objects FE required for localization processing with a certain level of accuracy or higher is defined as m. The required number m may be a fixed value or may differ for each location. The required number m, which differs for each location, is pre-registered in the feature object map information MAP-FE, for example. If the estimated number n of feature objects FE is less than the required number m, it is determined that the tolerance condition is not satisfied. In other words, safety control is executed when the estimated number n is less than the required number m. In that case, deceleration or stopping may be selected depending on the magnitude of the estimated number n. For example, if the required number m is 6, stopping is selected when n<2, and deceleration is selected when 2≦n<5.

[Second Example] The estimated number n of features FE is compared with the expected number f described above. More specifically, the ratio α (=n/f) of the estimated number n to the expected number f is calculated. If the ratio α is less than the threshold, it is determined that the tolerance condition is not satisfied. In other words, safety control is executed when the ratio α is less than the threshold. In this case, deceleration or stopping may be selected depending on the ratio α. For example, if the threshold is 70%, stopping is selected when α<50%, and deceleration is selected when 50%≦α<70%.

[Third example] The estimated number n of feature objects FE is compared with the above-mentioned expected number f. More specifically, the difference β (= f-n) between the expected number f and the estimated number n is calculated. If the difference β is greater than the threshold, it is determined that the tolerance condition is not satisfied. In other words, if the difference β is greater than the threshold, safety control is executed. In that case, deceleration or stopping may be selected depending on the difference β. For example, if the threshold is 1, stopping is selected if β>3, and deceleration is selected if 3≧β>1.

[Fourth Example] The reliability γ of the localization process based on the estimated number n of features FE is calculated. If the reliability γ is less than a threshold, it is determined that the tolerance condition is not satisfied. In other words, if the reliability γ is less than a threshold, safety control is executed. In this case, deceleration or stopping may be selected depending on the reliability γ. If the reliability γ is lower, stopping is selected.

Figure 6 is a flow chart summarizing the processing related to the pre-evaluation processing according to this embodiment. The flow related to the localization processing is executed separately.

In step S100, the processor 110 acquires various information such as sensor detection information 230, feature map information MAP-FE, localization result information 250 (vehicle position information), range definition information RNG, target route information, etc.

In step S110, the processor 110 obtains expected detection information EXP based on the vehicle position information, range definition information RNG, and feature map information MAP-FE.

In step S120, the processor 110 obtains actual detection information ACT based on the expected detection information EXP and the actually detected sensor detection information 230.

In step S130, the processor 110 estimates (determines) whether the accuracy of the future localization process satisfies the acceptable conditions based on at least the actual detection information ACT.

If the accuracy of the future localization process satisfies the acceptable conditions (step S130; Yes), the processor 110 continues the driving assistance control (step S140).

On the other hand, if the accuracy of the future localization process does not meet the acceptable conditions (step S130; No), the processor 110 interrupts the driving assistance control and executes safety control.

As described above, according to this embodiment, the accuracy of the localization process in the near future is evaluated in advance. If it is found that the accuracy of the localization process in the near future does not satisfy the allowable conditions, safety control is executed early. By executing safety control before the accuracy of the localization process deteriorates, it is possible to improve the safety of the vehicle 1.

1...vehicle, 10...vehicle control system, 20...internal sensor, 30...external sensor, 100...control device, 110...processor, 120...storage device, 230...sensor detection information, FE...feature, MAP-FE...feature map information

Claims (5)

A mobile object control system for controlling a mobile object,
one or more storage devices for storing feature map information indicating the positions of features;
and one or more processors that acquire sensor detection information detected by an external sensor mounted on the moving body, and execute a localization process that estimates a position of the moving body based on the sensor detection information and the feature map information;
The feature existing within a first range around the moving object is recognizable based on the sensor detection information and is used for the localization process;
the second range is a range detectable by the external sensor that is farther than the first range,
the expected detection information is the sensor detection information that is expected to be detected regarding the feature that exists in the second range;
The one or more processors further include:
obtaining the expected detection information based on the feature map information;
acquiring actual detection information, which is the sensor detection information that is actually detected, from the expected detection information;
Based on the actual detection information, estimate whether the accuracy of the future localization process satisfies an acceptable condition;
When the permissible condition is not satisfied, the moving object is slowed down or stopped. The mobile object control system according to claim 1,
If the permissible condition is not satisfied, the one or more processors initiate control to decelerate or stop the moving object before the moving object reaches the second range. 3. The mobile object control system according to claim 1,
The one or more processors:
estimating a number of the features to be utilized in future localization processes based on the actual detection information;
and estimating whether the accuracy of the future localization process satisfies the tolerance condition based on the estimated number of features. The mobile object control system according to claim 3,
The one or more processors:
obtaining an expected number of the features available for future localization processes based on the expected detection information;
and estimating whether or not the accuracy of the future localization process satisfies the tolerance condition based on a comparison between the estimated number of features and the expected number of features. A mobile object control method for controlling a mobile object, comprising:
obtaining feature map information indicating the location of the feature;
acquiring sensor detection information detected by an external sensor mounted on the moving object;
executing a localization process for estimating a position of the moving object based on the sensor detection information and the feature map information;
The feature existing within a first range around the moving object is recognizable based on the sensor detection information and is used for the localization process;
the second range is a range detectable by the external sensor that is farther than the first range,
the expected detection information is the sensor detection information that is expected to be detected regarding the feature that exists in the second range;
The moving object control method further comprises:
obtaining the expected detection information based on the feature map information;
acquiring actual detection information which is the sensor detection information actually detected from the expected detection information;
estimating whether a future accuracy of the localization process satisfies an acceptable condition based on the actual detection information;
and slowing down or stopping the moving body when the permissible condition is not satisfied.

Images