JP7597808B2 - Crash monitoring using statistical models. - Google Patents

Description

The present invention relates to collision monitoring using statistical models.

This application claims priority to U.S. Patent Application No. 16/682,971, entitled "COLLISION MONITORING USING STATISTIC MODELS," filed November 13, 2019, and U.S. Patent Application No. 16/683,005, entitled "COLLISION MONITORING USING SYSTEM DATA," filed November 13, 2019, the entire contents of both of which are incorporated herein by reference.

The autonomous vehicle may use an autonomous vehicle controller to guide the autonomous vehicle through an environment. For example, the autonomous vehicle controller may use planning methods, apparatus, and systems to determine a driving path and guide the autonomous vehicle through an environment that includes dynamic objects (e.g., vehicles, pedestrians, animals, etc.) and static objects (e.g., buildings, signs, stranded vehicles, etc.). The autonomous vehicle controller may take into account the predicted behavior of the dynamic objects as the vehicle navigates through the environment.

The detailed description will be described with reference to the accompanying drawings, in which the leftmost digit(s) of a reference number identifies the drawing in which that reference number first appears. Use of the same reference number in different drawings indicates similar or identical components or features.

FIG. 1 illustrates an environment including a vehicle performing crash monitoring using error models and/or system data in accordance with an embodiment of the present disclosure. FIG. 1 illustrates an example of a vehicle analyzing sensor data using an error model to determine an estimated position associated with an object in accordance with an embodiment of the disclosure. FIG. 2 illustrates another example of a vehicle analyzing sensor data using an error model to determine an estimated position associated with an object in accordance with an embodiment of the disclosure. FIG. 1 illustrates an example vehicle analyzing sensor and system data to determine an estimated position associated with an object in accordance with an embodiment of the disclosure. FIG. 2 illustrates another example of a vehicle analyzing sensor and system data to determine an estimated position associated with an object in accordance with an embodiment of the disclosure. FIG. 2 illustrates an exemplary graph showing a vehicle determining a probability of collision in accordance with an embodiment of the present disclosure. FIG. 1 illustrates generating error model data based at least in part on vehicle data and ground truth data in accordance with an embodiment of the present disclosure. FIG. 2 illustrates a computing device that generates perceptual error model data based, at least in part, on log data and ground truth data generated by a vehicle, in accordance with an embodiment of the present disclosure. FIG. 1 illustrates generating uncertainty data based at least in part on vehicle data and ground truth data in accordance with an embodiment of the present disclosure. 1 is a block diagram of an example system for implementing the techniques described herein in accordance with an embodiment of the present disclosure. FIG. 2 illustrates an example process for performing crash monitoring using an error model according to an embodiment of the present disclosure. FIG. 2 illustrates an example process for determining an estimated position associated with an object for using an error model according to an embodiment of the present disclosure. FIG. 1 illustrates an example process for performing collision monitoring with uncertainty according to an embodiment of the present disclosure. FIG. 1 illustrates an example process for performing collision monitoring with uncertainty according to an embodiment of the present disclosure. FIG. 2 illustrates an example process for determining an estimated position associated with an object for using uncertainty in accordance with an embodiment of the present disclosure.

As described above, an autonomous vehicle may use a controller to guide the autonomous vehicle through an environment. For example, the controller may use planning methods, apparatus, and systems to determine a driving path and guide the autonomous vehicle through an environment that includes dynamic objects (e.g., vehicles, pedestrians, animals, etc.) and/or static objects (e.g., buildings, signs, stranded vehicles, etc.). To ensure the safety of occupants and objects, the autonomous vehicle controller may use safety factors when operating in the environment. However, in at least some examples, such systems and controllers may include complex systems that cannot be verified. Despite the fact that there is no method to determine such systems and errors or uncertainties associated with them, such errors and uncertainties may be necessary to inform such vehicles of safe operation in the environment.

As such, the present disclosure is directed to techniques for performing collision monitoring using error models and/or system data by determining an error and/or uncertainty model for a complex system and the system. For example, an autonomous vehicle can use the error model and/or system uncertainty to subsequently determine an estimated position of both the autonomous vehicle and one or more objects. In one example, the estimated position may include a position of a probability distribution associated with the autonomous vehicle and one or more objects. The autonomous vehicle can then determine a probability of a collision between the autonomous vehicle and one or more objects using the estimated position. Based at least in part on the probability of the collision, the autonomous vehicle may perform one or more actions. In at least one example, such probability may be determined based on a determination made according to any of the techniques described in detail herein.

In more detail, the autonomous vehicle can traverse an environment and generate sensor data using one or more sensors. In one example, the sensor data can include data captured by sensors such as time-of-flight sensors, position sensors (e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measurement units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), lidar sensors, radar sensors, sonar sensors, infrared sensors, cameras (e.g., RGB, IR, intensity, depth, etc.), microphone sensors, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), ultrasonic transducers, wheel encoders, etc. The autonomous vehicle can then analyze the sensor data using one or more components (e.g., one or more systems) as it navigates through the environment.

For example, one or more of the components of the autonomous vehicle may use the sensor data to generate a trajectory for the autonomous vehicle. In one example, one or more components may also use the sensor data to determine attitude data associated with the position of the autonomous vehicle. For example, one or more components may use the sensor data to determine position, coordinate, and/or orientation data for the vehicle in the environment. In one example, the attitude data may include x-y-z coordinates and/or may include pitch, roll, and yaw data associated with the vehicle.

Further, one or more components of the autonomous vehicle can use the sensor data to perform operations such as detection, identification, segmentation, classification, and/or tracking of objects in the environment. For example, objects such as pedestrians, bicycles/bicyclists, motorbikes/bikers, buses, trams, trucks, animals, etc. may be present in the environment. One or more components can use the sensor data to determine a current location of the object, as well as an estimated location for the object at a future time point (e.g., 1 second in the future, 5 seconds in the future, etc.).

The autonomous vehicle can then use the trajectory of the autonomous vehicle along with the estimated position of the object to determine the probability of a collision between the autonomous vehicle and the object. For example, the autonomous vehicle can determine whether the estimated position of the object at a future time intersects with the position of the autonomous vehicle along the trajectory at the future time. To increase safety, the autonomous vehicle may use a distance and/or a time buffer when making the determination. For example, the autonomous vehicle can determine that there is a high probability of a collision if the position of the object at a future time is within a threshold distance (e.g., a distance buffer) to the position of the autonomous vehicle.

Additionally, the autonomous vehicle may determine the probability of a collision using an error model associated with a component and/or an uncertainty associated with the output of the component. The error model associated with a component may represent an error and/or an error percentage associated with the output of the component. For example, a perceptual error model may generate a perceptual error associated with a perceptual parameter (e.g., output) of a perceptual component, a predictive error model may generate a predictive error associated with a predictive parameter (e.g., output) from a predictive component, etc. In certain examples, the error may be represented without limitation by a lookup table determined at least in part based on ground truth data, a function (e.g., error based on input parameters), or a statistical set using any other model or data structure that maps an output to a particular error. In at least certain examples, such an error model may map a particular error with a probability/frequency of occurrence. As described, in certain examples, such an error model may be determined for a particular class of data (e.g., a different error model for a perceptual system for detection within a first range of distance and a perceptual system for detection within a second range of distance based on the speed of the vehicle, the speed of the object, etc.).

In one example, the error model may include a static error model. In other examples, the error model may include a dynamic error model that is updated by the autonomous vehicle and/or one or more computing devices. For example, the computing device may continuously receive vehicle data from the autonomous vehicle. The computing device may then update the error model using the vehicle data as well as ground truth data, which is described in more detail below. After updating the error model, the computing device may transmit the updated error model to the autonomous vehicle.

The component can analyze the sensor data and generate an output based at least in part on the analysis, which can represent one or more parameters. The error model can then indicate that an output of the vehicle component, such as a speed associated with an object, is associated with an error percentage. For example, the component can determine that the speed of an object in the environment is 10 meters per second. Using the error model, the autonomous vehicle can determine that the error percentage is X% (e.g., 20%), resulting in a range of speeds +/- X% (e.g., between 8 meters per second and 12 meters per second for an error percentage of 20%). In one example, the range of speeds can be associated with a probability distribution, such as a Gaussian distribution, indicating that some parts of the range are more likely to occur than other parts of the range. In one example, the probability distribution can be binned into various discrete probabilities. For example, 8 meters per second and 12 meters per second may be associated with a 5% probability, 9 meters per second and 11 meters per second may be associated with a 20% probability, and 10 meters per second may be associated with a 45% probability.

To use the error model, the autonomous vehicle can determine an estimated location of the object at a future time based, at least in part, on the outputs from the components and the error models associated with the components. The estimated location may correspond to a probability distribution, such as a Gaussian distribution of locations. In one example, the autonomous vehicle determines the estimated location of the object by first determining a respective probability distribution associated with each of the components and/or parameters. The autonomous vehicle can then determine the estimated location using the probability distributions for all of the components and/or parameters. For example, the autonomous vehicle can aggregate or combine the probability distributions for all of the components and/or parameters to determine the estimated location. Aggregating and/or combining the probability distributions may include multiplying the probability distributions, aggregating the probability distributions, and/or applying one or more other formulas to the probability distributions.

Additionally or alternatively, in one example, the autonomous vehicle may first determine an initial estimated position associated with the object using outputs from the components. The autonomous vehicle then determines a total error of each output using the error models. The autonomous vehicle may determine a total error by aggregating and/or combining the errors from each of the error models for the components. The autonomous vehicle then determines an estimated position using the total error and the initial estimated position. In such an example, the estimated position may include a distribution of possible positions around the initial estimated position.

For the first example, the autonomous vehicle can analyze the sensor data using one or more components to determine parameters associated with the object. The parameters may include, but are not limited to, the type of object, the current location of the object, the speed of the object, the direction of travel of the object, etc. Using the error model, the autonomous vehicle can then determine a probability distribution associated with the type of object, a probability distribution associated with the current location of the object, a probability distribution associated with the speed of the object, a probability distribution associated with the direction of travel of the object, etc. The autonomous vehicle can then determine an estimated location of the object at a future time using the probability distributions for the parameters. In this first example, each (or any one or more) of the estimated locations can be expressed as a probability distribution of locations.

For the second example, the autonomous vehicle can analyze the sensor data using one or more components to determine parameters associated with the object. The autonomous vehicle can then use the parameters to determine an initial estimated position of the object at a future time. Additionally, in one example, the autonomous vehicle can use an error model associated with the parameters to determine a total error and/or error percentage associated with determining the initial estimated position of the object. The autonomous vehicle can then use the initial estimated position and the total error and/or error percentage to determine an estimated position for the object. Again, in this second example, each (or any one or more) of the estimated positions can be expressed as a probability distribution of positions.

In any of the above examples, the autonomous vehicle may use a similar process to determine an estimated position of one or more other objects located in the environment. Additionally, the autonomous vehicle may use a similar process to determine an estimated position of the autonomous vehicle at a future time. For example, the autonomous vehicle may use one or more components to analyze sensor data to determine parameters associated with the autonomous vehicle. The parameters may include, but are not limited to, a position of the autonomous vehicle, a speed of the autonomous vehicle, a direction of travel of the autonomous vehicle, and the like. The autonomous vehicle may then use an error model associated with the parameters to determine an estimated position of the autonomous vehicle at a future time. In one example, each (or any one or more) of the estimated positions may correspond to a probability distribution of positions for the autonomous vehicle at a future time.

In addition to or as an alternative to using an error model to determine an estimated position of the autonomous vehicle and/or object, the autonomous vehicle can determine an estimated position using system data, such as an uncertainty model associated with the component and/or output. The uncertainty model associated with the parameter may correspond to a distribution of how much the output should be trusted and/or a measurement of how accurate the system is in the output. For example, if the component analyzes sensor data multiple times to determine the position of the object, the component will output a low uncertainty if the output includes a small distribution of values (e.g., within a first range) around the position indicated by the ground truth data. Furthermore, the component will output a high uncertainty if the output includes a large distribution of values (e.g., within a second range that is larger than the first range) around the position indicated by the ground truth data. The autonomous vehicle can use the uncertainty model to determine an estimated position of the object at a future time.

For the first example, the autonomous vehicle can analyze sensor data using one or more components to determine parameters associated with the object. The autonomous vehicle can then determine an uncertainty model associated with determining the type of object, an uncertainty model associated with determining the current position of the object, an uncertainty model associated with determining the speed of the object, an uncertainty model associated with determining the direction of travel of the object, etc. The autonomous vehicle can then determine an estimated position of the object at a future time using the uncertainty models associated with the parameters. In this first example, the estimated position may correspond to a probability distribution of positions.

For the second example, the autonomous vehicle can analyze sensor data using one or more components to determine parameters associated with the object. The autonomous vehicle can then determine an initial estimated position of the object at a future time using the parameters. Additionally, the autonomous vehicle can determine an estimated position of the object at a future time using an uncertainty model associated with the parameters and the components that determine the estimated position. Again, in this second example, the estimated position may correspond to a probability distribution of positions.

In any of the above examples, the autonomous vehicle may use a similar process to determine an estimated position of one or more other objects located in the environment. Additionally, the autonomous vehicle may use a similar process to determine an estimated position of the autonomous vehicle at a future time. For example, the autonomous vehicle may use one or more components to analyze sensor data to determine parameters associated with the autonomous vehicle. The parameters may include, but are not limited to, the position of the autonomous vehicle, the speed of the autonomous vehicle, the driving direction of the autonomous vehicle, etc. (any and/or all of which may be derived from an output trajectory from a planner system, for example). The autonomous vehicle may then use an uncertainty model associated with the parameters to determine an estimated position of the autonomous vehicle at a future time. In one example, the estimated position may correspond to a probability distribution of positions for the autonomous vehicle at a future time.

In one example, the autonomous vehicle can then determine a probability of collision using the estimated position of the autonomous vehicle and the estimated position of the object. For example, the probability of collision between the autonomous vehicle and the object may be calculated using an area of geometric overlap between the estimated position of the autonomous vehicle (e.g., a probability distribution of positions) and the estimated position of the object (e.g., a probability distribution of positions). In one example, when there are multiple objects located in the environment, the autonomous vehicle can determine a total probability of collision associated with the autonomous vehicle using the probability of collision determined for each of the objects. For example, the total probability of collision may include a sum of the probabilities of collision for each of the objects.

The autonomous vehicle may then determine whether the probability of a collision is equal to or greater than a threshold (e.g., 0.5%, 1%, 5%, and/or some other threshold percentage). In one example, if the probability of a collision is less than the threshold, the autonomous vehicle may then continue to navigate along the autonomous vehicle's current route. However, in one example, if the autonomous vehicle makes a determination that the probability of a collision is equal to or greater than the threshold, the autonomous vehicle may take one or more actions. For example, the autonomous vehicle may change the speed of the autonomous vehicle (e.g., slow down), may change the route of the autonomous vehicle, may park in a safe location, etc.

Additionally or alternatively, in one example, the autonomous vehicle may determine a total uncertainty associated with navigating the autonomous vehicle based, at least in part, on the uncertainty model used to determine the estimated position of the autonomous vehicle and the uncertainty model used to determine the estimated position of the object. The autonomous vehicle may then generate different routes and perform a similar process to determine a total uncertainty associated with the different routes. Further, the autonomous vehicle may select the route that includes the lowest uncertainty.

The autonomous vehicle and/or one or more computing devices may use the input data (e.g., log data and/or simulation data) to generate an error model and/or an uncertainty model. For example, the autonomous vehicle and/or one or more computing devices may compare the input data to ground truth data. In certain instances, the ground truth data may be manually labeled and/or determined from other verified machine learning components. For example, the input data may include sensor data and/or output data generated by components of the autonomous vehicle. The autonomous vehicle and/or one or more computing devices may compare the input data to the ground truth data, which may indicate actual parameters of objects in the environment. By comparing the input data to the ground truth data, the autonomous vehicle and/or one or more computing devices may determine errors and/or uncertainties associated with the components and/or parameters, and may generate a corresponding error model using the errors and/or a corresponding uncertainty model using the uncertainties.

In one example, the autonomous vehicle and/or one or more computing devices can determine an uncertainty associated with the component. For example, the autonomous vehicle and/or one or more computing devices can input input data to the component at various times to receive various outputs (e.g., parameters) from the component. The autonomous vehicle and/or one or more computing devices can then analyze the outputs and determine a distribution associated with the outputs. Using the distribution, the autonomous vehicle and/or one or more computing devices can determine the uncertainty. For example, if there is a large distribution, the autonomous vehicle and/or one or more computing devices can determine that there is high uncertainty. However, if there is a small distribution, the autonomous vehicle and/or one or more computing devices can determine that there is small uncertainty.

The techniques described herein may be implemented in multiple ways. Exemplary implementations are provided below with reference to the following drawings. Although described in the context of an autonomous vehicle, the methods, apparatus, and systems described herein may be applied to a variety of systems (e.g., sensor systems, or robotic platforms) and are not limited to autonomous vehicles. In another example, the techniques may be utilized in an aviation or marine context, or in any system that uses machine vision (e.g., in a system that uses image data). Additionally, the techniques described herein may be used with real data (e.g., captured using a sensor), simulated data (e.g., generated by a simulator), or any combination of the two.

1 illustrates an environment 100 including a vehicle 102 that performs collision monitoring using error models and/or system data in accordance with an embodiment of the present disclosure. For example, the vehicle 102 can navigate along a trajectory 104 within the environment 100. While navigating, the vehicle 102 can generate sensor data 106 using one or more sensors of the vehicle 102 and can analyze the sensor data 106 using one or more components 108 (e.g., one or more systems) of the vehicle 102. The components 108 may include, but are not limited to, a localization component, a perception component, a prediction component, a planning component, and the like. Based at least in part on the analysis, the vehicle 102 can identify at least a first object 110 and a second object 112 located within the environment 100.

Additionally, the vehicle 102 may analyze the sensor data 106 using the components 108 to determine an estimated location 114 associated with the vehicle 102, an estimated location 116 associated with the first object 110, and an estimated location 118 associated with the second object 112 at a future time. In one example, the estimated location 114 may include a probability distribution of locations associated with the vehicle 102, the estimated location 116 may include a probability distribution of locations associated with the first object 110, and/or the estimated location 118 may include a probability distribution of locations associated with the second object 112.

For example, the estimated location 114 may include an estimated location 120(1) associated with the vehicle 102, a first region (e.g., a first boundary) of the estimated location 120(2) associated with a first probability, a second region (e.g., a second boundary) of the estimated location 120(3) associated with a second probability, and a third region (e.g., a third boundary) of the estimated location 120(4) associated with a third probability. In one example, the first probability is greater than the second probability, which is greater than the third probability. For example, the vehicle 102 may determine that there is a higher probability that the vehicle 102 will be located within the first region of the estimated location 120(2) than within the second region of the approximate location 120(3). Additionally, the vehicle 102 may determine that the probability that the vehicle 102 will be located within a second region of the estimated location 120(3) is higher than within a third region of the estimated location 120(4).

Note that while the example of FIG. 1 illustrates only three separate regions of estimated location in this manner, in other examples there may be any number of regions of estimated location. Furthermore, regions located further away from estimated location 120(1) may have a lower probability than regions located closer to estimated location 120(1). This may be the case for each of the estimated locations of object 110 and object 112, as well.

Further, the estimated location 116 may include an estimated location 122(1) associated with the first object 110, a first region (e.g., a first boundary) of the estimated location 122(2) associated with a first probability, a second region (e.g., a second boundary) of the estimated location 122(3) associated with a second probability, and a third region (e.g., a third boundary) of the estimated location 122(4) associated with a third probability. In one example, the first probability is greater than the second probability, which is greater than the third probability. For example, the vehicle 102 may determine that there is a higher probability that the first object 110 will be located within the first region of the estimated location 122(2) than within the second region of the approximate location 120(3). Additionally, the vehicle 102 may determine that the first object 110 has a higher probability of being located within the second region of the estimated location 122(3) than within the third region of the estimated location 122(4).

Further, the estimated location 118 may include an estimated location 124(1) associated with the second object 112, a first region (e.g., a first boundary) of the estimated location 124(2) associated with a first probability, a second region (e.g., a second boundary) of the estimated location 124(3) associated with a second probability, and a third region (e.g., a third boundary) of the estimated location 120(4) associated with a third probability. In one example, the first probability is greater than the second probability, and the second probability is greater than the third probability. For example, the vehicle 102 may determine that there is a higher probability that the second object 112 will be located within the first region of the estimated location 124(2) than within the second region of the approximate location 124(3). Additionally, the vehicle 102 may determine that the second object 112 is more likely to be located within the second region of the estimated location 124(3) than within the third region of the estimated location 124(4).

In one example, the vehicle 102 can determine the estimated positions 114-118 using an error model 126 associated with the component 108. For example, for a first object 110, the vehicle 102 can analyze the sensor data 106 using the component 108 to determine one or more parameters 128 associated with the first object 110. The parameters 128 may include, but are not limited to, the type of the first object 110, the current position of the first object 110 (and/or the distance to the first object 110), the velocity of the first object 110, and the like. Using the error model 126, the vehicle 102 can then determine the estimated position 116 of the first object 110.

For the first example, the vehicle 102 can use the first error model 126 to determine a probability distribution associated with the type of the first object 110, the second error model 126 to determine a probability distribution associated with the current position of the first object 110, the third error model 126 to determine a probability distribution associated with the speed of the first object 110, and so on. For example, using the speed of the first object 110, the vehicle 102 can determine that the speed of the first object 110 is 1 meter per second. The vehicle 102 can then use the third error model 126 to determine that the error percentage may be X% (e.g., 20%), resulting in a range of speeds (e.g., speeds between 0.8 meters per second and 1.2 meters per second at 20%). In one example, the error model 126 can further indicate that some of the ranges have a higher probability of occurrence than other parts of the ranges. For example, 0.8 meters per second and 1.2 meters per second may be associated with a 5% probability, 0.9 meters per second and 1.1 meters per second may be associated with a 20 percent probability, and 1 meter per second may be associated with a 45% probability. The vehicle 102 may use similar processes to determine the probability distributions of other parameters 128.

The vehicle 102 can then use the probability distribution of the parameters 128 to determine the estimated location 116 of the first object 110. Additionally, the vehicle 102 can use a similar process to determine the parameters 128 for the vehicle 102, determine a probability distribution associated with the parameters 128 for the vehicle 102, and determine the estimated location 114 using the probability distribution. Additionally, the vehicle 102 can use a similar process to determine the parameters 128 for the second object 112, determine a probability distribution associated with the parameters 128 for the second object 112, and determine the estimated location 116 using the probability distribution.

For the second example, the vehicle 102 may use the parameters 128 for the first object 110 to determine an estimated position 122(1) for the first object 110. The vehicle 102 may then use the error model 126 associated with the parameters 128 used to determine the estimated position 122(1) to determine an error for the parameters 128. Using the total error and the estimated position 122(1), the vehicle 102 may determine an estimated position 116 for the first object 110. Additionally, the vehicle 102 may use a similar process to determine an estimated position 114 for the vehicle 102 and an estimated position 118 for the second object 112.

In addition to or as an alternative to using the error model 126 to determine the estimated position 114-118, in another example, the vehicle 102 may use one or more uncertainty models 130 associated with the components 108 and/or the parameters 128. For example, the output from the components 108 may include an uncertainty model 130 associated with determining the parameters 128. For example, the vehicle 102 may determine a first uncertainty model 130 associated with determining a type of the first object 110, a second uncertainty model 130 associated with determining a current position of the first object 110, a third uncertainty model 130 associated with determining a velocity of the first object 110, and so on. The vehicle 102 may then use the parameters 128 and the uncertainty models 130 to determine an estimated position 116 for the first object 110.

For the first example, the vehicle 102 can determine a probability distribution associated with the type of the first object 110 using a first uncertainty model 130, can determine a probability distribution associated with the current position of the first object 110 using a second uncertainty model 130, can determine a probability distribution associated with the speed of the first object 110 using a third uncertainty model 130, and so on. For example, using the speed of the first object 110, the vehicle 102 can determine that the speed of the first object 110 is 1 meter per second. The vehicle 102 can then determine that the uncertainty for the speed of the first object is 20% so that the certainty is 80%. Thus, the vehicle 102 can determine that the range of speeds is between 0.8 meters per second and 1.2 meters per second. In one example, the vehicle 102 can further determine that some of the ranges have a higher probability of occurrence than other parts of the range. For example, 0.8 meters per second and 1.2 meters per second may be associated with a 5% probability, 0.9 meters per second and 1.1 meters per second may be associated with a 20% probability, and 1 meter per second may be associated with a 45% probability. The vehicle 102 may use similar processes to determine the probability distributions of other parameters 128.

The vehicle 102 can then use the probability distribution of the parameters 128 to determine the estimated location 116 of the first object 110. Additionally, the vehicle 102 can use a similar process to determine the parameters 128 for the vehicle 102, determine a probability distribution associated with the parameters 128 for the vehicle 102, and determine the estimated location 114 using the probability distribution. Additionally, the vehicle 102 can use a similar process to determine the parameters 128 for the second object 112, determine a probability distribution associated with the parameters 128 for the second object 112, and determine the estimated location 116 using the probability distribution.

For the second example, the vehicle 102 may use the parameters 128 for the first object 110 to determine an estimated position 122(1) for the first object 110. The vehicle 102 may then use the uncertainty model 130 associated with the parameters 128 to determine a total uncertainty associated with the estimated position 122(1). Using the total uncertainty, the vehicle 102 may determine an estimated position 116 for the first object 110. Additionally, the vehicle 102 may use a similar process to determine an estimated position 114 for the vehicle 102 and an estimated position 118 for the second object 112.

In any of the above examples, after determining the estimated positions 114-118, the vehicle 102 can determine a probability of collision using the estimated positions 114-118. For example, the vehicle 102 can determine a probability of collision between the vehicle 102 and the first object 110. In one example, the vehicle 102 can determine a probability of collision using at least an area of geometric overlap between the estimated position 114 of the vehicle 102 and the estimated position 116 of the first object 110.

More specifically, the estimated position 114 of the vehicle 102 may be Gaussian distributed with parameters μ _v , σ _{v ,} which may be represented by N(μ _v , σ _v ² ). Furthermore, the estimated position 116 of the first object 110 may be Gaussian distributed with parameters μ ₀ , σ ₀ , which may be represented by N(μ ₀ , σ ₀ ² ). The probability of overlap between the estimated position 114 and the estimated position 116 may then be transformed to P[x=0], where x belongs to N(μ _v - σ ₀ ² + σ _v ² ). This may represent a one-dimensional problem associated with determining the probability of overlap.

In one example, the vehicle 102 may perform a similar process to expand the one-dimensional problem to a two-dimensional problem. Additionally, the vehicle 102 may perform a similar process to determine the probability of a collision between the vehicle 102 and the second object 112. In one example, the vehicle 102 may then determine a total probability of collision using the probability of a collision between the vehicle 102 and the first object 110 and the probability of a collision between the vehicle 102 and the second object 112. However, in the example of FIG. 1, the probability of a collision between the vehicle 102 and the second object 112 may be zero because there is no geometric overlap between the estimated position 114 and the estimated position 118.

The vehicle 102 may then determine whether the probability of a collision is equal to or greater than a threshold value. Based at least in part on a determination that the probability of a collision is less than the threshold value, the vehicle 102 may continue to navigate along the trajectory 104. However, based at least in part on a determination that the probability of a collision is equal to or greater than the threshold value, the vehicle 102 may take one or more actions. The one or more actions may include, but are not limited to, navigating along a new trajectory, changing speed (e.g., slowing down), parking, etc.

Note that in one example, vehicle 102 may perform a similar process to determine the probability of a collision between object 110 and object 112. Vehicle 102 may then perform one or more actions based, at least in part, on the probability of the collision. For example, if vehicle 102 determines that the probability of a collision between object 110 and object 112 is equal to or greater than a threshold, vehicle 102 may stop.

2 illustrates an example of a vehicle 102 analyzing sensor data 106 using an error model 126 to determine an estimated position associated with an object, according to an embodiment of the present disclosure. For example, a sensor system 202 of the vehicle 102 can generate sensor data 106. The sensor data 106 can then be analyzed by components 108 of the vehicle 102. In the example of FIG. 2, the components 108 can include a localization component 204, a perception component 206, a planning component 208, and a prediction component 210. However, in other examples, the vehicle 102 may not include one or more of the localization component 204, the perception component 206, the planning component 208, or the prediction component 210. Additionally or alternatively, in some examples, the vehicle 102 may include one or more additional components.

One or more of the components 204-210 may then analyze the sensor data 106 and generate outputs 212-218 based at least in part on the analysis. In one example, the outputs 212-218 may include parameters associated with the vehicle 102 and/or objects. For a first example, the output 212 from the localization component 204 may indicate a location of the vehicle 102. For a second example, the output 214 from the perception component 206 may include detection, segmentation, classification, etc. associated with the object. For a third example, the output 216 from the planning component 208 may include a path for the vehicle 102 to traverse within the environment.

2, it should be noted that one or more of the components 204-210 may use outputs 212-218 from one or more of the other components 204-210 to generate outputs 212-218. For example, the planning component 208 may use the output 212 from the localization component 204 to generate output 216. For another example, the planning component 208 may use the output 214 from the perception component 206 to generate output 216. In addition to or instead of using outputs 212-218 from one or more of the other components 204-210, the components 204-210 may use probability distributions 220-226, which are illustrated below.

The error component 228 may be configured to process the outputs 212-218 using the error model 126 to generate the probability distributions 220-226 associated with the outputs 212-218. In one example, the error component 228 may be included within the components 204-210. For example, the localization component 204 may analyze the sensor data 106 and generate both the output 212 and the probability distribution 220 associated with the output 212 based at least in part on the analysis. For another example, the perception component 206 may analyze the sensor data 106 and generate both the output 214 and the probability distribution 222 associated with the output 214 based at least in part on the analysis.

Probability distributions 220-226 may be associated with outputs 212-218, respectively. For example, error component 228 may process output 212 with error model 126 associated with localization component 204 to generate probability distribution 220. For example, if output 212 indicates a position of vehicle 102, probability distribution 220 may represent an estimated position of vehicle 102 based on a determined position and error represented by error model 126 for localization component 204. Furthermore, error component 228 may process output 214 with error model 126 associated with perception component 206 to generate probability distribution 222. For example, if output 214 indicates a velocity of an object, probability distribution 222 may represent a likely velocity of the object based on a determined velocity and error represented by error model 126 for perception component 206.

The estimation component 230 may be configured to process one or more of the probability distributions 220-226 and/or the sensor data 106 (not shown for clarity reasons) to generate an estimated position 232 associated with the vehicle 102 and/or the object. As described herein, the estimated position 232 may include a probability distribution, such as a Gaussian distribution of positions.

FIG. 3 illustrates an example of a vehicle 102 analyzing sensor data 106 using an error model 126 to determine an estimated position associated with an object in accordance with an embodiment of the present disclosure. In the example of FIG. 3, an estimation component 230 can analyze one or more of the outputs 212-218 from one or more of the components 204-210 to determine an estimated position 302 associated with the vehicle 102 and/or the object. An error component 228 can then use the error model 126 and the estimated position 302 to determine an estimated position 304 of the vehicle 102 and/or the object.

For example, the error component 228 may use the error model 126 to determine a total error and/or a total error percentage associated with the outputs 212-218 of the components 204-210 used to determine the estimated location 302. The error component 228 may then use the total error and/or the total error percentage to generate the estimated location 304. As described herein, the estimated location 304 may include a probability distribution, such as a Gaussian distribution of locations.

FIG. 4 illustrates an example of a vehicle 102 analyzing sensor data 106 using an uncertainty model 130 to determine an estimated position associated with an object according to an embodiment of the present disclosure. For example, an uncertainty component 402 may be configured to process the outputs 212-218 using the uncertainty model 130 to generate probability distributions 404-410 associated with the outputs 212-218. In one example, the uncertainty component 402 may be included within the components 204-210. For example, the localization component 204 may analyze the sensor data 106 and may generate both the output 212 and the probability distribution 404 associated with the output 212 based at least in part on the analysis. For another example, the perception component 206 may analyze the sensor data 106 and may generate both the output 214 and the probability distribution 406 associated with the output 214 based at least in part on the analysis.

4, it should be noted that one or more of the components 204-210 may use the output 212-218 from one or more of the other components 204-210 to generate the output 212-218. For example, the planning component 208 may use the output 212 from the localization component 204 to generate the output 216. For another example, the planning component 208 may use the output 214 from the perception component 206 to generate the output 216. In addition to or instead of using the output 212-218 from one or more other components 204-210, the components 204-210 may use probability distributions 404-410.

Probability distributions 404-410 may be associated with outputs 212-218, respectively. For example, the uncertainty component 402 may process the output 212 using the uncertainty model 130 associated with the localization component 204 to generate the probability distribution 404. For example, if the output 212 indicates a position of the vehicle 102, the probability distribution 404 may represent, at least in part, an estimated position of the vehicle 102 based on the determined position and the uncertainty model 130 for the localization component 204. Furthermore, the uncertainty component 402 may process the output 214 using the uncertainty model 130 associated with the perception component 206 to generate the probability distribution 406. For example, if the output 214 indicates a velocity of an object, the probability distribution 406 may represent a likely velocity of the object based on the determined velocity and the uncertainty model 130 for the perception component 206.

The estimation component 230 may be configured to process one or more of the probability distributions 404-410 and/or the sensor data 106 (not shown for clarity reasons) to generate an estimated position 412 associated with the vehicle 102 and/or the object. As described herein, the estimated position 412 may include a probability distribution, such as a Gaussian distribution of positions.

FIG. 5 illustrates an example of a vehicle 102 analyzing sensor data 106 using an uncertainty model 130 to determine an estimated position associated with an object in accordance with an embodiment of the present disclosure. In the example of FIG. 5, an estimation component 230 can analyze one or more of the outputs 212-218 from one or more of the components 204-210 to determine an estimated position 302 associated with the vehicle 102 and/or the object. An uncertainty component 402 can then use the uncertainty model 130 and the estimated position 302 to determine an estimated position 502 of the vehicle 102 and/or the object.

For example, the uncertainty component 402 may use the uncertainty model 130 for the components 204-210 to determine a total uncertainty associated with the outputs 212-218 of the components 204-210 used to determine the estimated position 302. The uncertainty component 402 may then use the total uncertainty to generate the estimated position 502. As described herein, the estimated position 502 may include a probability distribution, such as a Gaussian distribution of positions.

6 illustrates an example graph 600 illustrating a vehicle 102 determining a probability of collision over a period of time in accordance with an embodiment of the present disclosure. As shown, the graph 600 represents probability 602 along the y-axis and time points 604 along the x-axis. In the example of FIG. 6, the vehicle 102 may determine a probability of collision at time point 606(1). For example, at time point 606(1), the vehicle 102 may determine a probability of collision at three future time points: time point 606(2), time point 606(3), and time point 606(4). In some examples, the probability of collision is associated with the vehicle 102 and one object. In other examples, the probability of collision is associated with the vehicle 102 and more than one object.

As shown, the vehicle 102 may determine a first probability 608(1) of collision at time 606(2), a second probability 608(2) of collision at time 606(3), and no probability of collision at time 606(4). The first probability of collision 608(1) may be associated with a low risk and the second probability of collision 608(2) may be associated with a high risk, such that there is no probability of collision at time 606(4) and therefore no risk of collision at time 606(4). In one example, the first probability of collision 608(1) may be a low risk based, at least in part, on the first probability of collision 608(1) being less than a threshold probability. Additionally, the second probability of collision 608(2) may be a high risk based, at least in part, on the second probability of collision 608(2) being greater than a threshold probability.

Although the example in FIG. 6 illustrates determining the probability of a collision at discrete times, in some examples, the vehicle 102 may continuously determine the probability of a collision on an ongoing basis.

7 illustrates an example 700 of generating error model data based at least in part on vehicle data and ground truth data in accordance with an embodiment of the present disclosure. As depicted in FIG. 7, a vehicle 702 can generate vehicle data 704 and transmit the vehicle data 704 to an error model component 706. As described herein, the error model component 706 can determine an error model 126 that can indicate errors associated with parameters. For example, the vehicle data 704 can be data associated with components of the vehicle 702, such as the perception component 206, the planning component 208, the localization component 204, the estimation component 230, etc. By way of example and not limitation, the vehicle data 704 can be associated with the perception component 206, and the vehicle data 704 can include bounding boxes associated with objects detected by the vehicle 702 in the environment.

The error model component 706 can receive ground truth data 708, which may be determined from manual labeling and/or other authenticated machine learning components. By way of example and without limitation, the ground truth data 708 can include authenticated bounding boxes associated with objects in the environment. By comparing the bounding boxes of the vehicle data 704 to the bounding boxes of the ground truth data 708, the error model component 706 can determine an error associated with a system (e.g., component) of the vehicle 702. Such an error can include, for example, a difference between the ground truth and the output, a percentage difference, an error rate, etc. In one example, the vehicle data 704 can include one or more features (also referred to as parameters) associated with the detected entity and/or the environment in which the entity is located. In one example, the features associated with the entity can include, but are not limited to, an x-position (global position), a y-position (global position), a z-position (global position), an orientation, a type of entity (e.g., classification, etc.), a velocity of the entity, a range (size) of the entity, etc. Features associated with the environment may include, but are not limited to, the presence of another entity in the environment, the state of another entity in the environment, time of day, day of the week, season, weather conditions, indications of darkness/light, etc. Thus, the error may be associated with other features (e.g., environmental parameters). In at least some examples, such error models may be determined for various groupings of parameters (e.g., distinct models for different combinations of classification, distance, speed, etc.). In at least some examples, such parameters may further include, but are not limited to, environmental information such as number of objects, time of day, time of year, weather conditions, etc.

The error model component 706 can process the vehicle data 704 and the ground truth data 708 to determine the error model data 710. The error model data 710 can include errors calculated by the error model component 706, which can be represented as errors 712(1)-(3). Additionally, the error model component 706 can determine probabilities, represented as probabilities 714(1)-(3), associated with the errors 712(1)-(3) and associated with environmental parameters to provide the error models 716(1)-(3), which can represent the error model 126. By way of example and not limitation, the vehicle data 704 can include a bounding box associated with an object at a distance of 50 meters from the vehicle 702 in an environment including rainfall. The ground truth data 708 can provide a verified bounding box associated with the object. The error model component 706 can determine the error model data 710 that determines that an error is associated with the perception system of the vehicle 702. A distance of 50 meters and rainfall can be used as environmental parameters to determine which of the error models 716(1)-(3) to use. Once the error models are identified, the error models 716(1)-(3) can provide errors 712(1)-(3) based on probabilities 714(1)-(3), where errors 712(1)-(3) associated with higher probabilities 714(1)-(3) can be selected over errors 712(1)-(3) associated with lower probabilities 714(1)-(3).

FIG. 8 illustrates an example 800 of a vehicle 702 generating vehicle data 704 and transmitting the vehicle data 704 to a computing device 802 in accordance with an embodiment of the present disclosure. As described above, the error model component 706 can determine a perceptual error model that can indicate an error associated with a parameter. As described above, the vehicle data 704 can include sensor data generated by a sensor of the vehicle 702 and/or perceptual data generated by a perception system of the vehicle 702. The perceptual error model can be determined by comparing the vehicle data 704 to ground truth data 708. The ground truth data 708 can be manually labeled, associated with the environment, and represent known results. Thus, deviations in the vehicle data 704 from the ground truth data 708 can be identified as errors in the sensor system and/or perception system of the vehicle 702. By way of example and not limitation, the perception system can identify an object as a bicyclist, where the ground truth data 708 indicates that the object is a pedestrian. By way of another example, and not by way of limitation, a sensor system may generate sensor data that represents an object as having a width of 2 meters, where the ground truth data 708 indicates that the object has a width of 3 meters.

As described above, the error model component 706 can determine a classification associated with an object represented in the vehicle data 704 and can determine other objects of the same classification in the vehicle data 704 and/or other log data. The error model component 706 can then determine a probability distribution associated with a range of error associated with the object. Based on the comparison and the range of error, the error model component 706 can determine an estimated location 502.

As depicted in FIG. 8, an environment 804 may include objects 806(1)-(3) represented as bounding boxes generated by a perception system. Perceptual error model data 808 may indicate situation parameters as 810(1)-(3) and errors associated with the situation parameters as 812(1)-(3).

9 illustrates an example 900 of generating uncertainty data based at least in part on vehicle data and ground truth data in accordance with an embodiment of the present disclosure. As depicted in FIG. 9, a vehicle 702 can generate vehicle data 704 and transmit the vehicle data 704 to an uncertainty model component 902. As described herein, the uncertainty model component 902 can determine uncertainties associated with component decision parameters. For example, the vehicle data 704 can be data associated with components of the vehicle 702, such as the perception component 206, the planning component 208, the localization component 204, the prediction component 210, etc. By way of example and not limitation, the vehicle data 704 can be associated with the perception component 206, and the vehicle data 704 can include bounding boxes associated with objects detected by the vehicle 702 in the environment.

The uncertainty model component 902 can receive ground truth data 708, which may be determined from manual labeling and/or other verified machine learning components. By way of example and not limitation, the ground truth data 708 may include verified bounding boxes associated with objects in the environment. By comparing the vehicle data 704 to the ground truth data 708, the uncertainty model component 902 can determine the consistency for which the systems (e.g., components) of the vehicle 702 determine the ground truth. For example, the consistency can indicate the percentage for which the parameters represented by the vehicle data 704 are identical to the parameters represented by the ground truth data 708.

The uncertainty model component 902 can then use the consistency to generate uncertainty data 904 that is associated with the component determining the parameters and/or associated with the component determining the parameters. For example, if the consistency indicates that there is a low percentage, the uncertainty data 904 can indicate high uncertainty. However, if the data consistency indicates that there is a high percentage, the uncertainty data 904 can indicate low uncertainty.

More specifically, the uncertainty model component 902 can identify one or more types of uncertainty. The types of uncertainty can include, but are not limited to, epistemic uncertainty, aleatoric uncertainty (e.g., data-dependent, task-dependent, etc.), and the like. Epistemic uncertainty can be associated with ignorance about the data generated by the component. Aleatoric uncertainty can be associated with uncertainty about information that the data cannot explain. The uncertainty model component 902 can then use the identified uncertainties to generate the uncertainty model 130.

In one example, the uncertainty model component 902 can input data into the component multiple times, where one or more nodes of the component are changed when inputting the data, which causes the output of the component to differ. This can cause the output from the component to have a range. In one example, the component can further output the mean and/or variance of the output. The uncertainty model component 902 can generate an uncertainty model 130 for the component and/or type of output (e.g., parameter) with a distribution associated with the range, mean, and/or variance of the output.

FIG. 10 illustrates a block diagram of an example system 1000 for implementing the techniques described herein. In at least one example, the system 1000 may include a vehicle 102. In the illustrated example 1000, the vehicle 102 is an autonomous vehicle, but the vehicle 102 may be any other vehicle type (e.g., a driver-controlled vehicle that may provide an indication of whether it is safe to perform various operations).

The vehicle 102 may include a computing device 1002, one or more sensor systems 202, one or more emitters 1006 (also referred to as a communication device and/or modem), at least one direct connection 1008 (e.g., to physically couple to the vehicle 102 to exchange data and/or provide power), and one or more drive systems 1010. The one or more sensor systems 202 may be configured to capture sensor data 106 associated with the environment.

The sensor system 202 may include time-of-flight sensors, position sensors (e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measurement unit (IMU), accelerometer, magnetometer, gyroscope, etc.), lidar sensors, radar sensors, sonar sensors, infrared sensors, cameras (e.g., RGB, IR, intensity, depth, etc.), microphone sensors, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), ultrasonic transducers, wheel encoders, etc. The sensor system 202 may include various instantiations of each of these or other types of sensors. For example, the time-of-flight sensors may include individual time-of-flight sensors positioned at the corners, front, rear, sides, and/or top of the vehicle 102. As another example, the camera sensors may include various cameras positioned at various locations with respect to the exterior and/or interior of the vehicle 102. The sensor system 202 may provide input to the computing device 1002.

The vehicle 102 may also include one or more emitters 1004 for emitting light and/or sound. In this example, the one or more emitters 1004 include interior audio and visual emitters for communicating with occupants of the vehicle 102. By way of example and not by way of limitation, the interior emitters may include speakers, lights, signs, display screens, touch screens, haptic emitters (e.g., vibration and/or force feedback), mechanical actuators (e.g., seat belt tensioners, seat positioners, head rest positioners, etc.), and the like. In this example, the one or more emitters 1004 also include exterior emitters. By way of example and not by way of limitation, the exterior emitters in this example include lights that signal driving direction or other indicators of the vehicle's actions (e.g., indicator lights, signs, lighting arrays, etc.), as well as one or more audio emitters (e.g., speakers, speaker arrays, horns, etc.) that acoustically communicate with pedestrians or one or more other nearby vehicles, including acoustic beam steering technology.

The vehicle 102 may also include one or more communication connections 1006 that enable communication between the vehicle 102 and one or more other local or remote computing devices (e.g., remote teleoperated computing devices) or remote services. For example, the communication connections 1006 may facilitate communication with other computing devices local to the vehicle 102 and/or the drive system 1010. The communication connections 1006 may also enable the vehicle 102 to communicate with other nearby computing devices (e.g., other nearby vehicles, traffic signals, etc.).

The communications connection 1006 may include physical and/or logical interfaces for connecting the computing device 1002 to another computing device or to one or more external networks 1012 (e.g., the Internet). For example, the communications connection 1006 may enable Wi-Fi-based communications, such as over frequencies defined by the IEEE 802.11 standard, short-range radio frequencies such as Bluetooth, cellular communications (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), satellite communications, dedicated short-range communications (DSRC), or any suitable wired or wireless communications protocol that enables each computing device to interface with other computing devices. In at least some examples, the communications connection 1006 may include one or more modems, as described in detail above.

In at least one example, the vehicle 102 may include one or more drive systems 1010. In some examples, the vehicle 102 may have one drive system 1010. In at least one example, when the vehicle 102 has multiple drive systems 1010, the individual drive systems 1010 may be located at both ends of the vehicle 102 (e.g., the front and rear, etc.). In at least one example, the drive system 1010 may include one or more sensor systems 202 for detecting conditions surrounding the drive system 1010 and/or the vehicle 102. By way of example and not by way of limitation, the sensor systems 202 may include one or more wheel encoders (e.g., rotary encoders) for sensing the rotation of the wheels of the drive system, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) for measuring the orientation and acceleration of the drive system, cameras or other imaging sensors, ultrasonic sensors for acoustically detecting objects in the conditions surrounding the drive system, lidar sensors, radar sensors, etc. Some sensors, such as wheel encoders, may be unique to the drive system 1010. In some examples, the sensor system 202 for the drive system 1010 may overlap or complement a corresponding system (e.g., sensor system 506) of the vehicle 102.

The drive system 1010 may include many of the vehicle systems, including a high voltage battery, a motor for propelling the vehicle, an inverter for converting direct current from the battery to alternating current for use by other vehicle systems, a steering system including a steering motor and a steering rack (which may be electric), a brake system including hydraulic or electric actuators, a suspension system including hydraulic and/or pneumatic components, a stability control system for distributing braking force to mitigate loss of traction and maintain control, an HVAC system, lighting (e.g., lighting such as headlights/taillights for illuminating the perimeter of the vehicle), and one or more other systems (e.g., cooling systems, safety systems, on-board charging systems, other electrical components such as DC/DC converters, high voltage junctions, high voltage cables, charging systems, charging ports, etc.). Additionally, the drive system 1010 may include a drive system controller for controlling the operation of various vehicle systems that may receive and pre-process data from the sensor system 202. In one example, the drive system controller may include one or more processors and a memory communicatively coupled to the one or more processors. The memory can store one or more modules to perform various functions of the drive module 1010. In addition, the drive system 1010 also includes one or more communication connections that enable each drive system to communicate with one or more other local or remote computing devices.

The computing device 1002 may include one or more processors 1014 and a memory 1016 communicatively coupled to the processor 1014. In the illustrated example, the memory 1016 of the computing device 1002 stores the localization component 204, the perception component 206, the prediction component 210, the estimation component 230, the planning component 208, the error component 228, the uncertainty component 402, and one or more sensor systems 202. Although depicted as residing in the memory 1016 for illustrative purposes, it is contemplated that the localization component 204, the perception component 206, the prediction component 210, the estimation component 230, the planning component 208, the error component 228, the uncertainty component 402, and one or more system controllers 1018 may additionally or alternatively be accessible to the computing device 1002 (e.g., stored in a different component of the vehicle 102) and/or accessible to the vehicle 102 (e.g., stored remotely).

In the memory 1016 of the computing device 1002, the localization component 204 can include functionality for receiving data from the sensor system 202 to determine the position of the vehicle 102. For example, the localization component 204 can include and/or request/receive a three-dimensional map of the environment and continuously determine the position of the autonomous vehicle within the map. In one example, the localization component 204 can use simultaneous localization and mapping (SLAM) or calibration, localization and mapping, simultaneously (CLAMS) to receive time-of-flight data, image data, lidar data, radar data, sonar data, IMU data, GPS data, wheel encoder data, or any combination thereof, etc., to precisely determine the position of the autonomous vehicle. In one example, the localization component 204 can provide data to various components of the vehicle 102 to determine the initial position of the autonomous vehicle to generate a trajectory, as described herein.

The perception component 206 may include functionality for performing object detection, segmentation, and/or classification. In one example, the perception component 206 may provide processed sensor data indicative of the presence of an entity proximate to the vehicle 102 and/or the classification of the entity as a type of entity (e.g., automobile, pedestrian, bicyclist, building, tree, road surface, curb, sidewalk, unknown, etc.). In additional and/or alternative examples, the perception component 206 may provide processed sensor data indicative of one or more features (also referred to as parameters) associated with the detected entity and/or the environment in which the entity is located. In one example, the features associated with the entity may include, but are not limited to, x-position (global location), y-position (global location), z-position (global location), direction, type of entity (e.g., classification, etc.), velocity of the entity, range (size) of the entity, etc. Features associated with the environment may include, but are not limited to, the presence of another entity in the environment, the state of another entity in the environment, time of day, day of the week, season, weather conditions, geographic location, darkness/lightness indication, etc.

The perception component 206 may include functionality for storing the perception data generated by the perception component 206. In one example, the perception component 206 may determine a track corresponding to an object classified as a type of object. For purposes of example only, the perception component 206 using the sensor system 202 may capture one or more images of an environment. The sensor system 202 may capture an image of the environment including an object such as a pedestrian. The pedestrian may be in a first position at time T and in a second position at time T+t (e.g., movement during a span of time t after time T). That is, the pedestrian may move from the first position to the second position during this period of time. Such movement may be recorded, for example, as stored perception data associated with the object.

The stored sensory data may, in one example, include fused sensory data captured by the vehicle. The fused sensory data may include a fusion or other combination of sensor data from sensor systems 202, such as image sensors, lidar sensors, radar sensors, time-of-flight sensors, sonar sensors, global positioning system sensors, interior sensors, and/or any combination thereof. The stored sensory data may additionally or alternatively include classification data including a semantic classification of an object (e.g., pedestrian, vehicle, building, road surface, etc.) represented in the sensor data. The stored sensory data may additionally or alternatively include track data (a collection of historical positions, orientations, sensor characteristics, etc. associated with an object over time) corresponding to the movement of an object classified as a dynamic object through the environment. The track data may include various tracks of various different objects over time. This track data may be mined to identify images of a particular type of object (e.g., pedestrian, animal, etc.) when the object is not moving (e.g., stationary) or is moving (e.g., walking, running, etc.). In this example, the computing device determines a track corresponding to a pedestrian.

The prediction component 210 can generate one or more probability maps that represent predicted probabilities of predicted positions of one or more objects in the environment. For example, the prediction component 210 can generate one or more probability maps for vehicles, pedestrians, animals, etc., that are within a threshold distance from the vehicle 102. In one example, the prediction component 210 can measure the track of the object and generate a discretized predicted probability map, heat map, probability distribution, discretized probability distribution, and/or trajectory for the object based on the observed and predicted behavior. In one example, the one or more probability maps can represent the intent of one or more objects in the environment.

The planning component 208 can determine a path for the vehicle 102 to follow and traverse an environment. For example, the planning component 208 can determine various routes and paths as well as various levels of detail. In one example, the planning component 208 can determine a route to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For purposes of this description, the route may be a series of waypoints for travel between the two locations. As non-limiting examples, the waypoints include roadways, intersections, Global Positioning System (GPS) coordinates, and the like. Additionally, the planning component 208 can generate instructions for guiding the vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning component 208 can determine how to guide the vehicle 102 from a first waypoint in the series of waypoints to a second waypoint in the series of waypoints. In one example, the instructions may be a path or a portion of a path. In one example, the diverse paths may be generated substantially simultaneously (e.g., within technical tolerances) according to a receding horizon technique. One of the diverse paths in the receding data horizon having the highest confidence level may be selected for maneuvering the vehicle.

In other examples, the planning component 208 can alternatively or additionally use data from the perception component 206 and/or the prediction component 210 to determine a path for the vehicle 102 to follow to traverse the environment. For example, the planning component 208 and/or the prediction component 210 can receive data from the perception component 206 regarding objects associated with the environment. Using this data, the planning component 208 can determine a route to travel from a first location (e.g., a current location) to a second location (e.g., a target location) and avoid objects in the environment. In at least some examples, such a planning component 208 can determine that there is no such collision-free path and can then provide a path that brings the vehicle 102 to a safe stop, avoids any collisions, and/or otherwise mitigates damage.

In at least one example, the vehicle computing device 1002 may include one or more system controllers 1018, which may be configured to control steering, propulsion, braking, safety, emitter, communication, and other systems of the vehicle 102. These system controllers 1018 may communicate with and/or control corresponding systems of the drive system 1010 and/or other components of the vehicle 102, which may be configured to operate according to the path provided by the planning component 208.

The vehicle 102 may connect to the computing device 802 via a network 1012 and may include one or more processors 1020 and a memory 1022 communicatively coupled to the one or more processors 820. With at least one exception, the processor 820 may be similar to the processor 1014 and the memory 1022 may be similar to the memory 1016. In the illustrated example, the memory 1022 of the computing device 802 stores the vehicle data 704, the ground truth data 708, and the error model component 706. Although depicted as residing in the memory 1022 for illustrative purposes, it is contemplated that the vehicle data 704, the ground truth data 708, and/or the error model component 706 may additionally or alternatively be accessible to the computing device 802 (e.g., stored in a different component of the computing device 802) and/or accessible to the computing device 802 (e.g., stored remotely).

The processor 1014 of the computing device 1002 and the processor 1020 of the computing device 802 may be any suitable processor capable of processing data and executing instructions to perform operations as described herein. For purposes of illustration and not limitation, the processors 1014 and 1020 may include one or more Central Processing Units (CPUs), Graphics Processing Units (GPUs), or any other device or part of a device that processes electronic data and converts it into registers and/or other electronic data that may be stored in memory. In certain examples, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g., FPGAs, etc.), and other hardware devices may also be considered processors so long as they are configured to implement encoded instructions.

The memory 1016 of the computing device 1002 and the memory 1022 of the computing device 802 are examples of non-transitory computer-readable media. The memories 1016 and 1022 can store an operating system and one or more software applications, instructions, programs, and/or data that implement the methods described herein and the functions attributed to the various systems. In various implementations, the memories 1016 and 1022 can be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile/flash type memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein may include many other logical, programmatic, and physical components, of which those shown in the accompanying drawings are merely examples relevant to the description herein.

In some examples, some or all aspects of the components described herein may include any models, algorithms, and/or machine learning algorithms. For example, in some examples, the components in memories 1016 and 1022 may be implemented as neural networks.

11-14 illustrate exemplary processes according to embodiments of the present disclosure. These processes are illustrated as logical flow graphs in which each operation represents a sequence of operations that may be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more computer-readable media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform a particular function or implement a particular abstract data type. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and/or in parallel to implement a process.

FIG. 11 illustrates an example process 1100 for performing crash monitoring using an error model in accordance with an embodiment of the present disclosure. At operation 1102, the process 1100 may include receiving sensor data generated by one or more sensors. For example, the vehicle 102 may navigate along a route from a first location to a second location. While navigating, the vehicle 102 may generate sensor data using one or more sensors of the vehicle 102.

At operation 1104, the process 1100 may include determining, at least with a first system of the vehicle, at least a parameter associated with the vehicle based, at least in part, on the first portion of the sensor data. For example, the vehicle 102 may analyze the first portion of the sensor data with one or more systems. The one or more systems may include, but are not limited to, a localization system, a perception system, a planning system, a predictive system, etc. Based, at least in part, on the analysis, the vehicle 102 may determine a parameter associated with the vehicle 102. The parameters may include, but are not limited to, a position of the vehicle 102, a speed of the vehicle 102, a driving direction of the vehicle 102, etc.

At operation 1106, process 1100 may include determining an estimated location associated with the vehicle based, at least in part, on the parameters associated with the vehicle and a first error model associated with the first system. For example, vehicle 102 may process at least the parameters associated with vehicle 102 with the first error model. As described herein, the first error model may represent an error and/or an error percentage associated with an output of the first system. Based at least in part on the processing, vehicle 102 may subsequently determine an estimated location associated with vehicle 102. As also described herein, the estimated location may correspond to a probability distribution of locations.

At operation 1108, process 1100 may include at least determining, using at least a second system of the vehicle, a parameter associated with the object based, at least in part, on the second portion of the sensor data. For example, vehicle 102 may analyze the sensor data and, based at least in part on the analysis, may identify the object. Vehicle 102 may then analyze the second portion of the sensor data using one or more systems. Based, at least in part on the analysis, vehicle 102 may determine a parameter associated with the object. The parameters may include, but are not limited to, a type of object, a position of the object, a speed of the object, a direction of travel of the object, etc.

At operation 1110, process 1100 may include determining an estimated position associated with the object based, at least in part, on the parameters associated with the object and a second error model associated with the second system. For example, vehicle 102 may process at least the parameters associated with the object using the second error model. As described herein, the second error model may represent an error and/or an error percentage associated with an output of the second system. Based at least in part on the processing, vehicle 102 may subsequently determine an estimated position associated with the object. As also described herein, the estimated position may correspond to a probability distribution of positions.

At operation 1112, process 1100 may include determining a probability of collision based at least in part on an estimated position associated with the vehicle and an estimated position associated with the object. For example, the vehicle 102 may analyze an estimated position associated with the vehicle 102 and an estimated position associated with the object to determine a probability of collision. In one example, the probability of collision may be based at least in part on an amount of overlap between an estimated position associated with the vehicle 102 and an estimated position associated with the object.

At operation 1114, process 1100 may include determining whether the probability of a collision is equal to or greater than a threshold value. For example, vehicle 102 may compare the probability of a collision to a threshold value to determine whether the probability of a collision is equal to or greater than the threshold value.

If, at operation 1114, it is determined that the probability of a collision is not equal to or greater than the threshold, then, at operation 1116, process 1100 may include causing the vehicle to continue navigating along the path. For example, if the vehicle 102 determines that the probability of a collision is less than the threshold, the vehicle 102 may continue navigating along the path.

However, if at operation 1114 it is determined that the probability of a collision is equal to or greater than the threshold, then at operation 1118 process 1100 may include causing the vehicle to perform one or more actions. For example, if the vehicle 102 determines that the probability of a collision is equal to or greater than the threshold, the vehicle 102 may perform one or more actions. The one or more actions may include, but are not limited to, altering the route of the vehicle 102, altering the speed of the vehicle 102, parking the vehicle 102, etc.

FIG. 12 illustrates an example process 1200 for determining an estimated position associated with an object for use with an error model in accordance with an embodiment of the present disclosure. At operation 1202, the process 1200 may include receiving sensor data generated by one or more sensors. For example, the vehicle 102 may navigate along a path from a first location to a second location. While navigating, the vehicle 102 may generate sensor data using one or more sensors of the vehicle 102.

At operation 1204, process 1200 may include determining, with one or more systems of the vehicle, a first parameter associated with the object based at least in part on the sensor data. For example, vehicle 102 may analyze the sensor data with one or more systems. The one or more systems may include, but are not limited to, a localization system, a perception system, a planning system, a predictive system, etc. Based at least in part on the analysis, vehicle 102 may determine a first parameter associated with the object (e.g., a vehicle or another object). The parameters may include, but are not limited to, a position of the object, a speed of the object, a traveling direction of the object, etc.

At operation 1206, process 1200 may include determining a first probability distribution associated with the first parameter based at least in part on the first error model. For example, vehicle 102 may process at least the first parameter with the first error model. As described herein, the first error model may represent an error and/or an error percentage associated with the first parameter. Based at least in part on the processing, vehicle 102 may determine a first probability distribution associated with the first parameter.

At operation 1208, the process 1200 may include determining, using one or more systems of the vehicle, a second parameter associated with the object based at least in part on at least one of the sensor data or the first probability distribution. For example, the vehicle 102 may use one or more systems to analyze at least one of the sensor data or the first probability distribution. In one example, the vehicle 102 analyzes the first probability distribution where the second parameter is determined using the first parameter. Based at least in part on the analysis, the vehicle 102 may determine a second parameter associated with the object (e.g., a vehicle or another object). The second parameter may include, but is not limited to, a position of the object, a speed of the object, a direction of travel of the object, an estimated position of the object at a future time, etc.

At operation 1210, process 1200 may include determining a second probability distribution associated with the second parameter based at least in part on the second error model. For example, vehicle 102 may process at least the second parameter with the second error model. As described herein, the second error model may represent an error and/or an error percentage associated with the second parameter. Based at least in part on the processing, vehicle 102 may determine a second probability distribution associated with the second parameter.

At operation 1212, the process 1200 may include determining an estimated location associated with the object based at least in part on at least one of the first probability distribution or the second probability distribution. For example, the vehicle 102 may determine the estimated location based at least in part on the first probability distribution and/or the second probability distribution. In one example, the vehicle 102 may determine the estimated location using both the first probability distribution and the second probability distribution when the first parameter and the second parameter are independent, such as the first parameter indicating the current location of the object and the second parameter indicating the velocity of the object. In one example, the vehicle 102 may determine the estimated location using the second probability distribution when the second parameter is determined using the first parameter, such as when the second parameter indicates the estimated location of the object at a future time determined using the first parameter indicating the velocity of the object.

13A-13B illustrate an example process 1300 for performing crash monitoring using an uncertainty model in accordance with an embodiment of the present disclosure. At operation 1302, the process 1300 may include receiving sensor data generated by one or more sensors. For example, the vehicle 102 may navigate along a route from a first location to a second location. While navigating, the vehicle 102 may generate sensor data using one or more sensors of the vehicle 102.

At operation 1304, the process 1300 may include determining, at least with a first system of the vehicle, at least a parameter associated with the vehicle based, at least in part, on the first portion of the sensor data. For example, the vehicle 102 may analyze the first portion of the sensor data with one or more systems. The one or more systems may include, but are not limited to, a localization system, a perception system, a planning system, a predictive system, etc. Based, at least in part, on the analysis, the vehicle 102 may determine a parameter associated with the vehicle 102. The parameters may include, but are not limited to, a position of the vehicle 102, a speed of the vehicle 102, a driving direction of the vehicle 102, etc.

At 1306, process 1300 may include determining a first uncertainty model associated with a first system that determines the parameters associated with the vehicle. For example, vehicle 102 may determine the first uncertainty model. In one example, vehicle 102 determines the first uncertainty model by receiving the first uncertainty model from the first system. In one example, vehicle 102 determines the first uncertainty model using uncertainty data indicative of uncertainties associated with the first system that determines the first parameters.

At operation 1308, process 1300 may include determining an estimated location associated with the vehicle based, at least in part, on the parameters associated with the vehicle and the first uncertainty model. For example, the vehicle 102 may at least process the parameters associated with the vehicle 102 with the first uncertainty model. Based, at least in part, on the processing, the vehicle 102 may subsequently determine an estimated location associated with the vehicle 102. As also described herein, the estimated location may correspond to a probability distribution of locations.

At operation 1310, process 1300 may include at least determining, with at least a second system of the vehicle, a parameter associated with the object based, at least in part, on the second portion of the sensor data. For example, vehicle 102 may analyze the sensor data and, at least in part, based on the analysis, may identify the object. Vehicle 102 may then analyze the second portion of the sensor data with one or more systems. At least in part, based on the analysis, vehicle 102 may determine a parameter associated with the object. The parameters may include, but are not limited to, a type of object, a position of the object, a speed of the object, a direction of travel of the object, etc.

At 1312, process 1300 may include determining a second uncertainty model associated with a second system that determines the parameters associated with the object. For example, vehicle 102 may determine the second uncertainty model. In one example, vehicle 102 determines the second uncertainty model by receiving the second uncertainty model from the second system. In one example, vehicle 102 determines the second uncertainty model using uncertainty data indicative of uncertainties associated with the second system that determines the second parameters.

At operation 1314, process 1300 may include determining an estimated location associated with the object based, at least in part, on the parameters associated with the object and the second uncertainty model. For example, vehicle 102 may process at least the parameters associated with the object with the second uncertainty model. Based, at least in part, on the processing, vehicle 102 may subsequently determine an estimated location associated with the object. As also described herein, the estimated location may correspond to a probability distribution of locations.

At operation 1316, process 1300 may include determining a probability of collision based at least in part on the estimated position associated with the vehicle and the estimated position associated with the object. For example, the vehicle 102 may analyze the estimated position associated with the vehicle 102 and the estimated position associated with the object to determine the probability of collision. In one example, the probability of collision may be based at least in part on an amount of overlap between the estimated position associated with the vehicle 102 and the estimated position associated with the object.

At operation 1318, process 1300 may include determining whether the probability of a collision is equal to or greater than a threshold value. For example, vehicle 102 may compare the probability of a collision to a threshold value to determine whether the probability of a collision is equal to or greater than the threshold value.

If, at operation 1318, it is determined that the probability of a collision is not equal to or greater than the threshold, then, at operation 1320, process 1300 may include allowing the vehicle to continue navigating along the path. For example, if the vehicle 102 determines that the probability of a collision is less than the threshold, the vehicle 102 may continue navigating along the path.

However, if at operation 1318 it is determined that the probability of a collision is equal to or greater than the threshold, then at operation 1322 the process 1300 may include causing the vehicle to perform one or more actions. For example, if the vehicle 102 determines that the probability of a collision is equal to or greater than the threshold, the vehicle 102 may perform one or more actions. The one or more actions may include, but are not limited to, altering the route of the vehicle 102, altering the speed of the vehicle 102, parking the vehicle 102, etc.

Note that in one example, the vehicle 102 may perform steps 1304-1314 using a variety of possible paths associated with the vehicle 102. In such an example, the vehicle 102 may select the route that includes the lowest uncertainty and/or the lowest probability of collision.

FIG. 14 illustrates an example process 1400 for determining an estimated position associated with an object for use with an uncertainty model in accordance with an embodiment of the present disclosure. At operation 1402, the process 1400 may include receiving sensor data generated by one or more sensors. For example, the vehicle 102 may navigate along a path from a first location to a second location. While navigating, the vehicle 102 may generate sensor data using one or more sensors of the vehicle 102.

At operation 1404, process 1400 may include determining, with one or more systems of the vehicle, a first parameter associated with the object based at least in part on the sensor data. For example, vehicle 102 may analyze the sensor data with one or more systems. The one or more systems may include, but are not limited to, a localization system, a perception system, a planning system, a predictive system, etc. Based at least in part on the analysis, vehicle 102 may determine a first parameter associated with the object (e.g., the vehicle or another object). The first parameter may include, but is not limited to, a position of the object, a speed of the object, a traveling direction of the object, etc.

At operation 1406, process 1400 may include determining a first probability distribution associated with the first parameter based at least in part on the first uncertainty model. For example, vehicle 102 may at least process the first parameter with the first uncertainty model. Based at least in part on the processing, vehicle 102 may determine a first probability distribution associated with the first parameter.

At operation 1408, the process 1400 may include using one or more systems of the vehicle to determine a second parameter associated with the object based at least in part on at least one of the sensor data or the first probability distribution. For example, the vehicle 102 may use one or more systems to analyze at least one of the sensor data or the first probability distribution. In one example, the vehicle 102 analyzes the first probability distribution where the second parameter is determined using the first parameter. Based at least in part on the analysis, the vehicle 102 may determine a second parameter associated with the object (e.g., a vehicle or another object). The second parameter may include, but is not limited to, a position of the object, a speed of the object, a direction of travel of the object, an estimated position of the object at a future time, etc.

At operation 1410, process 1400 may include determining a second probability distribution associated with the second parameter based at least in part on the second uncertainty model. For example, vehicle 102 may at least process the second parameter with the second uncertainty model. Based at least in part on the processing, vehicle 102 may determine a second probability distribution associated with the second parameter.

At operation 1412, the process 1400 may include determining an estimated location associated with the object based at least in part on at least one of the first probability distribution or the second probability distribution. For example, the vehicle 102 may determine the estimated location based at least in part on the first probability distribution and/or the second probability distribution. In one example, the vehicle 102 may determine the estimated location using both the first probability distribution and the second probability distribution when the first parameter and the second parameter are independent, such as the first parameter indicating the current location of the object and the second parameter indicating the velocity of the object. In one example, the vehicle 102 may determine the estimated location using the second probability distribution when the second parameter is determined using the first parameter, such as when the second parameter indicates the estimated location of the object at a future time determined using the first parameter indicating the velocity of the object.

(Conclusion)
While one or more examples of the technology described herein have been described, various modifications, additions, permutations, and equivalents of those examples are included within the scope of the technology described herein.

In describing the examples, reference is made to the accompanying drawings which form a part of this application, which show specific examples of the claimed subject matter. It is understood that other examples can be used and that modifications or variations, such as structural changes, can be made. Such examples, modifications or variations do not necessarily depart from the intended scope of the claimed subject matter. While steps herein are provided in a particular order, in some cases the order may be changed such that certain inputs are provided at different times or in a different order without changing the functionality of the systems and methods described. The disclosed procedures may also be performed in different orders. Furthermore, the various computations herein need not be performed in the order disclosed, and other examples using alternative orders of computations may be readily implemented. In addition to being reordered, computations may also be decomposed into partial computations with the same results.

(Illustrative clauses)
A: An autonomous vehicle comprises one or more sensors, one or more processors, and one or more computer-readable media that, when executed by the one or more processors, cause the one or more processors to perform operations including obtaining sensor data from the one or more sensors; determining an estimated position of the autonomous vehicle at a future time based at least in part on a first portion of the sensor data; determining an estimated position of an object at a future time based at least in part on a system of the autonomous vehicle and a second portion of the sensor data; determining a distribution of estimated positions associated with the object based at least in part on an error model and the estimated position of the object, where the error model represents a probability of error associated with the system; determining a probability of a collision between the autonomous vehicle and the object based at least in part on the estimated position of the autonomous vehicle and the distribution of estimated positions associated with the object; and causing the autonomous vehicle to perform one or more actions based at least in part on the probability of the collision.

B: The autonomous vehicle of paragraph A, wherein the operations further include receiving an error model from one or more computing devices, the error model being generated using at least sensor data generated by the one or more vehicles.

C: An autonomous vehicle as described in either paragraph A or B, wherein the operations further include determining a distribution of estimated positions associated with the autonomous vehicle based at least in part on the further error model and the estimated position of the vehicle, and determining the probability of a collision between the autonomous vehicle and the object includes at least determining an amount of overlap between the distribution of estimated positions associated with the autonomous vehicle and the distribution of estimated positions associated with the object, and determining the probability of the collision based at least in part on the amount of overlap.

D: An autonomous vehicle as described in any one of paragraphs A-C, wherein the estimated position of the object at a future time is further determined based at least in part on a further system of the autonomous vehicle, and the distribution of the estimated positions is further determined based at least in part on a further error model, the further error model representing an error distribution associated with the further system.

E: The method comprises receiving sensor data from one or more sensors of the vehicle; determining an estimated position associated with the vehicle at a time based at least in part on a first portion of the sensor data; determining parameters associated with the object based at least in part on a system of the vehicle and a second portion of the sensor data; determining an estimated position associated with the object at a time based at least in part on an error model and parameters associated with the object, where the error model represents a probability of error associated with the system; and causing the vehicle to perform one or more actions based at least in part on the estimated position associated with the vehicle and the estimated position associated with the object.

F: The method of paragraph E, wherein the operations further include receiving an error model from one or more computing devices, the error model being generated using at least sensor data generated by one or more vehicles.

G: The method of any of paragraphs E or F, wherein the parameters include at least one of a type of object associated with the object, a position of the object within the environment, a velocity of the object, or a direction of travel of the object within the environment.

H: The method of any one of paragraphs E-F, wherein the step of determining an estimated position associated with the vehicle at the time includes at least a step of determining a parameter associated with the vehicle based, at least in part, on a further system of the vehicle and a first portion of the sensor data, and a step of determining an estimated position associated with the vehicle at the time based, at least in part, on a further error model and a parameter associated with the vehicle, the further error model representing a probability of an error associated with the further system.

I: The method of any one of paragraphs E-H, further comprising determining a further estimated position associated with the object at the time based at least in part on the parameters, wherein determining the further estimated position associated with the object at the time comprises determining an estimated position associated with the object at the time based at least in part on the error model and the further estimated position associated with the object.

J: The method of any one of paragraphs E-I, wherein determining an estimated position associated with the object at a time includes determining a distribution of estimated positions associated with the object at a time based, at least in part, on an error model and parameters associated with the object.

K: The method of any one of paragraphs E-J, wherein the step of determining an estimated position associated with the vehicle includes at least a step of determining a parameter associated with the vehicle based, at least in part, on a further system of the vehicle and a first portion of the sensor data, and a step of determining a distribution of the estimated position associated with the vehicle at a time based, at least in part, on a further error model and the parameter associated with the vehicle, the further error model representing a probability of error associated with the further system.

L: The method of any one of paragraphs E-K, further comprising determining an amount of overlap between a distribution of estimated positions associated with the vehicle and a distribution of estimated positions associated with the object, and determining a probability of collision based at least in part on the amount of overlap, wherein causing the vehicle to perform one or more actions is based at least in part on the probability of collision.

M: The method of any one of paragraphs E-L, further comprising selecting an error model based at least in part on the parameters.

N: The method of any one of paragraphs E-M, further comprising: determining a further parameter associated with the object based, at least in part, on a further system of the vehicle and a second portion of the sensor data; and determining an output associated with the object based, at least in part, on the further error model and the further parameter associated with the object, wherein determining the parameter associated with the object is based, at least in part, on the system and output of the vehicle.

O: The method of any one of paragraphs E to N, wherein the system is a perception system and the further system is a prediction system.

P: The method of any one of paragraphs E-O, further comprising the steps of determining a further estimated position associated with the vehicle at a further time point after the time point based at least in part on a first portion of the sensor data, determining further parameters associated with the object based at least in part on the vehicle's systems and a second portion of the sensor data, determining a further estimated position associated with the object at a further time point based at least in part on the error model and further parameters associated with the object, and causing the vehicle to perform one or more actions based at least in part on the further estimated position associated with the vehicle and the further estimated position associated with the object.

Q: The method of any one of paragraphs E-P, further comprising determining a probability of collision based at least in part on an estimated position associated with the vehicle and an estimated position associated with the object, and causing the vehicle to perform one or more actions includes causing the vehicle to at least one of change speed or change route based at least in part on the probability of collision.

R: One or more non-transitory computer-readable media store instructions that, when executed by one or more processors, cause one or more computing devices to perform operations including receiving sensor data generated by sensors associated with the vehicle, determining an estimated position associated with the object at a time based at least in part on a portion of the sensor data, determining an error model from a plurality of error models based at least in part on the estimated position, determining a distribution of estimated positions associated with the object based at least in part on the error model and the estimated position, and determining one or more actions to navigate the vehicle based at least in part on the distribution of estimated positions.

S: One or more non-transitory computer-readable media as described in paragraph R, wherein the operations further include determining a parameter associated with the vehicle based at least in part on a portion of the sensor data, and determining an estimated position based at least in part on the parameter, and the error model is associated with the parameter.

T: One or more non-transitory computer-readable media as described in paragraphs R or S, wherein determining the error model is further based, at least in part, on one or more of the following: classification of the object, speed of the object, size of the object, number of objects in the environment, weather conditions in the environment, time of day, or season.

U: An autonomous vehicle includes one or more sensors, one or more processors, and one or more computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including acquiring sensor data generated by the one or more sensors, determining an estimated position of the autonomous vehicle based at least in part on a first portion of the sensor data, determining an estimated position of the object based at least in part on a second portion of the sensor data, determining an uncertainty model associated with the estimated position of the object, determining a distribution of estimated positions associated with the object based at least in part on the uncertainty model and the estimated position of the object, determining a probability of a collision between the autonomous vehicle and the object based at least in part on the probability of the estimated position associated with the vehicle and the estimated position associated with the object, and causing the autonomous vehicle to perform one or more actions based at least in part on the probability of the collision.

V: The autonomous vehicle of paragraph U, wherein the operations further include determining a further uncertainty model associated with a further system that determines an estimated position of the autonomous vehicle, and determining a probability of an estimated position associated with the autonomous vehicle based at least in part on the further uncertainty model and the estimated position of the autonomous vehicle, and the probability of a collision between the autonomous vehicle and the object includes determining at least an amount of overlap between the probability of the estimated position associated with the autonomous vehicle and the probability of the estimated position associated with the object, and determining the probability of the collision based at least in part on the amount of overlap.

W: An autonomous vehicle as described in either paragraph U or V, wherein the estimated position of the object is further determined based, at least in part, on a further system of the autonomous vehicle, and the operations further include determining a further uncertainty model associated with the further system that determines the estimated position of the object, and the probability of the estimated position is further determined based, at least in part, on the further uncertainty model.

X: The method comprises receiving sensor data from one or more sensors of the vehicle, determining an estimated position associated with the vehicle based at least in part on a first portion of the sensor data, determining parameters associated with the object based at least in part on a system of the vehicle and a second portion of the sensor data, determining an uncertainty model associated with the system, determining an estimated position associated with the object based at least in part on the parameters and the uncertainty model associated with the object, and causing the vehicle to perform one or more actions based at least in part on the estimated position associated with the vehicle and the estimated position associated with the object.

Y: The method of paragraph X, further comprising receiving an uncertainty model from one or more computing devices, the uncertainty model being generated, at least in part, based on sensor data generated by one or more vehicles.

Z: The method of any of paragraphs X or Y, wherein determining the parameters associated with the object includes determining, at least in part, based on the second portion of the system and sensor data, at least one of a type of object associated with the object, a position of the object within the environment, a velocity of the object, or a traveling direction of the object within the environment.

AA: The method of any one of paragraphs X-Z, wherein determining an estimated position associated with the vehicle includes determining a parameter associated with the vehicle based at least in part on a further system of the vehicle and a first portion of the sensor data, determining a further uncertainty model associated with the further system that determines the parameter associated with the vehicle, and determining an estimated position associated with the vehicle based at least in part on the parameter associated with the vehicle and the further uncertainty model.

AB: The method of any one of paragraphs X-AA, further comprising determining a further estimated position associated with the object based at least in part on the parameters, wherein determining the estimated position associated with the object comprises determining an estimated position associated with the object based at least in part on the further estimated position associated with the object and an uncertainty model.

AC: The method of any one of paragraphs X-AB, wherein determining an estimated position associated with the object includes determining a distribution of estimated positions associated with the object based, at least in part, on parameters associated with the object and an uncertainty model.

AD: The method of any one of paragraphs X-AC, wherein determining an estimated position associated with the vehicle includes determining a parameter associated with the vehicle based at least in part on a further system of the vehicle and a first portion of the sensor data, determining a further uncertainty model associated with the further system that determines the parameter associated with the vehicle, and determining a distribution of estimated positions associated with the vehicle based at least in part on the parameter associated with the vehicle and the further uncertainty model.

AE: The method of any one of paragraphs X-AD, further comprising determining an amount of overlap between a distribution of estimated positions associated with the vehicle and a distribution of estimated positions associated with the object, and determining a probability of collision based at least in part on the amount of overlap, and causing the vehicle to perform one or more actions based at least in part on the probability of collision.

AF: The method of any one of paragraphs X-AE, further comprising determining a further parameter associated with the object based at least in part on a further system of the vehicle and a third portion of the sensor data, and determining a further uncertainty model associated with the further system that determines the further parameter associated with the object, wherein determining an estimated position associated with the object is based at least in part on the further parameter and the further uncertainty model.

AG: The method of any one of paragraphs X-AF, further comprising determining a further parameter associated with the object based at least in part on a further system of the vehicle and a second portion of the sensor data, determining a further uncertainty model associated with the further system that determines the further parameter associated with the object, and determining an output associated with the object based at least in part on the further parameter associated with the object and the further uncertainty model, wherein determining the parameter associated with the object comprises determining the parameter associated with the object based at least in part on the system and output of the vehicle.

AH: The method of any one of paragraphs X-AG, further comprising: determining a parameter associated with the further object based at least in part on a system of the vehicle and a third portion of the sensor data; determining a further uncertainty model associated with the system that determines the parameter associated with the further object; determining an estimated position associated with the further object based at least in part on the parameter associated with the further object and the further uncertainty model; and causing the vehicle to perform one or more actions, further based at least in part on the estimated position associated with the further object.

AI: The method of any one of paragraphs X-AH, further comprising determining a probability of collision based at least in part on an estimated position associated with the vehicle and an estimated position associated with the object, and causing the vehicle to perform one or more actions includes causing the vehicle to at least one of change speed or change route based at least in part on the probability of collision.

AJ: One or more non-transitory computer-readable media store instructions that, when executed by one or more processors, cause one or more computing devices to perform operations including receiving sensor data generated by sensors associated with the vehicle, determining an estimated position associated with the object based at least in part on a portion of the sensor data, determining an uncertainty model from a plurality of uncertainty models based at least in part on the estimated position, determining a distribution of estimated positions associated with the object based at least in part on the uncertainty model and the estimated position, and determining one or more actions to navigate the vehicle based at least in part on the distribution of estimated positions.

AK: One or more non-transitory computer-readable media as described in paragraph AJ, wherein the operations further include determining a parameter associated with the vehicle based at least in part on a portion of the sensor data, and determining an estimated position based at least in part on the parameter, and an uncertainty model is associated with the parameter.

AL: One or more non-transitory computer-readable media of any of paragraphs AJ or AK, wherein the operations further include determining an estimated position associated with the vehicle based at least in part on a further portion of the sensor data, determining a further uncertainty model from the plurality of uncertainty models based at least in part on the estimated position, and determining a distribution of the estimated position associated with the vehicle based at least in part on the further uncertainty model and the estimated position associated with the vehicle, wherein determining the one or more actions is based at least in part on the distribution of the estimated position associated with the vehicle.

AM: One or more non-transitory computer-readable media described in any one of paragraphs AJ-AL, wherein the operations further include determining a probability of collision based at least in part on a distribution of estimated positions associated with the vehicle and a distribution of estimated positions associated with the object, and determining the one or more actions is based at least in part on the probability of collision.

AN: One or more non-transitory computer-readable media described in any one of paragraphs AJ-AM, wherein determining the uncertainty model is further based, at least in part, on one or more of a classification of the object, a speed of the object, a size of the object, a number of objects in the environment, weather conditions in the environment, a time of day, or a season.

Claims (15)

1. An autonomous vehicle, comprising:
One or more sensors;
one or more processors;
When executed by one or more of the processors, the one or more processors are
acquiring sensor data from one or more of said sensors;
determining an estimated position of the autonomous vehicle at a future time based at least in part on the first portion of the sensor data;
determining an estimated position of an object at the future time based at least in part on a first system of the autonomous vehicle and a second portion of the sensor data;
determining a distribution of estimated positions associated with the object based at least in part on a first error model and the estimated positions of the object, the first error model representing a probability of error associated with an output of at least one component associated with position estimation in the first system;
determining a probability of a collision between the autonomous vehicle and the object based at least in part on the estimated position of the autonomous vehicle and a distribution of the estimated positions associated with the object;
and causing the autonomous vehicle to perform one or more actions based at least in part on the probability of a collision.
Autonomous vehicles. The operations further include receiving the first error model from one or more computing devices, the first error model being generated using at least sensor data generated by one or more vehicles.
The autonomous vehicle of claim 1 . The operation includes:
determining a distribution of estimated positions associated with the autonomous vehicle based at least in part on a second error model and the estimated positions of the autonomous vehicle;
Determining the probability of a collision between the autonomous vehicle and the object comprises at least:
determining an amount of overlap between a distribution of the estimated positions associated with the autonomous vehicle and a distribution of the estimated positions associated with the object;
determining the probability of collision based at least in part on the amount of overlap.
3. An autonomous vehicle as claimed in claim 1 or 2. The estimated position of the object at the future time is further determined based at least in part on a second system of the autonomous vehicle;
a distribution of the estimated positions is further determined based at least in part on a second error model, the second error model representing an error distribution associated with the second system.
The autonomous vehicle of claim 3 . receiving sensor data from one or more sensors of the vehicle;
determining an estimated position associated with the vehicle at a first time based at least in part on the first portion of the sensor data;
determining a first parameter associated with the object based at least in part on a first system of the vehicle and a second portion of the sensor data;
determining an estimated position associated with the object at the first time based at least in part on a first error model and the first parameter associated with the object, the first error model representing a probability of error associated with an output of at least one component associated with position estimation in the first system;
and causing the vehicle to perform one or more actions based at least in part on the estimated position associated with the vehicle and the estimated position associated with the object.
method. receiving the first error model from one or more computing devices, the first error model being generated using at least sensor data generated by one or more vehicles.
The method according to claim 5. The first parameter is
the type of object associated with said object;
the position of said object within the environment;
a velocity of the object; or a direction of travel of the object within the environment.
7. The method according to claim 5 or 6. The step of determining the estimated location associated with the vehicle at the first time point comprises at least:
determining a parameter associated with the vehicle based at least in part on a second system of the vehicle and the first portion of the sensor data;
determining the estimated position associated with the vehicle at the first time based at least in part on a second error model and the parameters associated with the vehicle, the second error model representing probabilities of errors associated with the second system ;
8. The method according to any one of claims 5 to 7. determining an additional estimated location associated with the object at the first time based at least in part on the first parameter;
determining the estimated position associated with the object at the first time point comprises determining the estimated position associated with the object at the first time point based at least in part on the first error model and the additional estimated position associated with the object.
8. The method according to any one of claims 5 to 7 . determining the estimated position associated with the object at the first time point includes determining a distribution of estimated positions associated with the object at the time point based at least in part on the first error model and the first parameter associated with the object;
8. The method according to any one of claims 5 to 7 . selecting at least one of the first error model and the second error model based at least in part on the first parameters.
The method according to claim 8 . determining a second parameter associated with the object based at least in part on a second system of the vehicle and the second portion of the sensor data;
determining an output of the second system associated with the object based at least in part on a second error model and the second parameter associated with the object, the second error model representing a probability of an error associated with the second system;
determining the second parameter associated with the object includes determining the second parameter associated with the object based at least in part on the second system of the vehicle and the output;
8. The method according to any one of claims 5 to 7 . determining an additional location estimate associated with the vehicle at a second time point, later than the first time point, based at least in part on the first portion of the sensor data;
determining a second parameter associated with the object based at least in part on the first system of the vehicle and the second portion of the sensor data;
determining an additional estimated position associated with the object at the second time based at least in part on the first error model and the second parameter associated with the object;
and causing the vehicle to perform one or more actions based at least in part on the additional estimated position associated with the vehicle and the additional estimated position associated with the object.
8. The method according to any one of claims 5 to 7 . determining a probability of a collision based at least in part on the estimated position associated with the vehicle and the estimated position associated with the object;
causing the vehicle to perform one or more of the actions includes causing the vehicle to at least one of change speed or change route based at least in part on the probability of collision.
8. The method according to any one of claims 5 to 7 . One or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform the method of any one of claims 5 to 14.

Images