JP7760732B2 - Information exchange method, device, and system - Google Patents

Description

This application relates to the field of intelligent vehicles, and in particular to information exchange methods, devices, and systems.

In recent years, autonomous driving has become an important development direction in the field of vehicle research and development. To improve the intelligence of autonomous vehicle driving, it is necessary to verify the capabilities of the vehicle's autonomous driving system under various operating conditions. In the process of testing an autonomous driving system, various environmental and traffic conditions, such as pedestrians, traffic vehicles, traffic lights, and road infrastructure, should be fully taken into consideration.

Currently, traffic police officers coordinate and direct traffic in scenarios such as traffic light failure, handling traffic accidents, and emergency traffic control. Therefore, identifying and responding to traffic police gestures is important for autonomous vehicle operation. However, there is currently little research on identifying and responding to traffic police gestures by autonomous driving systems, and there is an urgent need for a solution that can be applied to test the ability of autonomous driving systems to identify and respond to traffic police gestures.

Embodiments of the present application improve the safety and intelligence of autonomous driving by providing information exchange methods, devices, and systems for testing the ability of an autonomous driving system to identify and respond to traffic officer gestures.

According to a first aspect, there is provided an information exchange method. The method may be applied to a first device. The first device may be any device having an autonomous driving function or an assisted driving function, for example, an autonomous driving system (supporting the autonomous driving function or the assisted driving function), or a device or apparatus equipped with an autonomous driving system. The method includes the steps of receiving at least one test data, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture, the first parameter belonging to a predetermined gesture parameter set, and outputting a first control signal based on the at least one test data, the first control signal being used to control a first virtual terminal.

In the aforementioned solution, the autonomous driving system may receive at least one test data and output a first control signal based on the at least one test data to test the autonomous driving system's ability to identify and respond to gestures, thereby improving the safety and intelligence of the autonomous driving of the vehicle.

In a possible design, the gesture parameter set includes multiple parameters, which indicate any of a stop gesture, a go straight gesture, a left turn gesture, a left turn waiting gesture, a right turn gesture, a right turn waiting gesture, a lane change gesture, a slow down gesture, a pull over gesture, or an unknown gesture.

Please understand that the types listed above are merely examples and are not intended to be specific limitations.

In a possible design, the first parameter is an RGB value, and the RGB value represents an image corresponding to the first gesture. In other words, the test data may be in an image format.

In this way, the effect of identifying and responding to a first gesture by an autonomous driving system based on imaging technology can be simulated.

In a possible design, the first parameter is a true value that indicates a first gesture. For example, the first parameter is a true value agreed upon in the OSI standard.

In this way, the effect of the autonomous driving system identifying and responding to the first gesture based on the truth value can be simulated.

In a possible design, the gesture information further includes a second parameter, which indicates information about the lane in which the first virtual terminal is located.

In this manner, information regarding the lane in which the first virtual terminal is located may be provided as part of the gesture information for the autonomous driving system, allowing the autonomous driving system to identify and respond to the first gesture by referring to the information regarding the lane in which the first virtual terminal is located, thereby improving the reliability of the solution.

In a possible design, the gesture information further includes a third parameter, which indicates whether the first parameter is valid.

In this way, whether or not to test the autonomous driving system's ability to identify and respond to gestures can be flexibly decided based on requirements, thereby conserving the autonomous driving system's computing resources.

In possible designs, the at least one test data further includes one or more of the following information:
target identification information of the first gesture;
target position information of the first gesture;
Target-referring information of the first gesture; or Target-driving information of the first gesture.

In this way, information about the execution target corresponding to the first gesture can be provided to the autonomous driving system, further improving the reliability of the test solution.

In a possible design, the at least one test data may further include traffic light information.

In this way, the ability of an autonomous driving system to handle conflicting information (e.g., conflicts between gesture information and traffic light information) can be tested, potentially improving the robustness of the autonomous driving system.

In a possible design, at least one test data further includes obstacle information.

In this way, the ability of an autonomous driving system to process interference information is tested, which may improve the robustness of the autonomous driving system.

In a possible design, the at least one test data further includes sensory data from a second virtual terminal, the sensory data including reference gesture information, the reference gesture information indicating the first gesture.

In this way, the ability of an autonomous driving system to identify gestures based on V2X communication capabilities can be tested.

In a possible design, the at least one test data includes gesture information and traffic light information. Correspondingly, the step of outputting a first control signal based on the at least one test data may include the step of outputting a first control signal based on the traffic light information or the gesture information, whichever has a higher priority.

In this way, when traffic light information and gesture information are received simultaneously, the autonomous driving system can output a first control signal based on the information with higher priority, thereby improving the robustness of the autonomous driving system.

According to a second aspect, an information exchange method is provided. The method may be applied to a second device, which may be any device having a simulation function, such as a simulation system. The method includes the steps of outputting at least one test data to a first device, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture, the first parameter belonging to a predetermined gesture parameter set; receiving a first control signal from the first device; and controlling a first virtual terminal based on the first control signal.

In a possible design, the gesture parameter set includes multiple parameters, which indicate any of a stop gesture, a go straight gesture, a left turn gesture, a left turn waiting gesture, a right turn gesture, a right turn waiting gesture, a lane change gesture, a slow down gesture, a pull over gesture, or an unknown gesture.

In a possible design, the first parameter is an RGB value, where the RGB value represents an image corresponding to the first gesture, or the first parameter is a true value, where the true value indicates the first gesture.

In a possible design, the gesture information further includes a second parameter indicating information about the lane in which the first virtual terminal is located, and/or a third parameter indicating whether the first parameter is valid.

In possible designs, the at least one test data further includes one or more of the following information:
target identification information of the first gesture;
target position information of the first gesture;
Target-referring information of the first gesture; or Target-driving information of the first gesture.

In a possible design, the at least one test data further includes at least one of traffic light information or obstacle information.

In a possible design, the at least one test data further includes sensory data from a second virtual terminal, the sensory data including reference gesture information, the reference gesture information indicating the first gesture.

For the technical effects of the design in the second aspect, please refer to the technical effects of the corresponding design in the first aspect. Details will not be described again here.

In a possible design, when a first event is detected, the virtual object is controlled to perform a first gesture. For example, the first event may include one or more of an event in which the distance between the virtual object and the first virtual terminal is less than a preset distance, a traffic jam, or a traffic accident. It should be understood that the aforementioned types are merely examples and are not limiting.

In this design, driving a virtual subject to perform gestures based on events is closer to how gestures are triggered in real-world scenarios, which may improve the reliability of the test.

In a possible design, the virtual subject may be controlled to perform the same gesture as a gesture in the road collection data based on the road collection data, where the road collection data indicates that the real subject would perform the first gesture in a real road scenario.

In this design, driving a virtual subject to perform the same gestures as real gestures is closer to the gesture change rules in real scenarios, which can improve the reliability of the test.

In a possible design, the virtual subject is controlled to perform specified gestures at specified times, the specified gestures including the first gesture.

This design can reduce the complexity of the solution by driving virtual objects to perform gestures according to pre-set rules.

According to a third aspect, there is provided an information exchange device comprising a module/unit/technical means configured to perform the method according to the first aspect or any one of the possible designs of the first aspect.

For example, the device includes a transceiver module and a processing module. The transceiver module is configured to receive at least one test data, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture, the first parameter belonging to a predetermined gesture parameter set. The processing module is configured to generate a first control signal based on the at least one test data, the first control signal being used to control the first virtual terminal. The transceiver module is further configured to output the first control signal.

According to a fourth aspect, there is provided an information exchange device comprising a module/unit/technical means configured to perform the method according to the second aspect or any one of the possible designs of the second aspect.

For example, the device includes a transceiver module and a processing module. The transceiver module is configured to output at least one test data to the first device, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture, the first parameter belonging to a predetermined gesture parameter set. The transceiver module is further configured to receive a first control signal from the first device. The processing module is configured to control a first virtual terminal based on the first control signal.

According to a fifth aspect, there is provided an information exchange device. The information exchange device includes a processor and an interface circuit. The interface circuit is configured to receive a signal from another device other than the device and transmit the signal to the processor, or to transmit a signal from the processor to another communication device other than the device, and the processor is configured to implement a method according to the first aspect or any one of possible designs of the first aspect by using logic circuits or executing code instructions.

According to a sixth aspect, there is provided an information exchange device. The information exchange device includes a processor and an interface circuit, the interface circuit configured to receive a signal from another device other than the device and transmit the signal to the processor, or to transmit a signal from the processor to another communication device other than the device, and the processor configured to implement a method according to the second aspect or any one of possible designs of the second aspect by using logic circuits or executing code instructions.

According to a seventh aspect, there is provided an information exchange system including a device according to the third aspect and a device according to the fourth aspect, or including a device according to the fifth aspect and a device according to the sixth aspect.

According to an eighth aspect, there is provided a computer-readable storage medium containing a program or instructions. When the program or instructions are executed on a computer, it is possible to perform a method according to the first aspect, any one of the possible designs of the first aspect, the second aspect, or any one of the possible designs of the second aspect.

According to a ninth aspect, there is provided a computer program product comprising instructions. The computer program product stores the instructions. When the instructions are executed on a computer, the computer is enabled to perform a method according to the first aspect or any one of the possible designs of the first aspect, or a method according to the second aspect or any one of the possible designs of the second aspect.

According to a tenth aspect, a vehicle is provided. The vehicle includes an apparatus according to the third aspect, an apparatus according to the fifth aspect, or a computer-readable storage medium according to the eighth aspect.

1 is a schematic diagram of the architecture of an information exchange system according to an embodiment of the present application; 1 is a schematic diagram of a possible hypothetical traffic scenario according to an embodiment of the present application; 1 is a flowchart of an information exchange method according to an embodiment of the present application. 1 is a schematic diagram of possible test data according to an embodiment of the present application; 3A-3C are schematic diagrams of virtual object images in different virtual terminal fields of view according to an embodiment of the present application; 1 is a schematic diagram of the structure of an information exchange device according to an embodiment of the present application; FIG. 10 is a schematic diagram of the structure of another information exchange device according to an embodiment of the present application;

Embodiments of the present application are described in detail below with reference to the accompanying drawings.

Embodiments of the present application can be used in any scenario where the capabilities of a drive system can be tested.

For example, FIG. 1 is a schematic diagram of the architecture of an information exchange system to which embodiments of the present application are applicable. The system includes a second device and a first device. The second device is configured to generate at least one test data and output the at least one test data to the first device. The first device is an object under test (or an object under test is placed on the first device). The first device is configured to receive the at least one test data, process the at least one test data, generate a test result (e.g., a first control signal), and output the test result to the second device.

In a possible design, the first device may be an autonomous driving system or any device integrated with an autonomous driving system. The specific form of the device may be a chip (e.g., a controller, processor, or integrated circuit), a terminal (e.g., an in-vehicle terminal), etc. This is not a limitation in this application.

For example, autonomous driving systems in this embodiment of the present application include, but are not limited to, the following systems:

(1) Emergency assistance system: An emergency assistance system has the ability to continuously detect and respond to several targets and events in a dynamic driving task, rather than continuously performing horizontal or vertical movement control of the vehicle in a dynamic driving task.

(2) Partial driver assistance system: A partial driver assistance system has the ability to continuously perform horizontal or vertical movement control of a vehicle in a dynamic driving task under operating conditions designed by the partial driver assistance system, and to detect and respond to several targets and events corresponding to horizontal or vertical movement control of the vehicle.

(3) Combined driver assistance system: The combined driver assistance system has the ability to continuously perform horizontal and vertical movement control of the vehicle in a dynamic driving task under operating conditions designed by the combined driver assistance system, and to detect and respond to several targets and events corresponding to the horizontal and vertical movement control of the vehicle.

(4) Conditionally automated driving system: A conditionally automated driving system continuously performs all dynamic driving tasks under the operating conditions designed by the conditionally automated driving system.

(5) Highly automated driving system: A highly automated driving system continuously performs all dynamic driving tasks and automatically implements the minimum risk policy under operating conditions designed by the highly automated driving system.

(6) Fully automated driving system: A fully automated driving system continuously performs all dynamic driving tasks under any driving conditions and automatically implements a minimum risk policy.

It should be understood that the above types are merely examples and are not specific limitations. The autonomous driving system in the embodiments of this application is not limited to a country or level. Any system with assisted driving functions or autonomous driving functions falls within the scope of protection of the autonomous driving system in this application.

In a possible design, the second device may be a simulation system or any device (such as a chip or terminal) integrated with the simulation system. The second device is configured to generate test data related to traffic scenarios. The test data related to traffic scenarios is used to test the driving system's ability to identify traffic scenarios, its ability to respond to traffic scenarios, etc.

For a possible design, see Figure 1. The second device includes at least the following three functional modules:

(1) The scenario establishment module is configured to establish a virtual traffic scenario, for example, to establish environmental components (such as roads or buildings), traffic flows (such as vehicles or pedestrians), traffic lights (such as traffic lights or turn signals), or virtual objects (such as a traffic police officer model). The traffic flows include at least one virtual terminal, for example, a first virtual terminal. Figure 2 is a schematic diagram of a possible virtual traffic scenario according to one embodiment of the present application.

(2) The data analysis module is configured to acquire scenario data generated by the scenario establishment module, analyze the scenario data, and acquire test data that meets predetermined standards.

It can be understood that test data that meets pre-set standards can be identified and processed by the first device.

It should be understood that the preset standard is not particularly limited in this application. For example, the preset standard may be the Open Simulation Interface (OSI) standard established by the Association for Standardization of Automation and Measuring Systems (ASAM). The OSI standard defines a universal interface for connecting the development of autonomous driving functions with various driving simulation frameworks to achieve compatibility. This allows any autonomous driving function to be connected to any simulation tool, and various sensor models can be integrated.

(3) The vehicle dynamic model is configured to control the first virtual terminal in the virtual traffic scenario based on the control signal transmitted by the first device. For example, the vehicle dynamic model is used to obtain a dynamic response value of the first virtual terminal to the virtual traffic scenario based on the first control signal transmitted by the first device and the current attitude information of the first virtual terminal, and the attitude information of the first virtual terminal is updated based on the vehicle dynamic model, thereby achieving the effect of controlling the autonomous driving of the first virtual terminal by the autonomous driving system.

For a possible design, see FIG. 1. The first device includes at least a processing module. The processing module performs calculations on at least one test data and outputs a first control signal used to control a first virtual terminal to simulate the effect of an autonomous driving system identifying an event occurring in a scenario and controlling a vehicle in response to the event. For example, the processing module may execute instructions or data corresponding to an autonomous driving algorithm to perform calculations on at least one test data and output a first control signal.

It should be understood that the first device and the second device may be located in the same physical entity or in different physical entities. This is not a limitation in this application.

It should be understood that the system architecture shown in Figure 1 is merely an example and not a specific limitation. In actual applications, there may be other variations.

Figure 3 is a flowchart of an information exchange method according to one embodiment of the present application. An example is used in which this method is applied to the system shown in Figure 1. The method includes the following steps:

S101: A second device generates at least one test data.

The second device first establishes a virtual traffic scenario. For example, the scenario establishment module in the second device may call data in a scenario database to establish a virtual traffic scenario. The scenario database may provide a large number of typical scenarios to meet test requirements. The data in the scenario database may include static scenario data and dynamic scenario data. The static scenario data is used to describe static components in the traffic scenario, such as environmental components (such as roads). The dynamic scenario data is used to describe dynamic components in the traffic scenario, such as moving traffic flows (such as vehicles or pedestrians), changing traffic lights (such as traffic lights or turn signals), or virtual objects that may perform gestures. The scenario database may be stored in the second device or in a device other than the second device. This is not a limitation in the present application.

The established virtual traffic scenario may be a three-dimensional virtual traffic scenario model, which includes three-dimensional models of static components and three-dimensional models of dynamic components. Figure 2 is a schematic diagram of a virtual traffic scenario including roads, vehicles, pedestrians, traffic police officers, traffic lights, etc.

It can be understood that the virtual traffic scenario in this embodiment of the present application includes at least a first virtual terminal and a virtual object. The first virtual terminal is a virtual model of a terminal carrying an object to be tested (e.g., an autonomous driving system), for example, a virtual model of an autonomous driving vehicle. The virtual object may simulate a real person and perform the gestures of a real person. For example, the virtual object may be a virtual traffic police officer model and perform traffic police officer gestures, or the virtual object may be a virtual traffic assistant model and perform traffic assistant gestures. As shown in FIG. 2, an example is used in which the virtual object is a virtual traffic police officer model and the first virtual terminal is a vehicle.

After the virtual traffic scenario is established, the scenario establishment module may control (or drive) the virtual object to perform a first gesture.

In a possible design, the scenario establishment module may control (or drive) the virtual object to perform a gesture based on an event, where different events may correspond to the same gesture or different gestures. Events include, but are not limited to, an event in which the distance between the virtual object and the first virtual terminal is less than a preset distance, a traffic jam, a traffic accident, etc. For example, when the distance between the virtual object and the first virtual terminal is less than a preset distance, the object is controlled to perform a deceleration gesture. For example, when a traffic accident occurs in the virtual traffic scenario, the virtual object is controlled to perform a pullover gesture. For example, when a traffic jam occurs in the virtual traffic scenario, the scenario establishment module controls the virtual object to perform a lane change gesture. Correspondingly, when or after the first event occurs, the scenario establishment module controls the virtual object to perform a first gesture. Driving the virtual object to perform a gesture based on an event is closer to how gestures are triggered in a real scenario and may improve the reliability of the test.

In a possible design, the scenario establishment module may control (or drive) a virtual object to perform the same gesture as a gesture in the road collection data based on the road collection data, where the road collection data indicates a gesture performed by a real object in a real road scenario. For example, the road collection data may be video images, each of which may be three minutes long. The first minute of the video may show a real traffic police officer performing a stop gesture. The next minute of the video may show the real traffic police officer performing a left turn gesture. The next minute of the video may show the real traffic police officer performing a right turn gesture. Correspondingly, the scenario establishment module may control a virtual object to sequentially perform a stop gesture, a left turn gesture, and a right turn gesture within a three-minute duration. Correspondingly, the scenario establishment module may control a virtual object in the road data to perform a first gesture based on a first gesture performed by a real traffic police officer in the road collection data. Driving a virtual object to perform the same gesture as a real gesture may approximate the gesture change rules in a real scenario and improve the reliability of the test.

In a possible design, the scenario establishment module may control (or drive) the virtual object to perform specified gestures at specified times according to preset rules. For example, the duration of one test is 10 minutes, and the preset rule is that the 10 gestures shown in Table 1 are performed sequentially, with each gesture having an execution duration of 1 minute. Correspondingly, the specified gestures include the first gesture. Driving the virtual object to perform gestures according to preset rules may reduce the complexity of the solution.

During the process of the virtual object performing a gesture, the data analysis module may acquire scenario data of the virtual traffic scenario. It can be understood that the scenario data acquired by the data analysis module includes at least data related to the virtual object.

In a possible design, one or more sensor models, including, but not limited to, an image sensor model or a radar model, are integrated into the data analysis module. Based on the sensor models, the data analysis module may acquire perceptual data of a first virtual terminal on a virtual object, such as an image or point cloud corresponding to the virtual object. In this way, the scenario data acquired by the data analysis module is closer to the perceptual data of an actual vehicle in real life.

In another possible design, the data analysis module may also directly retrieve data in the scenario database used by the scenario establishment module to establish the virtual traffic scenario and use that data as the perception data of the first virtual terminal for the virtual object. In this way, the scenario data more accurately describes the virtual traffic scenario.

After the data analysis module acquires the scenario data of the virtual traffic scenario, if the scenario data of the virtual traffic scenario is in a preset format, the scenario data may be directly used as at least one piece of test data; or, if the scenario data of the virtual traffic scenario is not in a preset format, the data analysis module analyzes the scenario data and converts it into at least one piece of test data in a preset format. Optionally, the preset format is a data format that meets the OSI standard. In this way, a standardized connection between the second device and the autonomous driving system in the first device can be realized.

In this embodiment of the present application, the at least one test data includes at least gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture, and the first parameter can be understood to belong to a predetermined gesture parameter set. The at least one test data is used to test the ability of an autonomous driving system in the first device to identify and respond to gestures.

In one possible design, the first virtual object in the virtual traffic scenario is a virtual traffic police officer model. The gesture parameter set includes a plurality of parameters, each of which indicates a corresponding traffic police officer gesture, with different parameters indicating different traffic police officer gestures.

For example, the multiple parameters may indicate any of a stop gesture, a go straight gesture, a left turn gesture, a left turn waiting gesture, a right turn gesture, a right turn waiting gesture, a lane change gesture, a slow down gesture, a pull over gesture, or an unknown gesture.

It should be understood that the definitions of traffic police gestures may differ in different countries or regions. Therefore, in practical applications, the parameters included in the gesture parameter set may be flexibly designed based on requirements.

In a possible design, for example, the preset standard is the OSI standard. The first parameter may be a red, green, and blue (RGB) value, where the RGB value represents an image corresponding to the first gesture. For example, the data analysis module acquires, based on the sensor model, a two-dimensional or three-dimensional image of a virtual object in a virtual traffic scenario within the field of view of the first virtual terminal, and, based on the image sensor, simulates the effect of capturing an image of a traffic police officer by a vehicle in the real world. Because the OSI standard supports data in an image format, the data analysis module may directly transmit the acquired image data (RGB values) to the first device as test data.

In a possible design, for example, the preset standard is the OSI standard. The first parameter may be a true value, and the true value indicates a first gesture. For example, Table 1 is an example of a gesture parameter set. In this example, multiple parameters included in the gesture parameter set are all true values.

It should be understood that Table 1 is merely an example and not a specific limitation. In practical applications, the types of true values and the gesture types corresponding to each true value can be flexibly set based on requirements.

Of course, in addition to RGB values and true values, the first parameter may be in another data format specifically determined by the data formats supported by the first device.

S102: The second device outputs at least one piece of test data to the first device, and in response, the first device receives at least one piece of test data.

Specifically, the data analysis module transmits at least one test data to the first device.

S103: The first device generates a first control signal based on at least one test data.

Specifically, the processing module of the first device identifies an event occurring in the virtual traffic scenario based on at least one test data, and then generates a first control signal based on the event, where the first control signal is used to control the first virtual terminal such that the first terminal responds to the event.

In a possible implementation, the processing module of the first device identifies a gesture (e.g., a first gesture) performed by a virtual object in a virtual traffic scenario based on gesture information (e.g., a first parameter) included in at least one test data, and then generates a first control signal based on the first gesture.

During a particular implementation, the processing module may integrate one or more of the following functions: perception (e.g., perceiving an event that occurred in the virtual traffic scenario based on test data, e.g., a first gesture), prediction (e.g., predicting an event that will occur in the future in the virtual traffic scenario based on the sensing results), planning (e.g., planning the driving trajectory of the first virtual terminal based on the sensing results and/or the prediction results), control (e.g., generating a control signal used to control the driving trajectory of the first virtual terminal), etc.

For example, see FIG. 2. A first gesture performed by a virtual object at an intersection is a left turn gesture, and a processing module of a first device generates a first control signal used to control a first virtual terminal to drive a left turn.

S104: The first device outputs a first control signal to the second device, and in response, the second device receives the first control signal.

S105: The second device controls the first virtual terminal based on the first control signal.

For example, an event occurring in the virtual traffic scenario is a left turn gesture, and the first control signal is used to control the first virtual terminal to turn left. The vehicle dynamic model obtains a dynamic response value of the first virtual terminal to the virtual traffic scenario (e.g., a driving speed or direction required to execute the left turn of the first virtual terminal) based on the first control signal (left turn signal) transmitted by the first device and the current attitude information of the first virtual terminal (e.g., the lane in which the first virtual terminal is located, or the distance between the first virtual terminal and the intersection), and updates the attitude information of the first virtual terminal based on the dynamic response value (e.g., controlling the first virtual terminal to drive and turn left).

It can be appreciated that after taking control of the first virtual terminal, the second device can determine the autonomous driving system's ability to identify and respond to the first gesture by detecting the status of the first virtual terminal. For example, after the vehicle dynamics model updates the attitude information of the first virtual terminal, the second device determines the updated attitude information of the first virtual terminal to determine whether the autonomous driving system accurately identifies and responds to the first gesture, thereby determining the ability of the autonomous driving system in the first device to identify and respond to the gesture.

It can be understood that the first gesture is used as an example above. In an actual testing process, the gesture may be constantly changed to test the autonomous driving system multiple times. In the testing process, the algorithm of the autonomous driving system may be further continuously adjusted based on the test results to improve the autonomous driving system's ability to identify and respond to gestures.

According to steps S101 to S105, the second device generates test data and, based on the test data, performs closed-loop testing of the autonomous driving system's ability to identify and respond to traffic police officer gestures, thereby improving the safety and intelligence of the autonomous driving of the vehicle.

It may be understood that in real life, gestures performed by traffic police officers may be targeted (or directional). For example, gestures performed by traffic police officers typically indicate vehicles driving in a direction facing the traffic police officer. In other words, gestures performed by traffic police officers are valid for vehicles facing the traffic police officer and invalid for vehicles not facing the traffic police officer. Thus, in this embodiment of the present application, the data transmission module may transmit at least one test data to the first device when the first gesture is valid for the first virtual terminal (e.g., a virtual object is facing the first virtual terminal), and/or the processing module of the first device may output a first control signal to the first device when it determines that the first gesture is valid for the first virtual terminal (e.g., a virtual object is facing the first virtual terminal). Of course, in addition to the relative position between the virtual object and the first virtual terminal, the determination conditions for determining that the first gesture is valid for the first virtual terminal may further include other determination conditions. This is not a limitation in the present application.

In a possible design, the gesture information may further include a second parameter indicating information regarding the lane in which the first virtual terminal is located. The information regarding the lane in which the first virtual terminal is located may include, for example, an identifier (assigned_lane_id) of the lane in which the first virtual terminal is located. In other words, the information regarding the lane in which the first virtual terminal is located may be provided as part of the gesture information for the autonomous driving system, allowing the autonomous driving system to identify and respond to the first gesture by referring to the information regarding the lane in which the first virtual terminal is located, thereby improving the reliability of the solution.

In a possible design, the gesture information may further include a third parameter, which indicates whether the first parameter (is_out_of_service) is valid. Correspondingly, when the third parameter indicates that the first parameter is valid, the first device may generate a first control signal based on the first gesture.

Optionally, if the third parameter indicates that the first parameter is invalid and the at least one test data further includes other information other than gesture information, the first device may output a control signal based on the other information.

In this way, whether the gesture information is valid can be set, and whether to test the autonomous driving system's ability to identify and respond to the gesture can be flexibly determined based on requirements, thereby conserving computing resources of the first device.

In a possible design, the at least one test data further includes one or more of the following information:

(1) Target identification information for the first gesture, i.e., identification information for the target performing the first gesture, for example, identification information for the virtual target shown in FIG. 2.

(2) Target position information of the first gesture (police_base), i.e., position information of the target performing the first gesture, for example, position information of the virtual target shown in FIG. 2. The position information may be longitude and latitude, coordinates, etc., and is not limited in this application.

(3) Model reference information for the first gesture, i.e., source information for the target on which the first gesture is to be performed, e.g., source information for the virtual target shown in FIG. 2, where the model reference information may optionally be a link used to indicate a file address for storing three-dimensional model data of the virtual target, or

(4) Source reference of the first gesture, i.e., information used to drive the virtual object to perform the first gesture. For example, when the first gesture is driven by a first event, the source reference of the first gesture may be the first event; when the first gesture is driven by road collection data, the source reference of the first gesture may be the road collection data; or when the first gesture is driven by a preset rule, the source reference of the first gesture may be the preset rule.

In this way, the second device provides the first device with information related to the execution target corresponding to the first gesture, enabling the first device to identify more information related to the first gesture, thereby further improving the reliability of the test solution.

In real life, in addition to gesture information, a vehicle may also perceive other information used to indicate traffic rules, such as traffic light information or road sign information. Taking this into consideration, in a possible design, at least one test data may further include traffic light information, road sign information, etc. The information may be consistent with the gesture information or may contradict the gesture information. This is not a limitation in this application.

If the traffic light information, road sign information, etc. matches the gesture information (e.g., the gesture information indicates a left turn gesture and the traffic light indicates that traffic may turn left, or the road on which the first virtual terminal is located has a left turn identifier), and the third parameter indicates that the first parameter is valid, the first device may output a first control signal based on any one of the aforementioned information.

If traffic light information, road sign information, etc. contradicts the first gesture (e.g., the first gesture is a left turn gesture and the traffic light indicates that traffic may proceed straight, or the road on which the first virtual terminal is located has only a straight-through identifier), and the third parameter indicates that the first parameter is valid, the first device may output a first control signal based on the priority information of the aforementioned information.

For example, if at least one test data item includes gesture information and traffic light information, the first device outputs a first control signal based on the traffic light information or gesture information, whichever has the higher priority.

Correspondingly, when the priority of the gesture information is higher than the priority of the traffic light information, the first device generates a first control signal based on the first gesture. For example, if the priority of the traffic light is lower than the priority of the gesture information, the traffic light information indicates going straight, and the gesture information indicates turning left, the first control signal output by the first device is used to control the first virtual terminal to turn left.

Conversely, when the priority of the gesture information is lower than the priority of the traffic light information, the first device generates a first control signal based on the traffic light information. For example, if the priority of the traffic light is higher than the priority of the gesture information, the traffic light information indicates going straight, and the gesture information indicates turning left, the first control signal output by the first device is used to control the first virtual terminal to go straight.

Optionally, the priority of different types of information in the at least one test data may be determined according to local traffic rules.

In this way, the second device can support conflict configuration between gesture information and other information and can test the autonomous driving system's ability to handle conflicting information, thereby improving the robustness of the autonomous driving system.

In real life, there may be other traffic participants around the vehicle, such as other vehicles, pedestrians, or buildings, which may affect the vehicle's perception of the traffic officer's gestures. In a possible design, at least one test data may further include interference information.

For example, the interference information includes obstacle information. The obstacle information is information about an object (i.e., an obstacle) that blocks the line of sight of the first virtual terminal when the first virtual terminal perceives the virtual object, including, but not limited to, the shape, color, contour, spatial position, etc. of the obstacle. It should be understood that in an occlusion scenario, the gesture information may be partially missing. For example, the test data is an image. See FIG. 4. In the image received by the first device, part of the virtual object is blocked by another vehicle. Therefore, the gesture information of the virtual object is incomplete.

Of course, the interference information is not limited to information about obstacles, but may be other information. For example, if the test data is an image, the second device may further simulate interference caused by the hardware performance of the image sensor in real life by reducing parameters such as the brightness and resolution of the image.

In this way, the second device can support the configuration of interference information and test the autonomous driving system's ability to process the interference information, thereby improving the robustness of the autonomous driving system.

In a possible design, the at least one test data further includes sensory data from a second virtual terminal, the sensory data including reference gesture information, the reference gesture information indicating the first gesture. The relative position between the second virtual terminal and the virtual object is different from the relative position between the first virtual terminal and the virtual object.

For example, the test data and the perception data are both images. See Figure 5. The image of the virtual object in the field of view of the first virtual terminal is different from the image of the virtual object in the field of view of the second virtual terminal. Because the line of sight of the first virtual terminal is blocked by the second virtual terminal, only a portion of the image of the virtual object is present. The line of sight of the first virtual terminal is not blocked by an object. Therefore, a complete image of the virtual object can be obtained.

Correspondingly, the first device may refer to both an image of the virtual object in the field of view of the first virtual terminal and an image of the virtual object in the field of view of the second virtual terminal to identify the gesture performed by the virtual object.

Of course, in addition to the sensory data from the second virtual terminal, the at least one test data may further include data from another device, for example, image data collected by a roadside camera. This is not a limitation of the present application.

In this way, the second device can provide gesture information of a virtual object in a different field of view to the first device, thereby testing the autonomous driving system's ability to identify gestures based on Vehicle to Everything (V2X) communication capabilities.

It will be understood that the above implementations may be implemented separately or in combination with each other. This is not a limitation of the present application.

The above describes the method provided in the embodiment of the present application with reference to the accompanying drawings, and the following describes the device provided in the embodiment of the present application with reference to the accompanying drawings.

Based on the same technical concept, an embodiment of the present application provides an information exchange device. The device includes modules/units/means configured to perform the methods performed by the second device and/or the first device in the aforementioned method embodiments. The modules/units/means may be implemented by software, hardware, or hardware executing corresponding software.

See, for example, FIG. 6. The device may include a transceiver module 201 and a processing module 202. The device is configured to implement the functionality of the second device or the first device.

For example, when the device is used as a first device, the transceiver module 201 is configured to receive at least one test data, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture, the first parameter belonging to a predetermined gesture parameter set. The processing module 202 is configured to generate a first control signal based on the at least one test data, the first control signal being used to control the first virtual terminal. The transceiver module 201 is further configured to output the first control signal.

Optionally, the gesture parameter set includes a plurality of parameters, the plurality of parameters indicating any of a stop gesture, a go straight gesture, a left turn gesture, a left turn waiting gesture, a right turn gesture, a right turn waiting gesture, a lane change gesture, a slow down gesture, a pull over gesture, or an unknown gesture.

Optionally, the first parameter is an RGB value, the RGB value representing an image corresponding to the first gesture, or the first parameter is a true value, the true value indicating the first gesture.

Optionally, the gesture information further includes a second parameter indicating information about the lane in which the first virtual terminal is located, and/or a third parameter indicating whether the first parameter is valid.

Optionally, the at least one test data further includes one or more of the following information:
target identification information of the first gesture;
target position information of the first gesture;
Target-referring information of the first gesture; or Target-driving information of the first gesture.

Optionally, the at least one test data further includes at least one of the following information:
Traffic light information, or obstacle information.

Optionally, the at least one test data further includes sensory data from a second virtual terminal, the sensory data including reference gesture information, the reference gesture information being indicative of the first gesture.

Optionally, the at least one test data includes gesture information and traffic light information. The processing module 202 is configured to generate the first control signal based on the priority information of the traffic light information and the priority information of the gesture information, whichever has a higher priority.

For example, when the device is used as a second device, the transceiver module is configured to output at least one test data to the first device, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture, the first parameter belonging to a predetermined gesture parameter set. The transceiver module 201 is further configured to receive a first control signal from the first device. The processing module 202 is configured to control the first virtual terminal based on the first control signal.

Optionally, the gesture parameter set includes a plurality of parameters, the plurality of parameters indicating any of the following gestures:
Stop gesture, go straight gesture, left turn gesture, left turn waiting gesture, right turn gesture, right turn waiting gesture, lane change gesture, slow down gesture, pull over gesture, or unknown gesture.

Optionally, the first parameter is an RGB value, the RGB value representing an image corresponding to the first gesture, or the first parameter is a true value, the true value indicating the first gesture.

Optionally, the gesture information further includes a second parameter indicating information about the lane in which the first virtual terminal is located, and/or a third parameter indicating whether the first parameter is valid.

Optionally, the at least one test data further includes one or more of the following information:
target identification information of the first gesture;
target position information of the first gesture;
Target-referring information of the first gesture; or Target-driving information of the first gesture.

Optionally, the at least one test data further includes at least one of the following information:
Traffic light information, or obstacle information.

Optionally, the at least one test data further includes sensory data from a second virtual terminal, the sensory data including reference gesture information, the reference gesture information being indicative of the first gesture.

Optionally, the processing module 202 is further configured to: control the virtual object to perform a first gesture when a first event is detected; or, based on the road collection data, control the virtual object to perform a gesture that is the same as a gesture in the road collection data, where the road collection data indicates that the real object would perform the first gesture in a real road scenario; or, control the virtual object to perform a specified gesture at a specified time, where the specified gesture includes the first gesture.

Optionally, the first event includes one or more of an event in which the distance between the virtual object and the first virtual terminal is less than a preset distance, a traffic jam, or a traffic accident.

It should be understood that all relevant content of the steps in the above-mentioned method embodiments can be cited in the functional descriptions of the corresponding functional modules. The details will not be described again here.

In specific implementations, the device may have multiple product forms. Some possible product forms are described below.

See FIG. 7. An embodiment of the present application further provides an information exchange device. The device includes at least one processor 301 and an interface circuit 302. The interface circuit 302 is configured to receive a signal from another device other than the device and transmit the signal to the processor 301, or to transmit a signal from the processor 301 to another communication device other than the device. The processor 301 is configured to implement a method performed by the second device or the first device by using logic circuits or executing code instructions.

It should be understood that the processors referred to in the embodiments of this application may be implemented using hardware or software. When the processor is implemented using hardware, the processor may be a logic circuit, an integrated circuit, etc. When the processor is implemented using software, the processor may be a general-purpose processor and is implemented by reading software code stored in a memory.

For example, a processor may be a Central Processing Unit (CPU), another general-purpose processor, a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

It should be understood that the memory referred to in the embodiments of this application may be volatile or nonvolatile memory, or may include both volatile and nonvolatile memory. Nonvolatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM) used as an external cache. By way of example and not limitation, many forms of RAM may be used, such as static random access memory (Static RAM, SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), Synchlink dynamic random access memory (Synchlink DRAM, SLDRAM), and direct Rambus random access memory (Direct Rambus RAM, DR RAM).

Note that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) may be integrated into the processor.

Note that memory as described herein is intended to include, but is not limited to, these and any other suitable types of memory.

Based on the same technical concept, one embodiment of the present application further provides a computer-readable storage medium containing a program or instructions. When the program or instructions are executed on a computer, the method executed by the second device or the first device is performed.

Based on the same technical concept, an embodiment of the present application further provides a computer program product including instructions. The computer program product stores the instructions, and when the instructions are executed on a computer, a method performed by the second device or the first device is performed.

Based on the same technical concept, an embodiment of the present application further provides a vehicle including the device shown in Figure 6 or Figure 7.

Those skilled in the art should understand that embodiments of the present application may be provided as a method, system, or computer program product. Thus, the present application may take the form of a hardware-only embodiment, a software-only embodiment, or an embodiment having a combination of software and hardware. In addition, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

This application is described with reference to flowcharts and/or block diagrams of methods, devices (systems), and computer program products herein. It should be understood that computer program instructions may be used to implement each process and/or each block in the flowcharts and/or block diagrams, and combinations of processes and/or blocks in the flowcharts and/or block diagrams. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or any other programmable data processing device to create a machine, whereby the instructions executed by the processor of the computer or any other programmable data processing device create an apparatus for implementing the particular functionality of one or more processes in the flowcharts and/or one or more blocks in the block diagrams.

These computer program instructions may be stored in a computer-readable memory that can instruct a computer or any other programmable data processing device to operate in a particular manner, whereby the instructions stored in the computer-readable memory generate an artifact that includes an instruction apparatus that implements a particular function in one or more processes in the flowcharts and/or one or more blocks in the block diagrams.

The computer program instructions may alternatively be loaded onto a computer or other programmable data processing device, such that a sequence of operations and steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing one or more specific functions in one or more procedures in the flowcharts and/or one or more blocks in the block diagrams.

It will be apparent that those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. This application is intended to cover these modifications and variations of this application, provided that they fall within the scope of protection defined by the following claims and their equivalents.

Claims (22)

1. A method for exchanging information, comprising:
receiving at least one test data, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture to be performed by a virtual object , the first parameter belonging to a predetermined gesture parameter set;
outputting a first control signal based on the at least one test data, the first control signal being used to control a first virtual terminal ;
The gesture information includes:
a second parameter indicating information about the lane in which the first virtual terminal is located; and/or
a third parameter indicating whether the first parameter is valid;
further comprising:
method. The gesture parameter set includes a plurality of parameters, the plurality of parameters being:
10. The method of claim 1, wherein the gesture indicates any of a stop gesture, a go straight gesture, a left turn gesture, a wait left turn gesture, a right turn gesture, a wait right turn gesture, a lane change gesture, a slow down gesture, a pull over gesture to indicate stopping on the shoulder , or an unknown gesture. the first parameter is an RGB value, and the RGB value represents an image corresponding to the first gesture; or the first parameter is a true value , and the true value indicates the first gesture.
3. The method according to claim 1 or 2. The at least one test data
target identification information, which is identification information of the virtual target that performs the first gesture;
target position information, which is position information of the virtual target that performs the first gesture;
4. The method of claim 1, further comprising one or more of: target reference information indicating source information of the virtual object performing the first gesture; or target driving information used to drive the virtual object performing the first gesture . The at least one test data
The method of claim 1 , further comprising at least one of: traffic light information; or obstacle information. 6. The method of claim 1, wherein the at least one test data further includes sensory data from a second virtual terminal, the sensory data including reference gesture information, the reference gesture information indicating the first gesture, and a relative position between the second virtual terminal and the virtual object is different from a relative position between the first virtual terminal and the virtual object . the at least one test data includes the gesture information and the traffic light information;
The step of outputting a first control signal based on the at least one test data includes:
The method of claim 5 , further comprising: outputting the first control signal based on which of the traffic light information and the gesture information has a higher priority. 1. A method for exchanging information, comprising:
outputting at least one test data to a first device, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture to be performed by a virtual object , the first parameter belonging to a predetermined gesture parameter set;
receiving a first control signal from the first device;
and controlling a first virtual terminal based on the first control signal ;
The gesture information includes:
a second parameter indicating information about the lane in which the first virtual terminal is located; and/or
a third parameter indicating whether the first parameter is valid;
further comprising:
method. The gesture parameter set includes a plurality of parameters, the plurality of parameters being:
10. The method of claim 8, wherein the gesture indicates any of a stop gesture, a go straight gesture, a left turn gesture, a wait left turn gesture, a right turn gesture, a wait right turn gesture, a lane change gesture, a slow down gesture, a pull over gesture to indicate stopping on the shoulder , or an unknown gesture. the first parameter is an RGB value, and the RGB value represents an image corresponding to the first gesture; or the first parameter is a true value , and the true value indicates the first gesture.
10. The method according to claim 8 or 9 . The at least one test data
target identification information, which is identification information of the virtual target that performs the first gesture;
target position information, which is position information of the virtual target that performs the first gesture;
11. The method of claim 8, further comprising one or more of: target reference information indicating source information of the virtual object performing the first gesture; or target driving information used to drive the virtual object performing the first gesture . The at least one test data
The method of claim 8 , further comprising at least one of: traffic light information; or obstacle information. 13. The method of claim 8, wherein the at least one test data further includes sensory data from a second virtual terminal, the sensory data including reference gesture information, the reference gesture information indicating the first gesture, and a relative position between the second virtual terminal and the virtual object is different from a relative position between the first virtual terminal and the virtual object . The method comprises:
14. The method of claim 8, further comprising: controlling the virtual object to perform the first gesture when a first event is detected, the first event including one or more of an event in which a distance between the virtual object and the first virtual terminal is less than a preset distance, a traffic jam, or a traffic accident; or controlling the virtual object to perform a gesture that is the same as a gesture in the road collection data based on road collection data, the road collection data indicating that a real object would perform the first gesture in a real road scenario; or controlling the virtual object to perform a specified gesture at a specified time, the specified gesture including the first gesture . An information exchange device,
a transceiver module configured to receive at least one test data, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture to be performed by a virtual object , the first parameter belonging to a predetermined gesture parameter set;
a processing module configured to generate a first control signal based on the at least one test data, the first control signal being used to control a first virtual terminal;
the transceiver module is further configured to output the first control signal ;
The gesture information includes:
a second parameter indicating information about the lane in which the first virtual terminal is located; and/or
a third parameter indicating whether the first parameter is valid;
further comprising:
Device. An information exchange device,
a transceiver module configured to output at least one test data to a first device, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture to be performed by a virtual object , the first parameter belonging to a predetermined gesture parameter set;
wherein the transceiver module is further configured to receive a first control signal from the first device;
a processing module configured to control a first virtual terminal based on the first control signal ;
The gesture information includes:
a second parameter indicating information about the lane in which the first virtual terminal is located; and/or
a third parameter indicating whether the first parameter is valid;
further comprising:
Device. 8. An information exchange device comprising a processor and an interface circuit, the interface circuit being configured to receive a signal from another device other than the device and transmit the signal to the processor, or to transmit a signal from the processor to another communication device other than the device, the processor being configured to implement the method of any one of claims 1 to 7 by using logic circuits or executing code instructions. 15. An information exchange device comprising a processor and an interface circuit, the interface circuit configured to receive a signal from another device other than the device and transmit the signal to the processor, or to transmit a signal from the processor to another communication device other than the device, the processor configured to implement the method of any one of claims 8 to 14 by using logic circuits or executing code instructions. A computer readable storage medium containing a program or instructions which, when run on a computer, performs the method of any one of claims 1 to 7 . A computer readable storage medium containing a program or instructions which, when run on a computer, performs the method of any one of claims 8 to 14 . An information exchange system comprising the device according to claim 15 and the device according to claim 16 , or the device according to claim 17 and the device according to claim 18 . A vehicle comprising the device of claim 15 , or the device of claim 17 , or the computer-readable storage medium of claim 19 .