JP7632449B2 - Vehicle interior/exterior linkage device and method - Google Patents

Description

This disclosure relates to an in-vehicle/out-of-vehicle collaboration device and method. This application claims priority to Japanese Application No. 2020-031119, filed on February 27, 2020, and all contents of said Japanese application are incorporated herein by reference.

When driving a vehicle, it is necessary to pay close attention not only to the movement of the vehicle itself but also to the movement of other vehicles. Particular attention is required when pedestrians are present in addition to vehicles. Conventionally, as shown in FIG. 1, there is a technology that detects moving objects (hereinafter referred to as "moving objects") that exist in such a real space 50 using a number of sensors such as LiDAR (Light Detection and Ranging) and cameras, estimates their attributes (adult, child, vehicle, motorcycle, etc.), and creates a traffic situation overview map 52 using high-resolution road map data (hereinafter referred to as "high-resolution map") prepared in advance in a virtual space.

In order to create such a traffic condition bird's-eye view map 52, it is necessary to collect sensor data, which is the output of a large number of sensors, from vehicles equipped with those sensors and from infrastructure sensors such as cameras installed on the roadside. For this purpose, it is considered to use a fifth-generation mobile communication system (so-called "5G"). One example of such technology is disclosed in Patent Document 1, which is listed below.

As an example of 5G, Figure 2 shows a communication system 70 including a first slice to a fourth slice.

Referring to FIG. 2, the first slice includes vehicles 82 and 84 equipped with on-board devices 92 and 94 capable of 5G wireless communication, an infrastructure camera 88 equipped with a wireless communication device 98, a traffic signal controller 90 equipped with a wireless communication device 100 provided at a traffic light, a mobile phone 96 held by a pedestrian 86, etc., and is a slice defined to enable direct communication between them.

The second slice is a slice defined to include multiple base stations 110, 112 and 114 and to communicate with the communication terminals included in the first slice.

The third slice includes metro NWs (Networks) 120 and 122 and edge servers 126 and 128 provided in distributed DCs (Data Centers) 124, 130, etc. connected to them, and is defined so that each communication terminal can communicate with them via base stations 110, 112, and 114, etc. Note that the metro NW refers to a communication network constructed to connect buildings, social facilities, houses, etc. in a limited area, such as a city. Each of the base stations 110, 112, and 114, etc. is connected to one of the edge servers 126, 128, etc. The edge servers 126, 128, etc. are called so because they are located at the outermost part of the system connected by the network and connect the network system with the rest.

The fourth slice includes a core NW 140 capable of communicating with multiple metro networks, and a core server 144 provided in a core DC 142 connected to this core NW 140 is capable of communicating with communication devices connected to each metro network, such as edge servers 126 and 128.

Typically, the traffic condition overhead map described above is constructed and maintained for a specific range in the edge server 128 or the like. Each communication terminal belonging to the first slice transmits sensor data detected by its sensor to, for example, the edge server 126. The edge server 126 reconstructs the actual road conditions in a virtual space by integrating the sensor data, and creates and maintains the traffic condition overhead map 52. Based on the traffic condition overhead map 52 maintained in this way, the edge server 126 transmits information to each communication terminal to assist in its driving, etc.

JP 2018-018284 A JP 2008-263580 A JP 2018-073389 A

The in-vehicle/external linkage device according to a first aspect of the present disclosure is an in-vehicle/external linkage device used in a vehicle equipped with an in-vehicle network for data transfer, a wireless communication device for wirelessly communicating data with the outside of the vehicle, and a plurality of sensors, and includes a data receiving unit that receives sensor data from the plurality of sensors via the in-vehicle network, an end-to-end delay time estimation unit that estimates an end-to-end transmission delay time of the sensor data from each of the plurality of sensors to a predetermined device with which communication is possible via the wireless communication device, a value determination unit that determines a value of the sensor data output by the plurality of sensors based on the state of the vehicle and the state of the plurality of sensors, and a selection unit that selects a portion of the sensor data based on the transmission delay time estimated by the delay time estimation unit and the value determined by the value determination unit, and transmits a copy of the selected sensor data to the predetermined device via the wireless communication device.

A vehicle-exterior collaboration method according to a second aspect of the present disclosure is a vehicle-exterior collaboration method for linking the inside and outside of a vehicle in a vehicle equipped with an in-vehicle network for data transfer, a wireless communication device for wirelessly communicating data with the outside of the vehicle, and multiple sensors, by transmitting sensor data from the multiple sensors to a specified external device, the method including the steps of: a computer receiving sensor data from the multiple sensors mounted in the vehicle via the in-vehicle network; a computer estimating a transmission delay time of the sensor data between the end of the sensor and the specified device with which communication can be performed via the wireless communication device; a computer determining a value of the sensor data output by the multiple sensors based on the state of the vehicle and the state of the multiple sensors; and a computer selecting a portion of the sensor data based on the transmission delay time estimated in the step of estimating the transmission delay time and the value determined in the step of determining the value, and transmitting a copy of the selected sensor data to the specified device via the wireless communication device.

The technical scope and content of this disclosure will become apparent upon review of the detailed description of the disclosure with reference to the accompanying drawings.

FIG. 1 is a diagram showing a schematic relationship between actual road conditions and a road condition bird's-eye view map. FIG. 2 is a diagram showing a schematic configuration of the communication system disclosed in Patent Document 1. In FIG. FIG. 3 is a diagram showing a schematic configuration of a traffic assistance system according to the first embodiment of the present disclosure. FIG. 4 is a block diagram showing a configuration of elements related to a traffic assistance system in a vehicle according to the first embodiment of the present disclosure. FIG. 5 is a block diagram showing a configuration of an edge server that is a traffic assistance server that communicates with vehicles according to the first embodiment of the present disclosure. FIG. 6 is a block diagram illustrating a schematic configuration for communicating the sensor data of the vehicle according to the first embodiment. FIG. 7 is a block diagram showing a schematic hardware configuration of the vehicle interior/exterior linking unit used in the vehicle according to the first embodiment shown in FIG. FIG. 8 is a diagram showing data required for in-vehicle/out-of-vehicle cooperation, which is stored in the memory in the in-vehicle/out-of-vehicle cooperation unit shown in FIG. FIG. 9 is a diagram showing an example of the contents stored in the sensor data set table shown in FIG. FIG. 10 is a diagram showing an example of the sensor priority policy table shown in FIG. FIG. 11 is a diagram showing, in a table format, the correspondence relationship between the sensor types and the sensor type numbers in the vehicle according to the first embodiment. FIG. 12 is a diagram showing, in a table format, the correspondence between the sensor position type indicating the position of a sensor mounted on a vehicle and the sensor position type number. FIG. 13 is a schematic diagram showing an example of a mounting position of a camera in a vehicle. FIG. 14 is a diagram for explaining the priority of sensors based on the value of the sensor data. FIG. 15 is a flowchart showing a control structure of a program for controlling communication from a vehicle, executed by the transportation assistance server according to the first embodiment of the present disclosure. FIG. 16 is a flowchart showing a control structure of a program for controlling transmission of sensor data, executed by an on-board device mounted on a vehicle in the transportation assistance system in accordance with the first embodiment. FIG. 17 is a flowchart showing a control structure of a program for estimating transmission delay, executed by an on-board device mounted in a vehicle in the traffic assistance system according to the first embodiment. FIG. 18 is a flowchart showing a control structure of a program for updating the priority policy of a sensor, executed by an on-board device mounted in a vehicle in the transportation assistance system according to the first embodiment. FIG. 19 is a flowchart showing a control structure of a program for implementing sensor data selection and switch setting processing, executed by an on-board device mounted on a vehicle in the transportation assistance system in accordance with the first embodiment. FIG. 20 is a block diagram showing a schematic configuration of a traffic assistance system according to a second embodiment of the present disclosure. FIG. 21 is a block diagram showing a configuration of elements related to a traffic assistance system in a vehicle according to a second embodiment of the present disclosure. FIG. 22 is a flowchart showing a control structure of a program for controlling transmission of sensor data, executed by an on-board device mounted on a vehicle in the transportation assistance system in accordance with the second embodiment.

[Problem to be solved by this disclosure]
A communication system such as that disclosed in Patent Document 1 can construct and maintain a traffic condition bird's-eye view map 52. Dynamic information and caution information related to traffic obtained from the traffic condition bird's-eye view map 52 are transmitted to each communication terminal. As a result, for example, a vehicle equipped with such a communication terminal can know the road conditions when necessary, which can be useful for traffic safety.

However, the above-mentioned conventional technologies still have problems to be solved. The first problem is how to deal with dynamic fluctuations in the transmission capacity from the vehicle to a traffic support server such as an edge server. Large fluctuations in the transmission capacity may occur due to switching between 4G/5G areas and shadowing. In addition, large fluctuations in the transmission capacity may occur due to the internal state of the vehicle (in-vehicle network, load state of CPU (Central Processing Unit) resources) and changes in the state of the traffic support server. The traffic support system needs to appropriately deal with such dynamic fluctuations in the transmission capacity. The second problem is the selection of sensor data. If it is not possible to transmit all the sensor data from the vehicle to the traffic support server due to fluctuations in the transmission capacity, it becomes necessary to select and transmit part of the sensor data. To do this, it is necessary to determine what kind of sensor data should be transmitted from the vehicle to the traffic support server most efficiently.

There is no prior art that solves both the first and second problems. For example, the aforementioned Patent Document 2 proposes a technology that switches the type, resolution, and frame rate of images from a camera mounted on a vehicle in response to changes in transmission capacity. However, the technology described in Patent Document 2 does not take into account the state inside the vehicle-mounted device when performing such control. Furthermore, the technology described in Patent Document 2 does not take into account what conditions a sensor should be selected under depending on the various situations in which the vehicle is placed, i.e., the value of the sensor data.

On the other hand, the aforementioned Patent Document 3 discloses a technology for determining the priority when transmitting a data sample from a vehicle to a server based on the value of the data sample. In Patent Document 3, the value of the data sample is calculated based on a data value map in a table called a data value table. Based on the calculated value, the priority of transmitting the data sample to the server is determined. The data value map is determined, for example, based on the purpose for which the data sample will be used in the server, and is transmitted to the vehicle as needed. When the vehicle receives the data value map, the vehicle updates the existing data value map with the received data. Therefore, from the server's perspective, data can be collected from the vehicle with a priority according to the purpose of the data.

However, Patent Document 3 does not disclose any technology that changes the method of calculating the value of data in real time based on the situation in which the vehicle is placed. In particular, in a traffic assistance system, the value of data does not change depending on the use, but the priority of data to be sent to the server should always be processed in real time according to the situation of the vehicle and its surroundings. Patent Document 3 does not disclose such technology.

Therefore, the object of this disclosure is to provide an in-vehicle/external connection device and method that can determine the priority of data transmission according to changes in transmission capacity and the conditions of the vehicle and its surroundings, so that traffic assistance can be effectively provided by a specified device such as a traffic assistance server.

Description of the embodiments of the present disclosure
In the following description and drawings, the same parts are denoted by the same reference numerals. Therefore, detailed description thereof will not be repeated. Note that at least some of the following embodiments may be combined in any desired manner.

(1) An in-vehicle/external linkage device according to a first aspect of the present disclosure is an in-vehicle/external linkage device used in a vehicle equipped with an in-vehicle network for data transfer, a wireless communication device for wirelessly communicating data with the outside of the vehicle, and a plurality of sensors, and includes a data receiving unit that receives sensor data from the plurality of sensors via the in-vehicle network, an end-to-end delay time estimation unit that estimates an end-to-end transmission delay time of the sensor data from each of the plurality of sensors to a predetermined device with which communication is possible via the wireless communication device, a value determination unit that determines a value of the sensor data output by the plurality of sensors based on the state of the vehicle and the state of the plurality of sensors, and a selection unit that selects a portion of the sensor data based on the transmission delay time estimated by the delay time estimation unit and the value determined by the value determination unit, and transmits a copy of the selected sensor data to the predetermined device via the wireless communication device.

The end-to-end delay time estimation unit estimates the end-to-end transmission delay time of the sensor data from each of the multiple sensors mounted on the vehicle to a specified device. The value determination unit determines the value of the sensor data output by the multiple sensors based on the state of the vehicle and the state of the multiple sensors. The selection unit selects a portion of the sensor data based on the transmission delay time estimated by the delay time estimation unit and the value determined by the value determination unit, and transmits a copy of the selected sensor data to the specified device via the wireless communication device. The end-to-end transmission delay time from the sensor to a specific device, including the delay time in the in-vehicle network, is estimated, and the sensor data to be transmitted to the specified device is selected based on the estimated transmission delay time and the value determined for each of the sensor data. The in-vehicle/external linkage device selects and transmits sensor data that can be effectively used by the specified device, taking into account the end-to-end transmission delay time from the sensor to the specified device while following the fluctuations in the transmission capacity of the wireless communication device, so that it is possible to provide an in-vehicle/external linkage device that can maximize the transmission capacity while demonstrating the functions of the specified device as much as possible.

(2) The end-to-end delay time estimation unit may include a network transmission time estimation unit that estimates, for each sensor, the network delay time it takes for sensor data from the multiple sensors to reach the data receiving unit via the in-vehicle network; a processing time estimation unit that estimates the processing time required for processing the sensor data that has reached the data receiving unit until it is input into the wireless communication device; a wireless communication delay time estimation unit that estimates the wireless communication delay time it takes for the sensor data to be transmitted from the wireless communication device to a specified device; and a delay time calculation unit that calculates a transmission delay time based on the data amount, network delay time, processing time, and wireless communication delay time of each of the multiple sensors.

In addition to the processing time required for the sensor data that has arrived at the data receiving unit to be processed and input into the wireless communication device, and the wireless communication delay time required for the sensor data to be transmitted from the wireless communication device to a specified device, the network delay time required for the sensor data from multiple sensors to reach the data receiving unit via the in-vehicle network is estimated for each sensor, and the end-to-end transmission delay time is calculated based on these. Since the end-to-end transmission delay time is calculated taking into account the transmission delay time of the in-vehicle network, the amount of data that can be transmitted to a specified device within the allowable delay time can be more accurately estimated, and the sensor data to be transmitted can be more accurately selected.

(3) The value determination unit may include an infrastructure detection range estimation unit that estimates the detection range of an infrastructure sensor present in the vicinity of the vehicle based on the vehicle's position, and a value setting unit that estimates the detection ranges of multiple sensors based on the vehicle's position and attitude, and sets the value of the sensor data from each sensor so that the smaller the area of overlap with the detection range of the infrastructure sensor estimated by the infrastructure detection range estimation unit, the greater the value of the sensor data from that sensor.

Among the multiple sensors installed in a vehicle, sensor data from a sensor with a large detection range that overlaps with the detection range of an infrastructure sensor has low utility value for a specified device. The smaller the detection range of a sensor that overlaps with the detection range of an infrastructure sensor, the more valuable the sensor data it outputs is for a specified device. Therefore, by estimating the detection ranges of infrastructure sensors present in the vicinity of the vehicle and calculating the value of sensors whose detection ranges overlap little with the detection ranges of infrastructure sensors as high, the sensor data from these sensors is preferentially selected and transmitted to the specified device. Because high-value sensor data is transmitted to the specified device within limited transmission capacity, stable processing based on the sensor data can be performed in the specified device regardless of fluctuations in transmission capacity.

(4) The infrastructure detection range estimation unit may include an infrastructure detection range update unit that repeatedly updates the estimated detection range of an infrastructure sensor present in the vicinity of the vehicle based on the position of the vehicle at a predetermined time interval, and the value setting unit may include a value update unit that repeatedly updates the value of the sensor data from each sensor based on the position and attitude of the vehicle at a predetermined time interval so that the value of a sensor is greater for a sensor that has a smaller area of overlap with the detection range of the infrastructure sensor updated by the infrastructure detection range update unit.

The detection range of the infrastructure sensor is updated at a predetermined time interval. The value of the sensor data is also updated at a predetermined time interval based on the vehicle's position and attitude and the updated detection range of the infrastructure sensor. Therefore, the value of each sensor of the vehicle is updated at least at a predetermined time interval in accordance with the movement of the vehicle and the communication environment around the vehicle. As a result, regardless of the movement of the vehicle and changes in the communication environment around the vehicle, high-value sensor data is transmitted to a specified device. Therefore, regardless of changes in the environment, the specified device can perform stable processing.

(5) Each of the multiple sensors may be classified into one of multiple sensor types, and the infrastructure detection range estimation unit may include a sensor type detection range estimation unit that estimates the detection range of infrastructure sensors present in the vicinity of the vehicle for each sensor type of the infrastructure sensor based on the position of the vehicle, and the value setting unit may include a sensor type sensor data value setting unit that estimates, for each of the multiple sensor types, the detection range of each sensor belonging to that sensor type among the multiple sensors based on the position and attitude of the vehicle, and sets the value of sensor data from a sensor belonging to that sensor type among the multiple sensors such that the smaller the area of overlap with the detection range of the infrastructure sensor estimated for that sensor type by the sensor type detection range estimation unit, the greater the value.

The value setting unit sets the value of sensor data for each type of sensor, making more effective use of transmission capacity and allowing the selection of sensor data that is effective for processing by a specified device.

(6) The vehicle interior/exterior collaboration device may further include a sensor dataset storage unit that stores information specifying multiple sensor datasets together with an identifier for each sensor dataset, and the multiple sensor datasets may include information specifying, for each of multiple sensor types, the number of sensors to select from the multiple sensors and a data format related to the amount of sensor data from the sensors. The vehicle interior/exterior collaboration device may further include a sensor dataset reading unit that, in response to receiving one of the identifiers from a specific device, reads information about the sensor dataset corresponding to the identifier from the sensor dataset storage unit, a sensor data selection unit that selects, for each of the multiple sensor types, sensor data from the number of sensors specified by the information about the sensor dataset read by the sensor dataset reading unit in order of decreasing value based on the information about the sensor dataset read by the sensor dataset reading unit and the value set by the sensor type sensor data value setting unit for the sensor data from each sensor, and a network setting unit that sets the in-vehicle network to transfer the sensor data selected by the sensor data selection unit in priority to other sensor data.

When an identifier specifying one of the sensor data sets is given to the in-vehicle/out-of-vehicle integration device, the number of sensors specified by the sensor data set is selected for each sensor type in order of the sensor data's value. For all types of sensors, a predetermined number of valuable sensor data items are transmitted to a specified device based on the sensor data set designation. As a result, the specified device can effectively use the sensor data to perform stable processing.

(7) The vehicle may further include a vehicle control device that controls the vehicle using sensor data from the multiple sensors, and the vehicle control device may include an autonomous driving ECU (Electronic Control Unit) that performs autonomous driving of the vehicle using sensor data received from the multiple sensors via a network.

The autonomous driving ECU operates based on sensor data from multiple sensors installed in the vehicle. One of these pieces of sensor data is selected by the vehicle interior/exterior connection device and sent to a specific device, allowing the autonomous driving ECU and the specific device to share the sensor data, efficiently performing autonomous driving of the vehicle and processing of the specific device.

(8) The vehicle control device may include a remote control ECU that controls the vehicle in accordance with predetermined remote control commands, a sensor data transmission unit that transmits sensor data received from a plurality of sensors via a network to an external remote control device via a wireless communication device, and an input device that inputs remote control commands received from the remote control device via the wireless communication device to the remote control ECU.

The remote control device operates based on sensor data received from multiple sensors mounted on the vehicle and transmits remote control commands to the remote control ECU. Any of the sensor data transmitted to the remote control device is selected by the vehicle interior/exterior linkage device and transmitted to a specified device. Therefore, sensor data can be shared between the remote control device and the specified device, enabling efficient remote control of the vehicle and execution of processing by the specified device.

(9) A vehicle-exterior collaboration method according to a second aspect of the present disclosure is a vehicle-exterior collaboration method for linking the inside and outside of the vehicle in a vehicle equipped with an in-vehicle network for data transfer, a wireless communication device for wirelessly communicating data with the outside of the vehicle, and a plurality of sensors by transmitting sensor data from the plurality of sensors to a specified external device, the method including the steps of: a computer receiving sensor data from the plurality of sensors mounted in the vehicle via the in-vehicle network; a computer estimating a transmission delay time of the sensor data between the end of the sensor and the specified device with which communication can be performed via the wireless communication device; a computer determining a value of the sensor data output by the plurality of sensors based on a state of the vehicle and a state of the plurality of sensors; and a computer selecting a portion of the sensor data based on the transmission delay time estimated in the step of estimating the transmission delay time and the value determined in the step of determining the value, and transmitting a copy of the selected sensor data to the specified device via the wireless communication device.

In the step of estimating the end-to-end transmission delay time of the sensor data, the end-to-end transmission delay time of the sensor data from each of the multiple sensors mounted on the vehicle to a specified device is estimated. Based on the state of the vehicle and the state of the multiple sensors, the value of the sensor data output by the multiple sensors is determined. Based on the estimated transmission delay time and the value determined in the value determining step, a portion of the sensor data is selected, and a copy of the sensor data is transmitted to the specified device via the wireless communication device. The end-to-end transmission delay time from the sensor to a specific device is estimated, including the delay time in the in-vehicle network, and the sensor data to be transmitted to the specified device is selected based on the estimated transmission delay time and the value determined for each of the sensor data. While following the fluctuation in transmission capacity due to wireless communication, sensor data that can be effectively used by the specified device is selected and transmitted, taking into account the end-to-end transmission delay time from the sensor to the specified device, so that a method of in-vehicle/external collaboration can be provided that can maximize the transmission capacity while demonstrating the function of the specified device as much as possible.

[Effects of this disclosure]
As described above, according to this disclosure, it is possible to provide an in-vehicle/out-of-vehicle connection device and method that can determine the priority of data transmission according to changes in transmission capacity and the conditions of the vehicle and its surroundings, so that traffic assistance can be effectively provided by a specified device such as a traffic assistance server.

First Embodiment
<composition>
Overall composition
A schematic configuration diagram of the traffic assistance system according to this disclosure is shown in Fig. 3. Referring to Fig. 3, this traffic assistance system includes vehicles 82 and 84, infrastructure sensors (not shown), mobile phones carried by pedestrians, and an edge server 128 that is a traffic assistance server that communicates with these via a base station 110 and creates and maintains a communication status map 152 and a traffic status bird's-eye view map 150, similar to the system described in Patent Document 1.

The communication status map 152 is, for example, a high-definition map prepared in advance for traffic assistance, divided into a grid of multiple sections with sides of a certain length, and the communication status in each section is recorded. Typically, the communication status for each section is represented by the center position of the section and a representative value of the transmission speed of data transmitted to the traffic assistance server from vehicles in the section during a certain period of the past. In order to calculate the representative value of the transmission speed for a certain period of the past, data on the transmission speed of data received by the traffic assistance server during a certain period of the past is also accumulated. As the representative value of the transmission speed, typically, the average transmission speed, the most frequent value of the transmission speed, the median value of the transmission speed, representative values calculated from data excluding a certain number of the top and bottom, etc. can be used.

A traffic condition overhead map is a high-definition map prepared in advance in virtual space for a certain area, which stores the positions, speeds, attributes, etc. of moving objects such as vehicles and pedestrians and fixed objects such as buildings in the area in association with identification numbers. The traffic condition overhead map is constructed based on data transmitted from so-called infrastructure sensors installed in the area and on-board sensors installed in vehicles, as well as information obtained from specified sources regarding road construction, accidents, the locations of infrastructure sensors and their attributes, etc.

Below, the first embodiment is described as an example of how to transmit sensor data from a vehicle 82 to an edge server 128.

<<In-vehicle device>>
Referring to FIG. 4 , the vehicle 82 includes various sensors 170 and an autonomous driving ECU (Electronic Control Unit) 172 for controlling autonomous driving of the vehicle, an in-vehicle device 92 for collecting sensor data from the sensors 170 and controlling the autonomous driving ECU 172 while communicating with the autonomous driving ECU 172, and an exterior communication device 162 used by the in-vehicle device 92 when communicating with elements outside the vehicle, such as an edge server 128 and other vehicles.

The in-vehicle device 92 includes a network switch (NSW) 160 for in-vehicle communication to which the sensor 170 and the automatic driving ECU 172 are connected, and an in-vehicle/out-of-vehicle linking unit 164 that is connected to the sensor 170 and the automatic driving ECU 172 via the NSW 160, transmits sensor data from the sensor 170 to the outside via the external communication device 162, and controls the operating state of the automatic driving ECU 172, etc., in cooperation with information collected from the in-vehicle sensors and the like and data received from the outside via the external communication device 162. As described below, the in-vehicle/out-of-vehicle linking unit 164 controls the process of collecting sensor data and the process of transmitting sensor data to the outside in response to an instruction from the edge server 128 and a change in the situation of the vehicle 82. The in-vehicle device 92 shown in FIG. 4 includes the NSW 160 and the in-vehicle/out-of-vehicle linking unit 164, but may be a single in-vehicle device that includes the automatic driving ECU 172 or the external communication device 162, or both. The in-vehicle device 92 may be realized as a semiconductor integrated circuit such as an ASIC (Application-Specific Integrated Circuit) or a system LSI (Large Scale Integration), or as a device in which a plurality of semiconductor integrated circuits are mounted on a substrate.

Typically, sensor data from the sensor 170 is transmitted to the autonomous driving ECU 172, but as indicated by the arrow 174, a portion of the sensor data is selected by the vehicle interior/exterior linkage unit 164 and transmitted to the edge server 128. In the following embodiment, how the vehicle interior/exterior linkage unit 164 selects sensor data that enables the edge server 128 to most efficiently construct the traffic condition bird's-eye view map 150, depending on the transmission capacity between the exterior communication device 162 and the edge server 128, the processing time in the network within the vehicle 82 and the vehicle interior/exterior linkage unit 164 and other units, and the situation in which the vehicle 82 is placed, will be described.

The number of sensors mounted on vehicles is now very large, and is expected to continue to increase in the future, as well as the amount of data output from each sensor. For this reason, it is believed that the transmission speed of conventional in-vehicle networks will no longer be sufficient, and the use of networks with gigabit-class transmission speeds for in-vehicle networks is being considered. As will be described later with reference to FIG. 6, in the following embodiment, the in-vehicle devices of the vehicle 82 communicate with each other via such a network. Existing technology related to networks connecting computers can be applied to such networks, and it is expected that the cost of building an in-vehicle network will therefore decrease. In this example, TCP-IP (Transmission Control Protocol-Internet Protocol), which is currently the most widely used communication protocol for in-vehicle networks, is adopted.

<<Transportation Support Server>>
5, the edge server 128 includes a reception processing unit 210 for receiving signals from a plurality of infrastructure sensor equipment 180 (including any combination of cameras, millimeter wave radar, and LiDAR) and signals from sensors (any combination of cameras, LiDAR, and millimeter wave radar) 170 mounted on the vehicle 82 as described above. Each of the infrastructure sensor equipment 180 includes an infrastructure sensor 190 including any combination of cameras, millimeter wave radar, and LiDAR, and a communication device 192 for transmitting signals output from the infrastructure sensor 190 to the reception processing unit 210 of the edge server 128. Similarly, the vehicle 82 includes a sensor 170 including a camera, LiDAR, or millimeter wave radar, and an exterior communication device 162 for transmitting at least a part of the signals output from the sensor 170 to the reception processing unit 210 of the edge server 128.

The edge server 128 further includes a moving object tracking unit 212 which determines and tracks the position of each moving object in a predetermined first period by analyzing ranging signals from LiDAR, millimeter wave radar, etc. in the signals received by the receiving processing unit 210, and outputs the moving object tracking result 213, a moving object tracking result memory unit 214 for storing the moving object tracking result 213, and an attribute detection unit 216 which performs image analysis on image data from the camera in the signals received by the receiving processing unit 210 to determine the attributes and positions of moving objects such as vehicles and people in the image in a predetermined second period longer than the first period. The edge server 128 further includes an attribute memory unit 218 for storing attributes 217 output by the attribute detection unit 216, an integration processing unit 224 for repeatedly integrating the moving object tracking results 213 stored in the moving object tracking result memory unit 214 and the attributes 217 stored in the attribute memory unit 218 at a period shorter than the second period and outputting a traffic condition overhead map 225 which is the analysis result after the integration, and a traffic condition overhead map memory unit 226 for accumulating and storing the traffic condition overhead map 225 output by the integration processing unit 224.

The moving object tracking results 213, attributes 217 and traffic condition overhead map 225 are each calculated at a predetermined time interval, but analysis results calculated at a certain time in the past are also accumulated and stored as history in the moving object tracking result memory unit 214, attribute memory unit 218 and traffic condition overhead map memory unit 226. When the integration processing unit 224 performs integration processing, it may refer to the history of the traffic condition overhead map 225, which is the past integrated analysis result accumulated in the traffic condition overhead map memory unit 226.

The edge server 128 further includes a vehicle tracking unit 220 for obtaining vehicle information consisting of identification information of the managed vehicle, on-board sensor information, and vehicle attributes such as position, speed, and vehicle size based on the signal received by the receiving processing unit 210 from each vehicle, and a vehicle information storage unit 222 for storing vehicle information 221 of each vehicle analyzed by the vehicle tracking unit 220.

The edge server 128 further includes an information transmission unit 228 which performs processing such as comparing the moving object information of the traffic condition bird's-eye view map 225 with the vehicle information 221 stored in the vehicle information storage unit 222 and reporting information for traffic assistance, such as information on moving objects with attributes that are considered to be dangerous, such as children and pedestrians walking while looking at their smartphones, accident vehicles on the road, broken-down vehicles, fallen objects, etc., to vehicles located within a specified range of the object in question, and a transmission processing unit 230 for transmitting a signal for the information report by the information transmission unit 228 to the target vehicle.

The edge server 128 further includes a high-resolution map storage unit 262 that stores a high-resolution map, and a high-priority area extraction unit 240 for extracting coordinates that identify a high-priority area based on the traffic situation overhead map stored in the traffic situation overhead map storage unit 226 and the high-resolution map stored in the high-resolution map storage unit 262. Here, a high-priority area is an area for which sensor data related to that area needs to be collected preferentially. Examples of high-priority areas include areas in which there are a number of moving objects greater than or equal to a predetermined threshold, and areas in which moving objects with attributes that may behave dangerously, such as children, have been detected. The high-priority area extraction unit 240 extracts high-priority areas by, for example, dividing a road map into predetermined sections and determining whether each section meets the above-mentioned conditions. For the detection of an area where moving objects greater than or equal to the threshold are gathered, multiple areas (e.g., intersections) that should be noted on the road map may be determined in advance, and only those areas may be examined to see whether they satisfy the conditions.

The edge server 128 further includes a sensor priority policy storage unit 242 that stores a plurality of sensor priority policies described later, a policy determination unit 244 that determines which of the sensor priority policies stored in the sensor priority policy storage unit 242 to apply to each of the high priority areas extracted by the high priority area extraction unit 240 according to the situation of the extracted area, and a candidate vehicle selection unit 246 that selects vehicles that include the area in the sensor detection range of the area extracted by the high priority area extraction unit 240 as candidates for target vehicles for sensor data collection. That is, in the high priority area, the edge server 128 collects sensor data from only some of the vehicles. These some of the vehicles are vehicles equipped with high-performance sensors and transmission devices. By selecting such vehicles and collecting sensor data, the edge server 128 can collect the necessary sensor data with a margin even when there are many vehicles in the high priority area and the communication situation may deteriorate.

The sensor priority policy indicates a guideline for which type of sensor data from which sensors should be prioritized among sensor data. For example, suppose that the presence of a child is detected in a certain area. Such an area is referred to as a child detection area here. Unlike adults, children can suddenly make unexpected movements. Therefore, in a child detection area, it is necessary to obtain information about the position of the detected moving object at a high frequency. Therefore, it is necessary to transmit sensor data from a sensor that can detect the position of a moving object quickly, such as LiDAR, to the edge server 128 in priority over sensor data from a sensor that is difficult or takes a long time to detect the position of a moving object, such as a camera. On the other hand, for example, consider an area where an accident vehicle exists on the road. Here, such an area is referred to as an accident area. Since an accident vehicle does not usually move, there is no need to transmit the position coordinates to the edge server 128 at a high frequency. In addition, in order to know the situation of the accident, it is easier for the driver of each vehicle to understand the situation if it is distributed in the form of an image, such as a camera. Therefore, in such an area, it is better to prioritize a camera over a LiDAR. In this manner, the sensor priority policy provides a basic guideline as to what sensor should be preferentially transmitted to the edge server 128 depending on the situation in each area.

In this embodiment, it is assumed that each vehicle transmits sensor data to the edge server 128 in all areas other than the high priority area, and at that time, transmits coordinates (relative coordinates with respect to the vehicle's base position) that specify the detection range of the vehicle's sensor as vehicle information. Using these coordinates and the coordinates of the vehicle's base position, the absolute coordinates of the range that can be detected by each vehicle's sensor are calculated, and candidate vehicles can be selected by comparing these coordinates with the coordinates of each area.

The edge server 128 further includes a dataset query unit 250 for transmitting the sensor priority policy determined by the policy determination unit 244 for each of the candidate vehicles selected by the candidate vehicle selection unit 246, and for querying a sensor dataset table consisting of the sensor dataset that each vehicle transmits to the edge server 128 in accordance with the sensor priority policy; a sensor dataset table storage unit 248 for storing, for each vehicle, the sensor dataset table obtained from each candidate vehicle in response to the query by the dataset query unit 250; a vehicle selection unit 252 for selecting a vehicle that can transmit the most efficient sensor data for the edge server 128 to create a traffic condition overview map based on these sensor dataset tables, and transmitting an instruction to the selected vehicle to request the transmission of sensor data; and a query transmission/reception unit 254 for communicating with each vehicle by the dataset query unit 250 and the vehicle selection unit 252 via wireless communication.

Various standards can be considered from the designer's point of view regarding the transmission of the most effective sensor data for creating a traffic situation overview map. For example, it can be considered that the most important data from the viewpoint of driving support for traffic participants, that is, valuable data leading to accident prevention, is transmitted to the edge server 128 at a transmission speed sufficient for that purpose. From another viewpoint, it can be considered that the sensor data is transmitted to the edge server 128 so that the contents of the traffic situation overview map are as accurate as possible and can follow actual changes at a sufficient speed. In that case, it can be considered important to divide the traffic situation overview map into several areas, change the importance of each area, and make it possible to reflect changes in the traffic situation overview map at a higher following speed for important areas than for other areas. In short, it is equivalent to transmitting the "most effective sensor data most efficiently" to the edge server 128 to the edge server 128 in order to be able to send effective data from the viewpoint most important to the system designer in maintaining and managing the traffic situation overview map, and to satisfy the criteria set so as to prevent the load on the edge server 128 from becoming excessive.

5, the edge server 128 further includes a line speed information storage unit 256 that extracts line speed information when transmitting data from each section to the edge server 128 based on the sensor data and vehicle information, etc., received by the reception processing unit 210 from the vehicles in each section of the high-resolution map, and stores the line speed information for a predetermined period of time, a communication status map management unit 258 for creating and managing a communication status map showing the communication status for each section of the high-resolution map stored in the high-resolution map storage unit 262 based on the line speed information stored in the line speed information storage unit 256, and a communication status map storage unit 260 for storing the communication status map created by the communication status map management unit 258. When the data set inquiry unit 250 requests a transmission data set table set from each vehicle, it refers to this communication status map.

In FIG. 5, the inquiry transmission/reception unit 254 may use the same hardware as the reception processing unit 210 and the transmission processing unit 230. Whether or not the sensor priority policy is met can be determined quantitatively, but in this embodiment, it is determined in advance using a sensor data set table corresponding to each sensor priority policy as described later. The sensor data set table can be created based on subjective criteria, or it can be created by formulating a formula based on various factors such as transmission capacity such as line speed, the number of high priority areas that the traffic situation bird's-eye map is paying attention to, the position of the vehicle in that area, the capacity of the transmission equipment of each vehicle, the number of moving objects present in the area, and the like, and comparing the values obtained by applying these factors to this formula. However, in reality, it is more realistic to use a method using the sensor data set table described below.

In this embodiment, it is assumed that a sensor dataset table is prepared in advance for each vehicle. Each vehicle transmits these sensor dataset tables to the dataset query unit 250 in response to a query from the dataset query unit 250. The sensor dataset tables will be described later with reference to FIG. 9.

The sensor data set table to be used may be switched depending on the sensor priority policy adopted. Each vehicle or infrastructure sensor may have at least a sensor for detecting the position of an object, such as LiDAR or millimeter wave radar, and a sensor for acquiring images, such as a camera. In this embodiment, intersections with many moving objects and areas where children are present are both extracted as high priority areas at the same level. However, this disclosure is not limited to such an embodiment. Intersections are rather wide areas, while areas where children are detected are narrow areas, and the characteristics of the two are different. Therefore, only one of them may be extracted. Also, a wide area may be extracted first, and then a narrow area may be extracted from that.

Note that for both image sensors such as cameras and distance sensors such as LiDAR and millimeter wave radar, the level of definition (resolution) and amount of sensor data vary depending on the product specifications. Therefore, it is necessary to prepare a sensor data set table that is compatible with each sensor priority policy, taking into full consideration the type of sensor in question, the resolution of the sensor data, and the transmission interval.

6 shows an example of the sensor arrangement and network configuration of the vehicle 82. Referring to FIG. 6, the vehicle 82 includes an in-vehicle network 288 having a gigabit-class transmission speed to which the above-mentioned in-vehicle/outdoor linking unit 164 and the autonomous driving ECU 172 are connected, and sensor units 280, 282, 284, and 286 mounted on the right front, left front, right rear, and left rear of the vehicle 82, respectively.

Each of the sensor units 280, 282, 284 and 286 includes a millimeter wave radar, a camera, and a LiDAR.

In this embodiment, the in-vehicle network 288 includes four gigabit class network switches 292, 294, 296, and 298, each connected to a sensor belonging to the same sensor unit, a first multi-gigabit switch 300 bridging between the two network switches 292 and 294 at the front of the vehicle, and a second multi-gigabit switch 302 bridging between the two network switches 296 and 298 at the rear of the vehicle and connected to the first multi-gigabit switch 300. The vehicle interior/exterior linkage unit 164 is connected to the network switch 292, and the automatic driving ECU 172 is connected to the network switch 294. A TCU (Telematics Control Unit) 290, which corresponds to the exterior communication device 162 shown in FIG. 4, is connected to the network switch 292 together with the vehicle interior/exterior linkage unit 164.

In this manner, sensor units 280, 282, 284, and 286 are disposed in different positions on the vehicle. Therefore, depending on the situation of the vehicle, the value of the sensor data from these sensor units may differ as described below.

Figure 7 shows the hardware configuration of the vehicle interior/exterior linkage unit 164. Referring to Figure 7, the vehicle interior/exterior linkage unit 164 includes a microprocessor 320. The microprocessor 320 includes a bus 342, a CPU 340, a ROM (Read-Only Memory) 344, a RAM (Random Access Memory) 346, a DMAC (Direct Media Access Controller) 348, an input/output I/F (Interface) 352, a timer 350, and an interrupt controller 354 that is connected to the DMAC 348, the timer 350, and the input/output I/F 352 and generates an interrupt in the CPU 340 in response to signals from these.

The interrupt controller 354 periodically generates a timer interrupt to the CPU 340 based on the time measured by the timer 350. The interrupt controller 354 also generates an input/output interrupt to the CPU 340 when there is input/output from the outside of the input/output I/F 352. The interrupt controller 354 further generates an interrupt to the CPU 340 in response to a signal from the DMAC 348 when the data transfer by the DMAC 348 is completed.

Figure 8 shows the RAM 346 in which the vehicle interior/exterior linkage unit 164 shown in Figure 4 stores data for vehicle interior/exterior linkage. Of these data, all fixed data is stored in the ROM 344 shown in Figure 7, and is loaded into the RAM 346 when the CPU 340 executes the program. Other data is dynamically generated by the CPU 340 when the program is executed, and stored in the RAM 346.

8, the RAM 346 stores mounted sensor information 400, which is information about sensors mounted on the vehicle 82, a sensor data set table 402 in which a plurality of sensor data sets to be transmitted to the outside by the vehicle 82 are stored, a sensor priority policy table 404 in which a plurality of sensor priority policies are stored, which indicate how to determine the priority of each type of sensor depending on the vehicle situation, etc., when a sensor data set to be transmitted to the outside is specified, high-definition map data 406 for a navigation system (not shown) mounted on the vehicle 82, and data from various sensors mounted on the vehicle 82. The sensor data storage area 408 temporarily stores sensor data transmitted to the vehicle interior/exterior link unit 164 for transmission to the edge server 128, and a packet priority policy table 410 holds information for setting packet transfer priority (packet priority policy) in table format so that when a sensor data set to be transmitted is determined and a sensor priority is determined, the network switches 292, 294, 296, 298, the first multi-gigabit switch 300, and the second multi-gigabit switch 302 shown in FIG. 6 transfer packets of sensor data with a priority according to the determined sensor data set. The packet priority policy is information for specifying a packet priority that can be set in each switch, the number of queues for each packet priority, a packet transfer frequency for each priority, etc. Using this, a packet transfer policy for each priority is set for each packet, and information specifying the priority is stored in the packet header of the sensor data, so that each switch executes a transfer process according to the priority.

Figure 9 shows an example of a sensor data set table 402 stored in RAM 346. Referring to Figure 9, in this example, the sensor data set table 402 stores eight sensor data sets indicated by identification numbers (ID) 0 to 7 in the first column. In this embodiment, the identification numbers are determined so that when each sensor data set is selected, the smaller the amount of data transmitted, the smaller the identification number, and the larger the amount of data transmitted, the larger the identification number.

In the example shown in FIG. 9, the sensors are assumed to be LiDAR, millimeter wave radar, and a camera. The second column in FIG. 9 indicates the nature of the data transmitted in each sensor data set. In the example shown in FIG. 9, the resolution or compression rate at the time of transmission of the sensor data of LiDAR, millimeter wave radar, and camera are shown in order from the left. For example, in the sensor data set with identification number 2, LiDAR is uncompressed, millimeter wave radar is also uncompressed, and the image data of the camera is an SD (Standard Definition) image. The SD image is an image consisting of 720 x 480 pixels or 720 x 576 pixels. In addition, in the case of ID=5, LiDAR is uncompressed, millimeter wave radar is also uncompressed, but the camera image is a full HD (High Definition) image. A full HD image is an image consisting of 1900 x 1080 pixels. The "HD images" of identification numbers 3, 6, etc. are images consisting of 1280 x 780 pixels. These images are obtained from the same camera, and the center part is the same for all resolution images. That is, the HD image is an image of the center part of a full HD image, and the SD image is an image of the center part of an even HD image. By selecting data in this way, an image in the direction the camera is facing is always sent to the edge server 128.

The third column is the interval at which data is transmitted (data interval). In this example, this data interval is also determined for each ID and each sensor. In Figure 9, for example, for identification number 1, it is shown that both the LiDAR and millimeter wave radar transmit 10 times per second, and camera images are not transmitted. For identification number 5, both the LiDAR and millimeter wave radar transmit 10 times per second, and camera images are transmitted three times, full HD images, to the edge server 128.

The rightmost column indicates how many sensors of each type installed in the vehicle will transmit data from. For example, identification number 1 indicates that four pieces of data will be transmitted from both LiDAR and millimeter wave radar, and no cameras will be used. For identification number 4, data will be transmitted from four sensors each for LiDAR and millimeter wave radar, but there are also four cameras. In contrast, identification number 6 transmits the same four sensors for LiDAR and millimeter wave radar, but only two cameras are used.

In this way, the total amount of data to be transmitted can be controlled by specifying the resolution or compression rate of each sensor, the transmission interval, and the number of sensors to be used for each identification number. In other words, by specifying the sensor data set ID, the content and total amount of sensor data transmitted from the vehicle to the edge server 128 can be controlled.

In the above explanation, child detection areas and accident areas are given as characteristics of areas, but the characteristics of areas are not limited to these. For example, adults who are operating smartphones should be given the same caution as children. The same applies when a bicycle with two or three people on it is detected. Furthermore, examples of areas where camera images are given priority include areas where there are many parked vehicles, areas where traffic jams have existed for long periods of time, and areas where broken-down vehicles are parked in locations where vehicles would not normally park (broken-down vehicle areas).

In this embodiment, as described above, each vehicle transmits its own sensor data set table to the edge server 128. After determining the high priority area, the edge server 128 refers to the sensor data set table of each vehicle, determines which vehicle will receive the sensor data set corresponding to which identification number, and requests the vehicle to transmit the sensor data by specifying the identification number of the sensor data set. When the vehicle receives this request, the vehicle determines whether or not all of the sensor data sets corresponding to the identification number transmitted from the edge server 128 can be transmitted within the allowable delay time based on the available transmission capacity, the delay time of data communication through the network in the vehicle, the processing time by the CPU mounted on the in-vehicle device, and the transmission delay to the edge server 128. If the determination is positive, the transmission of the specified sensor data set is started. If the determination is negative, the edge server 128 is requested to decrease the identification number by 1, and the transmission of the sensor data is not started. When the edge server 128 receives a response from the vehicle to decrease the identification number by 1, the edge server 128 selects another vehicle and requests the vehicle to transmit a specific sensor data set. If any vehicle starts transmitting a sensor data set, communication continues, and if all vehicles request that the identification number be decreased by 1, the amount of sensor data to be transferred is decreased, a value that is 1 lower than the previously specified number is specified as the identification number of the sensor data set, and the vehicle selection and the sensor data transmission request are repeated again. Details of this procedure will be described later with reference to FIG.

Figure 10 shows an example of a sensor priority policy table 404 defined for the identification number of a sensor data set. The example shown in Figure 10 corresponds to identification number 3 or 6 in Figure 9. The number of sensors in the sensor data set with identification number 3 in Figure 9 is "4, 4, 2". That is, there are four millimeter wave radars and four LiDARs, and two cameras. In the example shown in Figure 10, if there are four millimeter wave radars, millimeter wave radars #0, 1, 2, and 3, they are adopted in this order. The same is true for LiDAR, and if there are LiDARs #0, #1, #2, and 3, they are adopted in this order. As for cameras, there are cameras #0, #1, #2, and 3, and among them, cameras #2 and 3 are adopted. In this way, which of the four cameras is adopted is determined based on the value of the sensor data obtained from each camera.

Below, we will explain one example of how to determine which camera to use (how to create the sensor priority policy table 404) when only two of four cameras are to be used, as in the case of the camera described above.

Referring to FIG. 11, in this example, sensor type numbers 0, 1, and 2 are assigned to the millimeter wave radar, LiDAR, and camera, respectively, according to the type of sensor. Furthermore, as shown in FIG. 12, sensor position type numbers are assigned according to the position type of the sensor (front/rear, left/right). In the example shown in FIG. 12, 0 is assigned when the sensor position is forward, 1 is assigned when the sensor position is rearward, 0 is assigned when the sensor position is left, and 1 is assigned when the sensor position is right. In this example, the sensor position is a combination of front/rear and left/right. That is, in the case of cameras, as shown in FIG. 13, when the above-mentioned sensor type numbers are combined with the front/rear and left/right sensor position types for the front left camera 440, the front right camera 442, the rear left camera 444, and the rear right camera 446 of the vehicle 82, the following codes are assigned to each camera.
Camera 440 200
Camera 442 201
Camera 444 210
Camera 446 211

Using this code, it is possible to determine which of four cameras to select when only two are to be used. The method is explained below. The following explanation concerns a method in which the value of the image data obtained from each camera is determined, and a code according to this value is selected, thereby allowing the camera corresponding to that code to be used.

With reference to Figure 14, consider a target area 500 managed by real space 50, and assume that an intersection 512 is located within this area. Assume that a building 514 exists near intersection 512, and that building 514 has a base station for high-speed data communications. Therefore, while high-speed communications are possible in high-speed communications area 510 centered on intersection 512, only low-speed communications are available in other areas. This information can be obtained from the communications status map 152 shown in Figure 3. It is also known that an infrastructure sensor is installed at intersection 512. This information can also be obtained from the traffic situation overhead map 150 shown in Figure 3.

In this state, the vehicle is traveling toward the intersection 512 in the high-speed communication area 510 as shown by the vehicle 520 in FIG. 14. The intersection 512 is provided with an infrastructure sensor as described above, and the information is transmitted to the edge server 128 at high speed. Therefore, even if the vehicle 520 transmits image data of the vicinity of the intersection 512 to the edge server 128, the overlap between the detection range of the camera of the vehicle 520 and the detection range of the infrastructure camera is large, and the data is duplicated and wasteful. On the other hand, there is no infrastructure sensor behind the vehicle 520. Therefore, there is little or no overlap between the detection range of the camera of the vehicle 520 and the detection range of the surrounding infrastructure cameras, and the image data obtained by the camera behind the vehicle 520 is highly useful if it is transmitted to the edge server 128. Furthermore, since the vehicle 520 is in an area where high-speed communication is possible, HD images or full HD images can be used as images.

Consider also vehicle 522 traveling away from intersection 512 in high-speed communication area 510. In this case, for the same reason as vehicle 520, images of the rear of vehicle 522 are of low value to edge server 128. On the other hand, images of the front of vehicle 522 are of high value to edge server 128. In the case of vehicle 522, since it is also in an area where high-speed communication is possible, HD images or full HD images can be used.

Furthermore, consider a vehicle 524 traveling on a road outside the high-speed communication area 510. Since the vehicle 524 is in a low-speed communication area, SD images should be used as the images. Also, since there are no infrastructure cameras in the vicinity of the vehicle 524, both the front and rear images of the vehicle 524 are useful to the edge server 128.

As described above, in this embodiment, the value of images from cameras that obtain images that do not overlap with images obtained by infrastructure cameras is set high, and the value of images from other cameras is set low, and a camera that obtains images with high value is selected. In other words, in this embodiment, the value of images from cameras that obtain images that complement the detection range of the infrastructure camera is set high, and the value of images that overlap with the detection range of the infrastructure camera is set low.

Therefore, for example, assuming that each of vehicles 520, 522 and 524 is equipped with four LiDARs, four millimeter wave radars and four cameras as shown in Figure 6, the following describes how to determine the sensor priority policy table 404 shown in Figure 10.

The sensor type is represented by the symbols shown in Figure 11, and the position of each sensor (front/back, left/right) is represented by the symbols shown in Figure 12. These are coded in the order of (sensor type, front/back, left/right). The same applies to sensors other than cameras.

For example, in the case of vehicle 520, the rear camera is used regardless of whether it is the left or right camera. In other words, of the four cameras, two cameras are used. Since vehicle 520 is in an area where high-speed communication is possible, the sensor data set in this case is identification number "6" as shown in Figure 9. In this case, if the codes of the 12 sensors are sorted in descending order, these 12 sensors are arranged in the following order:

211
210
201
200
111
110
101
100
011
010
001
000

Of these, the bottom four are millimeter wave radars, and all of these are adopted in the sensor data set with identification number 6. The middle four are LiDARs, and all of these are also adopted. Of the top four, the first two, "211" and "210", refer to the rear cameras of vehicle 520. As only two cameras are adopted in the sensor data set with identification number 6, the top two of these are adopted and the other cameras are not adopted.

Similarly, for vehicle 522, the codes are sorted in ascending order, and only the top code values "200" and "201" are used. For vehicle 524, all cameras are used without the need for sorting.

By coding each sensor in this way and sorting the codes in ascending order if the front sensors have a higher value, or in descending order if the rear sensors have a higher value, the desired sensor can be selected.

Note that in this embodiment, the entire code is sorted in ascending or descending order. However, this disclosure is not limited to such an embodiment. By using the second and third digits of the code as separate keys and sorting in ascending or descending order, respectively, sensors can be selected based on values defined for both the front and back and the left and right.

<<Program for realizing the edge server 128>>
15 is a flowchart showing a control structure of a program for executing, by a computer, the process of requesting specific vehicles in each area to transmit sensor data and collecting the sensor data in the edge server 128. This program runs repeatedly at a predetermined time interval in parallel with the program for creating a traffic condition bird's-eye view map and the program for creating a communication condition map.

With reference to FIG. 15, this program includes step 550 of extracting high priority areas from among the areas managed by edge server 128 in accordance with the above-mentioned criteria, step 552 of executing the following process 554 for each area extracted in step 550, and step 556 of, after completion of step 552, instructing vehicles that are not located in any of the high priority areas to transmit sensor data at a longer period (including the case where transmission is stopped) than vehicles transmitting sensor data from within the high priority areas, thereby terminating execution of the program.

The process 554 includes a step 570 for determining a sensor priority policy according to the characteristics of the area to be processed, a step 572 for calculating the sensor detection range of a vehicle present in the area to be processed from the data transmitted from the vehicle, and selecting as vehicle candidates a vehicle that includes the area to be processed in its sensor detection range or is likely to have the area to be processed in the near future in its sensor detection range based on the vehicle's moving speed and direction, and that is equipped with a specified sensor, a step 574 for determining an identification number of a sensor data set based on the sensor data set table received from the vehicle according to the sensor priority policy determined in step 570 for each vehicle selected in step 572, a step 576 for executing a process 578 to attempt to receive sensor data from each vehicle, a step 580 for determining whether or not reception of a sensor data set has started from the target vehicle as a result of executing step 576, and branching the flow of control according to the determination, and a step 582 for decrementing the value of the identifier of the sensor data set to be transmitted to each vehicle by 1 in response to a negative determination in step 580, and returning control to step 576. If the determination in step 580 is positive, this means that sensor data for the target area has been received, so the process 554 for that area ends and the process moves to the next area.

Process 578 includes step 590, which transmits the identifier of the sensor data set determined in step 574 for the target vehicle and requests the target vehicle to transmit the corresponding sensor data set; step 592, which determines whether transmission has started as a result of the processing of step 590 and branches the control flow according to the determination; step 594, which transmits an instruction to other vehicles in the high-priority area being processed to stop transmitting sensor data in response to the determination of step 592 being positive; and step 596, which exits the loop processing of process 578 for the high-priority area being processed. When the determination of step 592 is negative (a request to decrease the identifier of the sensor data set by 1 has been received from the target vehicle), process 578 is terminated and process 578 is started for the next vehicle among the candidate vehicles in the high-priority area being processed.

Note that although process 554 is performed for each area, there may be multiple candidate vehicles within an area. In such a case, it is desirable to execute process 578 in order of vehicles that are considered to be most efficient at collecting sensor data. For example, the candidate vehicles are sorted using criteria such as the number of types of sensors installed, the number of transmission capacities, and the number of RTTs (Round Trip Time: the time from sending a signal to receiving a response), and process 578 is executed starting from the vehicle with the highest ranking. Of course, this disclosure is not limited to such a method, and it is also possible to select vehicles in a random order, or to select vehicles in order of the length of time they are expected to be in the high-priority area.

When sensor data reception has started for all high priority areas, control proceeds to step 556, where vehicles outside the high priority areas are instructed to transmit sensor data at long intervals, and the process ends. The identifier of the sensor data set transmitted from each vehicle at this time may be a predetermined one, or may be selected by the vehicle depending on the communication speed of the area in which the vehicle is located.

<<Program for realizing the vehicle interior/exterior linkage unit 164>>
16, a program executed by a computer to realize the in-vehicle/outside linking unit 164 shown in FIG. 6 is executed when some event is received by the in-vehicle/outside linking unit 164. Here, the event includes an interrupt to the CPU.

This program includes step 640, which determines whether the received event is an instruction to transmit a sensor data set, and branches the control flow according to the determination. The instruction to transmit a sensor data set received in step 640 includes an identifier of one of the sensor data sets in the sensor data set table of the vehicle equipped with the on-board device executing this program.

The program further includes step 642, in response to the determination in step 640 being positive, estimating the transmission delay required to transmit the specified sensor data set to the edge server 128 based on the available wireless transmission capacity, the network delay in the vehicle, processing by the CPU, etc., and step 644, in which it is determined whether or not the specified sensor data set can be transmitted to the edge server 128 based on the transmission delay estimated in step 642, i.e., whether or not the transmission delay calculated in step 642 is equal to or less than the allowable delay, and branching the flow of control according to the determination. In response to the determination in step 644, the method includes a step 650 of selecting a sensor that constitutes the sensor data determined by the specified identifier, setting the network switches 292, 294, 296, 298, the first multi-gigabit switch 300, and the second multi-gigabit switch 302 in the in-vehicle network 288 to transmit the sensor data from the selected sensor preferentially, and terminating execution of the program; and in response to the determination in step 644 being negative, requesting the edge server 128 to decrease the value of the identifier of the data set to be transmitted by a predetermined number (e.g., 1), and terminating execution of the program.

The program further includes step 660, in response to the determination in step 640 being negative, determining whether the received content is an instruction from the edge server 128 to stop transmitting data and branching the flow of control, and step 664, in response to the determination in step 660 being negative, resetting each switch in the in-vehicle network 288 (see FIG. 6) from the state set in step 650 to a default state and terminating execution of the program.

Furthermore, the program includes step 670, in response to the determination in step 660 being negative, determining whether the received data is an instruction from the edge server 128 indicating that sensor data should be transmitted at a low speed, and branching the flow of control according to the determination, and step 674, in response to the determination in step 670 being positive, reselecting the sensor data to be transmitted at a low speed, and setting each switch in the in-vehicle network 288 (see Figure 6) to preferentially forward the transmission of sensor data from the selected sensor, and terminating execution of the program.

This program further includes step 680, in response to the negative determination of step 670, determining whether the received data is sensor data from a sensor mounted on the vehicle and branching the flow of control according to the determination; step 682, in response to the positive determination of step 680, transferring the received sensor data to the automatic driving ECU 172 (see FIG. 6); step 683, following step 682, selecting a sensor and setting a switch according to a priority policy; step 684, following step 683, determining whether the sensor data received in step 680 is to be transferred to the edge server 128 and branching the flow of control according to the determination result; step 686, in response to the positive determination of step 684, transferring the sensor data received in step 680 to the edge server 128 and terminating the execution of the program; and step 688, in response to the negative determination of step 680, performing processing according to the received data and terminating the execution of the program. When the determination of step 684 is negative, nothing is done and the execution of the program is terminated.

In this embodiment, in steps 650, 664, and 674, before transferring the sensor data, the presence or absence of transfer of sensor data to the edge server 128 and the transfer speed are confirmed, and if necessary according to the confirmation result, the switch priority is set and the sensor data is transferred. If transfer is not necessary, the sensor data is not transferred. Specifically, in this embodiment, the data output unit from each sensor is instructed to add information to a packet of sensor data from a specified sensor indicating that the data should be added to one of the priority queues of each switch set according to the priority policy. As a result, whether or not sensor data should be transmitted, and if so, which sensor data to transmit, are controlled in almost real time.

Figure 17 shows, in the form of a flowchart, the control structure of the program executed in step 642 in Figure 16. Referring to Figure 17, this program includes step 700 for executing process 702 of estimating end-to-end delay time from the sensor that is the output source of each sensor data mounted on the vehicle to the destination (i.e., edge server 128) of the sensor data, and step 704 for determining whether the estimated value of the transmission delay for all the sensor data is smaller than the allowable delay time as a result of step 700, and branching the control flow according to the determination result.

This program further includes step 706, in response to the determination in step 704 being positive, setting a flag indicating whether or not the sensor data can be transmitted to the edge server 128 within the tolerable delay time to 0 and terminating execution of this program, and step 708, in response to the determination in step 704 being negative, setting the flag to 9 and terminating execution of this program. In step 644 of FIG. 16, if the flag is 0, it is determined that transmission of the sensor data is possible, and if it is 9, it is determined that transmission of the sensor data is not possible.

Process 702 in Figure 17 includes step 730 of confirming (estimating) the transmission capacity (in this case, wireless transmission delay) from the in-vehicle/external connection unit 164 to the edge server 128, step 732 of confirming (estimating) the transmission delay of data from the sensor to be processed to the TCU 290 in the in-vehicle network 288 shown in Figure 6, step 734 of confirming (estimating) the system processing time required by the CPU 340 (see Figure 7) and the TCU 290 to transmit the sensor data, and step 736 of estimating the delay time that occurs in the transmission of the sensor data to be processed from the sensor to the edge server 128 by summing steps 730, 732 and 734, and terminating process 702.

The wireless transmission delay that is a problem in step 730 is often caused by fluctuations in the transmission capacity of the wireless communication used by the TCU 290 in FIG. 6. The transmission capacity of wireless communication may fluctuate significantly due to switching between 4G communication and 5G communication, shadowing by large vehicles, etc. In step 730, for example, the observation result of the most recent actual data communication between the vehicle and the edge server 128 may be used as the transmission delay.

Transmission delay is calculated using the following formula.

Wireless transmission delay = sensor data volume / line speed
+ In-car network transmission delay
+ System processing time
+Margin

The in-vehicle network transmission delay and system processing time can be tabulated based on the most recent observation results. Alternatively, a margin can be set including the in-vehicle network transmission delay and system processing time.

When calculating the delay time in this manner, it is necessary to synchronize the timer maintained by the vehicle 82 with an external reference timer. For example, when determining the line speed based on the time required to transmit data from the edge server 128 to the vehicle, an accurate transmission delay cannot be calculated unless the vehicle's timer and the edge server 128's timer are synchronized. Therefore, the timer 350 shown in FIG. 7 is synchronized with an external time based on protocols such as NTP (Network Time Protocol), AVB (Audio-Video Bridging), and TSN (Time-Sensitive Networking). In addition, the timers of each part in the vehicle are synchronized with the timer 350 to prevent any inconvenience in the calculation of the transmission delay.

The transmission delay of the in-vehicle network refers to the sum of the time it takes for the sensor data from each sensor to reach the in-vehicle external linking unit 164 through the network and the time it takes for the sensor data to reach the TCU 290 from the in-vehicle external linking unit 164 in FIG. 6. The processing time of the CPU refers to the processing time required for the CPU 340 shown in FIG. 7 to perform processing to transmit the sensor data to the edge server 128. Since the in-vehicle external linking unit 164 controls the vehicle, it is necessary to provide a margin for the in-vehicle network and CPU resources for in-vehicle control. It is easy to check the status of the transmission capacity and the in-vehicle network, and it is easy to estimate the impact on the transmission of sensor data. However, since the CPU resources are easily affected by other processes and cannot be clearly known until the actual transmission of sensor data is started, it is more difficult to estimate them than the other two elements. Therefore, the process of transmitting sensor data under various conditions may be executed, and the CPU resources consumed as a result may be measured and stored in the form of a table or the like.

18 is a flowchart showing a control structure of a program for implementing a priority policy update process executed by CPU 340 of in-vehicle/out-vehicle linkage unit 164 shown in FIG. 7 in response to an interrupt signal generated by timer 350 at predetermined time intervals. Referring to FIG. 18, this program includes step 800 for detecting the position of the vehicle from an output of a GPS (Global Positioning System) or the like, step 802 for searching for infrastructure sensor equipment present around the vehicle from map information based on step 800, and step 804 for determining whether the infrastructure sensor searched for in step 802 is within a predetermined distance ahead of the vehicle and branching the control flow in response to the determination result.

The program further includes step 806, which, when the determination in step 804 is positive, determines whether or not there is an infrastructure sensor within a specified distance behind the vehicle and branches the control flow based on the determination result; step 808, which is executed when the determination in step 806 is positive, which compares the distance to the forward infrastructure sensor with the distance to the rear infrastructure sensor, determines whether the forward infrastructure sensor is farther away, and branches the control flow in response to the determination; step 812, which, when the determination in step 808 is positive, decides to give priority to the forward sensors mounted on the vehicle and terminates execution of the program; and step 810, which, when the determination in step 808 is negative, decides to give priority to the rear sensors mounted on the vehicle and terminates execution of the program.

The program further includes step 814, which, in response to the negative determination in step 806, decides to give priority to the rear sensor and terminates execution of the program; step 816, in response to the negative determination in step 804, determines whether or not an infrastructure sensor is present within a predetermined distance behind the vehicle and branches the control flow according to the determination; step 818, in response to the positive determination in step 816, decides to give priority to the front sensor among the sensors installed in the vehicle and terminates execution of the program; and step 820, which is executed when the negative determination in step 816 is made, randomly sets the priority of the sensors installed in the vehicle and terminates execution of the program.

By executing this program, sorting the sensor codes in ascending or descending order according to priority as described above, and further selecting the number of high-priority sensors for each sensor type determined by the sensor data set, a sensor priority policy table 404 is obtained for determining the sensor data set to be sent to the edge server 128. This program is repeated at predetermined time intervals, and the sensor priority policy table 404 is updated each time. Therefore, if the predetermined time is set sufficiently short, an appropriate sensor priority policy table 404 can always be obtained in almost real time according to the vehicle's position and surrounding conditions.

Figure 19 is a flowchart showing a control structure of a program for implementing step 650 in Figure 16. Referring to Figure 19, this program includes step 850 of selecting a sensor that outputs sensor data to be transmitted to edge server 128 for each sensor type by performing the following process 852 on each sensor data in the sensor data set identified by an identifier received from edge server 128, and step 854 of setting a packet forwarding priority policy corresponding to the priority determined in step 850 in each switch (or the output unit of each sensor) and terminating execution of this program.

Process 852 includes step 870 of determining whether the number of sensors defined in the specified sensor data set is less than the number of sensors of the sensor type being processed that are installed in the vehicle, and branching the control flow depending on the determination, step 872 of determining the value of each sensor using the sensor priority policy table 404 when the determination in step 870 is positive, step 874 of selecting, following step 872, the number of sensors specified for the sensor type being processed by the specified sensor data set starting with the sensors with the highest value, and step 878 of selecting all sensors of the sensor type being processed and terminating process 852 in response to the determination in step 870 being negative.

Actions
Referring to FIG. 6, each sensor included in the sensor units 280, 282, 284, and 286 outputs sensor data according to the surrounding situation. These sensor data are transmitted to the vehicle interior/exterior linking unit 164 via the network switches 292, 294, 296, and 298, the first multi-gigabit switch 300, and the second multi-gigabit switch 302, etc. Among the sensors, the LiDAR measures the distance to the lattice point of the part of the moving object facing the LiDAR that exists within a predetermined range around, and outputs a ranging signal as a point cloud that is a collection of three-dimensional position coordinates of each point. The millimeter wave radar measures the distance to the moving object existing in the surroundings, and outputs the signal. The camera captures an image of the imaging range at a predetermined frame rate. Of the images, an image (SD image, HD image, full HD image) of the range specified by the sensor data set including the center of the image is output. 6, the sensor data specified by the sensor data set specified by the edge server 128 is also transmitted to the edge server 128. When determining the sensor data set, the edge server 128 and the in-vehicle/out-of-vehicle linking unit 164 operate as follows.

Timer 350 shown in Figure 7 generates a timer interrupt at predetermined time intervals and provides an interrupt signal to CPU 340. In response to this interrupt signal, CPU 340 executes a program whose control structure is shown in Figure 18. By periodically executing this program, a priority policy for determining the priority when selecting a sensor according to a sensor data set from among multiple sensors possessed by the vehicle, depending on the relationship between the vehicle and surrounding infrastructure sensors, is updated in real time.

As described above, the reception processing unit 210 of the edge server 128 receives signals from the multiple infrastructure sensor equipment 180 and signals from the sensors 170 mounted on the vehicles 82, provides signals from the LiDAR and millimeter wave radar to the moving object tracking unit 212, and provides image data from the cameras to the attribute detection unit 216. Meanwhile, the reception processing unit 210 provides information indicating the vehicle's position, speed, and direction of travel from the information received from the sensors 170 to the vehicle tracking unit 220. Based on the received information, the vehicle tracking unit 220 maintains data indicating the current position and direction of travel of each vehicle. Coordinates indicating the detection range of the sensors mounted on each vehicle are transmitted from each vehicle to the edge server 128. The vehicle information storage unit 222 stores such information together with the coordinates of each vehicle.

The moving object tracking unit 212 of the edge server 128 determines the position of each moving object in a first cycle by analyzing the distance measurement signals from the LiDAR etc. received from the reception processing unit 210. The time required for this analysis is short, and the moving object tracking result 213 is updated in the first cycle. The attribute detection unit 216 determines the attributes of moving objects such as vehicles and people in the image by performing image analysis on the image data received from the reception processing unit 210. Since image processing takes time, the cycle of attribute detection by the attribute detection unit 216 is a second cycle that is longer than the update cycle of the moving object tracking result 213 by the moving object tracking unit 212. The attributes 217 of the attribute detection unit 216 are stored in the attribute memory unit 218.

With reference to FIG. 5, the reception processing unit 210 receives sensor data and vehicle data from infrastructure sensor equipment 180 and vehicles 82 in the target area. The moving object tracking unit 212 detects the position of the moving object in the target area from the data and tracks its movement. The tracking results are stored in the moving object tracking result storage unit 214. Similarly, the attribute detection unit 216 detects the attributes of each moving object. The detected attributes of the moving object are stored in the attribute storage unit 218. The integration processing unit 224 integrates the tracking results of the moving objects and the attributes of the moving objects, creates a traffic situation overhead map 225 using the high-resolution map stored in the traffic situation overhead map storage unit 226, and stores the data in the traffic situation overhead map storage unit 226. The integration processing unit 224 constantly updates the traffic situation overhead map 225 stored in the traffic situation overhead map storage unit 226 based on newly received data.

The vehicle tracking unit 220 stores the vehicle information 221 received from each vehicle in the vehicle information storage unit 222. The vehicle tracking unit 220 further tracks each vehicle in the area based on the vehicle information 221 stored in the vehicle information storage unit 222 and newly received vehicle information 221. The vehicle tracking unit 220 further requests a vehicle that has newly entered the area to transmit a transmission data set table, and receives the received transmission data set table in the sensor data set table storage unit 248.

When information to be transmitted to each vehicle occurs in the traffic condition overhead map 225, the information transmission unit 228 generates the information to be transmitted to each vehicle from the data stored in the traffic condition overhead map memory unit 226, and transmits the information to each vehicle via the transmission processing unit 230.

The line speed information storage unit 256 extracts and stores pairs of the transmission time held in the data received by the reception processing unit 210 from each vehicle in the area, the reception time in the edge server 128, and the position of each vehicle stored in the vehicle information storage unit 222. The communication status map management unit 258 creates a communication status map relating to the transmission time between the vehicle and the edge server 128 for each section obtained by dividing the map into a grid based on the data stored in the line speed information storage unit 256 and the high-resolution map stored in the high-resolution map storage unit 262, and stores the communication status map in the communication status map storage unit 260. The communication status map management unit 258 always keeps the communication status map storage unit 260 up to date based on the most recent line speed information stored in the line speed information storage unit 256.

The high priority area extraction unit 240, the policy determination unit 244, the candidate vehicle selection unit 246, the dataset inquiry unit 250 and the vehicle selection unit 252 operate as follows. Referring to FIG. 15, the following process is repeated at a predetermined time.

First, the edge server 128 extracts high priority areas from the areas that the edge server 128 is responsible for according to the above-mentioned criteria (step 550). Then, for each area extracted in step 550, processing 554 is performed in step 552.

In process 554, a sensor priority policy according to the characteristics of the area to be processed is determined (step 570). Next, in step 572, the sensor detection range of the vehicle is calculated from the data transmitted from the vehicle present in the area to be processed, and it is determined whether the vehicle includes the area to be processed in the detection range of the sensor, and whether the vehicle is likely to enter the detection range of the sensor in the near future and is equipped with a specified sensor, and a vehicle that satisfies the conditions is selected as a vehicle candidate. In the following step 574, for each vehicle selected in step 572, a sensor data set to be collected from the vehicle is determined and stored according to the sensor priority policy determined in step 570, the communication status map, and the vehicle information regarding the sensor equipped in each vehicle.

In step 576, process 578 is executed for each candidate vehicle selected in step 572. In this embodiment, process 578 is executed in order from the candidate vehicles having the most favorable attributes (type, number and performance of installed sensors, data transmission performance, etc.).

In process 578, a transmission of sensor data is sent to the target vehicle along with the identifier of the sensor data set determined in step 574.

Referring to FIG. 16, in the vehicle that receives this, the determination in step 640 becomes positive, step 642 is executed, and the transmission delay when transmitting the specified sensor data set to the edge server 128 is estimated. This process is as shown in FIG. 17. That is, for each sensor data, the wireless transmission delay (step 730), the in-vehicle network transmission delay (step 732), and the system processing time (step 734) are confirmed (estimated), and based on them, the total end-to-end delay time from the sensor to be processed to the edge server 128 is estimated. If the estimated transmission delay value for all sensor data is smaller than the allowable delay, the flag is set to 0 in step 706. If the estimated transmission delay value for even one sensor data is equal to or greater than the allowable delay time, the flag is set to 9 in step 708.

If, in step 644, it is determined that the transmission delay calculated in step 642 is less than the allowable delay and therefore the specified sensor data set can be transmitted to the edge server 128, then, in step 650, a sensor for transmitting sensor data to the edge server 128 is selected according to the sensor data set, and each switch is configured to preferentially forward the sensor data from that sensor.

That is, referring to FIG. 19, the process 852 is executed for each sensor data in the sensor data set corresponding to the identifier specified by the edge server 128. In the process 852, it is first determined whether the number of sensors specified by the sensor data set for a certain type of sensor is less than the total number of sensors equipped in the vehicle. If the determination is negative, the sensors of the target type mounted on the vehicle are selected as sensors that transmit sensor data to all edge servers 128 (step 878). If the determination in step 870 is positive, in step 872, the value of each sensor is determined according to the sensor priority policy at that time, and in step 874, sensors starting from the most valuable sensor are selected up to the number of sensors determined by the specified sensor data set. When sensors have been selected for all sensor types in this way, in step 854, each switch is set to transfer sensor data from the selected center with priority.

As a result, the selected sensor starts outputting sensor data addressed to the edge server 128. When the CPU 340 (FIG. 7) receives this sensor data, the sensor data is transferred to the autonomous driving ECU 172 in step 682 via the route of step 640 → step 660 → step 670 → step 680 in FIG. 16. Furthermore, if the sensor data is to be transferred to the edge server 128, the sensor data is transferred to the edge server 128 via the route of step 684 → step 686.

15 again, in this case, the determination in step 592 becomes positive (flag = 0), the processes in steps 594 and 596 are executed, and the process in step 576 ends. In the following step 580, it is determined that the transmission of sensor data from the target vehicle is being executed, so the process for the target area ends, and the process 554 is executed for the next high priority area.

If the transmission delay in a vehicle that requested the transfer of the sensor data set in step 590 is greater than or equal to the allowable delay time, the following processing is performed.

With reference to Figure 16, in this case, processing is performed in the following order: step 640 → step 642 → step 644. Since the determination in step 644 is negative, processing in step 648 is executed. That is, the vehicle transmits information to the edge server 128 requesting that the identifier of the sensor data set be decreased by one.

Referring again to FIG. 15, in the edge server 128 that receives this request, the determination in step 592 is negative (flag ≠ 0). Execution of process 578 for the target vehicle is terminated, and process 578 is executed for the next vehicle. When sensor data cannot be transferred within the allowable delay time for all candidate vehicles, the determination in step 580 is negative, and in step 582, 1 is subtracted from the identifier value of the sensor data set for each vehicle. Process 578 is then repeated for each vehicle using the new identifier. This process is repeated until a vehicle capable of transmitting sensor data is found. Although not shown, when transmission is not possible for all candidate vehicles, processing can be resumed from step 570.

Once sensor data transmission from the selected vehicles has commenced for all of the targeted high priority areas, step 556 is executed and instructions are sent to vehicles outside the area indicating that sensor data should be sent at long intervals.

With reference to Figure 16, when each vehicle receives this instruction, step 674 is executed via the path of step 640 → step 660 → step 670 in Figure 16. In step 674, sensors are selected, data is output from the sensors, and switches in the network are reconfigured (reset) so that a different data set from that in step 650 (in most cases, all sensor data) is sent to the edge server 128 at long intervals. As a result, sensor data is sent to the edge server 128 at long intervals from vehicles other than the selected vehicle.

For vehicles other than those selected in the high priority area, an instruction to stop sending sensor data is sent from the edge server 128. In this case, for those vehicles, the process of step 664 is executed via the route of step 640 → step 660 in FIG. 16. In other words, step 664 is reset, and all sensor data is excluded from being transferred to the edge server 128.

16, the operation of the program executed in each vehicle when a data set transmission instruction, a data transmission stop instruction, and a low-speed data transmission instruction are received is as described above. When the CPU 340 receives sensor data from the sensor, the sensor data is transferred to the autonomous driving ECU 172 (see FIG. 6) in step 682 via the route of step 640 → step 660 → step 670 → step 680, and in the subsequent steps 684 and 686, if the sensor data is to be transferred to the edge server 128, it is transferred to the edge server 128, and if not, nothing is done and the execution of this program is terminated.

When it is determined in step 680 that the received data is not sensor data, control proceeds to step 688, where processing is performed according to the received data and processing is terminated.

As described above, according to this embodiment, in an area where a certain condition is satisfied (high priority area), only vehicles that can transmit the most effective sensor data for creating a traffic situation overview map to the edge server 128 most efficiently within a certain allowable delay according to a sensor priority policy corresponding to the conditions of the area transmit the sensor data, and other vehicles do not transmit sensor data. The amount of data transmitted to the edge server 128 is not excessive, and the traffic situation overview map can be created and maintained in real time in a timely manner. In this case, the transmission delay is estimated by estimating the transmission capacity (line speed) from the vehicle transmitter to the edge server 128, as well as the transmission delay of the in-vehicle network and the processing time in the CPU, and then adding them together, and comparing them with the allowable delay by adding a margin. As a result, the edge server 128 can collect information necessary for creating and maintaining the traffic situation overview map from the selected vehicle with a high probability, taking into consideration the line speed with the edge server 128 and the factors of the transmission delay in the vehicle. As a result, the risk of communication congestion between the vehicle and the edge server 128 in the target area can be reduced. For the target area, a sensor priority policy according to the reason for the target area is adopted, and vehicles that can transmit sensor data in a form that conforms to the sensor priority policy are selected. Therefore, the most necessary data according to the traffic situation is transmitted to the edge server 128, and a traffic situation bird's-eye view map that fully reflects the necessary information can be created.

17, the end-to-end transmission delay time of the sensor data from each of the multiple sensors mounted on the vehicle 82 to the edge server 128 is estimated. The process shown in FIG. 18 and the process of step 872 shown in FIG. 19 determine the value of the sensor data output by the multiple sensors mounted on the vehicle 82 based on the vehicle state including the position and attitude of the vehicle 82, the installation positions of the multiple sensors mounted on the vehicle 82 on the vehicle 82, and the overlap state of the detection ranges of the multiple sensors and the detection ranges of the infrastructure sensors. The process of step 874 in FIG. 19 selects a part of the sensor data based on the end-to-end transmission delay time estimated in process 702 in FIG. 17, and the value determined in the process of FIG. 18 and step 872 in FIG. 19 according to the position and traveling direction of the vehicle 82, etc., and transmits a copy of the sensor data to the edge server 128 via the TCU 290 in FIG. 6. The end-to-end transmission delay time from the sensor to the edge server 128 is estimated, including the delay time in the in-vehicle network, and sensor data to be transmitted to the edge server 128 is selected based on the estimated transmission delay time and the value determined for each piece of sensor data by the process in Fig. 18 and step 874 in Fig. 19. While following the fluctuation in transmission capacity by the TCU 290, the in-vehicle/out-of-vehicle linking unit 164 selects and transmits sensor data that can be effectively used by the edge server 128, taking into account the end-to-end transmission delay time from the sensor to the edge server 128, so that the edge server 128 can function as well as possible while making the most of the available transmission capacity.

In addition to the processing time in the CPU 340 until the sensor data that has arrived at the input/output I/F 352 shown in Fig. 7 is input to the TCU 290 in Fig. 6, and the communication delay time required for the sensor data to be transmitted from the TCU 290 to the edge server 128, the network delay time until the sensor data from multiple sensors reaches the input/output I/F 352 in Fig. 7 via the in-vehicle network 288 shown in Fig. 6 is estimated for each sensor, and the end-to-end transmission delay time from each sensor to the edge server 128 is calculated based on these. Since the end-to-end transmission delay time is calculated taking into account the transmission delay time of the in-vehicle network 288, the amount of data that can be transmitted to the edge server 128 within the allowable delay time can be more accurately estimated, and the sensor data to be transmitted can be more accurately selected.

Among the multiple sensors mounted on the vehicle 82, sensor data from a sensor with a large overlapping detection range with the detection range of the infrastructure sensor has low utility value in the edge server 128. The smaller the overlapping detection range of a sensor with the detection range of the infrastructure sensor, the more valuable the sensor data output by that sensor is in the edge server 128. Therefore, by estimating the detection range of the infrastructure sensor present in the vicinity of the vehicle 82 using an algorithm such as that illustrated in FIG. 18 described with reference to FIG. 14, the value of the sensor whose detection range overlaps with the detection range of the infrastructure sensor is calculated to be high and the value of the sensor whose detection range overlaps with the detection range of the infrastructure sensor is calculated to be low, and the sensor data from those sensors is preferentially selected and transmitted to the edge server 128. Since high-value sensor data is transmitted to the edge server 128 within a limited transmission capacity, the edge server 128 can perform stable processing based on the sensor data regardless of fluctuations in the transmission capacity. Note that the algorithm shown in FIG. 18 does not check the details of the detection range of the infrastructure sensor, but at least among the sensors mounted on the vehicle 82, sensors whose detection range is unlikely or almost unlikely to overlap with the infrastructure sensor are selected, and thus the above-mentioned condition is satisfied.

As shown by the algorithm in FIG. 18, the detection range of the infrastructure sensor is updated at a predetermined time interval. The value of the sensor data is also updated at a predetermined time interval based on the position and attitude of the vehicle 82 and the updated detection range of the infrastructure sensor, as shown in FIG. 18. Therefore, the value of each sensor of the vehicle 82 is updated at least at a predetermined time interval in accordance with the movement of the vehicle 82 and the environment around the vehicle 82, etc. As a result, regardless of the movement of the vehicle 82 and changes in the environment around the vehicle, etc., high-value sensor data is transmitted to the edge server 128. Therefore, regardless of vehicle distribution and changes in the environment, the edge server 128 can perform stable processing.

As shown in Figure 19, the value of sensor data is set for each type of sensor. This allows the edge server 128 to select effective sensor data for processing by making more effective use of transmission capacity.

As can be seen from the algorithm shown in FIG. 19, when an identifier specifying one of the sensor data sets is provided to the vehicle interior/exterior linkage unit 164, the sensor data set corresponding to that identifier is determined as shown in FIG. 9. The number of sensors specified by the sensor data set are selected for each sensor type in order of the sensor data with the most valuable. For all types of sensors, a predetermined number of valuable sensor data items based on the designation by the sensor data set are transmitted to the edge server 128. As a result, the edge server 128 can effectively use the sensor data to perform stable processing.

The autonomous driving ECU 172 shown in Fig. 6 operates based on sensor data from multiple sensors mounted on the vehicle 82. Among these sensor data, sensor data selected by the vehicle interior/exterior linkage unit 164 is transmitted to the edge server 128, so that the autonomous driving ECU 172 and the edge server 128 can share the sensor data, and autonomous driving of the vehicle and processing of the edge server 128 can be efficiently performed.

Second Embodiment
<composition>
In the first embodiment, a part of the sensor data transmitted to the automatic driving ECU 172 is transmitted to the edge server 128. However, this disclosure is not limited to such an embodiment. Referring to Fig. 20, a traffic assistance system 900 according to the second embodiment includes a vehicle 914 having a configuration similar to that of the vehicle 82 according to the first embodiment, a vehicle 916 having a configuration similar to that of the vehicle 82 instead of the vehicle 84 in Fig. 4 but equipped with a remote control ECU (not shown) instead of the automatic driving ECU 172 shown in Figs. 4 and 6, and a remote control server 910 for receiving sensor data from the vehicle 916 via the Internet 912, generating a signal (remote control command) for controlling the remote control ECU of the vehicle 916 based on the sensor data, and transmitting the signal to the vehicle 916 via the Internet 912.

21, the vehicle 916 includes an in-vehicle device 932 instead of the in-vehicle device 92 shown in FIG. 4, and a remote control ECU 930 instead of the automatic driving ECU 172 shown in FIG. 4. The in-vehicle device 932 includes a NSW 950, an in-vehicle/external linkage unit 952, and an external communication device 954 instead of the NSW 160, the in-vehicle/external linkage unit 164, and the external communication device 162 of the in-vehicle device 92. The remote control ECU 930 does not have an automatic driving function like the automatic driving ECU 172, and in this embodiment, it simply operates according to remote control commands transmitted from the remote control server 910.

21, this in-vehicle device 932 does not transmit sensor data from the sensor 170 to the remote control ECU 930, but transmits it to the remote control server 910 via the NSW 950, the in-vehicle/outside linkage unit 952, and the outside-vehicle communication device 954, as shown by arrow 960. The remote operation command transmitted from the remote control server 910 is received by the in-vehicle/outside linkage unit 952 via the reverse path of the sensor data transmission path, and is provided to the remote control ECU 930 via the NSW 950.

In such an embodiment, basically all of the sensor 170 is transmitted to the remote control server 910, while a portion of it is transmitted to the edge server 128. For this purpose, the control structure of the program executed by the CPU of the vehicle interior/exterior linkage unit 952 is shown in FIG. 22.

Referring to Figure 22, this program differs from the program shown in Figure 16 in that, instead of step 682 in Figure 16, it includes step 972 for transferring sensor data to the remote control server 910 when the sensor data is received, step 980 for determining whether the received data is a remote control command from the remote control server 910 and branching the control flow in response to a negative determination in step 680, and step 982 for transferring the received remote control command to the remote control ECU 930 shown in Figure 21 and terminating execution of this program when the determination in step 980 is positive.

As can be easily understood by comparing Figures 4 and 21, and Figures 16 and 22, respectively, the vehicle 916 of this embodiment operates in the same manner as the vehicle 82 of the first embodiment, and can achieve the same effects, except that it operates by remote control rather than automatic driving.

In this second embodiment, the hardware may be the same as that shown in the first embodiment, except for the remote control ECU 930, and the difference from the first embodiment is in the control structure of the program executed by the vehicle 916, etc. The edge server 128 may be exactly the same as that in the first embodiment.

As described above, in the second embodiment, vehicles that use sensor output for remote control rather than autonomous driving can also be used as targets for collecting sensor data. This allows a wider range of targets for sensor data collection, minimizing the impact on vehicles with autonomous driving functions, while also maintaining high quality of the traffic situation overview map maintained by the edge server.

20 and 21 operates based on sensor data received from a sensor 170 mounted on a vehicle 916, and transmits remote control commands to a remote control ECU 930. Any of the sensor data transmitted to the remote control server 910 is selected by an in-vehicle/out-of-vehicle linking unit 952 and transmitted to the edge server 128. Therefore, the sensor data can be shared between the remote control server 910 and the edge server 128, and remote control of the vehicle 916 and execution of processing by the edge server 128 can be performed efficiently.

In the above embodiment, the line speed is measured by the vehicle interior/exterior linkage device to determine whether the sensor data set can be transmitted. However, this disclosure is not limited to such an embodiment. For example, the edge server 128 may transmit to the vehicle interior/exterior linkage device the line speed between each vehicle and the edge server 128, which is estimated from the communication status map 152 (see FIG. 3) within the area managed by the edge server 128 and the position of each vehicle. In this case, the vehicle interior/exterior linkage device uses this line speed to determine whether the sensor data of the sensor data set can be transmitted.

In the above embodiment, the edge server 128 further collects sensor data set tables from each vehicle and requests candidate vehicles to transmit sensor data by specifying the identifiers of the sensor data sets based on the tables. However, this disclosure is not limited to such an embodiment. For example, it is conceivable that the edge server 128 does not collect sensor data set tables, but asks each vehicle for a data set that will enable the edge server 128 to transmit the most sensor data from a specified sensor according to conditions such as communication speed. The edge server 128 may select the vehicle that can transmit the most data among the vehicles that responded.

Furthermore, in the above embodiment, the edge server 128 and the vehicle (mainly the vehicle interior/exterior linkage unit 164) work together to determine the sensor data set to be sent from the vehicle to the edge server 128. However, this disclosure is not limited to such an embodiment, and the vehicle interior/exterior linkage unit 164 may determine the sensor data set independently. For example, if the sensor is a camera, when the image output by the camera becomes blurred, it is possible to stop transmitting the image from that camera to the edge server 128 and send the image of another camera that can obtain a clear image to the edge server 128. Whether the image is clear or not can be determined, for example, by performing Fourier analysis on the image and judging whether there are many high frequency components.

In the above embodiment, time synchronization using a timer as shown in FIG. 7 is essential for the vehicle interior/external linkage unit 164 to accurately estimate the line speed. However, the accuracy of time synchronization depends on the protocol used for time synchronization and the hardware performance of the vehicle interior/external linkage unit 164 and an ECU with a similar configuration. Therefore, it is desirable to set a large margin in the calculation of the delay time when the synchronization error is large, and to set a small margin when the synchronization error is small.

In this embodiment, the computer programs for operating the vehicle interior/external linking units 164 and 952 as the functions of the vehicle interior/external linking unit 952 and its components are written into the ROM 344 when the vehicle interior/external linking units 164 and 952 are shipped. However, if necessary, the computer programs written into the ROM 344 can be updated via a network. Furthermore, by connecting a removable storage medium reading device to the in-vehicle device 932, these computer programs can also be updated from media such as CD-ROM (Compact Disk-Read Only Memory) or DVD-ROM (Digital Versatile Disk-Read Only Memory). In that case, it is conceivable that CD-ROMs or DVD-ROMs, etc. on which the computer programs are recorded, will be distributed on the market.

The embodiments disclosed herein are merely examples, and the disclosure is not limited to the above-described embodiments. The scope of the disclosure is indicated by each claim in the scope of the claims, taking into consideration the detailed description of the disclosure, and includes all modifications within the meaning and scope equivalent to the wording described therein.

50 Real space 52, 150, 225 Traffic situation bird's-eye view map 70 Communication system 82, 84, 520, 522, 524, 914, 916 Vehicle 86 Pedestrian 88 Infrastructure camera 90 Traffic signal controller 92, 94, 932 Vehicle-mounted device 96 Mobile phone 98, 100 Wireless communication device 110, 112, 114 Base station 120, 122 Metro NW
124, 130 Distributed DC
126, 128 Edge server 140 Core NW
142 Core DC
144 Core Server 152 Communication Status Map 160, 950 NSW
162, 954 Vehicle exterior communication device 164, 952 Vehicle interior/exterior linkage unit 170 Sensor 172 Automatic driving ECU
174, 960 Arrow 180 Infrastructure sensor equipment 190 Infrastructure sensor 192 Communication device 210 Reception processing unit 212 Moving object tracking unit 213 Moving object tracking result 214 Moving object tracking result storage unit 216 Attribute detection unit 217 Attribute 218 Attribute storage unit 220 Vehicle tracking unit 221 Vehicle information 222 Vehicle information storage unit 224 Integration processing unit 226 Traffic condition bird's-eye view map storage unit 228 Information transmission unit 230 Transmission processing unit 240 High priority area extraction unit 242 Sensor priority policy storage unit 244 Policy decision unit 246 Candidate vehicle selection unit 248 Sensor data set table storage unit 250 Data set inquiry unit 252 Vehicle selection unit 254 Enquiry transmission/reception unit 256 Line speed information storage unit 258 Communication status map management unit 260 Communication status map storage unit 262 High-resolution map storage unit 280, 282, 284, 286 Sensor unit 288 In-vehicle network 290 TCU
292, 294, 296, 298 Network switch 300 First multi-gigabit switch 302 Second multi-gigabit switch 320 Microprocessor 340 CPU
342 bus 344 ROM
346 RAM
348 DMAC
350 Timer 352 Input/Output I/F
354 Interrupt controller 400 On-board sensor information 402 Sensor data set table 404 Sensor priority policy table 406 High-resolution map data 408 Sensor data storage area 410 Packet priority policy tables 440, 442, 444, 446 Camera 500 Target area 510 High-speed communication area 512 Intersection 514 Building 550, 552, 556, 570, 572, 574, 576, 580, 582, 590, 592, 594, 596, 640, 642, 644, 648, 650, 660, 664, 670, 674, 680, 682, 683, 684, 686, 688, 700, 704, 706, 708, 730, 732, 734, 736, 800, 802, 804, 806, 808, 810, 812, 814, 816, 818, 820, 850, 854, 870, 872, 874, 878, 972, 980, 982 Step 554, 578, 702, 852 Process 900 Traffic assistance system 910 Remote control server 912 Internet 930 Remote control ECU

Claims (11)

A vehicle interior/exterior linking device used in a vehicle equipped with a wireless communication device that wirelessly communicates data with the outside of the vehicle and a plurality of sensors,
a data receiving unit that receives sensor data from the plurality of sensors;
a delay time estimation unit that estimates a transmission delay time of the sensor data from each of the plurality of sensors to a predetermined device that can communicate with the wireless communication device;
a selection unit that selects a portion of the sensor data based on the transmission delay time estimated by the delay time estimation unit, and transmits the selected sensor data to the predetermined device via the wireless communication device ;
a value determination unit that determines a value of sensor data output from the plurality of sensors based on a state of the vehicle and states of the plurality of sensors;
the selection unit includes a data selection unit that selects a portion of the sensor data based on the transmission delay time estimated by the delay time estimation unit and the value determined by the value determination unit, and transmits the selected sensor data to the predetermined device via the wireless communication device;
The value determination unit is
an infrastructure detection range estimation unit that estimates a detection range of an infrastructure sensor present in the vicinity of the vehicle based on a position of the vehicle;
a value setting unit that estimates the detection ranges of the multiple sensors based on the position and attitude of the vehicle, and sets a value of the sensor data from each sensor so that the value of the sensor data from the sensor is greater for a sensor with a smaller area overlapping with the detection range of the infrastructure sensor estimated by the infrastructure detection range estimation unit . The vehicle interior/exterior linkage device according to claim 1, wherein the delay time estimation unit includes an end-to-end delay time estimation unit that estimates a transmission delay time of the sensor data between each of the plurality of sensors and a predetermined device that can communicate with the wireless communication device. The end-to-end delay time estimation unit
a transmission time estimation unit that estimates a delay time for each of the sensors until the sensor data from the plurality of sensors arrives at the data receiving unit;
a processing time estimation unit that estimates a processing time required for processing the sensor data that has arrived at the data receiving unit until the sensor data is input to the wireless communication device;
a wireless communication delay time estimation unit that estimates a wireless communication delay time required for the sensor data to be transmitted from the wireless communication device to the predetermined device;
The vehicle interior/exterior linking device according to claim 2 , further comprising: a delay time calculation unit that calculates the transmission delay time based on an amount of data from each of the plurality of sensors, the delay time, the processing time, and the wireless communication delay time. the infrastructure detection range estimation unit includes an infrastructure detection range update unit that repeatedly updates an estimated detection range of an infrastructure sensor present in the vicinity of the vehicle at a predetermined time interval based on a position of the vehicle;
2. The vehicle interior/exterior integration device according to claim 1, wherein the value setting unit includes a value updating unit that repeatedly updates the value of the sensor data from each sensor at a predetermined time interval based on the position and attitude of the vehicle, so that the value of a sensor having a smaller area overlapping with the detection range of the infrastructure sensor updated by the infrastructure detection range updating unit is higher . Each of the plurality of sensors is classified into one of a plurality of sensor types;
the infrastructure detection range estimation unit includes a sensor type detection range estimation unit that estimates detection ranges of the infrastructure sensors present in the vicinity of the vehicle for each sensor type of the infrastructure sensors based on a position of the vehicle;
2. The vehicle interior/exterior collaboration device according to claim 1, wherein the value setting unit includes a sensor type sensor data value setting unit that estimates, for each of the plurality of sensor types, a detection range of each of the sensors belonging to that sensor type among the plurality of sensors based on a position and attitude of the vehicle, and sets a value of sensor data from a sensor belonging to that sensor type among the plurality of sensors such that a sensor having a smaller area overlapping with the detection range of the infrastructure sensor estimated for that sensor type by the sensor type detection range estimation unit has a higher value. The vehicle further comprises an in-vehicle network for data transfer;
The vehicle interior/exterior linking device according to claim 5 , wherein the data receiving unit includes a network receiving unit that receives the sensor data from the plurality of sensors via the vehicle interior network. The sensor data set storage unit stores information specifying a plurality of sensor data sets together with an identifier of each of the sensor data sets,
the plurality of sensor data sets include information for specifying, for each of the plurality of sensor types, the number of sensors to be selected from the plurality of sensors and a data format related to the amount of sensor data from the corresponding sensor;
The vehicle interior/exterior linking device further includes:
a sensor data set reading unit configured to read, in response to receiving any one of the identifiers from the predetermined device, information on the sensor data set corresponding to the identifier from the sensor data set storage unit;
a sensor data selection unit that selects, for each of the plurality of sensor types, sensor data from a number of sensors designated by the information of the sensor data set read by the sensor data set reading unit, in order of decreasing value, based on information of the sensor data set read by the sensor data set reading unit and the value set by the sensor type sensor data value setting unit for the sensor data from each sensor;
The vehicle interior/exterior linking device according to claim 6 , further comprising: a network setting unit that sets the vehicle interior network so that the sensor data selected by the sensor data selection unit is transferred with priority over other sensor data. The vehicle further comprises an in-vehicle network for data transfer;
The vehicle interior/exterior linking device according to claim 1 , wherein the data receiving unit includes a network receiving unit that receives the sensor data from the plurality of sensors via the in-vehicle network. The vehicle further includes a vehicle control device that controls the vehicle using the sensor data from the plurality of sensors.
The vehicle interior/exterior linking device according to claim 1 , wherein the vehicle control device includes an autonomous driving ECU that performs autonomous driving of the vehicle using the sensor data received from the plurality of sensors. The vehicle further comprises:
a remote control ECU that controls the vehicle in accordance with a predetermined remote control command;
The vehicle interior/exterior linking device includes:
Transmitting the sensor data received from the plurality of sensors to an external remote control device via the wireless communication device;
The vehicle interior/exterior linking device according to claim 1 , further comprising: a remote control command received from the remote control device via the wireless communication device, the remote control ECU being configured to input the remote control command to the remote control ECU. A vehicle having a wireless communication device that wirelessly communicates data with the outside of the vehicle and a plurality of sensors, the vehicle having the wireless communication device and a method for linking the inside and outside of the vehicle by transmitting sensor data from the plurality of sensors to an external predetermined device,
A computer receives sensor data from a plurality of sensors mounted on a vehicle;
A step of estimating a transmission delay time of the sensor data from the sensor to the predetermined device that can communicate with the wireless communication device by a computer;
a step of the computer selecting a portion of the sensor data based on the transmission delay time estimated in the step of estimating the transmission delay time, and transmitting the selected sensor data to the predetermined device via the wireless communication device ;
and a step of determining, by a computer, a value of sensor data output by the plurality of sensors based on a state of the vehicle and states of the plurality of sensors;
the transmitting step includes a data selection step in which a computer selects a portion of the sensor data based on the transmission delay time estimated in the estimating step and the value determined in the determining step, and transmits the selected sensor data to the predetermined device via the wireless communication device;
The step of determining the value comprises:
A step of estimating a detection range of an infrastructure sensor present in the vicinity of the vehicle based on a position of the vehicle by a computer;
a step of a computer estimating the detection ranges of the multiple sensors based on the position and attitude of the vehicle, and setting a value of the sensor data from each sensor so that the value of the sensor data from each sensor is greater for a sensor with a smaller area overlapping with the detection range of the infrastructure sensor estimated by the infrastructure detection range estimation unit .

Images