JP7838738B2 - Air traffic control system - Google Patents

Description

This invention relates to a control system that utilizes equipment installed along a road to switch between autonomous driving and remote control of an autonomous vehicle.

As a remote support system for autonomous vehicles, there is a system that switches between remote control mode and autonomous driving mode based on judgments made from the surrounding conditions of the autonomous vehicle, and if autonomous driving is difficult, an operator at a control center takes over remote control (Patent Document 1).

The system described in Patent Document 1 does not address blind spots in autonomous vehicles. Therefore, in dangerous roads with poor visibility, relying solely on the autonomous vehicle's surrounding conditions is insufficient to determine the conditions for switching to remote control.

Japanese Patent Publication No. 2020-3890

This invention was made in view of the above background, and aims to provide a control system that can safely and smoothly switch between autonomous driving and remote control of an autonomous vehicle using information gathering equipment installed along a road.

To achieve the above objective, the control system according to the present invention comprises: a roadside unit that communicates with the autonomous vehicle and monitors its surroundings; a roadside unit monitoring system that monitors the roadside unit; a remote control system that remotely controls the autonomous vehicle; and an operation management system that acquires monitoring results from the roadside unit monitoring system, makes a decision on switching between autonomous driving and remote control based on the monitoring results, and, if it decides to switch to remote control, instructs the remote control system to remotely control the autonomous vehicle. The monitoring results of the roadside unit acquired from the roadside unit monitoring system include not only surrounding information such as the presence or absence of obstacles around the roadside unit, but also status information such as malfunctions of the roadside unit.

The above control system utilizes roadside monitoring results when deciding whether to switch between automated and remote control modes. This allows for the proactive identification of areas where automated driving would be dangerous, enabling a smooth transition from automated to remote control. Furthermore, it can address performance degradation due to roadside unit failures. In short, by considering the roadside unit's monitoring status when deciding whether to switch between automated and remote control modes, the system can implement appropriate mode changes.

According to a specific aspect of the present invention, in the above-mentioned control system, the operation management system communicates with the autonomous vehicle and, upon receiving abnormal information from the autonomous vehicle, instructs the remote control system to remotely control the autonomous vehicle. In this case, the remote control can be switched over, taking into account the abnormal information from the autonomous vehicle.

According to another aspect of the present invention, the roadside unit has a roadside unit sensor that monitors a detection area including the travel path of the autonomous vehicle. When the roadside unit sensor detects an obstacle on the autonomous vehicle's travel path, it notifies the roadside unit monitoring system of the obstacle information. The roadside unit monitoring system then notifies the operation management system of the obstacle information as a result of the roadside unit's monitoring. The operation management system then determines, based on the obstacle information, to switch from autonomous driving to remote control. In this case, even if an obstacle is present on the autonomous vehicle's travel path, the switch to remote control can be performed smoothly. Furthermore, it is possible to detect obstacles ahead of the autonomous vehicle, allowing for a sufficient margin of error in switching to remote control and safely avoiding obstacles.

According to yet another aspect of the present invention, the roadside unit has a roadside unit sensor that monitors a detection area including the route of the autonomous vehicle. If the roadside unit sensor fails, it notifies the roadside unit monitoring system of the failure information. The roadside unit monitoring system then notifies the operation management system of the failure information as a result of the roadside unit monitoring. The operation management system then determines, based on the failure information, to switch from autonomous driving to remote control. In this case, even if the roadside unit sensor fails, the switch to remote control can be performed smoothly.

According to yet another aspect of the present invention, if communication with the roadside unit monitoring system is lost, the roadside unit monitoring system notifies the operation management system of the communication loss information as a result of the roadside unit monitoring. The operation management system then determines, based on the communication loss information, to switch from autonomous driving to remote control. In this case, even if communication with the roadside unit is lost, the switch to remote control can be performed smoothly.

According to yet another aspect of the present invention, if communication between an autonomous vehicle and a roadside unit that monitors a detection area including the autonomous vehicle's travel path is lost, the autonomous vehicle notifies the fleet management system of the communication loss information as abnormal information. The fleet management system then determines, based on the communication loss information, to switch from autonomous driving to remote control. In this case, even if communication between the autonomous vehicle and the roadside unit is lost, the switch to remote control can be performed smoothly.

According to yet another aspect of the present invention, if the remote control system determines that it is possible to return from remote control to autonomous driving, it notifies the fleet management system of the termination of remote control, and the fleet management system instructs the autonomous vehicle to resume autonomous driving.

This is a conceptual plan view showing a road equipped with the control system according to the first embodiment. This is a block diagram showing an example configuration of an air traffic control system. This is a flowchart illustrating a series of operations in an air traffic control system. This is a flowchart illustrating a series of operations in an air traffic control system. This is a flowchart illustrating a series of operations in an air traffic control system.

[First Embodiment]
Hereinafter, an example of a control system according to the first embodiment of the present invention will be described with reference to Figure 1 and other figures. Figure 1 is a conceptual diagram for illustrating the overview of the case in which the control system 100 according to this embodiment is applied to a road RA.

Figure 1 shows an example of the operation of an autonomous vehicle VE, which is supported by the control system 100, when traveling on a straight road RA with one lane in each direction. While a bus BU traveling along a predetermined route is shown as an example of an autonomous vehicle VE, the control system 100 can support various autonomous vehicles VE, including not only buses BU but also passenger cars, trucks, trailers, and towing vehicles. The autonomous vehicle VE travels along road RA according to the driving route TR1 corresponding to the future position information TR.

The control system 100 comprises an on-board system VS installed in the autonomous vehicle VE, a roadside unit PV installed along the road RA, and a control center CC that monitors and controls the on-board system VS and the roadside unit PV. The control system 100 functions as a system through the cooperation of the on-board system VS, the roadside unit PV, and the control center CC. The control system 100 uses the on-board system VS and the roadside unit PV to monitor the conditions of the road RA and the autonomous vehicle VE, while supporting the driving of the autonomous vehicle VE as it travels along the road RA. Specifically, the control system 100 switches between autonomous driving and remote control of the autonomous vehicle VE based on the monitoring status of the road RA, the roadside unit PV, and the autonomous vehicle VE.

Figure 2 is a block diagram showing an example configuration of the control system. As shown in Figures 1 and 2, the on-board system VS is mounted on the autonomous vehicle VE, and communication CT1 with the control center CC, utilizing communication networks NT1a and NT1b, enables switching between autonomous driving and remote control. In addition to communication CT1 with the control center CC, the on-board system VS also communicates with the roadside unit PV, utilizing communication network NT2, via CT2, to mutually provide information about the vehicle and its surroundings.

In an autonomous vehicle (VE), the on-board system (VS) comprises a driving control unit 11, an autonomous driving device 12, a remote control device 13, an on-board sensor 14, a GNSS acquisition medium 15, a vehicle-to-infrastructure communication device 16, a control communication device 17, a route data unit 18, and a map data unit 19.

The driving control unit 11 consists of various parts necessary for normal driving operations such as steering, acceleration, and braking. The automatic driving device 12 performs automatic control of engine operation and other functions corresponding to the operations of the driving control unit 11 in order to perform automatic driving of the autonomous vehicle VE. The remote control device 13 performs remote control of engine operation and other functions corresponding to the operations of the driving control unit 11. The on-board sensor 14 consists of an imaging unit (camera) and a distance measuring unit such as LiDAR (not shown) and detects the surrounding conditions of the autonomous vehicle VE. The GNSS acquisition medium 15 consists of a GPS receiver, etc. The vehicle-to-infrastructure communication device 16 performs wireless communication with the roadside unit PV. The control communication device 17 performs wireless communication with the control center CC. The route data unit 18 stores information about the driving route TR1. The map data unit 19 is for determining the location indicated as the driving route TR1 in the route data unit 18.

Furthermore, the map data unit 19 stores map data for at least the area that could be included in the driving route TR1, and it is assumed that, for example, nationwide road map data is incorporated. The autonomous vehicle VE drives along a driving route TR1 set from the route data unit 18, following the map data stored in the map data unit 19.

In the case of an autonomous vehicle VE with the above configuration, for example, the autonomous driving system 12 controls the operation of various components constituting the on-board sensors 14, in addition to various programs necessary for autonomous driving, to acquire information about the surrounding conditions of the autonomous vehicle VE (for example, information about the position of white lines defining lanes on road RA), and to acquire self-position information via GNSS (GPS, etc.) through a GNSS acquisition medium (GPS receiver, etc.) 15. This makes it possible to accurately estimate the self-position and perform autonomous driving based on the estimation results.

The automated driving system 12 includes a future location information generation unit 12a for generating future location information TR. The future location information TR consists of the current position of the automated driving vehicle VE on a single driving route TR1 set from the route data unit 18 using a GNSS acquisition medium 15, and information about the planned future route based on the current position. The future location information TR may also include a detour route TR2 in case an obstacle OB exists on the automated driving vehicle VE's driving path. Driving route TR1 and detour route TR2 are the movement trajectories of the automated driving vehicle VE based on the future location information TR. The future location information TR is created by the automated driving vehicle VE based on its route information, speed, and time. The detour route TR2 may be provided by a roadside unit PV or a control center CC.

Furthermore, in this example, we will explain that in the autonomous vehicle VE, a predetermined route, TR1, is set to travel through a set of predetermined locations. In other words, the autonomous vehicle VE performs autonomous driving along the map data unit 19, following the TR1 route in the route data unit 18. During this process, it transmits future position information TR, corresponding to the TR1 route during autonomous driving, to the roadside unit PV and the control center CC via the vehicle-to-infrastructure communication device 16 and the control communication device 17.

The automated driving system 12 performs automated driving of the automated driving vehicle VE based on images captured by the on-board sensors 14 and sensor detection information. The automated driving system 12 transmits the captured images and sensor detection information corresponding to the video information to the control center CC via the control communication device 17. The images and other data from the on-board sensors 14 transmitted to the control center CC are used for remote control.

The remote control device 13 operates the automated vehicle VE according to the remote control information received from the remote control device 62 of the control center CC, which will be described later. Here, the remote control information is the driving control signal based on the remote control operations performed by the control center CC operator using the remote control terminal 61.

The vehicle-to-infrastructure communication device 16 has the function of performing wireless communication by connecting to the communication network NT2. The vehicle-to-infrastructure communication device 16 incorporates communication circuits that utilize communication methods such as mobile communication lines represented by 5G and 4G LTE, mid-range wireless communication methods represented by wireless LAN, narrow-range wireless communication methods such as DSRC, and spot communication methods such as beacons. It performs digital data communication with roadside PVs while identifying the other device. The vehicle-to-infrastructure communication device 16 can employ one or more of the above communication methods.

The control communication device 17 performs digital data communication between the remote control system 60 and the operation management system 70 of the control center CC, which will be described later. In this embodiment, the wireless communication between the control communication device 17 and the remote control system 60 and the wireless communication between the control communication device 17 and the operation management system 70 utilize different communication networks NT1a and NT1b. The communication network NT1a between the control communication device 17 and the remote control system 60 is a high-speed line with enhanced security, a communication line that enables real-time remote control, and utilizes a mobile communication line such as IP-VPN. The communication network NT1b between the control communication device 17 and the operation management system 70 can be a general line, and utilizes a mobile communication line such as 5G or 4G LTE. Note that the wireless communication between the control communication device 17 and the remote control system 60 and the wireless communication between the control communication device 17 and the operation management system 70 are not limited to using different communication networks NT1a and NT1b, but may also use a common communication line.

Next, the roadside unit PV is a roadside device installed near the road RA. The roadside unit PV monitors the detection range SA, which includes the driving route TR1 of the autonomous vehicle VE, and communicates with the autonomous vehicle VE to obtain information about the autonomous vehicle VE itself. Furthermore, the roadside unit PV notifies the autonomous vehicle VE of the status of the road RA based on the monitoring results and information about the autonomous vehicle VE. The roadside unit PV also notifies the roadside unit monitoring system 50 of the monitoring status of the road RA and the monitoring status of the roadside unit PV via communication CT3 with the control center CC using the communication network NT3. The roadside unit PV is installed on traffic lights, utility poles, streetlights, etc., and is not limited to being installed on existing structures; it may also be installed as a separate, independent unit.

The roadside PV monitors the area surrounding the autonomous vehicle VE, including the first lane RA1 corresponding to the lane in which the autonomous vehicle VE is traveling, and the second lane RA2 corresponding to the opposing lane of the first lane RA1, within a predetermined section where the autonomous vehicle VE is traveling. By monitoring the surrounding area, i.e., the detection range SA, with the roadside PV, it is possible to obtain surrounding information over a wider range than the surrounding information obtained from the on-board sensors 14 installed on the autonomous vehicle VE, thereby compensating for the blind spots of the autonomous vehicle VE. Although not shown in the diagram, blind spots are particularly likely to occur when there is a curve in the road RA and visibility around the curve is poor. The position and size of the detection range SA can be changed as appropriate.

The roadside PV monitors traffic conditions within the detection range SA and acquires target information such as moving objects MB and obstacles OB. It also acquires route information (future position information, described later) from the automated driving vehicle VE, indicating the driving route TR1 when traveling along the road RA. To avoid collisions with moving objects MB and obstacles OB within the detection range SA, the roadside PV notifies the automated driving vehicle VE and the roadside monitoring system 50 of hazard information and obstacle information. Furthermore, based on the hazard information and obstacle information, the roadside PV can also predict collision avoidance with moving objects MB and obstacles OB and provide information for collision avoidance. Obstacle information includes, for example, current time, type, location, and speed. Hazard information includes, for example, the possibility of collision with moving objects MB and obstacles OB, and specifically, it includes determining whether or not there will be a collision with other vehicles on the automated driving vehicle VE's driving path and stopping accordingly.

The roadside PV unit comprises a roadside control device 21, a roadside sensor 22, a vehicle-to-road communication device 23, a control communication device 24, and a map data unit 25.

The roadside control device 21 includes a roadside determination unit 21a, which is composed of various circuit boards, a CPU, a storage device, etc., and a sensor interface 21b for connecting to a roadside sensor 22 that monitors the detection range SA.

The roadside determination unit 21a collects future position information TR of the autonomous vehicle VE received by the vehicle-to-infrastructure communication device 23, vehicle information, and object information as detection results from the roadside sensor 22, and uses this information to determine the road conditions. Typically, as shown in Figure 1, if the vehicle is about to travel straight along road RA, the road conditions in the detection range SA can be determined by extracting the status of obstacles OB as object information through image analysis processing, the driving status of the forward vehicle GM (a moving object MB), and acquiring data related to the road shape in the detection range SA stored in the map data unit 25. Based on the determination results, obstacle information and hazard information are transmitted to the autonomous vehicle VE. Vehicle information includes, for example, the current time, type, location, and speed.

The sensor interface 21b receives information acquired by the roadside sensor 22, i.e., target information, and outputs it to the roadside determination unit 21a. In other words, the sensor interface 21b is for acquiring target information within the detection range SA.

Furthermore, the roadside control device 21 performs a health check on the roadside unit sensor 22 at regular intervals to monitor whether the roadside unit sensor 22 is operating normally. The health information, which is the result of the health check, is periodically transmitted from the roadside unit sensor 22.

The roadside sensor 22 consists of an imaging unit (camera) and a distance measuring unit (not shown), and detects moving objects MB and obstacles OB, etc., that are present in the detection range SA, which is a predetermined range to be monitored. In other words, the roadside sensor 22 monitors the detection range SA, which is a detection area that includes the travel path of the autonomous vehicle VE. Note that moving objects MB may include vehicles, bicycles, pedestrians, etc. The imaging unit takes images and generates image data to monitor the detection range SA. The distance measuring unit may employ, for example, a LiDAR, a millimeter-wave sensor, or radar, and generates distance measurement data to acquire the position of moving objects MB, etc. Note that the roadside sensor 22 may be configured with either an imaging unit or a distance measuring unit, as long as it can acquire target information such as moving objects MB. Although only one roadside sensor 22 is shown in the figure, a configuration with multiple cameras, etc., can be used to thoroughly monitor the detection range SA. Furthermore, while the detection range SA is shown here as an example, if the detection range SA changes depending on the direction of travel A1 of the autonomous vehicle VE to which the information is provided, it is possible to appropriately select the cameras, etc., to be used accordingly. Here, the detection results acquired by the roadside sensor 22, including various information such as image data and distance measurement data concerning the moving object MB present in the detection range SA, are considered as target information. In other words, the target information includes information such as the operating status of the moving object MB present in the detection range SA, as well as the presence of obstacles OB.

Furthermore, the roadside PV may transmit the captured images and sensor detection information corresponding to the video information acquired by the roadside sensor 22 to the control center CC via the control communication device 24 described later. In this case, the captured images and other information from the roadside sensor 22 transmitted to the control center CC can be used for remote control.

The vehicle-to-infrastructure communication device 23 communicates wirelessly with the autonomous vehicle VE. The vehicle-to-infrastructure communication device 23 has the same communication functions as the vehicle-to-infrastructure communication device 16, and communicates digital data with the autonomous vehicle VE located in a predetermined communication zone while identifying the other device, via the communication network NT2 which enables communication using mobile communication lines such as 5G and 4G LTE, medium-range wireless communication methods such as wireless LAN, narrow-range wireless communication methods such as DSRC, and spot communication methods such as beacons. The vehicle-to-infrastructure communication device 23 can employ one or a combination of any of the above communication methods. Here, the autonomous vehicle VE, which is the communication partner, transmits future position information TR, which indicates its future position, to the roadside unit PV as data for making a determination, as described above. The future location information TR includes the current location of the autonomous vehicle VE (location at the current time), the future location calculated based on this (including the predicted arrival time), and information such as speed and direction (azimuth angle) at each of these times (scheduled times). Therefore, by receiving the future location information TR from the autonomous vehicle VE, the roadside unit PV can determine, for example, the predicted arrival time of the autonomous vehicle VE at its destination and the time required to pass through the detection range SA.

The control communication device 24 communicates wirelessly with the control center CC. The control communication device 24 communicates digital data with the roadside unit monitoring system 50 of the control center CC (described later) via a communication network NT3 that enables communication using mobile communication lines, such as 5G or 4G LTE. Note that the communication network NT3 may be a wired communication line.

The map data unit 25 stores detailed data regarding the location where the roadside PV unit is installed and its surrounding area, such as road information (coordinates, shape, width, lanes, road-to-road connections such as intersections, traffic regulations, etc.) and topographic information.

Next, the control center (CC) determines whether it is necessary to switch between autonomous driving and remote control of the autonomous vehicle (VE) based on the monitoring status of the road (RA), autonomous vehicle (VE), and roadside monitoring devices (PV). If it is determined that remote control is necessary, it initiates remote control.

The control center (CC) is equipped with a roadside aircraft monitoring system (50), a remote control system (60), and an operation management system (70).

The roadside unit monitoring system 50 monitors the roadside unit PV. The monitoring results obtained from the roadside unit monitoring system 50 include surrounding information (obstacle information), such as the presence or absence of moving objects MB and obstacles OB around the roadside unit PV, as well as hazard information, and health information, such as malfunctions of the roadside unit PV.

The roadside monitoring system 50 comprises a monitoring and control device 51 and a roadside communication device 52. Although not shown in the figures, the roadside monitoring system 50 is also capable of communicating with the operation management system 70 via wired, wireless, or a combination thereof.

The monitoring and control device 51 communicates with the roadside PV unit at predetermined intervals via the roadside communication device 52 to check the status of the roadside PV unit. The monitoring and control device 51 monitors whether communication from the roadside PV unit is interrupted or malfunctioning. The monitoring and control device 51 performs a health check on the roadside PV unit at regular intervals to monitor whether the roadside PV unit is operating normally. The health information, which is the result of the health check, is periodically transmitted from the roadside PV unit via the roadside communication device 52.

Furthermore, the monitoring and control device 51 transmits, via the roadside communication device 52, the obstacle information acquired by the roadside sensor 22 and the hazard information determined by the roadside determination unit 21a of the roadside control device 21 from the roadside PV.

The monitoring and control device 51 transmits obstacle information, hazard information, health information, and sensor failure information, etc., acquired from the roadside PV unit, to the operation management system 70, which will be described later.

The roadside communication device 52 performs wireless communication with the roadside PV. The roadside communication device 52 has the same communication functions as the control communication device 24, and performs digital data communication with the roadside PV via the communication network NT3, which enables communication using mobile communication lines such as 5G and 4G LTE.

The remote control system 60 is for remotely controlling the autonomous vehicle VE. The remote control system 60 includes a remote control terminal 61, a remote control device 62, and a vehicle communication device 63. Although not shown in the figures, the remote control system 60 can communicate with the operation management system 70 via wired, wireless, or a combination thereof.

The remote control terminal 61 is for the operator to remotely control the autonomous vehicle VE. The remote control terminal 61 is equipped with a monitor screen and control panel (not shown). The control panel has functions equivalent to a steering wheel and brakes. The monitor screen displays captured images corresponding to video information acquired from the autonomous vehicle VE and the roadside unit PV. The operator remotely controls the autonomous vehicle VE by operating the control panel while viewing the monitor screen.

The remote control device 62 converts the operation information from the remote control terminal 61 into driving control signals corresponding to the remote control of the autonomous vehicle VE. The remote control device 62 transmits the driving control signals, which are the remote control information, to the autonomous vehicle VE via the vehicle communication device 63. The remote control device 13 of the autonomous vehicle VE performs remote control based on the driving control signals. Furthermore, when the problem hindering autonomous driving is resolved and autonomous driving can be resumed, the remote control device 62 notifies the operation management system 70 of the termination of remote control.

The vehicle communication device 63 performs wireless communication with the autonomous vehicle VE. Similar to the control communication device 17, the vehicle communication device 63 has a high-speed communication function with enhanced security. For example, it performs digital data communication with the autonomous vehicle VE via the communication network NT1a, which enables communication using a mobile communication line such as IP-VPN.

The operation management system 70 acquires monitoring results from the roadside vehicle PV (Pedestrian View) system 50 and makes a decision on switching between automated driving and remote control based on the monitoring results. Furthermore, the operation management system 70 acquires abnormal information from the automated driving vehicle VE (Vehicle View) and makes a decision on switching between automated driving and remote control based on the monitoring results.

The operation management system 70 has the function of monitoring the status of the autonomous vehicle VE and the surrounding conditions of the autonomous vehicle VE. The operation management system 70 acquires hazard information, obstacle information, and status information of roadside units PV installed within the detection range SA where the autonomous vehicle VE travels. Based on this information, the operation management system 70 manages whether the roadside units PV are malfunctioning or functioning correctly. The information from the roadside units PV is aggregated by area by the roadside unit monitoring system 50 and notified to the operation management system 70.

The operation management system 70 comprises an operation management control device 71 and a vehicle-side communication device 72. Although not shown in the figures, the operation management system 70 is also capable of communicating with the roadside monitoring system 50 and the remote control system 60 via wired, wireless, or a combination thereof.

The operation management control device 71 has a driving mode switching determination unit 71a. The driving mode switching determination unit 71a makes a decision on switching between automatic driving and remote control based on the monitoring results of the roadside unit PV obtained from the roadside unit monitoring system 50 and the status of the automated driving vehicle VE. If the operation management control device 71 determines to switch to remote control, it instructs the remote control system 60 to remotely control the automated driving vehicle VE. Furthermore, when the operation management control device 71 receives notification from the remote control system 60 that remote control has ended, it transmits information regarding the instruction to switch back to automatic driving (automatic driving return request) to the automated driving vehicle VE via the vehicle-side communication device 72. In addition, if the operation management control device 71 determines that there is an urgent risk of collision with an obstacle OB, etc., based on hazard information from the roadside unit PV, it can notify the automated driving vehicle VE of an emergency stop request.

The vehicle-side communication device 72 performs wireless communication with the autonomous vehicle VE. Similar to the control communication device 17, the vehicle-side communication device 72 has general communication functions and performs digital data communication between the vehicle-side communication device 72 and the autonomous vehicle VE via the communication network NT1b, which enables communication using mobile communication lines such as 5G and 4G LTE.

In the above, the autonomous vehicle VE and the roadside unit PV communicate to each other to send and receive information such as future location information TR and vehicle information (vehicle status) when entering and passing through the detection range SA. At this time, the roadside unit PV notifies the roadside unit monitoring system 50 of each piece of information and simultaneously communicates directly with the autonomous vehicle VE within the roadside unit PV's communication area (communication range). The autonomous vehicle VE communicates with the operation management system 70 and the remote control system 60 at regular intervals or when switching driving modes. Table 1 summarizes the communication content exchanged between each device. In Table 1, health information of the autonomous vehicle VE and the roadside unit PV is transmitted at regular intervals.
[Table 1]

The control system 100 of this embodiment performs appropriate mode switching by adding the monitoring status of the roadside unit PV to the information from the automated driving vehicle VE regarding the decision information for switching between automated driving and remote control modes. In other words, the control system 100 can switch the driving mode of the automated driving vehicle VE using the status monitoring of the roadside unit PV, and can perform driving mode switching that encompasses information from the surrounding area of the automated driving vehicle VE and information from the roadside unit PV. In particular, the monitoring status of the roadside unit PV includes the following items.
(1) Communication abnormality between roadside unit PV and autonomous vehicle VE (vehicle-to-infrastructure communication abnormality)
(2) Obstacle detection by the roadside unit sensor 22 of the roadside unit PV (3) Danger determination by the roadside unit PV (4) Failure of the roadside unit sensor 22 of the roadside unit PV (5) Failure detection of the roadside unit PV by the roadside unit monitoring system 50 (abnormality detection by health check)

The above (1) corresponds to communication content a1 in Table 1, the above (2) corresponds to communication content a2 in Table 1, the above (3) corresponds to communication content a3 in Table 1, the above (4) corresponds to communication content a4 and a5 in Table 1, and the above (5) corresponds to communication content a5 in Table 1. Furthermore, the health information of the autonomous vehicle VE shown in Table 1 corresponds to the abnormal information transmitted from the autonomous vehicle VE to the operation management system 70 shown in Figure 1. The abnormal information, based on the health information of the autonomous vehicle VE, includes, for example, cases where communication with the roadside unit PV is interrupted or where there is a problem with the equipment of the autonomous vehicle VE.

The following describes an example of the operation of the control system 100. The operation management system 70, for example, determines to switch from automated driving to remote control in the following situations and switches between automated driving and remote control for the automated driving vehicle VE under its management. In the control center CC, when a problem occurs, i.e., when a remote control switch trigger occurs, the operation management system 70 is notified, a decision is made to switch the driving mode, and an alarm is issued. The control center CC has an operator stationed at the remote control system 60 to ensure a smooth start to remote control.

The operation management system 70 automatically switches from automatic driving to remote driving after determining the driving mode switch when a remote control switch trigger occurs. Similarly, the operation management system 70 automatically switches from remote driving to automatic driving after determining the driving mode switch when a return trigger occurs. In other words, after driving in remote control mode, if the remote control system 60 determines that it is possible to return from remote control to automatic driving, it notifies the operation management system 70 of the end of remote control. The operation management system 70 then instructs the automatic driving vehicle VE to resume automatic driving upon the automatic driving return request.

<Obstacle detection by roadside unit sensors>
The trigger source is the roadside unit PV. When the roadside unit PV detects an obstacle OB that cannot be avoided by autonomous driving on the path of the autonomous driving vehicle VE using the roadside unit sensor 22, it notifies the roadside unit monitoring system 50 of the obstacle information. The roadside unit monitoring system 50 notifies the operation management system 70 of the obstacle information as a result of monitoring by the roadside unit PV. The operation management system 70 determines that it will switch from autonomous driving to remote control based on the obstacle information and instructs the remote control system 60 to remotely control the autonomous driving vehicle VE. After the obstacle OB is bypassed by remote control, the operation management system 70 determines that it will switch back to autonomous driving for the autonomous driving vehicle VE, and the autonomous driving vehicle VE resumes autonomous driving. This allows for a smooth switch to remote control even if an obstacle OB is present on the path of the autonomous driving vehicle VE. Furthermore, it is possible to detect obstacle OB ahead of the autonomous driving vehicle VE, allowing for a switch to remote control with sufficient margin and enabling safe avoidance of the obstacle OB.

<Roadside unit sensor failure>
The trigger source is the roadside unit PV. If the roadside unit sensor 22 fails, the roadside unit PV notifies the roadside unit monitoring system 50 of the failure information of the roadside unit sensor 22. The roadside unit monitoring system 50 notifies the operation management system 70 of the failure information as a result of monitoring the roadside unit PV. The operation management system 70's driving mode switching determination unit 71a determines that it will switch from automatic driving to remote control based on the failure information and instructs the remote control system 60 to remotely control the automatic driving vehicle VE. After the vehicle passes through the detection range SA via remote control, the operation management system 70 determines that it will switch back to automatic driving for the automatic driving vehicle VE, and the automatic driving vehicle VE resumes automatic driving. This ensures that the switch to remote control can be performed smoothly even if the roadside unit sensor 22 fails. Furthermore, for subsequent automatic driving vehicles VE, remote control is initiated before they enter the roadside unit PV that is malfunctioning.

<Malfunction of the roadside control communication device>
The trigger source is the control center CC. If communication with the roadside monitoring system 50 is lost with the roadside monitoring system PV, which monitors the detection range SA, which includes the path of the autonomous vehicle VE, the roadside monitoring system 50 notifies the operation management system 70 of the communication loss information of the roadside monitoring system PV as a result of the roadside monitoring system. The operation management system 70 determines that it will switch from autonomous driving to remote control based on the communication loss information and instructs the remote control system 60 to remotely control the autonomous vehicle VE. After the vehicle has passed through the detection range SA via remote control, the operation management system 70 determines that it will switch back to autonomous driving for the autonomous vehicle VE, and the autonomous vehicle VE resumes autonomous driving. This allows for a smooth switch to remote control even if communication with the roadside monitoring system PV is lost. Furthermore, for subsequent autonomous vehicles VE, remote control is initiated before they enter the roadside monitoring system PV that is malfunctioning.

<Failure of the vehicle-to-infrastructure communication device on the roadside unit>
The trigger source is the autonomous vehicle VE. If communication between the autonomous vehicle VE and the roadside unit PV, which monitors the detection range SA (a detection area including the autonomous vehicle VE's travel path), is lost, the autonomous vehicle VE notifies the operation management system 70 of the communication failure of the roadside unit PV as abnormal information. The operation management system 70 determines, based on the communication failure information, to switch from autonomous driving to remote control and instructs the remote control system 60 to remotely control the autonomous vehicle VE. After the autonomous vehicle VE has passed through the communication area (communication range) of the roadside unit PV, which includes the detection range SA, the operation management system 70 determines to switch back to autonomous driving for the autonomous vehicle VE, and the autonomous vehicle VE resumes autonomous driving. This allows for a smooth switch to remote control even if communication between the autonomous vehicle VE and the roadside unit PV is lost. Furthermore, for subsequent autonomous vehicle VEs, remote control is initiated before they enter the roadside unit PV that is malfunctioning.

If the vehicle-to-infrastructure communication device 16 of the autonomous vehicle VE malfunctions, it is possible to switch to remote control without using information from the roadside unit PV. However, this switch is not as smooth as one using information from the roadside unit PV. Therefore, it is preferable for the autonomous vehicle VE to travel slowly within the detection range SA.

<Equipment malfunction in an autonomous vehicle>
The trigger source is the autonomous vehicle VE. If the autonomous vehicle VE detects an equipment problem through self-diagnosis while driving, it determines that it is impossible to continue autonomous driving and notifies the operation management system 70 of the equipment problem information (vehicle information) as abnormal information. Examples of equipment problems include abnormal output of sensors necessary for autonomous driving (LiDAR, GNSS, etc.). The operation management system 70 determines to switch from autonomous driving to remote control based on the communication interruption information and instructs the remote control system 60 to remotely control the autonomous vehicle VE. After the equipment problem is resolved, the operation management system 70 determines to switch the autonomous vehicle VE back to autonomous driving, and the autonomous vehicle VE resumes autonomous driving.

<Obstacle detection using on-board sensors in autonomous vehicles>
The trigger source is the autonomous vehicle VE. When the autonomous vehicle VE detects an obstacle OB that cannot be avoided by autonomous driving on its path using the onboard sensor 14, it notifies the operation management system 70 of the obstacle information. The operation management system 70 determines, based on the obstacle information, to switch from autonomous driving to remote control and instructs the remote control system 60 to remotely control the autonomous vehicle VE. After the obstacle OB is bypassed by remote control, the operation management system 70 determines to switch the autonomous vehicle VE back to autonomous driving, and the autonomous vehicle VE resumes autonomous driving.

In the above, the operational status of the roadside PV and the autonomous vehicle VE is confirmed by periodic communication between each device. However, a failure in the communication device between the roadside PV and the autonomous vehicle VE can only be detected if the autonomous vehicle VE enters the communication range with the roadside PV. Furthermore, if no information is received from either the roadside PV or the autonomous vehicle VE, the operation management system 70 of the control center CC makes the final decision.

The following describes a series of operational examples for some of the situations in the control system 100 described above, referring to the flowchart. Note that the autonomous vehicle VE continuously transmits its future position information TR to the roadside unit PV while receiving information from the roadside unit PV.

<Obstacle detection by roadside unit sensors>
Figure 3 is a flowchart illustrating a series of operations of the control system 100 when an obstacle occurs within the detection range SA of the roadside unit PV. In Figure 3, the autonomous vehicle VE is initially driving automatically on the road RA according to the driving route TR1 (step S10).

First, the roadside control device 21 of the roadside unit PV, when it detects an obstacle OB on the path of the autonomous vehicle VE using the roadside unit sensor 22 (Yes in step S11), notifies the roadside unit monitoring system 50 of the obstacle information (step S12). Examples of obstacle information include large parked vehicles and road closures due to construction.

When the monitoring control device 51 of the roadside unit monitoring system 50 receives obstacle information from the roadside unit PV (Yes in step S13), it notifies the operation management system 70 of the obstacle information as a result of monitoring the roadside unit PV (step S14).

The operation management control device 71 of the operation management system 70, after receiving obstacle information from the roadside unit monitoring system 50, searches for an automated driving vehicle VE entering the detection range SA of the corresponding roadside unit PV (step S15).

The operation management control device 71 of the operation management system 70, acting as a driving mode switching determination unit 71a, determines whether the notified obstacle OB can be avoided using the detour route TR2 registered in the operation management system 70 (step S16). If the obstacle OB cannot be avoided (No. in step S16), it determines, based on the obstacle information, to switch from automatic driving to remote control, and instructs the remote control system 60 to remotely control the automatic driving vehicle VE, requesting remote control from the automatic driving vehicle VE (step S17). In other words, the operation management system 70 notifies the remote control system 60 of the remote control request.

The remote control device 62 of the remote control system 60 initiates remote control of the automated driving vehicle VE traveling within the detection range SA of the corresponding roadside unit PV (step S18). Specifically, the remote control device 62 transmits a driving control signal corresponding to the operator's operation of the remote control terminal 61 to the remote control device 13 of the automated driving vehicle VE. In response to the remote control request from the remote control system 60 (Yes in step S19), the automated driving vehicle VE switches from automated driving to remote control, and driving is performed under remote control (step S20). Remote control continues until the avoidance of obstacles OB that would interfere with automated driving is completed.

The remote control device 62 of the remote control system 60 notifies the operation management system 70 of a request to return to automatic driving, indicating the end of remote control, after the vehicle has detoured around an obstacle (OB) via remote control, i.e., after recovering from a situation that would have interfered with automatic driving (step S21). This request to return to automatic driving is notified, for example, by an operator via the remote control system 60 to the operation management system 70.

The operation management control device 71 of the operation management system 70 receives notification of the end of remote control from the remote control system 60 and, as the driving mode switching determination unit 71a, switches from remote control to automatic driving and restores automatic driving of the automatic driving vehicle VE (step S22). That is, the operation management system 70 notifies the automatic driving vehicle VE of a request to resume automatic driving. The automatic driving vehicle VE, in response to the automatic driving resume request from the operation management system 70 (Yes in step S23), resumes automatic driving (step S24).

In step S16, the operation management control device 71 of the operation management system 70 notifies the automated driving vehicle VE of the detour route TR2 if it is possible to avoid the obstacle OB on the detour route TR2 (Yes in step S16) (step S25). The automated driving vehicle VE receives the detour route TR2 from the operation management system 70 and, without switching from automated driving mode (No in step S19), drives automatically according to the detour route TR2 (step S26). After detouring the obstacle OB by automated driving, the automated driving vehicle VE returns to the original driving route TR1 and maintains automated driving (step S27).

<Failure of the vehicle-to-infrastructure communication device on the roadside unit>
Figure 4 is a flowchart illustrating a series of operations of the control system 100 when a communication failure occurs between the roadside unit PV and the autonomous vehicle VE (failure of the roadside unit PV's vehicle-to-infrastructure communication device 23). In Figure 4, the autonomous vehicle VE is initially driving automatically on road RA according to the driving route TR1 (step S30). Also, the roadside unit PV has a malfunction in its vehicle-to-infrastructure communication device 23 (step S31).

First, the autonomous driving vehicle VE, as the autonomous driving device 12, determines, based on its own position, that it has entered the communication area with the roadside unit PV, which includes the detection range SA (Yes in step S32). If it determines that it cannot communicate with the roadside unit PV (Yes in step S33), it notifies the operation management system 70 of the communication failure information (abnormal information) as a communication failure with the roadside unit PV (Step S34). Note that the communication area with the roadside unit PV can be set by both the autonomous driving vehicle VE and the roadside unit PV.

When the operation management control device 71 of the operation management system 70 receives a communication interruption notification from the autonomous vehicle VE to the roadside unit PV (Yes in step S35), the operation mode switching determination unit 71a determines, based on the communication interruption information, that it will switch from autonomous driving to remote control. It then instructs the remote control system 60 to remotely control the autonomous vehicle VE and requests remote control from the autonomous vehicle VE (step S36).

The remote control device 62 of the remote control system 60 initiates remote control of the corresponding autonomous vehicle VE (step S37). Upon receiving a remote control request from the remote control system 60 (Yes in step S38), the autonomous vehicle VE switches from autonomous driving to remote control and begins driving under remote control (step S39). Remote control continues until the vehicle has completed passing through the communication area of the roadside unit PV. For subsequent autonomous vehicles VE, remote control is initiated before they enter the roadside unit PV that is malfunctioning.

The remote control device 62 of the remote control system 60 notifies the operation management system 70 of a request to return to automatic driving, indicating the end of remote control, after the roadside PV has passed through the communication area via remote control, that is, after recovering from a situation that would interfere with automatic driving (step S40).

The operation management control device 71 of the operation management system 70 receives notification of the end of remote control from the remote control system 60 and, as the driving mode switching determination unit 71a, switches from remote control to automatic driving and restores automatic driving of the automatic driving vehicle VE (step S41). The automatic driving vehicle VE resumes automatic driving in response to the automatic driving restoration request from the operation management system 70 (Yes in step S42) (step S43).

In step S38, if the automatic driving mode is maintained, the automatic driving vehicle VE does not switch from automatic driving to a driving mode (No. in step S38), but continues to drive automatically according to the driving route TR1 (step S44).

<Roadside unit sensor failure>
Figure 5 is a flowchart illustrating a series of operations of the control system 100 when a failure occurs in the roadside unit PV. In Figure 5, the automated driving vehicle VE is initially driving automatically on the road RA according to the driving route TR1 (step S50). Failures in the roadside unit PV include failures of the roadside unit sensor 22 and failures of the roadside control device 21, and the same operations occur in these cases as well. If the control communication device 24 of the roadside unit PV is faulty, the failure of the roadside unit PV is notified to the roadside unit monitoring system 50. In this case, the roadside unit PV failure is determined by the interruption of communication of the roadside unit PV's health information, which is notified to the roadside unit monitoring system 50 at a certain time interval.

First, the roadside PV unit, upon detecting a communication interruption or output abnormality of the roadside sensor 22's health information (Yes in step S51), determines that the roadside sensor 22 has malfunctioned. If this determination is made, the roadside PV unit notifies the roadside monitoring system 50 of the malfunction (step S52). Examples of output abnormalities in the roadside sensor 22 include the camera used as the roadside sensor 22 failing to transmit information at the specified fps, or the output image having missing parts.

When the monitoring control device 51 of the roadside unit monitoring system 50 receives fault information for a roadside unit PV (Yes in step S53), it notifies the operation management system 70 of the fault information as a result of monitoring the roadside unit PV (step S54).

The operation management control device 71 of the operation management system 70, after receiving fault information from the roadside unit monitoring system 50, searches for an automated driving vehicle VE entering the detection range SA of the relevant roadside unit PV (step S55).

The operation management control device 71 of the operation management system 70, acting as an operation mode switching determination unit 71a, determines, based on fault information, to switch from automatic operation to remote control. It then instructs the remote control system 60 to remotely control the automatic vehicle VE, requesting remote control from the automatic vehicle VE (step S56).

The remote control device 62 of the remote control system 60 initiates remote control of the corresponding autonomous vehicle VE (step S57). Upon receiving a remote control request from the remote control system 60 (Yes in step S58), the autonomous vehicle VE switches from autonomous driving to remote control and begins driving under remote control (step S59). Remote control continues until the vehicle has completed passing through the communication area of the roadside unit PV. For subsequent autonomous vehicles VE, remote control is initiated before they enter the roadside unit PV that is malfunctioning.

The remote control device 62 of the remote control system 60 notifies the operation management system 70 of a request to return to automatic driving, indicating the end of remote control, after the roadside PV has passed through the communication area via remote control, that is, after recovering from a situation that would interfere with automatic driving (step S60).

The operation management control device 71 of the operation management system 70 receives notification of the end of remote control from the remote control system 60 and, as the driving mode switching determination unit 71a, switches from remote control to automatic driving and restores automatic driving of the automatic driving vehicle VE (step S61). The automatic driving vehicle VE resumes automatic driving in response to the automatic driving restoration request from the operation management system 70 (Yes in step S62) (step S63).

In step S58, if the automatic driving mode is maintained, the automatic driving vehicle VE does not switch from automatic driving to a driving mode (No. in step S58), but continues to drive automatically according to the driving route TR1 (step S64).

In the control system 100 of the embodiment described above, by utilizing the monitoring results of the roadside PV when deciding whether to switch between automatic driving and remote control, it is possible to identify in advance locations where it would be dangerous for the automatic driving vehicle VE to travel under automatic driving conditions, and to smoothly switch from automatic driving to remote control. Furthermore, it can also respond to a decrease in the functionality of automatic driving due to a failure of the roadside PV. In other words, by taking into account the monitoring status of the roadside PV, such as its status (liveness/death information), when deciding whether to switch between automatic driving and remote control modes, an appropriate mode switch can be performed.

〔others〕
This invention is not limited to the embodiments described above, and can be implemented in various forms without departing from its spirit.

In the above embodiment, the shape of the road RA is merely an example and is not limited to this; the method can be applied to various shapes and structures.

In the above embodiment, the reference point of the travel route TR1 was set to the center of the front of the autonomous vehicle VE. However, it can be appropriately changed to, for example, the center of the vehicle body or the center of the rear end of the autonomous vehicle VE.

In the above embodiment, the control system 100 was described as comprising an on-board system VS, a roadside unit PV, a roadside unit monitoring system 50 in the control center CC, a remote control system 60, and an operation management system 70. However, it may also be composed of the roadside unit PV, the roadside unit monitoring system 50, the remote control system 60, and the operation management system 70, regardless of whether or not information is provided from the autonomous vehicle VE.

Furthermore, while the above assumes that the roadside PVs and other components of the control system 100 are installed near the site, i.e., near the road RA, this is not limited to this arrangement. For example, sections responsible for various information processing and data management could be located remotely as management servers for the control center CC, or various processing and data storage could be performed on the cloud.

11... Driving operation unit, 12... Automatic driving device, 12a... Future position information generation unit, 13... Remote control device, 14... On-board sensor, 15... Acquisition medium, 16... Infrastructure-to-vehicle communication device, 17... Control communication device, 18... Route data unit, 19... Map data unit, 21... Roadside control device, 21a... Roadside determination unit, 21b... Sensor interface, 22... Roadside machine sensor, 23... Infrastructure-to-vehicle communication device, 24... Control communication device, 25... Map data unit, 50... Roadside machine monitoring system, 51... Monitoring and control device, 52... Roadside communication device, 60... Remote control Vertical System, 61...Remote Control Terminal, 62...Remote Control Device, 63...Vehicle Communication Device, 70...Operation Management System, 71...Operation Management Control Device, 71a...Driving Mode Switching Determination Unit, 72...Vehicle-side Communication Device, 100...Control System, BU...Bus, CC...Control Center, GM...Forward Vehicle, MB...Moving Object, OB...Obstacle, PV...Roadside Unit, RA...Road, RA1...Lane, RA2...Lane, SA...Detection Range, TR...Future Position Information, TR1...Driving Route, TR2...Detour Route, VE...Autonomous Driving Vehicle, VS...In-vehicle System

Claims (7)

A roadside unit that communicates with the autonomous vehicle and monitors the surroundings,
A roadside unit monitoring system that monitors the aforementioned roadside unit,
A remote control system for remotely controlling the aforementioned autonomous vehicle,
An operation management system that obtains monitoring results from the roadside unit monitoring system, makes a decision on switching between automated driving and remote control based on the monitoring results, and if it decides to switch to remote control, instructs the remote control system to remotely control the automated driving vehicle,
Equipped with ,
The roadside unit monitoring system is a control system having a monitoring and control device that performs health checks on the roadside unit at regular intervals to monitor whether the roadside unit is operating normally . The control system according to claim 1, wherein the operation management system communicates with the autonomous vehicle and, upon receiving abnormal information from the autonomous vehicle, instructs the remote control system to remotely control the autonomous vehicle. The roadside unit has a roadside unit sensor that monitors a detection area including the travel path of the autonomous vehicle, and when the roadside unit sensor detects an obstacle on the travel path of the autonomous vehicle, it notifies the roadside unit monitoring system of the obstacle information.
The roadside unit monitoring system notifies the operation management system of the obstacle information as a result of the monitoring of the roadside unit.
The control system according to any one of claims 1 and 2, wherein the operation management system determines to switch from automatic driving to remote control based on the obstacle information. The roadside unit has a roadside unit sensor that monitors a detection area including the travel path of the autonomous vehicle, and if the roadside unit sensor fails, it notifies the roadside unit monitoring system of the failure information of the roadside unit sensor.
The roadside equipment monitoring system notifies the operation management system of the failure information as a result of monitoring the roadside equipment.
The control system according to any one of claims 1 and 2, wherein the operation management system determines to switch from automatic driving to remote control based on the fault information. If communication with the roadside unit monitoring system is lost when communication with the roadside unit monitoring the detection area including the route of the autonomous vehicle is lost, the roadside unit monitoring system notifies the operation management system of the communication loss information of the roadside unit as a result of the roadside unit monitoring.
The control system according to any one of claims 1 and 2, wherein the operation management system determines to switch from automatic driving to remote control based on the communication interruption information. If the autonomous vehicle loses communication with the roadside unit that monitors the detection area including the autonomous vehicle's travel path, it notifies the operation management system of the communication loss information of the roadside unit as abnormal information.
The control system according to claim 2, wherein the operation management system determines to switch from automatic driving to remote control based on the communication interruption information. When the remote control system determines that it is possible to return from remote control to automatic driving, it notifies the operation management system of the termination of the remote control.
The operation management system is a control system according to any one of claims 1 to 6, which instructs the autonomous vehicle to perform autonomous driving.