JP7679762B2 - Remotely operated vehicle control device - Google Patents

Description

This disclosure relates to a control device for a remotely operated vehicle.

Patent Document 1 discloses a technology for acquiring information indicating the equipment status of a remotely driven vehicle that is the subject of remote driving. An information processing device controls the remote driving device based on information indicating the equipment status of the remotely driven vehicle and information indicating the equipment status of a remote driving device that remotely drives the remotely driven vehicle.

JP 2019-174993 A

As shown in Patent Document 1, a technology for remotely driving a vehicle is known. A vehicle that is the subject of remote driving is called a remotely driven vehicle or a remotely driven vehicle. A remotely driven vehicle is driven based on operations input to a remote driving device called a remote cockpit. When a request to start remote driving is received, remote driving starts. However, if remote driving is started immediately without checking the state of the remotely driven vehicle and the remote cockpit, a malfunction may occur due to a large discrepancy between the state of the remotely driven vehicle and the state of the remote cockpit. Therefore, before starting remote driving, it is necessary to accurately determine whether or not remote driving is possible. Here, the decision to start remote driving may be made in the remote cockpit. However, the remote cockpit is merely an operating system, and the occurrence rate of erroneous judgment due to malfunctions, etc. is higher than that of a vehicle. In order to suppress erroneous judgment, it is preferable to make the decision to start remote driving on the vehicle side.

The objective of this disclosure is to provide technology that can reliably determine whether or not remote driving can be started for a vehicle that is the subject of remote driving.

The present disclosure relates to a control device for a remotely operated vehicle.
The control device includes:
Obtaining a target tire angle from a remote driving device and obtaining an actual tire angle of the remote driving vehicle;
In response to the match between the target tire angle and the actual tire angle, switching the driving of the remotely driven vehicle to remote driving by the remote driving device.
It is characterized by:

According to the present disclosure, in a vehicle that is the subject of remote driving, a highly reliable determination can be made as to whether remote driving can be started. By making a highly reliable determination, the occurrence of problems when remote driving is started is suppressed.

FIG. 4 is a conceptual diagram for explaining a remote operation switching condition in the embodiment of the present disclosure. FIG. 4 is a conceptual diagram for explaining a remote operation switching condition in the embodiment of the present disclosure. 1 is a conceptual diagram for explaining visual support provided by a remote driving system according to an embodiment of the present disclosure. 1 is a conceptual diagram for explaining visual support provided by a remote driving system according to an embodiment of the present disclosure.

An embodiment of the present disclosure will be described with reference to the attached drawings.

1. Control device for remote driving vehicle The control device according to this embodiment is mounted on a vehicle and enables remote driving of the vehicle. The control device according to this embodiment may be a device built into the vehicle as a vehicle component, or may be an external device that can be attached to and detached from the vehicle. The latter is called a remote driving kit. In the following description, the control device is assumed to be a remote driving kit. A vehicle equipped with a remote driving kit and capable of remote driving is called a remote driving vehicle. A remote driving device for remotely driving the remote driving vehicle is called a remote cockpit. The remote driving kit enables remote driving of the vehicle by transmitting a target control amount to the remote driving vehicle, communicating with the outside of the vehicle, and transmitting the state of the remote driving vehicle to the remote cockpit. The remote cockpit is equipped with a monitor for displaying information acquired by an on-board camera and an operation system for inputting a target control amount of the remote driving vehicle, and is operated by a remote driver. In addition, the remote driving vehicle may be simultaneously equipped with an automatic driving kit that enables automatic driving.

The remotely driven vehicle starts remote driving when a request to start remote driving is made. Examples of the timing at which the start of remote driving is requested include when the remotely driven vehicle, which has been driven automatically by an automatic driving kit, approaches a location that requires assistance from remote driving, such as an intersection, or when the occupant of the remotely driven vehicle requests remote driving. When remote driving of the remotely driven vehicle starts and the vehicle is switched to a remote driving state, the vehicle is driven according to the operations input to the remote cockpit. However, the remotely driven vehicle is not necessarily in a state in which it can be immediately switched to a remote driving state when a request to start remote driving is made. For example, the tires of the remotely driven vehicle may be turned to the right, and the steering wheel in the remote cockpit may be reset to the center position. If remote driving is started in such a state, a malfunction may occur in which the remotely driven vehicle is not driven as intended by the remote driver.

In this embodiment, therefore, a remote driving switching condition is set as a condition for safely switching the remotely driven vehicle to the remote driving state, and the remotely driven vehicle is switched to the remote driving state only when the remote driving switching condition is satisfied. For example, by setting the remote driving switching condition to match the state of the operation system in the remote cockpit and the state of the control system of the remotely driven vehicle, the control of the vehicle becomes smoother when switching to the remote driving state, improving the ride comfort of the passengers in the remotely driven vehicle. Alternatively, for example, by setting the remote driving switching condition to be sufficient brake operation amount by the remote driver, sudden acceleration of the remotely driven vehicle can be avoided.

In this embodiment, when a request to start remote driving is made, the decision of whether or not to switch to the remote driving state is made not on the remote cockpit side, but on the remote driving vehicle side in which the remote driving kit is installed. Devices installed on the vehicle side have a lower risk of failure and are also highly expandable compared to operation systems such as remote cockpits. Therefore, by making the decision on the vehicle side, the reliability of the decision can be greatly improved. Also, by making the decision on the vehicle side, the remote cockpit can be made of general-purpose parts, which reduces costs. To further improve reliability, the decision system of the remote driving kit may be designed to be redundant.

Here, it is appropriate that the remote driving switching condition is that the state of the remote cockpit's operation system and the state of the remotely driven vehicle's control system have a specified correspondence. For example, it is assumed that the remote driving switching condition is that the range of the shift lever in the remote cockpit and the range of the shift lever in the remotely driven vehicle match. Alternatively, for example, the remote driving switching condition may be that the acceleration amount input to the remote cockpit is compared with the vehicle speed of the remotely driven vehicle and that the acceleration amount is such that sudden acceleration or deceleration does not occur. Among these, one of the most important remote driving switching conditions is that the actual tire angle and the target tire angle match. The actual tire angle is the actual tire angle of the remotely driven vehicle, and the target tire angle is the value input by the remote driver operating the remote cockpit's operation system, such as the steering wheel, converted into the tire angle of the remotely driven vehicle.

A typical operation system for controlling the tire angle of a remote-driven vehicle is a steering wheel. However, the operation system is not limited to a steering wheel, and a joystick, a touch panel, an input device in the shape of a game controller, etc. may also be used as the operation system. For example, the target tire angle may be calculated by the remote driver operating the joystick left and right. When the operation system for controlling the tire angle is a steering wheel, the operation angle of the steering wheel is called the steering angle.

Here, the agreement between the actual tire angle and the target tire angle includes not only a perfect agreement, but also the difference between the actual tire angle and the target tire angle being smaller than a certain value. Furthermore, the agreement between the actual tire angle and the target tire angle may be determined based on the tire angle, or, if the operation system is steering, based on the steering angle. In other words, the remote driving switching condition may be satisfied when the value obtained by converting the actual tire angle into a steering angle matches the steering angle of the remote cockpit.

When the remotely driven vehicle is being driven autonomously, the actual tire angle is more likely to change compared to the state of other control systems of the remotely driven vehicle. Therefore, differences are particularly likely to occur between the actual tire angle and the target tire angle between the remote cockpit and the remotely driven vehicle. Therefore, if the remote driving switching condition is that the actual tire angle and the target tire angle match, the discomfort felt by the remote driver when switching to the remote driving state can be particularly reduced. In addition, the movement of the vehicle when switching to the remote driving state becomes smoother, making it possible to achieve both comfortable operation for the remote driver and a comfortable ride for the passengers.

The control device according to this embodiment therefore sets the condition for switching to remote driving as being that the actual tire angle and the target tire angle match. In other words, the remote driving kit acquires information on the actual tire angle and the target tire angle. Then, when the actual tire angle and the target tire angle match, it determines that the condition for switching to remote driving is met, and switches the remotely driven vehicle to a remote driving state.

Figure 1 conceptually shows an example of a remote driving switching condition in this embodiment. In Figure 1, the operation system for operating the tire angle of the remotely driven vehicle is the steering wheel. Information about the vehicle state, including the actual tire angle, is notified to the remote cockpit. The remote driver operates the remote cockpit while checking the information about the vehicle state. At this time, the remote cockpit may assist the operation of the remote driver by applying a reaction force to the steering wheel. The steering angle input by the operation of the remote driver is converted to a target tire angle and transmitted to the remotely driven vehicle. The remote driving kit compares the actual tire angle with the target tire angle, and when the actual tire angle and the target tire angle match, the remotely driven vehicle is switched to a remote driving state.

This flow is shown, for example, in FIG. 2. When the remote driving kit is activated, the remote driving vehicle is in a state where remote driving is possible. When the start of remote driving is requested, the remote driving kit notifies the remote cockpit of information about the vehicle state acquired from the remote driving vehicle. The remote cockpit receives the notification from the remote driving kit and notifies the remote driving kit of information about the state of the operation system. The information notified from the remote cockpit to the remote kit includes the target tire angle or the steering angle corresponding to the target tire angle. If the actual tire angle and the target tire angle match based on the notified information, the remote driving kit determines that the remote driving switching condition is met and sends a request to switch to the remote driving state to the remote driving vehicle. When the switching is completed, a switching completion notification is sent from the remote driving vehicle to the remote cockpit through the remote driving kit, and the operation system of the remote cockpit displays that the switching is completed. After the switching, information for remote driving is transmitted between the remote driving vehicle, the remote driving kit, and the remote cockpit, and the remote driving vehicle is remotely driven.

2. Remote Driving System The control device according to this embodiment constitutes a remote driving system together with a remote cockpit.

If the remote driving switching condition is not satisfied, it is necessary to make the actual tire angle and the target tire angle coincide in order to start remote driving. There are two possible methods for making the actual tire angle and the target tire angle coincide: changing the actual tire angle and changing the target tire angle. The remote driving system according to this embodiment does not change the actual tire angle. Instead, the remote driver is prompted to operate the steering wheel and other operating systems so that the target tire angle coincides with the actual tire angle, and the operation of the remote driver to make the target tire angle coincide with the actual tire angle is supported. Since the remote driver himself is required to operate the vehicle in order to make the target tire angle coincide with the actual tire angle, the remote driver can more realistically grasp the state of the remotely driven vehicle, and remote driving becomes smoother. Furthermore, it is not necessary to install a control mechanism for making the target tire angle coincide with the actual tire angle in the operating system of the remote cockpit, which reduces costs.

The operation for matching the target tire angle with the actual tire angle and satisfying the remote driving switching condition is called initial state preparation. In a flow similar to that of FIG. 2, when the remote cockpit receives information about the vehicle state, the remote driving system determines whether the actual tire angle and the target tire angle match. If they do not match, the remote driving system requests the remote driver to match the actual tire angle with the target tire angle. This request is conveyed to the remote driver, for example, by being displayed on a monitor in the remote cockpit or by audio guidance being output from a speaker in the remote cockpit.

The remote driver who receives the request must operate the steering wheel and other operating systems himself to prepare for the initial state. The remote driving system receives a confirmation request from the remote driver who has prepared the initial state and determines again whether the target tire angle and actual tire angle match. If the target tire angle and actual tire angle do not match even after the second determination, the remote driving system again requests the remote driver to match the target tire angle and actual tire angle. If the target tire angle and actual tire angle match, the remote driving vehicle switches to a remote driving state.

The remote driving system according to this embodiment supports the operation of the remote driver who is preparing for the initial state. Support can be provided in the form of visual support or tactile support. Here, visual support will be explained first. Visual support is provided by visually instructing the remote driver on the operation direction and amount of the remote cockpit operation system required to satisfy the remote driving switching conditions by displaying the operation direction and amount of the remote cockpit operation system required to satisfy the remote driving switching conditions on a monitor in the remote cockpit, etc.

The remote driving system acquires information about the actual tire angle and information about the target tire angle input to the remote cockpit. Then, the difference between the actual tire angle and the target tire angle is visually displayed. For example, as shown in FIG. 3, the predicted tire trajectory is displayed on the monitor. In the monitor, the tire trajectory is drawn with dotted lines and dashed lines superimposed on the scenery around the vehicle acquired by the vehicle-mounted camera. The tire trajectory drawn with dotted lines displays the predicted trajectory based on the actual tire angle, and the tire trajectory drawn with dashed lines displays the predicted trajectory based on the target tire angle. The remote driver can match the actual tire angle and the target tire angle by operating the steering wheel or other operating system so that the predicted trajectory based on the actual tire angle and the predicted trajectory based on the target tire angle match. In FIG. 3, the predicted trajectory based on the actual tire angle and the predicted trajectory based on the target tire angle are displayed with dotted lines and dashed lines, but the two predicted trajectories may be distinguished by displaying them with solid lines of different colors. In addition, in the display of the predicted tire trajectory line shown in FIG. 3, when the remote driver operates the steering wheel or other operating system in the opposite direction to the target, the predicted tire trajectory line may be changed to a display that emphasizes the correct operating direction by blinking or changing to a highlighted color.

The tire angle may also be displayed numerically. The numerical display may be displayed above or below the monitor projecting the scenery around the vehicle, or may be superimposed on the monitor projecting the scenery around the vehicle. For example, it may be displayed as "Required operation amount: 50 degrees left remaining (remotely driven vehicle tire angle: 150 degrees left)," allowing the remote driver to know the required operation amount specifically as a numerical value.

Figure 4 shows another example of visual support. (1) is an example of a display using bars. The left dotted line indicates the maximum tire angle when the tires of the remotely driven vehicle are turned to the left, the right dotted line indicates the maximum tire angle when the tires of the remotely driven vehicle are turned to the right, and the middle dotted line indicates the neutral position of the tires of the remotely driven vehicle. The actual tire angle is represented as the position of the solid line of the bar, and the amount of change in the target tire angle from the neutral position is shown by the diagonal lines. The left or right end of the diagonal line bar is the target tire angle. The remote driver can operate the steering wheel and other operating systems according to the display to match the actual tire angle with the target tire angle.

Figure 4 (2) is an example of visual support that displays the amount of rotation. The actual tire angle is indicated by a solid line, and the target tire angle is indicated by a dashed line, and the amount of rotation is shown. The dotted line indicates the neutral position of the tire. The position of the dashed line changes depending on the steering angle input to the remote cockpit. The required amount of rotation is displayed by an arrow, and the remote driver can match the actual tire angle with the target tire angle by operating the steering wheel or other control system so as to satisfy the amount of rotation displayed by the arrow. At this time, if the remote driver operates the steering wheel or other control system in the opposite direction, the display indicating the target tire angle may be highlighted by flashing or changing to a highlighted color.

Any one of the visual supports described above may be selected, or multiple may be selected simultaneously. The remote driver can easily match the actual tire angle with the target tire angle by operating the steering wheel and other control systems in accordance with the visual support.

Next, an example of haptic support when the operating system for manipulating the tire angle is a steering wheel will be described. Haptic support is provided by applying a reaction force to the steering wheel. When the remote driver operates the steering wheel in a direction that causes the target tire angle and the actual tire angle to move apart, a reaction force to the steering wheel is applied as a steering reaction force that prevents this. Conversely, when the steering operation by the remote driver is in the correct direction, that is, in a direction that brings the target tire angle and the actual tire angle closer together, the remote driving system does not apply a steering reaction force. This allows the remote driver to know haptically whether or not he is steering in the correct direction.

When the remote driving switching condition is satisfied, the initial state preparation is completed. When the initial state preparation is completed by visual support or tactile support, that is, when the target tire angle and the actual tire angle match, the remote driver may be visually or tactilely instructed. Here, when the target tire angle and the actual tire angle match, it may include when the difference between the target tire angle and the actual tire angle becomes smaller than a certain value. For example, when the predicted tire trajectory line is displayed as in FIG. 3, when the remote driving switching condition is satisfied, the predicted trajectory line is merged into one or the color of the predicted trajectory line is changed. Alternatively, when the remote driving switching condition is satisfied, the remote driver may be instructed by applying a small vibration to the steering wheel, driver's seat, brake pedal, etc. of the remote cockpit.

After the driver is notified that the initial state preparation is complete, the remotely driven vehicle is switched to a remote driving state. If the remote driving system has functions associated with remote driving, the functions may be enabled at the same time when the remotely driven vehicle is switched to the remote driving state. An example of a function associated with remote driving is a virtual steering reaction force generation function. The virtual steering reaction force generation function is a function for improving the operating comfort of the remote operator. By generating a virtual steering reaction force, a reaction force that mimics the state of the vehicle is applied to the remote cockpit based on information about the vehicle state, such as the vehicle speed and steering current, of the remotely driven vehicle acquired by the remote driving system.

3. Modifications As a modification of the control device according to the present embodiment, the remote driving switching condition may be set for each control mechanism, rather than for all control mechanisms of the remotely driven vehicle. For example, the agreement between the target tire angle and the actual tire angle is set as the remote driving switching condition only for the tire control mechanism of the remotely driven vehicle. When the target tire angle and the actual tire angle match, the remote driving kit switches only the tire control mechanism of the remotely driven vehicle to the remote driving state, and causes the tire angle of the remotely driven vehicle to start following the target tire angle. Similarly, the remote driving switching condition for each control mechanism can be set for control mechanisms other than the tire control mechanism. By setting the remote driving switching condition for each control mechanism, a smoother switch to the remote driving state can be performed.

As described above, the control device according to the present embodiment allows the vehicle to determine whether or not the remotely driven vehicle can be switched to a remotely driven state when a request to start remote driving is received. By performing the determination on the vehicle side, the reliability of the determination is improved, the occurrence of malfunctions is suppressed, and the vehicle can be safely switched to a remotely driven state.

Claims (4)

A remote driving device for remotely driving a remote driving vehicle,
The remote-controlled vehicle is configured to switch to remote control driving when the target tire angle and the actual tire angle match,
The remote operation device is
providing the target tire angle to the remotely driven vehicle and obtaining the actual tire angle from the remotely driven vehicle;
A remote driving device that urges a remote driver to perform an operation to match the target tire angle with the actual tire angle by superimposing and displaying an image acquired from an on-board camera of the remote-driven vehicle, a first trajectory line indicating a predicted tire trajectory based on the target tire angle, and a second trajectory line indicating a predicted tire trajectory based on the actual tire angle. The remote driving device according to claim 1 ,
A remote driving device comprising: a notification means for visually or tactilely notifying that the steering device is being operated in a direction in which the first trajectory line and the second trajectory line are separated. The remote driving device according to claim 1 ,
A remote driving device comprising: a notification means for visually or tactilely notifying that the first trajectory line and the second trajectory line substantially coincide with each other. A remote driving system comprising the remote driving device according to any one of claims 1 to 3 and the remote driving vehicle.

Images