JP7616133B2 - Remote Operation System - Google Patents

Description

The present invention relates to a remote operation system.

Japanese Patent Publication No. 10-145777 is known as a technical document related to a remote driving system. This publication describes a system in which a vehicle is equipped with multiple cameras and remotely controlled while viewing the camera images.

Japanese Patent Application Publication No. 10-145777

However, if all vehicles to be remotely driven are equipped with cameras, microphones, and other equipment for remote driving, a large amount of equipment would be required for each vehicle, which would increase vehicle costs.

One aspect of the present invention is a remote driving system for a remote driver to remotely drive a remotely driven vehicle, comprising a remote driving interface for inputting operation information of the remote driver, an operation information acquisition unit for acquiring the operation information of the remote driver from the remote driving interface via wireless communication, a camera for acquiring image information of the surroundings of the remotely driven vehicle, a microphone for acquiring sound information of the surroundings of the remotely driven vehicle, a vehicle information transmission unit for transmitting the image information and sound information to the remote driving interface, and a portable remote driving kit having a vehicle instruction unit for transmitting remote driving instructions to the remotely driven vehicle based on the operation information of the remote driver, and a remote driving drone for transporting the portable remote driving kit.

According to one aspect of the present invention, it is possible to realize remote driving of a vehicle by a remote driver without equipping the vehicle with equipment such as a camera or microphone for remote driving, or equipment for direct communication with a remote driving interface.

FIG. 1 is a diagram for explaining a remote driving system according to an embodiment. 1 is a diagram for explaining the state in which the portable remote driving kit is mounted on a vehicle to be remotely driven. FIG. 2 is a block diagram showing an example of the configuration of a portable remote operation kit.

The following describes an embodiment of the present invention with reference to the drawings.

Figure 1 is a diagram for explaining a remote driving system according to one embodiment. The remote driving system shown in Figure 1 is a system for realizing remote driving of a remotely driven target vehicle M. The remotely driven target vehicle M is a vehicle that has a function that allows it to be remotely driven by this remote driving system.

As shown in FIG. 1, the remote driving system includes a remote driving interface 1 through which a remote driver R performs remote driving, a portable remote driving kit 2, and a remote driving drone 3.

The remote driving interface 1 is an interface that the remote driver R uses to remotely drive the remotely driven vehicle M. The remote driving interface 1 is provided with an information providing unit and an operation input unit. The information providing unit is a device that provides the remote driver R with image information and sound information about the surroundings of the remotely driven vehicle M. The information providing unit has a display that provides image information to the remote driver and a speaker that provides sound information.

The operation input unit is a device into which remote driving operations are input by the remote driver R. The operation input unit is composed of various input devices that imitate, for example, a steering wheel, an accelerator pedal, and a brake pedal. The operation input unit may be capable of operating the remote driving drone 3. The operation input unit may be equipped with a separate controller for operating the remote driving drone 3, or may be configured to operate the remote driving drone 3 and the remote driving target vehicle M using the same device. The operation input unit may be composed of a keyboard, a mouse, or a touch panel.

The portable remote driving kit 2 is a device separate from the vehicle, and realizes remote driving of the remotely driven vehicle M by communicating with the remotely driven vehicle M. The portable remote driving kit 2 may be configured to be detachable from the remotely driven vehicle M. Figure 2 is a diagram for explaining the state in which the portable remote driving kit 2 is attached to the remotely driven vehicle M.

As shown in FIG. 2, the portable remote driving kit 2 is attached to the remotely driven vehicle M and communicates with the remotely driven vehicle M, allowing the remote driver R to perform remote driving via the portable remote driving kit 2. The portable remote driving kit 2 may perform wireless communication with the remotely driven vehicle M, or when attached to the remotely driven vehicle M, wired communication may be performed by connecting a communication unit. Details of the portable remote driving kit 2 will be described later.

The remote driving drone 3 is a drone (small unmanned aerial vehicle) that carries the portable remote driving kit 2. The remote driving drone 3 may have a well-known drone configuration. The remote driving drone 3 may be configured so that it can be remotely controlled by a remote driver from the remote driving interface 1. The remote driving drone 3 may be configured so that it can be remotely controlled by a remote driver separate from the vehicle. The remote driving drone 3 may carry the portable remote driving kit 2 and attach the portable remote driving kit 2 to the remote driving target vehicle M to be remotely driven. The remote driving drone 3 and the portable remote driving kit 2 may be configured as an integrated device.

Next, the configuration of the portable remote operation kit 2 will be described with reference to FIG. 3. FIG. 3 is a block diagram showing an example of the configuration of the portable remote operation kit 2. As shown in FIG. 3, the portable remote operation kit 2 has a remote operation calculation unit 30 that performs calculation processing.

The remote operation calculation unit 30 is an electronic control unit having, for example, a CPU [Central Processing Unit] and a storage unit such as a ROM [Read Only Memory], a RAM [Random Access Memory], and a HDD [Hard Disk Drive]. In the remote operation calculation unit 30, for example, various functions are realized by the CPU executing programs stored in the storage unit.

A camera 21, a microphone 22, and a communication unit 23 are connected to the remote driving calculation unit 30. The camera 21 is a device for acquiring image information of the surroundings of the remotely driven vehicle M. Multiple cameras 21 may be provided to capture images in multiple directions, not just the front of the remotely driven vehicle M. The microphone 22 is a device for acquiring sound information of the surroundings of the remotely driven vehicle M. Multiple microphones 22 may also be provided facing different directions.

The communication unit 23 is a device for communicating with the remote driving interface 1 and the remotely driven vehicle M. The communication unit 23 is configured to be capable of wireless communication with the remote driving interface 1. The communication unit 23 is connected to the remote driving interface 1 by global communication such as LTE [Long Term Evolution] communication.

The communication unit 23 is configured to be able to communicate with the remotely driven vehicle M, and is connected to the remotely driven vehicle M by local communication such as Wi-Fi. The communication unit 23 may be connected by wire when the portable remote driving kit 2 is attached to the remotely driven vehicle M.

Next, the functional configuration of the remote driving calculation unit 30 will be described. As shown in FIG. 3, the remote driving calculation unit 30 has an operation information acquisition unit 31, a vehicle information transmission unit 32, and a vehicle instruction unit 33.

The operation information acquisition unit 31 acquires operation information of the remote driver from the remote driving interface 1 via the communication unit 23. The operation information acquisition unit 31 acquires, for example, the steering amount, the accelerator pedal depression amount, and the brake pedal depression amount as operation information. The operation information acquisition unit 31 may acquire the operation of turning on the direction indicator as operation information. The operation information acquisition unit 31 may receive operation information as an instruction to go straight at 40 km/h, turn right, turn left, etc.

The vehicle information transmission unit 32 transmits vehicle surrounding information relating to the surrounding conditions of the remotely driven vehicle M to the remote driving interface 1 (remote driver R) via the communication unit 23. The vehicle information transmission unit 32 transmits image information of the surroundings of the remotely driven vehicle M acquired by the camera 21 and sound information of the surroundings of the remotely driven vehicle M acquired by the microphone 22 to the remote driving interface 1. The vehicle information transmission unit 32 may acquire vehicle speed information and fuel information (or remaining battery information) from the remotely driven vehicle M and transmit it to the remote driving interface 1 as vehicle status information. The vehicle information transmission unit 32 may also include acceleration information and steering angle information as vehicle status information.

The vehicle instruction unit 33 transmits remote driving instructions to the remotely driven vehicle M based on the operation information of the remote driver R. The vehicle instruction unit 33 may transmit the operation information of the remote driver R as a remote driving instruction as is to the remotely driven vehicle M. The vehicle instruction unit 33 may set a speed limit when recognizing a speed limit sign from image information around the remotely driven vehicle M. The vehicle instruction unit 33 may issue a remote driving instruction that limits the accelerator pedal depression amount so that the vehicle speed does not exceed the speed limit when the vehicle speed of the remotely driven vehicle M would exceed the speed limit if the accelerator pedal depression amount of the remote driver R is reflected as is.

According to the remote driving system of this embodiment described above, by consolidating functions in the portable remote driving kit 2, remote driving by the remote driver R can be realized by communicating with the portable remote driving kit 2 transported by the remote driving drone 3, even if the remote driving target vehicle M does not have to be equipped with a camera or microphone for remote driving or direct communication equipment with the remote driving interface. As a result, remote driving is possible by installing the portable remote driving kit 2 in the remote driving target vehicle M in a situation where remote driving is required, so there is no need to provide equipment for remote driving in the remote driving target vehicle M, which contributes to reducing vehicle costs.

In addition, with this remote driving system, the remote driver R can operate the remote driving drone 3 to land on and dock with the remote driving target vehicle M that he/she wishes to remotely drive, thereby achieving remote driving via the remote driving drone 3 equipped with the portable remote driving kit 2.

Such a remote driving system solves the issues of remote driving services. Specifically, in rental car delivery (out-of-service) services, the user drives in the driver's seat for most sections, so the operating time requiring remote driving is short. Also, in remote driving of mining machinery, one person can be in charge of a backhoe and a dump truck, but they are not driven at the same time. For this reason, if the portable remote driving kit 2 can be attached and detached only to the mobility required when needed (vehicle M to be remotely driven), it is possible to achieve both cost reduction and the addition of functions. Being able to remotely drive multiple different types of vehicle M to be remotely driven with one portable remote driving kit 2 expands the possibilities of remote driving services.

As an expansion of the possibilities of remote driving services, for example, application to logistics is considered. After an autonomous truck (remotely driven vehicle M) is autonomously driven to the exit of a highway, an attendant or a remote driving drone 3 can operate the autonomous truck at the exit of the highway and remotely drive the truck to a logistics warehouse. It is also possible to transfer the luggage to a small mobility (remotely driven vehicle M) at the logistics warehouse and re-attach the portable remote driving kit 2 to the small mobility. This makes it possible for the remote driver R to drive multiple different vehicles without moving. It is also possible to improve the efficiency of logistics by assigning the most suitable remote driver R at the appropriate time according to the driving qualifications of the remote driver R.

Remote driving can also be used in airport parking lots. When entering the parking lot, the customer would abandon the remotely driven vehicle M at the airport entrance, and a portable remote driving kit 2 would be attached to the remotely driven vehicle M by an attendant or a remote driving drone 3 at the airport entrance. The remote driver R would then remotely drive the remotely driven vehicle M to the parking lot, and the attendant in the parking lot would remove the portable remote driving kit 2 and drive the vehicle a short distance to a parking space.

When leaving the parking lot, the customer waits at the airport entrance, and the portable remote driving kit 2 is attached to the customer's remotely driven vehicle M by a parking lot attendant or a remote driving drone 3, and the remote driver R remotely drives the remotely driven vehicle M to the airport entrance, where the portable remote driving kit 2 is retrieved from the remotely driven vehicle M by the attendant or the remote driving drone 3. This reduces the movement of people and makes it possible to efficiently use parking lots at facilities such as airports.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-mentioned embodiments. The present invention can be implemented in various forms, including the above-mentioned embodiments, with various modifications and improvements based on the knowledge of those skilled in the art.

1... Remote driving interface, 2... Portable remote driving kit, 3... Remote driving drone, 21... Camera, 22... Microphone, 23... Communication unit, 30... Remote driving calculation unit, 31... Operation information acquisition unit, 32... Vehicle information transmission unit, 33... Vehicle instruction unit, M... Vehicle to be remotely driven, R... Remote driver.

Claims (1)

A remote driving system for a remote driver to remotely drive a remote driving target vehicle,
a remote driving interface for inputting operation information of the remote driver;
a portable remote driving kit having an operation information acquisition unit that acquires operation information of the remote driver from the remote driving interface by wireless communication, a camera for acquiring image information of the surroundings of the remotely driven target vehicle, a microphone for acquiring sound information of the surroundings of the vehicle, a vehicle information transmission unit that transmits the image information and the sound information to the remote driving interface, and a vehicle instruction unit that transmits remote driving instructions to the remotely driven target vehicle based on the operation information of the remote driver;
A remote-operated drone carrying the portable remote-operated kit;
A remote driving system equipped with:

Images