JP7822291B2 - Remote control device, operation method for remote control device, and program - Google Patents

Description

The present invention relates to a remote control device, a method for operating a remote control device, and a program.

Patent Document 1 discloses a technology that identifies the last parked location of a vehicle to be remotely controlled, determines whether an operation requested by a user is acceptable based on the last parked location, and rejects the operation if it is not acceptable.

Japanese Patent Application Laid-Open No. 2006-231964

However, Patent Document 1 does not anticipate multiple people sharing a vehicle. For example, if multiple people share a vehicle using the technology in Patent Document 1, a situation may arise where one user's remote control operation renders the vehicle unable to be remotely controlled, thereby reducing convenience for other users. In the context of sharing services, which have become increasingly popular in recent years, there is a desire to improve convenience for users who share a vehicle with multiple people.

In consideration of the above issues, the present invention aims to provide technology that can improve convenience for users who share a vehicle with multiple people. Furthermore, the present invention provides access to a safe, affordable, and easily accessible sustainable transportation system for all people by improving traffic safety through the expansion of public transportation, with particular consideration given to the needs of vulnerable people, women, children, people with disabilities, and the elderly. One aspect of the present invention contributes to the development of sustainable transportation systems.

A remote control device according to one aspect of the present invention is a remote control device for remotely controlling a vehicle, the remote control device comprising: a storage unit for registering biometric information of a user in association with identification information set in a terminal device of the user;
a communication means for acquiring biometric information of a first user present inside the vehicle;
a computing means for granting authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage means ,
The storage means has a function of registering a plurality of users as a group, and when the first user and the second user are registered as the same group,
When the communication unit acquires biometric information of a second user after the first user,
The computing means maintains the remote control authority granted to the identification information associated with the terminal device of the first user and the remote control authority granted to the identification information associated with the terminal device of the second user .

This invention can improve convenience for users who share a vehicle with multiple people. For example, being able to board a vehicle can be inferred to mean having the authority to operate the vehicle. Therefore, by granting remote operation authority to users authenticated by biometric authentication and allowing remote operation from outside the vehicle, user convenience can be improved.

FIG. 1 is a diagram illustrating an example of the configuration of a vehicle remote control system. FIG. 2 is a diagram showing the hardware configuration of a remote control device, a vehicle, and a terminal device. FIG. 2 is a diagram illustrating an example of the operation of the vehicle remote control system. FIG. 4 is a diagram illustrating an example of a screen displayed on a terminal device. FIG. 3 is a diagram illustrating authority management information stored in a storage device of a remote control device. 10 is a flowchart showing an example of the operation of the terminal device. A diagram that schematically explains the authorization of remote control (authorization). FIG. 7B is a diagram illustrating a processing flow of a modified example of the processing of S705 in FIG. 7A. 10 is a flowchart showing an example of the operation of the remote control device (registering fellow passengers as the same group). 10 is a flowchart showing an example of the operation of the remote control device (output of operation instructions). 10 is a flowchart showing an example of the operation of the remote control device (deletion of authority). 10 is a flowchart showing an example of vehicle operation (display of usage code). 10 is a flowchart showing an example of vehicle operation (transmission of identification information). 10 is a flowchart showing an example of vehicle operation (operation based on instructions).

The following embodiments are described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the claimed invention, and not all combinations of features described in the embodiments are necessarily essential to the invention. Two or more of the features described in the embodiments may be combined in any desired manner. Furthermore, the same reference numbers are used for identical or similar components, and duplicate descriptions will be omitted.

[System Overview]
An example configuration of a vehicle remote control system 150 including a remote control device 100 according to an embodiment will be described with reference to Fig. 1 . The remote control device 100 provides a user with a service for remotely controlling a vehicle 110 using a terminal device 120. Hereinafter, a service for remotely controlling a vehicle 110 using the terminal device 120 will be referred to as a remote control service. Remote control may be performed not only from a location remote from the vehicle 110, but also near the vehicle 110 or inside the vehicle 110.

In the remote control service, the remote control device 100 remotely controls the vehicle 110 in accordance with remote control instructions transmitted from the terminal device 120. The remote control device 100 may be capable of providing the remote control service to multiple users, each of whom has their own terminal device 120. While only two terminal devices 120 are shown in FIG. 1, the remote control device 100 may provide the remote control service to three or more users. The remote control device 100 may also be capable of remotely controlling multiple vehicles 110. Although only two vehicles 110 are shown in FIG. 1, the remote control device 100 may be capable of remotely controlling three or more vehicles 110.

The remote control device 100, the vehicle 110, and the terminal device 120 are each connected to a wide area network 130, such as the Internet. The remote control device 100 may be connected to the wide area network 130 via a local area network (LAN) (not shown). The vehicle 110 may be connected to the wide area network 130 via a cellular network (not shown). The terminal device 120 may be connected to the wide area network 130 via a cellular network (not shown) or via a LAN (not shown). The remote control device 100 and the vehicle 110 can communicate with each other via the wide area network 130. The remote control device 100 and the terminal device 120 can communicate with each other via the wide area network 130. The vehicle 110 and the terminal device 120 may also be able to communicate with each other via the wide area network 130. Alternatively or additionally, the vehicle 110 and the terminal device 120 may be able to communicate with each other directly via short-range communication such as Bluetooth (registered trademark).

Figure 2 is a diagram showing the hardware configuration of the remote control device 100, vehicle 110, and terminal device 120 in the vehicle remote control system 150 according to the embodiment.

(Remote control device 100)
The remote control device 100 (server) performs processes related to the registration of user information, the granting of authorization for remote control, the determination of the presence or absence of authorization for remote control, and remote control instructions received from the terminal device 120. The remote control device 100 includes a processor 51, a storage device 52, and a communication device 53.

The processor 51 (arithmetic unit) is a general-purpose integrated circuit for executing operations by the remote control device 100. The processor 51 may be configured, for example, by a central processing unit (CPU). The processor 51 performs various processes related to the vehicle remote control service by reading and executing programs stored in the storage device 52. When the processor 51 receives a request for account registration for using the remote control service from the user's terminal device 120 via the communication device 53, it registers user information in the storage device 52. In response to the registration request from the user's terminal device 120, the processor 51 transmits (downloads) a predetermined remote control program 37 to the terminal device 120 via the communication device 53.

The storage device 52 is a device for storing information and software used in the operation of the remote control device 100. The storage device 52 may be configured with a hard disk drive (HDD) or a solid state drive (SSD). The storage device 52 stores user information registration and various information, such as authority management information 57. The authority management information 57 is information for managing the authority to perform remote control of the vehicle 110. The authority to perform remote control of the vehicle 110 will be referred to as operation authority below. Specific examples of the authority management information 57 will be described later. The storage device 52 stores programs to be read and executed by the processor 51 and remote control programs to be provided to the user's terminal device 120. It can also store information obtained from the terminal device 120 and the vehicle 110 via the communication device 53.

The communication device 53 wirelessly transmits and receives various information. The communication device 53 is a device that enables the remote control device 100 to communicate with external devices (e.g., a router or base station). The communication device 53 may be configured, for example, as a network card. The communication device 53 may perform wireless communication or wired communication. The communication device 53 may support short-range wireless communication such as Bluetooth (registered trademark). Furthermore, the communication device 53 may support cellular communication, Wi-Fi (registered trademark) communication, etc. The communication device 53 mediates wireless communication between the control device 15 of the vehicle 110 and the communication device 36 of the terminal device 120.

(Configuration of vehicle 110)
The vehicle 110 has a propulsion device 4, a braking device 5, a steering device 6, an external sensor 7, a vehicle sensor 8, a communication device 9, a navigation device 10, a driving operation device 11, a biometric information acquisition device 12, an interface device (HMI device) 13, and a control device 15.

Each component of vehicle 110 is connected to enable signal transmission via an in-vehicle communication network such as a Controller Area Network (CAN).

The propulsion device 4 is a device that provides driving force to the vehicle and includes, for example, a power source and a transmission. The power source has at least one of an internal combustion engine such as a gasoline engine and an electric motor.

The brake device 5 is a device that applies braking force to the vehicle and includes, for example, a brake caliper that presses pads against a brake rotor and an electric cylinder that supplies hydraulic pressure to the brake caliper.

The steering device 6 is a device for changing the steering angle of the wheels, and includes, for example, a rack-and-pinion mechanism for steering the wheels and an electric motor for driving the rack-and-pinion mechanism. The steering device 6 includes an electric power steering device. The electric power steering device assists steering operations or includes a motor that generates driving force for automatically steering the front wheels, a sensor that detects the steering angle, and the like. The propulsion device 4, braking device 5, and steering device 6 are controlled by the control device 15.

The external sensor 7 is a sensor that detects objects around the vehicle. The external sensor 7 includes a radar 16, a lidar (Light Detection and Ranging) 17, and a camera 18, and outputs the detection results to the control device 15.

Radar 16 is, for example, a millimeter-wave radar, and is capable of detecting objects around vehicle 110 using radio waves and measuring the distance to the objects. Multiple radars 16 are provided around vehicle 110; for example, one radar 16 is provided in the center of the front of the vehicle, one at each front corner, and one at each rear corner.

The LIDAR 17 uses light to detect objects around the vehicle 110 and measure the distance to the objects. Multiple LIDARs 17 are provided around the vehicle; for example, one LIDAR 17 is provided at each corner of the front of the vehicle, one in the center of the rear, and one on each side of the rear.

Camera 18 is a device that captures images of the surroundings of vehicle 110, and is, for example, a digital camera that uses a solid-state image sensor such as a CCD or CMOS. Camera 18 includes a front camera that captures images in front of vehicle 110 and a rear camera that captures images behind it.

The vehicle sensors 8 include a vehicle speed sensor that detects the speed of the vehicle 110, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity around a vertical axis, and a direction sensor that detects the orientation of the vehicle 110. The yaw rate sensor is, for example, a gyro sensor. The vehicle sensors 8 may also include sensors that detect the remaining fuel or battery charge for driving the propulsion device 4, and sensors that detect the operating status of the brake device 5, etc. The vehicle sensors 8 may also include sensors that detect the lock status (locked/unlocked) of the doors and sensors that detect the open/closed status of the windows. The detection results of the vehicle sensors 8 are transmitted to the terminal device 120 and the remote control device 100 via the communication device 9. In the terminal device 120, the detection results of the vehicle sensors 8 may be displayed on the UI screen of the remote control program 37 in the input/output device 30. In the remote control device 100, the processor 51 may use the detection results of the vehicle sensors 8 to perform various processes, such as authorization.

The communication device 9 provides the vehicle 110 with the ability to communicate with the outside world. The communication device 9 may support short-range wireless communication such as Bluetooth (registered trademark). Furthermore, the communication device 9 may support cellular communication, Wi-Fi (registered trademark) communication, etc. The communication device 9 mediates wireless communication between the control device 15 of the vehicle 110 and the terminal device 120 and remote control device 100. In other words, the control device 15 can communicate with the terminal device 120 and remote control device 100 carried by the user via the communication device 9, using communication methods such as infrared communication and Bluetooth (registered trademark).

The navigation device 10 acquires the vehicle's current position and provides route guidance to the destination, and includes a GNSS receiver 20 and a map memory 21. The GNSS receiver 20 determines the vehicle's position (latitude and longitude) based on signals received from artificial satellites (positioning satellites). The map memory 21 is composed of a storage device such as flash memory or a hard disk, and stores map information.

The driving operation device 11 is installed inside the vehicle cabin and accepts input operations performed by the user to control the vehicle. The driving operation device 11 includes, as driving operation units, for example, a steering wheel, accelerator pedal, brake pedal, parking brake device, shift lever, and push start switch (engine start button). The push start switch accepts input operations to start the vehicle through driving operations by the user. The driving operation device 11 includes a sensor that detects the amount of operation and outputs a signal indicating the amount of operation to the control device 15.

The biometric information acquisition device 12 acquires biometric information of a user present inside the vehicle 110. The biometric information acquisition device 12 is, for example, a camera (indoor camera) installed inside the cabin of the vehicle 110, and acquires images including the face of a user present inside the vehicle 110 (in-vehicle user). The biometric information acquisition device 12 is capable of acquiring images including the faces of users seated in the front seats (driver's seat and passenger seat) of the vehicle 110, and users seated in the rear seats located behind the front seats. The biometric information acquisition device 12 outputs the acquired biometric information to the control device 15. Note that the biometric information acquisition device 12 may also be any type of detection device that detects one or more of the user's fingerprint, retina, and iris as biometric information.

The interface device 13 (HMI device) provides an interface (HMI: Human Machine Interface) between the control device 15 and the user, notifying the user of various information by displaying and outputting audio, and accepting input operations from the user. The interface device 13 has a display device 23 made of liquid crystal, organic EL, etc., that functions as a touch panel that can accept input operations from the user, and an input device 24 such as buttons and switches.

The control device 15 is an electronic control unit (ECU) that includes a CPU, non-volatile memory (ROM), and volatile memory (RAM). The control device 15 is able to perform various vehicle controls by executing program-based arithmetic processing using the CPU. At least some of the functional units of the control device 15 may be implemented using hardware such as an LSI, ASIC, or FPGA, or may be implemented using a combination of software and hardware.

The control device 15 has an external environment recognition unit 41, a position identification unit 42, a user recognition unit 43, a vehicle control unit 44, and a storage device 45.

The external environment recognition unit 41 acquires information about other vehicles and obstacles present around the vehicle 110 based on the detection results of the external environment sensor 7. For example, the external environment recognition unit 41 acquires information about the position and size of other vehicles and obstacles, as well as information about traffic light displays, lane markings on roads, and markings (such as white lines) painted on the road surface in parking lots, etc. The external environment recognition unit 41 also analyzes images acquired by the camera 18 using image analysis techniques such as pattern matching to acquire the presence and size of other vehicles and obstacles. Furthermore, the external environment recognition unit 41 can calculate the distance to obstacles using signals from the radar 16 and lidar 17 and acquire the position of the obstacles.

The position determination unit 42 is capable of detecting the vehicle's position based on signals from the GNSS receiving unit 20 of the navigation device 10. In addition to signals from the GNSS receiving unit 20, the position determination unit 42 can also obtain vehicle speed and yaw rate from the vehicle sensor 8 and determine the vehicle's position and attitude using so-called inertial navigation.

The user recognition unit 43 analyzes the biometric information (e.g., facial images) acquired by the biometric information acquisition device 12 based on image analysis techniques such as feature extraction processing, and acquires facial information by extracting the user's facial features from the facial images acquired through the analysis. When the biometric information acquisition device 12 acquires facial images including the faces of multiple users present inside the vehicle 110, the user recognition unit 43 acquires multiple pieces of facial information by extracting the facial features of each of the multiple users from the facial images as biometric information.

The processor 51 of the remote control device 100 references the storage device 52 to determine whether the identification information associated with the terminal device 120 has been granted remote control authority. If the identification information has been granted remote control authority, the processor 51 transmits a remote control signal from the terminal device 120 to the vehicle 110.

The vehicle control unit 44 controls the vehicle 110 in accordance with remote operation instructions. For example, it controls the locking and unlocking of the vehicle 110 doors, the activation of the power system that supplies power to various parts of the vehicle 110, and the activation of the propulsion unit 4. For example, upon receiving a remote operation instruction to unlock the doors, the vehicle control unit 44 unlocks the doors. Furthermore, upon receiving a remote operation instruction to activate the propulsion unit 4, the vehicle control unit 44 activates the propulsion unit 4. If the propulsion unit 4 is an internal combustion engine, the vehicle control unit 44 turns on the ignition. Furthermore, the vehicle control unit 44 controls predetermined operations of the vehicle 110 based on the remote operation instructions from the driving operation device 11.

The storage device 45 is composed of flash memory, a hard disk, etc., and stores various programs executed by the control device 15 and various parameters required for program execution. The storage device 45 also registers various information obtained from the remote control device 100 via the communication device 9. This information includes user information registered in the authority management information 57 obtained from the remote control device 100.

(Terminal device 120)
The terminal device 120 is a device that can be carried by a user, and is capable of communicating with the remote control device 100 and the control device 15 of the vehicle 110. The remote control program 37 provides various functions for remotely controlling the vehicle. The terminal device 120 may be a mobile device such as a smartphone. As shown in FIG. 2 , the terminal device 120 has an input/output device 30, an imaging device 31, a processor 33, a storage device 35, a communication device 36, and a remote control program 37. The remote control program 37 is a program obtained from the remote control device 100. When the remote control program 37 is executed in the terminal device 120,
The input/output device 30 presents information to a user operating the terminal device 120 and accepts input from the user operating the terminal device 120. The input/output device 30 functions as, for example, a touch panel, and upon accepting input from the user, outputs a signal corresponding to the input to the processor 33. The input/output device 30 functions as a UI (user interface) for various functions related to vehicle remote control provided by the remote control program 37. Operation input to the remote control program 37 and display of information processed by the remote control program 37 can be performed via the input/output device 30.

The imaging device 31 is capable of capturing images (still images and moving images) in an imaging mode set by the input/output device 30. The imaging device 31 is, for example, a digital camera configured with a CMOS or the like. The processor 33 is able to acquire facial image features as biometric information of the user by performing predetermined image processing on an image captured of the user operating the terminal device 120. When the user registers an account to the remote control device 100 to use the remote control service, the terminal device 120 may transmit biometric information along with a registration request to the remote control device 100, and the remote control device 100 may associate the identification information (user ID or terminal ID) with the biometric information and register it in the storage device 52.

The processor 33 controls various processes in the terminal device 120. By executing programs stored in the storage device 35, the processor 33 realizes various functions related to vehicle remote control using the remote control program 37. The processor 33 controls the display screen of the terminal device 120, and for example, controls the display of a UI corresponding to the various functions related to vehicle remote control provided by the remote control program 37.

The storage device 35 is a device for storing information and software used in the operation of the terminal device 120. The storage device 52 may be configured as a hard disk drive (HDD) or a solid state drive (SSD). The storage device 52 stores, for example, the remote control program 37.

The communication device 36 provides a function that enables the terminal device 120 to communicate with the outside world. The communication device 36 may support short-range wireless communication such as Bluetooth (registered trademark). Furthermore, the communication device 36 may support cellular communication, Wi-Fi (registered trademark) communication, etc.

(Example of system operation)
An example of the operation of the vehicle remote control system 150 including the remote control device 100 according to the embodiment will be described with reference to Fig. 3. Before executing this operation, it is assumed that the user has registered an account for using the remote control service with the remote control device 100 and installed the remote control program 37 in the user's terminal device 120. It is also assumed that the user has registered the user ID of the account for using the remote control service in the remote control program 37.

At S300, the user pairs the communication device 9 of the vehicle 110 with the communication device 36 of the terminal device 120. This enables the communication device 9 of the vehicle 110 and the communication device 36 of the terminal device 120 to communicate with each other. The terminal device 120 may acquire the vehicle ID of the vehicle 110 during pairing, and may transmit this vehicle ID to the remote control device 100 along with its own terminal ID or a user ID registered in the remote control program 37. The remote control device 100 may store this vehicle ID as the vehicle ID of the vehicle to be remotely controlled by the terminal device 120 (in column 503 of the authority management information 57, described below).

At S301, the biometric information acquisition device 12 acquires biometric information of a user present inside the vehicle 110. The biometric information acquisition device 12 is, for example, a camera (interior camera) installed inside the passenger compartment of the vehicle 110, and acquires an image including the face of a user (e.g., a first user) present inside the vehicle 110.

The communication device 9 of the vehicle 110 transmits the biometric information of the user (first user) acquired by the biometric information acquisition device 12 to the remote control device 100.

In S306, the communication device 53 of the remote control device 100 communicates with the vehicle 110 to acquire biometric information of the first user present inside the vehicle 110. The remote control device 100 determines whether the biometric information of the first user matches the biometric information registered in the authority management information 57. If the biometric information of the first user matches the biometric information registered in the authority management information 57, the remote control device 100 grants remote control authority, on the prerequisite that identification information (user ID, terminal ID) associated with the first user's terminal device 120 is registered in the authority management information 57. In this case, the user (first user) can omit the operations of S302 and S305 using the terminal device 120.

If the first user's biometric information does not match the biometric information registered in the authority management information 57, or if the biometric information is not registered, the user obtains operation authority to perform remote operation through processing using the usage code (S302-S306). When the usage code presented by the interface device 13 inside the vehicle 110 matches the usage code transmitted from the user's terminal device 120, the remote operation device 100 grants remote operation authority to the identification information associated with the first user's terminal device, which is registered in the storage device 52.

In steps S302 to S306, the user acquires operating authority to perform remote control. Specifically, in step S302, the user uses the remote control program 37 to request the remote control device 100 to issue a usage code. The usage code is information that must be sent to the remote control device 100 to acquire operating authority to perform remote control.

In S303, the remote control device 100 generates a usage code in response to the user's request and transmits it to the vehicle 110. The remote control device 100 may, for example, randomly generate a four-digit number as the usage code. The remote control device 100 associates the generated usage code with identification information associated with the terminal device 120 that requested the issuance of the usage code and stores them in the storage device 52 for use in a later step.

In S304, the vehicle 110 presents the usage code using the display device 23 in accordance with the user's operation using the input device 24. The presentation of the usage code may be by displaying the usage code on the display device 23, by playing a sound representing the usage code from the speaker of the interface device 13, or by a combination of these. Presenting the usage code requires operation of the input device 24. Therefore, the user must unlock the vehicle 110 using the vehicle 110 key to obtain the usage code.

At S305, the user uses the remote control program 37 to send the usage code displayed by the display device 23 to the remote control device 100. At S306, the remote control device 100 grants the terminal device 120 remote control authority to perform remote control, on the condition that the usage code sent at S305 matches the usage code created at S303, and that the usage code was received at S305 within a predetermined time (e.g., within 5 minutes) after the usage code was generated at S303. A validity period may be set for the operation authority.

In addition, in S306, it is also possible to grant remote control authority using the user's biometric information transmitted in S301. For example, the storage device 52 registers the user's biometric information in association with identification information set in the user's terminal device 120. The communication device 53 may acquire biometric information of a first user present inside the vehicle 110 from the vehicle 110, and the remote control device 100 may grant remote control authority for the vehicle 110 to identification information associated with the first user's biometric information among the biometric information registered in the storage device 52.

The remote control device 100 may grant operation authority to individual terminal devices 120, or may grant operation authority to terminal devices 120 on a user-by-user basis. Assume that a user has multiple terminal devices 120. In this case, the remote control device 100 may grant operation authority to the terminal device 120 used to send the usage code, and not to the other terminal devices 120. The remote control device 100 may grant operation authority to not only the terminal device 120 used to send the usage code, but also to the other terminal devices 120, among the user's multiple terminal devices 120. In this case, the remote control device 100 may be considered to have granted operation authority to the user. In the following description, a terminal device 120 that has been granted operation authority may be referred to as a terminal device 120 with operation authority. Furthermore, the user of a terminal device 120 with granted operation authority may be referred to as a user with operation authority. The following description will be given of a case where operation authority is granted to an individual terminal device 120, but instead, operation authority may be granted to the user of an individual terminal device 120. As with granting operation authority, the remote operation device 100 may revoke operation authority for an individual terminal device 120, or may revoke operation authority for the terminal device 120 on a user-by-user basis.

At S307, the user uses the remote control program 37 of the terminal device 120 for which the user has operating authority to send a remote control instruction for the vehicle 110 to the remote control device 100. Specific examples of remote control instructions will be described later. At S308, the remote control device 100 confirms that the terminal device 120 that sent the remote control instruction has operating authority.

At S309, the remote control device 100 transmits a remote control instruction to the vehicle 110 in order to remotely operate the vehicle 110 in accordance with the remote control instruction from the user. The remote control device 100 outputs a signal to operate the vehicle 110 based on a remote control request received from a terminal device 120 with identification information that has been granted remote control authority.

At S310, the vehicle 110 executes the operation instructed by the remote control device 100. By repeating the operations of S307 to S310, the user can repeatedly instruct remote operation as long as the operation authority is valid.

In S311, the remote control device 100 cancels the operation authority in accordance with a remote operation instruction from the user. In S311, the remote control device 100 may cancel the operation authority granted to the terminal device 120 when the validity period of the operation authority has expired. By periodically canceling the operation authority in this manner, unauthorized use of the remote control service can be prevented. The remote control device 100 may update the validity period of the operation authority if S302 to S306 are executed within the validity period.

(Screen display of terminal device 120)
4, an example of a screen displayed on the input/output device 30 of the terminal device 120 by the remote operation program 37 will be described. In addition to the screen 400 and the screen 410, the input/output device 30 may display a menu screen for transitioning to these screens, for example.

ST41 in Figure 4 shows an example of a screen 400 used to issue and transmit a usage code. Screen 400 includes an issue request button 401 for requesting the issuance of a usage code in S302 described above. When the user selects (e.g., presses) the issue request button 401 using the input/output device 30, a request to issue a usage code is transmitted to the remote control device 100. The address of the remote control device 100 may be set in advance in the remote control program 37.

Screen 400 further includes a field 402 for inputting a usage code, and a usage code send button 403 for transmitting the usage code input in field 402 to remote control device 100. When the user inputs the usage code into field 402 using input/output device 30 and selects (e.g., presses) usage code send button 403, the usage code input in field 402 is transmitted to remote control device 100. In the example of ST41 in FIG. 4, the issue request button 401, field 402, and usage code send button 403 are included on the same screen 400, but these objects may be distributed across multiple screens.

ST42 in Figure 4 shows an example of a screen 410 used to remotely control the vehicle 110. The screen 410 includes a button 411 for unlocking the doors of the vehicle 110. The door of the vehicle 110 is unlocked when the user selects (e.g., presses) the button 411 using the input/output device 30.

The screen 410 further includes a button 412 for activating the power supply or propulsion device 4 of the vehicle 110. When the user selects (e.g., presses) the button 412 using the input/output device 30, the power supply or propulsion device 4 of the vehicle 110 is turned on (e.g., power on state, ignition on state). This allows the user to start the vehicle 110. To further improve security, the remote control program 37 may require the user to enter an activation code presented by the display device 23 of the vehicle 110 after the button 412 is pressed.

The screen 410 further includes a button 413 for starting the air conditioner of the vehicle 110. The air conditioner of the vehicle 110 is started when the user selects (e.g., presses) the button 413 using the input/output device 30. The remote control program 37 may provide remote control for changing the settings (e.g., temperature) of the air conditioner of the vehicle 110. In addition, it may provide remote control for activating the defroster.

The screen 410 further includes a button 414 for changing the state of the vehicle 110 to make it easier to find. When the user selects (e.g., presses) the button 414 using the input/output device 30, the vehicle 110's lights, for example, flash for a predetermined period of time.

As described above, if the remote operation program 37 includes an unlock operation and a power-on operation as remote operations, the user can start the vehicle 110 using the terminal device 120 instead of a key for the vehicle 110. Therefore, the terminal device 120 can function as a digital key. The remote operation program 37 may include operations other than those described in ST42 of FIG. 4 as remote operations, or may not include some or all of the operations described in ST42 of FIG. 4. In the example of ST42 of FIG. 4, buttons 411 to 414 are included on the same screen 410, but these objects may also be distributed across multiple screens.

(Authority Management Information)
5 is a diagram illustrating an example of authority management information 57 stored in the storage device 52 of the remote operation device 100. The storage device 52 registers user identification information, information on the group to which the user belongs, biometric information, etc. in the authority management information 57. In FIG. 5, the operation authority of each user is managed in a table format. Alternatively, the operation authority of each user may be managed in another format. In the authority management information 57, a record is generated for each user who has registered an account for using the remote operation service in the remote operation device 100.

Column 500 (User ID) represents identification information (i.e., a user ID) for uniquely identifying a user. The identification information (user ID) may be the account name of an account that the user uses to access the remote control service. The account name may be a character string specified by the user (e.g., an email address), or a unique character string generated by the remote control device 100. When a user registers an account, the remote control device 100 adds a record to the authority management information 57 and registers the user ID in column 500 of that record.

Column 501 (Terminal ID) represents identification information (i.e., terminal ID) for uniquely identifying the terminal device 120 used by the user. The terminal ID may be, for example, the address of the communication device 36 (or a BD address in the case of Bluetooth). The terminal ID may also be the serial number of the terminal device 120. The terminal ID may also be a character string uniquely assigned to the terminal device 120 by the remote operation device 100. When the user inputs an account name, the remote operation program 37 of the terminal device 120 may transmit the user ID of the terminal device 120 to the remote operation device 100. The remote operation device 100 registers the received terminal ID in column 501. When the remote operation device 100 assigns a terminal ID to the terminal device 120, the remote operation device 100 registers the terminal ID in column 501 and transmits the terminal ID to the terminal device 120. The terminal device 120 stores the received terminal ID for use in subsequent processing.

A single user may be able to register only one terminal device 120 to be used for remote control, or may be able to register multiple terminal devices 120. If a single user registers multiple terminal devices 120, multiple terminal IDs are registered in column 501.

Column 502 (Vehicle ID) represents identification information (i.e., the vehicle ID) for uniquely identifying the vehicle that the user remotely controls. The vehicle ID may be, for example, an IP address assigned to the communication device 9 of the vehicle 110. The vehicle ID may be stored in the vehicle 110. The terminal device 120 may obtain the vehicle ID from the vehicle 110 when performing pairing in S300 of FIG. 3, and may transmit this vehicle ID to the remote control device 100 along with a request to issue a usage code in S302. The remote control device 100 registers the received vehicle ID in column 502.

A single user may be able to register only one vehicle 110 to be remotely controlled, or may be able to register multiple vehicles 110. If a single user registers multiple vehicles 110, multiple vehicle IDs are registered in column 502. If multiple vehicle IDs are registered, the remote control device 100 may obtain the vehicle ID of the vehicle to be remotely controlled along with the remote control instruction.

Column 503 (operation authority) indicates whether the user has operation authority to perform remote operation. When remote operation authority is granted, "Operation authority granted" is set in column 503. Operation authority may include multiple operation authorities with different scopes of authority. In addition, the scope of operation authority may change depending on whether the vehicle 110 is in a moving state.

Column 504 (Group ID) represents identification information (i.e., group ID) for identifying the group to which each user belongs. When a user registers an account to use the remote control service, the remote control device 100 can also add a record to the authority management information 57 and register the group ID to which the user belongs.

Column 505 represents the user's biometric information. The biometric information is image information including the user's face. For example, when registering identification information (user ID or terminal ID), image information including the user's face may be registered as biometric information.

Image information including the user's face may be image information captured by the imaging device 31 of the terminal device 120. Also, image information including the user's face acquired by the biometric information acquisition device 12 inside the vehicle 110 may be registered as biometric information.

(Example of operation of terminal device 120)
Fig. 6 is a flowchart showing an example of the operation of the terminal device 120. An example of the operation performed by the terminal device 120 will be described with reference to Fig. 6. Each step in Fig. 6 may be performed by the processor 33 executing a computer program stored in the storage device 35. Alternatively, some or all of the multiple steps in Fig. 6 may be performed by a dedicated integrated circuit such as an ASIC. In such a case, the dedicated integrated circuit may operate as a functional unit for executing each step.

In S601, the terminal device 120 determines whether the user has started using the remote control program 37. If it is determined that the user has started using the remote control program 37 ("YES" in S601), the terminal device 120 transitions the process to S602, and otherwise ("NO" in S601) repeats S601. The terminal device 120 may determine that the user has started using the remote control program 37 when the remote control program 37 is started or when the remote control program 37 is changed from background to active.

At S602, the terminal device 120 sends an inquiry to the remote control device 100 as to whether the terminal device 120 has operation authority. At S603, the terminal device 120 determines whether the terminal device 120 has operation authority based on the result of the inquiry. If the terminal device 120 determines that the terminal device 120 has operation authority ("YES" at S603), the process proceeds to S604; otherwise ("NO" at S603), the process proceeds to S607.

For example, if the biometric information of the user in the vehicle 110 matches the biometric information registered in the authority management information 57 and identification information (user ID or terminal ID) associated with the user's terminal device 120 is registered, processing proceeds to S604 (YES in S603). On the other hand, if the user's biometric information does not match the biometric information registered in the authority management information 57 (NO in S603), processing proceeds to S607, and the user obtains operating authority to perform remote operation through processing using the usage code (S607 to S611).

If the terminal device 120 has the operation authority, the terminal device 120 performs remote operation of the vehicle 110 through steps S604 to S606. In step S604, the terminal device 120 uses the output device 404 to display a screen for remote operation (e.g., screen 410 of step ST42 in Figure 4).

In S605, the terminal device 120 determines whether it has received a remote operation instruction from the user using the input/output device 30. If it is determined that a remote operation instruction has been received from the user ("YES" in S605), the terminal device 120 transitions the process to S605; otherwise ("NO" in S605), the terminal device 120 repeats S605. The remote operation instruction may be, for example, pressing any of buttons 411 to 414 in ST42 of FIG. 4. In S606, the terminal device 120 transmits the remote operation instruction received from the user to the remote operation device 100.

If the terminal device 120 does not have operation authority, the terminal device 120 acquires operation authority for the vehicle 110 through steps S607 to S611. In step S607, the terminal device 120 uses the input/output device 30 to display a screen (e.g., screen 400 in step ST41 in Figure 4) for requesting the issuance of a usage code.

In S608, the terminal device 120 determines whether an issuance instruction has been received from the user using the input/output device 30. An issuance instruction is an instruction to request the issuance of a usage code. If it is determined that an issuance instruction has been received from the user ("YES" in S608), the terminal device 120 transitions the process to S609; otherwise ("NO" in S608), the terminal device 120 repeats S608. The issuance instruction may be, for example, pressing the issuance request button 401 in ST41 of FIG. 4. In S609, the terminal device 120 transmits a request to issue a usage code to the remote control device 100.

In S610, the terminal device 120 determines whether it has received a code transmission instruction from the user using the input/output device 30. A code transmission instruction is an instruction to transmit a usage code. If it is determined that a code transmission instruction has been received from the user ("YES" in S610), the terminal device 120 transitions the process to S611; otherwise ("NO" in S610), the terminal device 120 repeats S610. The usage code instruction may be, for example, pressing the issuance request button 401 in ST41 of FIG. 4. In S611, the terminal device 120 transmits the usage code entered in field 402 to the remote control device 100.

The data transmitted from the terminal device 120 to the remote control device 100 may include identification information associated with the terminal device 120. The identification information associated with the terminal device 120 may be a user ID or a terminal ID. The user ID may be the same type as the user ID described in column 500 of Figure 5. The terminal ID may be the same type as the terminal ID described in column 501 of Figure 5.

(Example of operation of the remote control device 100)
7A to 10 are flowcharts showing an example of the operation of the remote control device 100. Each step in FIGS. 7A to 10 may be performed by the processor 51 executing a computer program stored in the storage device 52. Alternatively, some or all of the steps in FIGS. 7A to 10 may be performed by a dedicated integrated circuit such as an ASIC. In such a case, the dedicated integrated circuit may operate as a functional unit for executing each step. Note that the example of the operation of the remote control device 100 described in FIGS. 7A to 10 may be performed by the control device 15 of the vehicle 110.

(Authorization of remote control)
Fig. 7A is a flowchart showing an example of the operation of the remote control device 100. An example of the operation of the remote control device 100 regarding the authorization of remote operation will be described with reference to Fig. 7A.

In S700, the remote control device 100 determines whether the biometric information of the user (first user) transmitted from the vehicle 110 in S301 matches the biometric information registered in the authority management information 57. If the biometric information of the user (first user) matches the biometric information registered in the authority management information 57 ("YES" in S700), the remote control device 100 proceeds to S702. If the biometric information does not match ("NO" in S700), the remote control device 100 proceeds to S701.

In S701, the remote control device 100 determines whether the usage code created in S303 matches the usage code transmitted in S305. If they match ("YES" in S701), the process proceeds to S702; otherwise ("NO" in S701), the process related to authorization is terminated.

In S702, the remote control device 100 determines whether the identification information (user ID or terminal ID) included in the usage code transmitted in S305 is registered in the authority management information 57. If it is registered ("YES" in S702), the process proceeds to S703; otherwise ("NO" in S702), the process related to authorization is terminated. Note that the identification information (user ID or terminal ID) may be obtained inside the vehicle 110 by the communication device 9 acquiring the identification information from the user's terminal device 120 inside the vehicle and transmitting it to the remote control device 100.

In S703, the remote operation device 100 references the authority management information 57 to determine whether a group is registered for the identification information (user ID or terminal ID) to which authority is to be granted. Specifically, the remote operation device 100 references column 504 of the authority management information 57 to determine whether a group is registered.

If group registration has been performed ("YES" in S703), the remote control device 100 transitions the process to S704, where the remote control device 100 grants remote control authority to the identification information (user ID or terminal ID) registered in the group. For example, in ST51 of FIG. 5, the remote control device 100 grants remote control authority to the identification information of user A (user ID = UA) and user B (user ID = UA), who are registered in the same group 1 (ST51 of FIG. 5).

When the remote control device 100 grants remote control authorization to the identification information associated with the user's (first user's) terminal device through a comparison of the usage codes, the storage device 52 associates and registers the user's (first user's) biometric information acquired by the communication device 53 with the identification information associated with the user's (first user's) terminal device.

The target of authorization may be, for example, the current operator (user) or the last operator (user) at the time of starting up the vehicle 110, and remote operation authorization may be granted to the identification information of a user belonging to the same group as the current operator or last operator. For example, when the propulsion device 4 of the vehicle 110 starts or stops, the communication device 9 of the vehicle 110 may acquire identification information associated with the terminal device 120 of the first user inside the vehicle and transmit it to the communication device 53. The communication device 53 may acquire, via the communication device 9 of the vehicle 110, identification information associated with the terminal device 120 of the first user present inside the vehicle 110 when the vehicle 110 starts or stops. The processor 51 of the remote operation device 100 may register the identification information acquired by the communication device 53 in the storage device 52 and grant remote operation authorization to the identification information registered in the storage device 52.

The storage device 52 has the function of registering multiple users as a group, and can register, for example, a first user (e.g., user ID = UA in ST51 in Figure 5) and a second user (e.g., user ID = UB in ST51 in Figure 5) as Group 1. If the communication device 53 acquires the biometric information of the second user after the first user, the remote control device 100 maintains the remote control authority granted to the identification information associated with the terminal device of the first user and the remote control authority granted to the identification information associated with the terminal device of the second user.

On the other hand, if the determination in S703 is that group registration has not been performed ("NO" in S703), the process proceeds to S705, where the remote control device 100 grants remote control authority to the identification information determined to be registered in S702, and ends the process. In this case, the remote control device 100 deletes the remote control authority of the first user and grants remote control authority to the identification information associated with the terminal device 120 of the new user, the third user (for example, user ID = UC in ST52 of Figure 5).

When a new user who has never driven a particular vehicle 110 before drives the vehicle 110, such as when the vehicle 110 is resold, the previous remote control authority may be deleted. For example, if the communication device 53 acquires the biometric information of a third user after the first user, the remote control device 100 deletes the remote control authority granted before the new user, the third user. The remote control device 100 deletes the remote control authority granted to the identification information associated with the terminal device of the first user. If the first user is registered in a group, the information of the group ID to which the first user belongs may be deleted. Furthermore, the biometric information of the first user may be deleted along with the deletion of the remote control authority. The remote control device 100 then grants remote control authority to the identification information associated with the terminal device of the new user, the third user (e.g., user ID = UC in Figure 3) (ST52 in Figure 5).

(Modification)
Fig. 7B is a diagram illustrating a process flow of a modified example of the process of S705 in Fig. 7A . In the modified example of Fig. 7B , the remote control device 100 may switch between maintaining the previously granted authority and granting the authority to a new user depending on whether the location where the biometric information was acquired is a parking area where the vehicle 110 is normally parked (a normal parking area).

The position determination unit 42 of the vehicle 110 determines the position of the vehicle 110 based on signals from the GNSS receiving unit 20 of the navigation device 10 and signals from the vehicle sensor 8. If the vehicle remains stopped continuously for a predetermined period of time, the position determination unit 42 determines that the vehicle is parked and determines the position of the vehicle 110 as a parking position. The parking position determined by the position determination unit 42 is transmitted to the remote control device 100 by the communication device 9. Date and time information specifying the date and time of parking may be transmitted along with the parking position. Furthermore, when biometric information is acquired by the biometric information acquisition device 12, the communication device 9 may associate the biometric information with the parking position determined by the position determination unit 42 and the date and time information and transmit them to the remote control device 100.

The communication device 53 acquires biometric information, the parking location of the vehicle 110, and date and time information specifying the date and time of parking from the vehicle 110.

The remote control device 100 obtains the parking frequency for each parking position of the vehicle 110 over a specified period based on information about the parking position of the vehicle 110 and the parking date and time, and identifies the parking position with the highest parking frequency as the parking area where the vehicle 110 is normally parked (normal parking area).

Here, parking frequency refers to the number of times a vehicle parks at each parking location over a given period (e.g., one month), such as four times at parking location A, 20 times at parking location B, and six times at parking location C. Note that parking frequency may also be the cumulative (total) amount of time parked at each parking location.

In S711, the remote control device 100 determines whether the location where the biometric information was acquired is within the normal parking area. If the location where the biometric information was acquired is not within the normal parking area ("NO" in S711), the process proceeds to S713, where the remote control device 100 deletes the remote control authority granted to the identification information associated with the biometric information of the first user. Then, the remote control device 100 grants remote control authority to the identification information associated with the biometric information acquired at a location outside the normal parking area.

For example, the remote control device 100 deletes the remote control authority of the first user who was previously granted remote control authority, and grants remote control authority to the identification information associated with the terminal device 120 of the new user, the third user (for example, user ID = UC in ST52 of Figure 5).

On the other hand, if the determination in S711 indicates that the location where the biometric information was acquired is within the normal parking area ("YES" in S711), the process proceeds to S712, where the remote control device 100 maintains the remote control authority granted to the identification information associated with the first user's biometric information. For example, the remote control device 100 grants (maintains) remote control authority to the identification information associated with the terminal device 120 of the first user (user ID = UA in ST51 of Figure 5), who was the user to whom remote control authority was first granted.

When the location where the biometric information was acquired is within a normal parking area, it may be assumed that the vehicle 110 is used when shared by a family. In such a case, when new biometric information is acquired, the previously granted remote control authority is not deleted, and the authority of the user who was previously granted remote control authority is maintained. This improves convenience when the vehicle 110 is shared by multiple users. Furthermore, by maintaining the authority of the user who was granted remote control authority, it becomes possible to remotely restrict the operation of the vehicle 110 even if the vehicle 110 is stolen, for example, thereby improving the security of the vehicle 110.

(Register passengers as the same group)
Fig. 8 is a flowchart showing an example of the operation of the remote control device 100. An example of the operation of the process executed by the remote control device 100 to register fellow passengers of the vehicle 110 as part of the same group will be described with reference to Fig. 8 .

At S800, the communication device 53 receives biometric information from the vehicle 110. The communication device 53 acquires multiple facial images including the faces of multiple users present inside the vehicle 110.

In S801, the remote control device 100 determines whether the facial information of each of multiple users is registered as biometric information in the storage device 52. If the facial information is not registered as the user's biometric information in the storage device 52 ("NO" in S801), the processing ends. In this case, the user obtains operating authority to perform remote control by processing using the usage code (S302 to S306).

On the other hand, if it is determined in S801 that facial information is registered in the storage device 52 as the user's biometric information ("YES" in S801), processing proceeds to S802.

In S802, the remote control device 100 registers multiple users as the same group in the storage device 52.

In S803, the remote control device 100 grants remote control authority to the identification information (user ID or terminal ID) registered in the group. If there are multiple passengers inside the vehicle 110, the remote control device 100 grants remote control authority to the identification information (user ID or terminal ID) of the multiple passengers.

(Remote control instruction output)
Fig. 9 is a flowchart showing an example of the operation of the remote control device 100. An example of the operation relating to the output of a remote operation instruction executed by the remote control device 100 will be described with reference to Fig. 9. The output of a remote operation instruction refers to a process of transmitting a remote operation finger received from the terminal device 120 to the vehicle 110 to be operated.

At S901, the remote control device 100 receives a remote control instruction from the terminal device 120.

In S902, the remote control device 100 references the authority management information 57 and determines whether or not it has authority to perform remote control based on the identification information (user ID or terminal ID) included in the remote control instruction. If it is determined that it has authority to perform remote control ("YES" in S902), the process proceeds to S903; otherwise ("NO" in S902), S902 is repeated.

At S903, the remote control device 100 refers to the authority management information 57 and identifies the vehicle 110 to be operated (e.g., vehicle ID = V1 in Figure 5) based on the identification information included in the remote control instruction. The remote control device 100 then transmits a remote control instruction (remote control signal) to the identified vehicle 110 in accordance with the remote control instruction.

(Removal of remote control authority)
10 is a flowchart showing an example of the operation of the remote control device 100. An example of the operation of the remote control device 100 regarding the deletion of the authority of remote operation will be described with reference to FIG.

At S1001, the remote control device 100 receives an instruction to delete remote control authority from the terminal device 120. The remote control device 100 references the authority management information 57 (e.g., column 503 in Figure 5) and identifies the remote control authority to be deleted based on the identification information (user ID or terminal ID) included in the instruction to delete authority.

In S1002, the remote control device 100 references the authority management information 57 (e.g., column 504 in Figure 5) and determines whether the identification information of the remote control authority to be deleted is group registered. If it is determined that the identification information is group registered ("YES" in S1002), the process proceeds to S1003.

In S1003, the remote control device 100 deletes the remote control authority for the entire group. For example, in ST51 of FIG. 5, if an instruction to delete remote control authority is received from a terminal device 120 having identification information for user A (user ID = UA) or user B (user ID = UA), the remote control device 100 deletes the remote control authority associated with the terminal devices 120 of all users (A, B) in group 1 to which user A and user B belong.

On the other hand, if group registration has not been performed ("NO" in S1002), the process proceeds to S1004. In S1004, the remote control device 100 deletes the remote control authority identified in S1001. For example, in ST52 of FIG. 5, if an instruction to delete remote control authority is received from the terminal device 120 having the identification information of user C (user ID = UC), the remote control device 100 deletes the remote control authority associated with user C's terminal device 120.

(Example of operation of vehicle 110)
11 to 13 are flowcharts showing an example of the operation of vehicle 110. Each of the steps in FIGS. 11 to 13 may be performed by processor 51 executing a computer program stored in storage device 45. Alternatively, some or all of the steps in FIGS. 11 to 13 may be performed by a dedicated integrated circuit such as an ASIC. In such a case, the dedicated integrated circuit may operate as a functional unit for executing each step.

(Display usage code)
Fig. 11 is a flowchart showing an example of the operation of the vehicle 110. An example of the operation of the vehicle 110 regarding the display of a usage code will be described with reference to Fig. 11. In S1101, the vehicle 110 determines whether or not it has received the usage code transmitted from the remote control device 100 in S303 (Fig. 3). If it is determined that the usage code has been received ("YES" in S1101), the vehicle 110 transitions the process to S1102, and otherwise ("NO" in S1101) repeats S1101.

At S1102, the vehicle 110 displays the usage code using the display device 23 in response to the user's operation of the input device 24.

(Transmission of identification information)
Fig. 12 is a flowchart showing an example of the operation of the vehicle 110. An example of the operation related to the transmission of identification information executed by the vehicle 110 will be described with reference to Fig. 12. In the operation shown in Fig. 12, the vehicle 110 transmits, to the remote operation device 100, identification information associated with the terminal device 120 of the current operator (user) or the last operator (user) at the time of startup.

In S1201, the vehicle control unit 44 of the vehicle 110 determines whether the propulsion device 4 is started or stopped by controlling the ignition ON or OFF. If the vehicle 110 determines that the propulsion device 4 has started or stopped ("YES" in S1201), it transitions to S1202, and otherwise ("NO" in S1201) it repeats S1201.

In S1202, the vehicle 110 acquires identification information (user ID or terminal ID) associated with the terminal device 120 within the vehicle 110. The identification information (user ID or terminal ID) may be acquired using the communication device 9 of the vehicle 110. The identification information associated with the terminal device 120 may be the terminal ID of the terminal device 120 or the user ID of the user of the terminal device 120. If multiple terminal devices 120 are present within the vehicle 110, the vehicle 110 may acquire identification information (user ID or terminal ID) from each of the multiple terminal devices 120.

In S1203, the vehicle 110 transmits the identification information (user ID or terminal ID) acquired in S1202 to the remote control device 100. This identification information is used for registration in the authority management information 57 (e.g., FIG. 5) and for determining whether to grant remote control authorization (e.g., S702 in FIG. 7A). Note that in the above example, an example was described in which identification information associated with the terminal device 120 in the vehicle 110 is acquired when the vehicle 110 is started or stopped. However, the vehicle 110 may also transmit to the remote control device 100 identification information associated with a terminal device 120 that has traveled with the vehicle 110 for a predetermined period of time.

The communication device 53 of the remote control device 100 acquires, from the vehicle 110, identification information (user ID or terminal ID) associated with the terminal device 120 of the first user present inside the vehicle 110 when the vehicle 110 is started or stopped. The processor 51 of the remote control device 100 registers the identification information (user ID or terminal ID) acquired by the communication device 53 in the storage device 52, and grants remote control authority to the identification information (user ID or terminal ID) registered in the storage device 52.

(Operation based on remote control instructions)
13 is a flowchart showing an example of the operation of the vehicle 110. An example of the operation relating to vehicle control based on a remote operation instruction executed by the vehicle 110 will be described with reference to FIG.

In the operation shown in FIG. 13, the vehicle 110 operates in accordance with a remote control instruction. In S1301, the vehicle 110 determines whether it has received the remote control instruction transmitted in S309 from the remote control device 100 via the communication device 9. If it is determined that the remote control instruction has been received ("YES" in S1301), the vehicle 110 transitions the process to S1302; otherwise ("NO" in S1301), it repeats S1301.

In S1302, the vehicle 110 performs vehicle control in accordance with the remote operation instructions. The vehicle control unit 44 controls the vehicle 110 in accordance with the remote operation instructions. For example, it controls the locking and unlocking of the doors of the vehicle 110, the activation of the power system that supplies power to each part of the vehicle 110, the activation of the propulsion unit 4, etc.

[Other embodiments]
Furthermore, a remote control program that realizes one or more functions described in each embodiment can be supplied to a terminal device via a network or a storage medium, and one or more processors in a computer of the system or device can read and execute the remote control program. The present invention can also be realized in such an embodiment.

<Summary of the embodiment>
(Item 1) The remote control device of the above embodiment is a remote control device (100) that remotely controls a vehicle,
a storage means (52) for registering biometric information of a user in association with identification information set in the terminal device of the user;
A communication means (53) for acquiring biometric information of a first user present inside the vehicle;
and a computing means (51) for granting the authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage means (52).

The remote control device of item 1 can improve convenience for users who share a vehicle with multiple people. For example, being able to board a vehicle can be inferred to mean having the authority to operate the vehicle. Therefore, by granting remote control authority to users authenticated by biometric authentication and allowing remote operation from outside the vehicle, user convenience can be improved.

(Item 2) When the biometric information of the first user is not registered in the storage means (52), the calculation means (51)
When the usage code presented by the interface device inside the vehicle matches the usage code transmitted from the user's terminal device,
The remote control authority is granted to identification information associated with the terminal device of the first user, which is registered in the storage means (52).

With the remote control device of item 2, if biometric information has not been registered, the user can enter the vehicle using a mechanical unlocking method and authenticate the usage code via an in-vehicle operation, thereby activating the remote control functions provided by the vehicle remote control system.

(Item 3) When the calculation means (51) grants the authorization for remote operation to the identification information associated with the terminal device of the first user by comparing the usage code,
The storage means (52) registers the biometric information of the first user acquired by the communication means (53) in association with identification information associated with the terminal device of the user.

The remote control device of item 3 makes it possible to register the biometric information of the user inside the vehicle in association with the identification information associated with the user's terminal device when activating the remote control function provided by the vehicle remote control system.

(Item 4) When the communication means (53) acquires biometric information of a second user after acquiring biometric information of the first user, the calculation means (51) deletes the remote control authority granted to the second user before the second user.

With the remote control device of item 4, when the vehicle user changes, the previous user's authority can be revoked by acquiring the subsequent user's biometric information. This improves the security of the vehicle when remotely controlled. For example, when a vehicle is resold, the previous user's authority to remotely control the vehicle can be revoked by having a user (e.g., a staff member) enter the vehicle at the resale destination and acquire the biometric information, without leaving any authority to the previous user.

(Item 5) The communication means (53) acquires an image including a face of the first user as the biometric information,
The calculation means (51) acquires facial information by extracting facial features of the first user from the image,
The storage means (52) registers the face information and the identification information as biometric information of the first user in association with each other.

The remote control device of item 5 can associate facial information extracted as biometric information of a user present inside a vehicle with identification information.

(Item 6) The storage means (52) has a function of registering a plurality of users as a group, and when the first user and the second user are registered as the same group,
When the communication means (53) acquires biometric information of a second user after the first user,
The computing means (51) maintains the remote control authority granted to the identification information associated with the terminal device of the first user and the remote control authority granted to the identification information associated with the terminal device of the second user.

The remote control device of item 6 allows users to be registered as a group, thereby ensuring security in remote control while improving convenience for multiple users when multiple users share a single vehicle. By adding or removing users from the group, the range of users who can use the remote control functions provided by the vehicle remote control system can be set.

(Item 7) The communication means (53) acquires images including faces of a plurality of users present inside the vehicle,
When the facial information of the multiple users extracted from the image is registered in the storage means (52) as biometric information of the multiple users, the calculation means (51) registers the multiple users as the same group.

The remote control device of item 7 allows passengers to be registered as part of the same group and granted remote control authority, improving convenience for multiple users.

(Item 8) The calculation means (51) outputs a signal for operating the vehicle based on a remote control request received from a terminal device having identification information authorized for remote control.

The remote control device of item 8 allows a vehicle to be remotely controlled based on a remote control request sent from a terminal device with identification information that has been granted remote control authority.

(Item 9) The communication means (53) acquires the biometric information, the parking position of the vehicle, and date and time information specifying the parking date and time from the vehicle,
The calculation means (51)
Based on the information on the parking position of the vehicle and the parking date and time, a parking frequency for each parking position of the vehicle during a predetermined period is acquired, and the parking position with the highest parking frequency is identified as a normal parking area in which the vehicle is normally parked;
When the location where the biometric information was acquired is not within the normal parking area, the calculation means (51) deletes the authority of remote operation assigned to the identification information associated with the biometric information of the first user,
The identification information associated with the biometric information acquired at the location is given authority to remotely operate the vehicle.

(Item 10) The communication means (53) acquires the biometric information, the parking position of the vehicle, and date and time information specifying the parking date and time from the vehicle,
The calculation means (51)
Based on the information on the parking position of the vehicle and the parking date and time, a parking frequency for each parking position of the vehicle during a predetermined period is acquired, and the parking position with the highest parking frequency is identified as a normal parking area in which the vehicle is normally parked;
When the location where the biometric information was acquired is within the normal parking area, the calculation means (51) maintains the remote control authority granted to the identification information associated with the biometric information of the first user.

The remote control device of items 9 and 10 can maintain or update remote control authority based on the location where biometric information was acquired. When the location where biometric information was acquired is within a normal parking area, use such as when a family shares a vehicle can be envisioned. In such cases, when new biometric information is acquired, the authority of the user who was previously granted remote control authority is maintained without deleting the previously granted remote control authority. This improves convenience when a vehicle is shared by multiple users. Furthermore, by maintaining the authority of the user who was granted remote control authority, it becomes possible to remotely restrict the operation of the vehicle even if the vehicle is stolen, for example, thereby improving the security of the vehicle 110.

(Item 11) The control method for the remote control device of the above embodiment is a control method for a remote control device that remotely operates a vehicle,
a step of registering the user's biometric information and the identification information set in the user's terminal device in association with each other in a storage means (52);
A step in which a communication means (53) acquires biometric information of a first user present inside the vehicle;
The method includes a step in which a calculation means (51) grants the authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user from among the biometric information registered in the storage means (52).

The control method for a remote control device in Item 11 can improve convenience for users who share a vehicle with multiple people. For example, being able to board a vehicle can be inferred to mean having the authority to operate the vehicle. Therefore, by granting remote control authority to users authenticated by biometric authentication and allowing remote operation from outside the vehicle, user convenience can be improved.

(Item 12) A program that causes a computer to execute the control method described in Item 11.

The program of item 12 can provide a program that causes a computer to execute a control method for a remote control device.

The present invention is not limited to the above-described embodiments, and various modifications and variations are possible within the scope of the present invention.

100: Remote control device, 110: Vehicle, 120: Terminal device

Claims (13)

A remote control device for remotely operating a vehicle,
a storage means for registering biometric information of a user in association with identification information set in a terminal device of the user;
a communication means for acquiring biometric information of a first user present inside the vehicle;
a computing unit that grants authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage unit; and
Equipped with
The storage means has a function of registering a plurality of users as a group, and when the first user and the second user are registered as the same group,
When the communication unit acquires biometric information of a second user after the first user,
A remote control device characterized in that the calculation means maintains the remote control authority granted to the identification information associated with the terminal device of the first user and the remote control authority granted to the identification information associated with the terminal device of the second user . When the biometric information of the first user is not registered in the storage means, the calculation means
When the usage code presented by the interface device inside the vehicle matches the usage code transmitted from the user's terminal device,
2. The remote control device according to claim 1, wherein the authority to perform the remote control is given to identification information associated with the terminal device of the first user, which is registered in the storage means. When the computing means grants the authorization for remote operation to the identification information associated with the terminal device of the first user by comparing the usage code,
3. The remote control device according to claim 2, wherein the storage means registers the biometric information of the first user acquired by the communication means in association with identification information associated with the terminal device of the user. The remote control device described in claim 1, characterized in that when the communication means acquires biometric information of a second user after the first user, the calculation means deletes the remote control authority granted before the second user. the communication means acquires an image including a face of the first user as the biometric information;
the computing means acquires facial information by extracting facial features of the first user from the image;
2. The remote control device according to claim 1, wherein the storage unit registers the face information and the identification information as biometric information of the first user in association with each other. the communication means acquires images including faces of a plurality of users present inside the vehicle;
The remote control device according to claim 1, characterized in that, when the facial information of the plurality of users extracted from the image is registered in the storage means as biometric information of the plurality of users, the calculation means registers the plurality of users as the same group. The remote control device described in claim 1, characterized in that the calculation means outputs a signal for operating the vehicle based on a remote control request received from a terminal device having identification information authorized for remote control. A remote control device for remotely operating a vehicle,
a storage means for registering biometric information of a user in association with identification information set in a terminal device of the user;
a communication means for acquiring biometric information of a first user present inside the vehicle;
a computing means for granting authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage means,
The communication means acquires the biometric information, the parking position of the vehicle, and date and time information specifying the parking date and time from the vehicle;
The calculation means
Based on the information on the parking position of the vehicle and the parking date and time, a parking frequency for each parking position of the vehicle during a predetermined period is acquired, and the parking position with the highest parking frequency is identified as a normal parking area in which the vehicle is normally parked;
When the location where the biometric information was acquired is not within the normal parking area, the calculation means deletes the authority of remote operation assigned to the identification information associated with the biometric information of the first user, and
A remote control device characterized in that authority to remotely control the vehicle is granted to the identification information associated with the biometric information acquired at the location. A remote control device for remotely operating a vehicle,
a storage means for registering biometric information of a user in association with identification information set in a terminal device of the user;
a communication means for acquiring biometric information of a first user present inside the vehicle;
a computing means for granting authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage means,
The communication means acquires the biometric information, the parking position of the vehicle, and date and time information specifying the parking date and time from the vehicle;
The calculation means
Based on the information on the parking position of the vehicle and the parking date and time, a parking frequency for each parking position of the vehicle during a predetermined period is acquired, and the parking position with the highest parking frequency is identified as a normal parking area in which the vehicle is normally parked;
A remote control device characterized in that the calculation means maintains the remote control authority granted to the identification information corresponding to the biometric information of the first user when the location where the biometric information was acquired is within the normal parking area. A control method for a remote control device that remotely controls a vehicle, comprising:
a step of registering the biometric information of the user and the identification information set in the terminal device of the user in association with each other in a storage means;
a step of acquiring biometric information of a first user present inside the vehicle by a communication means;
a step of granting, by a computing means, authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage means;
and
The storage means has a function of registering a plurality of users as a group, and when the first user and the second user are registered as the same group,
When the communication unit acquires biometric information of a second user after the first user,
A control method characterized in that the calculation means maintains the remote control authority granted to the identification information associated with the terminal device of the first user and the remote control authority granted to the identification information associated with the terminal device of the second user . A control method for a remote control device that remotely controls a vehicle, comprising:
a step of registering the biometric information of the user and the identification information set in the terminal device of the user in association with each other in a storage means;
a step of acquiring biometric information of a first user present inside the vehicle by a communication means;
and a step by a computing means of granting authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage means,
The communication means acquires the biometric information, the parking position of the vehicle, and date and time information specifying the parking date and time from the vehicle;
The calculation means
Based on the information on the parking position of the vehicle and the parking date and time, a parking frequency for each parking position of the vehicle during a predetermined period is acquired, and the parking position with the highest parking frequency is identified as a normal parking area in which the vehicle is normally parked;
When the location where the biometric information was acquired is not within the normal parking area, the calculation means deletes the authority of remote operation assigned to the identification information associated with the biometric information of the first user, and
A control method comprising: granting authority to remotely control the vehicle to the identification information associated with the biometric information acquired at the location. A control method for a remote control device that remotely controls a vehicle, comprising:
a step of registering the biometric information of the user and the identification information set in the terminal device of the user in association with each other in a storage means;
a step of acquiring biometric information of a first user present inside the vehicle by a communication means;
and a step by a computing means of granting authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage means,
The communication means acquires the biometric information, the parking position of the vehicle, and date and time information specifying the parking date and time from the vehicle;
The calculation means
Based on the information on the parking position of the vehicle and the parking date and time, a parking frequency for each parking position of the vehicle during a predetermined period is acquired, and the parking position with the highest parking frequency is identified as a normal parking area in which the vehicle is normally parked;
A control method characterized in that the calculation means maintains the remote operation authority granted to the identification information corresponding to the biometric information of the first user when the location where the biometric information was acquired is within the normal parking area. A program for causing a computer to execute the control method according to any one of claims 10 to 12 .