JP7731312B2 - Vehicle control device, vehicle control method, and program - Google Patents

Description

The present invention relates to a vehicle control device, a vehicle control method, and a program.

Technology for controlling the driving of a vehicle based on road dividing lines recognized by a camera mounted on the vehicle is known. For example, Patent Document 1 describes a technology for driving a vehicle based on recognized road dividing lines, and for driving the vehicle based on the trajectory of a preceding vehicle if the degree of recognition of the road dividing lines does not meet a predetermined standard.

Japanese Patent Application Laid-Open No. 2020-050086

The technology described in Patent Document 1 controls the driving of a vehicle based on road dividing lines recognized by a camera and map information carried by the vehicle. However, with conventional technology, if the road dividing lines recognized by the camera differ from the content of the map information carried by the vehicle, it may not be possible to appropriately change the vehicle's driving control.

The present invention was made in consideration of these circumstances, and one of its objectives is to provide a vehicle control device, vehicle control method, and program that can appropriately change vehicle driving control even when the road dividing lines recognized by the camera differ from the content of the map information installed in the vehicle.

A vehicle control device, a vehicle control method, and a program according to the present invention employ the following configuration.
(1): A vehicle control device according to one aspect of the present invention includes an acquisition unit that acquires camera images of a vehicle's surroundings, a driving control unit that controls steering and acceleration/deceleration of the vehicle based on the camera images and map information without relying on an operation by a driver of the vehicle, and a driving mode of the vehicle is determined to be one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task assigned to the driver is lighter than that assigned to the first driving mode, at least some of the plurality of driving modes including the second driving mode being controlled by the driving control unit, and a task associated with the determined driving mode not being performed by the driver. and a determination unit that determines whether or not there is a discrepancy between the road dividing line shown in the camera image and the road dividing line shown in the map information, and determines whether or not the vehicle is at a branch point shown in the map information.When it is determined that there is a discrepancy between the road dividing line shown in the camera image and the road dividing line shown in the map information, and it is determined that the vehicle is at a branch point shown in the map information, the mode determination unit determines the driving mode of the vehicle based on the branching direction of the branch point and the direction of the road dividing line shown in the camera image.

(2): In the aspect (1) above, if the deviation between the branching direction at the branching point and the direction of the road dividing line shown in the camera image is equal to or greater than a threshold, the mode determination unit changes the second driving mode to the first driving mode and continues the first driving mode using the road dividing line shown in the camera image.

(3): In the above aspect (1) or (2), if the difference between the branching direction at the branch point and the direction of the road dividing line shown in the camera image is less than a threshold, the mode determination unit continues the second driving mode using the road dividing line shown in the map information.

(4): In any of the above aspects (1) to (3), the second driving mode is a driving mode in which the driver is not tasked with holding an operator that accepts steering operations of the vehicle, and the first driving mode is a driving mode in which the driver is tasked with at least holding the operator.

(5): Another aspect of the present invention relates to a vehicle control method in which a computer acquires a camera image capturing a situation around the vehicle, and controls the steering and acceleration/deceleration of the vehicle based on the camera image and map information without relying on the operation of the driver of the vehicle, and determines the driving mode of the vehicle to one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which the task assigned to the driver is lighter than that of the first driving mode, and at least some of the plurality of driving modes including the second driving mode are performed by controlling the steering and acceleration/deceleration of the vehicle without relying on the operation of the driver of the vehicle, and the determined If the driver does not perform a task related to the selected driving mode, the driving mode of the vehicle is changed to a driving mode with a more severe task, and the system determines whether there is a discrepancy between the road dividing lines shown in the camera image and the road dividing lines shown in the map information, and determines whether the vehicle is at a branch point shown in the map information. If it is determined that there is a discrepancy between the road dividing lines shown in the camera image and the road dividing lines shown in the map information, and it is determined that the vehicle is at a branch point shown in the map information, the system determines the driving mode of the vehicle based on the branch direction at the branch point and the direction of the road dividing lines shown in the camera image.

(6): A program according to another aspect of the present invention causes a computer to acquire camera images of the vehicle's surroundings, and based on the camera images and map information, controls the steering and acceleration/deceleration of the vehicle without relying on the driver's operation, and determines the vehicle's driving mode to one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which the driver's task is lighter than that of the first driving mode, and at least some of the plurality of driving modes including the second driving mode are performed by controlling the steering and acceleration/deceleration of the vehicle without relying on the driver's operation, and the determined If the driver does not perform a task related to a driving mode, the system changes the vehicle's driving mode to a driving mode with a more severe task, determines whether there is a discrepancy between the road dividing lines shown in the camera image and the road dividing lines shown in the map information, and determines whether the vehicle is at a branch point shown in the map information, and if it is determined that there is a discrepancy between the road dividing lines shown in the camera image and the road dividing lines shown in the map information and the vehicle is at a branch point shown in the map information, determines the vehicle's driving mode based on the branch direction at the branch point and the direction of the road dividing lines shown in the camera image.

(7): A program according to another aspect of the present invention causes a computer to acquire camera images of the vehicle's surroundings, and based on the camera images and map information, controls the steering and acceleration/deceleration of the vehicle without relying on the operation of the driver of the vehicle, and determines the driving mode of the vehicle to one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which the task assigned to the driver is lighter than that of the first driving mode, and at least some of the plurality of driving modes including the second driving mode are performed by controlling the steering and acceleration/deceleration of the vehicle without relying on the operation of the driver of the vehicle, and the determined If the driver does not perform a task related to a driving mode, the system changes the vehicle's driving mode to a driving mode with a more severe task, determines whether there is a discrepancy between the road dividing lines shown in the camera image and the road dividing lines shown in the map information, and determines whether the vehicle is at a branch point shown in the map information, and if it is determined that there is a discrepancy between the road dividing lines shown in the camera image and the road dividing lines shown in the map information and the vehicle is at a branch point shown in the map information, determines the vehicle's driving mode based on the branch direction at the branch point and the direction of the road dividing lines shown in the camera image.

(1) to (7) enable the vehicle's driving control to be appropriately changed even if the road dividing lines recognized by the camera differ from the map information installed in the vehicle.

1 is a configuration diagram of a vehicle system 1 using a vehicle control device according to an embodiment. FIG. 2 is a functional configuration diagram of a first control unit 120 and a second control unit 160. A figure showing an example of the correspondence between driving modes, control states of the vehicle M, and tasks. FIG. 2 is a diagram illustrating an example of a scene in which the operation of the vehicle control device according to the embodiment is performed. FIG. 10 is a diagram illustrating another example of a scene in which the operation of the vehicle control device according to the embodiment is performed. 4 is a flowchart illustrating an example of a flow of an operation executed by the vehicle control device according to the embodiment.

Embodiments of the vehicle control device, vehicle control method, and program of the present invention will be described below with reference to the drawings.

[Overall configuration]
1 is a configuration diagram of a vehicle system 1 using a vehicle control device according to an embodiment. The vehicle on which the vehicle system 1 is installed may be, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source may be an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of these. The electric motor operates using power generated by a generator connected to the internal combustion engine, or discharged power from a secondary battery or a fuel cell.

The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) device 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, vehicle sensors 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driver monitor camera 70, driving controls 80, an autonomous driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are connected to each other via multiplexed communication lines such as a CAN (Controller Area Network) communication line, serial communication lines, a wireless communication network, etc. Note that the configuration shown in FIG. 1 is merely an example, and some of the components may be omitted or additional components may be added.

Camera 10 is a digital camera that uses a solid-state imaging element such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). Camera 10 is attached to any location on the vehicle (hereinafter referred to as host vehicle M) in which vehicle system 1 is installed. When capturing images of the front, camera 10 is attached to the top of the front windshield, the back of the rearview mirror, or the like. Camera 10, for example, periodically captures images of the surroundings of host vehicle M. Camera 10 may also be a stereo camera.

The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by objects (reflected waves) to detect at least the position (distance and direction) of the object. The radar device 12 may be mounted at any location on the vehicle M. The radar device 12 may also detect the position and speed of an object using the FM-CW (Frequency Modulated Continuous Wave) method.

The LIDAR 14 irradiates the area around the vehicle M with light (or electromagnetic waves with wavelengths similar to light) and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time between light emission and light reception. The irradiated light is, for example, pulsed laser light. The LIDAR 14 can be attached to any location on the vehicle M.

The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the camera 10, radar device 12, and LIDAR 14 to recognize the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition results to the autonomous driving control device 100. The object recognition device 16 may output the detection results from the camera 10, radar device 12, and LIDAR 14 directly to the autonomous driving control device 100. The object recognition device 16 may be omitted from the vehicle system 1.

The communication device 20 communicates with other vehicles in the vicinity of the vehicle M, for example, using a cellular network, Wi-Fi network, Bluetooth (registered trademark), or DSRC (Dedicated Short Range Communication), or communicates with various server devices via a wireless base station.

The HMI 30 presents various information to the occupants of the vehicle M and accepts input operations from the occupants. The HMI 30 includes various display devices, speakers, buzzers, touch panels, switches, keys, etc.

The vehicle sensors 40 include a vehicle speed sensor that detects the speed of the host vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects angular velocity around a vertical axis, and a direction sensor that detects the orientation of the host vehicle M.

The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as a hard disk drive (HDD) or flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be determined or supplemented by an inertial navigation system (INS) that uses the output of the vehicle sensors 40. The navigation HMI 52 includes a display device, speaker, touch panel, keys, etc. The navigation HMI 52 may share some or all of the components with the HMI 30 described above. The route determination unit 53 determines a route (hereinafter, a map route) from the position of the vehicle M determined by the GNSS receiver 51 (or any input position) to a destination input by the occupant using the navigation HMI 52, for example, by referring to the first map information 54. The first map information 54 is information that represents road shapes using, for example, links indicating roads and nodes connected by the links. The first map information 54 may also include information such as road curvature and POI (Point of Interest) information. The route on the map is output to the MPU 60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be implemented, for example, by the functions of a terminal device such as a smartphone or tablet device owned by the occupant. The navigation device 50 may transmit the current position and destination to a navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server.

The MPU 60 includes, for example, a recommended lane determination unit 61, and stores second map information 62 in a storage device such as an HDD or flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into multiple blocks (for example, every 100 m in the vehicle's direction of travel) and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 determines, for example, which lane from the left to use in the route on the map. If there is a branch point on the route on the map, the recommended lane determination unit 61 determines a recommended lane so that the host vehicle M can travel on a reasonable route to the branch point.

The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the centers of lanes or information on lane boundaries. The second map information 62 may also include road information, traffic regulation information, address information (address and postal code), facility information, telephone number information, and information on prohibited sections where mode A or mode B, described below, is prohibited. The second map information 62 may be updated as needed by the communication device 20 communicating with other devices. In this embodiment, the second map information 62 particularly includes location information of lane branching points as meta information.

The driver monitor camera 70 is a digital camera that uses a solid-state imaging element such as a CCD or CMOS. The driver monitor camera 70 is mounted at any location on the vehicle M in a position and orientation that allows it to capture an image of the head of an occupant (hereinafter referred to as the driver) seated in the driver's seat of the vehicle M from the front (in an orientation that captures the face). For example, the driver monitor camera 70 is mounted above a display device located in the center of the instrument panel of the vehicle M.

The driving operators 80 include, for example, a steering wheel 82, as well as an accelerator pedal, brake pedal, shift lever, and other operators. The driving operators 80 are equipped with sensors that detect the amount of operation or the presence or absence of operation, and the detection results are output to the automatic driving control device 100 or some or all of the driving force output device 200, brake device 210, and steering device 220. The steering wheel 82 is an example of an "operator that accepts steering operation by the driver." The operator does not necessarily have to be annular, and may be in the form of an irregular steering wheel, joystick, button, or the like. A steering grip sensor 84 is attached to the steering wheel 82. The steering grip sensor 84 is realized by a capacitance sensor or the like, and outputs a signal to the automatic driving control device 100 that can detect whether the driver is gripping the steering wheel 82 (meaning that the driver is in contact with the steering wheel in a manner that allows force to be applied).

The autonomous driving control device 100 includes, for example, a first control unit 120 and a second control unit 160. The first control unit 120 and the second control unit 160 are each implemented by a hardware processor, such as a CPU (Central Processing Unit), executing a program (software). Some or all of these components may be implemented by hardware (including circuitry), such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or GPU (Graphics Processing Unit), or by a combination of software and hardware. The program may be stored in advance on a storage device (storage device with a non-transitory storage medium) such as a HDD or flash memory of the autonomous driving control device 100, or may be stored on a removable storage medium such as a DVD or CD-ROM, and installed on the HDD or flash memory of the autonomous driving control device 100 by inserting the storage medium (non-transitory storage medium) into a drive device. The automatic driving control device 100 is an example of a "vehicle control device," and the combination of the action plan generation unit 140 and the second control unit 160 is an example of a "driving control unit."

Figure 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130, an action plan generation unit 140, and a mode determination unit 150. The first control unit 120, for example, implements functions based on AI (Artificial Intelligence) and functions based on pre-specified models in parallel. For example, the "intersection recognition" function may be implemented by executing intersection recognition using deep learning or the like and recognition based on pre-specified conditions (such as traffic lights and road markings that can be pattern-matched) in parallel, and then scoring and comprehensively evaluating both. This ensures the reliability of autonomous driving.

The recognition unit 130 recognizes the position, speed, acceleration, and other status of objects around the vehicle M based on information input from the camera 10, radar device 12, and LIDAR 14 via the object recognition device 16. The position of an object is recognized as a position on an absolute coordinate system with a representative point of the vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or by an area. The "state" of an object may include the acceleration or jerk of the object, or its "behavioral state" (for example, whether or not the vehicle is changing lanes or is about to change lanes).

The recognition unit 130 also recognizes, for example, the lane in which the host vehicle M is traveling (driving lane). For example, the recognition unit 130 recognizes the driving lane by comparing the pattern of road dividing lines (e.g., an arrangement of solid and dashed lines) obtained from the second map information 62 with the pattern of road dividing lines around the host vehicle M recognized from the image captured by the camera 10. Note that the recognition unit 130 may recognize the driving lane by recognizing road boundaries (road boundaries) including not only road dividing lines but also road dividing lines, shoulders, curbs, medians, guardrails, etc. This recognition may take into account the position of the host vehicle M obtained from the navigation device 50 and the processing results from the INS. The recognition unit 130 also recognizes stop lines, obstacles, red lights, toll booths, and other road phenomena.

When recognizing the driving lane, the recognition unit 130 recognizes the position and orientation of the host vehicle M relative to the driving lane. For example, the recognition unit 130 may recognize the deviation of the host vehicle M's reference point from the center of the lane and the angle it makes with a line connecting the centers of the lanes in the host vehicle M's direction of travel as the relative position and orientation of the host vehicle M relative to the driving lane. Alternatively, the recognition unit 130 may recognize the position of the host vehicle M's reference point relative to either side edge of the driving lane (a road dividing line or road boundary) as the relative position of the host vehicle M relative to the driving lane.

The behavior plan generation unit 140 essentially drives the recommended lane determined by the recommended lane determination unit 61, and further generates a target trajectory for the host vehicle M to travel automatically (without relying on driver operation) in the future so that the host vehicle M can respond to the surrounding conditions. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequential arrangement of points (trajectory points) that the host vehicle M should reach. Trajectory points are points that the host vehicle M should reach at every predetermined driving distance (e.g., several meters) along the road. Separately, target speeds and target accelerations are generated as part of the target trajectory for every predetermined sampling time (e.g., several tenths of a second). Furthermore, the trajectory points may be the positions that the host vehicle M should reach at each predetermined sampling time. In this case, the target speed and target acceleration information is expressed as the interval between trajectory points.

When generating a target trajectory, the behavior plan generation unit 140 may set an autonomous driving event. Autonomous driving events include a constant speed driving event, a low-speed following driving event, a lane change event, a branching event, a merging event, and a takeover event. The behavior plan generation unit 140 generates a target trajectory according to the activated event.

The mode determination unit 150 determines the driving mode of the host vehicle M to be one of multiple driving modes that assign different tasks to the driver. The mode determination unit 150 includes, for example, a determination unit 152. The function of the determination unit 152 will be described later.

Figure 3 is a diagram showing an example of the correspondence between driving modes, control states of the host vehicle M, and tasks. The host vehicle M has five driving modes, for example, Mode A to Mode E. The control state, i.e., the degree of automation of the host vehicle M's driving control, is highest in Mode A, followed by Mode B, Mode C, and Mode D, with Mode E being the lowest. Conversely, the tasks imposed on the driver are lightest in Mode A, followed by Mode B, Mode C, and Mode D, with Mode E being the most severe. Note that Modes D and E are non-autonomous control states, and therefore the autonomous driving control device 100 is responsible for terminating autonomous driving control and transitioning to driving assistance or manual driving. The following provides examples of the contents of each driving mode.

In Mode A, the vehicle is in an autonomous driving state, and the driver is not required to monitor the road ahead or hold the steering wheel 82 (in the figure, the driver is holding the steering wheel). However, even in Mode A, the driver is required to be in a position where they can quickly switch to manual driving in response to a request from the system centered on the autonomous driving control device 100. Note that autonomous driving here means that both steering and acceleration/deceleration are controlled independently of the driver's operation. "Ahead" refers to the space in the direction of travel of the host vehicle M, as seen through the front windshield. Mode A is a driving mode that can be implemented, for example, on a highway or other motorway, when certain conditions are met, such as the host vehicle M traveling at a predetermined speed (e.g., approximately 50 km/h) or less and there being a vehicle ahead to be followed. Mode A is sometimes referred to as TJP (Traffic Jam Pilot). If these conditions are no longer met, the mode determination unit 150 changes the driving mode of the host vehicle M to Mode B.

In Mode B, the vehicle is in a driving assistance state, and the driver is tasked with monitoring the area ahead of the vehicle M (hereinafter referred to as "forward monitoring"), but is not tasked with holding the steering wheel 82. In Mode C, the vehicle is in a driving assistance state, and the driver is tasked with monitoring the area ahead and holding the steering wheel 82. Mode D is a driving mode in which the driver must perform some degree of driving operation for at least one of steering and accelerating/decelerating the vehicle M. For example, in Mode D, driving assistance such as ACC (Adaptive Cruise Control) and LKAS (Lane Keeping Assist System) is provided. Mode E is a manual driving state in which the driver must perform both steering and accelerating/decelerating operations. In both Modes D and E, the driver is naturally tasked with monitoring the area ahead of the vehicle M.

The automatic driving control device 100 (and the driving assistance device (not shown)) executes an automated lane change according to the driving mode. Automated lane changes include system-requested automated lane changes (1) and driver-requested automated lane changes (2). Automated lane changes (1) include automated lane changes for overtaking, which are performed when the speed of a leading vehicle is slower than the speed of the vehicle itself by a certain standard, and automated lane changes for proceeding toward a destination (automated lane changes due to a change in the recommended lane). Automated lane changes (2) are performed when the driver operates a turn signal and conditions related to speed and positional relationship with surrounding vehicles are met, causing the vehicle M to change lanes in the direction of the operation.

In mode A, the automatic driving control device 100 does not perform either automated lane change (1) or (2). In modes B and C, the automatic driving control device 100 performs either automated lane change (1) or (2). In mode D, the driving assistance device (not shown) does not perform automated lane change (1), but performs automated lane change (2). In mode E, neither automated lane change (1) nor (2) is performed.

When the driver does not perform a task related to the determined driving mode (hereinafter referred to as the current driving mode), the mode determination unit 150 changes the driving mode of the host vehicle M to a driving mode with a more severe task.

For example, in mode A, if the driver is in a position where they cannot switch to manual driving in response to a request from the system (for example, if they continue to look away from the road outside the permitted area or if signs of driving difficulty are detected), the mode determination unit 150 uses the HMI 30 to prompt the driver to switch to manual driving, and if the driver does not comply, it controls the vehicle M to move to the shoulder of the road and gradually stop, and stops the autonomous driving. After autonomous driving is stopped, the vehicle enters mode D or E, and the driver can manually operate the vehicle M to start moving. The same applies below to "stopping autonomous driving." In mode B, if the driver is not monitoring the road ahead, the mode determination unit 150 uses the HMI 30 to prompt the driver to monitor the road ahead, and if the driver does not comply, it controls the vehicle M to move to the shoulder of the road and gradually stop, and stops the autonomous driving. In mode C, if the driver is not monitoring the road ahead or is not gripping the steering wheel 82, the mode determination unit 150 uses the HMI 30 to prompt the driver to monitor the road ahead and/or grip the steering wheel 82, and if the driver does not comply, the mode determination unit 150 performs control such as pulling the vehicle M over to the shoulder of the road and gradually bringing it to a halt, thereby terminating the autonomous driving.

The second control unit 160 controls the driving force output device 200, the braking device 210, and the steering device 220 so that the host vehicle M passes through the target trajectory generated by the action plan generation unit 140 at the scheduled time.

Returning to FIG. 2, the second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires information about the target trajectory (trajectory points) generated by the action plan generation unit 140 and stores it in memory (not shown). The speed control unit 164 controls the driving force output device 200 or the brake device 210 based on the speed element associated with the target trajectory stored in memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in memory. The processing of the speed control unit 164 and the steering control unit 166 is realized, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road ahead of the host vehicle M and feedback control based on the deviation from the target trajectory.

The driving force output device 200 outputs driving force (torque) to the drive wheels to propel the vehicle. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU (Electronic Control Unit) that controls these. The ECU controls the above components in accordance with information input from the second control unit 160 or information input from the driving operator 80.

The brake device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or information input from the driving operator 80, so that brake torque corresponding to the braking operation is output to each wheel. The brake device 210 may include a backup mechanism that transmits hydraulic pressure generated by operation of the brake pedal included in the driving operator 80 to the cylinder via a master cylinder. Note that the brake device 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic brake device that controls an actuator according to information input from the second control unit 160 to transmit hydraulic pressure from the master cylinder to the cylinder.

The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor applies force to a rack and pinion mechanism to change the direction of the steered wheels. The steering ECU drives the electric motor to change the direction of the steered wheels in accordance with information input from the second control unit 160 or information input from the driving operator 80.

[Operation of vehicle control device]
Next, the operation of the vehicle control device according to the embodiment will be described. In the following description, it is assumed that the host vehicle M is traveling in the driving mode of mode B. Fig. 4 is a diagram showing an example of a scene in which the operation of the vehicle control device according to the embodiment is executed. Fig. 4 shows a situation in which the host vehicle M is traveling on lane L1, just before a branch point, and is set in the driving mode of mode B so as to continue straight ahead.

While the host vehicle M is traveling in lane L1, the recognition unit 130 recognizes the surrounding conditions of the host vehicle M, particularly the road dividing lines on both sides of the host vehicle M, based on images captured by the camera 10. Hereinafter, the road dividing lines recognized based on the images captured by the camera 10 will be represented by CL (hereinafter referred to as "camera road dividing lines CL"), and the road dividing lines recognized based on the second map information 62 will be represented by ML (hereinafter referred to as "map road dividing lines ML"). In Figure 4, the map road dividing lines ML deviate from the actual road dividing lines AL. This represents a situation in which the map road dividing lines ML and the actual road dividing lines AL deviate from each other due to, for example, construction or redrawing of road dividing lines.

While the host vehicle M is traveling, the determination unit 152 determines whether there is a discrepancy (mismatch) between the camera road dividing line CL and the map road dividing line ML. Here, discrepancy means, for example, that the distance between the camera road dividing line CL and the map road dividing line ML is equal to or greater than a predetermined value. The determination unit 152 further determines whether the host vehicle M is at a branch point based on the branch point position information obtained from the second map information 62. More specifically, for example, the recognition unit 130 determines whether the host vehicle M is at a branch point by determining whether the host vehicle M is within a predetermined distance from the branch point position information (e.g., GPS coordinates).

When the mode determination unit 150 determines that there is a discrepancy between the camera road dividing line CL and the map road dividing line ML and that the vehicle M is at a branching point, it determines the driving mode of the vehicle M based on the branching direction of the branching point obtained from the second map information 62 and the direction of the camera road dividing line CL.

More specifically, as shown in FIG. 4, for example, the mode determination unit 150 calculates the deviation between the branch direction (extension direction of the map road dividing line ML) of the branch point obtained from the second map information 62 and the direction of the camera road dividing line CL. If the calculated deviation is equal to or greater than a threshold, it determines that the camera road dividing line CL is more reliable than the map road dividing line ML. This is because, generally, when the vehicle M passes by a branch point, there is a high frequency of erroneous recognition in which the recognized camera road dividing line CL is closer to the branch road ML than to the main road due to, for example, the special paint conditions at the branch point. In other words, if the recognized camera road dividing line CL is not closer to the branch road ML, this means that the reliability of the camera road dividing line CL is high.

Therefore, if the deviation between the branch direction at the branch point obtained from the second map information 62 and the direction of the camera road dividing line CL is equal to or greater than a threshold, the mode determination unit 150 changes the driving mode from mode B to mode C and continues driving in mode C driving mode using the camera road dividing line CL as a reference line. Alternatively, the mode determination unit 150 may continue driving in mode B driving mode using the camera road dividing line CL as a reference line without changing the driving mode from mode B. This makes it possible to control the driving of the host vehicle M using more reliable road dividing lines, even if there is a mismatch between the camera road dividing line CL and the map road dividing line ML at the branch point.

Figure 5 is a diagram showing another example of a scenario in which the operation of the vehicle control device according to the embodiment is performed. The scenario in Figure 5 illustrates a situation in which the camera road dividing line CL has shifted toward the branching road ML, resulting in the camera road dividing line CL being drawn in a direction different from the lane L1 in which the host vehicle M is originally traveling. In this case, the determination unit 152 determines that the deviation between the branching direction of the branching point obtained from the second map information 62 and the direction of the camera road dividing line CL is less than a threshold, and the mode determination unit 150 continues the driving mode in mode B using the map road dividing line ML as a reference line. In other words, if the deviation between the branching direction of the branching point obtained from the second map information 62 and the direction of the camera road dividing line CL is less than the threshold, this indicates the possibility of erroneous recognition in which the camera road dividing line CL deviates toward the branching road ML, and therefore continues the driving mode in mode B using the map road dividing line ML. As a result, the vehicle M can travel straight along the lane L1 along the map road dividing line ML without being guided by the camera road dividing line CL that is leaning towards the branch road ML.

Alternatively, if the deviation between the direction of the branch road ML and the direction of the camera road dividing line CL is less than a threshold, the mode determination unit 150 may notify the occupant of the vehicle M and change the driving mode from mode B to mode C. For example, if the deviation between the direction of the branch road ML and the direction of the camera road dividing line CL is less than a threshold, the mode determination unit 150 may notify the occupant of the vehicle M of information indicating the possibility of misrecognition of the branch road ML and recommend that the occupant change to mode C, or may change to mode C a certain period of time after the deviation determination.

Next, the flow of operations executed by the vehicle control device according to the embodiment will be described with reference to Figure 6. Figure 6 is a flowchart showing an example of the flow of operations executed by the vehicle control device according to the embodiment. The processing according to this flowchart is executed in a predetermined cycle while the host vehicle M is traveling in driving mode B using camera road-dividing lines CL.

First, the mode determination unit 150 acquires the camera road dividing lines CL and the map road dividing lines ML via the recognition unit 130 (step S100). Next, the determination unit 152 determines whether there is a discrepancy between the acquired camera road dividing lines CL and the map road dividing lines ML (step S102).

Next, if the determination unit 152 determines that there is a discrepancy between the acquired camera road dividing lines CL and the map road dividing lines ML, it determines whether the vehicle M is at a branch point based on the second map information 62 (step S104). If it determines that there is no discrepancy between the acquired camera road dividing lines CL and the map road dividing lines ML, or if it determines that the vehicle M is not at a branch point, the mode determination unit 150 returns the process to step S100.

On the other hand, if it is determined that the vehicle M is at a junction, the determination unit 152 then determines whether the deviation between the direction of the camera road dividing line CL and the branching direction of the junction is equal to or greater than a threshold (step S106). If it is determined that the deviation between the direction of the camera road dividing line CL and the branching direction of the junction is less than the threshold, the mode determination unit 150 determines to continue driving in mode B using the map road dividing line ML as a reference line (step S108). On the other hand, if it is determined that the deviation between the direction of the camera road dividing line CL and the branching direction of the junction is equal to or greater than the threshold, the mode determination unit 150 changes the driving mode from mode B to mode C (step S110). At this time, the mode determination unit 150 temporarily discards the map road dividing line ML and determines to continue driving in mode C using the camera road dividing line CL as a reference line. This ends the processing of this flowchart.

According to the present embodiment described above, when a deviation occurs between the camera road dividing lines and the map road dividing lines and the vehicle is at a junction, the driving mode of the vehicle is controlled based on the direction of the camera road dividing lines and the branching direction of the junction. This allows the vehicle's driving control to be appropriately changed even if the road dividing lines recognized by the camera differ from the content of the map information installed in the vehicle, or even if the road dividing lines recognized by the camera differ from the content of the map information installed in the vehicle.

The above-described embodiment can be expressed as follows.
a storage device storing a program;
a hardware processor;
The processor executes the computer-readable instructions to:
Acquire camera images of the vehicle's surroundings,
Controlling the steering and acceleration/deceleration of the vehicle based on the camera image and map information without relying on the operation of the driver of the vehicle;
determining a driving mode of the vehicle to one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task imposed on the driver is lighter than that imposed on the driver in the first driving mode, and at least a part of the plurality of driving modes including the second driving mode being performed by controlling the steering and acceleration/deceleration of the vehicle without relying on an operation by the driver of the vehicle, and changing the driving mode of the vehicle to a driving mode in which the task is heavier when the driver does not perform a task related to the determined driving mode;
determining whether or not there is a discrepancy between the road dividing line shown in the camera image and the road dividing line shown in the map information, and determining whether or not the vehicle is at a branch point shown in the map information;
when it is determined that there is a discrepancy between the road dividing line shown in the camera image and the road dividing line shown in the map information, and when it is determined that the vehicle is at a branch point shown in the map information, determining a driving mode of the vehicle based on a branch direction of the branch point and a direction of the road dividing line shown in the camera image;
The vehicle control device is configured as follows.

The above describes the form for carrying out the present invention using an embodiment, but the present invention is in no way limited to such an embodiment, and various modifications and substitutions can be made without departing from the spirit of the present invention.

10 Camera 12 Radar device 14 LIDAR
16 Object recognition device 100 Automatic driving control device 120 First control unit 130 Recognition unit 140 Action plan generation unit 150 Mode determination unit 152 Determination unit 160 Second control unit

Claims (6)

an acquisition unit that acquires a camera image of the surroundings of the vehicle;
a driving control unit that controls the steering and acceleration/deceleration of the vehicle based on the camera image and map information without relying on an operation by a driver of the vehicle;
a mode determination unit that determines a driving mode of the vehicle to be one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task imposed on the driver is lighter than that imposed on the driver in the first driving mode, at least some of the plurality of driving modes including the second driving mode being controlled by the driving control unit, and that changes the driving mode of the vehicle to a driving mode in which the task is heavier when the driver does not perform a task related to the determined driving mode;
a determination unit that determines whether or not there is a discrepancy between a road dividing line shown in the camera image and a road dividing line shown in the map information, and that determines whether or not the vehicle is at a branch point shown in the map information,
when it is determined that there is a deviation between the road dividing line shown in the camera image and the road dividing line shown in the map information and the vehicle is determined to be at a branching point shown in the map information, the mode determination unit determines a driving mode of the vehicle based on a branching direction of the branching point and a direction of the road dividing line shown in the camera image.
Vehicle control device. the mode determination unit, when a deviation between a branching direction at the branching point and a direction of a road dividing line shown in the camera image is equal to or greater than a threshold, changes the second driving mode to the first driving mode and continues the first driving mode using the road dividing line shown in the camera image;
The vehicle control device according to claim 1 . the mode determination unit continues the second driving mode using the road dividing line shown in the map information when a difference between a branch direction at the branch point and a direction of a road dividing line shown in the camera image is less than a threshold value;
The vehicle control device according to claim 1 or 2. the second driving mode is a driving mode in which the driver is not required to hold an operator that receives a steering operation of the vehicle;
The first driving mode is a driving mode in which the driver is assigned at least a task of gripping the operating element.
The vehicle control device according to any one of claims 1 to 3. The computer
Acquire camera images of the vehicle's surroundings,
Controlling the steering and acceleration/deceleration of the vehicle based on the camera image and map information without relying on the operation of the driver of the vehicle;
determining a driving mode of the vehicle to one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task imposed on the driver is lighter than that imposed on the driver in the first driving mode, and at least a part of the plurality of driving modes including the second driving mode being performed by controlling the steering and acceleration/deceleration of the vehicle without relying on an operation by the driver of the vehicle, and changing the driving mode of the vehicle to a driving mode in which the task is heavier when the driver does not perform a task related to the determined driving mode;
determining whether or not there is a discrepancy between the road dividing line shown in the camera image and the road dividing line shown in the map information, and determining whether or not the vehicle is at a branch point shown in the map information;
when it is determined that there is a discrepancy between the road dividing line shown in the camera image and the road dividing line shown in the map information, and when it is determined that the vehicle is at a branch point shown in the map information, determining a driving mode of the vehicle based on a branch direction of the branch point and a direction of the road dividing line shown in the camera image;
Vehicle control method. On the computer,
Acquire a camera image capturing the surroundings of the vehicle,
and controlling the steering and acceleration/deceleration of the vehicle based on the camera image and map information without relying on the operation of the driver of the vehicle;
determining a driving mode of the vehicle to one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task imposed on the driver is lighter than that of the first driving mode, and at least a part of the plurality of driving modes including the second driving mode being performed by controlling the steering and acceleration/deceleration of the vehicle without relying on an operation by the driver of the vehicle, and changing the driving mode of the vehicle to a driving mode in which the task is heavier when the driver does not perform a task related to the determined driving mode;
determining whether or not there is a discrepancy between a road dividing line shown in the camera image and a road dividing line shown in the map information, and determining whether or not the vehicle is at a branch point shown in the map information;
when it is determined that there is a discrepancy between the road dividing line shown in the camera image and the road dividing line shown in the map information, and when it is determined that the vehicle is at a branch point shown in the map information, a driving mode of the vehicle is determined based on a branch direction of the branch point and a direction of the road dividing line shown in the camera image.
program.