JP7780995B2 - Vehicle control device, vehicle control method, and program - Google Patents

Description

The present invention relates to a vehicle control device, a vehicle control method, and a program.

In recent years, there has been considerable research into autonomous driving, which recognizes the vehicle's surroundings and automatically controls its driving. Related technologies include detecting information about the driving environment by recognizing the state of road markings, determining control content by estimating the lane markings that separate the current driving lane from line types pre-stored in a memory unit, and determining the vehicle's position using camera recognition results and map information (see, for example, Patent Documents 1 to 4).

Japanese Patent Application Laid-Open No. 2000-105898 JP 2010-221859 A Japanese Patent Application Publication No. 10-300494 JP 2013-032953 A

However, with conventional technology, it was sometimes impossible to determine from images captured by a camera whether the road on which a vehicle was traveling was a two-way road. As a result, even if lane markings could be recognized from the image, the lane the vehicle was traveling in could not be identified or could be identified incorrectly.

The present invention has been made in consideration of these circumstances, and one of its objectives is to provide a vehicle control device, vehicle control method, and program that can more accurately identify the vehicle's travel lane, even when the road on which the vehicle is traveling is a two-way traffic section.

A vehicle control device, a vehicle control method, and a program according to the present invention employ the following configuration.
(1): A vehicle control device according to one embodiment of the present invention includes a recognition unit that recognizes the surrounding conditions of a vehicle; a driving control unit that controls one or both of the steering and speed of the vehicle based on the surrounding conditions recognized by the recognition unit; an acquisition unit that acquires map information including lane information around the vehicle and reference information for identifying the position of the vehicle; and an identification unit that identifies the driving lane of the vehicle based on information about the road on which the vehicle is traveling, acquired from the map information based on the reference information, and the type of road dividing lines that divide one or more lanes around the vehicle recognized by the recognition unit.When it is determined that the road is a two-way traffic section based on at least one of the road information and the type of the road dividing lines, the identification unit identifies the driving lane from the recognition results of the surroundings of the vehicle recognized by the recognition unit, without using at least information about the line type of the two-way traffic dividing lines.

(2): In the aspect (1) above, when the identification unit acquires, based on the map information, that the road is a two-way traffic section with multiple parallel lanes, it identifies the driving lane from the recognition results of the surroundings of the vehicle recognized by the recognition unit without using information on the line type of the two-way traffic lane markings.

(3): In the aspect (1) above, when the road is a two-way traffic section with one lane in each direction, the identification unit identifies the vehicle's travel lane as one lane in each direction if the types of road dividing lines on the left and right of the vehicle are solid lines, dashed lines, special lines, or shoulders.

(4): In any one of the above aspects (1) to (3), the identification unit identifies the vehicle as traveling in the rightmost lane of a two-lane road if the road is a two-lane road with two-way traffic, the road dividing line on the left side of the vehicle is a broken line or a special line, and the dividing line further back than the left road dividing line as seen from the vehicle is a solid line.

(5): In any one of the above aspects (1) to (4), when the road is a two-way traffic section with three or more lanes on each side, and the road dividing line on the right side of the vehicle is a yellow lane, and the road dividing line on the left side of the vehicle is a dashed line or a special line, the identification unit identifies the vehicle's traveling lane as the rightmost lane of the road with three or more lanes on each side.

(6): In any one of the above aspects (1) to (5), the driving control unit drives the vehicle by executing one of a plurality of driving modes including at least a first driving mode and a second driving mode in which a task imposed on the vehicle occupant is more severe than that in the first driving mode, and executes the first driving mode when the identification unit identifies the vehicle's driving lane, and executes the second driving mode when the identification unit does not identify the vehicle's driving lane.

(7): In any one of the above aspects (1) to (5), the driving control unit drives the vehicle by executing one of a plurality of driving modes that assign different tasks to the vehicle's occupants, and if the identification unit does not identify the vehicle's driving lane, the driving mode currently being executed is continued to drive the vehicle.

(8): A vehicle control method according to one aspect of the present invention is a vehicle control method in which a computer recognizes the surrounding conditions of a vehicle, executes driving control to control one or both of the steering and speed of the vehicle based on the recognized surrounding conditions, acquires map information including lane information around the vehicle and reference information for identifying the position of the vehicle, identifies the vehicle's traveling lane based on information about the road on which the vehicle is traveling, acquired from the map information based on the reference information, and the recognized types of road dividing lines that define one or more lanes around the vehicle, and, when the road is determined to be a two-way traffic section based on at least one of the road information and the type of road dividing lines, identifies the traveling lane from the recognition results around the recognized vehicle without using at least information about the line type of the two-way traffic dividing lines.

(9): A program according to one aspect of the present invention causes a computer to recognize the conditions surrounding a vehicle, execute driving control to control one or both of the steering and speed of the vehicle based on the recognized conditions, acquire map information including lane information around the vehicle and reference information for identifying the position of the vehicle, identify the vehicle's travel lane based on information about the road on which the vehicle is traveling, acquired from the map information based on the reference information, and the type of road dividing lines that divide one or more recognized lanes around the vehicle, and, when the road is determined to be a two-way traffic section based on at least one of the road information and the type of road dividing lines, identify the travel lane from the recognition results around the recognized vehicle without using at least information about the line type of the two-way traffic dividing lines.

According to aspects (1) to (9) above, the vehicle's travel lane can be identified more accurately, even if the road on which the vehicle is traveling is a two-way traffic section.

1 is a configuration diagram of a vehicle system 1 using a vehicle control device according to an embodiment. FIG. 2 is a functional configuration diagram of a first control unit 120 and a second control unit 160 according to the embodiment. A diagram showing an example of the relationship between driving modes, control states of vehicle M, and tasks. FIG. 10 is a diagram showing an example of the contents of a judgment table 182. FIG. 10 is a diagram illustrating an example of a first determination pattern. FIG. 10 is a diagram illustrating an example of a second determination pattern. FIG. 10 is a diagram illustrating an example of a third determination pattern. FIG. 10 is a diagram illustrating an example of a fourth determination pattern. FIG. 10 is a diagram illustrating an example of a fifth determination pattern. FIG. 10 is a diagram illustrating an example of a sixth determination pattern. FIG. 10 is a diagram illustrating an example of a seventh determination pattern. 4 is a flowchart showing an example of the flow of a driving control process executed by the automatic driving control device 100.

Embodiments of a vehicle control device, vehicle control method, and program of the present invention will be described below with reference to the drawings. Note that the following describes a case where laws stipulating left-hand traffic apply, but if laws stipulating right-hand traffic apply, the terms left and right can be reversed.

[Overall configuration]
FIG. 1 is a configuration diagram of a vehicle system 1 using a vehicle control device according to an embodiment. The vehicle (hereinafter, vehicle M) on which the vehicle system 1 is mounted may be, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source may be an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using power generated by a generator connected to the internal combustion engine or discharged power from a secondary battery or a fuel cell. In the following, an embodiment in which the vehicle control device is applied to an autonomous vehicle will be described as an example. Autonomous driving refers to automatically controlling one or both of the steering and speed of the vehicle M to perform driving control. The driving control of the vehicle M may include various driving assistance systems, such as adaptive cruise control (ACC), auto lane changing (ALC), and lane keeping assistance system (LKAS). Some or all of the driving of an autonomous vehicle may be manually controlled by an occupant (driver).

The vehicle system 1 includes, for example, a camera (an example of an imaging unit) 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, vehicle sensors 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driver monitor camera 70, driving controls 80, an autonomous driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are connected to each other via multiplexed communication lines such as a Controller Area Network (CAN) communication line, serial communication lines, a wireless communication network, etc. Note that the configuration shown in FIG. 1 is merely an example; some of the components may be omitted, or additional components may be added. The combination of the camera 10, radar device 12, and LIDAR 14 is an example of an "external environment sensor ES." The external sensor ES may include other detection units (e.g., sonar) that recognize the vehicle's surroundings, or may include an object recognition device 16. The HMI 30 is an example of an "output device." The autonomous driving control device 100 is an example of a "vehicle control device."

The camera 10 is a digital camera that uses a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 can be mounted anywhere on the vehicle M. For example, when capturing images of the area in front of the vehicle M, the camera 10 can be mounted on the top of the front windshield or the back of the rearview mirror. When capturing images of the area behind the vehicle M, the camera 10 can be mounted on the top of the rear windshield or the back door. When capturing images of the sides and rear of the vehicle M, the camera 10 can be mounted on a door mirror. The camera 10 periodically captures images of the area around the vehicle M. The camera 10 may be a stereo camera. The camera 10 may also be provided with multiple cameras (e.g., a first camera and a second camera), and multiple cameras may capture images in the same direction. Alternatively, the camera 10 may normally capture images using the first camera, and when certain conditions are met, the camera 10 may capture images using the second camera, or both the first and second cameras. The predetermined condition is, for example, when road dividing lines (hereinafter referred to as dividing lines) that divide lanes, etc., included in the road on which the vehicle M is traveling are recognized from an image captured by a camera (hereinafter referred to as a camera image). The dividing lines may include, for example, line segment information other than dividing lines that divide lanes.

The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by objects (reflected waves) to detect at least the object's position (distance and direction). The radar device 12 may be mounted at any location on the vehicle M. The radar device 12 may also detect the object's position and speed using the FM-CW (Frequency Modulated Continuous Wave) method.

The LIDAR 14 irradiates the area around the vehicle M with light (or electromagnetic waves with wavelengths similar to light) and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time between light emission and light reception. The irradiated light is, for example, pulsed laser light. The LIDAR 14 can be attached to any location on the vehicle M.

The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the camera 10, radar device 12, and LIDAR 14 included in the external sensor ES to recognize the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition results to the autonomous driving control device 100. The object recognition device 16 may output the detection results from the camera 10, radar device 12, and LIDAR 14 directly to the autonomous driving control device 100. The object recognition device 16 may be omitted from the vehicle system 1.

The communication device 20 communicates with other vehicles in the vicinity of vehicle M using, for example, a cellular network, a Wi-Fi network, Bluetooth (registered trademark), or DSRC (Dedicated Short Range Communication), or with various server devices via a wireless base station.

The HMI 30 presents various information to the occupants of the vehicle M under the control of the HMI control unit 170 and accepts input operations by the occupants. The HMI 30 includes, for example, various display devices, speakers, switches, a microphone, a buzzer, a touch panel, keys, etc. The various display devices are, for example, LCD (Liquid Crystal Display) and organic EL (Electro Luminescence) display devices. The display device is, for example, located near the front of the driver's seat (the seat closest to the steering wheel) on the instrument panel, in a position where the occupant can see it through the gap in the steering wheel or over the steering wheel. The display device may also be located in the center of the instrument panel. The display device may also be a HUD (Head Up Display). The HUD projects an image onto a portion of the front windshield in front of the driver's seat, allowing the occupant sitting in the driver's seat to see a virtual image. The display device displays an image generated by the HMI control unit 170, which will be described later. The HMI 30 may also include a driving selector switch that switches between automatic driving and manual driving by the occupant. The switch may include, for example, a turn signal switch (directional indicator) 32. The turn signal switch 32 is provided, for example, on the steering column or steering wheel. The turn signal switch 32 is an example of an operating unit that accepts, for example, an instruction from the occupant to change lanes of the vehicle M.

The vehicle sensors 40 include a vehicle speed sensor that detects the speed of the vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity around a vertical axis, and a direction sensor that detects the orientation of the vehicle M. The vehicle sensors 40 may also include a steering angle sensor that detects the steering angle of the vehicle M (which may be the angle of the steering wheels or the angle of the steering wheel). The vehicle sensors 40 may also include a position sensor that acquires the position of the vehicle M. The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. The position sensor may also be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver 51 of the navigation device 50.

The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as a hard disk drive (HDD) or flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be determined or supplemented by an inertial navigation system (INS) that uses the output of the vehicle sensors 40. The navigation HMI 52 includes a display device, speaker, touch panel, keys, etc. The navigation HMI 52 may share some or all of the components with the HMI 30 described above. The route determination unit 53 determines a route (hereinafter, a map route) from the position of the vehicle M determined by the GNSS receiver 51 (or any input position) to a destination input by the occupant using the navigation HMI 52, for example, by referring to the first map information 54.

The first map information 54 includes, for example, information with lane information added to each road (hereinafter referred to as lane information). Lane information includes, for example, nodes indicating the start and end of road sections and links representing the road shape between the nodes. Furthermore, lane information may include the number of lanes (number of parallel lanes), the number of lanes to be added or removed, and the direction of lane addition or removal (information indicating whether lanes are added or removed on the left or right side of the road's direction of travel) in a given section, such as a road section. Furthermore, lane information may include, for example, information regarding the type of lane markings (e.g., solid lines, dashed lines, shoulder markings) and their colors. The first map information 54 may also include information indicating whether the road is a two-way traffic section. A two-way traffic section is, for example, a road section on which two-way traffic is carried out, where the lanes for each direction of travel are not clearly separated by a central divider but by markings, lines similar to markings, or objects (e.g., poles or road studs), and the road structure does not clearly separate the two directions of travel. The first map information 54 may also include information such as the distance and curvature of road sections, road type (e.g., expressway, general road), and POI (Point of Interest) information. The route on the map is output to the MPU 60.

The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be implemented, for example, by the functions of a terminal device such as a smartphone or tablet device owned by the occupant. The navigation device 50 may transmit the current position and destination to a navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server. Note that the first map information 54 may be stored in the memory unit 180 instead of in the navigation device 50.

The MPU 60 includes, for example, a recommended lane determination unit 61. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, every 100 m in the vehicle's travel direction) and determines a recommended lane for each block from the lane information in the first map information 54. The recommended lane determination unit 61 may also determine a recommended lane for each road stored in the first map information 54. For example, the recommended lane determination unit 61 determines which lane from the left (or right) to use. When there is a branch point on the route on the map, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel along a reasonable route to the branch point.

The driver monitor camera 70 is, for example, a digital camera that uses a solid-state imaging element such as a CCD or CMOS. The driver monitor camera 70 is mounted at any location on the vehicle M in a position and orientation that allows it to capture an image of the head of an occupant (hereinafter referred to as the driver) seated in the driver's seat of the vehicle M from the front (in an orientation that captures the face). For example, the driver monitor camera 70 is mounted above a display device provided in the center of the instrument panel of the vehicle M.

The driving operators 80 include, for example, a steering wheel 82, as well as an accelerator pedal, brake pedal, shift lever, and other operators. The driving operators 80 are equipped with sensors that detect the amount of operation or the presence or absence of operation, and the detection results are output to the automatic driving control device 100 or some or all of the driving force output device 200, brake device 210, and steering device 220. The steering wheel 82 is an example of an "operator that accepts steering operation by the driver." The operator does not necessarily have to be annular and may be in the form of an irregular steering wheel, joystick, button, etc. A steering grip sensor 84 is attached to the steering wheel 82. The steering grip sensor 84 is realized by a capacitance sensor or the like, and outputs a signal to the automatic driving control device 100 that can detect whether the driver is gripping the steering wheel 82 (meaning that the driver is in contact with the steering wheel in a state where force can be applied).

The autonomous driving control device 100 includes, for example, a first control unit 120, a second control unit 160, an HMI control unit 170, and a memory unit 180. The first control unit 120, the second control unit 160, and the HMI control unit 170 are each realized by a hardware processor, such as a CPU (Central Processing Unit), executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as the HDD or flash memory of the autonomous driving control device 100, or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the autonomous driving control device 100 by inserting the storage medium (non-transitory storage medium) into a drive device. The combination of the action plan generation unit 140 and the second control unit 160 is an example of a "driving control unit." The HMI control unit 170 is an example of an "output control unit."

The memory unit 180 may be realized by the various storage devices mentioned above, or alternatively, a solid state drive (SSD), an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), or a random access memory (RAM). The memory unit 180 stores, for example, a determination table 182, programs, and various other information. The determination table 182 is, for example, a table referenced to identify the lane in which the vehicle M is traveling. The determination table 182 will be described later. The memory unit 180 may also store first map information 54.

Figure 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160 according to the embodiment. The first control unit 120 includes, for example, a recognition unit 130, an action plan generation unit 140, and a mode determination unit 150. The first control unit 120, for example, implements functions based on AI (Artificial Intelligence) and functions based on a pre-specified model in parallel. For example, the "intersection recognition" function may be implemented by executing intersection recognition using deep learning or the like and recognition based on pre-specified conditions (such as traffic lights and road markings that can be pattern-matched) in parallel, and then scoring and comprehensively evaluating both. This ensures the reliability of autonomous driving.

The recognition unit 130 recognizes the position, speed, acceleration, and other status of objects around the vehicle M based on information input from the external sensor ES. The position of an object is recognized as a position on an absolute coordinate system with a representative point of the vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or by an area. The "state" of an object may include the acceleration or jerk of the object, or its "behavioral state" (for example, whether or not the object is changing lanes or is about to change lanes).

The recognition unit 130 also recognizes, for example, the lane in which the vehicle M is traveling (driving lane). For example, the recognition unit 130 recognizes the dividing lines on the left and right of the vehicle M from the camera image captured by the camera 10, and recognizes the driving lane based on the position of the recognized dividing lines. Note that the recognition unit 130 may recognize the driving lane by recognizing landmarks (road boundaries, road boundaries) that can identify the lane position, including not only dividing lines but also shoulders, curbs, medians, guardrails, fences, walls, etc. This recognition may take into account the position of the vehicle M obtained from the navigation device 50 and the processing results from the INS. The recognition unit 130 may also recognize stop lines, obstacles, red lights, toll booths, and other road phenomena.

When recognizing the driving lane, the recognition unit 130 recognizes the position and orientation of vehicle M relative to the driving lane. For example, the recognition unit 130 may recognize the deviation of vehicle M's reference point from the center of the lane and the angle it makes with a line connecting the centers of the lanes in the vehicle M's direction of travel as the relative position and orientation of vehicle M relative to the driving lane. Alternatively, the recognition unit 130 may recognize the position of vehicle M's reference point relative to one of the side edges of the driving lane (a dividing line or road boundary) as the relative position of vehicle M relative to the driving lane. Note that recognition of the driving lane and the position and orientation of vehicle M relative to the driving lane by the recognition unit 130 may be performed by the identification unit 153, which will be described later.

The behavior plan generation unit 140 essentially drives the recommended lane determined by the recommended lane determination unit 61, and further generates a target trajectory for vehicle M to travel automatically (without driver input) in the future so that it can respond to the conditions around vehicle M. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequence of points (trajectory points) that vehicle M should reach. Trajectory points are points that vehicle M should reach at every predetermined travel distance (e.g., several meters) along the road. Separately, target speeds and target accelerations are generated as part of the target trajectory for every predetermined sampling time (e.g., several tenths of a second). Furthermore, trajectory points may be positions that vehicle M should reach at each predetermined sampling time. In this case, information on target speed and target acceleration is expressed as the interval between trajectory points.

When generating a target trajectory, the behavior plan generation unit 140 may set an autonomous driving event (function). Autonomous driving events include a constant speed driving event, a low-speed following driving event, a lane change event, a branching event, a merging event, and a takeover event. The behavior plan generation unit 140 generates a target trajectory according to the activated event.

The mode determination unit 150 determines the driving mode of the vehicle M to be one of multiple driving modes that impose different tasks on the driver (in other words, multiple modes with different degrees of automation). The mode determination unit 150 includes, for example, a driver state determination unit 151, a first acquisition unit 152, an identification unit 153, and a mode change processing unit 154. The individual functions of these units will be described later. The first acquisition unit 152 is an example of an "acquisition unit."

Figure 3 is a diagram showing an example of the relationship between driving modes, the control state of vehicle M, and tasks. In the example of Figure 3, vehicle M has five driving modes, for example, Mode A to Mode E. In Figure 3, Modes A and B are examples of "first driving modes," and Modes C, D, and E are examples of "second driving modes." Note that there may be driving modes other than Modes A to E, and there may be driving modes other than the first and second driving modes. Among Modes A to E, the control state, i.e., the degree of automation (control degree) of driving control of vehicle M, is highest in Mode A, followed by Mode B, Mode C, and Mode D, with Mode E being the lowest. Conversely, the tasks imposed on the driver (occupant) are lightest in Mode A, followed by Mode B, Mode C, and Mode D, with Mode E, which involves manual driving, being the most severe. Note that modes B to E are not autonomous driving, and the autonomous driving control device 100 is responsible for ending autonomous driving control and transitioning to driving assistance or manual driving. The contents of each mode are described below as examples.

In Mode A, the vehicle is in an autonomous driving state, and the driver is not required to monitor the surroundings of the vehicle M or grip the steering wheel 82 (hereinafter referred to as "steering grip"). Surroundings monitoring includes monitoring at least the direction of travel of the vehicle M (e.g., forward). "Forward" refers to the space in the direction of travel of the vehicle M as seen through the front windshield. However, even in Mode A, the driver is required to be ready to quickly transition to manual driving in response to a request from the system centered on the autonomous driving control device 100. Note that autonomous driving here refers to control of both the steering and speed of the vehicle M independently of the driver's operation. Mode A is a driving mode that can be implemented, for example, on a highway or other expressway when certain conditions are met, such as when the vehicle M is traveling at a predetermined speed (e.g., approximately 50 km/h) or less and there is a vehicle ahead to be followed. Mode A is sometimes referred to as Traffic Jam Pilot (TJP) mode. If these conditions are no longer met, the mode determination unit 150 changes the driving mode of the vehicle M to Mode B.

Furthermore, while mode A is being executed, the occupant can perform a second task. A second task is, for example, an action other than driving that is permitted by the occupant while vehicle M is in autonomous driving. Second tasks include, for example, watching television, using a terminal device (e.g., a smartphone or tablet) carried by the occupant (e.g., making calls, sending and receiving emails, using SNS (Social Networking Service), browsing the web, etc.), eating, etc.

In mode B, the system is in a driving assistance state, and the driver is tasked with monitoring the surroundings of vehicle M (hereinafter referred to as "peripheral monitoring"), but is not tasked with gripping the steering wheel 82. For example, in mode B, lane change instructions from the occupant are not accepted, and lane changes of vehicle M are performed at the discretion of the vehicle system 1 based on route settings to the destination by the navigation device 50, etc. A lane change refers to moving vehicle M from the lane in which vehicle M is traveling to an adjacent lane adjacent to the lane in which vehicle M is traveling, and may include lane changes due to branching or merging. The main driver in modes A and B is vehicle system 1.

In Mode C, the system enters a driving assistance state, and the driver is tasked with monitoring the surroundings and holding the steering wheel 82. For example, in Mode C, if the vehicle system 1 determines that vehicle M needs to change lanes, it queries the occupant via the HMI 30, and if approval for the lane change from the occupant is received from the HMI 30, driving assistance is provided to execute the lane change. Lane change control in Modes B and C is system-driven lane change.

Mode D is a driving mode that requires some degree of driver operation for at least one of steering and acceleration/deceleration of vehicle M. For example, in Mode D, driving assistance such as ACC (Adaptive Cruise Control) and LKAS (Lane Keeping Assist System) is provided. Also, in Mode D, when an instruction to change lanes for vehicle M is received through the driver's operation of the turn signal switch 32, driving assistance is provided to execute the lane change in the instructed direction. A lane change in Mode D is a lane change intentionally made by the driver. The driver's operation of the turn signal switch 32 is an example of a driving operation. Also, driving operations in Mode D may include driving operations to control steering or acceleration/deceleration.

In Mode E, vehicle M enters a manual driving state, requiring the driver to perform steering, acceleration, and deceleration. In both Mode D and Mode E, the driver is naturally tasked with monitoring the surroundings of vehicle M. In Modes C to E, the driver is the primary driver.

If the driver does not perform a task related to the determined driving mode, the mode determination unit 150 changes the driving mode of the vehicle M to a driving mode with a more severe task.

For example, if the driver is unable to transition to manual driving in response to a request from the system while mode A is in progress (for example, if the driver continues to look away from the road outside the permitted area or if signs of driving difficulty are detected), the mode determination unit 150 causes the HMI control unit 170 to execute control using the HMI 30 to prompt the driver to transition to manual driving in mode E. Furthermore, if the driver does not respond within a predetermined time after the mode determination unit 150 has executed control to prompt the driver to transition to manual driving, or if the mode determination unit 150 determines that the driver is not in a state to perform manual driving, the mode determination unit 150 performs control such as gradually decelerating the vehicle M while pulling it toward a target position (for example, the shoulder of the road) and stopping the autonomous driving. Furthermore, after autonomous driving is stopped, the vehicle M enters mode D or E, and the driver can manually start the vehicle M. The same applies below to "stopping autonomous driving."

In mode B, if the driver is not monitoring the road ahead, the mode determination unit 150 uses the HMI 30 to prompt the driver to monitor the road ahead, and if the driver does not comply, the mode determination unit 150 controls the vehicle M to approach the target position and gradually stop, and the autonomous driving is stopped. In mode C, if the driver is not monitoring the road ahead or is not gripping the steering wheel 82, the mode determination unit 150 uses the HMI 30 to prompt the driver to monitor the road ahead and/or grip the steering wheel 82, and if the driver does not comply, the mode determination unit 150 controls the vehicle M to approach the target position and gradually stop, and the autonomous driving is stopped.

The driver state determination unit 151 determines whether the occupant (driver) is in a state suitable for driving. For example, the driver state determination unit 151 monitors the driver's state for the above-mentioned mode change and determines whether the driver's state is suitable for the task. For example, the driver state determination unit 151 analyzes images captured by the driver monitor camera 70 and performs posture estimation processing to determine whether the driver is in a position that prevents him or her from switching to manual driving in response to a request from the system. The driver state determination unit 151 also analyzes images captured by the driver monitor camera 70 and performs gaze estimation processing to determine whether the driver is monitoring the surroundings of the vehicle M (more specifically, the front). If the driver state determination unit 151 determines that the driver is not in a state suitable for the task for a predetermined period of time or longer, it determines that the driver is in an unsuitable state for driving the task. If the driver state determination unit 151 determines that the driver is in a state suitable for driving the task, it determines that the driver is in a suitable state for driving the task. The driver state determination unit 151 may also determine whether the occupant is in a state where they can take over driving.

The first acquisition unit 152 acquires first map information 54. The first acquisition unit 152 also acquires reference information for identifying the position of the vehicle M. The reference information is, for example, position information of the vehicle M detected by the vehicle sensor 40 or a camera image captured by the camera 10. The reference information may also include some or all of the recognition results obtained by the recognition unit 130.

The identification unit 153 refers to the first map information 54 based on the position information of vehicle M included in the reference information, and identifies the driving lane of vehicle M from among one or more lanes included in the road on which vehicle M is traveling. Furthermore, the identification unit 153 identifies the driving lane, for example, based on road information acquired from the first map information 54 and the type of lane markings recognized by the recognition unit 130. Furthermore, when the identification unit 153 determines that the road is for oncoming traffic based on at least one of the road information and the lane markings, it identifies the driving lane by adopting special rules (predetermined relaxed conditions). For example, the identification unit 153 identifies the driving lane from the recognition results of the area around vehicle M recognized by the recognition unit 130, without using at least information on the line type of lane markings for oncoming traffic. Oncoming traffic lane marks are, for example, lane markings that define lanes on the road where vehicles travel opposite vehicle M, and are lane markings other than lane markings that define lanes in the same direction as vehicle M's direction of travel. Furthermore, oncoming traffic lane marks do not include lane marks that simultaneously separate the lane in the direction of travel of vehicle M and the lane for oncoming traffic. More specifically, the identification unit 153 identifies the driving lane using, for example, recognized road lane marks excluding oncoming traffic lane marks. Note that oncoming traffic lane marks may be used to identify the position of vehicle M relative to the driving lane, assuming that some line segment is present. The following mainly describes the case where the driving lane is identified using lane marks excluding oncoming traffic lane marks.

The identification unit 153 also identifies the lane in which vehicle M is traveling at a predetermined timing. The predetermined timing may be, for example, a predetermined cycle, or may be when autonomous driving begins, when driving begins on a specific road such as an expressway, or when a road section changes in the first map information 54. The predetermined timing may be, for example, when the number of lanes in the map information and the number of lanes obtained from the camera image do not match, causing the current lane in which vehicle M is traveling to be reset and the lane in which vehicle M needs to be identified again, when the lane in which vehicle M is traveling cannot be identified and there is no change in the road within a predetermined distance, or when the occupant performs an operation to start autonomous driving. The predetermined timing may also be, for example, when the number of lanes on the road on which vehicle M is traveling has increased or decreased, or will increase or decrease in the near future. The function of the identification unit 153 will be described in detail below.

The mode change processing unit 154 determines the driving mode of the vehicle M based on the determination result of the driver state determination unit 151, the determination result of the determination unit 153, etc. The mode change processing unit 154 may also determine whether to continue the current driving mode or switch to another mode. The mode change processing unit 154 also performs various processes for changing to the driving mode determined by the mode determination unit 150. For example, the mode change processing unit 154 may instruct a driving assistance device (not shown) to operate, cause the HMI control unit 170 to output information to prompt the driver to take action to the HMI 30, or instruct the action plan generation unit 140 to generate a target trajectory according to the driving mode.

The second control unit 160 controls the driving force output device 200, the braking device 210, and the steering device 220 so that the vehicle M passes through the target trajectory generated by the action plan generation unit 140 at the scheduled time.

The second control unit 160 includes, for example, a second acquisition unit 162, a speed control unit 164, and a steering control unit 166. The second acquisition unit 162 acquires information about the target trajectory (trajectory points) generated by the action plan generation unit 140 and stores it in memory (not shown). The speed control unit 164 controls the driving force output device 200 or the brake device 210 based on the speed element associated with the target trajectory stored in memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in memory. The processing of the speed control unit 164 and the steering control unit 166 is realized, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road ahead of the vehicle M and feedback control based on the deviation from the target trajectory.

The HMI control unit 170 notifies the occupant of predetermined information via the HMI 30. The predetermined information includes, for example, information related to the driving of the vehicle M, such as information related to the state of the vehicle M and information related to driving control. Information related to the state of the vehicle M includes, for example, the speed of the vehicle M, engine RPM, and shift position. Information related to driving control also includes, for example, an inquiry about whether to change lanes, whether a driving mode is active, information related to changing the driving mode, information assigned to the occupant necessary to switch driving modes (task request information for the occupant), and information related to the status of driving control (for example, the content of an event currently being executed). The predetermined information may also include information unrelated to the driving control of the vehicle M, such as television programs, content (for example, movies) stored on storage media such as DVDs, etc. The predetermined information may also include, for example, information regarding the current location and destination of vehicle M, information regarding the remaining fuel level, information indicating whether the lane vehicle M is traveling in has been identified, the remaining distance until the driving mode is switched, the direction in which lanes are added or removed, the number of lanes added or removed, the number of lanes running parallel to the traveling lane (number of parallel lanes), etc.

For example, the HMI control unit 170 may generate an image including the above-mentioned predetermined information and display the generated image on the display device of the HMI 30, or may generate audio indicating the predetermined information and output the generated audio from the speaker of the HMI 30. The HMI control unit 170 may also output information received by the HMI 30 to the communication device 20, the navigation device 50, the first control unit 120, etc.

The driving force output device 200 outputs driving force (torque) to the drive wheels to propel the vehicle. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, etc., and an ECU (Electronic Control Unit) that controls these. The ECU controls the above components in accordance with information input from the second control unit 160 or information input from the driving operator 80.

The brake device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or information input from the driving operator 80, so that brake torque corresponding to the braking operation is output to each wheel. The brake device 210 may include a backup mechanism that transmits hydraulic pressure generated by operation of the brake pedal included in the driving operator 80 to the cylinder via a master cylinder. Note that the brake device 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic brake device that controls an actuator according to information input from the second control unit 160 to transmit hydraulic pressure from the master cylinder to the cylinder.

The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor applies force to a rack and pinion mechanism to change the direction of the steered wheels. The steering ECU drives the electric motor to change the direction of the steered wheels in accordance with information input from the second control unit 160 or information input from the driving operator 80.

[Functions of specific parts and details of operation control]
Below, the details of the function of the identification unit 153 and the content of driving control based on the processing content of the identification unit 153 will be described. Note that, below, it is assumed that the driver state determination unit 151 has determined that the driver is appropriately performing the task assigned according to the driving mode, and an example will be described in which the driving mode is determined based on the processing content by the identification unit 153. Note that, when the driver state determination unit 151 has determined that the driver is not performing the task assigned according to the mode, the mode determination unit 150 determines to change to a mode corresponding to the task being performed by the driver, or determines to perform control to stop the autonomous driving.

First, the identification unit 153 acquires information about the lane markings around the vehicle M recognized by the recognition unit 130, for example. Specifically, the recognition unit 130 analyzes an image (camera image) including the road on which the vehicle M is traveling, captured by the camera 10 (the first camera, or one or both cameras, if present), extracts edge points in the image that have a large difference in brightness from adjacent pixels, and recognizes landmarks that can identify lane markings or lane positions on the image plane by connecting the edge points. The recognition unit 130 may also extract image information from the image by feature extraction, image enhancement processing, etc., and recognize the lane markings and the landmarks by matching the extracted image information with a predefined pattern matching model, etc. The recognition unit 130 may also recognize type information, such as type (solid line, dashed line) and color, for each lane marking based on the image analysis results, or may recognize the type of landmark. The recognition unit 130 may also recognize the relative positions of the recognized lane markings and landmarks, and the relative positions (relative positions) between the lane markings and landmarks and the vehicle M. The recognition unit 130 may also recognize characters displayed on road signs or characters drawn on the road.

The marking lines around vehicle M recognized by the recognition unit 130 are, for example, the left and right marking lines closest to vehicle M and the left and right marking lines next closest to vehicle M after the left and right marking lines. Hereinafter, the closest marking line on the left side of vehicle M will be referred to as the "left-side first marking line," and the next closest marking line (the marking line located further back from vehicle M than the left-side first marking line) will be referred to as the "left-side second marking line." Furthermore, the closest marking line on the right side of vehicle M will be referred to as the "right-side first marking line," and the next closest marking line (the marking line located further back from vehicle M than the right-side first marking line) will be referred to as the "right-side second marking line."

The identification unit 153 refers to the first map information 54 based on the location information included in the reference information acquired by the first acquisition unit 152, and acquires information about the road on which vehicle M is traveling. Road information is, for example, information about the number of lanes. The road information may also include information about the number of lanes on each side, information indicating whether there is two-way traffic, and information indicating whether the road on which vehicle M is traveling is a deceleration zone.

Next, the identification unit 153 identifies the lane vehicle M is traveling in based on information about the road vehicle M is traveling on, obtained from the first map information 54, and information about the type of dividing lines around vehicle M recognized by the recognition unit 130. For example, if the road vehicle M is traveling on is a two-way traffic section with one lane in each direction, and the type of dividing lines on the left and right of vehicle M is a solid line, a dashed line, a special line, or a shoulder dividing line, the identification unit 153 identifies the lane vehicle M is traveling in as one lane in each direction.

Furthermore, when the road on which vehicle M is traveling, obtained from the first map information 54, is a two-way traffic section and has multiple parallel lanes (two or more lanes that can be traveled in the same direction), the identification unit 153 identifies the traveling lane using the markings in the recognition results around vehicle M recognized by the recognition unit 130, excluding the markings for oncoming traffic. For example, when the road on which vehicle M is traveling is a two-lane two-way traffic section and the marking on the left side of vehicle M is a dashed or special line, the identification unit 153 identifies the traveling lane of vehicle M as the rightmost lane of the two-lane section (the second lane from the left). Furthermore, when the road on which vehicle M is traveling is a two-lane two-way traffic section with three or more lanes on each side and the marking on the right side of vehicle M is a yellow lane and the marking on the left side of vehicle M is a dashed or special line, the identification unit 153 identifies the traveling lane of vehicle M as the rightmost lane of a three or more lanes section.

When identifying the driving lane as described above, the identification unit 153 may refer to a judgment table 182 stored in advance in the memory unit 180, and identify the driving lane of the vehicle M if the relevant conditions are met.

Figure 4 shows an example of the contents of the judgment table 182. The judgment table 182 is information in which judgment results are associated with judgment conditions and lane marking conditions. The example of the judgment table 182 shown in Figure 4 also includes a condition ID as identification information for identifying each judgment condition. The judgment start condition is a condition under which the identification unit 153 identifies the driving lane. "Always" includes the meaning of a predetermined period. The lane marking conditions include conditions for the line type of the two lane markings on each side of the vehicle M (left-side first lane marking, left-side second lane marking, right-side first lane marking, and right-side second lane marking). The "-" included in the lane marking conditions in Figure 4 indicates that the lane markings can be of any type (including cases where they are not detected). The judgment result identifies the lane in which the vehicle M is traveling when the conditions set forth in the judgment start condition and lane marking conditions are met.

The following explains the determination results (identification of the driving lane) according to the combination of the determination start condition and the lane marking information for each determination pattern (condition ID) in accordance with Figure 4. The following also explains cases where it has not been determined from the first map information 54 that the road on which vehicle M is traveling is a two-way traffic section (or it has been determined that it is not a two-way traffic section) (determination patterns 1 to 4), and cases where it has been determined (determination patterns 5 to 7).

<First Determination Pattern>
Figure 5 is a diagram showing an example of a first determination pattern. In the example of Figure 5, vehicle M is traveling at speed VM in the direction of extension of road RD1 (the X-axis direction in the figure). In the example of Figure 5, demarcation line RL11 corresponds to the left-side second demarcation line, demarcation line RL12 corresponds to the left-side first demarcation line, demarcation line RL13 corresponds to the right-side first demarcation line, and demarcation line RL14 corresponds to the right-side second demarcation line. The same applies to the subsequent figures.

When it is necessary to identify the lane in which vehicle M is traveling, the identification unit 153 periodically (always) determines whether the lane marking conditions of condition IDs "C101" to "C104" are met. In the example of Figure 5, it is assumed that the recognition unit 130 has not detected (recognized) lane markings RL11 and RL14 (undetected), but has recognized lane markings RL12 and RL13 as solid lines. In this case, the identification unit 153 determines that vehicle M's lane is an end lane of a single lane (a single lane) because the lane marking condition of condition ID "C101" shown in Figure 4 is met. Similarly, the identification unit 153 determines that vehicle M's lane is an end lane of a single lane when, in the first determination pattern, at least one of lane markings RL11 and RL14 is not detected but is instead recognized by the recognition unit 130 as a shoulder (shoulder marking).

<Second Determination Pattern>
FIG. 6 is a diagram illustrating an example of a second determination pattern. In the example of FIG. 6, the recognition unit 130 recognizes that the lane marking RL11 is a road shoulder (or side strip), the lane marking RL12 is a solid line, and the lane marking RL13 is a dashed line. In the second determination pattern, the lane marking RL14 may be undetected or may be detected as some kind of lane marking. In this case, the identification unit 153 determines that the lane in which the vehicle M is traveling is the leftmost lane of the multiple lanes because the lane marking condition of the condition ID "C102" shown in FIG. 4 is satisfied.

<Third Determination Pattern>
FIG. 7 is a diagram illustrating an example of a third determination pattern. In the example of FIG. 7, the recognition unit 130 recognizes that the lane marking RL11 is a solid line and the lane markings RL12 and RL13 are dashed lines. In the third determination pattern, the lane marking RL14 may be undetected, or some lane marking may be detected. In this case, the identification unit 153 determines that the lane of the vehicle M is the second lane (the second lane from the left) of a road with three or more lanes because the lane marking condition of condition ID "C103" shown in FIG. 4 is satisfied. In addition to the conditions of the third determination pattern, the identification unit 153 may also refer to the first map information 54 based on the position information of the vehicle M and determine that the condition "C103" is satisfied if the road RD1 is a road with three or more lanes.

<Fourth Determination Pattern>
FIG. 8 is a diagram illustrating an example of a fourth determination pattern. In the example of FIG. 8, the recognition unit 130 recognizes that the lane markings RL11 and RL13 are solid lines and the lane marking RL12 is a dashed line. The lane marking RL14 may be undetected, or the road shoulder may be detected. In this case, the identification unit 153 determines that the lane of the vehicle M is the second lane (the second lane from the left) of a two-lane road because the lane marking condition of condition ID "C104" shown in FIG. 4 is satisfied. In addition to the above conditions, the identification unit 153 may refer to the first map information 54 based on the position information of the vehicle M and determine that the condition "C104" is satisfied if the road RD1 is a two-lane road.

<Fifth Determination Pattern>
FIG. 9 is a diagram illustrating an example of a fifth determination pattern. In the example of FIG. 9 , it is assumed that map information (first map information 54) indicates that road RD1 on which vehicle M is traveling is a two-way traffic section with one lane (one lane in each direction). In this case, the identification unit 153 satisfies the condition of condition ID "C105" shown in FIG. 4 , and therefore next determines whether the lane marking condition of condition ID "C105" is satisfied. Here, in the example of FIG. 9 , it is assumed that the recognition unit 130 recognizes lane marks RL12 and RL13 as solid lines. Furthermore, in the fifth determination pattern, it is assumed that lane marks RL11 and RL14 may be undetected, or that some kind of lane marking may be detected. In this case, the identification unit 153 satisfies the determination start condition of condition ID "C105" shown in FIG. 4 , and therefore determines that the lane on which vehicle M is traveling is the end lane of a one-lane road in a two-way traffic section.

In addition, in the fifth determination pattern, instead of the dividing lines RL12 and RL13 being solid lines, the identification unit 153 may determine that at least one of them is a dashed line, a special line, or a shoulder (shoulder dividing line). A special line is, for example, a line whose presence can be recognized from a camera image, but whose type (solid line, dashed line, etc.) cannot be recognized. Examples of special lines include lines that have been partially obscured due to rubbing or dirt, and deceleration dashed lines. Deceleration dashed lines are, for example, road markings that make the width of the road appear narrower to the driver of the vehicle. In the fifth determination pattern, the identification unit 153 determines that the lane on which vehicle M is traveling is an edge lane of a one-lane road if the road on which vehicle M is traveling is a two-way traffic section with one lane in each direction, and if the type of at least one of the dividing lines on the left and right of vehicle M is a solid line, dashed line, special line, or shoulder dividing line.

Identifying which lane vehicle M is traveling in is very important when performing driving control in the first driving mode, for example. Therefore, even if the first map information 54 recognizes that there is one lane in each direction in a two-way traffic section, it is necessary to ensure that vehicle M is actually traveling in that lane. Therefore, as in the fifth determination pattern, by identifying that vehicle M is traveling in the edge lane of a one-lane road from the types of dividing lines on the left and right of vehicle M, more appropriate driving control can be executed (or continued).

In the fifth determination pattern, as shown in FIG. 9 , when a nearby vehicle m1 (another vehicle present within a predetermined distance from vehicle M) is present, the identification unit 153 may determine whether road RD1 is a road for oncoming traffic or identify the oncoming traffic lane based on the movement direction of nearby vehicle m1 recognized by the recognition unit 130. In the example of FIG. 9 , nearby vehicle m1 is traveling at a speed Vm1 in the opposite direction to the traveling direction of vehicle M. For example, as shown in FIG. 9 , when nearby vehicle m1 is recognized traveling in the opposite direction to the traveling direction of vehicle M in an adjacent lane separated from the traveling lane of vehicle M by a single dividing line RL13, the identification unit 153 identifies the adjacent lane to the right of the traveling lane of vehicle M as the oncoming traffic lane. Furthermore, instead of identifying the lane as the oncoming traffic lane, the identification unit 153 may identify the traveling lane of vehicle M as the lane closest to the oncoming traffic lane.

<Sixth Determination Pattern>
FIG. 10 is a diagram illustrating an example of a sixth determination pattern. In the example of FIG. 10 , it is assumed that map information (first map information 54) indicates that road RD1 on which vehicle M is traveling is a two-way traffic section with two lanes (two lanes on each side). In this case, the identification unit 153 satisfies the determination start condition for condition ID "C106" shown in FIG. 4 , and therefore next determines whether the lane marking condition for condition ID "C106" is satisfied. Here, in the example of FIG. 10 , it is assumed that the recognition unit 130 recognizes at least lane marking RL11 as a solid line and lane marking RL12 as a dashed line or a special line. Furthermore, in the sixth determination pattern, lane marking RL13 may be recognized as a solid line, and lane marking RL14 may be undetected or may be in a state where some lane marking has been detected. In this case, the identification unit 153 determines that the lane in which vehicle M is traveling is the rightmost lane (or the second lane from the left) of a two-lane road in a two-way traffic section, because the condition ID "C106" shown in Figure 4 is satisfied.

In the sixth determination pattern, the dividing line RL13 may be a dashed line, a special line, a road shoulder (shoulder dividing line), or a pole instead of a solid line. Examples of poles include lane dividers installed on the road surface and pylons (color cones (registered trademark)) temporarily placed (not permanently installed) due to construction or other reasons. In the sixth determination pattern, when road RD1 is a two-lane road with two or more parallel lanes (two or more lanes in the same direction), the identification unit 153 identifies the driving lane using the dividing lines recognized around vehicle M by the recognition unit 130, excluding the dividing line RL14 for oncoming traffic. Specifically, when road RD1 is a two-lane road with two-lane roads and the dividing line RL12 on the left side of vehicle M is a dashed line, the identification unit 153 identifies the driving lane of vehicle M as the rightmost lane of the two-lane road, regardless of the line type of the dividing line RL13 on the right side.

In the sixth determination pattern, when nearby vehicles (other vehicles) m1 and m2 are present as shown in FIG. 10, the identification unit 153 may determine whether road RD1 is a two-way traffic road or identify the driving lane of vehicle M based on the movement directions of nearby vehicles m1 and m2 recognized by the recognition unit 130. In the example of FIG. 10, nearby vehicle m1 is traveling in the opposite direction to the traveling direction of vehicle M at a speed Vm1, and nearby vehicle m2 is traveling in the same direction as the traveling direction of vehicle M at a speed Vm2. In this case, the identification unit 153 identifies the lane adjacent to the right of the traveling lane of vehicle M as a two-way traffic lane. Furthermore, because nearby vehicle m2 is traveling in a lane divided by dividing lines RL11 and RL12, the identification unit 153 identifies the traveling lane of vehicle M as the rightmost lane of a two-lane road.

<Seventh Determination Pattern>
FIG. 11 is a diagram illustrating an example of a seventh determination pattern. In the example of FIG. 11 , it is assumed that map information (first map information 54) indicates that road RD1 on which vehicle M is traveling is a two-way traffic section with three or more lanes (three lanes on each side). In this case, the identification unit 153 satisfies the determination start condition for condition ID "C107" shown in FIG. 4 , and therefore next determines whether the lane marking condition for condition ID "C107" is satisfied. Here, in the example of FIG. 11 , it is assumed that the recognition unit 130 recognizes lane marking RL12 as a dashed line and lane marking RL13 as a solid yellow line. Furthermore, in the seventh determination pattern, it is assumed that lane markings RL11 and RL14 may be undetected, or that some lane markings may be detected. In this case, the identification unit 153 satisfies the condition for condition ID "C107" shown in FIG. 4 , and therefore determines that vehicle M's traveling lane is the rightmost lane of a lane with three or more lanes on each side in a two-way traffic section.

In the seventh determination pattern, the dividing line RL12 may be a special line instead of a dashed line. In the seventh determination pattern, when road RD1 is a two-way traffic section and has multiple parallel lanes (three or more lanes for travel in the same direction), the identification unit 153 identifies the driving lane using the dividing lines recognized around vehicle M by the recognition unit 130, excluding the dividing line RL14 for oncoming traffic. Specifically, when the road on which vehicle M is traveling is a two-way traffic section with three or more lanes on each side, and the dividing line to the right of vehicle M is a yellow lane and the dividing line to the left of vehicle M is a dashed line, the identification unit 153 identifies the driving lane of vehicle M as the rightmost lane of a lane with three or more lanes on each side.

Furthermore, in the seventh determination pattern, as in the sixth determination pattern described above, when surrounding vehicles (other vehicles) m1 and m2 are present around vehicle M, the identification unit 153 may determine whether road RD1 is an oncoming traffic road or identify the driving lane of vehicle M based on the direction of movement of surrounding vehicles m1 and m2, etc.

In the fifth to seventh determination patterns described above, the identification unit 153 may determine whether road RD1 is a two-way traffic section based on the line type of the lane markings recognized from the camera image, instead of (or in addition to) obtaining from the information included in the first map information 54 that road RD1 is a two-way traffic section. In this case, the identification unit 153 may recognize road RD1 as a two-way traffic section if, for example, the lane markings recognized from the camera image include solid yellow lines or have a shape indicating a two-way traffic section. The identification unit 153 may also determine that road RD1 is a two-way traffic section if the road information recognized from the camera image includes a road sign indicating two-way traffic.

By identifying the driving lane of vehicle M using the above-described first to seventh determination patterns, it is possible to more accurately identify the driving lane of vehicle M, even in road conditions that are difficult to recognize from camera images. Therefore, even in cases where oncoming traffic is difficult to recognize from camera images, it is possible to prevent mismatches between road information obtained from map information and road (driving lane) information recognized from camera images. Therefore, it is possible to continue the first driving mode without switching from the first driving mode to the second driving mode. Note that the identification unit 153 may identify the driving lane of vehicle M using patterns other than the first to seventh determination patterns.

Furthermore, if none of the above-mentioned first to seventh judgment patterns (condition IDs "C101" to "C107") apply, the identification unit 153 is unable to identify which lane the vehicle M is traveling in among one or more lanes included in road RD1.

When the identification unit 153 identifies the driving lane of vehicle M, the mode change processing unit 154 executes the first driving mode, and performs driving control such as driving vehicle M without deviating from the identified driving lane, following a vehicle ahead, or changing lanes to drive toward the destination. Furthermore, when the identification unit 153 does not identify the driving lane of vehicle M, the mode change processing unit 154 executes the second driving mode. As a result, when it is not possible to identify which lane on the road vehicle M is driving in, autonomous driving is restricted, allowing vehicle M to drive more safely.

Furthermore, if the identification unit 153 cannot identify the lane in which vehicle M is traveling, the mode change processing unit 154 may continue to drive vehicle M in the currently active driving mode. In this way, by maintaining the current driving mode without switching driving modes in situations in which the lane in which vehicle M is traveling cannot be identified, more stable driving can be achieved.

When the lane in which vehicle M is traveling cannot be identified, whether to switch to the second driving mode or continue the current driving mode (first driving mode or second driving mode) may be determined based on, for example, the surrounding conditions of vehicle M and road information obtained from the first map information 54. For example, if the recognition unit 130 determines that a predetermined number of other vehicles are present around vehicle M (within a predetermined distance), it may be impossible to recognize lane markings due to the influence of the other vehicles. Therefore, the mode change processing unit 154 may continue the current driving mode even if the lane in which vehicle M is traveling cannot be identified if there are a predetermined number of other vehicles or more around vehicle M, and switch to the second driving mode if there are fewer than the predetermined number of other vehicles around vehicle M. Furthermore, the mode change processing unit 154 may refer to the first map information 54 and obtain information about the roads around vehicle M based on the position information of vehicle M. If the number of lanes on the road within a predetermined distance from vehicle M is not increasing or decreasing or the curvature of the road is small (below a threshold), the mode change processing unit 154 may continue the current driving mode. This allows the vehicle M to implement a more appropriate driving mode depending on the surrounding conditions.

Furthermore, if the identification unit 153 is unable to identify the lane in which vehicle M is traveling, the HMI control unit 170 may cause the HMI 30 to output a message that the lane in which vehicle M is traveling cannot be identified, thereby notifying the occupant. This allows the occupant to be notified of the situation of vehicle M, and enables the occupant to monitor the area around the vehicle, switch to manual driving as necessary, or otherwise perform driving control according to the situation.

[Processing flow]
Next, a flow of processing executed by the automatic driving control device 100 of the embodiment will be described. Note that, of the processing executed by the automatic driving control device 100, the following description will mainly focus on the processing of identifying the driving lane of the vehicle M and the processing of switching the driving mode based on the result of the identification processing. Furthermore, the processing of this flowchart may be executed repeatedly at a predetermined timing, for example.

Figure 12 is a flowchart showing an example of the flow of driving control processing executed by the automatic driving control device 100. In the example of Figure 12, the recognition unit 130 recognizes the surrounding conditions of the vehicle M (step S100). Next, the mode determination unit 150 drives the vehicle in one of a plurality of pre-set driving modes based on the surrounding conditions, etc. (step S102).

Next, the mode determination unit 150 acquires the first map information, the recognition result by the recognition unit 130, and reference information for identifying the position of vehicle M (step S104), and performs processing to identify the driving lane of vehicle M based on the acquired information (step S106). Here, the identification unit 153 of the mode determination unit 150 determines whether the judgment conditions related to oncoming traffic (e.g., the conditions of condition IDs "C105" to "C107" in the judgment table 182 described above) are met based on the acquired information (step S108). If it is determined that the conditions are met, the identification unit 153 employs special rules (predetermined relaxed conditions) and identifies the driving lane without using at least information on the line type of the lane markings for oncoming traffic. More specifically, the identification unit 153 identifies the driving lane using lane markings that exclude oncoming traffic lane markings (step S110). Furthermore, if it is determined in the processing of step S108 that the conditions are not met, the identification unit 153 does not adopt the special rule, but identifies the driving lane of vehicle M using the recognized lane markings (step S112).

Next, the mode change processing unit 154 determines whether the driving mode of vehicle M is currently in the first driving mode (step S114). If it is determined that the first driving mode is currently being executed, the mode change processing unit 154 determines whether the identification unit 153 has identified the driving lane of vehicle M (step S116). If it is determined that the driving lane has been identified, the mode change processing unit 154 continues the first driving mode based on the identified driving lane (step S118). If it is determined that the driving lane has not been identified, the mode change processing unit 154 performs control to switch from the first driving mode to the second driving mode (step S120). This ends the processing of this flowchart. Furthermore, if it is determined in the processing of step S114 that the first driving mode is not currently being executed, the processing of this flowchart ends.

[Modification]
Below, several modified examples of this embodiment will be described. For example, in the above-described embodiment, when the identification unit 153 is unable to identify the driving lane of the vehicle M, the driving control unit may perform a lane change or the like to position the vehicle M in the leftmost or rightmost lane of the road on which the vehicle M is traveling. This makes it possible to identify the driving lane more accurately after the lane change. Note that the driving control unit may also cause the identification unit 153 to perform processing to identify the driving lane of the vehicle M immediately after positioning the vehicle M in the leftmost or rightmost lane of the road. This makes it possible to identify the driving lane more quickly when the driving lane cannot be identified.

In addition, in the above-described embodiment, if map information (second map information) with higher accuracy than the first map information 54 is held in addition to the first map information 54, and if the second map information cannot be obtained, control may be performed to identify the driving lane using the first map information 54 or an image captured by the camera 10.

Here, the second map information is, for example, map information in which road information is defined for each lane over a shorter section than the first map information 54. The second map information may also include, for example, information on the center of lanes or information on lane boundaries. The second map information may also include road information, traffic regulation information, address information (address and postal code), facility information, telephone number information, etc. The second map information may be updated as needed by the communication device 20 communicating with other devices. The second map information may be held in a storage device such as the HDD or flash memory of the MPU 60, or may be stored in the memory unit 180.

For example, if the MPU 60 or the memory unit 180 has second map information, the recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into multiple blocks and determines the recommended lane for each block by referring to the second map information. In this case, the recommended lane determination unit 61 may use the lane information included in the second map information to determine which lane from the left the vehicle M should travel in.

Here, for example, if vehicle M is executing the first driving mode and the second map information is unavailable due to an abnormality in the data of the second map information or an abnormality during the update of the second map information, the mode determination unit 150 identifies the driving lane of vehicle M as described above based on information acquired from the first map information 54, and continues the first driving mode. This makes it possible to execute a driving mode with a high degree of automation even in a situation where the second map information is unavailable.

In addition, in an embodiment, when switching from the first driving mode to the second driving mode, the mode determination unit 150 may determine which of multiple modes included in the second driving mode to switch to, depending on the driving state and driving environment of the vehicle M. The driving state is, for example, the state of the driver determined by the driver state determination unit 151. The driving environment is, for example, the shape of the road around the vehicle M, the number of lanes, the presence or absence of branching or merging, the number and relative positions of other vehicles in the vicinity, etc. For example, when the conditions for switching from the first driving mode to the second driving mode are met, the mode determination unit 150 determines to switch to mode C if the number of lanes when the driving lane of the vehicle M cannot be identified is three lanes, to mode D if it is four lanes, or to mode E if it is five lanes or more. This allows the vehicle M to drive in a more appropriate mode depending on the driving state and driving environment. Furthermore, the lane marking conditions in the determination table 182 in the above-described embodiment may be replaced with other lane markings as appropriate depending on the traffic laws (Road Traffic Act) of the foreign country to which this embodiment applies.

According to the embodiment described above, the vehicle control device includes a recognition unit 130 that recognizes the surrounding conditions of vehicle M, a driving control unit (action plan generation unit 140, second control unit 160) that controls one or both of the steering and speed of vehicle M based on the surrounding conditions recognized by the recognition unit 130, an acquisition unit (first acquisition unit) 152 that acquires map information including lane information around vehicle M and reference information for identifying the position of vehicle M, and an identification unit 153 that identifies the driving lane of vehicle M based on information about the road on which the vehicle is traveling, acquired from the map information based on the reference information, and the type of lane markings that demarcate one or more lanes around vehicle M recognized by the recognition unit 130.When it is determined that the road is a two-way traffic section based on at least one of the road information and the type of lane markings, the identification unit 153 identifies the driving lane of vehicle M using the road markings from the recognition results around vehicle M recognized by the recognition unit 130, without using at least information about the line type of the lane markings for two-way traffic.This makes it possible to more accurately identify the driving lane of the vehicle even if the road on which the vehicle is traveling is a two-way traffic section. Therefore, the first driving mode can be continued using information about the identified driving lane, or the degree of driving control can be changed in more appropriate situations.

Specifically, for example, if there are no nearby vehicles, it may not be possible to determine from camera images alone whether the road is a two-way traffic section, and the lane vehicle M is traveling in may not be identified. Therefore, in this embodiment, if it is determined that the road is a two-way traffic section based on map information or the like, special rules (predetermined relaxed conditions) are used to identify the lane vehicle M is traveling in (for example, identifying the lane vehicle M is traveling in based on the type of dividing line corresponding to the number of lanes on each side), thereby improving the rate at which the lane vehicle is identified and reducing erroneous identification.

Furthermore, according to the embodiment, the driving lane is identified based on the recognition results of two lane markings on each side of vehicle M (first left lane marking, second left lane marking, first right lane marking, second right lane marking), thereby improving the identification rate of vehicle M's driving lane without increasing the recognition processing load. Furthermore, according to the embodiment, even if vehicle M is not equipped with high-precision map information or the equipped high-precision map cannot be used, the position (driving lane) of vehicle M can be more accurately identified using a navigation map (first map information 54) such as that used in navigation device 50, and the first driving mode can be continued. This eliminates the need for frequent map information updates or management by a map server, as is required with high-precision maps, thereby reducing operational costs.

The above-described embodiment can be expressed as follows.
a storage device storing a program;
a hardware processor;
The hardware processor executes the program stored in the storage device,
Recognizes the vehicle's surroundings,
Execute driving control to control one or both of the steering and the speed of the vehicle based on the recognized surrounding conditions;
acquiring map information including lane information around the vehicle and reference information for identifying the position of the vehicle;
identifying a lane in which the vehicle is traveling based on information about the road on which the vehicle is traveling, obtained from the map information based on the reference information, and based on the type of road dividing line that divides each of the one or more lanes around the vehicle that has been recognized;
When the road is determined to be a two-way traffic section based on at least one of the road information and the type of the road dividing line, the travel lane is identified from the recognition results of the surroundings of the recognized vehicle without using at least information on the line type of the two-way traffic dividing line.
Vehicle control device.

The above describes the form for carrying out the present invention using an embodiment, but the present invention is in no way limited to such an embodiment, and various modifications and substitutions can be made without departing from the spirit of the present invention.

1...vehicle system, 10...camera, 12...radar device, 14...LIDAR, 16...object recognition device, 20...communication device, 30...HMI, 32...turn signal switch, 40...vehicle sensor, 50...navigation device, 60...MPU, 70...driver monitor camera, 80...driving operator, 82...steering wheel, 84...steering grip sensor, 100...automatic driving control device, 120...first control unit, 1 30...Recognition unit, 140...Action plan generation unit, 150...Mode determination unit, 151...Driver state determination unit, 152...First acquisition unit, 153...Identification unit, 154...Mode change processing unit, 160...Second control unit, 162...Second acquisition unit, 164...Speed control unit, 166...Steering control unit, 170...HMI control unit, 180...Memory unit, 200...Driving force output device, 210...Brake device, 220...Steering device, M...Vehicle

Claims (12)

a recognition unit that recognizes the surrounding situation of the vehicle;
a driving control unit that controls one or both of the steering and the speed of the vehicle based on the surrounding conditions recognized by the recognition unit;
an acquisition unit that acquires map information including lane information around the vehicle and reference information for identifying the position of the vehicle;
an identification unit that identifies a driving lane of the vehicle based on information about the road on which the vehicle is traveling, obtained from the map information based on the reference information, and based on the type of road dividing line that divides each of one or more lanes around the vehicle recognized by the recognition unit,
when it is determined that the road is a two-way traffic section based on at least one of the information about the road and the type of the road dividing line, the identification unit identifies a driving lane from among the recognition results of the periphery of the vehicle recognized by the recognition unit without using at least information about the line type of the two-way traffic dividing line ,
When the identification unit acquires, based on the map information, that the road is a two-way traffic section and has a plurality of parallel lanes, it identifies the driving lane from the recognition results of the periphery of the vehicle recognized by the recognition unit without using information on the line type of the two-way traffic lane markings.
Vehicle control device. a recognition unit that recognizes the surrounding situation of the vehicle;
a driving control unit that controls one or both of the steering and the speed of the vehicle based on the surrounding conditions recognized by the recognition unit;
an acquisition unit that acquires map information including lane information around the vehicle and reference information for identifying the position of the vehicle;
an identification unit that identifies a driving lane of the vehicle based on information about the road on which the vehicle is traveling, obtained from the map information based on the reference information, and based on the type of road dividing line that divides each of one or more lanes around the vehicle recognized by the recognition unit,
when it is determined that the road is a two-way traffic section based on at least one of the information about the road and the type of the road dividing line, the identification unit identifies a driving lane from among the recognition results of the periphery of the vehicle recognized by the recognition unit without using at least information about the line type of the two-way traffic dividing line ,
The operation control unit
running the vehicle by executing one of a plurality of driving modes including at least a first driving mode and a second driving mode in which a task imposed on an occupant of the vehicle is heavier than that in the first driving mode;
When the identification unit identifies the driving lane of the vehicle, the first driving mode is executed;
When the identification unit does not identify the driving lane of the vehicle, the second driving mode is executed.
Vehicle control device. a recognition unit that recognizes the surrounding situation of the vehicle;
a driving control unit that controls one or both of the steering and the speed of the vehicle based on the surrounding conditions recognized by the recognition unit;
an acquisition unit that acquires map information including lane information around the vehicle and reference information for identifying the position of the vehicle;
an identification unit that identifies the lane in which the vehicle is traveling based on information about the road on which the vehicle is traveling, obtained from the map information based on the reference information, and based on the type of road dividing line that divides each of the one or more lanes around the vehicle recognized by the recognition unit,
when it is determined that the road is a two-way traffic section based on at least one of the information about the road and the type of the road dividing line, the identification unit identifies a driving lane from among the recognition results of the periphery of the vehicle recognized by the recognition unit without using at least information about the line type of the two-way traffic dividing line ,
the driving control unit runs the vehicle by executing one of a plurality of driving modes in which tasks are differently assigned to an occupant of the vehicle, and when the identification unit does not identify a driving lane of the vehicle, continues running the driving mode to run the vehicle.
Vehicle control device. When the road is a two-way traffic section with one lane in each direction, the identification unit identifies the vehicle's travel lane as one lane in each direction if the types of road dividing lines on the left and right sides of the vehicle are solid lines, broken lines, special lines, or shoulders.
The vehicle control device according to any one of claims 1 to 3 . The identification unit identifies the vehicle as traveling in the rightmost lane of the two-lane road when the road is a two-lane road section for two-way traffic, the road dividing line on the left side of the vehicle is a broken line or a special line, and the dividing line further back than the left road dividing line as seen from the vehicle is a solid line.
The vehicle control device according to any one of claims 1 to 3. the identification unit identifies the vehicle's travel lane as the rightmost lane of the road with three or more lanes in each direction when the road is a two-way traffic section with three or more lanes in each direction and the road dividing line on the right side of the vehicle is a yellow lane and the road dividing line on the left side of the vehicle is a dashed line or a special line;
The vehicle control device according to any one of claims 1 to 3 . The computer
Recognizes the vehicle's surroundings,
Execute driving control to control one or both of the steering and the speed of the vehicle based on the recognized surrounding conditions;
acquiring map information including lane information around the vehicle and reference information for identifying the position of the vehicle;
identifying a driving lane of the vehicle based on information about the road on which the vehicle is traveling, obtained from the map information based on the reference information, and based on the type of road dividing line that divides each of the one or more lanes around the vehicle that has been recognized;
When the road is determined to be a two-way traffic section based on at least one of the road information and the type of the road dividing line, the vehicle's driving lane is identified without using information on the type of the two -way traffic dividing line from among the results of the recognition of the surroundings of the recognized vehicle ;
When it is determined based on the map information that the road is a two-way traffic section and has a plurality of parallel lanes, the system identifies the driving lane from the recognition results of the surroundings of the vehicle without using information on the line type of the two-way traffic lane markings.
Vehicle control method. The computer
Recognizes the vehicle's surroundings,
Execute driving control to control one or both of the steering and the speed of the vehicle based on the recognized surrounding conditions;
acquiring map information including lane information around the vehicle and reference information for identifying the position of the vehicle;
identifying a driving lane of the vehicle based on information about the road on which the vehicle is traveling, obtained from the map information based on the reference information, and based on the type of road dividing line that divides each of the one or more lanes around the vehicle that has been recognized;
When the road is determined to be a two-way traffic section based on at least one of the road information and the type of the road dividing line, the vehicle's driving lane is identified without using information on the type of the two -way traffic dividing line from among the results of the recognition of the surroundings of the recognized vehicle ;
running the vehicle by executing one of a plurality of driving modes including at least a first driving mode and a second driving mode in which a task imposed on an occupant of the vehicle is heavier than that in the first driving mode;
When the driving lane of the vehicle is identified, the first driving mode is executed;
When the driving lane of the vehicle is not identified, the second driving mode is executed.
Vehicle control method. The computer
Recognizes the vehicle's surroundings,
Execute driving control to control one or both of the steering and the speed of the vehicle based on the recognized surrounding conditions;
acquiring map information including lane information around the vehicle and reference information for identifying the position of the vehicle;
identifying a driving lane of the vehicle based on information about the road on which the vehicle is traveling, obtained from the map information based on the reference information, and based on the type of road dividing line that divides each of the one or more lanes around the vehicle that has been recognized;
When the road is determined to be a two-way traffic section based on at least one of the road information and the type of the road dividing line, the vehicle's driving lane is identified without using information on the type of the two -way traffic dividing line from among the results of the recognition of the surroundings of the recognized vehicle ;
running the vehicle by executing one of a plurality of driving modes in which tasks are assigned to the vehicle occupants differently, and, if the driving lane of the vehicle is not specified, continuing the running driving mode to run the vehicle;
Vehicle control method. On the computer,
Recognize the vehicle's surroundings,
Execute driving control to control one or both of the steering and the speed of the vehicle based on the recognized surrounding conditions;
acquiring map information including lane information around the vehicle and reference information for identifying the position of the vehicle;
Identifying the lane in which the vehicle is traveling based on information about the road on which the vehicle is traveling, which is acquired from the map information based on the reference information, and based on the type of road dividing line that divides each of the one or more lanes around the recognized vehicle;
When the road is determined to be a two-way traffic section based on at least one of the road information and the type of the road dividing line, the driving lane is identified without using at least information on the type of the two-way traffic dividing line from the recognition results of the surroundings of the recognized vehicle ,
When it is determined based on the map information that the road is a two-way traffic section and has multiple parallel lanes, the system identifies the driving lane from the recognition results of the surroundings of the vehicle without using information on the line type of the two-way traffic lane markings.
program. On the computer,
Recognize the vehicle's surroundings,
Execute driving control to control one or both of the steering and the speed of the vehicle based on the recognized surrounding conditions;
acquiring map information including lane information around the vehicle and reference information for identifying the position of the vehicle;
Identifying the lane in which the vehicle is traveling based on information about the road on which the vehicle is traveling, which is acquired from the map information based on the reference information, and based on the type of road dividing line that divides each of the one or more lanes around the recognized vehicle;
When the road is determined to be a two-way traffic section based on at least one of the road information and the type of the road dividing line, the driving lane is identified without using at least information on the type of the two-way traffic dividing line from the recognition results of the surroundings of the recognized vehicle ,
running the vehicle by executing one of a plurality of driving modes including at least a first driving mode and a second driving mode in which a task imposed on an occupant of the vehicle is heavier than that in the first driving mode;
When the driving lane of the vehicle is identified, the first driving mode is executed;
When the driving lane of the vehicle is not identified, the second driving mode is executed.
program. On the computer,
Recognize the vehicle's surroundings,
Execute driving control to control one or both of the steering and the speed of the vehicle based on the recognized surrounding conditions;
acquiring map information including lane information around the vehicle and reference information for identifying the position of the vehicle;
Identifying the lane in which the vehicle is traveling based on information about the road on which the vehicle is traveling, which is acquired from the map information based on the reference information, and based on the type of road dividing line that divides each of the one or more lanes around the recognized vehicle;
When the road is determined to be a two-way traffic section based on at least one of the road information and the type of the road dividing line, the driving lane is identified without using at least information on the type of the two-way traffic dividing line from the recognition results of the surroundings of the recognized vehicle ,
running the vehicle by executing one of a plurality of driving modes in which tasks are assigned to an occupant of the vehicle differently, and, if the driving lane of the vehicle is not specified, continuing the running driving mode to run the vehicle;
program.