JP7618629B2 - Route recognition system for autonomous vehicles - Google Patents

Description

The present invention relates to a route recognition system for autonomous vehicles that follows a driving route using a camera installed in the vehicle.

A driver's driving actions involve repeated chronological processing of cognition, judgment, and operation. Research has ranged from elucidating driver driving behavior to improving automobile safety, to research on automobiles and control systems that are suited to human characteristics, to research on driver steering, such as where drivers look on the route and how they steer.

Non-Patent Document 1 proposes a forward gaze primary prediction model in which "the steering wheel is operated in proportion to the deviation between the target course at a gaze point a certain distance L ahead in the vehicle's travel direction and the predicted future position of the vehicle."

In chapter 11 of Non-Patent Document 2, it is stated that the key to automatic steering is to detect the deviation of the front and rear ends of the vehicle from the desired course using an appropriate method, and to perform steering that is sensitive to the sum and difference of the two.

In addition, in Non-Patent Document 3, when driving around a sharp curve, the primary prediction model is unable to follow the course well. Therefore, as a more realistic model, a secondary prediction model is proposed, which considers that "the future vehicle position is regarded as a position predicted based on the current motion state, and steering is performed according to the deviation from the target course."
The forward gaze models of Non-Patent Documents 1 and 3 use a camera mounted on a vehicle to recognize the forward direction of travel, leading to technology for driving assistance and self-driving.

Patent document 1 describes that the device is equipped with a means for notifying the user of conditions under which recognition processing by the camera is impossible, such as when the image is too bright due to backlighting or after rain, when the image is too dark at night, or when the image has low contrast due to twilight or snow.

The key points of the automatic steering described in Non-Patent Document 2 are connected to path following control that recognizes lanes using a side camera, and further to a magnetic sensor method that follows a route using magnets buried in the road surface.
Patent Document 2 states that a magnetic sensor installed under the vehicle floor can detect the position of magnets buried in the road, making it possible to recognize the position without using GPS or IMU, and to travel without derailing from the magnetic track, from straight ahead to all curvature radii including the minimum turning radius.

Patent document 3 describes a method for detecting changes in the position of the center of gravity that accompany changes in the load, and shows a redundant system that obtains the sideslip angle and yaw rate of the vehicle's center of gravity from a vehicle model, calculates its coordinates and direction, and matches these with the detection values from an IMU (inertial navigation unit), GPS, and magnetic markers.

Patent Document 4 describes how an autonomous vehicle is equipped with multiple sensors to recognize the surroundings of the vehicle, and how the external recognition method using each sensor involves constantly replacing the surrounding recognition coordinates recognized by the multiple sensors attached in different positions with the vehicle's center of gravity position. In other words, the coordinates detected by each sensor are converted and captured on vehicle coordinates with the center of gravity of the vehicle, which is the origin of vehicle motion, as the origin.

JP 2000-207563 A JP 2020-098566 A JP 2021-25925 A [Patent Document 4] JP 2022-0343345

Masaichi Kondo: On the fundamental relationship between steering and motion of an automobile, Transactions of the Society of Automotive Engineers of Japan, No. 5 (1958), p. 40-43. Masaichi Kondo: Basic Automotive Engineering, Late Edition, Yokendo Co., Ltd., 1st edition published April 20, 1962, p.21. Kenichi Yoshimoto: Simulation of the human-vehicle system using a steering model including prediction, Journal of the Japan Society of Mechanical Engineers, Vol. 71, No. 596 (September 1968), pp. 13-18.

White line recognition devices have been developed for self-driving vehicles that photograph the left and right white lines and recognize them on the road. These white line recognition systems photograph the shapes of the left and right white lines using cameras inside the windshield and on top of the vehicle while driving, but they are exposed to disturbances from the natural environment, such as sunlight and shadows, and sometimes cannot recognize the white lines.

Non-Patent Documents 1 to 3 relate to a forward-focused model that uses a camera mounted on a vehicle to recognize the road ahead, but make no proposals about resolving the difficulty of recognizing markers on the road, such as white lines.

Patent Document 1 also relates to a forward gaze model, and this prior document only notifies the driver when it becomes difficult to recognize indicators due to twilight or other reasons, but does not propose any means to resolve the situation.

In Patent Document 2, magnets embedded in the road are given information such as the route coordinates, curvature, and azimuth, resulting in excellent route tracking performance. However, while the magnetic sensor method solves the problem of being unable to recognize white lines, it has concerns about its resistance to lightning strikes and the cost of burying magnetic markers.

In order to solve the above problems, the route recognition system for autonomous driving according to the present invention sets a route line on the road surface, and attaches a camera to the vehicle body in a position where the route line can be viewed without being exposed to disturbances from the natural environment such as the effects of sunlight or shadows, and where the relative position with the vehicle body's center of gravity can be determined. By referring to the position of this camera, the lateral deviation (e2) between the route line and the vehicle's center of gravity and the angular deviation (e3) of the vehicle body axle relative to the route line are detected, and the detected lateral deviation (e2) and angular deviation (e3) are eliminated, and steering control is performed so that the vehicle's center of gravity coincides with the route line.

In addition to existing white lines, the route lines can be dotted or colored lines, and the route lines themselves can have information such as curvature change information, road gradient change information, planned vehicle speed information, route branch information, route junction information, and longitude, latitude, and orientation. This makes it possible to perform look-ahead control.

Regarding the position of the center of gravity of the vehicle body, the position of the center of gravity when the vehicle is unloaded can be calculated by multiplying the wheelbase by the ratio of the front axle load to the rear axle load listed on the vehicle inspection certificate, and the position of the center of gravity when the vehicle is loaded or has passengers on board can be calculated by detecting the front axle load to the rear axle load from the air pressure of the air suspension on the front and rear axles and multiplying that ratio by the wheelbase, so the positional relationship between the installation position of the camera itself and the position of the center of gravity can be calculated. It is preferable that the changing position of the center of gravity falls within the field of view of the camera.

An example of an installation location where the route line can be seen and where the relative position to the vehicle's center of gravity can be known is under the floor of the vehicle body. This location allows accurate recognition of the route line and detection of the lateral deviation (e2) between the route line and the vehicle's center of gravity. However, measures against stone chipping, water intrusion, etc. will be necessary.

The cameras may also be installed in two locations: one where the route line can be seen, and one where the relative position to the center of gravity can be determined. In this case, the positional relationship between the two cameras must be corrected when calculating the lateral deviation (e2) and angular deviation (e3).

According to the present invention, by mounting a camera on the front of the vehicle, as well as on the underfloor surface and within the range of change in the center of gravity of the vehicle, and illuminating this range with lighting devices installed on the vehicle body, it is possible to recognize the route line even in twilight or in bad weather. Then, the lateral and angular displacement of the vehicle body at the position of the center of gravity of the vehicle relative to the route line can be captured to determine the steering angle and control the following of the route. Furthermore, in addition to determining the steering angle by associating the type, shape, and color of the route line with route information, it is possible to control the following of the route in response to operational management such as acceleration/deceleration, stopping, waiting, and U-turns.

Diagram of underfloor camera field of view A diagram illustrating how the robot follows a route by detecting the deviation of its own position from the route line and the curvature of the route. Diagram of how the vehicle recognizes its own location from route information and follows the route Diagram of detecting junctions and other intersections from route information

Hereinafter, an embodiment of the present invention will be described with reference to FIGS.
Figure 1 is an explanatory diagram of the field of view arrangement of an underfloor camera, showing a vehicle turning with a minimum turning radius _R0 . The position of the center of gravity of the vehicle changes with the load or number of passengers. The camera is placed at a position (camera field of view (W) in the figure) where the locus of the center of gravity is within the camera's field of view. The axis of the camera passes through the center of the camera's field of view (W).

Note that the center of gravity trajectory is not visible to the camera, but rather it is the route line that is projected in the camera's field of view (W). In this embodiment, the center of gravity trajectory traced by the vehicle's center of gravity in a planar view is contained in the camera's field of view (W), which projects the route line. In other words, by aligning the x-axis of the vehicle's xy coordinate system (the vehicle's front-to-rear central axis) with the camera axis and determining and installing the camera's front-to-rear position (y value of the xy coordinate system), the center of gravity trajectory traced by the vehicle's center of gravity is placed within the camera's field of view (W).

When turning with the minimum turning radius, the path traced by the center of gravity is an arc with its center at "Point O" on the extension of the rear axle. The field of view must be long and wide enough to include this arc, and the field of view must be long (lsf-lsr) and wide (B) to capture the path traced by the center of gravity over the entire range from straight driving to the minimum turning radius, and must be set so that the curvature change of the path line traced on the road surface can be captured.

Although the above-mentioned "capturing the trajectory of the center of gravity over the entire range from straight driving to the minimum radius" is possible with an underfloor camera, it is better that "for the front-of-vehicle camera, the trajectory of the center of gravity is limited to within the field of view captured by the front-of-vehicle camera." Therefore, autonomous driving using only the front-of-vehicle camera is limited to a turning range within which the trajectory of the center of gravity of the vehicle can be captured in the field of view. The fields of view of both the front-of-vehicle camera and the underfloor camera are recognized on an x-y plane coordinate system with the center of gravity of the vehicle as the origin. Note that the origin of the field of view image may be the intersection of the vehicle's front-to-rear center line and the rear axle center line instead of the vehicle's center of gravity position.

FIG. 2 is an explanatory diagram of detecting the self-position deviation from the route line and the curvature of the route, and tracing the route by detecting the curvature of the route. The route line is shown by a dotted line. The route line is captured at points _P1 , Pc, and _P2 . To capture point Pc, it is a prerequisite that the center of gravity position Gc is known. The distance between the center of gravity position Gc and point Pc is taken as the lateral deviation e2, and the angle between the route tangent at point Pc (a line perpendicular to the normal line) and the vehicle center line (x-axis) is taken as the angular deviation e3. The actual front wheel steering angle for tracing the route is calculated from the captured e2, e3, and the vehicle's longitudinal speed vx.

That is, dividing the differential value e'2 of the lateral deviation e2 by the vehicle longitudinal speed vx gives the yaw rate r _e2 due to e2, i.e., formula (1). The differential value of e3 gives the yaw rate r _e3 due to e3, i.e., formula (2). The intersection _0-12 of the normal line to point p1 and the normal line to point p2 is found to determine the radius of curvature (R _1-= R ₂ ), and the reciprocal is taken to determine the route curvature ρ _route . Multiplying this route curvature by the route tangential speed (i.e., the value obtained by dividing the vehicle longitudinal speed by the cosine of the angular deviation e3) gives the route yaw rate r _route , i.e., formula (3). Thus, the sum of formulas (1), (2), and (3) gives the vehicle yaw rate r, i.e., formula (4).

Dividing the vehicle's yaw rate by the vehicle speed v (i.e., _vx /cos _e2 ) gives the vehicle's turning curvature ρ, i.e., equation (5), and the actual front wheel steering angle δ for aligning the vehicle's center of gravity with this path is given by equation (6).

Fig. 3 is an explanatory diagram of how the vehicle recognizes its own position from route line information and follows a route. The origin of the coordinate system (earth coordinates) is placed at the starting point (starting point), and the length of the dashed line and the coordinate x and coordinate y of the dashed line interval are accumulated (integrated) from there to calculate the coordinates of the current location ( _xn , _yn ) using equations (7) and (8) and the azimuth angle ( _Φn ) using equation (9). Each dashed line is associated with its ID number, and map data of latitude, longitude, azimuth angle, and other information is associated with it, and table lookup control is performed.

Now, the lateral deviation of the vehicle with respect to the pn(xn,yn) point on the route and its tangent line is e2, and the angular deviation is e3. The x coordinate of the vehicle's center of gravity is given by formula (10), and the y coordinate by formula (11). The straight-line distance l _GCm from the vehicle's center of gravity coordinate (xGC,yGC) to the junction pm on the route and the route arc angle φ _GCm are given by formula (12) and formula (13). The side slip angle β included in formula (13) is given by formula (14). Thus, the route curvature ρ _GCm from the center of gravity to the junction is given by formula (15), and the front wheel actual steering angle δ to follow the route curvature from the center of gravity to the junction is given by formula (16). In formula (16), l is the wheelbase of the vehicle, and the product of this with the route curvature ρ _GCm corresponds to the actual steering angle at low speed, and (1 + K _SF v^2) in parentheses indicates the speed dependency of the actual steering angle. The merging point is located farther away as the speed increases and the curvature decreases, and closer as the speed increases and the curvature increases. Here, the lateral slip coefficient β included in equation (14) and the stability factor K _SF included in equation (16) are expressed by equations (17) and (18). In equation (17), lf is the distance from the center of gravity of the vehicle to the front axle, Ccr is the rear wheel tire cornering coefficient, Nr is the rear axle load, lf is the distance from the center of gravity of the vehicle to the rear axle, and m is the vehicle mass. In equation (18), Ccf is the front wheel tire cornering coefficient, and Nf is the front axle load.

Figure 4 is an explanatory diagram of how branching and merging are detected from route information. In this invention, route lines are solid or dotted, white or colored, and laid in the center of the route, making it possible to preemptively transmit curvature change information, road gradient change information, planned vehicle speed information, route branching information, and route merging information.

Figure 4 (A) is an example of distinguishing between cruising on a single track section and branching and merging in a waiting area section by the type and color of the line. The single track section is drawn with a dashed white line, and upbound and downbound vehicles pass back and forth there. A waiting area is provided for the need for passing, with the upbound and downbound passing lines being dashed white lines, the upbound waiting lines being solid white lines, and the downbound waiting lines being solid colored (e.g. blue) lines, ensuring control over determining branching and merging.

Figure 4 (B) is an example of identifying entry and exit from straight and small curvature curved roads to large curvature curved roads by the type of line. Straight and small curvature curved roads are shown as white dotted lines, which meet the branching and merging requirements described above, while large curvature curved roads, which require relatively strict route control, are shown as solid colored lines (e.g., blue).

As mentioned above, the dashed route drawn on the road surface is captured by cameras installed under the floor and in addition to the front of the vehicle, along with lighting, and the route is followed. In addition to the current location and current lateral deviation, angular deviation, and route curvature of the vehicle relative to the route line captured by the camera, route information is expressed by the type of route line (solid line, dashed line) and color of the route line (white, blue), making it possible to control the vehicle to anticipate and prepare for the route information ahead.

In order to travel along the route at the planned vehicle speed and with appropriate acceleration/deceleration and lateral acceleration, the drive and braking systems are controlled to accommodate changes in the gradient of the route (control of drive gear selection, accelerator opening, engine brake, auxiliary brake, and wheel brake), and the drive, braking, and steering systems are controlled to accommodate changes in the curvature of the route.

The route includes branching stops for entering, stopping, and departing at bus stops, decelerating, steering, stopping, starting, steering, and accelerating, boarding and alighting, merging for starting, cruising on single-track sections, and diverging, waiting, and merging at single-track waiting areas.

In both Figures 4 (A) and (B), the main route is represented by a dotted line, and the dotted lines are counted and associated with route information such as distance from the starting point, gradient, curvature, and command speed according to their rank. Branches are represented by white solid lines and colored solid lines. Furthermore, solid lines are used for sections with large curvature to further improve the detection accuracy of the lateral deviation e2, angular deviation e3, and curvature by the camera. Instead of counting dashed lines, wheel pulses can also be counted. The dashed lines can also be solid lines.

As described above, by installing cameras under the floor in addition to the front, the system follows colored route lines painted on the road surface, resolving the issue with the camera system of being exposed to disturbances from the natural environment and being unable to recognize white lines.

The camera is mounted in a position that can accommodate the route line having a curvature from the minimum turning radius of the vehicle to straight ahead in its field of view, and detects the lateral deviation e2, angular deviation e3, and route line curvature between the vehicle's center of gravity position and the route line to perform feedback control, and also obtains route information such as distance, gradient, curvature, planned speed, branch position, merging position, and latitude and longitude orientation from the route line to perform feedforward control. Thus, it functions as a route recognition system for an autonomous vehicle equipped with a visual device that follows a driving route laid on a road.
This corresponds to the case where it functions as a route recognition system using a front-of-vehicle camera, or the case where it functions as a parallel redundant system using a front-of-vehicle camera and an under-floor camera.

Claims (6)

A route recognition system for autonomous driving that travels along a route line drawn on the road surface, in which a camera is attached under the floor of the vehicle body that can visually confirm the route line without being exposed to disturbances from the natural environment such as the effects of sunlight or shadows, can determine the relative position with respect to the vehicle's center of gravity, and can bring the center of gravity trajectory traced by the vehicle's center of gravity within the visible range in a plan view, detects the lateral deviation (e2) between the route line and the center of gravity of the vehicle and the angular deviation (e3) of the vehicle axle relative to the route line by referring to the position of this camera, and performs steering control to eliminate the detected lateral deviation (e2) and angular deviation (e3) and align the center of gravity of the vehicle with the route line. 2. The route recognition system for automatic driving according to claim 1, wherein the lateral deviation (e2) of the center of gravity of the vehicle and the angular deviation (e3) of the vehicle axle relative to the route line are substituted into equation (4) to calculate the yaw angular velocity (yaw rate) of the vehicle, the yaw angular velocity (yaw rate) is substituted into equation (5) to calculate the turning curvature of the vehicle, and the yaw angular velocity (yaw rate) and the turning curvature are substituted into equation (6) to calculate the front wheel steering angle.
2. The route recognition system for autonomous driving according to claim 1, wherein the lateral deviation (e2) of the center of gravity of the vehicle and the angular deviation (e3) of the vehicle axle relative to the coordinates and orientation of the route line are substituted into equations (10) and (11) to calculate the coordinates of the center of gravity position (center of gravity point) of the vehicle, the straight-line distance from the center of gravity point coordinates to the route junction coordinates are calculated using equation (12), the route arc angle is calculated using equations (13), (14), (17), and (18), and the route curvature is calculated using equation (15), and the actual front wheel steering angle reaching the route junction is calculated using equation (16).
The route recognition system for automated driving described in claim 1, characterized in that the route lines on the road surface have different colors and shapes for each piece of route information. 2. The route recognition system for automated driving according to claim 1, wherein the route lines are dotted or colored, enabling the advance transmission of curvature change information, road gradient change information, planned vehicle speed information, route branch information, route merging information, and longitude, latitude, and orientation, thereby enabling look-ahead control. The route recognition system for an autonomous vehicle according to claim 1, characterized in that the camera has a field of view of a length and width that can capture the arc trajectory of the vehicle's center of gravity, the center of which is on the extension line of the rear axle, when turning with the minimum turning radius.

Images