JPH0812686B2 - Road recognition method - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a road recognition method for recognizing the shape of a road based on image information obtained by imaging the road.

[Conventional technology]

In order to automatically drive a vehicle (automobile) on a road (road), it is necessary to recognize the shape of the road in real time. As such a traveling road recognition method, for example, a traveling road in front of the vehicle may be imaged by a TV camera or the like, and the obtained data (image surface) may be image-processed.

FIG. 10 shows the image plane of the road to be discriminated. In the figure, HL corresponds to the horizon, and LL and RL correspond to the left and right road edges. SR is the confluence of the roads. In addition to the straight road as shown in FIG. 10A, a curved road (shown in FIG. 11B), a branch road (shown in FIG. 13C), and a branch flow path from the main line (shown in FIG. (Figure (d) shown)
And a confluent channel to the main line (shown in (e) and (f) of the figure).

[Problems to be Solved by the Invention]

It is not easy to recognize such a wide variety of shapes of traveling paths.

For example, the edges corresponding to the ends LL and RL of the traveling road are extracted at the lower end of the image plane (the front side of the traveling road), and the traveling road edges LL and RL are located above the image plane (the traveling road) based on the extraction result. The method of tracking toward the front side) can be considered. However, with this method, it is extremely difficult to recognize a merged channel as shown in FIGS. 10 (e) and 10 (f), for example.

Therefore, it is an object of the present invention to provide a traveling road recognition method capable of accurately and easily recognizing various shapes of traveling roads.

[Means for solving the problem]

A traveling road recognition method according to a first aspect of the present invention is a traveling road recognition method on two-dimensional image information composed of image information obtained by capturing an image of a traveling road. The first step of setting at least one window to be searched and tracked, and when a discontinuity of a predetermined portion of the traveling road is detected in this window, the search range by the window is expanded to the surrounding area to determine a predetermined vicinity of the discontinuity. A second step of detecting the portion.

A method for recognizing a road according to a second aspect of the present invention, on a two-dimensional image plane composed of image information obtained by imaging a road,
In a method for recognizing an end of a road, a window for searching an end of the road on an image plane is set.
Steps and while exploring the edge of this window,
The second step of setting new windows one after another in the direction in which the end extends and searching for the end, and when a break in the end of the road is detected in the window, The third step of setting a new window in the direction and searching for another road edge in the above direction, and when another road edge is detected near the point where the road edge is interrupted, the other road The fourth to pursue the roadside
And the steps of.

(Operation) According to the traveling road recognition method of the first aspect of the present invention, since a local area called a window is a processing target, processing data is reduced. Further, by expanding the search range of the window and detecting the running path that should be continued, it is possible to easily recognize the shape such as the joining of the running paths.

Further, according to the roadway recognition method of the second aspect of the present invention, since the window is set to search for the roadway end, the data to be processed is reduced and the speedup is achieved. Then, since the processing of such windows (small areas) is piled up and the entire traveling road shape is recognized, highly accurate traveling road recognition can be performed. Also, if you find a break in the road edge on the window, the window is set virtually and the search process is executed to find another road edge, so
It is possible to easily recognize variations such as various mergings of the road shapes.

〔Example〕

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

The traveling road recognition method of the embodiment whose main part is shown in FIG.
In the system shown in FIG. 2, FIGS.
It is applied while the series of processes in the figure is executed. Therefore,
These will be described in order.

FIG. 2 is a block diagram of a system to which the embodiment method is applied. In the figure, a camera 11 captures an image of a road ahead of the vehicle and outputs analog image data, and the output is converted by an A / D conversion board 12 into digital image data of 256 gradations. Then, spatial differentiation is performed by the Sobel board 13, and a place (edge) where the density of the image surface suddenly changes is detected. Processing result of Sobel board 13 is multi-valued memory 1
Stored in 4 with 256 gradations. Next, the data stored in the multi-valued memory 14 is sent to the binarization board 15 and the binary data (0,
It is stored in the binarization memory 16 as 1).

The processing executed in the configuration from the camera 11 to the binarization memory 16 is the processing in the previous stage of the present invention, and is schematically shown in FIG. That is, the analog moving image captured by the camera 11 is first digitized in the directions of the space axis and the time axis, and then digitized into 256 gradations with respect to the gray value. After that, by binarizing, binary image data of 512 × 512 pixels is obtained. Processing for this binary image data (main processing)
Is performed by the CPU 17 in FIG.

Next, prior to the description of the present processing, the tracking processing and the window processing applied by the traveling road recognition method according to the present invention will be described.

Extracting the features (for example, the boundary line of an object) from an image is essential for processing the image. There is an edge in the typical one of the features of the image, and in order to extract this, in the embodiment, the image is spatially differentiated,
Adopt a method of detecting edges. Edges are detected from where features (generally density) change abruptly, which form the boundaries (contours) of the object. Specifically, since the density of the image surface changes rapidly on the white line (shoulder line) of the road, the road edge can be recognized by detecting the edge.

[Sequential Tracking Process] The edge detected by the feature extraction is detected as a line by the following sequential tracking type process.

(Step-i) Select a pixel to start processing.

(Step-ii) A pixel to be processed next is obtained (searched) based on the pixel processed immediately before, and a predetermined process is performed.

(Step-iii) When processing is performed on the next pixel, step-ii is repeated or the processing is ended.

The above tracking process has the following features. First,
Which pixel is to be processed depends on the previous processing result (that is, the position of the start pixel). Secondly, it is possible to limit the processing range by utilizing the processing result before that, and thus it is possible to avoid performing unnecessary processing. Further, the processing accuracy can be improved by limiting the processing range.

A specific example of the tracking process will be briefly described with reference to FIG.
First, the edge to be tracked (binary data is not "0" but "1"
However, it is assumed that it has become like the hatching in FIG. First, the start pixel is selected for the cross-hatching in FIG. 9B, and the edge continuous with this is searched. This search is performed for the pixel of the (*) mark in the figure, whereby the pixel of the next edge can be detected as shown in (c) of the figure. The next edge search is preferentially performed in the direction in which the edge arrangement is predicted, as indicated by the (*) mark in FIG. Hereinafter, the processing range is limited by using the processing results before that, and the edges are searched as shown in (d), (e), and (f) of FIG.

Here, in FIG. 4 (f), since there is no edge in the continuous pixels, the tracking process cannot be performed. Therefore, in this case, the search range is jumped by one pixel as shown in FIG. Then, the next edge can be detected as shown in FIG. If a jump for one pixel cannot be detected, a jump for two pixels is performed.

[Window Processing] In the present invention, the edge tracking is not performed on the entire image surface, but is performed only on a part of the image surface by the window processing. As a result, it is possible to extract and process only the range in which the object exists on the image surface.
Further, in the present invention, based on the above-mentioned sequential tracking processing,
The contour line of the object is extracted.

(Setp-i) First, a window to be processed is set based on pixels satisfying appropriate conditions.

(Setp-ii) The pixel is set as a start pixel, the pixel to be processed next is obtained, and the edge is traced.

(Setp-iii) If the tracked pixel does not go out of the window, the process of step-ii is repeated. If it is outside the window, return to step-i and set the next window newly. If there is a break in the edge, a few pixels are moved (jump) in a direction in which there is a high possibility of having an edge, and the edge is searched by scanning in the horizontal direction.

(Setp-iv) Repeat setp-ii.

 The features of this method are shown below.

Since the processing results before that are used, it is possible to limit the processing range, and it is possible to avoid wasteful processing.

Since the processing window is set, it is not affected by information outside the window.

Fine tuning is possible depending on how the processing window is set.

 It is also effective for discontinuous lines.

Next, a specific example of the sequential window (small area) tracking processing will be described with reference to FIG.

First, as shown in FIG. 7A, the window WD ₁ is set around the pixel G ₁ tracked in the previous process. Then, only performs tracking processing in the window WD _1, this window when initially prompted pixels G ₂ exiting out of the WD within _1, a new window WD ₂ around the pixel G ₂ as shown in FIG. (B) To set. Window WD ₂
The tracking is performed in the same manner as in the window WD ₁ , but here, as shown in FIG. 7C, the edge that should follow the pixel G ₃ cannot be detected from the adjacent pixel and the pixel jumped by one pixel. To do. In this case, it sets the next window WD ₃ on the window WD _2, searches the edge figure shown in (c) (*) marks the pixels. Then, the pixel G ₄ shown in FIG. 7D can be detected, and the search in the window WD ₃ is continued using this as a base point. Then, when the last pixel G ₅ in the window WD ₃ can be detected, the next window WD ₄ is set thereon.

Next, the main processing of the traveling road recognition method according to the present embodiment will be described.

FIG. 6 is a flowchart showing the whole of this processing. As shown, this process consists of an initial process (step 601), which is shown in detail in FIG. 7, and left and right tracking (steps 602 and 603), which includes left (right) side tracking, which is shown in detail in FIG. .

The initial processing will be described in detail with reference to FIGS. 7 and 9.

First, when the camera 11 is placed at an appropriate height on the road (traveling road) and oriented in the substantially extended direction of the road, the white lines on the left and right of the road (running road edges LL, RL) are as shown in FIG. 9 (a). , Respectively to the left and right of the center of the screen. Taking advantage of this property,
An initial window WD _0L , WD _{0R on} each of the left and right at the bottom of the image plane (immediately above the screen if part of the car moves to the screen)
Is set (step 701 in FIG. 7). Next, the processes of steps 702 to 705 and the processes of steps 706 to 709 in FIG. 7 are successively performed. That is, as shown in FIG. 9 (a), there is no edge in the road, so when the inside of the initial windows WD _0L and WD _0R is scanned from the center side to the outside, the first edge found is the white line (LL , RL) edge. Actually, a small noise (No. 9
Considering the existence of G _b and G _{c in} FIGS. 2B and 2C, when edges are found to be gathered to some extent, it is determined that the line is a white line corresponding to the road edges LL and RL. (Step 7
02,706).

Further, when an edge due to a white line on the road is found, tracking is started in the upward direction (forward direction), and ends when outside the initial windows WD _0L and WD _0R . If the edge cannot be found in one scan, move a few pixels above the screen,
Scan similarly, find edge or initial window
This scan is repeated until it goes out of WD _0L , WD _0R . When an edge cannot be found in the initial window WD _{0L as} in FIGS. 9A and 9B (steps 703 and 707), the virtual tracking point E _0L is set at the outer corner of the upper side of the window WD _0L ,
Treat the edge as if it was found (steps 704, 70)
8). However, this virtual tracking point E _0L is not output as a point sequence.

Next, the left (right) tracking will be described with reference to FIGS. 8 and 9.

The edge traced by the initial processing (E _0L ,
Based on E _0R ), the left and right initial windows WD _0L , WD
Set small left and right windows WD _IL and WD _IR on _0R . As for setting, as shown in FIGS. 9B and 9C, the start pixel is set to the center of the lower side of the windows WD _IL and WD _IR . After that, a small area (window) is set in the same manner as above, based on the edge traced last time. Left, right window _{WD 1L, WD 2L, ......,} WD 1R, WD 2R, size the initial window WD _0L of ..., the reason for the smaller than the WD _0R is to reduce the influence of noise. If the window WD to be set extends from the left and right sides of the image surface, the window WD is set to the inside of the image surface so that it does not extend.

In the window WD, the above-mentioned tracking processing is performed from the disclosed pixel, and ends when it goes out of the window being tracked,
Sets the next new window. The search is basically performed by alternating right and left on a window-by-window basis (FIG. 6), and the central processing described below is performed once for each left and right tracking. When the edge is completely lost or the set number of left and right windows reaches a predetermined number (obtained from the position of the horizon), the search ends. Further, only when the edge goes out from the left and right of the screen while tracking the edge (step 802), the virtual search point is set similarly to the initial processing (step 803). By doing this, it is possible to find an edge that enters the screen again after it has gone out of the screen.

Next, an essential part of the embodiment of the present invention will be described with reference to FIG.

1 (a) to 1 (d) show a procedure for recognizing a traveling road according to the embodiment. The traveling road to be recognized is a straight road as shown in the figure, and one road merges from the left side. First, the left and right windows WD _1L , WD _1R are set on the lower side of the screen as shown in FIG. 9, and the windows WD _2L , WD _2R , WD _3L , WD _3R , ... are set one after another on top of this. , The end of the road is searched and tracked. Here, it is assumed that the left end of the road is interrupted in the windows WD _5L and WD _{5R as} shown in Fig. 1 (a). In this case, as shown in FIG. 1B, the left windows WD _6L , WD _7L , ... _Are virtually set one after another in the upper direction of the screen with the left window WD _5L as a base point, and the end portions are searched. At this time, the right edge is tracked by the right windows WD _6R , WD _7R , .... Then, another road edge is found when the window WD _12L is set as shown in FIG. 1 (b). Therefore, as shown in FIG. 1 (c), the search and tracking of the traveling path are restarted.

If the edge at the end of the search and tracking is represented as a point sequence, it becomes as shown in FIG. 1 (d). The point sequence on the left side is curved at the confluence, but the shape of the road can be roughly recognized.

FIGS. 1 (e), (f), (g), and (h) illustrate cases in which different ways are used for recognizing a traveling path that joins a straight road.

First, as shown in FIG. 1 (e), the windows WD _1L , W
Set D _1R , WD _2L , WD _2R , ... sequentially and trace the end of the road. If you find a break in the road edge in the right window WD _3R , move upward to the right window WD _4R , WD as in Fig. 1 (b).
Set _5R , ... Then, when another road edge is found in the right window WD _9R , the search and pursuit along the edge is restarted. As a result, a point sequence as shown in FIG. 1 (f) can be obtained.

In the method of FIG. 1 (g), the direction of virtual setting of windows is different. That is, in the above-described example, the virtual setting of the window is performed above the screen, but in the case of FIG. 1 (g), the virtual setting is performed along the direction in which the running end extends. . According to this method, the obtained edge point sequence is shown in FIG.
As a result, the shape of the confluence portion of the traveling road can be recognized as a smoother shape.

The direction of virtual setting is not limited to these, and may be an intermediate direction between the directions shown in FIGS. 1 (e) and 1 (g).

〔The invention's effect〕

As described above in detail, according to the present invention, the window is set to search the road edge, so that the data to be processed is reduced and the processing speed is increased. Then, since the high-precision window processing is accumulated to recognize the entire traveling road shape, the traveling road can be discriminated with high accuracy. Further, when a break in the road edge is found in the window, a forward search process is executed to find another road edge, and therefore, there is an effect that various merging variations of the road shape can be easily recognized.

[Brief description of drawings]

FIG. 1 is a diagram for explaining a main part of a roadway recognition method according to an embodiment of the present invention, FIG. 2 is a configuration diagram of a system to which the roadway recognition method is applied, and FIG. Figure to explain,
FIG. 4 is a diagram for explaining the edge tracking process, FIG. 5 is a diagram for explaining the sequential small area tracking type window process, and FIG. 6 is a flowchart showing the overall process of the road recognition method according to the embodiment. , FIG. 7 is a flow chart showing the initial processing,
FIG. 8 is a flowchart for left (right) tracking, FIG. 9 is a diagram for explaining initial processing and window setting, and FIG. 10 is a diagram showing various traveling paths to be recognized. 11 …… Camera, 12 …… A / D conversion board, 13 …… Sobel board, 14 …… Multi-valued memory, 15 …… Binarization board, 16 ……
Binary memory, 17 …… CPU, WD …… Window, HL ……
Horizon, LL ... left end of road, RL ... right end of road.

Claims (3)

[Claims] 1. A method for recognizing a road on a two-dimensional image information consisting of image information obtained by picking up an image of a road, wherein at least one window for searching and tracking a predetermined portion of the road on the image plane. And the step of expanding the search range by the window from the position where the interruption is detected to the front side of the traveling path when the interruption of a predetermined portion of the traveling path is detected by the window. A second step of detecting the predetermined portion in the vicinity of the traveling path recognition method. 2. The second step is a step of sequentially expanding the search range by the window in a direction in which the extension of the traveling path is predicted by using the past processing results. The described road recognition method. 3. Image information obtained by imaging a traveling road 2
A method for recognizing an end of the road on a three-dimensional image surface, comprising: searching the end of the road on the image surface.
A first step of setting two windows; a second step of searching for the end portion in the window and setting new windows one after another in a direction in which the end portion extends, and tracing and searching the end portion; When detecting a discontinuity of the end of the traveling road in the window, at least a new window is set in the direction in which the traveling road end extends or in the front side, and a third traveling road end is searched in the direction. And a fourth step of tracking and searching for another running road end when another running road end is detected in the vicinity of the point where the running road end is interrupted. How to recognize the road.