JPH0510685B2 - - Google Patents

Description

[Detailed Description of the Invention] Purpose of the Invention (Field of Industrial Application) The present invention relates to an image-based steering control device for an unmanned vehicle, and more specifically, it relates to an image-based steering control device for an unmanned vehicle. The present invention relates to a safety device for driving along a travel route determined by image processing of an image of a travel line taken by an imaging device.

(Prior art) In recent years, unmanned vehicles have been using an imaging device equipped with an imaging device to capture a driving line that indicates the driving route of an unmanned vehicle, and then image-processing the image of the driving line taken by the imaging device to drive the vehicle along the determined driving route. Image-based unmanned vehicles have been proposed. In this type of image-based unmanned vehicle, the driving line in front of the vehicle is simply imaged, the image is processed, and the steering angle and the like are determined from the deviation of the driving line in the image.

(Problem to be solved by the invention) However, in image-based unmanned vehicles, the imaging device may not be able to capture the driving line accurately or at all for some reason. Image processing is performed and the steering mechanism is operated based on the processing results. Therefore, some kind of safety measures have been necessary for image-based unmanned vehicles.

An object of the present invention is to provide a safety device for an image-based steering control device for an unmanned vehicle that allows the unmanned vehicle to travel safely even when the imaging device cannot accurately or at all photograph the traveling line for some reason. .

Structure of the Invention (Means for Solving Problems) In order to achieve the above object, the present invention provides an imaging device that is mounted on an unmanned vehicle and captures an image of a traveling line indicating the traveling route of the unmanned vehicle while traveling; an imaging control means for controlling the imaging device to cause the imaging device to capture a new imaging position at least twice before the unmanned vehicle reaches the imaging position previously captured by the imaging device; a discriminating means for discriminating each pixel signal of an image as to whether it is the traveling line or not and encoding it into pixel data; and a discriminating means for discriminating each pixel signal of an image as to whether it is the traveling line or not and encoding it into pixel data; a storage means for storing information, an determining means for calculating a travel route based on a group of pixel data of the latest image newly stored in the storage means, and a determining means for determining a travel route based on a group of pixel data of the latest image newly stored in the storage means; When determining a route, a first steering control means operates a steering mechanism based on data of the latest determined travel route, and the determining means determines the latest travel route based on a group of pixel data of the latest image. The present invention provides a safety device in a steering control device for an image-based unmanned vehicle, which comprises a second steering control device that continues to operate the steering mechanism based on the data of the travel route previously determined by the indexing device when the vehicle cannot be used. That is.

(Function) When the indexing means determines the latest travel route based on the image data group of the latest images, the first steering control means operates the steering mechanism based on the data of the determined latest travel route. On the other hand, when the determining means is unable to determine the latest travel route based on the pixel data group of the latest image, the second steering control means The steering mechanism is subsequently operated based on the data of the previous travel route determined from the pixel data group of the previous image.

(Example) Hereinafter, an example of a safety device in a steering control device for an unmanned vehicle embodying the present invention will be described with reference to the drawings.

In FIG. 1, a support frame 2 is erected at the upper center position on the front side of the unmanned vehicle 1, and an image pickup device is installed at the upper center position of the frame 2.
A CCD (charge coupled device) camera 3 is provided. The CCD camera 3 is set on the support frame 2 so as to photograph an area 4a on the road surface 4 several meters ahead of the unmanned vehicle 1. and,
The CCD camera 3 covers the area 4a shown in FIG.
This is captured in image 9 shown in the figure. Further, in this embodiment, it is assumed that the image 9 of the area 4a is captured with 256×256 pixels.

Illumination lamps 5 are installed below both sides of the CCD camera 3, and are set to illuminate the area 4a. Incidentally, in the embodiment described above, an illumination lamp was used to facilitate imaging, but the present invention may be carried out without using this. On the road surface 4, as shown in FIG. 2, a travel line 6 indicating the travel route of the unmanned vehicle 1 is drawn with a constant line width D, and in this embodiment, a white paint different from the color of the road surface 4 is drawn. It is depicted in. Then, the CCD camera 3 photographs the traveling line 6 having the constant line width D.

Note that the level of the signal (hereinafter referred to as a pixel signal) from the CCD camera 3 that photographed the white running line 6 is high, and on the contrary, the level of the pixel signal from the CCD camera 3 that photographed the dark road surface 4 is low.

Next, the electrical configuration of the safety device in the steering control device mounted on the unmanned vehicle 1 will be explained with reference to FIG.

The microcomputer 10 includes a central processing unit (hereinafter simply referred to as CPU) 11, a program memory 12 consisting of a read-only memory (ROM) that stores control programs, and a readout memory 12 that temporarily stores arithmetic processing results of the CPU 11, pixel data, etc. Consists of a working memory 13 made of rewritable memory (RAM), a timer 14, etc.
The CPU 11 determines a travel route using a control program stored in a program memory 12 and executes various arithmetic processing operations for steering control.

The CPU 11 connects the input/output interface 15 and the A/
The CCD camera 3 is scan-controlled via a D converter 16, and in this embodiment, as shown in FIG. 6, a new area 4a is formed in which the previous area 4a partially overlaps. , and the image is taken several times (twice in this embodiment) before the unmanned vehicle 1 reaches the imaged area 4a. For convenience of explanation, this CCD is used in this example.
The scanning control of the camera 3 is performed at fixed intervals based on the time measured by the timer 14 on the premise that the unmanned vehicle 1 always travels at a constant speed, but the traveling speed of the unmanned vehicle 1 is not constant. In some cases, it is controlled by the traveling speed and time or by an odometer. The scanning control of the CCD camera 3 uses a scanning method in which scanning is performed in the horizontal direction (X-axis direction) and the scanning moves from the top of the screen 9 to the bottom (Y-axis direction), but this is not limiting. However, other scanning methods may be used instead.

Further, the A/D converter 16 converts pixel signals from each image sensor of the CCD camera 3 into pixel data, and stores the pixel data in the working memory 13 via the bus controller 17. When converting pixel signals from the CCD camera 3 from analog values to digital values, the A/D converter 16 determines whether each pixel signal is greater than or equal to a predetermined set value. Each pixel that is input sequentially is set to "1" as a pixel in the white driving line 6, and "0" as a pixel in the dark road surface 4 if the pixel signal is less than 1. The signal is binarized and stored as pixel data in a predetermined storage area 13a of the working memory 13. Therefore, the image 9 taken by the CCD camera 3 is stored in the working memory 13 as a 256×256 pixel data group.

Each time a new image 9 pixel data group is input, the previous image 9 pixel data group is rewritten to the newest image 9 pixel data group.

A binarization level controller 18 converts setting value data for binarization by the A/D converter 16 to the A/D converter 16 based on a control signal from the CPU 11.
Output to converter 16. Drive controller 2
0 controls a driving motor and a steering mechanism 21 (not shown) based on control signals from the CPU 11 as well. The steering mechanism 21 then controls the steering angle based on the control signal. or,
The CPU 11 inputs a detection signal from the speed detector 22 that detects the rotational speed of the drive system of the unmanned vehicle 1 via the input/output interface 15, and calculates the running speed V of the unmanned vehicle 1 at the time. ing.

Next, the processing operation of the CPU 11 will be explained.

Now, based on the control signal from CPU11, the CCD
When the camera 3 is scan-controlled, the CCD camera 3 is not perpendicular to the road surface 4, but is tilted at a certain angle to capture an image, so the front area 4a shown in FIG. The fifth with different
An image 9 as shown in the figure is captured. The image 9 captured by this CCD camera 3 is converted into an A/D signal as a pixel signal.
It is output to the converter 16, and the A/D converter 16 determines whether each pixel signal is a pixel signal of the traveling line 6 or a pixel signal of the road surface 4 and stored in the working memory 13 as pixel data. be done.

The CPU 11 performs image recognition of the travel line 6 based on pixel data stored in a predetermined storage area 13a of the working memory 13. The CPU 11 sequentially reads out each pixel data in the scanning direction (X-axis direction) from the top of the image 9 and determines whether there is a running line 6 having a constant line width D and at which position in the scanning line it is located. The center position of the range determined to be the travel line 6.
Find the position of Cn and working memory 13
to be memorized.

As shown in Figure 8, in each pixel that makes up image 9, the 128th vertical pixel column counting from the left is taken as the y-axis, and the 128th horizontal pixel column counting from the top is used as the y-axis. The position and center position Cn of the running line 6 are determined by the x, y coordinates, with each pixel of the image 9 being expressed by the x, y coordinates, defined as the x axis. Then, the center positions C0 to C255 of each scanning line up to the bottom of the image 9 are determined.

Next, the CPU 11 determines the center position C0~
A plurality of center positions are selected from C225 as selection points. In this embodiment, the scanning line (y-axis direction) to be selected is determined in advance, and four scanning lines are selected in advance so that when image 9, which will be described later, is projectively transformed, all adjacent selected points are equidistant with respect to the traveling direction. Scan line La
The center positions Ca, Cb, Cc, and Cd of ~Ld are defined as selection points Z1 to Z4, as shown in FIG.

The CPU 11 projects the selected points Z1 to Z4,
That is, calculation processing is performed to determine at which positions (hereinafter referred to as base points) Q1 to Q4 on the actual area 4a shown in FIG. 9 the selected points Z1 to Z4 found in the image 9 are located. This is a CCD as mentioned above.
This is a process to match the difference between the traveling line 6 in the image 9 and the actual traveling line 6 in the area 4a since the camera 3 does not image the road surface 4 vertically. Note that this projective transformation processing operation is performed based on the preset focal length and tilt of the CCD camera 3,
Projective transformation, ie, coordinate transformation, is performed based on installation conditions such as height. The general formula for this projective transformation is as follows.

Let the position coordinates of the selected point shown in the input image of the CCD camera 3 be x, y, and let the position coordinates of the base point to be converted into the actual area 4a be the X, Y coordinates with the position of the CCD camera 3 as the origin. Furthermore, let H be the height of the CCD camera 3, θ be the inclination of the CCD camera 3, and F be a constant that determines the magnification of the corresponding position.

X=x・secθ/1−(y/F)tanθ・H/F Y=y+F・tanθ/1−(y/F)tanθ・H/F Next, the CPU 11 selects these four base points Q1 to Q4.
As shown in FIG. 9, the area 4a is divided into eight areas in the X-axis direction, and it is determined which of the divided areas B1 to B8 the area belongs to. In this embodiment, the judgment as to whether it belongs to any of these is made in advance by the position of the center of each area on the partition line that partitions each area B1 to B8 and the lines a to d on the area 4a corresponding to each of the scanning lines La to Ld. For example, as shown in FIG. 10, the distances D3 and D4 to the center positions P3 and P4 of each region B3 and B4 on both sides of the base point Q3 are calculated, respectively, and the center located at the closest distance D4 is determined. The area B4 on the side of the position P4 is determined to be the area to which the base point Q3 belongs.

On the other hand, each of the divided regions B1 to B of the four base points 1 to Q4, including the case where the base points Q1 to Q4 do not exist,
The combination (pattern) to which 8 belongs is 9 to the 4th power (=
6561), and 6561 travel data corresponding to the 6561 patterns are stored in a predetermined storage area 12a of the program memory 12 as shown in FIG.
The CPU 11 searches for a pattern that matches the determined arrangement of base points Q1 to Q4, and reads out travel data of the matching pattern.

The travel data is the pattern, that is, the steering angle at times during the travel in order to rationally and efficiently travel from the current position to the imaged area 4a with respect to the travel line 6 based on the arrangement of the base points Q1 to Q4. The steering mechanism 19 is operated based on this data, which has been determined experimentally in advance. In addition, among the 6561 patterns, for example, as shown in FIG.
exists in regions B4 and B5 and only the base point Q1 does not exist in any region, when the inner base point Q3 does not exist as shown in FIG. 13, or as shown in FIGS. 14 and 15. This includes a pattern of the travel line 6 that is actually impossible, such as when the eccentric position between base points is large, and the travel data for that pattern is used for steering control assuming that the CCD camera 3 could not accurately image the travel line 6 for some reason. The data is used to indicate an imaging error, rather than the data indicating the imaging error.

Now, the CPU 11 searches for a pattern that matches the arrangement of the base points Q1 to Q4 based on the latest image 9 taken by the CCD camera 3, reads out travel data for that pattern from the program memory 12, and stores it in a predetermined storage in the working memory 13. When stored in the area 13b, the CPU 11 adjusts the steering mechanism 21 based on the latest travel data stored in the working memory 13.
Activate. That is, at this point, the travel route of the unmanned vehicle 1 in the latest imaging area 4a has been determined based on this travel data.

The CPU 11 causes the unmanned vehicle 1 to travel toward its imaging area 4a while controlling the steering angle based on the latest travel data.

During this travel, when a predetermined period of time has elapsed, the CPU 11 controls the CCD camera 3 to store a group of pixel data of an image 9 of a new area 4a ahead of the previous imaging area 4a in the working memory 13. Based on this latest pixel data group, the CPU 11 determines base points Q1 to Q4 in the same manner as described above, searches for a matching pattern, and reads travel data corresponding to the matching pattern. Then, the CPU 1 determines that the read driving data is data for steering control, not data indicating an imaging error.
1 judges, the CPU 11 uses the working memory 13
The content of is rewritten from the previous traveling data to this new traveling data, and the steering mechanism 21 is operated using this latest traveling data. That is, the driving of the unmanned vehicle 1 based on the previous driving data is changed to driving based on the driving data for the latest imaging area 4a during the driving.

After that, by repeating this, CPU11
Each time the CCD camera 3 images a new area 4a, it reads out new travel data based on the image of that area 4a and operates the steering mechanism 21.
An unmanned vehicle 1 is caused to travel along a travel line 6.

Furthermore, if the driving data read out by the CPU 11 through a pattern search is not data for steering control but data indicating an imaging error,
The CPU 11 immediately determines that the pixel data group in the latest imaging area 4a is not the correct data group, and continues to perform steering control based on the previously read travel data. Therefore, the unmanned vehicle 1 will travel based on the previous travel data.

At this time, the previous travel data is data up to area 4a corresponding to the previous travel data, and since the unmanned vehicle 1 has not yet reached area 4a corresponding to the previous travel data, the future travel data is Even when traveling based on the data, the unmanned vehicle 1 can safely travel to the area 4a corresponding to the destination travel data.

Then, when the unmanned vehicle 1 controls the CCD camera 3 again to image a new area 4a on the way to the area 4a corresponding to the previous travel data, the CPU 11 searches for a corresponding pattern in the same manner as described above, and The driving data corresponding to the pattern is read out and steering control is performed based on the latest driving data. Therefore, even if the previous pixel data group is incorrect, this can be compensated for and the unmanned vehicle 1
can be made to travel safely along the travel line 6. Note that when the next group of pixel data is also incorrect, the CPU 11 determines that there is no travel data ahead of the current position and stops the vehicle from traveling.

In this way, in this embodiment, before reaching the area 4a imaged by the CCD camera 3, a new area 4a is imaged twice, and the unmanned vehicle 1 is driven based on the latest pixel data group at the time of image capture. ,
When the latest running data is incorrect, the unmanned vehicle 1 is made to run based on the running data based on the previous pixel data group, so if the pixel data group in the area 4a imaged by the CCD camera 3 is incorrect for some reason. Even if it is incorrect, the travel data based on the previous pixel data group can be used as is until the travel data based on the latest image data group of the area 4a to be imaged next is read out.
The unmanned vehicle 1 can be driven safely.

Furthermore, in this embodiment, since the image 9 of the travel line 6 taken by the CCD camera 3 is patterned and travel data corresponding to the pattern is prepared in advance, the image processing operation for determining the travel route is extremely fast. It will be possible to do it. As a result, the speed of the unmanned vehicle 1 can be increased accordingly.

The present invention is not limited to the above-mentioned embodiment. For example, in the above-mentioned embodiment, the CCD camera 3 images a new area 4a twice before reaching the imaged area 4a. The number may be further increased. In this case, CCD
Even if the pixel data group of the image 9 captured by the camera 3 is incorrect two or more times in a row, this can be compensated for.

Further, in the above embodiment, the base point Q1 on the lines a to d
The running line 6 was patterned based on which area B1 to B8 Q4 belongs to, and for example, the area 4a is divided into squares as shown in FIG. The traveling line 6 can be patterned by finding blocks Ba to Be in which the area occupied by the line 6 is large, or the first
As shown in FIG. 7, in the above example, base points Q1 to Q4 on lines a to d in the X-axis direction were obtained, but lines L1 to L7 for defining base points Q1 to Q5 in the Y-axis direction were determined. The running line 6 may be patterned by setting a plurality of divided regions in the Y-axis direction.

Furthermore, in the above embodiment, the pattern is determined by projectively transforming the selected point selected in the image 9, replacing it on the actual area 4a as a base point, and searching for the arrangement of each base point at that time. Since there is only a certain relationship between 4a and the image, the array of selected points in image 9 is patterned in advance without projective transformation, and the travel data is generated based on the array pattern of the selected points. It may be decided. Further, in the embodiment described above, the steering control of the unmanned vehicle 1 was performed based on the patterned driving data, but a steering control device that calculates a function of the driving route from a group of pixel data and controls the steering of the unmanned vehicle based on the function, The present invention may be applied to a steering control device that controls the steering of an unmanned vehicle simply based on the amount of deviation of a travel line in an image.

Furthermore, in the embodiment described above, the traveling line 6 was imaged on the road surface 4, but a panel or the like depicting the traveling line may be disposed above the unmanned vehicle 1, or the traveling line may be drawn on the ceiling, and the traveling line 6 may be imaged on the road surface 4. It may also be an image-based unmanned vehicle that drives by capturing images.
In this case, the traveling line is less dirty than the traveling line on the road surface, and the traveling line can be imaged with high reliability without the risk of dirt being scattered near the traveling line.

Further, although a CCD camera was used as the imaging device in the above embodiment, it may be implemented using other imaging devices, and in the above embodiment, the CCD camera 3
The pixel configuration (resolution) of the image was 256 x 256 pixels, but it is not limited to this, for example.
Of course, the pixels may be changed to 512 x 512 pixels, 1024 x 1024 pixels, etc. as appropriate.

Effects of the Invention As detailed above, according to the present invention, image-based unmanned vehicle steering allows the unmanned vehicle to travel safely even when the imaging device cannot accurately or at all photograph the traveling line for some reason. It has excellent effects as a safety device in control equipment.

[Brief explanation of the drawing]

Figure 1 is a side view of an unmanned vehicle embodying this invention, Figure 2 is a plan view, Figure 3 is an electric block circuit diagram of the safety device, Figure 4 is a diagram showing the area imaged by the CCD camera, Figure 5 is an explanatory diagram to explain the image taken by the CCD camera, and Figure 6 is
Figure 7 is a diagram to explain the relationship between the number of images taken by the CCD camera and its area, Figure 7 is a diagram to explain the contents of the working memory, Figure 8 is a diagram to explain the coordinates of the image, and Figure 9 is a diagram to explain the image coordinates. The figure shows each base point when the selected point is projectively transformed into the actual area, Figure 10 is a diagram to explain the area to which the base point belongs, Figure 11 is a diagram to explain the contents of the program memory, FIGS. 12 to 15 are diagrams for explaining impossible travel line patterns, and FIGS. 16 and 17 are diagrams for explaining other patterns of travel lines. In the figure, 1 is an unmanned vehicle, 3 is a CCD camera, 4 is a road surface, 4a is an area, 5 is a lighting lamp, 6 is a running line, 9 is an image, 10 is a microcomputer,
11 is a central processing unit (CPU), 12 is a program memory, 13 is a working memory, 14 is a timer, 1
6 is an A/D converter, 18 is a binary level controller, 20 is a drive controller, 21 is a steering mechanism, and Q1 to Q4 are reference points.

Claims (1)

[Scope of Claims] 1. An imaging device that is mounted on an unmanned vehicle and captures an image of a traveling line that indicates the traveling route of the unmanned vehicle while the vehicle is traveling; and until the unmanned vehicle reaches the imaging position previously captured by the imaging device. an image capturing control means for controlling the image capturing apparatus to cause the image capturing apparatus to image a new image capturing position at least twice; and a code for determining whether each pixel signal of an image captured by the image capturing apparatus is on the traveling line or not. discriminating means for converting the pixel data into pixel data; storage means for storing the pixel data group of the image captured by the imaging device, determined by the discriminating means, in a pre-allocated storage area; determining means for determining a travel route based on a group of pixel data of an image; a first steering control means for operating the steering mechanism; and when the determining means is unable to determine the latest travel route based on the pixel data group of the latest image, the travel route previously determined by the determining means; and a second steering control means that continuously operates a steering mechanism based on the data.