JP7415853B2 - autonomous vehicle - Google Patents

Description

The present invention relates to autonomous vehicles.

SLAM (Simultaneous Localization and Mapping) is a technology for estimating the state (location information, etc.) of unmanned vehicles. SLAM is a technology that simultaneously estimates the self-position of a mobile object and creates an environmental map, and allows the mobile object to create an environmental map in an unknown environment. It uses the map information it has built to perform specific tasks while avoiding obstacles.

It is known to use road images as a technique for estimating the self-position of an autonomous vehicle (see, for example, Patent Document 1). This technology installs a camera on the underside of the vehicle body to take pictures of the road surface, detects characteristics from both pre-captured images (map images) and current road surface images (actual images), and then The self-position is estimated by comparison.

US Patent No. 8,725,413

However, as shown in FIG. 8, when estimating the self-position of an autonomous vehicle using features (patterns) F11 to F15 present in a road surface image P10 acquired by a camera, The position of the feature in the image differs depending on the illuminance of the light source. In particular, in an image taken of a dark spot on a road surface, for example, as shown in FIG. 9, objects that should be there may not be acquired as feature points, and the image may be characterized by noise N10. As a result, when estimating the self-position of the autonomous vehicle, the feature points due to the noise N10 are used, and there is a concern that the correct self-position of the autonomous vehicle cannot be estimated.

An object of the present invention is to provide an autonomous vehicle that can improve self-position estimation accuracy.

An autonomous vehicle for solving the above problems includes a camera installed on the lower surface of the vehicle body to photograph the road surface below the vehicle body, and a camera installed on the lower surface of the vehicle body that emits light into the photographing area of the camera on the road surface. A light source for irradiation, a storage unit for storing a map image taken of the road surface in advance, a feature extracted from the image of the road surface taken by the camera, and a feature extracted from the map image. an autonomous vehicle, comprising: a self-position estimating unit that estimates the self-position of the autonomous vehicle by comparing the self-position of the autonomous vehicle; a determination unit that detects noise by comparing the positions of points; and an image of the road surface that is obtained by the self-position estimating unit by masking the noise detected by the determination unit in the image of the road surface taken by the camera at the same geographical point. The gist of the present invention is to include a noise mask section having the following functions.

According to this, noise is detected by comparing the positions of feature points in a plurality of road surface images acquired by cameras at the same geographical point, and the noise in the road surface images is masked. As a result, by obtaining an image with less noise, the self-position estimation accuracy of the autonomous vehicle can be improved by making the self-position estimation less susceptible to noise.

Further, in the autonomous vehicle, the determination unit preferably detects noise based on distances between feature points in each of the images.
Further, in the autonomous vehicle, it is preferable that the determination unit detects noise based on a distance from a reference point on a plane forming an image of the camera to a feature point in each of the images.

According to the present invention, self-position estimation accuracy can be improved.

FIG. 1 is a side view of an autonomous vehicle in an embodiment. An enlarged view of part A in FIG. 1. A diagram showing an example of an image. A block diagram showing the electrical configuration of an autonomous vehicle. Flowchart for explaining the action. (a), (b), and (c) are diagrams showing examples of images. The figure which shows an example of the image for demonstrating another example. The figure which shows an example of the image for demonstrating a subject. The figure which shows an example of the image for demonstrating a subject.

An embodiment embodying the present invention will be described below with reference to the drawings.
As shown in FIG. 1, the autonomous vehicle 10 is a four-wheeled vehicle, and includes a vehicle body 11, a driving wheel 102 located at the bottom of the vehicle body 11, a steering wheel 105 located at the bottom of the vehicle body 11, Equipped with

As shown in FIGS. 1 and 2, the autonomous vehicle 10 includes a camera 20. The camera 20 is installed at the center of the lower surface of the vehicle body 11 and can photograph the road surface Sr below the vehicle body 11. FIG. 3 shows an example of an image P1 taken by the camera 20. Image P1 has a circular shape. In the image P1 of FIG. 3, the traveling direction of the autonomous vehicle 10 is the X direction, and the direction (width direction of the autonomous vehicle 10) orthogonal to the traveling direction (X direction) is the Y direction. In addition, in FIGS. 1 and 2, the traveling direction of the autonomous vehicle 10 in the horizontal direction is expressed as the X direction, the direction orthogonal to the X direction is expressed as the Y direction, and the vertical direction is expressed as the Z direction. There is.

As shown in FIGS. 1, 2, and 4, the autonomous vehicle 10 includes a light source 30 for illumination. The light source 30 is installed on the lower surface of the vehicle body 11. The light source 30 includes a plurality of light emitting diodes (hereinafter simply referred to as LEDs) 31. The light source 30 is for irradiating light onto the photographing area of the camera 20 on the road surface Sr. The light source 30 includes a plurality of LEDs 31 arranged on the lower surface of the vehicle body 11 so as to surround the camera 20 . As shown in FIG. 2, the optical axis Ax of each LED 31 extends directly below (in the vertical direction) from the LED 31.

Each LED 31 of the light source 30 lights up in synchronization with the photographing timing of the camera 20 and irradiates the photographing area of the camera 20 with light.
As shown in FIG. 4, the autonomous vehicle 10 includes a control device 40, motor drivers 100 and 103, a travel motor 101, and a steering motor 104. A camera 20 is connected to the control device 40 . The control device 40 is capable of driving the drive wheels 102 by controlling the travel motor 101 via the motor driver 100. The control device 40 is capable of driving the steered wheels 105 by controlling the steering motor 104 via the motor driver 103.

As shown in FIG. 4, the autonomous vehicle 10 includes an LED driver 90. The control device 40 can control each LED 31 that is the light source 30 via the LED driver 90.
As shown in FIG. 4, the control device 40 includes a processing section 50, a storage section 60, a determination section 70, and a noise mask section 80. The processing unit 50 includes a self-position estimating unit 51. The self-position estimation section 51 includes a feature extraction section 52 and a matching section 53. The determination unit 70 includes an inter-feature point distance measurement unit 71 and a comparison unit 72.

The storage unit 60 stores various programs for controlling the autonomous vehicle 10. The control device 40 may include dedicated hardware that executes at least some of the various processes, such as an application-specific integrated circuit (ASIC). The controller 40 may be configured as a circuit including one or more processors operating according to a computer program, one or more dedicated hardware circuits such as an ASIC, or a combination thereof. The processor includes a CPU and memory such as RAM and ROM. The memory stores program codes or instructions configured to cause the CPU to perform processes. Memory, or computer readable media, includes anything that can be accessed by a general purpose or special purpose computer.

The control device 40 operates the autonomous vehicle 10 by controlling the travel motor 101 and the steering motor 104 according to a program stored in the storage unit 60. The autonomous vehicle 10 of this embodiment is a vehicle that automatically runs and steers under the control of the control device 40 without any operation by a passenger.

The storage unit 60 stores a map image 61 obtained by photographing the road surface Sr in advance. The map image 61 is created with geographical location information linked thereto.
For example, as shown in FIG. The self-position of the autonomous vehicle can be estimated by comparing the The self-position is the coordinates indicating one point on the vehicle body 11, for example, the coordinates of the center of the vehicle body 11 in the horizontal direction.

Specifically, as shown in FIG. 3, the feature extraction unit 52 detects a feature point from the image P1, which is the current road surface image (actual image), and extracts the feature amount of the feature point, that is, the feature point. A feature value representing the degree of brightness of a certain pixel in surrounding pixels is detected. Similarly, the feature extraction unit 52 detects a feature point from a map image taken in advance, and also determines the feature amount of the feature point, that is, the magnitude of the brightness of the pixel surrounding the feature point. Detect the feature amount that represents the degree of Then, the matching unit 53 compares the feature amount of each feature point in the current road surface image (actual image) with the feature amount of each feature point in the map image taken in advance. Estimate your location.

When the map image 61 is stored in the storage unit 60 in advance, the coordinates of the road surface pattern are stored as an environmental map. The environmental map is created by mapping using SLAM (Simultaneous Localization and Mapping). SLAM simultaneously estimates the self-position of a mobile object and creates an environmental map, and allows the mobile object to create an environmental map in an unknown environment. Use the constructed map information to accomplish specific tasks. When estimating the self-position, the self-position of the autonomous vehicle is estimated by comparing the road surface image obtained by the camera with the map image obtained in advance.

The control device 40 can move the autonomous vehicle 10 to a desired position by controlling the travel motor 101 and the steering motor 104 while performing self-position estimation to estimate the position of the autonomous vehicle 10 on the map. It is possible.

Next, the effect will be explained.
FIG. 5 shows the processing executed by the control device 40.
In step S10 of FIG. 5, the control device 40 prepares a plurality of images taken at different times of the road surface Sr at the same geographical point (same location) on the travel path of the autonomous vehicle 10 using the camera 20. In the flowchart of FIG. 5, the plurality of images are three images P1a, P1b, and P1c.

An example of the process in step S10 will be described using FIGS. 6(a), 6(b), and 6(c).
A plurality of images P1a, P1b, and P1c at different times are shown in FIGS. 6(a), 6(b), and 6(c). FIG. 6(a) is an image P1a at time t1. FIG. 6(b) is an image P1b at time t2 (=t1+T1) after a certain period of time T1 has elapsed from time t1. FIG. 6(c) is an image P1c at time t3 (=t2+T2) after a certain period of time T2 has elapsed from time t2.

In step S11 of FIG. 5, the control device 40 extracts the feature points of each of the images P1a, P1b, and P1c, and records the position of each feature point in the image.
An example of the process in step S11 will be explained using FIGS. 6(a), 6(b), and 6(c).

6(a) has feature points Pf1 to Pf8, FIG. 6(b) has feature points Pf11 to Pf18, and FIG. 6(c) has feature points Pf21 to Pf28.

In step S12 of FIG. 5, the control device 40 calculates the distance for each feature point of each of the images P1a, P1b, and P1c.
An example of the process in step S12 will be described using FIGS. 6(a), 6(b), and 6(c).

In FIG. 6(a), distances L1 to L7 are measured for each feature point. That is, the distance L1 between the feature point Pf1 and the feature point Pf2, and the distance between the feature point Pf2 and the feature point Pf3, using the feature point Pf1 at the lower left in the figure as a common reference, and measuring the feature points closer to each other in order. L2, distance L3 between feature point Pf3 and feature point Pf4, distance L4 between feature point Pf4 and feature point Pf5, distance L5 between feature point Pf5 and feature point Pf6, feature point Pf6 and feature point Measure the distance L6 between the point Pf7 and the distance L7 between the feature point Pf7 and the feature point Pf8. In FIG. 6(b), distances L11 to L17 for each feature point are measured. That is, using the feature point Pf11 at the lower left in the figure as a common reference, and using the feature points that are closer in order as measurement targets, the distance L11 between the feature point Pf11 and the feature point Pf12, and the distance between the feature point Pf12 and the feature point Pf13. L12, distance L13 between feature point Pf13 and feature point Pf14, distance L14 between feature point Pf14 and feature point Pf15, distance L15 between feature point Pf15 and feature point Pf16, feature point Pf16 and feature point A distance L16 between the feature point Pf17 and a distance L17 between the feature point Pf17 and the feature point Pf18 are measured. In FIG. 6(c), distances L21 to L27 for each feature point are measured. That is, the distance L21 between the feature point Pf21 and the feature point Pf22, and the distance between the feature point Pf22 and the feature point Pf23, using the feature point Pf21 at the lower left in the figure as a common reference, and measuring the feature points closer to each other in order. L22, distance L23 between feature point Pf23 and feature point Pf24, distance L24 between feature point Pf24 and feature point Pf25, distance L25 between feature point Pf25 and feature point Pf26, feature point Pf26 and feature point A distance L26 between the feature point Pf27 and a distance L27 between the feature point Pf27 and the feature point Pf28 are measured.

In step S13 of FIG. 5, the determination unit 70 of the control device 40 determines feature points for which the difference in distance between feature points is equal to or greater than a certain value as noise feature points.
An example of the process in step S13 will be described using FIGS. 6(a), 6(b), and 6(c).

In FIGS. 6A, 6B, and 6C, the inter-feature point distance L1, the inter-feature point distance L11, and the inter-feature point distance L21 are approximately equal (L1≈L11≈L21). The inter-feature point distance L2, the inter-feature point distance L12, and the inter-feature point distance L22 are approximately equal (L2≈L12≈L22). The inter-feature point distance L3, the inter-feature point distance L13, and the inter-feature point distance L23 are approximately equal (L3≈L13≈L23). The inter-feature point distance L4, the inter-feature point distance L14, and the inter-feature point distance L24 are approximately equal (L4≈L14≈L24).

On the other hand, the inter-feature point distance L5, the inter-feature point distance L15, and the inter-feature point distance L25 are different (L5≠L15≠L25). The inter-feature point distance L6, the inter-feature point distance L16, and the inter-feature point distance L26 are different (L6≠L16≠L26). The inter-feature point distance L7, the inter-feature point distance L17, and the inter-feature point distance L27 are different (L7≠L17≠L27).

Therefore, in FIG. 6(a), the feature points Pf1, Pf2, Pf3, Pf4, Pf5, in FIG. 6(b), the feature points Pf11, Pf12, Pf13, Pf14, Pf15, and in FIG. 6(c), The feature points Pf21, Pf22, Pf23, Pf24, and Pf25 are regular feature points. That is, in the image, the features of the road surface appear at almost the same position at any timing. On the other hand, in FIG. 6(a), the feature points Pf6, Pf7, Pf8, in FIG. 6(b), the feature points Pf16, Pf17, Pf18, and in FIG. 6(c), the feature points Pf26, Pf27, Pf28. This is a feature point due to noise. That is, in an image, the position where noise features appear differs depending on the timing.

In this way, the determination unit 70 detects noise by comparing the positions of the feature points in each of the images P1a, P1b, and P1c in the plurality of images P1 of the road surface Sr acquired by the camera 20 at the same geographical point.

Specifically, the inter-feature point distance measurement section 71 of the determination section 70 in FIG. 4 measures the distance between the feature points in each of the images P1a, P1b, and P1c in the plurality of images P1 of the road surface Sr acquired by the camera 20. If the difference in distance between the feature points in each image P1a, P1b, P1c (for example, |L1-L11|, |L11-L21|) is larger than a threshold, the comparison unit 72 performs Suppose that noise is detected as a result of comparing the positions of feature points.

In step S14 of FIG. 5, the noise mask unit 80 of the control device 40 determines an area (range) in the image with many noise feature points, and dynamically masks the area. That is, an operation is performed to mask noise during processing as necessary.

An example of the process in step S14 will be described using FIGS. 6(a), 6(b), and 6(c).
In FIG. 6A, it is determined that an elliptical region Z1 including the feature points Pf6, Pf7, and Pf8 is an area (range) with many noise feature points in the image P1a. Then, the noise is hidden by masking the area Z1 in the image P1a where there are many noise feature points. In FIG. 6B, it is determined that an elliptical region Z2 including the feature points Pf16, Pf17, and Pf18 is an area (range) with many noise feature points in the image P1b. Then, the noise is hidden by masking the area Z2 in the image P1b where there are many noise feature points. In FIG. 6C, it is determined that an elliptical area Z3 including feature points Pf26, Pf27, and Pf28 is an area (range) with many noise feature points in the image P1c. Then, the noise is hidden by masking the area Z3 in the image P1c where there are many noise feature points.

Areas Z1, Z2, and Z3 are areas where the illuminance of the illumination light source 30 (LED 31) is low, and noise feature points are likely to appear.
Regarding the timing of performing the process in FIG. 5, basically it is performed once when the settings of the camera 20 or light source 30 (illumination) change, and the same location (area) is masked until the settings etc. change. However, the process shown in FIG. 5 may be performed at startup, or may be performed periodically to change the location (area) to be masked.

Then, in the self-position estimating unit 51, the features extracted from the image in which noise is hidden by masking the area with many noise feature points in the image of the road surface Sr photographed by the camera 20, and the map in the storage unit 60 are used. By comparing the characteristics extracted from the image 61, the self-position of the autonomous vehicle is estimated.

In this way, the noise masking section 80 masks the detected noise in the image of the road surface Sr taken by the camera 20 at the same geographical point, and sets it as the image of the road surface Sr in the self-position estimating section 51 . In other words, by comparing images taken at fixed points, a region (range) characterized by noise is identified and the noise is masked. That is, processing to hide noise is performed. This allows us to identify noisy areas (ranges) Z1, Z2, and Z3 using multiple images obtained at fixed points, that is, the same geographical location, and dynamically generate masks to determine the self-location. Improve estimation accuracy.

In other words, noise is determined by acquiring multiple camera images at the same geographical point and comparing the positions of their feature points. Thereby, by acquiring an image with less noise, it is possible to suppress a decrease in self-position estimation accuracy due to noise.

explain in detail.
Feature points due to noise in a camera image exist at periodically different positions. In other words, if a feature is based on a road surface pattern, when multiple camera images are acquired at the same geographical point, the feature exists at the same position in each camera image, and the feature exists at the same position in the camera images.

However, when multiple camera images are acquired at the same geographical point, the characteristics due to noise that exist due to the brightness and darkness of camera images tend to differ in their positions in the camera images.
Therefore, by acquiring multiple camera images at the same geographical point and comparing the positions of feature points in the camera images, it is possible to determine that locations where the positions of feature points change are noise.

This makes it possible to improve self-position estimation accuracy by using an algorithm that obtains images with less noise and making self-position estimation less susceptible to noise.
According to the above embodiment, the following effects can be obtained.

(1) A self-position estimation unit 51 that estimates the self-position of the autonomous vehicle 10 by comparing the features extracted from the image P1 of the road surface Sr photographed by the camera 20 and the features extracted from the map image 61 The configuration of the autonomous vehicle 10 equipped with the following is as follows. Noise is detected by comparing the positions of feature points Pf1 to Pf8, Pf11 to Pf18, and Pf21 to Pf28 in each image P1a, P1b, and P1c in a plurality of images P1 of the road surface Sr acquired by the camera 20 at the same geographical point. A determination unit 70 is provided. Further, a noise masking section 80 is provided which masks the noise detected by the determination section 70 in the image of the road surface Sr taken by the camera 20 at the same geographical point and uses it as an image of the road surface Sr in the self-position estimating section 51.

According to this, noise is detected by comparing the positions of feature points in a plurality of road surface images acquired by a camera at the same geographical point, and the noise in the road surface image is masked. As a result, by obtaining an image with less noise, the self-position estimation accuracy of the autonomous vehicle can be improved by making the self-position estimation less susceptible to noise.

(2) Since the determination unit 70 detects noise based on the distance between feature points in each image, noise can be easily detected.
(3) Since the noise masking unit 80 masks a region where there are many feature points due to noise, noise can be easily masked.

The embodiment is not limited to the above, and may be embodied as follows, for example.
Although the plurality of images in FIG. 5 are three images P1a, P1b, and P1c, the number of images may be two or more.

○ In order to determine the features caused by noise, the distances between feature points (for example, the distances L1 to L7 in Fig. 6(a), the distances L11 to L17 in Fig. 6(b), and the distances L21 to L27 in Fig. 6(c)) are We used the following, but the following may be used instead: As shown in FIG. 7, a reference point Pr is provided on the plane that constitutes the camera image (an XY plane made up of orthogonal X and Y axes), and the distances L31 to L38 from the reference point Pr to each of the feature points Pf31 to Pf38 are But that's fine. That is, the distance L31 between the reference point Pr and the feature point Pf31, the distance L32 between the reference point Pr and the feature point Pf32, the distance L33 between the reference point Pr and the feature point Pf33, and the distance L33 between the reference point Pr and the feature point Pf33. Distance L34 between reference point Pr and feature point Pf35, distance L36 between reference point Pr and feature point Pf36, distance L37 between reference point Pr and feature point Pf37, A distance L38 between the reference point Pr and the feature point Pf38 is measured. If the difference in the distance from the reference point Pr to the feature point in each image is smaller than the threshold, it is a regular feature point, and if the difference in the distance from the reference point Pr to the feature point in each image is larger than the threshold, it is a noise. It is determined that the point is a feature point.

In FIG. 7, regarding the distances L31 to L38 from the reference point Pr to each of the feature points Pf31 to Pf38, the feature points Pf36 to Pf38 are noises in which the difference in distance from the reference point Pr to the feature point in each image is larger than the threshold value. The elliptical region Z10 including the feature points Pf36 to Pf38 is a region (range) in which there are many noise feature points in the image. Then, the noise is hidden by applying a mask to the area Z10 in which there are many noise feature points in the image.

In this way, the determination unit 200 shown by the virtual line in FIG. It has a reference point/feature point distance measurement unit 201 that measures the distance to a feature point. Furthermore, if the difference in distance from the reference point Pr to the feature point in each image is larger than a threshold, the determination unit 200 shown by a virtual line in FIG. It has a comparison unit 202 that detects the detection.

In this manner, the determination unit 200 can easily detect noise by detecting noise based on the distance from the reference point Pr on the plane that constitutes the image of the camera 20 to the feature point in each image. At this time, the noise masking unit 80 masks the area where there are many feature points due to noise, so that the noise can be easily masked.

○Although an LED is used as a light source, the present invention is not limited to this, and a halogen lamp or the like may also be used. Moreover, the color of the light source may be any color, for example, red, blue, white, etc.
〇The type of autonomous vehicle does not matter. For example, it may be a towing car that moves the aircraft at an airport.

DESCRIPTION OF SYMBOLS 10... Autonomous vehicle, 11... Vehicle body, 20... Camera, 30... Light source, 51... Self-position estimation part, 60... Storage part, 61... Map image, 70... Judgment part, 80... Noise mask part, 200... Judgment part , L1-L7...distance, L11-L17...distance, L21-L27...distance, L31-L38...distance, P1...image, P1a, P1b, P1c...image, Pf1-Pf8...feature point, Pf11-Pf18...feature point , Pf21 to Pf28...Feature points, Pf31 to Pf38...Feature points, Pr...Reference points, Sr...Road surface.

Claims (3)

a camera installed on the lower surface of the vehicle body and photographing the road surface below the vehicle body;
a light source that is installed on the lower surface of the vehicle body and irradiates a shooting area of the camera on the road surface;
a storage unit that stores a map image taken in advance of the road surface;
autonomous driving, comprising: a self-position estimating unit that estimates the self-position of the autonomous vehicle by comparing features extracted from the image of the road surface taken by the camera and features extracted from the map image; It's a car,
a determination unit that detects noise by comparing the positions of feature points in a plurality of images of the road surface obtained at different times by the same camera at the same geographical point;
and a noise masking unit that masks the noise detected by the determination unit in the image of the road surface taken by the same camera at the same geographical point to make the image of the road surface in the self-position estimating unit. autonomous vehicles. The autonomous vehicle according to claim 1, wherein the determination unit detects noise based on a distance between feature points in each of the images. The autonomous vehicle according to claim 1, wherein the determination unit detects noise based on a distance from a reference point on a plane forming an image of the camera to a feature point in each image.