JP7805201B2 - Object recognition method and object recognition device - Google Patents

Description

The present invention relates to an object recognition method and an object recognition device.

There is a known invention that identifies shaded and non-shaded areas on an image, calculates light source color information based on the hue and luminance information of the shaded and non-shaded areas, and detects and removes the shaded areas (Patent Document 1).

JP 2010-237976 A

However, because the invention described in Patent Document 1 determines whether an object is the same based on hue, there is a risk that adjacent asphalt areas and white line areas may be mistakenly determined to be shaded and non-shaded areas of the same object.

The present invention was made in consideration of the above problems, and its purpose is to provide an object recognition method and object recognition device that are capable of recognizing road markings with high accuracy.

An object recognition method according to one aspect of the present invention extracts road surface reflection points, which are reflection points from the road surface, from among multiple distance measurement points based on the positions of the distance measurement points detected by a position detection means; extracts a road surface marking area indicating the area where road surface markings exist in the captured image captured by the imaging means and a pavement area indicating the area of the road surface excluding the road surface marking area based on the reflection intensity of the road surface reflection points; extracts a reference reflection point group that serves as a standard for brightness in the captured image in the pavement area; calculates a reference pixel value that indicates the standard for the reference reflection point group; estimates the relative illuminance of each pixel in the captured image using the reference pixel value; corrects the luminance value of each pixel in the captured image using the relative illuminance; and recognizes road surface markings from the corrected image.

This invention makes it possible to recognize road markings with high accuracy.

FIG. 1 is a configuration diagram of an object recognition device 1 according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a reference reflection point group. FIG. 3 is a diagram illustrating an example of relative illuminance. FIG. 4 is a diagram illustrating an example of an image correction method. FIG. 5 is a flowchart illustrating an example of the operation of the object recognition device 1. FIG. 6 is a flowchart illustrating an example of the operation of the object recognition device 1. FIG. 7 is a flowchart illustrating an example of the operation of the object recognition device 1. FIG. 8 is a diagram illustrating an example of object recognition.

Embodiments of the present invention will be described below with reference to the drawings. In the drawings, identical parts will be designated by the same reference numerals and their descriptions will be omitted.

An example configuration of the object recognition device 1 will be described with reference to Figure 1. As shown in Figure 1, the object recognition device 1 includes a sensor group 10, a camera 11, and a controller 20.

The object recognition device 1 may be installed in a vehicle with an autonomous driving function, or in a vehicle without an autonomous driving function. The object recognition device 1 may also be installed in a vehicle that is capable of switching between autonomous driving and manual driving. The autonomous driving function may also be a driving assistance function that automatically controls only some of the vehicle control functions, such as steering control, braking force control, and driving force control, to assist the driver in driving. In this embodiment, the object recognition device 1 will be described as being installed in a vehicle with an autonomous driving function.

Although not shown in Figure 1, the object recognition device 1 may control various actuators, such as the steering actuator, accelerator pedal actuator, and brake actuator, based on the recognition results (object position, shape, orientation, etc.) from the image recognition unit 24. This may enable highly accurate autonomous driving.

The sensor group 10 primarily includes sensors that measure the distance and direction to objects around the vehicle. One example of such a sensor is a LIDAR (Laser Imaging Detection and Ranging). LIDAR measures the distance and direction to objects and recognizes the shape of objects by emitting light (laser light) at objects around the vehicle and measuring the time it takes for the reflected light to hit the object and bounce back. LIDAR can also obtain the positional relationship of objects in three dimensions. The principles and mechanisms of LIDAR are well known, so a detailed explanation is omitted. Mapping is also possible using the reflected intensity of laser light. The sensor group 10 may also include a GPS or GNSS receiver that detects the position of the vehicle. The sensor group 10 may also include a speed sensor, acceleration sensor, steering angle sensor, gyro sensor, brake oil pressure sensor, accelerator position sensor, etc. that detect the state of the vehicle. Information obtained by the sensor group 10 is output to the controller 20. Unless otherwise specified, the sensor group 10 will be described below as being represented by a lidar, and the information output to the controller 20 will be described as information acquired by the lidar.

The camera 11 has an imaging element such as a CCD (charge-coupled device) or CMOS (complementary metal oxide semiconductor). While there are no particular limitations on where the camera 11 may be installed, as an example, the camera 11 may be installed in front, to the side, or behind the vehicle. The camera 11 continuously captures images of the area around the vehicle at a predetermined interval. The camera 11 detects objects around the vehicle (pedestrians, bicycles, motorbikes, other vehicles, etc.) and information ahead of the vehicle (demarcation lines, traffic lights, signs, crosswalks, intersections, etc.). Images captured by the camera 11 are output to the controller 20. The images captured by the camera 11 are stored in a storage device (not shown), and the controller 20 may refer to the images stored in the storage device.

The area measured (detected) by the lidar and the area imaged (detected) by the camera 11 overlap in whole or in part.

The controller 20 is a general-purpose microcomputer equipped with a CPU (Central Processing Unit), memory, and input/output units. A computer program for functioning as the object recognition device 1 is installed on the microcomputer. By executing the computer program, the microcomputer functions as the multiple information processing circuits provided in the object recognition device 1. Note that while an example is shown here in which the multiple information processing circuits provided in the object recognition device 1 are realized by software, it is of course possible to configure the information processing circuits by providing dedicated hardware for executing each of the information processes described below. The multiple information processing circuits may also be configured as individual hardware. The controller 20 includes, as examples of multiple information processing circuits (information processing functions), a point cloud acquisition unit 21, a road surface reflection point estimation unit 22, an image acquisition unit 23, an image recognition unit 24, a reference reflection point extraction unit 25, a reference pixel value calculation unit 26, a relative illuminance estimation unit 27, and an image illuminance correction unit 28. Note that the controller 20 may also be referred to as an ECU (Electronic Control Unit).

The point cloud acquisition unit 21 acquires a ranging point cloud (a collection of multiple ranging points in three-dimensional coordinates) from the lidar. The point cloud acquisition unit 21 outputs the acquired ranging point cloud to the road surface reflection point estimation unit 22.

The road surface reflection point estimation unit 22 estimates road surface reflection points based on the three-dimensional position and shape of the range-finding point cloud. In this embodiment, "road surface reflection points" refer to reflection points from the road surface among the multiple range-finding points. Examples of reflection points other than those on the road surface while driving include reflection points from buildings. The road surface reflection point estimation unit 22 outputs the estimated road surface reflection points to the reference reflection point extraction unit 25.

The image acquisition unit 23 acquires the image captured by the camera 11. The image acquisition unit 23 outputs the acquired image to the image recognition unit 24. Note that if an area where the object to be recognized is likely to exist is known, the image acquisition unit 23 may extract and output only that area.

The image recognition unit 24 performs road marking recognition processing on the image acquired from the image acquisition unit 23. Road marking recognition processing is an example of image processing, and is a process of detecting and identifying road markings around the vehicle (mainly ahead of the vehicle) and associating their attribute information with each pixel. Such image processing is well known, and for example, semantic segmentation can be used, which estimates the likelihood that each pixel is a road marking. Attributes may also be distinguished by color, such as white lines and yellow lines. In this embodiment, "road markings" refer to markings on the road surface with special paint to provide guidance, direction, caution, regulation, instructions, etc. necessary for road traffic, and examples include dividing lines (dashed white lines), pedestrian crossings, stop lines, and direction arrows. The image recognition unit 24 outputs the processing results to the reference reflection point extraction unit 25.

Based on the reflection intensity of the laser light from the road surface reflection points and the image recognition results, the reference reflection point extraction unit 25 extracts reflection point groups (reference reflection point groups) that serve as a reference for the brightness of pixels on the image for each of the paved area and road marking area on the road surface. In this embodiment, "paved area" refers to the asphalt portion of the road surface excluding areas where road markings exist. "Road marking area" refers to areas where road markings exist. "Reference reflection point group for paved area" refers to a collection of reflection points from the paved area. Furthermore, "reference reflection point group for road marking area" refers to a collection of reflection points from the road marking area. The reference reflection point extraction unit 25 outputs the extracted reference reflection point groups to the reference pixel value calculation unit 26. Note that for road marking areas, clustering may be performed based on the color (hue, saturation) of the pixel values of points projected onto the image plane, with each cluster being treated as a separate reference reflection point group. Furthermore, because the reflectivity of light (laser light) from paved road surfaces such as asphalt and concrete is higher than that of light (laser light) from white lines and other road markings, lidar can determine whether a road reflection point is a paved road surface such as asphalt or concrete, or a road marking such as a white line, based on the reflectivity (i.e., the intensity of the reflected light). In other words, areas of the road surface with relatively high reflection intensity can be determined to be road marking areas, and areas of the road surface with relatively low reflection intensity can be determined to be paved areas other than road marking areas.

The reference pixel value calculation unit 26 projects the reference reflection point cloud acquired from the reference reflection point extraction unit 25 onto the image plane, and calculates reference pixel values for each of the pavement area and the road marking area from the pixel values of the projected point cloud. There are no particular restrictions on the method for calculating the reference pixel value, but as an example, it is calculated using the median and average values. If there are multiple reference reflection point clouds corresponding to different road marking colors, a reference pixel value is calculated for each of them. The reference pixel value calculation unit 26 outputs the calculated reference pixel values to the relative illuminance estimation unit 27.

The relative illuminance estimation unit 27 estimates the relative illuminance for each pixel of the image based on the reference pixel value calculated by the reference pixel value calculation unit 26, and generates a relative illuminance image. The relative illuminance estimation unit 27 outputs the generated relative illuminance image to the image illuminance correction unit 28. The relative illuminance is calculated by dividing the pixel value at each pixel by the reference pixel value. Furthermore, since lidar ranging points are generally sparser than in an image, areas on the image where there are no corresponding ranging points are calculated by interpolation.

The image illuminance correction unit 28 corrects the brightness of the image captured by the camera 11 using the relative illuminance image generated by the relative illuminance estimation unit 27. The correction is performed by dividing the image by the relative illuminance image. The corrected image is output to the image recognition unit 24, where it is subjected to object recognition processing again and output as the final recognition result. As described above, the object recognition device 1 may control various actuators such as the steering actuator, accelerator pedal actuator, and brake actuator based on the final recognition result (object position, shape, orientation, etc.) by the image recognition unit 24. This can realize highly accurate autonomous driving.

Next, we will explain the details of each function of the controller 20 with reference to Figures 2 to 4.

The scene shown in Figure 2 (image 30) is a view ahead of the vehicle traveling on a road. Image 30 was captured by camera 11 installed in front of the vehicle. As shown in Figure 2, the vehicle is traveling in the left lane of a two-lane road. In Figure 2, there is a space on the left side where vehicles can stop; this is a bus stop. Reference numerals 40 and 41 in Figure 2 indicate asphalt areas (traffic lane areas) on the road surface. More specifically, reference numeral 40 indicates the asphalt area on the left side, and reference numeral 41 indicates the asphalt area on the right side. Reference numerals 42 and 43 indicate dividing lines (white lines). Reference numeral 50 indicates an area illuminated by the sun (sunlight). Because the light in the area indicated by reference numeral 50 is strong, it may be difficult for the camera 11 to recognize objects (e.g., white lines). Therefore, in this embodiment, the area indicated by reference numeral 50 is corrected (corrected on the image) using information acquired by the lidar and image 30 captured by camera 11, thereby accurately recognizing objects (e.g., white lines) on the image. The area indicated by the reference numeral 50 may also be expressed as an area where the spatial frequency is above a predetermined value.

The road surface reflection point estimation unit 22 estimates road surface reflection points based on the three-dimensional position and shape of the range-finding point cloud. In Figure 2, road surface reflection points refer to reflection points from the areas indicated by reference numerals 40 to 43 and 50. Note that in Figure 2, the road surface includes areas other than those indicated by reference numerals 40 to 43 and 50, so the road surface also includes reflection points from areas other than those indicated by reference numerals 40 to 43 and 50. However, for convenience of explanation, road surface reflection points will be described here as reflection points from the areas indicated by reference numerals 40 to 43 and 50.

The reference reflection point extraction unit 25 extracts reflection point groups (reference reflection point groups) that serve as a reference for pixel brightness on the image for each paved area and road marking area on the road surface, based on the reflection intensity of the road surface reflection points and the image recognition results. In Figure 2, the "reference reflection point group for the paved area" refers to the collection of reflection points from the areas indicated by reference numerals 40 and 41. The "reference reflection point group for the road marking area" refers to the collection of reflection points from the areas indicated by reference numerals 42 and 44. In this embodiment, the lidar detection area and the camera 11 imaging area overlap. By comparing the detection results of both, the reference reflection point group is extracted with high accuracy. For the area indicated by reference numeral 50, the "reference reflection point group for the pavement area" and the "reference reflection point group for the road marking area" are extracted based on the lidar detection results (reflection intensity of the road surface reflection points).

The reference pixel value calculation unit 26 projects the reference reflection point cloud onto the image plane and calculates reference pixel values for each of the pavement area and road marking area from the pixel values of the projected point cloud. The relative illuminance estimation unit 27 calculates the relative illuminance for each pixel of the image based on the reference pixel values calculated by the reference pixel value calculation unit 26, generating a relative illuminance image. Specifically, in the area indicated by the symbol 50, in areas (areas on the image) detected by the lidar as white lines (road marking areas), the pixel values of the white lines are divided by the "reference pixel value of the road marking area." Similarly, in the area indicated by the symbol 50, in areas (areas on the image) detected by the lidar as asphalt (pavement area), the pixel values of the asphalt are divided by the "reference pixel value of the pavement area." This division corrects the brightness value of each pixel in the image 30. This division calculates the relative illuminance indicated by the symbol 60 in Figure 3. By projecting this relative illuminance onto a two-dimensional image, a relative illuminance image is generated, as shown at 61 in Figure 3.

Next, the image illuminance correction unit 28 corrects the brightness of the image captured by the camera 11 using the relative illuminance image 61 generated by the relative illuminance estimation unit 27. Specifically, the image 30 in Figure 2 is used as the input image, and the image illuminance correction unit 28 divides this input image by the relative illuminance image 61. This generates the illuminance-corrected image 70 shown in Figure 4. As can be seen from the illuminance-corrected image 70, the brightness of the area 50 is corrected. Road markings such as white lines can be recognized with high accuracy on the corrected image.

Next, an example of the operation of the object recognition device 1 will be described with reference to the flowchart in Figure 5. In step S101, the camera 11 captures an image of the area ahead of the vehicle. Processing proceeds to step S103, where the image recognition unit 24 performs road marking recognition processing on the image acquired in step S101. This recognizes the white lines, asphalt, etc. shown in Figure 2. In step S105, the point cloud acquisition unit 21 acquires a ranging point cloud (a collection of multiple ranging points in three-dimensional coordinates) from the lidar. Processing proceeds to step S107, where the road surface reflection point estimation unit 22 estimates road surface reflection points based on the three-dimensional positions and shapes of the ranging point cloud acquired in step S105.

The process proceeds to step S109, where the reference reflection point extraction unit 25 extracts reflection point groups (reference reflection point groups) that serve as a reference for the brightness of pixels on the image for each of the paved area and road marking area on the road surface, based on the reflection intensity of the road surface reflection points estimated in step S107 and the image recognition results. The process proceeds to step S111, where the reference pixel value calculation unit 26 projects the reference reflection point groups extracted in step S109 onto the image plane and calculates reference pixel values that serve as references for each of the paved area and road marking area from the pixel values of the projected point groups. The process proceeds to step S113, where the relative illuminance estimation unit 27 estimates relative illuminance using the reference pixel values calculated in step S111, and generates the relative illuminance image 61 shown in Figure 3.

The process proceeds to step S115, where the image illuminance correction unit 28 corrects the brightness of the image captured by the camera 11 using the relative illuminance image 61 generated in step S113 (see Figure 4). The process proceeds to step S117, where the image recognition unit 24 recognizes road markings such as white lines using the image corrected in step S115. This makes it possible to recognize road markings on the image even when the sun's rays are too strong for the camera 11 to recognize them.

An example of a method for extracting the reference reflection point cloud for a road marking area and the reference reflection point cloud for a pavement area will be described with reference to Figures 6 and 7. First, an example of a method for extracting the reference reflection point cloud for a road marking area will be described with reference to Figure 6. In step S201, the road surface reflection point estimation unit 22 determines whether each ranging point has been reflected from the road surface or something else (a three-dimensional object) based on the three-dimensional position and shape of the ranging point cloud. From step S201 onwards, the description will be given assuming that each ranging point has been determined to have reflected from the road surface. If there are unprocessed road surface reflection points in step S203 (YES in step S203), processing proceeds to step S205, where one unprocessed road surface reflection point is selected. If NO in step S203, processing ends.

If the reflection intensity of the ranging point is equal to or greater than the threshold in step S207 (YES in step S207), processing proceeds to step S209, where the ranging point is projected onto the image plane. If the projected ranging point in step S201 is recognized as a road marking on the image (YES in step S211), processing proceeds to step S213, where the ranging point is determined to be a reference reflection point in the road marking area. The threshold may be changed depending on the distance to the ranging point and the normal direction. Specifically, the threshold may be made smaller the longer the distance to the ranging point. The threshold may be made smaller the larger the angle between the laser incident direction and the normal direction of the ranging point. If NO in step S207, NO in step S211, or after processing in step S213, processing returns to step S203.

Next, an example of a method for extracting a reference reflection point group for a pavement area will be described with reference to Figure 7. However, since the processing of steps S301 to S305 and S309 is the same as the processing of steps S201 to S205 and S209 shown in Figure 6, the description will be omitted. In step S307, if the reflection intensity of the ranging point is below the threshold (YES in step S307), it is determined to be a reflection point for a pavement area. In step S311, if the projected ranging point is not recognized as a road marking on the image (YES in step S311), processing proceeds to step S315, where it is determined whether or not a reference reflection point for the road marking area extracted in the processing of step S213 in Figure 6 exists near the ranging point. If a reference reflection point for the road marking area exists near the ranging point (YES in step S315), processing proceeds to step S317, where it is determined to be a reference reflection point for a pavement area. If the result is NO in step S307, NO in step S311, NO in step S315, or after the processing of step S317, the processing returns to step S303. Note that, as in Figure 6, the threshold value may be changed depending on the distance to the ranging point and the normal direction.

Figure 8 shows an example of an image after correction by step S115 shown in Figure 5. In Figure 8, reference numeral 81 indicates a crosswalk, reference numeral 82 indicates an arrow line, and reference numeral 83 indicates a dividing line (white line). Reference numerals 81 to 83 indicate areas where strong sunlight hits and objects cannot be recognized by the camera 11. Image correction according to this embodiment makes it possible to recognize road markings on the image even in areas where strong sunlight makes them difficult for the camera 11 to recognize. Note that the scene shown in Figure 8 is different from the scene shown in Figure 2.

(Action and effect)
As described above, the object recognition device 1 according to this embodiment provides the following advantageous effects.

The object recognition device 1 includes an imaging means for capturing an image of a predetermined range around the vehicle, including the road surface; a position detection means for emitting light in at least an area overlapping the imaging means and detecting the positions of ranging points, which are reflection points, based on the reflected light; and a controller 20 for processing data acquired from the imaging means and position detection means. An example of the "imaging means" is a camera 11. An example of the "position detection means" is a lidar. An example of "light" is laser light. Based on the positions of the ranging points detected by the position detection means, the controller 20 extracts road surface reflection points, which are reflection points from the road surface, from among the multiple ranging points. Based on the reflection intensity of the road surface reflection points, the controller 20 extracts a road surface marking area, which indicates the area where road markings exist on the captured image captured by the imaging means, and a pavement area, which indicates the area of the road surface excluding the road surface marking area. The controller 20 extracts a reference reflection point group that serves as a reference for the brightness of the captured image in the pavement area. It then calculates a reference pixel value that indicates the reference for the reference reflection point group. The controller 20 estimates the relative illuminance of each pixel in the captured image using the reference pixel value. The controller 20 corrects the brightness value of each pixel in the captured image using the relative illuminance. The controller 20 recognizes road markings from the corrected image. This embodiment can correct for differences in brightness on the road surface due to the shadows of structures and objects around the road, enabling accurate recognition of road markings. Furthermore, because the reflection intensity of the laser light emitted from the lidar is used, it is possible to distinguish between objects with high reflection intensity (road markings) and objects with low reflection intensity (asphalt) regardless of the hue of the objects or the adjacent relationship between the areas. In the above-mentioned embodiment, an example was described in which reference pixel values of both the "reference reflection point cloud of the pavement area" and the "reference reflection point cloud of the road marking area" are used, but this is not limiting. Only the reference pixel value of the "reference reflection point cloud of the pavement area" may be used.

The reference reflection point group may be defined as a first reference reflection point group, and the reference pixel value may be defined as a first reference pixel value. The controller 20 may extract a second reference reflection point group that serves as a reference for brightness on the captured image in the road marking area, calculate a second reference pixel value that indicates the reference for the second reference reflection point group, estimate the relative illuminance of each pixel on the captured image using the second reference pixel value, and correct the luminance value of each pixel in the captured image using the relative illuminance. This enables road markings to be recognized with high accuracy.

The controller 20 may cluster the second reference reflection point group by the hue and saturation of the corresponding pixels and estimate the relative illuminance of each cluster. By performing clustering processing for each road marking color (white, yellow, etc.), it is possible to set an appropriate reference pixel value, making it possible to calculate an appropriate relative illuminance and correct shading even when road markings of multiple colors are present.

The greater the distance from the position detection means to the road surface reflection point, the smaller the reflection intensity threshold used to determine whether the road surface reflection point is a road marking. The greater the distance to the road surface reflection point (distance measurement point), the weaker the reflection intensity becomes, but by setting the threshold in this way, this effect can be suppressed, making it possible to accurately distinguish between paved areas (asphalt) and road markings.

The controller 20 may estimate the normal direction for each road surface reflection point based on its shape. The larger the angle between the incident direction of the light (laser light) and the normal direction, the smaller the reflection intensity threshold used to determine whether the road surface reflection point is a road marking. The greater the incident angle of the lidar's light with respect to the target surface (the closer the angle between the normal to the target surface and the incident direction of the light is to a right angle), the weaker the reflection intensity will be; however, by setting the threshold in this way, this effect can be suppressed, making it possible to accurately distinguish between paved areas (asphalt) and road markings.

The controller 20 may perform a smoothing process when interpolating the relative illuminance on the captured image to generate a relative illuminance image. Because pavement areas and road markings have a variety of colors microscopically, small errors will occur when calculating relative illuminance based on a single point. However, in outdoor environments, illuminance generally changes smoothly except at the boundaries between shaded and non-shaded areas. Therefore, smoothing the relative illuminance image makes it possible to reduce the effects of small color differences on the road surface.

The controller 20 may recognize road markings from the corrected image only in areas of the captured image where the relative illuminance is outside a specified range. An example of an "area where the relative illuminance is outside a specified range" is the area indicated by reference numeral 50 in Figure 2. By performing image recognition on only a portion of the image, the computational load is reduced. Of course, as described above, image recognition may also be performed on the entire image. By performing image recognition on the entire image, it is possible to accurately recognize road markings using the entire image as a clue.

Each function described in the above embodiments may be implemented by one or more processing circuits. Processing circuits include programmed processing devices, such as processors including electrical circuitry. Processing circuits also include devices, such as application-specific integrated circuits (ASICs) or circuit components, arranged to perform the described functions.

As described above, an embodiment of the present invention has been described. However, the descriptions and drawings that form part of this disclosure should not be understood as limiting this invention. Various alternative embodiments, examples, and operating techniques will become apparent to those skilled in the art from this disclosure.

1 Object Recognition Device, 10 Sensor Group, 11 Camera, 20 Controller, 21 Point Cloud Acquisition Unit, 22 Road Surface Reflection Point Estimation Unit, 23 Image Acquisition Unit, 24 Image Recognition Unit, 25 Reference Reflection Point Extraction Unit, 26 Reference Pixel Value Calculation Unit, 27 Relative Illuminance Estimation Unit, 28 Image Illuminance Correction Unit,

Claims (8)

An object recognition method for an object recognition device comprising: an imaging means for imaging a predetermined range around a vehicle including a road surface; a position detection means for emitting light into at least a range overlapping with the imaging means and detecting the position of a distance measurement point, which is a reflection point, based on the reflected light; and a controller for processing data acquired from the imaging means and the position detection means,
The controller
extracting road surface reflection points, which are reflection points from the road surface, from among the plurality of distance measurement points based on the positions of the distance measurement points detected by the position detection means;
extracting a road marking area indicating an area where road markings exist on the image captured by the imaging means and a pavement area indicating an area of the road surface excluding the road marking area based on the reflection intensities of the road surface reflection points;
extracting a first reference reflection point group that serves as a reference for brightness on the captured image of the paved area;
extracting a second reference reflection point group that serves as a reference for brightness on the captured image in the road marking area;
calculating a first reference pixel value indicating a reference for the first reference reflection point group from a pixel value of each pixel of the captured image ;
calculating a second reference pixel value indicating a reference for the second reference reflection point group from the pixel value of each pixel of the captured image;
estimating relative illuminance of each pixel on the captured image using the first reference pixel value and the second reference pixel value ;
correcting the luminance value of each pixel of the captured image using the relative illuminance;
an object recognition method for recognizing the road marking from the corrected image; The object recognition method according to claim 1 , wherein the controller clusters the second reference reflection point group according to the hue and saturation of the corresponding pixels, and estimates the relative illuminance of each cluster. An object recognition method for an object recognition device comprising: an imaging means for imaging a predetermined range around a vehicle including a road surface; a position detection means for emitting light into at least a range overlapping with the imaging means and detecting the position of a distance measurement point, which is a reflection point, based on the reflected light; and a controller for processing data acquired from the imaging means and the position detection means,
The controller
extracting road surface reflection points, which are reflection points from the road surface, from among the plurality of distance measurement points based on the positions of the distance measurement points detected by the position detection means;
extracting a road marking area indicating an area where road markings exist on the image captured by the imaging means and a pavement area indicating an area of the road surface excluding the road marking area based on the reflection intensities of the road surface reflection points;
extracting a reference reflection point group that serves as a reference for brightness on the captured image of the pavement area;
calculating a reference pixel value indicating a reference for the reference reflection point group from the pixel value of each pixel of the captured image;
estimating the relative illuminance of each pixel on the captured image using the reference pixel value;
correcting the luminance value of each pixel of the captured image using the relative illuminance;
Recognizing the road marking from the corrected image;
An object recognition method, characterized in that the greater the distance from the position detection means to the road surface reflection point, the smaller the threshold value of the reflection intensity used to determine whether the road surface reflection point is a road marking or not. An object recognition method for an object recognition device comprising: an imaging means for imaging a predetermined range around a vehicle including a road surface; a position detection means for emitting light into at least a range overlapping with the imaging means and detecting the position of a distance measurement point, which is a reflection point, based on the reflected light; and a controller for processing data acquired from the imaging means and the position detection means,
The controller
extracting road surface reflection points, which are reflection points from the road surface, from among the plurality of distance measurement points based on the positions of the distance measurement points detected by the position detection means;
extracting a road marking area indicating an area where road markings exist on the image captured by the imaging means and a pavement area indicating an area of the road surface excluding the road marking area based on the reflection intensities of the road surface reflection points;
extracting a reference reflection point group that serves as a reference for brightness on the captured image of the pavement area;
calculating a reference pixel value indicating a reference for the reference reflection point group from the pixel value of each pixel of the captured image;
estimating the relative illuminance of each pixel on the captured image using the reference pixel value;
correcting the luminance value of each pixel of the captured image using the relative illuminance;
Recognizing the road marking from the corrected image;
The controller estimates a normal direction for each of the road surface reflection points based on the shape of the road surface reflection points;
The object recognition method is characterized in that the larger the angle between the incident direction of the light and the normal direction, the smaller the threshold value of the reflection intensity used to determine whether the road surface reflection point is a road marking. An object recognition method for an object recognition device comprising: an imaging means for imaging a predetermined range around a vehicle including a road surface; a position detection means for emitting light into at least a range overlapping with the imaging means and detecting the position of a distance measurement point, which is a reflection point, based on the reflected light; and a controller for processing data acquired from the imaging means and the position detection means,
The controller
extracting road surface reflection points, which are reflection points from the road surface, from among the plurality of distance measurement points based on the positions of the distance measurement points detected by the position detection means;
extracting a road marking area indicating an area where road markings exist on the image captured by the imaging means and a pavement area indicating an area of the road surface excluding the road marking area based on the reflection intensities of the road surface reflection points;
extracting a reference reflection point group that serves as a reference for brightness on the captured image of the pavement area;
calculating a reference pixel value indicating a reference for the reference reflection point group from the pixel value of each pixel of the captured image;
estimating the relative illuminance of each pixel on the captured image using the reference pixel value;
correcting the luminance value of each pixel of the captured image using the relative illuminance;
Recognizing the road marking from the corrected image;
The object recognition method is characterized in that the controller performs a smoothing process when interpolating the relative illuminance on the captured image to generate a relative illuminance image. An object recognition method for an object recognition device comprising: an imaging means for imaging a predetermined range around a vehicle including a road surface; a position detection means for emitting light into at least a range overlapping with the imaging means and detecting the position of a distance measurement point, which is a reflection point, based on the reflected light; and a controller for processing data acquired from the imaging means and the position detection means,
The controller
extracting road surface reflection points, which are reflection points from the road surface, from among the plurality of distance measurement points based on the positions of the distance measurement points detected by the position detection means;
extracting a road marking area indicating an area where road markings exist on the image captured by the imaging means and a pavement area indicating an area of the road surface excluding the road marking area based on the reflection intensities of the road surface reflection points;
extracting a reference reflection point group that serves as a reference for brightness on the captured image of the pavement area;
calculating a reference pixel value indicating a reference for the reference reflection point group from the pixel value of each pixel of the captured image;
estimating the relative illuminance of each pixel on the captured image using the reference pixel value;
correcting the luminance value of each pixel of the captured image using the relative illuminance;
Recognizing the road marking from the corrected image;
The object recognition method, wherein the controller recognizes the road marking from the corrected image only in an area on the captured image where the relative illuminance is outside a predetermined range. an imaging means for imaging a predetermined range around the vehicle including the road surface;
a position detection means for emitting light to at least an area overlapping with the imaging means and detecting the position of a distance measurement point, which is a reflection point, based on the reflected light;
a controller that processes data acquired from the imaging means and the position detection means,
The controller
extracting road surface reflection points, which are reflection points from the road surface, from among the plurality of distance measurement points based on the positions of the distance measurement points detected by the position detection means;
extracting a road marking area indicating an area where road markings exist on the image captured by the imaging means and a pavement area indicating an area of the road surface excluding the road marking area based on the reflection intensities of the road surface reflection points;
extracting a reference reflection point group that serves as a reference for brightness on the captured image of the pavement area;
calculating a reference pixel value indicating a reference for the reference reflection point group from the pixel value of each pixel of the captured image ;
estimating the relative illuminance of each pixel on the captured image using the reference pixel value;
correcting the luminance value of each pixel of the captured image using the relative illuminance;
Recognizing the road marking from the corrected image ;
The controller recognizes the road marking from the corrected image only in an area on the captured image where the relative illuminance is outside a predetermined range.
An object recognition device characterized by: an imaging means for imaging a predetermined range around the vehicle including the road surface;
a position detection means for emitting light to at least an area overlapping with the imaging means and detecting the position of a distance measurement point, which is a reflection point, based on the reflected light;
a controller that processes data acquired from the imaging means and the position detection means,
The controller
extracting road surface reflection points, which are reflection points from the road surface, from among the plurality of distance measurement points based on the positions of the distance measurement points detected by the position detection means;
extracting a road marking area indicating an area where road markings exist on the image captured by the imaging means and a pavement area indicating an area of the road surface excluding the road marking area based on the reflection intensities of the road surface reflection points;
extracting a reference reflection point group that serves as a reference for brightness on the captured image of the pavement area;
calculating a reference pixel value indicating a reference for the reference reflection point group from the pixel value of each pixel of the captured image;
estimating the relative illuminance of each pixel on the captured image using the reference pixel value;
correcting the luminance value of each pixel of the captured image using the relative illuminance;
Recognizing the road marking from the corrected image;
The controller performs a smoothing process when interpolating the relative illuminance on the captured image to generate a relative illuminance image.
An object recognition device characterized by: