JP7701680B2 - Road surface condition determination device for vehicles - Google Patents

Description

The present invention relates to a road surface condition determination device for a vehicle.

Road surface condition determination devices that analyze images of the road ahead captured by an onboard camera to determine the condition of the road surface, for example, whether the road surface is paved or not, or whether the road surface is wet due to rain, have been put to practical use. In addition to being used for vehicle driving control, the determined road surface information is also used for driving control and route search of other vehicles that receive road surface information via the cloud, for example, in the case of connected cars equipped with a Data Communication Module (DCM).

Such road surface condition determination devices have the following problems. For example, in backlit conditions where the sun, headlights of oncoming vehicles, or building lights are in the direction of the camera's field of view, light from these light sources is reflected on the road surface. Also, when exiting a tunnel or parking lot, the difference in brightness between the vehicle and the direction of the camera's field of view becomes large. Under these conditions, reflections from the road surface can prevent the determination of road surface conditions, or extreme brightness differences can exceed the dynamic range of the camera. In either case, the captured image cannot be analyzed properly, leading to an erroneous determination of road surface conditions.

As a countermeasure, for example, Patent Document 1 discloses a technology for detecting lane markers on the road for the purpose of driving assistance. In this technology, when the influence of the light source or the like is large as described above, if the front image captured by the front camera is determined to be unsuitable for judging the road surface condition, the rear image captured by the rear camera is selected and analyzed instead to judge the road surface condition.

JP 2002-99999 A

In the technology of Patent Document 1, the rear image is selected when the front image is unsuitable, but the imaging point of the rear camera is not taken into consideration. For example, a method of imaging the road surface with the rear camera at the timing when the front image is judged to be unsuitable is considered. In this case, since the vehicle is traveling during the process of judging suitability, the rear camera may capture an image of a point far beyond the imaging point of the front camera. In addition, a method of capturing and storing a rear image with the rear camera at the same timing as the imaging of the front camera, and reading and analyzing the stored rear image when the front image is unsuitable is also considered. In this case, an image of a point behind the front image is captured by the rear camera by an amount equivalent to the front-rear distance between the imaging points of the front camera and the rear camera when the vehicle is stopped.

In either case, the road surface condition is determined based on the rear image captured at a point different from the front image. Although there is no problem when detecting lane markers provided continuously along the road as in Patent Document 1, the following problems arise when determining the road surface condition.
For example, in developing countries where roads are not well developed, paved roads may have unpaved areas scattered thereon due to partial construction work, and on mountain roads, small rivers may form across the road surface due to rainwater or spring water seeping from the mountainside. For example, on a road where unpaved areas are scattered on a paved road, if a front image capturing the paved part is determined to be inappropriate, a rear image of the unpaved road may be selected and analyzed, but in that case, the road surface condition is determined based on the rear image, which has completely different conditions from the front image. This can lead to problems such as incorrect determination of road surface conditions and missing parts in the road surface information data.

The present invention has been made to solve these problems, and its purpose is to provide a road surface condition judgment device for a vehicle that can judge the condition of the road surface without being affected by light sources, and can effectively utilize road surface information by preventing problems such as erroneous judgments and missing data.

In order to achieve the above object, the road surface condition determination device for a vehicle of the present invention includes a front imaging device that images the road surface at a point in front of the vehicle, a rear imaging device that images the road surface at a point behind the vehicle, a road surface position estimation unit that estimates the imaging points of the front image captured by the front imaging device and the rear image captured by the rear imaging device, respectively, a road surface condition determination unit that determines the condition of the road surface in the front image and the rear image, respectively, a light source influence determination unit that determines the degree of influence of a light source on the front image and the rear image, respectively, and when the light source influence determination unit determines that the degree of influence of a light source on either the front image or the rear image is low, the road surface condition determination unit The present invention is characterized in that it is equipped with an acquired image control unit that outputs the determined road surface condition of one of the images and the imaging point of the one of the images estimated by the road surface position estimation unit as road surface information, and when the light source influence determination unit determines that the degree of influence of the light source on one of the images is high and the degree of influence of the light source on the other of the images is low, it regards the other of the images captured at the same point as the one of the images as an alternative image, and outputs the road surface condition of the alternative image determined by the road surface condition determination unit and the imaging point of the alternative image estimated by the road surface position estimation unit as road surface information instead of the road surface information for the one of the images (Claim 1).

Therefore, if it is determined that the degree of influence of the light source on either the front image or the rear image is low, the road surface condition and the imaging point of either image are output as road surface information, and if it is determined that the degree of influence of the light source on either image is high and the degree of influence of the light source on the other image is low, the other image captured at the same point as the one image is considered to be an alternative image, and the road surface condition and the imaging point of the alternative image are output as road surface information. As a result, if either image is influenced by the light source and is unsuitable for determining the road surface condition, the other image that is likely to be free from the influence of the light source as an alternative image is analyzed to determine the road surface condition. Therefore, it is possible to accurately determine the road surface condition without being influenced by the light source.

If either of the images is unsuitable for determining the road surface conditions, another image taken at the same location as the first image is selected as an alternative image, and the road surface conditions are determined based on the other image under the same conditions as the first image.
In another aspect, the captured image control unit outputs road surface information regarding the one of the images when the light source influence determination unit determines that the degree of influence of the light source on the one of the images is less than a preset first determination value, and when the light source influence determination unit determines that the degree of influence of the light source on the one of the images is equal to or greater than the first determination value and determines that the degree of influence of the light source on the other of the images is less than the first determination value, regards the other of the images captured at the same point as the one of the images as the alternative image. Road surface information may be output, and if it is determined that the degree of influence of the light source on either one of the images and the other of the images is both equal to or greater than a second judgment value set to a value greater than the first judgment value, the output of the road surface information may be stopped, and if it is determined that the degree of influence of the light source on either one of the images and the other of the images is both equal to or greater than the first judgment value and at least one of the degrees of influence of the light source on both images is less than the second judgment value, road surface information regarding the image with a smaller degree of influence of the light source may be output (claim 2).

Therefore, when the degree of influence of the light source on either one image and the other image is both equal to or greater than the first judgment value, and at least one of the degrees of influence of the light source on both images is judged to be less than the second judgment value, road surface information for the image with the smaller degree of influence of the light source is output. Images with a degree of influence of the light source less than the second judgment value still have room to analyze the road surface condition, although not as much as images with a degree of influence less than the first judgment value. When there are road surface locations where the road surface condition cannot be analyzed due to the influence of the light source, there will be missing parts in the road surface information, but this prevents such a situation.

In another aspect, the system may further include a recording unit that links the front image and the rear image to the imaging point estimated by the road surface position estimation unit and records them, and when the captured image control unit determines that the degree of influence of the light source on one of the images is high, it may read out from the recording unit the other of the images captured at the same point as the one of the images, and when it determines that the degree of influence of the light source on the read out other of the images is low, it may consider the other of the images to be the alternative image (Claim 3).

For example, when it is determined that the influence of the light source on one of the images is high, it is possible to capture the other image, but if the vehicle is traveling at a high speed, the imaging device may have already passed the imaging point for one of the images and may not be able to capture the image. In the present invention, an image recorded in advance in the recording unit is read out, so that the other image of the same point can be reliably obtained as a substitute image regardless of various conditions such as the vehicle's traveling speed and the calculation speed of the road surface condition determination device.

In another aspect, the vehicle may further include a movement direction determination unit that determines the movement direction of the vehicle, and when the movement direction determination unit determines that the vehicle is moving forward, the captured image control unit may regard the front image as one of the images and the rear image as the other of the images, and when the movement direction determination unit determines that the vehicle is moving backward, the captured image control unit may regard the rear image as one of the images and the front image as the other of the images (Claim 4).

Therefore, the movement direction determination unit determines the direction of movement of the vehicle, and if the vehicle is moving forward, the front image is selected, and if the front image is unsuitable, the rear image is selected, and each image is applied to determine the road surface condition. Also, if the vehicle is reversing, the rear image is selected, and if the rear image is unsuitable, the front image is selected, and each image is applied to determine the road surface condition. As a result, it becomes possible to determine the road surface condition without being affected by the light source, not only when the vehicle is moving forward but also when it is reversing.

In another aspect, the captured image control unit may output road surface information regarding the front image and the rear image to at least one of a driving control unit that controls the driving of the vehicle and a communication device that communicates with the outside (claim 5).
Therefore, the road surface condition can be accurately determined without being affected by the light source, and the determined condition can be effectively utilized for vehicle travel control, other vehicle travel control, route guidance, and the like.

The vehicle road surface condition judgment device of the present invention can judge the condition of the road surface without being affected by light sources, and can effectively utilize road surface information by preventing problems such as erroneous judgments and missing data.

1 is a configuration diagram showing a vehicle equipped with a road surface condition determination device according to an embodiment; FIG. 2 is a block diagram showing a configuration of a controller. 5 is a flowchart showing an image and position information recording routine executed by the controller. 5 is a flowchart showing a road surface condition determination routine executed by the controller. 1 is an explanatory diagram showing a situation in which a light source is located in front of a vehicle moving forward, and the road surface reflecting light from the light source is imaged by a front camera and then by a rear camera. FIG.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a road surface condition determination device for a vehicle according to the present invention will now be described.
FIG. 1 is a diagram showing the configuration of a vehicle equipped with a road surface condition determination device according to the present embodiment.
The controller 2 mounted on the vehicle 1 executes various controls such as control of an engine (not shown) which is a power source for driving, and driving control for appropriately controlling the behavior of the vehicle 1 while driving. In addition, the controller 2 also functions as a road surface condition determination device 3 which determines the condition of the road surface on which the vehicle 1 is driving, for example, whether the road surface is paved or not, or whether the road surface is wet due to rain or not. The determined road surface information is used for driving control of the vehicle 1 by a driving control unit 4 described later, and is also provided to other vehicles from a DCM 5 (corresponding to the "communication device" of the present invention) via the cloud and is also used for driving control and route search of other vehicles. Note that the configurations of the driving control unit 4 and the DCM 5 are well known, so detailed explanations thereof will be omitted.

As one of the pieces of equipment that make up the road surface condition determination device 3, a front camera 6 (corresponding to the "front image capture device" of the present invention) is mounted on the front of the vehicle 1, and a rear camera 7 (corresponding to the "rear image capture device" of the present invention) is mounted on the rear of the vehicle 1. The front camera 6 captures an image of the road surface at a point ahead of the vehicle 1, which is a predetermined distance away, and the rear camera 7 captures an image of the road surface at a point behind the vehicle 1, which is a predetermined distance away.

FIG. 2 is a block diagram showing the configuration of the controller 2, and in this figure, the portion mainly related to the road surface condition determination device 3 is extracted and shown.
The front camera 6 and the rear camera 7 are each connected to an acquired image control unit 9 via an image acquisition unit 8, and the acquired image control unit 9 is connected to a road surface condition acquisition unit 11 via an image processing unit 10, as well as to a road surface position estimation unit 12, a movement direction determination unit 13, a recording unit 14, the driving control unit 4 and a DCM 5. A wheel speed sensor 15, an angular velocity sensor 16, an acceleration sensor 17 and a GPS 18 are connected to the road surface position estimation unit 12, and a wheel speed sensor 15 and a transmission 19 are connected to the movement direction determination unit 13.

The road surface position estimation unit 12 estimates the relative or absolute position of the image capturing points captured by the front camera 6 and the rear camera 7 based on various information such as the wheel speed detected by the wheel speed sensor 15, the yaw rate detected by the angular velocity sensor 16, the longitudinal acceleration detected by the acceleration sensor 17, and the position of the vehicle 1 acquired by the GPS 18, and outputs this as position information to the captured image control unit 9.

The movement direction determination unit 13 determines the movement direction of the vehicle 1 based on the forward/reverse switching information obtained from the transmission 19 or the wheel speed information obtained from the wheel speed sensor 15 , and outputs the result to the captured image control unit 9 .
The captured image control unit 9 acquires the front and rear images captured by the front and rear cameras 6 and 7 via the image acquisition unit 8, and sequentially records them in the recording unit 14. The recorded images are held in the recording unit 14 for a period during which the vehicle 1 travels a predetermined distance or for a predetermined time (hereinafter referred to as a predetermined travel distance or a predetermined time). Although details will be described later, this is a process for substituting the other image of the same point when one of the images is affected by a light source and is not suitable for determining the road surface condition. In addition, the front and rear images are linked to position information of the image capturing points of the front and rear cameras 6 and 7 acquired from the road surface position estimation unit 12 and sequentially recorded in the recording unit 14.

The acquired image control unit 9 also acquires the moving direction of the vehicle 1 from the moving direction determination unit 13, and first outputs an image corresponding to the moving direction, specifically, an image captured by the cameras 6 and 7 facing the moving direction (hereinafter referred to as a moving direction image) and position information associated therewith, to the road surface condition acquisition unit 11 via the image processing unit 10. The moving direction image is a front image when the vehicle 1 is moving forward, and a rear image when the vehicle 1 is moving backward. After outputting the moving direction image, the road surface condition acquisition unit 11 may request an image corresponding to the opposite direction to the moving direction of the vehicle 1, specifically, an image captured by the cameras 6 and 7 facing in the opposite direction to the moving direction (hereinafter referred to as a counter moving direction image). In that case, the counter moving direction image having position information of the same point as the moving direction image is read from the recording unit 14 in response to the request, and is output to the road surface condition acquisition unit 11 as a substitute image.

Furthermore, when the road surface condition determined based on the moving direction image or the counter-moving direction image and a command to output the road surface condition are input from the road surface condition acquisition unit 11, the acquired image control unit 9 outputs the road surface condition and the position information associated with the road surface condition as road surface information to the driving control unit 4 and the DCM 5. The road surface information output to the DCM 5 is provided to other vehicles via the cloud.
The image processing unit 10 performs processing such as distortion conversion, noise reduction, and color correction on the image input from the acquired image control unit 9 , and outputs the image to the road surface condition acquisition unit 11 .

On the other hand, the road surface condition acquisition unit 11 has a road surface condition determination unit 20 , a light source effect determination unit 21 , and an output determination unit 22 .
The road surface condition determination unit 20 analyzes the moving direction image or the opposite moving direction image acquired from the acquired image control unit 9, and determines the road surface condition in front of or behind the vehicle 1. Since the determination process is well known, detailed explanation will be omitted, but for example, a neural network trained in advance using image data of various road surfaces and reflections other than road surfaces outputs text or assigned numbers indicating the road surface condition, and a reliability value of the inference of the road surface. Example: {"asphalt": 0.6, "dirt": 0.2, "snow": 0.1, "others": 0.1 }
The light source influence determination unit 21 analyzes the moving direction image or the counter moving direction image acquired from the acquired image control unit 9, and determines the degree of influence of the light source on the image. The influence of the light source on the image includes, for example, the influence of the light source reflected on the road surface, or the influence of an extreme brightness difference between the vehicle 1 and the angle of view of the cameras 6 and 7, and the degree of influence of such light sources is determined comprehensively. Various known methods can be applied to the determination process, and for example, as shown in the flow chart of FIG. 7 of Patent Document 1 (JP Patent Publication 2002-99999), the difference between the maximum brightness and the minimum brightness in the captured image may be used as an index for the determination.

Based on the degree of influence of the light source determined by the light source influence determination unit 21, the output determination unit 22 determines whether the moving direction image or the reverse moving direction image acquired from the acquired image control unit 9 is suitable for determining the road surface condition, in other words, whether it should be output as road surface information. In detail, if the degree of influence of the light source on the moving direction image acquired in advance from the acquired image control unit 9 is less than a first judgment value set in advance, it is considered that there is no influence of the light source (including cases where the influence is very slight). In this case, the road surface condition of the moving direction image determined by the road surface condition determination unit 20 and an output command are output to the acquired image control unit 9. In addition, if the degree of influence of the light source on the moving direction image is equal to or greater than the first judgment value, it is considered that the influence of the light source is not negligible, and a request is made to the acquired image control unit 9 for a reverse moving direction image linked to the position information of the same point as the moving direction image.

When the reverse-movement-direction image is acquired from the acquired image control unit 9 in response to a request, the degree of influence of the light source on the reverse-movement-direction image is compared with the first judgment value. If the degree of influence of the light source is less than the first judgment value, the road surface condition of the reverse-movement-direction image and an output command are output to the acquired image control unit 9. If the degree of influence of the light source on the reverse-movement-direction image is equal to or greater than the first judgment value, the degree of influence of the light source on each of the moving direction image and the reverse-movement-direction image is compared with a second judgment value (>first judgment value) that is set in advance to a value greater than the first judgment value. If the degrees of influence of both images are equal to or greater than the second judgment value, the output to the acquired image control unit 9 is stopped as neither road surface condition is suitable for road surface information. In other cases, that is, if the degree of influence of at least one of the images is less than the second judgment value, the moving direction image and the reverse-movement-direction image with the smaller degree of influence of the light source are selected, and the road surface condition of the selected image and an output command are output to the acquired image control unit 9.

The division of roles between the acquired image control unit 9 and the output determination unit 22 is not limited to this and can be changed arbitrarily. For example, the determination function of the output determination unit 22 may be included in the acquired image control.
Next, the process of determining the road surface condition by the road surface condition determining device 3 will be described.
FIG. 3 is a flowchart showing an image and position information recording routine executed by the road surface condition determining device 3, which is executed at predetermined control intervals while the vehicle 1 is traveling.

First, in step S1, position information is read from the road surface position estimation unit 12, and in step S2, a front image captured by the front camera 6 and a rear image captured by the rear camera 7 are read. In the following step S3, position information is linked to each of the read front and rear images, and in step S4, these images and position information are recorded in the recording unit 14 (acquired image control unit 9). The images and position information are held in the internal recording area for a predetermined driving distance or a predetermined time, and are updated sequentially each time a new image is acquired.

The specified driving distance or the specified time is set to a length that allows both the front image and the rear image of the same location to be recorded and held in the recording unit 14. For example, the specified driving distance is set to be longer than the front-to-rear distance between the image capturing points of the front camera 6 and the rear camera 7 when the vehicle is stopped. Also, for example, the specified time is set to a length that allows a rear image of the same location to be captured and recorded before the front image in the recording unit 14 is erased by updating, even when the vehicle 1 is traveling at a low speed.

On the other hand, FIG. 4 is a flowchart showing a road surface condition determination routine executed by the road surface condition determining device 3, which is executed in synchronization with the routine of FIG.
For ease of explanation, it is assumed that the vehicle 1 is moving forward. In step S11, various sensor information is read out, and in the following step S12, the moving direction of the vehicle 1 is determined (moving direction determination unit 13). In the following step S13, the images captured by the cameras 6 and 7 corresponding to the moving direction are selected as moving direction images. In this case, since the vehicle 1 is moving forward, first, the front image is selected as the moving direction image, and the latest front image, i.e., the front image and position information captured and recorded by the processing of step S4 in FIG. 3 at that time, are captured.

Then, in step S14, the road surface condition of the moving direction image is judged (road surface condition judgment unit 20), in step S15, the degree of influence of the light source on the moving direction image Fa is judged (light source influence judgment unit 21), and in step S16, it is judged whether the degree of influence of the light source on the moving direction image Fa is less than the first judgment value F1 (output judgment unit 22). If there is no light source such as the sun, the headlights of an oncoming vehicle, or building lights in front of the vehicle 1, a clear front image that is not influenced by the light source is captured. For this reason, a Yes (affirmative) judgment is made in step S16, and in step S17, the road surface condition of the front image and its position information are output as road surface information to the driving control unit 4 and the DCM 5 (acquired image control unit 9).

Also, as shown in FIG. 5A, for example, in backlight conditions where light source A is present in front of vehicle 1, light from light source A is reflected on road surface E corresponding to the imaging point of front camera 6, and front camera 6 facing forward images road surface E under backlight conditions. Since the captured front image is affected by light source A, a No (negative) judgment is made in step S16 and the process proceeds to step S18. In step S18, from among the images and position information recorded in recording unit 14, a counter-movement direction image linked to the position information of the same point as the movement direction image is selected as a substitute image, and this counter-movement direction image and position information are read out (output determination unit 22, acquired image control unit 9). In this case, a rear image having position information of the same point as the front image and its position information are read out from recording unit 14.

The time required from when the front image is captured until the rear image is captured depends on the traveling speed of the vehicle 1. Therefore, if the traveling speed of the vehicle 1 is unexpectedly slow, the rear image may not yet have been captured and recorded at the time of step S18. In that case, the process waits until the capturing and recording are completed.
In step S18, instead of reading out the captured rear image from the recording unit 14, it is also possible to capture a rear image when the rear camera 7 reaches the imaging point of the front image as the vehicle 1 travels. However, if the traveling speed of the vehicle 1 is high, the rear camera 7 may have already passed the imaging point of the front image at the time when the front image is determined to be inappropriate in step S16. In this embodiment, since an image recorded in advance in the recording unit 14 is read out, the rear image of the same point can be reliably used as a substitute image in the determination process regardless of various conditions such as the traveling speed of the vehicle 1 and the calculation speed of the controller 2.

However, in the present invention, it is not essential to record the captured image in the recording unit 14. In cases where the calculation speed of the controller 2 is sufficient, the rear image may be captured when the rear camera 7 reaches the imaging point of the front image as the vehicle 1 travels, as described above.
In the next step S19, the state of the road surface in the reverse direction image is judged (road surface state judgement unit 20), in step S20, the influence degree Fb of the light source A in the reverse direction image is judged (light source influence judgement unit 21), and in step S21, it is judged whether or not the influence degree Fb of the light source A in the reverse direction image is less than the first judgment value F1 (output judgement unit 22). Even if the road surface is captured by the front camera 6 under backlight conditions, as shown in FIG. 5B, the rear camera 7 facing backward captures the road surface E under frontlight conditions, so that the rear image is often free from the influence of the light source A. In this case, the judgement is Yes in step S21 and the process proceeds to step S22, where the state of the road surface in the rear image and its position information are output as road surface information to the driving control unit 4 and the DCM 5 (acquired image control unit 9).

If the judgment in step S21 is No, the process proceeds to step S23, where it is judged whether the degree of influence Fa of the light source A in the moving direction image is equal to or greater than the second judgment value F2, and whether the degree of influence Fb of the light source A in the counter-movement direction image is equal to or greater than the second judgment value F2. If the judgment is Yes, the routine ends. Therefore, in this case, both images are deemed unsuitable for judging the road surface condition, and their road surface information is not output. At this time, it is also possible to output to the driving control unit 4 and the DCM 5 a message that the road surface condition cannot be judged.

If the determination in step S23 is No, i.e., if at least one of the degrees of influence of light source A Fa, Fb is less than the second determination value, the process proceeds to step S24. In step S24, the image with the smaller degree of influence of light source A Fa, Fb is selected, and the road surface condition and position information of the selected image are output as road surface information to the driving control unit 4 and the DCM 5 (acquired image control unit 9).

The processing when the vehicle 1 is reversing is similar, so a duplicate explanation will not be given, but first the rear image is read out to determine the degree of influence of light source A, and if this is inappropriate, the front image is read out as an alternative image and the degree of influence of light source A is also determined.
As described above, when the travel direction image is affected by the light source A and is not suitable for determining the road surface condition, the road surface condition determination device 3 of the vehicle 1 in this embodiment determines the road surface condition by analyzing the opposite travel direction image, which is likely to be free from the influence of the light source A as an alternative image. Therefore, the road surface condition can be accurately determined without being affected by the light source A, and can be effectively utilized for driving control of the vehicle 1 and driving control and route guidance of other vehicles via the DCM 5.

In this embodiment, when the moving direction image is not suitable for determining the road surface condition, a counter moving direction image of the same location as the moving direction image is selected as an alternative image and applied to the determination of the road surface condition. Therefore, for example, on a road where unpaved areas are scattered on a paved road, if the paved areas are captured and analyzed as a moving direction image and determined to be unsuitable, a counter moving direction image of the paved areas at the same location is selected as an alternative image. As a result, the road surface condition is determined based on the counter moving direction image under the same conditions as the moving direction image, so that it is possible to prevent erroneous determinations such as those in the technology of Patent Document 1, and also to prevent cases where missing areas occur in the road surface information data.

In addition, if the degree of influence of the light source A on both the moving direction image and the counter-moving direction image is equal to or greater than the first judgment value (steps S16 and S21 are No), and at least one of the degrees of influence of the light source A on both images is less than the second judgment value (step S23 is No), road surface information on the image with the smaller degree of influence of the light source A is output (step S24). In this case, the degree of influence of the light source A on either one image or both images is less than the second judgment value, and there is room to analyze the road surface condition, although not as much as an image with the degree of influence less than the first judgment value. If either one image is less than the second judgment value, that image is selected, and if both images are less than the second judgment value, the image with the smaller degree of influence of the light source A is selected to analyze the road surface condition. If there is a road surface location where the road surface condition cannot be analyzed due to the influence of the light source A, there will be missing parts in the road surface information, but this has the effect of preventing such a situation.

In addition, the movement direction determination unit 13 determines the movement direction of the vehicle 1, and when the vehicle 1 is moving forward, the front image is selected as the movement direction image, and when the front image is unsuitable, the rear image is selected as the opposite movement direction image, and each image is applied to the determination of the road surface condition. In addition, when the vehicle 1 is reversing, the rear image is selected as the movement direction image, and when the rear image is unsuitable, the front image is selected as the opposite movement direction image, and each image is applied to the determination of the road surface condition. Therefore, not only when the vehicle 1 is moving forward but also when it is reversing, the road surface condition can be accurately determined without being affected by the light source A and can be used as road surface information.

This concludes the description of the embodiment, but the aspects of the present invention are not limited to this embodiment. For example, in the above embodiment, the road surface condition is determined not only while the vehicle 1 is moving forward but also while it is moving backward, but the road surface condition may be determined only while the vehicle is moving forward. In more detail, a front image is selected while the vehicle 1 is moving forward to determine the road surface condition, and if the front image is determined to be inappropriate due to the influence of light source A, a rear image is selected instead of the front image to determine the road surface condition, and such processing is not performed while the vehicle 1 is moving backward.

In addition, in the above embodiment, the front image is selected as the movement direction image while the vehicle 1 is moving forward, and the rear image is selected as the movement direction image while the vehicle 1 is moving backward, but both images may be reversed. In detail, the rear image is selected to determine the road surface condition while the vehicle 1 is moving forward, and the front image is selected to determine the road surface condition while the vehicle 1 is moving backward. Even in this case, if the image on the opposite side is selected as an alternative image when the selected image is inappropriate, the same effect as the above embodiment can be achieved, and therefore such an aspect is also included in the present invention.

1 Vehicle 3 Road surface condition determination device 4 Travel control unit 5 DCM (communication device)
6 Front camera (front imaging device)
7 Rear camera (rear imaging device)
9 Acquired image control unit 12 Road surface position estimation unit 13 Movement direction determination unit 14 Recording unit 20 Road surface condition determination unit 21 Light source effect determination unit

Claims (5)

A front imaging device that captures an image of a road surface in front of the vehicle;
A rear imaging device that images a road surface at a rear point of the vehicle;
a road surface position estimating unit that estimates imaging points of the front image captured by the front imaging device and the rear image captured by the rear imaging device,
a road surface condition determination unit that determines a road surface condition of each of the front image and the rear image;
a light source influence determination unit that determines the degree of influence of a light source on each of the front image and the rear image;
a recording unit that records the front image and the rear image in association with the imaging point estimated by the road surface position estimation unit;
an acquired image control unit which, when the light source influence determination unit determines that the degree of influence of the light source on one of the front image and the rear image is low, outputs, as road surface information, the road surface condition of the one of the images determined by the road surface condition determination unit and the imaging point of the one of the images estimated by the road surface position estimation unit; and, when the light source influence determination unit determines that the degree of influence of the light source on the one of the images is high and the degree of influence of the light source on the other of the images is low, reads from the recording unit the other of the images captured at the same point as the one of the images, regards the read other of the images as an alternative image, and outputs, instead of road surface information regarding the one of the images, the road surface condition of the alternative image determined by the road surface condition determination unit and the imaging point of the alternative image estimated by the road surface position estimation unit;
A road surface condition determination device for a vehicle, comprising: the captured image control unit outputs road surface information regarding the one of the images when the light source influence determination unit determines that the degree of influence of the light source on the one of the images is less than a preset first determination value, and when the light source influence determination unit determines that the degree of influence of the light source on the one of the images is equal to or greater than the first determination value and that the degree of influence of the light source on the other of the images is less than the first determination value, the captured image control unit outputs road surface information by regarding the other of the images captured at the same point as the one of the images as the alternative image, The road surface condition judgment device for a vehicle as described in claim 1, characterized in that when it is judged that the degree of influence of the light source on either one of the images and the other of the images is both equal to or greater than a second judgment value set to a value greater than the first judgment value, the output of the road surface information is canceled, and when it is judged that the degree of influence of the light source on either one of the images and the other of the images is both equal to or greater than the first judgment value and at least one of the degrees of influence of the light source on both images is less than the second judgment value, the road surface information relating to the image with a smaller degree of influence of the light source is output. (delete) A moving direction determination unit that determines a moving direction of the vehicle,
The road surface condition determination device for a vehicle as described in claim 1, characterized in that when the movement direction determination unit determines that the vehicle is moving forward, the captured image control unit regards the front image as one of the images and the rear image as the other image, and when the movement direction determination unit determines that the vehicle is moving backward, the captured image control unit regards the rear image as one of the images and the front image as the other image. The road surface condition determination device for a vehicle described in any one of claims 1, 2, and 4, characterized in that the captured image control unit outputs road surface information regarding the front image and the rear image to at least one of a driving control unit that controls the driving of the vehicle and a communication device that communicates with the outside.

Images