JP6468568B2 - Object recognition device, model information generation device, object recognition method, and object recognition program - Google Patents

Description

  The present invention relates to an object recognition device, a model information generation device, an object recognition method, and an object recognition program.

  Conventionally, there is an apparatus for detecting an object such as the periphery of a vehicle. For example, an invention of an apparatus for detecting a pedestrian from an image captured by a far-infrared camera that captures an intensity distribution of thermal radiation of an object as an image is disclosed (for example, Patent Document 1). This apparatus stores pedestrian model information in an image in association with an environment outside the vehicle such as daily illuminance, and detects the pedestrian using the pedestrian model information corresponding to the environment outside the vehicle.

JP 2007-264732 A

However, the apparatus disclosed in Patent Document 1 is intended for an image captured by a far-infrared camera, and does not consider detecting a pedestrian by analyzing an image captured by a visible light camera.
For this reason, there are cases in which an object such as a pedestrian cannot be accurately recognized with respect to an image captured by a visible light camera.
The present invention has been made in view of such circumstances, and provides an object recognition device, a model information generation device, an object recognition method, and an object recognition program that can recognize an object with higher accuracy. One of the purposes.

The invention described in claim 1 is an image acquisition unit (30) that acquires an image captured by the imaging unit (12), a brightness detection unit (32) that detects brightness around the vehicle, and the brightness. according to the brightness detected by the detection unit, the image recognition unit to change the contents of processing to recognize the object included in the image acquired by the image acquiring unit (34), wherein the image recognition unit Based on the brightness detected by the brightness detection unit, one or a plurality of object model information is selected from a plurality of object model information in which an edge distribution and brightness of an image are associated with each other, and the brightness detection is performed. An object recognition device that recognizes the object by extracting an edge from the image and comparing the selected object model information with the extracted edge with the content of the processing according to the brightness detected by the unit. )

A second aspect of the present invention is the object recognition apparatus according to the first aspect, comprising a storage unit that stores the plurality of pieces of object model information .

According to a third aspect of the present invention, in the object recognition device according to the first or second aspect, the plurality of object model information (66, 68) includes a color edge acquired based on information on color, a first edge First object model information associated with brightness greater than or equal to brightness, second object model information associated with a brightness edge acquired based on information on brightness and brightness less than the second brightness; The image recognition unit acquires a color edge from the image captured by the imaging unit when the brightness detected by the brightness detection unit is equal to or higher than the first brightness, and acquires the acquired color. The object is recognized by comparing the edge and the first object model information, and when the brightness detected by the brightness detection unit is less than the second brightness, the image is captured by the imaging unit Get luminance edge from image It is intended to recognize the object by comparing the luminance edge acquired, and the second object model information.

The invention according to claim 4 is the object recognition device according to any one of claims 1 to 3, wherein the image recognition unit is irradiated by a headlamp device including a headlamp that emits light. The content of the process for recognizing the object included in the image acquired by the image acquisition unit is changed between the irradiated region and the non-irradiated region that is not irradiated .

The invention according to claim 5 is the object recognition apparatus according to claim 4 , further comprising a storage unit that stores a plurality of object model information, wherein the plurality of object model information is acquired based on information about color. The first object model information in which the color edge is associated with the brightness equal to or higher than the first brightness, the brightness edge acquired based on the information related to the brightness, and the brightness less than the second brightness are associated with each other. The second object model information, and the image recognition unit, when the brightness detected by the brightness detection unit is less than the second brightness, an irradiation area in the image acquired by the image acquisition unit Is obtained by acquiring a color edge from an image obtained by capturing an object, comparing the color edge with the first object model information, and recognizing the object, and obtaining an image acquired by the image acquisition unit. Non-irradiation included For frequency, the object acquires a luminance edge from an image captured, and the luminance edge is intended to recognize the object by comparing the second object model information.

  According to the seventh aspect of the present invention, the first correspondence information in which the first image in which an object is imaged in an environment of the first brightness or higher and the identification information of the object are associated with each other, and the second brightness An information acquisition unit that acquires a second correspondence information in which a second image obtained by imaging the object in the environment of less than the object and identification information of the object are associated, and the first acquired by the information acquisition unit A feature quantity extraction unit that extracts a first feature quantity related to color from the image of FIG. 2 and a second feature quantity related to luminance from the second image; the first feature quantity extracted by the feature quantity extraction unit; First object model information is generated by associating with object identification information, and second object model information is determined by associating the second feature amount extracted by the feature amount extraction unit with the object identification information. Generating model information including a learning generation unit for generating It is the location.

In the invention according to claim 8, the in-vehicle computer acquires an image captured by the imaging unit, detects brightness around the vehicle, and is included in the acquired image according to the detected brightness. The content of the process for recognizing the object to be detected is changed, and based on the detected brightness, one or more object model information is selected from the plurality of object model information in which the edge distribution and brightness of the image are associated The object recognition method recognizes the object by extracting an edge from the image with the content of the processing according to the detected brightness and comparing the selected object model information with the extracted edge .

The invention according to claim 9 causes the in-vehicle computer to acquire an image captured by the imaging unit, detects brightness around the vehicle, and is included in the acquired image according to the detected brightness. The content of the process for recognizing the object to be changed is changed, and one or a plurality of object model information is selected from a plurality of object model information in which the edge distribution and the brightness of the image are associated with each other based on the detected brightness is allowed, the contents of processing corresponding to the brightness obtained by the detection, by extracting an edge from the image, the object recognition program to recognize the object by comparing the previously described and selected so object model information extracted edge It is.

According to the first, third, seventh and eighth aspects of the invention, the image recognition unit recognizes an object included in the image acquired by the image acquisition unit according to the brightness detected by the brightness detection unit. By changing the processing content, the object can be recognized more accurately.

According to the third aspect of the present invention, the image recognizing unit selects the edge used for object recognition and the object model information according to the brightness detected by the brightness detecting unit. The object can be recognized stably without being.

According to the invention described in claim 4 or 5 , it is possible to more accurately recognize objects included in a region irradiated with the headlamp and a region not irradiated on the headlamp in the image. .

According to the invention described in claim 6 , the model information generating apparatus associates the color information in the image of the object captured in the environment of the first brightness or higher with the object identification information, and daytime object model information. , And generating the night object model information by associating the luminance information in the image in which the object is imaged in the environment less than the first brightness with the object identification information, Object model information that can recognize an object with higher accuracy can be generated.

2 is a diagram illustrating an example of a functional configuration of a vehicle control system 1 including an object recognition device 20. FIG. It is the figure which showed typically the content of the object model information 66 for daytime. 5 is a flowchart showing a flow of processing executed by an image recognition unit 34. 3 is a diagram illustrating a functional configuration of a model information generation device 200. FIG. 5 is a flowchart showing a flow of processing executed by the model information generating apparatus 200. It is a figure which shows an example of a function structure of 1 A of vehicle control systems of 2nd Embodiment. It is a figure which shows an example of image IM imaged with the camera 12 containing an irradiation area | region and a non-irradiation area | region. It is a flowchart which shows the flow of the process performed by 34 A of image recognition parts of 2nd Embodiment. It is a figure which shows an example of a function structure of the vehicle control system 1B of 3rd Embodiment.

  Hereinafter, embodiments of an object recognition device, a model information generation device, an object recognition method, and an object recognition program according to the present invention will be described with reference to the drawings.

<First Embodiment>
FIG. 1 is a diagram illustrating an example of a functional configuration of a vehicle control system 1 including an object recognition device 20. The vehicle control system 1 includes, for example, a radar device 10, a camera 12, an illuminance sensor 14, a vehicle sensor 16, an object recognition device 20, a vehicle control device 70, a travel drive device 72, and a brake device 74.

  The radar apparatus 10 is provided, for example, in the vicinity of a bumper of a vehicle (hereinafter referred to as the host vehicle) on which the vehicle control system 1 is mounted, a front grille, and the like. For example, the radar apparatus 10 radiates a millimeter wave in front of the host vehicle, receives a reflected wave reflected by the radiated millimeter wave hitting the object, and analyzes the received reflected wave, thereby specifying the position of the object. The position of the object includes, for example, at least a distance from the own vehicle to the object, and may include an azimuth or a lateral position of the object with respect to the own vehicle. The radar apparatus 10 detects the position of an object by, for example, FM-CW (Frequency-Modulated Continuous Wave) method, and outputs the detection result to the object recognition apparatus 20.

  The camera 12 is a digital camera using a solid-state imaging device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The camera 12 is attached to the upper part of the front windshield, the rear surface of the rearview mirror, etc. For example, the camera 12 periodically images the front of the host vehicle. For example, the camera 12 captures an image and outputs the captured image to the object recognition device 20. The camera 12 is not limited to one, and a plurality of cameras 12 may be provided in the host vehicle, or a stereo camera including a plurality of cameras may be used.

  The illuminance sensor 14 detects the brightness around the vehicle and outputs the detection result to the object recognition device 20. The vehicle sensor 16 includes a vehicle speed sensor that detects a vehicle speed, an acceleration sensor that detects acceleration, a yaw rate sensor that detects an angular velocity around a vertical axis, a direction sensor that detects the direction of the host vehicle, and the like. The vehicle sensor 16 outputs the detection result of each sensor to the vehicle control device 70.

  The object recognition device 20 includes, for example, an image acquisition unit 30, a brightness detection unit 32, an image recognition unit 34, a surrounding situation recognition unit 50, and a storage unit 60. Some or all of the image acquisition unit 30, the brightness detection unit 32, the image recognition unit 34, and the surrounding situation recognition unit 50 are realized by a processor executing a program (software). Some or all of these may be realized by hardware such as LSI (Large Scale Integration) and ASIC (Application Specific Integrated Circuit), or may be realized by a combination of software and hardware. Further, each functional unit included in the object recognition device 20 may be distributed by a plurality of computer devices. The storage unit 60 is realized by a ROM (Read Only Memory), a RAM (Random Access Memory), an HDD (Hard Disk Drive), a flash memory, or the like.

  The image acquisition unit 30 acquires an image captured by the camera 12. The image is a color image including color information and luminance information. The color information is, for example, RGB information (information of R value, G value, and B value), lightness information (information indicating the degree of brightness of the color), saturation information (information indicating the vividness of the color), And information including part or all of hue information (information indicating hue).

  The brightness detection unit 32 acquires the detection result of the illuminance sensor 14 and detects the brightness around the vehicle based on the acquired detection result. The brightness detection unit 32 acquires the exposure time set when the camera 12 captures an image from the camera 12 instead of the detection result of the illuminance sensor 14, and based on the acquired exposure time, The brightness around the vehicle may be derived. Specifically, the camera 12 controls the exposure time of the next imaging process so that the luminance value of an arbitrary region in the captured image approaches the target luminance value. When the brightness value of an arbitrary area is higher than the target brightness value, the exposure time of the next imaging process is shortened, and the brightness value of the arbitrary area is lower than the target brightness value. Increases the exposure time of the next imaging process. In this way, the exposure time is obtained by the camera 12, and information related to the exposure time is output to the brightness detection unit 32, and the brightness detection unit 32 derives the brightness around the vehicle based on the exposure time.

  The brightness detection unit 32 may derive the brightness around the vehicle based on the luminance distribution in the image acquired by the image acquisition unit 30 instead of the detection result of the illuminance sensor 14. For example, the brightness detection unit 32 detects that the brightness around the vehicle tends to be bright when the brightness value distribution tends to have a large number of regions having a brightness value equal to or higher than a predetermined value.

  The image recognition unit 34 includes, for example, a feature amount extraction unit 36, an identification unit 38, and a position specifying unit 40. The feature amount extraction unit 36 determines the content of the processing according to the brightness detected by the brightness detection unit 32, and based on the determined processing content, the feature amount extraction unit 36 extracts the feature amount from the image acquired by the image acquisition unit 30. Extract.

  When identifying the object included in the image, the identification unit 38 refers to information (the daytime object model information 66 or the nighttime object model information 68 described later) that is referred to according to the brightness detected by the brightness detection unit 32. At least one) is selected, and the object is identified using the selected information.

  The position specifying unit 40 derives the position of the object in the real space identified by the identifying unit 38. For example, the position specifying unit 40 derives the position of the object based on parameters such as the position of the camera 12, the focal length of the camera 12, and an infinite point (for example, a road vanishing point) in the image.

  The surrounding situation recognition unit 50 acquires the detection result of the radar device 10, and recognizes the position of the object based on the acquired result. Further, the surrounding situation recognition unit 50 acquires the position of the object in the real space as the processing result of the image recognition unit 34. Then, the surrounding state recognition unit 50 recognizes the speed and moving direction of the object based on the position of the object in the plurality of images. The peripheral situation recognition unit 50 places importance on the distance from the host vehicle among the positions of the objects specified by the radar device 10 and places importance on the azimuth or lateral position among the positions of the objects acquired from the image recognition unit 34. By trend, these positions may be integrated to recognize the position of the object. Further, instead of (or in addition to) the radar apparatus 10, a sensor such as a laser radar or an ultrasonic sensor may be provided.

  Further, the surrounding state recognition unit 50 may detect the position and speed of an object based on information acquired from a vehicle-to-vehicle communication or a sensor that detects a vehicle traveling on a road. In this case, the host vehicle includes a communication unit that communicates with another vehicle, a sensor that detects a vehicle traveling on the road, and the like.

  The vehicle control device 70 controls the host vehicle based on the detection result of the vehicle sensor 16 and the position of the object recognized by the surrounding situation recognition unit 50. For example, the vehicle control device 70 performs an automatic brake control that causes the brake device 74 to output a braking force when a TTC (Time To Collision) with the recognized pedestrian falls below a certain level. Further, the vehicle control device 70 may cause a speaker (not shown) to output an alarm. Further, the vehicle control device 70 may perform inter-vehicle distance control for controlling the travel drive device 72 or the brake device 74 so as to keep the inter-vehicle distance with a recognized object (for example, a vehicle) constant.

  The storage unit 60 stores, for example, a daytime processing program 62, a nighttime processing program 64, daytime object model information 66, nighttime object model information 68, and other control programs for controlling the vehicle. The daytime object model information 66 and the nighttime object model information 68 are examples of object model information.

  The daytime processing program 62 uses the image recognition unit 34 to process an image captured by the camera 12 (hereinafter referred to as a daytime captured image) when the brightness around the vehicle is equal to or higher than the first brightness. This is an image processing program used. The first brightness or higher is, for example, the brightness around the vehicle in the daytime. The daytime processing program 62 is a program for extracting feature values of color information from daytime captured images. The feature amount of color information is a part of the feature amount of RGB information, the feature amount of lightness information, the feature amount of saturation information, and the feature amount of hue information for each pixel of an image or a group of pixels obtained by grouping pixels. Or it is the total feature amount.

  The daytime processing program 62 is a program for more accurately extracting edges from the above-described color information feature amount. The edge extracted from the feature amount of the color information is an example of a color edge. An edge is a pixel or a pixel group in which the characteristic amount changes greatly between neighboring pixels or a pixel group obtained by grouping pixels. For example, the edge is extracted by obtaining a change in the feature amount between neighboring pixels or pixel groups using a SOBEL filter or the like. The daytime processing program 62 includes, for example, a threshold for extracting an edge from the feature amount of color information, a correction value, information for instructing to use the daytime object model information 66, and the like.

  The daytime processing program 62 may be a program for extracting feature amounts of color information and luminance information. In this case, the daytime processing program 62 includes threshold values, correction values, and the like for extracting edges from the feature amounts of the color information and the luminance information.

  The night processing program 64 uses the image recognition unit 34 when processing an image captured by the camera 12 (hereinafter referred to as a night captured image) when the brightness around the vehicle is less than the first brightness. This is an image processing program used. The less than the first brightness is, for example, the brightness of the surroundings of the vehicle at night, the brightness of the inside of the tunnel at daytime and nighttime, and the like. The night processing program 64 is a program for extracting a feature amount of luminance information from a night captured image. The feature amount of luminance information is information based on the luminance of each pixel or pixel group of the image.

  The night processing program 64 is a program for extracting an edge with higher accuracy from a feature amount of luminance information. The edge extracted from the feature amount of the luminance information is an example of the luminance edge. The night processing program 64 includes, for example, a threshold for extracting an edge from the feature amount of luminance information, correction values, information for instructing to use the night object model information 68, and the like.

  The daytime object model information 66 is object model information that is referred to when the daytime processing program 62 is executed and the type of the object is specified, and at least represents an edge distribution according to the type of the object. Model information is stored together with identification information of the object. The daytime object model information 66 is an example of second object model information. The type of object is a type of object to be watched when the host vehicle travels, such as a pedestrian, a vehicle, or an animal.

  FIG. 2 is a diagram schematically showing the contents of the daytime object model information 66. The daytime object model information 66 indicates, for example, an edge distribution assumed to be extracted from an image of an object such as a pedestrian. In this regard, the object model information for night 68 is the same, but even in the same image, the distribution of the edge of the color information and the distribution of the edge of the luminance information are different. The object model information 66 is prepared, and the edge distribution of the luminance information is prepared as the night object model information 68.

  Further, in the daytime object model information 66, the object model information may be subdivided and stored for each part of the object. The daytime object model information 66 is information generated based on the edge extracted from the daytime captured image and correct information (which object image is) for the daytime captured image. Details of the method for generating the daytime object model information 66 will be described later.

  The night object model information 68 is object model information that is referred to when the night processing program 64 is executed and the type of the object is specified, and at least represents an edge distribution according to the type of the object. Model information is stored together with identification information of the object. The night object model information 68 is an example of first object model information. Normally, the number of edges corresponding to one object included in the nighttime object model information 68 is smaller than the number of edges corresponding to one object included in the daytime object model information 66. Therefore, the amount of information included in the nighttime object model information 68 is smaller than the amount of information included in the daytime object model information 66.

  Further, in the nighttime object model information 68, the object model information may be subdivided and stored for each part of the object. The night object model information 68 is information generated based on the edge extracted from the night-time captured image and the correct answer information for the night-time captured image. Details of the method for generating the nighttime object model information 68 will be described later.

  The travel drive device 72 is a drive source such as an engine or a travel motor, for example. The travel drive device 72 outputs torque for traveling the host vehicle to the drive wheels in accordance with the control amount output by the object recognition device 20. The brake device 74 includes an electric motor, for example. For example, the electric motor causes each wheel to output a brake torque corresponding to a braking operation in accordance with a control amount output by the object recognition device 20.

  FIG. 3 is a flowchart showing the flow of processing executed by the image recognition unit 34. This process is executed each time the image acquisition unit 30 acquires an image captured by the camera 12, for example. Note that the process of this flowchart is expressed as t in the current routine on the premise that the process is repeatedly executed.

  First, the image acquisition unit 30 acquires an image captured by the camera 12 (step S100). Next, the brightness detection unit 32 acquires the brightness detected by the illuminance sensor 14 (step S102), and determines whether the acquired brightness is equal to or higher than the first brightness (step S104). . If the current time is in the first time zone instead of the brightness, the process proceeds to step S106. If the current time is not the first time zone, the process in step S110 is performed. You may proceed to. A 1st time slot | zone is a time slot | zone corresponding to the daytime in the time, for example. In this case, the vehicle control system 1 includes a timekeeping unit such as a radio clock or a car navigation device, and the object recognition device 20 acquires time from the radio clock or the car navigation device.

  If the acquired brightness is equal to or higher than the first brightness, the feature amount extraction unit 36 uses the daytime processing program 62 to extract an edge from the acquired image (step S106). Next, the identification unit 38 compares the edge extracted in step S106 with the daytime object model information 66 to identify an object included in the image (step S108). For example, the identifying unit 38 derives the degree of similarity between the edge extracted in step S106 and the edge associated with the type of object included in the daytime object model information 66. Then, the identification unit 38 identifies the object associated with the edge whose similarity degree is greater than or equal to the predetermined value in the daytime object model information 66 as the object corresponding to the edge extracted in step S106.

  If the acquired brightness is less than the first brightness, the feature amount extraction unit 36 uses the night processing program 64 to extract an edge from the acquired image (step S110). Next, the identification unit 38 compares the edge extracted in step S110 with the nighttime object model information 68 to identify an object included in the image (step S112). For example, the identification unit 38 derives the degree of similarity between the edge extracted in step S110 and the edge associated with the type of object included in the nighttime object model information 68. Then, the identification unit 38 identifies the object associated with the edge whose similarity degree is greater than or equal to the predetermined value in the nighttime object model information 68 as the object corresponding to the edge extracted in step S110.

  Next, the surrounding situation recognition unit 50 derives a relative position of the identified object with respect to the host vehicle (step S114). The relative position is a relative position on the real plane and is a position mapped from a position on the image plane.

  Next, the surrounding state recognition unit 50 determines whether or not the object recognized from the image captured at time t-1 (that is, in the previous routine) and the object recognized in the process of the current routine are the same object. Is determined (step S116). In this step, the surrounding state recognition unit 50 determines whether or not they are the same object based on the transition of the relative position derived in step S114 in the current and previous routines.

  When it is determined that they are the same object, the surrounding state recognition unit 50 derives the moving direction and moving speed of the object based on the relative position derived in step S114 in the routine before this time and the previous time (step S118). ). The movement direction and the movement speed derived here are used for the process of estimating the position of the object in the image acquired by the process of the next routine. If the objects recognized in the routine processing are not the same, the processing in step S118 is skipped. Thereby, the processing of this flowchart is completed.

  By the above-described processing, the object recognition device 20 changes the content of the processing for recognizing the object included in the image acquired by the image acquisition unit 30 according to the brightness detected by the illuminance sensor 14, thereby An object can be recognized with high accuracy. Thereby, the object recognition apparatus 20 can control the own vehicle according to the object which exists around the own vehicle. As a result, the host vehicle can perform appropriate control according to the surrounding situation.

  In the above-described processing, the daytime processing program 62 is used when the brightness is equal to or higher than the first brightness, and the processing is performed using the nighttime processing program 64 when the brightness is lower than the first brightness. Alternatively, the daytime processing program 62 may be used when the brightness is equal to or higher than the first brightness, and the nighttime processing program 64 may be used when the brightness is less than the second brightness. In this case, the first brightness (“first brightness” in the claims) and the second brightness (“second brightness” in the claims) have the same value. There may be different values.

  In addition, as described above, instead of performing processing by classifying the brightness in two stages, the brightness is classified into three or more stages, and the type of the object is determined using a processing program corresponding to each brightness. You may specify.

  For example, when the brightness is classified into three levels, the brightness is set to brightness 1, brightness 2, and brightness 3 from the lowest brightness. In this case, the image recognition unit 34 processes the image using the night processing program 64 having different parameters for extracting the edge of the luminance information for the brightness 1 and the brightness 2, and for the day 3 for the brightness 3. The image may be processed using a processing program. The image recognizing unit 34 processes the image using the nighttime processing program 64 for the brightness 1 and processes for the daytime when the parameters for extracting the edge of the color information are different for the brightness 2 and the brightness 3, respectively. The program 66 may be used to process the image.

<Generation method of object model information>
Hereinafter, a method of generating daytime object model information 212 corresponding to daytime object model information 66 and nighttime object model information 214 corresponding to nighttime object model information 68 will be described. FIG. 4 is a diagram illustrating a functional configuration of the model information generation apparatus 200. The model information generation device 200 includes, for example, an information acquisition unit 202, a feature amount extraction unit 204, a learning unit 206, and a storage unit 210.

  The model information generating apparatus 200 performs the following processing for both daytime and nighttime use. The model information generation device 200 performs processing for obtaining daytime object model information 212 for daytime stored in the storage unit 210 based on color information of an image obtained by capturing an object in an environment of the first brightness or higher. Then, based on the luminance information of the image obtained by capturing the object in the environment of less than the first brightness, processing for obtaining the night-time object model information 214 stored in the storage unit 210 is performed. Hereinafter, when the daytime object model information 212 and the nighttime object model information 214 are not distinguished, they are referred to as “object model information”. Further, when the daytime captured image and the nighttime captured image are not distinguished, they are simply referred to as “captured images”. Note that the model information generating apparatus 200 uses the night information for nighttime stored in the storage unit 210 based on luminance information of an image obtained by capturing an object in an environment less than the second brightness different from the first brightness. Processing for obtaining the object model information 214 may be performed.

  The information acquisition unit 202 acquires learning information transmitted or input from another device. The learning information is, for example, information for generating object model information, and is a combination of a captured image and identification information of an object included in the captured image.

  The feature amount extraction unit 204 extracts an edge from the captured image acquired by the information acquisition unit 202. The learning unit 206 learns the relationship between the edge extracted by the feature amount extraction unit 204 and the identification information of the object included in the captured image. The learning unit 206 statistically processes the learned result, thereby deriving object model information in which the edge included in the captured image is associated with the object identification information. As described above, the object model information obtained for the daytime and the nighttime is reflected in the daytime object model information 66 and the nighttime object model information 68 in the object recognition apparatus 20.

  FIG. 5 is a flowchart showing a flow of processing executed by the model information generating apparatus 200. First, the information acquisition unit 202 acquires a combination of a captured image and identification information (correct information) of an object included in the captured image (step S200). Next, the feature amount extraction unit extracts an edge from the captured image acquired by the information acquisition unit 202 (step S202). Next, the learning unit 206 learns the relationship between the edge extracted by the feature amount extraction unit 204 and the identification information of the object included in the captured image (step S204). Thereby, the process of this flowchart is complete | finished.

  Through the above-described processing, the model information generation apparatus 200 generates daytime object model information 212 by associating color information in an image of an object captured in an environment of the first brightness or higher with object identification information. Even in an environment where the brightness is different by generating the night-time object model information 214 by associating the luminance information in the image in which the object is captured in the environment of the first brightness and the identification information of the object. It is possible to generate object model information that can recognize an object with higher accuracy.

  According to the first embodiment described above, the image recognition unit 34 recognizes an object included in the image acquired by the image acquisition unit 30 according to the brightness detected by the brightness detection unit 32. By changing the content of, the object can be recognized with higher accuracy.

<Second Embodiment>
Hereinafter, the second embodiment will be described. In the first embodiment, the irradiation area of the headlamp device is not considered in the processing of the night-time captured image. On the other hand, in 2nd Embodiment, in the process of a night captured image, a different process is performed by the irradiation area | region and non-irradiation area | region of a headlamp apparatus. Here, the difference from the first embodiment will be mainly described, and description of functions and the like common to the first embodiment will be omitted.

  FIG. 6 is a diagram illustrating an example of a functional configuration of the vehicle control system 1A according to the second embodiment. The vehicle control system 1A further includes a headlamp device 76 in addition to the functional configuration of the first embodiment. Further, the vehicle control system 1A includes an object recognition device 20A instead of the object recognition device 20 of the first embodiment.

  The headlamp device 76 includes, for example, a headlamp (not shown) and a headlamp control unit (not shown). The headlamp illuminates the front of the host vehicle based on the control of the headlamp control unit. The headlamp control unit controls the headlamp to adjust the height direction of the irradiation range, the vehicle width direction of the irradiation range, or the intensity of irradiation light. For example, the headlamp control unit switches between so-called low beam and high beam. The headlamp device 76 outputs information indicating the irradiation range of the headlamp, the intensity of irradiation light, and the like to the object recognition device 20A.

  The object recognition device 20A includes an image recognition unit 34A in place of the image recognition unit 34 of the first embodiment. The image recognition unit 34A further includes an irradiation area specifying unit 42 in addition to the functional units of the image recognition unit 34 of the first embodiment. The irradiation area specifying unit 42 acquires the irradiation area of the headlamp from the headlamp device 76 and the imaging area of the image from the camera 12. The irradiation region specifying unit 42 specifies an irradiation region and a non-irradiation region in the image acquired by the image acquisition unit 30 based on the acquired irradiation region and the imaging region of the camera 12.

  The feature amount extraction unit 36 acquires information indicating the irradiation region and the non-irradiation region in the image from the irradiation region specifying unit 42. The feature amount extraction unit 36 processes the image using the daytime processing program 62 for the irradiation region in the image, and processes the image using the nighttime processing program 64 for the non-irradiation region in the image. To do. The identification unit 38 is executed using a part or all of the objects identified based on the processing result executed using the daytime processing program 62 and the daytime object model information 66, and the nighttime processing program 64. The object is recognized by integrating a part or all of the objects identified based on the processing result and the night object model information 68.

  FIG. 7 is a diagram illustrating an example of an image IM captured by the camera 12 including an irradiation region and a non-irradiation region. In the illustrated example, the lower body of the person H exists in the irradiation area A1, and the upper body of the person H exists in the non-irradiation area A2. The feature amount extraction unit 36 performs processing on the irradiation area A1 including the lower half of the person H in the drawing using the daytime processing program 62, and the non-irradiation area A2 including the upper half of the person H in the drawing. Is processed using the night processing program 64. The identification unit 38 identifies a part centered on the lower half of the person H based on the edge extracted by the feature amount extraction unit 36 and the daytime object model information 66, and the edge extracted by the feature amount extraction unit 36 Then, based on the nighttime object model information 68, a part centered on the upper body of the person H is identified, and the identification result is integrated to identify that the object is the person H.

  FIG. 8 is a flowchart showing a flow of processing executed by the image recognition unit 34A of the second embodiment. This process is executed each time the image acquisition unit 30 acquires an image captured by the camera 12, for example. Note that the process of this flowchart is expressed as t in the current routine on the premise that the process is repeatedly executed.

  First, the image acquisition unit 30 acquires an image captured by the camera 12 (step S300). Next, the brightness detection unit 32 acquires the brightness detected by the illuminance sensor 14 (step S302), and determines whether the acquired brightness is equal to or higher than the first brightness (step S304). .

  When the acquired brightness is equal to or higher than the first brightness, the feature amount extraction unit 36 uses the daytime processing program 62 to extract an edge from the acquired image (step S306). Next, the identifying unit 38 identifies the object included in the image by comparing the edge extracted in step S306 with the daytime object model information 66 (step S308). Hereinafter, the process of extracting the edge in step S306 and the process of identifying the object in step S308 are collectively referred to as “daytime object recognition process”.

  When the brightness acquired in step S304 is less than the first brightness, the irradiation area specifying unit 42 determines whether or not the irradiation area exists in the image acquired in step S300 (step S310).

  When the irradiation area does not exist (when the image is only the non-irradiation area), the feature amount extraction unit 36 uses the night processing program 64 to extract the edge from the acquired image (step S312). Next, the identification unit 38 compares the edge extracted in step S314 with the nighttime object model information 68 to identify an object included in the image (step S314), and the process proceeds to step S320. Hereinafter, the process of extracting the edge in step S312 and the process of identifying the object in step S314 are collectively referred to as “night object recognition process”.

  When the irradiation area exists, the image recognition unit 34A performs “daytime object recognition processing” on the irradiation area of the image (step S316). Here, 34 A of image recognition parts may change the threshold value when extracting an edge with the process program for daytime for every area | region included in an irradiation area | region in an image. For example, when the image recognition unit 34A converts the region to be processed included in the irradiation region into a distance from the own vehicle in the real space, the image recognition unit 34A sets a threshold for extracting an edge as the distance is farther from the own vehicle. Execute the process with a downward trend. In this manner, the image recognition unit 34A can extract the edge more accurately by taking into account the attenuation degree of the headlamp.

  Next, the feature amount extraction unit 36 performs “night object recognition processing” on the non-irradiated region of the image (step S318). Next, the surrounding situation recognition unit 50 derives the relative position of the object with respect to the host vehicle (step S320). Next, the surrounding state recognition unit 50 determines whether or not the object recognized in the image captured at time t-1 and the object recognized in the processing of this routine are the same object ( Step S322).

  If they are the same object, the surrounding situation recognition unit 50 derives the moving direction and moving speed of the object based on the relative positions derived in step S320 in the current and previous routines (step S324). If the object recognized in the routine processing is not the same object, the processing in step S324 is skipped. Thereby, the process of this flowchart is complete | finished.

  According to the second embodiment described above, the image recognizing unit 34 obtains an image using the irradiation area irradiated by the headlamp device 76 and the non-irradiated area not irradiated by the headlamp device 76. By changing the content of the process for recognizing the object included in the image acquired by the unit 30, the same effect as in the first embodiment can be obtained, and included in the irradiation area irradiated by the headlamp device 76 Can be recognized more accurately.

<Third Embodiment>
Hereinafter, a third embodiment will be described. In the first embodiment, the vehicle control device 70 executes automatic brake control, alarm output, or inter-vehicle distance control based on the processing result of the object recognition device 20, but in the third embodiment, The automatic driving control device 100 performs automatic driving control based on the processing result of the object recognition device 20. Here, the difference from the first embodiment will be mainly described, and description of functions and the like common to the first embodiment will be omitted.

  FIG. 9 is a diagram illustrating an example of a functional configuration of the vehicle control system 1B according to the third embodiment. The vehicle control system 1B includes an automatic operation changeover switch 120 in addition to the functional configuration of the vehicle control system 1 of the first embodiment. The vehicle control system 1B includes an automatic driving control device 100 in place of the vehicle control device 70. The automatic driving control device 100 includes a storage unit 102, a target lane determining unit 104, and an automatic driving control unit 110. The storage unit 102 stores, for example, information such as high-precision map information, target lane information, and action plan information.

  The target lane determining unit 104 is realized by, for example, an MPU (Micro-Processing Unit). The target lane determination unit 104 divides the route provided from the navigation device into a plurality of blocks (for example, every 100 [m] with respect to the vehicle traveling direction), and refers to the high-accuracy map information for each block. Determine the lane. The target lane determining unit 104 determines, for example, what number lane from the left to travel. For example, the target lane determination unit 104 determines the target lane so that the host vehicle can travel on a reasonable travel route for proceeding to the branch destination when there is a branch point or a merge point in the route. The target lane determined by the target lane determination unit 104 is stored in the storage unit 102 as target lane information.

  The automatic driving control unit 110 includes, for example, an action plan generation unit 112, a track generation unit 114, a travel control unit 116, and a switching control unit 118.

  The action plan generation unit 112 sets a starting point of automatic driving and / or a destination of automatic driving. The action plan generation unit 112 generates an action plan in a section between the start point and the destination for automatic driving. The action plan is composed of, for example, a plurality of events that are sequentially executed. Events include, for example, a deceleration event for decelerating the host vehicle, an acceleration event for accelerating the host vehicle, a lane keeping event for driving the host vehicle so as not to deviate from the driving lane, a lane change event for changing the driving lane, Events that follow the vehicle are included. Information indicating the action plan generated by the action plan generation unit 112 is stored in the storage unit 102 as action plan information.

  The track generation unit 114 determines and determines one of the traveling modes such as constant speed traveling, following traveling, low speed following traveling, deceleration traveling, curve traveling, obstacle avoidance traveling, lane change traveling, merging traveling, branch traveling, and the like. A trajectory candidate is generated based on the travel mode. For example, the track generation unit 114 generates a track as a collection of target positions (track points) that the reference position (for example, the center of gravity and the center of the rear wheel axis) of the host vehicle should reach at predetermined future times.

  The travel control unit 116 controls the travel drive device 72 or the brake device 74 so that the host vehicle passes the track generated by the track generation unit 114 at a scheduled time. The switching control unit 118 switches between the automatic operation mode and the manual operation mode based on a signal input from the automatic operation switch 120.

  According to the third embodiment described above, the same effects as those of the first embodiment and the second embodiment can be achieved, and automatic operation can be performed with higher accuracy.

  As mentioned above, although the form for implementing this invention was demonstrated using embodiment, this invention is not limited to such embodiment at all, In the range which does not deviate from the summary of this invention, various deformation | transformation and substitution Can be added.

DESCRIPTION OF SYMBOLS 1,1A, 1B ... Vehicle control system, 12 ... Camera, 14 ... Illuminance sensor, 20, 20A ... Object recognition device, 30 ... Image acquisition part, 32 ... Brightness detection part, 34, 34A ... Image recognition part, 36 ... Feature amount extraction unit 38. Identification unit 50 Peripheral situation recognition unit 66, 212 Daytime object model information 68, 214 Nighttime object model information 76 Headlight device 200 Model information generation device 202 Information acquisition unit 204 Feature amount extraction unit 206 Learning unit

Claims (8)

An image acquisition unit for acquiring an image captured by the imaging unit;
A brightness detector for detecting the brightness of the surroundings of the vehicle;
An image recognition unit that changes the content of processing for recognizing an object included in the image acquired by the image acquisition unit according to the brightness detected by the brightness detection unit;
The image recognition unit
Based on the brightness detected by the brightness detection unit, selecting one or a plurality of object model information from a plurality of object model information associated with the distribution of the edge of the image and the brightness,
With the content of the processing according to the brightness detected by the brightness detection unit, an edge is extracted from the image,
Recognizing the object by comparing the selected object model information with the extracted edge;
Object recognition device. A storage unit storing the plurality of object model information;
The object recognition apparatus according to claim 1. The plurality of object model information is:
First object model information in which a color edge acquired based on information on color is associated with a brightness greater than or equal to the first brightness;
Second object model information in which a luminance edge acquired based on information on luminance and a brightness less than the second brightness are associated with each other,
The image recognition unit
When the brightness detected by the brightness detection unit is equal to or higher than the first brightness, a color edge is acquired from the image captured by the imaging unit, the acquired color edge, and the first object model information To recognize the object,
When the brightness detected by the brightness detection unit is less than the second brightness, a brightness edge is acquired from the image captured by the imaging unit, the acquired brightness edge, the second object model information, To recognize the object by comparing
The object recognition apparatus according to claim 1 or 2. The image recognition unit is included in the image acquired by the image acquisition unit in an irradiation region irradiated by a headlamp device including a headlamp that irradiates light and a non-irradiation region that is not irradiated. Change the process of recognizing objects,
The object recognition device according to claim 1. A storage unit storing a plurality of object model information;
The plurality of object model information is:
First object model information in which a color edge acquired based on information on color is associated with a brightness greater than or equal to the first brightness;
Second object model information in which a luminance edge acquired based on information on luminance and a brightness less than the second brightness are associated with each other,
The image recognition unit
When the brightness detected by the brightness detection unit is less than the second brightness, a color edge is acquired from the image of the object captured for the irradiation area in the image acquired by the image acquisition unit. The object is recognized by comparing the color edge with the first object model information, and the non-irradiation region included in the image acquired by the image acquisition unit Recognizing the object by obtaining a luminance edge and comparing the luminance edge with the second object model information;
Object recognition apparatus according to claim 4 Symbol mounting. First correspondence information in which a first image in which an object is imaged in an environment of a first brightness or higher is associated with identification information of the object, and the object is imaged in an environment of less than a second brightness An information acquisition unit configured to acquire second correspondence information in which the second image thus obtained is associated with the identification information of the object;
A feature quantity extraction unit that extracts a first feature quantity related to color from the first image acquired by the information acquisition section and a second feature quantity related to luminance from the second image;
First object model information is generated by associating the first feature amount extracted by the feature amount extraction unit with the identification information of the object,
A learning generation unit that generates second object model information by associating the second feature amount extracted by the feature amount extraction unit with the identification information of the object;
A model information generating apparatus comprising: In-vehicle computer
Obtain an image captured by the imaging unit,
Detect the brightness around the vehicle,
In accordance with the detected brightness, change the content of the process for recognizing the object included in the acquired image,
Based on the detected brightness, select one or a plurality of object model information from a plurality of object model information associated with the distribution of the edge of the image and the brightness,
In the content of the processing according to the detected brightness, an edge is extracted from the image,
Recognizing the object by comparing the selected object model information with the extracted edge;
Object recognition method. On-board computer
The image captured by the imaging unit is acquired,
Detect the brightness around the vehicle,
According to the detected brightness, change the content of the process for recognizing the object included in the acquired image,
Based on the detected brightness, one or a plurality of object model information is selected from a plurality of object model information associated with the distribution of the edge of the image and the brightness,
In the content of the processing according to the detected brightness, the edge is extracted from the image,
Recognizing the object by comparing the selected object model information with the extracted edge;
Object recognition program.

Images