JP3960223B2 - Object recognition apparatus and object recognition method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an object recognition apparatus and an object recognition method for recognizing a recognition target object from an image.
[0002]
[Prior art]
Conventionally, when recognizing a target object from a color image obtained by copying a recognition target object such as a road sign, the color information is recognized based on lightness information of red, blue, or green, or hue information derived from the lightness information. Recognize based on. Thus, for example, Patent Document 1 discloses a technique for recognizing a marker that is a recognition target object from a color image based on lightness information and hue information of the color image.
[0003]
[Patent Document 1]
Japanese Patent Laid-Open No. 9-185703
[Problems to be solved by the invention]
However, when the recognition target is recognized based on the hue information according to the conventional technique, the red position in the hue circle is arranged around 0 degrees, and therefore, for example, 0 degrees and 359 degrees of the hue circle. Although the red corresponding to the position of is a red corresponding to the same hue, the normalized hue value is divided into 0 and 255. Therefore, when the recognition process is performed based on such a hue value, the degree of correlation between the hue values in the calculation process of the recognition process is lowered, and the recognition performance is degraded.
[0005]
Also, when recognizing a recognition object based on lightness information according to the conventional technique, if the object projected in the original color image has a sunlit part and a shaded part, the color is originally the same color. Will be recognized as different colors in the sunny and shaded areas. That is, when recognizing a recognition object from a color image in which the object has a sunlit part and a shaded part, the recognition performance is degraded.
[0006]
Accordingly, an object of the present invention is to provide an object recognition apparatus and an object recognition method capable of improving the recognition performance when recognizing a recognition target object in order to solve the above-described problems.
[0007]
[Means for Solving the Problems]
The object recognition device according to the present invention is an object recognition device that recognizes a recognition target in which a color to be used is preset, and hue information of a color used for the recognition target is captured by an imaging unit. The hue value included in the hue information extracted by the extraction means is shifted so that the hue value corresponding to the color other than the color used for the recognition object and the extraction means extracted from the image becomes the reference value of the hue. Shift means, normalization means for normalizing the hue value shifted by the shift means, recognition means for recognizing the recognition object based on the values normalized by the normalization means, and saturation information of the captured image Based on the brightness information, a black and white extraction unit that extracts a region corresponding to white or black from the captured image, and a hue value corresponding to the region extracted by the black and white extraction unit corresponds to white or black. color Comprising a hue value changing means for changing the value, the normalization means is characterized by normalizing the changed hue values by the hue-value change means.
[0008]
Further, the object recognition method of the present invention is an object recognition method for recognizing a recognition target in which a color to be used is set in advance, and the extraction means calculates the hue information of the color used for the recognition target, extracted from the image captured by the imaging unit, the hue shift means, a hue value corresponding to the color other than the color used for the recognition object is contained in such that the reference value of the hue, the extracted color information The value is shifted, the normalization means normalizes the shifted hue value, the recognition means recognizes the recognition object based on the normalized value, and the black- and- white extraction means detects the color of the captured image. Based on the saturation information and lightness information, an area corresponding to white or black is extracted from the captured image, and the hue value changing means corresponds to the hue value corresponding to the extracted area corresponding to white or black. change in hue value, normalizing means, normalization In, characterized in that said normalizing the changed hue values.
[0009]
According to these inventions, the reference value of the hue is shifted to the position of the hue corresponding to the color other than the color used for the recognition object, and the recognition of the recognition object is performed after normalizing the hue value after the shift. Since it can be performed, it is possible to prevent a situation in which the color used for the recognition object is divided after the normalization of the hue value. Therefore, since recognition can be performed without reducing the degree of correlation between hue values, recognition performance can be improved. In addition, since the recognition object is recognized based on the hue information without using the brightness information, the recognition performance can be improved even if the recognition object projected in the captured image has a sunlit part and a shaded part. The recognition object can be recognized from the captured image without being lowered.
[0012]
In this way, a specific hue value can be assigned to an area corresponding to white or black included in the recognition object, and the recognition object can be recognized after the hue value is normalized. Therefore, the recognition performance can be further improved.
[0013]
In the object recognition apparatus of the present invention, the recognition means preferably performs recognition based on pattern recognition.
[0014]
Thus, recognition of a recognition target object can be performed more effectively by using pattern recognition.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of a sign recognition device according to the present invention will be described with reference to the drawings. First, the functional configuration of the sign recognition apparatus 1 in the present embodiment will be described with reference to FIG. As shown in FIG. 1, the sign recognition device 1 includes an imaging unit (imaging unit) 11, an HSI conversion unit 12, a hue image generation unit (extraction unit) 13, a hue value shift unit (shift unit) 14, and a monochrome area extraction unit. (Monochrome extraction means) 15, monochrome / hue value changing unit (hue value changing unit) 16, hue value normalizing unit (normalizing unit) 17, sign recognition unit (recognizing unit) 18, and output unit 19. Note that the sign recognition device 1 according to the present embodiment is a device mounted on, for example, an automobile. The sign recognition device 1 is a device that, when recognizing a road sign installed beside a road, etc., prompts the driver to be aware of traffic by displaying the content of the road sign on a display. is there. Hereinafter, each component will be described.
[0016]
The imaging unit 11 is a device incorporating an imaging device such as a CCD camera, for example, and captures a landscape developed near the front of the automobile to which the sign recognition device 1 is attached, and displays a color image (for example, an RGB image). get.
[0017]
The HSI converter 12 converts the RGB color image into a color image having hue, saturation, and lightness component information using HSI (Hue Saturation Intensity) conversion. Note that the method of converting a color image into a color image having each component information of hue, saturation, and brightness is not limited to the above-described HSI conversion, and may be, for example, HSV (Hue Saturation intensity Value) conversion.
[0018]
Based on the hue component information (hue information) and the saturation component information (saturation information) included in the color image, the hue image generation unit 13 selects pixels (regions) corresponding to red, blue, and yellow from the color image. Extraction is performed to generate a hue image having hue component information of red, blue, and yellow.
[0019]
The hue value shift unit 14 shifts the hue value corresponding to each pixel included in the hue image by −120 degrees. The hue value shift will be described with reference to FIG. FIG. 2A is a diagram showing a general hue circle. Red is arranged at the reference value (0 degree position) of the hue circle shown in FIG. Further, green is arranged at a position of 120 degrees on the hue circle. FIG. 2B is a diagram showing the hue circle after the hue values of the hue circle shown in FIG. 2A are shifted by −120 degrees. The reference value of the hue circle shown in FIG. 2B is shifted to the green position of the hue circle shown in FIG. As a result, the hue value (120 degrees) corresponding to green, which is not used in principle for road signs among the colors represented in the hue circle, is shifted to the 0 degree position (reference value) of the hue circle. Become. Therefore, the hue value corresponding to the color (red / blue / yellow) used for the road sign that is the recognition target is not divided into 0 degree and 359 degree. Therefore, it is possible to prevent a decrease in the correlation degree in the recognition process performed after normalizing the hue value, and thus improve the recognition performance. The shift of the hue value is not limited to the above-described −120 degree shift, and the hue values corresponding to the colors (red, blue, yellow) used for the road sign that is the recognition target are 0 degrees and 359. What is necessary is just to shift so that it may not be divided. That is, the hue value corresponding to the color (green) that is not used in principle for the road sign that is the recognition target may be shifted so as to be arranged at the 0 degree position. In addition, the color used for road signs is almost never green. Road signs are often placed beside roads where roadside trees are planted. In order to make it easier to distinguish from green, it is considered that green is not used in principle.
[0020]
Based on the saturation component information included in the color image, the black and white area extraction unit 15 converts pixels (areas) whose saturation value is smaller than a predetermined threshold (for example, 90) from the color image to white or black. Extract as corresponding pixels. The reason why pixels corresponding to white or black are extracted in this way is that the saturation value of white or black is smaller than the saturation values of other colors. Here, the predetermined threshold may be set near the upper limit of the saturation value that can be assumed to be white or black. In addition, the black and white area extraction unit 15 determines a lightness value from a predetermined threshold (for example, 128) from each pixel corresponding to the extracted white or black based on the lightness component information (lightness information) included in the color image. The smaller pixel is extracted as a pixel corresponding to black, and a hue image having black hue component information is generated. The reason why pixels corresponding to black are extracted in this way is that the brightness value of black is smaller than the brightness value of white. Further, the black and white area extraction unit 15 selects pixels having a lightness value equal to or greater than a predetermined threshold (for example, 128) from the pixels corresponding to the extracted white or black based on the lightness component information included in the color image. Extracted as pixels corresponding to white, a hue image having hue component information of white is generated. Here, the predetermined threshold value may be set near the upper limit of the brightness value that can be assumed to be black or near the lower limit of the brightness value that can be assumed to be white.
[0021]
The black and white / hue value changing unit 16 changes the hue value of the pixel corresponding to white extracted by the black and white region extracting unit 15 to, for example, 359 degrees, and the pixel corresponding to black extracted by the black and white region extracting unit 15 Is changed to, for example, 2 degrees (see FIG. 2B). Thereby, white / black hue component information is generated. Note that the hue values of the pixels corresponding to white and black do not necessarily have to be changed to 359 degrees or 2 degrees as described above. What is necessary is just to change into the hue value which can distinguish clearly from the color (red, blue, yellow) currently used as a road sign.
[0022]
Here, since hues generally do not include white and black, when an object with white or black is represented in a hue image, the white or black part of the object is a chromatic color that approximates that color. Expressed as hue. Therefore, conventionally, it is extremely difficult to clearly distinguish between white and black parts of an object and parts represented by other colors by hue, and an object with white or black is displayed as a hue image. It was impossible to recognize on the basis of Therefore, in the sign recognition apparatus 1 according to the present embodiment, the black and white area extraction unit 15 extracts pixels that are highly likely to correspond to white or black based on the saturation component information and the lightness component information, and changes the monochrome / hue value. The unit 16 assigns a specific hue value to the extracted pixel so that the white portion or the black portion of the target object can be clearly identified by the hue.
[0023]
The hue value normalization unit 17 converts (normalizes) each hue value (0 degrees to 359 degrees) of the hue image having the hue component information of red, blue, yellow, white, and black into 0 to 255. Thus, for example, the normalized value of the hue value (359 degrees) corresponding to white is 255, and the normalized value of the hue value (2 degrees) corresponding to black is 1. In the present embodiment, the hue image after being normalized by the hue value normalization unit 17 is used as a hue image for sign recognition (HITS: Hue Image for Traffic Sign).
[0024]
The sign recognition unit 18 performs a recognition process for recognizing a road sign that is a recognition target object from a sign recognition hue image, for example, based on a pattern recognition method using normalized correlation. The recognition process will be described in detail. The sign recognition unit 18 compares the hue image of the road sign and the hue image for sign recognition on a pixel-by-pixel basis, calculates the degree of coincidence, and recognizes the sign based on the degree of coincidence. It is recognized whether or not a road sign is included in the hue image for use. Note that the pattern recognition method is not limited to the normalized correlation, and may be, for example, template matching using a sum of differences or a neural network.
[0025]
The output unit 19 displays the content of the sign corresponding to the road sign recognized by the sign recognition unit 18 on the display. Note that the output by the output unit 19 is not limited to displaying the content of the sign on the display. For example, the content of the sign may be output as audio from a speaker.
[0026]
Next, the operation of the sign recognition apparatus 1 in this embodiment will be described with reference to FIG.
[0027]
As shown in FIG. 3, first, the imaging unit 11 of the sign recognition apparatus 1 acquires a color image of a landscape imaged by a CCD camera (step S1).
[0028]
Next, the HSI conversion unit 12 converts the color image acquired by the imaging unit 11 into a color image having each component information of hue, saturation, and brightness based on the HSI conversion (step S2).
[0029]
Next, the hue image generation unit 13 extracts pixels corresponding to red, blue, and yellow from the color image based on the hue component information and the saturation component information included in the color image converted by the HSI conversion unit 12. Then, a hue image having the hue component information of red, blue, and yellow is generated (step S3).
[0030]
Next, the hue value shift unit 14 shifts the hue value corresponding to each pixel included in the hue image generated by the hue image generation unit 13 by −120 degrees (step S4).
[0031]
Next, the black and white area extraction unit 15 is a pixel extracted by the hue image generation unit 13 among the pixels included in the color image based on the saturation component information included in the color image converted by the HSI conversion unit 12. A pixel having a saturation value smaller than a predetermined threshold (for example, 90) is extracted as a pixel corresponding to white or black from each of the other pixels (step S5).
[0032]
Next, the black and white area extraction unit 15 determines that the brightness value of the pixel corresponding to white or black extracted in step S5 is a predetermined threshold based on the brightness component information included in the color image converted by the HSI conversion unit 12. It is determined whether it is smaller than (for example, 128) (step S6). When this determination is NO (step S6; NO), the monochrome / hue value changing unit 16 changes the hue value of the determined pixel to, for example, 359 degrees, and changes the changed hue value to the step. This is reflected in the hue image after the hue value is shifted in S4 (step S7). Thereby, the hue value of the pixel corresponding to white is changed to 359 degrees, and this hue value is reflected in the hue image.
[0033]
On the other hand, when it is determined in step S6 that the brightness value of the pixel corresponding to white or black extracted in step S5 is smaller than a predetermined threshold (for example, 128) (step S6; YES), black and white The hue value changing unit 16 changes the hue value of the determined pixel, for example, twice, and reflects the changed hue value in the hue image after the hue value is shifted in step S4 ( Step S8). As a result, the hue value of the pixel corresponding to black is changed twice and this hue value is reflected in the hue image. In this way, by reflecting the hue value corresponding to the changed white or black in the hue image, the hue image having the hue component information of red, blue, yellow is changed to the hue of red, blue, yellow, white, black. It is converted into a hue image having component information.
[0034]
Next, the hue value normalization unit 17 normalizes each hue value (0 degrees to 359 degrees) of the hue image having the hue component information of red, blue, yellow, white, and black to 0 to 255, and signs A recognition hue image is generated (step S9).
[0035]
Next, the sign recognition unit 18 performs a recognition process for recognizing a road sign based on the sign recognition hue image generated by the hue value normalization unit 17 (step S10).
[0036]
Then, when the sign recognition unit 18 recognizes that the road sign is included in the sign recognition hue image, the output unit 19 displays the content of the sign corresponding to the recognized road sign on the display. (Step S11).
[0037]
In the operation of the sign recognition device 1 described above, after the hue image generation unit 13 generates a hue image having the hue component information of red, blue, and yellow in step S3 shown in FIG. The area extraction unit 15 extracts pixels corresponding to white or black, but the order of processing is not limited to this, and the order of processing may be reversed, or both processes are performed simultaneously. Also good.
[0038]
As described above, the sign recognition device 1 according to the present embodiment does not perform the recognition process based on the lightness component information as in the conventional case, but the red, blue, yellow, white corresponding to the color used for the road sign.・ Because recognition processing is performed based on a hue image having black hue component information, recognition is performed even if there is a sunlit part and a shaded part on the road sign that is a recognition target imaged in the original color image. The road sign can be recognized from the original color image without degrading the performance. Therefore, the recognition performance can be improved as compared with the prior art.
[0039]
Further, in the sign recognition device 1 according to the present embodiment, the hue value corresponding to the color (green) that is not used in principle for the road sign is shifted to the reference value (0 degree position) of the hue circle. The color (red, blue, yellow) used for the can be prevented from being divided after the normalization of the hue value. Therefore, the recognition process can be performed without reducing the degree of correlation between the hue values, and the recognition performance can be improved.
[0040]
Furthermore, in the sign recognition device 1 according to the present embodiment, a specific hue value is assigned to a pixel that is highly likely to correspond to white or black, and the hue value corresponding to white and black is set to the hue value corresponding to another color. Therefore, an object having white or black can be recognized based on the hue image. Thereby, recognition performance can be further improved as compared with the prior art.
[0041]
Further, in the sign recognition device 1 according to the present embodiment, since only the hue corresponding to the color used for the road sign is the object to be recognized, the colors other than the color used for the road sign among the hues on the color wheel. The hue portion corresponding to the color is in an empty state. Therefore, a hue value corresponding to white or black, which cannot be represented on the hue circle, can be represented on the hue circle by using the position in the empty state. Thereby, the object in which white and black exist can be recognized based on a hue image.
[0042]
In addition, although the case where the recognition target object is a road sign is described as an example in the sign recognition device 1 in the present embodiment, the recognition target object is not limited to a road sign. For example, road signs use white and black in addition to red, blue and yellow, but can also be applied to recognition objects that do not contain white or black in the colors used. is there. In this case, the monochrome area extracting unit 15 and the monochrome / hue value changing unit 16 described above can be omitted. That is, the hue value normalization unit 17 in the embodiment described above normalizes the hue value shifted by the hue value shift unit 14 and the hue value changed by the monochrome / hue value change unit 16. The hue value normalization unit 17 in this example may normalize the hue value after being shifted by the hue value shift unit 14.
[0043]
【The invention's effect】
According to the object recognition device and the object recognition method of the present invention, it is possible to improve the recognition performance when recognizing a recognition target object.
[Brief description of the drawings]
FIG. 1 is a functional configuration diagram of a sign recognition apparatus according to an embodiment of the present invention.
FIG. 2A is a diagram showing a general hue circle, and FIG. 2B is a diagram showing a hue circle after a hue value is shifted.
FIG. 3 is a flowchart showing a sign recognition process.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Sign recognition apparatus, 11 ... Imaging part, 12 ... HSI conversion part, 13 ... Hue image generation part, 14 ... Hue value shift part, 15 ... Monochrome area extraction part, 16 ... monochrome / hue value changing unit, 17 ... hue value normalizing unit, 18 ... label recognition unit, 19 ... output unit.

Claims (3)

An object recognition device for recognizing a recognition object whose color to be used is set in advance,
Extraction means for extracting hue information of a color used for the recognition object from an image captured by the imaging means;
Shift means for shifting a hue value included in the hue information extracted by the extraction means so that a hue value corresponding to a color other than the color used for the recognition object becomes a reference value of the hue;
Normalizing means for normalizing the hue value shifted by the shifting means;
Recognizing means for recognizing an object to be recognized based on a value normalized by the normalizing means;
Black and white extraction means for extracting a region corresponding to white or black from the captured image based on the saturation information and lightness information of the captured image;
Hue value changing means for changing a hue value corresponding to the region extracted by the black and white extracting means to a hue value corresponding to white or black,
The object recognizing apparatus, wherein the normalizing means normalizes the hue value changed by the hue value changing means. The object recognition apparatus according to claim 1 , wherein the recognition unit performs the recognition based on pattern recognition. An object recognition method for recognizing a recognition target whose color to be used is preset,
An extracting unit that extracts hue information of a color used for the recognition object from an image captured by the imaging unit;
The shift means shifts the hue value included in the extracted hue information so that a hue value corresponding to a color other than the color used for the recognition object becomes a reference value of the hue,
Normalization means normalizes the shifted hue value;
A recognition means recognizes a recognition object based on the normalized value;
The black and white extraction means extracts a region corresponding to white or black from the captured image based on the saturation information and lightness information of the color of the captured image,
A hue value changing unit changes the hue value corresponding to the extracted region to a hue value corresponding to white or black;
The object recognizing method , wherein the normalizing means normalizes the changed hue value during the normalization.

Images