JP4578767B2 - Object recognition device and object recognition program - Google Patents

Description

【００２５】
【数２】

【００２６】
式（１）および式（２）において、φはガボールフィルタの方向、Ｆはガボールフィルタの周波数、λはガウス関数の扁平比、σはスケールパラメータ（ぼかしの程度）を表している。
【００２７】
これらを図で表すと、周波数成分は図４に示す符号４１４、ガウス関数は図４に示す符号４１５のように表され、ガボールフィルタはこれらの積から図４の符号４１６に示すウェーブレットプロフィールで表現することができる。
【００２８】
図５は、ガボールフィルタによってドーナツのテクスチャを求める様子を示している。式（１）のφとＦを変えることにより、図５（ａ）に示すように、ガボールフィルタとして方向と周波数の異なるもの５１ａ，５１ｂ，５１ｃを作成しておき、これらを図５（ｂ）に示すようにドーナツ全体５２を小領域５３に分けたそれぞれの部分と掛け合わせ、ドーナツの周波数別、方向別の成分を抽出する。ガボールフィルタの周波数Ｆの取り得る値は、Ｆ０＝１／Ｌ，Ｆ１＝２Ｆ０，Ｆ２＝２Ｆ１，・・・である。ここで、Ｌはフィルタの大きさ（図４参照）である。
【００２９】
そして、小領域５３の画像データをｘｙ座標上でｆ（ｘ、ｙ）と表せば、式（３）のように画像データとガボールフィルタを畳み込み積分することにより、小領域５３の周波数成分を求めることができる。
【００３０】
【数３】

【００３１】
図６は、画像保持部２に保持された画像を色情報を持つ画像に変換する方法を示している。撮像部１で撮像された画像のＲＧＢの信号を色情報変換部４において輝度Ｌ^*、色相ａ^*、ｂ^*の情報３１，３２，３３に変換する。この変換は式（４）によって行うことができる。
【００３２】
【数４】

【００３３】
なお、本実施の形態においては、色情報としてＣＩＥ均等色空間であるＬ^*,ａ^*,ｂ^*を用いたが、これに限られるわけではなく、ＣＩＥ均等色空間Ｌ^*,ｕ^*,ｖ^*、輝度信号Ｙと２つの色差信号Ｃｒ,Ｃｂからなる画像信号Ｙ,Ｉ,ＱおよびＹ,Ｕ,Ｖ、あるいはマンセル空間のような均等視知覚空間上のＨ,Ｖ,Ｃ、Ｈ,Ｌ,Ｓなどの色情報を用いても良い。
【００３４】
図７は、変換画像保持部５、読込単位保持部１４、画像読込部８、特徴抽出部９および特徴量保持部１０の関係を示すブロック図である。変換画像保持部５には、画像が輝度Ｌ^*、色相ａ^*、ｂ^*の情報を持つ画像に変換されて保持されている。また、読込単位保持部１４には、変換画像保持部５に保持された画像を小領域６８に分割する際の大きさと各々の画像上での位置が保持されている。画像読込部８は、これら３つの画像情報（Ｌ^*、ａ^*、ｂ^*）を小領域６８毎に読み込む。
【００３５】
図７に示すように、特徴抽出部９は、フィルタ７１と判別部７２とによって構成されている。フィルタ７１は、画像読込部８が読み込んだ領域の画像に対して、式（２）によるフィルタ処理を施す。当該フィルタ処理では、読み込んだ領域の画像が小領域６７のようにドーナツ以外の情報を多く（例えば、所定割合以上）含む情報を破棄する。そして、特徴量保持部１０は、特徴抽出部９で得られた結果をフィルタの周波数Ｆ０，Ｆ１，…Ｆｎ毎かつＬ^*、ａ^*、ｂ^*の各情報毎に合計して、情報７４，７５，７６として保持する。
【００３６】
図８は、判別部１１によるドーナツの種類の決定方法を説明するブロック図である。学習データ保持部１２は、Ｌ^*、ａ^*、ｂ^*の比率に関する情報をドーナツの種類（例えば、チョコレート、アーモンドクランチ、コーヒーロール等）毎に保持している。判別部１１は、特徴量保持部１０が保持するドーナツの周波数成分毎の色情報（７４，７５，７６）と、学習データ保持部１２が保持する特徴量である各ドーナツの周波数成分毎の色情報（１２１，１２２，１２３）とを比較して、最も近いものをドーナツの種類として決定する。
【００３７】
以上説明したように、本実施形態の物体認識装置によれば、撮像部１で撮像された画像からドーナツ等の物体が撮影されている領域を決定した後、当該領域を複数の小領域に分割している。そして、当該分割された小領域毎にテクスチャ（模様）の解析を行い、各小領域の周波数成分毎の輝度Ｌ^*、色相ａ^*、ｂ^*を各情報毎に合計して得られた色情報から物体の特徴を抽出して、その比率を学習データ保持部１２に保持されている情報と比較することにより物体を判別している。このため、テクスチャと色の配分が同じで色が反転している物体であっても正確に判別することができる。
【００３８】
〔第２の実施形態〕
図９は、本発明に係る物体認識装置の第２の実施形態を示すブロック図である。同図に示すように、第２の実施形態の物体認識装置は、第１の実施形態の物体認識装置が有する構成要素に加えて、特許請求の範囲のパラメータ変更指示手段に該当する判別領域変更部１５、物体領域保持部９１および学習データ変更部９２が設けられている。また、学習データ保持部１２には、第１の実施の形態で使用したのと同様の学習データ１２４〜１２６と共に、学習データ１２７〜１２９が付加されている。さらに、本実施形態の特徴量保持部１０は、第１の特徴量保持部１０ａと第２の特徴量保持部１０ｂを有している。
【００３９】
図１０は、本実施形態の読込単位保持部１４の内部構成を示すブロック図である。同図に示すように、読込単位保持部１４は、小領域生成部１０１と、小領域保持部１０２と、小領域走査部１０３と、比較部１０４と、特徴保持部１０５と、比較結果保持部１０６とを内部に有して構成されている。小領域生成部１０１は、物体領域決定部３によって決定された領域を小領域に分割するものである。また、小領域保持部１０２は、小領域生成部１０１によって分割された結果生成された各小領域の画像上の位置に関する情報を保持するものである。また、小領域走査部１０３は、小領域保持部１０２に保持された小領域を順に走査するものであり、各小領域の画像上の位置を画像読込部８に知らせる。
【００４０】
また、特徴保持部１０５は、ドーナツの一部が特殊な形状を有していると現れる特徴量を保持するものである。また、比較部１０４は、特徴抽出部９から得られた特徴と特徴保持部１０５から得られた特徴量とを比較するものであり、一致した場合はその小領域の番号を比較結果保持部１０６に格納する。また、比較結果保持部１０６は、特徴保持部１０５から送られた比較結果を保持するものである。
【００４１】
以上の構成要素を備えた読込単位保持部１４に判別領域変更部１５からの入力があった場合、小領域走査部１０３は、比較結果保持部１０６に保持された小領域の番号を順に走査し、小領域保持部１０２に保持されている小領域の画像上の位置を画像読込部８に出力する。
【００４２】
次に、本実施形態の物体認識装置の動作について、図１１〜図１３を参照して説明する。まず、図１１に示すように、ドーナツの内容物充填口（以下「充填口」という。）１１２の部分に判定領域１１１を設定する。この判定領域内の模様を調べると、領域内に充填口またはその一部Ｃ３やＤ３が含まれているため、図１１に示す符号Ｂ２やＢ３等の他の領域とは異なる特徴量が得られる。このような特徴量を予め特徴保持部１０５に記憶させておき、特徴抽出部９が抽出した特徴量と一致した場合は、その小領域番号を比較結果保持部１０６に保持する。
【００４３】
一方、判別部１１でドーナツ全体の特徴量を調べた結果、充填口の部分を詳しく調べないと判別が困難な種類のドーナツであることが判明した場合、判別領域変更部１５は、パラメータ設定部６にパラメータの変更を指示する。パラメータ設定部６は、小領域の一辺の長さを第１の実施形態で説明した小領域の半分とするパラメータを設定し、パラメータ保持部７に保持する。なお、判定領域を半分としたパラメータは、図４に示したガボールフィルタのウェーブレットプロフィールにおける一辺Ｌが１／２の場合に相当する。また、画像読込部８は、読込単位保持部１４に保持された、図１１に示した小領域Ｃ３，Ｄ３を参照して、変換画像保持部５に保持された画像の中から小領域Ｃ３，Ｄ３に対応する画像を読み込む。
【００４４】
図１２は、小領域Ｃ３，Ｄ３をさらに分割した様子を示す説明図である。また、図１３は、さらに分割された小領域によって画像が読み込まれた様子を示す説明図である。同図に示すように、小領域Ｃ３，Ｄ３は４分割されている。特徴抽出部９によって得られた色情報が第２の特徴量保持部１０ｂに入力されると、判別部１１は、図１３（ａ）に示す画像から図１３（ｂ）の領域Ｃ３，Ｄ３を取り出した後、充填口１１２にかからない部分Ｅは図１３（ｃ）に示すように破棄され、最終的には、図１３（ｄ）に示すように充填口１１２の部分のみを残す。そして、当該残った部分の特徴を学習データ保持部１２の各学習データを参照することにより、ドーナツの種類を判別する。
【００４５】
以上説明したように、本実施形態の物体認識装置によれば、判別部１１で物体全体の特徴を調べた結果、判別が困難な種類の物体であることが判明したときは、小領域をさらに分割し、判別可能な領域を絞り込んだ上で当該領域の特徴を抽出して、学習データ保持部１２に保持されている情報と比較している。このため、全体の色と模様は同じで一部分のみが異なる物体を正確に判別することができる。
【００４６】
〔第３の実施形態〕
第３の実施形態の物体認識装置は、図１４に示すように、ドーナツの撮影画像に対して小領域Ａ２１を設定し、当該小領域Ａ２１をドーナツの撮影画像全体にわたって少しずつ走査していく。小領域Ａ２１は、図１５に示すように１／４や１／１６に細分化され、図１６に示すようにフィルタを用いて畳み込み積分することによって、ドーナツの充填口の位置を決定することができる。
【００４７】
図１５は、小領域Ａ２１の細分化についての説明図である。小領域Ａ２１は、小領域Ａ２１と同じ大きさの領域Ａ３１、小領域Ａ２１の１／４の大きさの領域Ａ３２、小領域Ａ２１の１／１６の大きさの領域Ａ３３に分割される。それぞれ１個、４個、１６個の領域があり、合計２１個の領域になる。
【００４８】
図１６は、小領域Ａ２１がドーナツの充填口の位置する場合についての説明図である。符号Ａ４１はドーナツの充填口に位置する小領域Ａ２１を拡大した様子であり、符号Ａ３２は小領域Ａ４１と同じ大きさの領域、符号Ａ４３は小領域Ａ４１の１／４の大きさの領域、符号Ａ４４は小領域Ａ４１の１／１６の大きさの領域を示している。
【００４９】
本実施形態では、図４でも説明した方向と周波数の異なるガボールフィルタを図１６の符号Ａ４５，Ａ４６のように作成しておき、領域Ａ４３に対してはフィルタＡ４５を用いて、領域Ａ４４に対してはフィルタＡ４６を用いて畳み込み積分を行う。このようなフィルタリングを行うことにより、小領域Ａ２１が充填口の位置以外に位置した場合よりも、畳み込み積分の結果として大きな値を得ることができるため、ドーナツの充填口の位置を知ることができる。
【００５０】
図１７は、本発明に係る物体認識装置の第３の実施形態を示すブロック図である。同図に示すように、本実施形態の物体認識装置は、第１の実施形態の物体認識装置が有する撮像部１、画像保持部２、特許請求の範囲の物体領域決定手段に該当する物体領域決定部３、パラメータ設定手段に該当するパラメータ設定部６、パラメータ保持部７、画像読込手段に該当する画像読込部８、特徴判別手段に該当する特徴抽出部９、特徴記憶手段に該当する特徴量保持部１０、物体判別手段に該当する判別部１１および結果保持部１３に加えて、読込領域決定手段に該当する読込領域決定部４１と、充填口特徴量保持部４３と、小領域位置判別手段に該当する小領域位置判別部４５と、色情報記憶手段に該当する色データ保持部４７とを備えて構成されている。
【００５１】
読込領域決定部４１は、上記説明した小領域Ａ２１をドーナツの撮影画像上で走査して画像の読込領域を決定するものである。また、各位置において、小領域Ａ２１を細分化した領域も順に指定していく。画像読込部８は、この領域の位置と大きさに合わせて画像データを読み込む。一方、パラメータ保持部７は、読込領域決定部４１が決定した領域に応じて、図１６に示したフィルタＡ４５，Ａ４６のパラメータを特徴抽出部９に提供する。そして、特徴抽出部９は、画像読込部８が読み込んだ画像データに対してパラメータ保持部７から提供されたパラメータのフィルタを用いて畳み込み積分を行うことによって画像の特徴を抽出する。
【００５２】
充填口特徴量保持部４３は、小領域Ａ２１がドーナツの充填口に位置したときに得られる特徴抽出部９による畳み込み積分の典型的な特徴を保持するものである。また、小領域位置判別部４５は、特徴抽出部９で得られた特徴と、充填口特徴量保持部４３が保持する特徴とを比較することによって、小領域Ａ２１がドーナツの充填口に位置するか否かを判別するものである。
【００５３】
また、色データ保持部４７は、ドーナツ毎に充填口の色に関するデータを保持するものである。判別部１１は、小領域位置判別部４５によって小領域Ａ２１がドーナツの充填口に位置すると判別された場合に、小領域Ａ２１の画像の色を色データ保持部４７に保持されている色データと比較することによって、撮像部１で撮像されたドーナツの種類を判別する。
【００５４】
以上説明したように、本実施形態の物体認識装置によれば、ドーナツの充填口の位置を判別して、充填口の色からドーナツの種類を判別することができる。
【００５５】
【発明の効果】
以上説明したように、本発明に係る物体認識装置および物体認識プログラムによれば、模様が同じで色が反転している物体や、一部の領域で違いがあるものの全体として模様および色が同じ物体を正確に判別することができる。
【図面の簡単な説明】
【図１】本発明に係る物体認識装置の第１の実施形態を示すブロック図
【図２】撮像部によってドーナツを撮影した画像を示す説明図
【図３】撮影画像中のドーナツに枠を設定し、当該枠内を小領域に分割した様子を示す説明図
【図４】ガボールフィルタの周波数成分、ガウス関数およびウェーブレットプロフィールを示す説明図
【図５】ガボールフィルタによってドーナツのテクスチャを求める様子を示す説明図
【図６】画像を輝度、色相、彩度の情報を持つ画像に変換する方法を示すブロック図
【図７】変換画像保持部５、読込単位保持部１４、画像読込部８、特徴抽出部９および特徴量保持部１０の関係を説明するブロック
【図８】判別部１１によるドーナツの種類の決定方法を説明するブロック図
【図９】本発明に係る物体認識装置の第２の実施形態を示すブロック図
【図１０】第２の実施形態の読込単位保持部の内部構成を示すブロック図
【図１１】撮像部によるドーナツの撮影画像を示す説明図
【図１２】小領域をさらに分割した様子を示す説明図
【図１３】さらに分割された小領域によって画像が読み込まれた様子を示す説明図
【図１４】ドーナツの撮影画像に対して小領域Ａ２１を設定した説明図
【図１５】小領域Ａ２１の細分化についての説明図
【図１６】小領域Ａ２１がドーナツの充填口の位置する場合についての説明図
【図１７】本発明に係る物体認識装置の第３の実施形態を示すブロック図
【図１８】従来の物体認識装置を示すブロック図
【図１９】模様は全く同じであるが色が反転している互いに種類の異なる２つのドーナツを示す説明図
【図２０】従来の物体認識装置では識別が不可能な異なる種類のドーナツの他の例を示す説明図
【符号の説明】
１ 撮像部
３ 物体領域決定部
４ 色情報変換部
５ 変換画像保持部
６ パラメータ設定部
８ 画像読込部
９ 特徴抽出部
１１ 判別部
１２ 学習データ保持部
１４ 読込単位保持部
１５ 判別領域変更部
１０１ 小領域生成部
１０２ 小領域保持部
１０３ 小領域走査部
１０４ 比較部
１０５ 特徴保持部
１０６ 比較結果保持部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an object recognition device and an object recognition program for recognizing and determining the type of an object such as a donut.
[0002]
[Prior art]
FIG. 18 is a block diagram showing a conventional object recognition apparatus. As shown in the figure, the conventional object recognition apparatus includes an imaging unit 141, an image holding unit 142, an object region determination unit 143, a texture learning database 144, a color learning database 145, a texture analysis unit 146, A color analysis unit 147 and a determination unit 148 are provided.
[0003]
The imaging unit 141 is a camera, for example, and images an object such as a donut. Hereinafter, a donut will be described as an example of the object. The image holding unit 142 holds the image picked up by the image pickup unit 141, and the object region determination unit 143 determines a region where the donut in the image is shot. On the other hand, the texture analysis unit 146 analyzes the donut pattern (texture). However, the texture learning database 144 stores the pattern for each type of donut, and the texture analysis unit 146 determines the donut pattern by referring to the information stored in the texture learning database 144.
[0004]
In addition, the color analysis unit 147 checks the color in the region determined by the object region determination unit 143. However, the color learning database 145 stores the color for each type of donut, and the color analysis unit 147 refers to the information stored in the color learning database 145 to determine the color of the donut. Then, the determination unit 148 determines the type of donut based on each analysis result by the texture analysis unit 146 and the color analysis unit 147.
[0005]
[Problems to be solved by the invention]
However, in the conventional object recognition apparatus described above, it has been difficult to identify donuts having the same pattern but inverted colors. FIG. 19 shows two different types of donuts 151 and 152 having the same pattern but having the colors reversed. As shown in the figure, the two donuts 151 and 152 each have a brown portion 153 and a white portion 154, and the areas of the two color donuts 151 and 152 are the same. In such donuts 151 and 152, the result of the texture analysis is the same, and since the area occupied by the brown portion 153 and the white portion 154 is the same, the result of the color analysis is also the same. For this reason, the conventional object recognition apparatus has a problem that it is not possible to determine that these two donuts 151 and 152 are of different types.
[0006]
FIG. 20 shows another example of different types of donuts that cannot be identified by a conventional object recognition apparatus. As shown in the figure, the types of donuts 161 and donuts 162 are different from each other and have almost the same pattern and color throughout, but the colors of the regions 163 and 164 which are filling ports for contents such as cream (that is, the color of the cream) Only) is different. In the conventional object recognition apparatus, after the entire image is analyzed, the pattern and color are discriminated in consideration of the entire pattern. Therefore, in the donuts having the same pattern and color as in the donuts 161 and 162, each kind of There was a problem that it was difficult to discriminate them from different ones.
[0007]
The present invention has been made in view of the above-described conventional problems, and an object having the same pattern and reversed color, or an object having the same pattern and color as a whole although there are differences in some areas. An object of the present invention is to provide an object recognition apparatus and an object recognition program capable of accurately discriminating.
[0008]
[Means for Solving the Problems]
In order to achieve the above object, an object recognition apparatus according to the present invention captures an image holding unit that holds an image obtained by shooting an object, and the object in the image held by the image holding unit. Object area determining means for determining an area, area dividing means for dividing the area determined by the object area determining means into a plurality of small areas, and information on brightness and hue for the image held in the image holding means Color information conversion means for converting to color information of the image converted by the color information conversion means For each luminance and hue component Feature extraction means for analyzing as a pattern of the object for each small area divided by the area dividing means, and pattern information analyzed by the feature extraction means for each small area divided by the area dividing means The Feature holding means for holding, feature storage means for storing features relating to color and pattern for each type of object, features of the object determined by the feature extraction means, and objects stored in the feature storage means Object discriminating means for discriminating the type of the object in the image by comparing with the feature. Therefore, it is possible to accurately discriminate an object having the same pattern and reversed color.
[0010]
Further, the object recognition apparatus according to the present invention is a parameter for setting a parameter for determining the length of one side of a small area constituting the area according to the size of the area determined by the object area determining means. Comprising setting means, Feature extraction The means analyzes the pattern of the object according to the parameter set by the parameter setting means. Therefore, the pattern of the object can be analyzed accurately.
[0011]
In the object recognition apparatus according to the present invention, the parameter setting unit determines a parameter based on a frequency component of each small region obtained by dividing the region determined by the object region determination unit and a direction of the frequency component. Therefore, an appropriate parameter can be determined according to the pattern, that is, the frequency component.
[0012]
Further, the object recognition apparatus according to the present invention provides the above Characteristic The holding means preferably stores information on luminance and hue for each pattern for each type of object.
[0013]
Further, the object recognition apparatus according to the present invention includes parameter change instruction means for instructing change of a parameter set by the parameter setting means, and when the object determination means cannot determine the type of the object in the image, The parameter change instructing unit instructs the region dividing unit to change the parameter determined by the object region determining unit to a parameter for dividing the region into a plurality of small regions smaller than the small region. As described above, when the type of the object in the image cannot be determined, it is divided into smaller sub-regions, so that it is possible to extract detailed features.
[0015]
Further, the object recognition method according to the present invention includes an image holding step for holding an image in which an object is shot, an object region determining step for determining a region in which the object is shot in the held image, An area dividing step for dividing the area determined by the object area determining step into a plurality of small areas, a color information converting step for converting the held image into color information of luminance and hue, and the color information converting step The color information of the image converted by For each luminance and hue component A feature extraction step for analyzing as a pattern of the object for each small region divided by the region division step, and a pattern information analyzed by the feature extraction analysis step for each small region divided by the region division step The A feature holding step for holding, a feature storing step for storing features relating to color and pattern for each type of object, a feature of the object determined by the feature holding step, and an object stored in the feature storing step An object discriminating step for discriminating the type of the object in the image by comparing with the feature. Therefore, it is possible to accurately discriminate an object having the same pattern and reversed color.
[0017]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of an object recognition apparatus according to the present invention will be described below in detail with reference to the drawings. In the description of each embodiment, the object recognition apparatus according to the present invention will be described in detail. However, the object recognition program according to the present invention is a program to be realized as each component included in the object recognition apparatus. Therefore, the description regarding the program is included in the following description.
[0018]
[First Embodiment]
FIG. 1 is a block diagram showing a first embodiment of an object recognition apparatus according to the present invention. As shown in the figure, the object recognition apparatus according to the first embodiment includes an imaging unit 1, an image holding unit 2, an object region determining unit 3 corresponding to the object region determining means in the claims, and color information. Color information conversion unit 4 corresponding to the conversion unit, converted image holding unit 5, parameter setting unit 6 corresponding to the parameter setting unit, parameter holding unit 7, image reading unit 8, pattern analysis unit and color feature determination A feature extraction unit 9 corresponding to the means, a feature amount holding unit 10, a discrimination unit 11 corresponding to the object discrimination unit, a learning data holding unit 12 corresponding to the feature storage unit, a result holding unit 13, and a region dividing unit. And a reading unit holding unit 14 corresponding to the above, which captures an image of an object such as a donut and finally recognizes and determines the type of the object. Hereinafter, an object to be recognized and discriminated will be described by taking a donut as an example.
[0019]
Hereinafter, each component which the object recognition apparatus of this embodiment has will be described.
The imaging part 1 is a camera, for example, and images a donut. The image holding unit 2 holds an image captured by the imaging unit 1. The object region determination unit 3 determines a region where a donut is photographed in the image. The color information conversion unit 4 converts the image held in the image holding unit 2 into an image having color information of luminance and hue. The converted image holding unit 5 holds the image converted by the color information converting unit 4.
[0020]
The parameter setting unit 6 is for setting a parameter for determining a donut pattern (texture) according to the size of the region determined by the object region determination unit 3. The parameter set by the parameter setting unit 6 is determined by the frequency component of each small region obtained by dividing the region determined by the object region determining unit 3 into a plurality of parts and the direction of the frequency component. The parameter holding unit 7 holds parameters determined by the parameter setting unit 6. Further, the reading unit holding unit 14 holds the area determined by the object area determining unit 3 by subdividing it into small areas.
[0021]
The image reading unit 8 reads the image held in the converted image holding unit 5 in units of small areas held in the reading unit holding unit 14. The feature extraction unit 9 extracts the feature of the image of the area read by the image reading unit 8 according to the parameters held by the parameter holding unit 7. The feature amount holding unit 10 holds the features extracted by the feature extraction unit 9. The determining unit 11 compares the feature held in the feature amount holding unit 10 with the content held in the learning data holding unit 12 to determine the type of donut imaged by the imaging unit 1. Further, the result holding unit 13 holds the discrimination result by the discrimination unit 11.
[0022]
Hereinafter, the object recognition apparatus of this embodiment will be described in more detail.
FIG. 2 is an explanatory diagram showing an image 21 obtained by photographing the donut 22 by the imaging unit 1. When photographing the donut 22, the size of the donut 22 is not constant because the positional relationship between the imaging unit 1 and the donut changes. Further, the object area determination unit 3 sets an area so that the square frame 23 is in contact with the donut 22. FIG. 3 shows how the area within the frame 23 is divided into small areas 31 by the reading unit holding unit 14 after the frame 23 is set. After the image of the area in the frame 23 is divided into the small areas 31, the feature extraction unit 9 analyzes the pattern (texture) for each small area 31.
[0023]
This texture analysis is performed using a Gabor filter shown in FIG. The Gabor filter is “Multichannel Texture Analysis Using Localized Spatial Filters” (Alan Conrad Bovik, Marianna Clark, Wilson S. Geilser, IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 12, No. 1, January 1990). And defined by equation (2). The Gabor filter is represented by the product of a part represented by a Gaussian function 412 and a part represented by a frequency component 413, as shown in Expression (1).
[0024]
[Expression 1]

[0025]
[Expression 2]

[0026]
In Expressions (1) and (2), φ represents the Gabor filter direction, F represents the Gabor filter frequency, λ represents the Gaussian function flat ratio, and σ represents the scale parameter (the degree of blurring).
[0027]
When these are represented graphically, the frequency component is represented by reference numeral 414 shown in FIG. 4, the Gaussian function is represented by reference numeral 415 shown in FIG. 4, and the Gabor filter is expressed by the wavelet profile indicated by reference numeral 416 in FIG. can do.
[0028]
FIG. 5 shows how a donut texture is obtained using a Gabor filter. By changing φ and F in the equation (1), as shown in FIG. 5A, the Gabor filters 51a, 51b, 51c having different directions and frequencies are prepared, and these are shown in FIG. As shown in FIG. 4, the entire donut 52 is multiplied with each portion divided into small areas 53, and components for each frequency and direction of the donut are extracted. Possible values of the frequency F of the Gabor filter are F0 = 1 / L, F1 = 2F0, F2 = 2F1,. Here, L is the size of the filter (see FIG. 4).
[0029]
If the image data of the small region 53 is expressed as f (x, y) on the xy coordinates, the frequency component of the small region 53 is obtained by convolving and integrating the image data and the Gabor filter as shown in Expression (3). be able to.
[0030]
[Equation 3]

[0031]
FIG. 6 shows a method for converting an image held in the image holding unit 2 into an image having color information. The RGB signal of the image picked up by the image pickup unit 1 is converted into luminance L by the color information conversion unit 4. ^* , Hue a ^* , B ^* Information 31, 32, 33. This conversion can be performed by equation (4).
[0032]
[Expression 4]

[0033]
In the present embodiment, L, which is a CIE uniform color space, is used as color information. ^* , a ^* , b ^* However, the present invention is not limited to this, and the CIE uniform color space L ^* , u ^* , v ^* , Image signals Y, I, Q and Y, U, V consisting of luminance signal Y and two color difference signals Cr, Cb, or H, V, C, H, L, on a uniform vision perception space such as Munsell space Color information such as S may be used.
[0034]
FIG. 7 is a block diagram showing the relationship between the converted image holding unit 5, the reading unit holding unit 14, the image reading unit 8, the feature extraction unit 9, and the feature amount holding unit 10. The converted image holding unit 5 displays an image with luminance L ^* , Hue a ^* , B ^* It is converted into an image having the information and stored. Further, the reading unit holding unit 14 holds the size when the image held in the converted image holding unit 5 is divided into small areas 68 and the position on each image. The image reading unit 8 reads these three pieces of image information (L ^* , A ^* , B ^* ) Is read for each small area 68.
[0035]
As illustrated in FIG. 7, the feature extraction unit 9 includes a filter 71 and a determination unit 72. The filter 71 performs the filtering process according to the expression (2) on the image in the area read by the image reading unit 8. In the filtering process, information including a large amount of information other than the donut (for example, a predetermined ratio or more) such as the small area 67 in the image of the read area is discarded. Then, the feature amount holding unit 10 uses the result obtained by the feature extracting unit 9 for each of the filter frequencies F0, F1,. ^* , A ^* , B ^* Are stored as information 74, 75, 76.
[0036]
FIG. 8 is a block diagram illustrating a method for determining the type of donut by the determination unit 11. The learning data holding unit 12 is L ^* , A ^* , B ^* Is stored for each type of donut (for example, chocolate, almond crunch, coffee roll, etc.). The determination unit 11 includes color information (74, 75, 76) for each frequency component of the donut held by the feature amount holding unit 10 and a color for each frequency component of each donut that is the feature amount held by the learning data holding unit 12. The information (121, 122, 123) is compared, and the closest one is determined as the type of donut.
[0037]
As described above, according to the object recognition device of the present embodiment, after determining an area where an object such as a donut is captured from an image captured by the imaging unit 1, the area is divided into a plurality of small areas. is doing. Then, the texture (pattern) is analyzed for each of the divided small areas, and the luminance L for each frequency component of each small area is analyzed. ^* , Hue a ^* , B ^* The features of the object are extracted from the color information obtained by summing the information for each piece of information, and the ratio is compared with the information held in the learning data holding unit 12 to discriminate the object. For this reason, even an object having the same texture and color distribution and an inverted color can be accurately identified.
[0038]
[Second Embodiment]
FIG. 9 is a block diagram showing a second embodiment of the object recognition apparatus according to the present invention. As shown in the figure, the object recognition apparatus according to the second embodiment is configured to change the discrimination area corresponding to the parameter change instruction means in the claims in addition to the constituent elements of the object recognition apparatus according to the first embodiment. A unit 15, an object area holding unit 91, and a learning data changing unit 92 are provided. In addition, learning data 127 to 129 are added to the learning data holding unit 12 together with learning data 124 to 126 similar to those used in the first embodiment. Furthermore, the feature amount holding unit 10 of the present embodiment includes a first feature amount holding unit 10a and a second feature amount holding unit 10b.
[0039]
FIG. 10 is a block diagram showing an internal configuration of the reading unit holding unit 14 of the present embodiment. As shown in the figure, the reading unit holding unit 14 includes a small region generation unit 101, a small region holding unit 102, a small region scanning unit 103, a comparison unit 104, a feature holding unit 105, and a comparison result holding unit. 106 inside. The small area generation unit 101 divides the area determined by the object area determination unit 3 into small areas. The small area holding unit 102 holds information regarding the position on the image of each small area generated as a result of the division by the small area generation unit 101. The small area scanning unit 103 sequentially scans the small areas held in the small area holding unit 102 and notifies the image reading unit 8 of the position of each small area on the image.
[0040]
The feature holding unit 105 holds a feature amount that appears when a part of the donut has a special shape. The comparison unit 104 compares the feature obtained from the feature extraction unit 9 and the feature quantity obtained from the feature holding unit 105. If they match, the comparison result holding unit 106 assigns the number of the small region. To store. The comparison result holding unit 106 holds the comparison result sent from the feature holding unit 105.
[0041]
When there is an input from the discrimination region changing unit 15 to the reading unit holding unit 14 having the above components, the small region scanning unit 103 sequentially scans the numbers of the small regions held in the comparison result holding unit 106. The position on the image of the small area held in the small area holding unit 102 is output to the image reading unit 8.
[0042]
Next, the operation of the object recognition apparatus of the present embodiment will be described with reference to FIGS. First, as shown in FIG. 11, a determination region 111 is set in a portion of a doughnut filling port (hereinafter referred to as “filling port”) 112. When the pattern in the determination area is examined, since the filling port or a part thereof C3 or D3 is included in the area, a characteristic amount different from that of other areas such as reference numerals B2 and B3 shown in FIG. 11 is obtained. . Such a feature amount is stored in advance in the feature holding unit 105, and when the feature amount matches the feature amount extracted by the feature extraction unit 9, the small region number is held in the comparison result holding unit 106.
[0043]
On the other hand, as a result of investigating the feature amount of the whole donut by the discriminating unit 11, if it is found that the type of donut is difficult to discriminate unless the portion of the filling port is examined in detail, the discriminating region changing unit 15 6 is instructed to change the parameter. The parameter setting unit 6 sets a parameter that sets the length of one side of the small region to half of the small region described in the first embodiment, and stores the parameter in the parameter holding unit 7. Note that the parameter with the determination region being halved corresponds to the case where the side L in the wavelet profile of the Gabor filter shown in FIG. 4 is ½. The image reading unit 8 refers to the small regions C3 and D3 shown in FIG. 11 held in the reading unit holding unit 14, and selects the small regions C3 and C3 from the images held in the converted image holding unit 5. The image corresponding to D3 is read.
[0044]
FIG. 12 is an explanatory diagram showing a state in which the small areas C3 and D3 are further divided. FIG. 13 is an explanatory diagram showing a state in which an image is read by a further divided small area. As shown in the figure, the small areas C3 and D3 are divided into four. When the color information obtained by the feature extraction unit 9 is input to the second feature amount holding unit 10b, the determination unit 11 extracts the regions C3 and D3 in FIG. 13B from the image shown in FIG. After removal, the portion E that does not reach the filling port 112 is discarded as shown in FIG. 13C, and finally only the portion of the filling port 112 is left as shown in FIG. Then, the type of the donut is determined by referring to each learning data of the learning data holding unit 12 for the feature of the remaining part.
[0045]
As described above, according to the object recognition apparatus of the present embodiment, when the characteristics of the entire object are checked by the determination unit 11, if it is determined that the object is a kind of object that is difficult to determine, the small area is further reduced. After dividing and narrowing down the discriminable area, the features of the area are extracted and compared with the information held in the learning data holding unit 12. For this reason, it is possible to accurately discriminate an object having the same overall color and pattern but different only in part.
[0046]
[Third Embodiment]
As illustrated in FIG. 14, the object recognition apparatus according to the third embodiment sets a small area A <b> 21 for a captured image of a donut, and scans the small area A <b> 21 little by little over the entire captured image of the donut. The small region A21 is subdivided into 1/4 or 1/16 as shown in FIG. 15, and the position of the filling port of the donut can be determined by performing convolution integration using a filter as shown in FIG. it can.
[0047]
FIG. 15 is an explanatory diagram for subdividing the small area A21. The small area A21 is divided into an area A31 having the same size as the small area A21, an area A32 having a size 1/4 of the small area A21, and an area A33 having a size 1/16 of the small area A21. There are 1, 4 and 16 areas, respectively, for a total of 21 areas.
[0048]
FIG. 16 is an explanatory diagram for the case where the small region A21 is located at the filling port of the donut. Reference sign A41 is an enlarged view of the small area A21 located at the filling port of the donut. Reference sign A32 is an area having the same size as the small area A41. Reference sign A43 is an area having a quarter size of the small area A41. A44 indicates an area that is 1/16 the size of the small area A41.
[0049]
In the present embodiment, Gabor filters having different directions and frequencies as described in FIG. 4 are created as indicated by reference numerals A45 and A46 in FIG. 16, and the filter A45 is used for the region A43 and the region A44 is used. Performs convolution integration using filter A46. By performing such filtering, a larger value can be obtained as a result of the convolution integral than when the small region A21 is located at a position other than the position of the filling port, so that the position of the filling port of the donut can be known. .
[0050]
FIG. 17 is a block diagram showing a third embodiment of the object recognition apparatus according to the present invention. As shown in the figure, the object recognition apparatus according to the present embodiment includes an image pickup unit 1, an image holding unit 2, and an object region corresponding to the object region determination unit of the claims included in the object recognition device according to the first embodiment. The determining unit 3, the parameter setting unit 6 corresponding to the parameter setting unit, the parameter holding unit 7, the image reading unit 8 corresponding to the image reading unit, the feature extracting unit 9 corresponding to the feature discrimination unit, and the feature quantity corresponding to the feature storage unit In addition to the holding unit 10, the discriminating unit 11 corresponding to the object discriminating unit, and the result holding unit 13, the reading region determining unit 41 corresponding to the reading region determining unit, the filling port feature amount holding unit 43, and the small region position determining unit Are provided with a small region position determination unit 45 corresponding to the above and a color data holding unit 47 corresponding to the color information storage means.
[0051]
The reading area determination unit 41 scans the small area A21 described above on the image of the donut and determines an image reading area. In each position, areas obtained by subdividing the small area A21 are also specified in order. The image reading unit 8 reads image data in accordance with the position and size of this area. On the other hand, the parameter holding unit 7 provides the parameters of the filters A45 and A46 shown in FIG. 16 to the feature extraction unit 9 according to the region determined by the reading region determination unit 41. Then, the feature extraction unit 9 extracts image features by performing convolution integration on the image data read by the image reading unit 8 using a parameter filter provided from the parameter holding unit 7.
[0052]
The filling port feature amount holding unit 43 holds typical features of the convolution integral performed by the feature extraction unit 9 obtained when the small region A21 is located at the filling port of the donut. In addition, the small region position determination unit 45 compares the feature obtained by the feature extraction unit 9 with the feature held by the filling port feature amount holding unit 43, so that the small region A21 is positioned at the filling port of the donut. Or not.
[0053]
The color data holding unit 47 holds data related to the color of the filling port for each donut. When the small region position determining unit 45 determines that the small region A21 is located at the filling port of the donut, the determining unit 11 determines the color of the image of the small region A21 and the color data held in the color data holding unit 47. By comparing, the type of the donut imaged by the imaging unit 1 is determined.
[0054]
As described above, according to the object recognition device of the present embodiment, the position of the dough filling port can be determined, and the type of the donut can be determined from the color of the filling port.
[0055]
【The invention's effect】
As described above, according to the object recognition device and the object recognition program of the present invention, the same pattern and color as a whole although the pattern is the same and the color is reversed, or there are differences in some areas. An object can be accurately identified.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a first embodiment of an object recognition apparatus according to the present invention.
FIG. 2 is an explanatory diagram illustrating an image obtained by photographing a donut by an imaging unit.
FIG. 3 is an explanatory diagram showing a state in which a frame is set on a donut in a photographed image and the inside of the frame is divided into small regions.
FIG. 4 is an explanatory diagram showing frequency components, Gaussian functions, and wavelet profiles of a Gabor filter.
FIG. 5 is an explanatory diagram showing a state where a texture of a donut is obtained by a Gabor filter
FIG. 6 is a block diagram illustrating a method for converting an image into an image having luminance, hue, and saturation information.
FIG. 7 is a block for explaining the relationship among a converted image holding unit 5, a reading unit holding unit 14, an image reading unit 8, a feature extracting unit 9, and a feature amount holding unit 10;
FIG. 8 is a block diagram illustrating a method for determining the type of donut by the determination unit 11
FIG. 9 is a block diagram showing a second embodiment of the object recognition apparatus according to the present invention.
FIG. 10 is a block diagram showing an internal configuration of a reading unit holding unit according to the second embodiment.
FIG. 11 is an explanatory diagram showing a captured image of a donut by an imaging unit.
FIG. 12 is an explanatory diagram showing a state where a small area is further divided;
FIG. 13 is an explanatory diagram showing a state in which an image is read by a further divided small area.
FIG. 14 is an explanatory diagram in which a small area A21 is set for a captured image of a donut.
FIG. 15 is an explanatory diagram of subdivision of the small area A21.
FIG. 16 is an explanatory diagram of a case where the small region A21 is located at the filling port of the donut.
FIG. 17 is a block diagram showing a third embodiment of the object recognition apparatus according to the present invention.
FIG. 18 is a block diagram showing a conventional object recognition device.
FIG. 19 is an explanatory diagram showing two different types of donuts having the same pattern but having the colors reversed.
FIG. 20 is an explanatory diagram showing another example of different types of donuts that cannot be identified by a conventional object recognition device.
[Explanation of symbols]
1 Imaging unit
3 Object region determination unit
4-color information converter
5 Conversion image holding unit
6 Parameter setting section
8 Image reading part
9 Feature extraction unit
11 Discriminator
12 Learning data holding unit
14 Reading unit holding part
15 Discrimination area changing section
101 Small region generator
102 Small area holding part
103 Small area scanning unit
104 comparison part
105 Feature holding unit
106 Comparison result holding unit

Claims (6)

Image holding means for holding an image of an object photographed;
An object area determining means for determining an area where the object in the image held in the image holding means is photographed;
Area dividing means for dividing the area determined by the object area determining means into a plurality of small areas;
Color information conversion means for converting the image held in the image holding unit into luminance and hue color information;
Analyzing the color information of the image converted by the color information conversion means as a pattern of the object for each small area divided by the area dividing means for each component of luminance and hue ;
Feature holding means for holding pattern information analyzed by the feature extraction means for each of the small areas divided by the area dividing means;
Feature storage means for storing features relating to color and pattern for each type of object;
An object discriminating unit that discriminates the type of the object in the image by comparing the feature of the object discriminated by the feature holding unit with the feature of the object stored in the feature storage unit;
An object recognition apparatus characterized by comprising: In accordance with the size of the area determined by the object area determination means, parameter setting means for setting a parameter for determining the length of one side of the small area constituting the area,
The object recognition apparatus according to claim 1, wherein the feature extraction unit analyzes the pattern of the object in accordance with the parameter set by the parameter setting unit.   3. The object recognition according to claim 2, wherein the parameter setting unit determines a parameter based on a frequency component of each small region obtained by dividing the region determined by the object region determining unit into a plurality of regions and a direction of the frequency component. apparatus.   4. The object recognition apparatus according to claim 1, wherein the feature holding means stores information on luminance and hue for each pattern for each type of object. Parameter change instruction means for instructing change of parameters set by the parameter setting means,
When the object determination unit cannot determine the type of the object in the image, the parameter change instruction unit divides the region determined by the object region determination unit into a plurality of small regions smaller than the small region. The object recognition apparatus according to claim 2, wherein the region dividing unit is instructed to change the parameter to a parameter for the purpose. An image holding step for holding an image of an object photographed;
An object area determination step for determining an area in which the object in the held image is captured;
A region dividing step of dividing the region determined by the object region determining step into a plurality of small regions;
A color information conversion step for converting the held image into color information of luminance and hue;
A feature extraction step of analyzing the color information of the image converted by the color information conversion step as a pattern of the object for each small region divided by the region division step for each component of luminance and hue ;
A feature holding step for holding pattern information analyzed by the feature extraction analysis step for each small region divided by the region dividing step;
A feature storage step for storing features relating to colors and patterns for each type of object;
An object determination step of determining the type of the object in the image by comparing the feature of the object determined in the feature holding step with the feature of the object stored in the feature storage step;
An object recognition method comprising:

Images