JP3725508B2 - Parameter estimation device and data collation device - Google Patents

Description

次に、パラメタ入力部１０２に含まれるディスプレイが、サンプルとなる顔画像を表示する。そして、使用者が、パラメタ入力部１０２に含まれるマウスを用いて、顔画像から顔の特徴点の座標を選択すると、パラメタ入力部１０２は、使用者が選択した顔の特徴点座標を学習部１０３に出力する（Ｓ１２）。
【０１０７】
ここで、入力された顔の特徴点の座標を、図４に示す。顔画像１２０１の原点Ｏ１２０８を基準に、右眉１２０２、右目１２０３、左眉１２０４、左目１２０５、鼻１２０６及び口１２０７のそれぞれの座標（Ｘ座標、Ｙ座標）が、使用者により特徴点の座標として選択される。
【０１０８】
次に、学習部１０３の第２の教示ベクトル計算部２０１が、入力された特徴点の座標に、（数１）を用いて、入力した各特徴点の座標をデータ照合装置として定めた順番、例えば、右眉、右目、左眉、左目、鼻、口の順に並べることで連結してひとつのベクトルデータを作成し、これを教示ベクトルＶ₂とし（Ｓ１３）、平均ベクトル計算部２０３ａに出力する。
【０１０９】
次に、パラメタ入力部１０２、学習部１０３の第２の教示ベクトル計算部２０１及び学習部１０３の第１の教示ベクトル計算部２０２は、以上Ｓ１０からＳ１３までの処理を、必要とするＮ個分繰り返し、処理が終了した場合は、（Ｓ１６）に進み、終了していない場合は、Ｓ１１からＳ１４を繰り返す（Ｓ１５）。
【０１１０】
次に、平均ベクトル計算部２０３ａ及び２０３ｂは、Ｎ個分の教示ベクトルＶ₁、Ｖ₂の集合｛Ｖ₁｝、｛Ｖ₂｝に、（数２）及び（数３）を用いて、教示ベクトルＶ₁の平均ベクトルである平均ベクトルＭ₁と、教示ベクトルＶ₂の平均ベクトルである平均ベクトルＭ₂を計算する（Ｓ１６）。
【０１１１】
そして、平均ベクトル計算部２０３ｂは、平均ベクトルＭ₁を、自己相関情報計算部２０５及び相互相関情報計算部２０４に出力し、平均ベクトル計算部２０３ａは、平均ベクトルＭ₂を、相互相関情報計算部２０４に出力する。
【０１１２】
【数２】

【数３】

次に、自己相関情報計算部２０５は、Ｎ個分の教示ベクトルＶ₁の集合｛Ｖ₁｝及び平均ベクトルＭ₁を（数４）に代入し、教示ベクトルＶ₁の分布である共分散行列Ｃ₁を計算することで、画像入力部１０１から入力したＮ個の顔画像の分布を算出する。また、自己相関情報計算部２０５は、共分散行列Ｃ₁を、擬似逆行列Ｃ₁ ^*に変形し、特徴抽出行列計算部２０６に出力する。
【０１１３】
【数４】

更に、相互相関情報計算部２０４は、Ｎ個の教示ベクトルＶ₁と教示ベクトルＶ₂、平均ベクトルＭ₁及び平均ベクトルＭ₂を（数５）に代入し、教示ベクトルＶ₁及びＶ₂の相関である共分散行列Ｃ₂を計算することで、Ｎ個の顔画像と顔画像における特徴点の座標との相関を算出し、特徴抽出行列計算部２０６に出力する（Ｓ１７）。
【０１１４】
【数５】

次に、特徴抽出行列計算部２０６は、Ｓ１７において求めた擬似逆行列Ｃ₁ ^*及び共分散行列Ｃ₂と、オンライン処理で入力される照合用顔画像とから特徴点の座標を抽出するパラメタである特徴抽出行列Ｃ₃を、（数６）に従って計算する（Ｓ１８）。
【０１１５】
【数６】

そして、特徴抽出行列計算部２０６は、求めた平均ベクトルＭ₁、Ｍ₂及び特徴抽出行列Ｃ₃を相関情報データベース１０４に出力する（Ｓ１９）。このようにして、オフライン処理は終了する。
【０１１６】
次に、オンライン処理について、図５及び図６を用いて説明する。図５は、パラメタ推定部１０５の構成を示す図である。図６は、実施の形態１にかかるデータ照合装置のオンライン処理の動作フロー図である。
【０１１７】
パラメタ推定部１０５には、第２の電子データから入力ベクトルＸを計算し、出力するベクトル計算部３０１が設けられている。
【０１１８】
また、パラメタ計算部３０２は、パラメタ推定部１０５が計算した入力ベクトルＸ、平均ベクトルＭ₁、Ｍ₂及び特徴抽出行列Ｃ₃から特徴点の座標を含む期待値ベクトルを計算し、出力する。
【０１１９】
また、テンプレートマッチング部３０３は、パラメタ計算部３０２が計算した期待値ベクトルから特徴点の座標を算出し、出力する。
【０１２０】
オンライン処理の目的は、入力された顔画像から、入力に対して固有なパラメタである特徴点の座標を推定し、推定した特徴点の座標から照合に用いる顔の領域画像である顔領域を求め、求めた顔領域と画像データベース１０６に登録された画像と照合することにある。
【０１２１】
まず、画像入力部１０１が、第２の電子データである照合用画像を入力し（Ｓ２０）、パラメタ推定部１０５のベクトル計算部３０１に出力する。次に、ベクトル計算部３０１は、第２の電子データにおける各画素の値をラスタスキャン順に並べて入力ベクトルデータＸに変換し（Ｓ２１）、パラメタ計算部３０２に出力する。
【０１２２】
次に、パラメタ計算部３０２は、入力ベクトルデータＸ、オフライン処理で求めた特徴抽出行列Ｃ₃、平均ベクトルＭ₁及び平均ベクトルＭ₂を（数７）に代入し、入力ベクトルデータＸに対する特徴点の座標を含む期待値ベクトルＥを計算し（Ｓ２２）、テンプレートマッチング部３０３に出力する。
【０１２３】
【数７】

ここで、（数７）は、オフライン処理で求めたＮ個の顔画像と顔画像の特徴点の座標との相互相関を用いて、照合用画像の特徴点の座標を算出する式である。
【０１２４】
ここで、テンプレートマッチによる計算コスト算出の場合と、同じ例で本発明の計算コストを算出する。入力画像サイズを縦１５０×横１５０画素＝２２５００画素として、入力に対し両目の座標（右眼Ｘ座標、右眼Ｙ座標、左眼Ｘ座標、左眼Ｙ座標）計４次元を推定する場合を想定する。この場合、１５０×１５０の入力画像を縦一列に変換したベクトルを（数７）に代入し、乗算を単位演算とした場合、Ｃ₃は縦２２５００×横４の行列、（Ｘ−Ｍ₁）は２２５００次元のベクトルであるため、乗算回数は４×２２５００＝９００００（ｐｅｌ）となる。つまり、乗算だけでテンプレートマッチの５００分の１の計算コストとなり、その効果は大きい。
【０１２５】
ここで、（数７）を用いた計算によって特徴点の座標の推定が可能な理由を説明する。
【０１２６】
（数７）で得られる特徴点の座標を示す期待値ベクトルＥは、教示ベクトルＶ₁と教示ベクトルＶ₂との関係を、２つのベクトルの分布が正規分布であるという仮定の元でベイズ推定を用いて学習した時の、入力ベクトルデータＸに対して得られる出力の期待値に等しい。
【０１２７】
また、ベイズ推定とは、母数の分布と適当な損失関数を定義し、損失関数の期待値が最小となるように推定する統計的推定法をいう。即ち、（数７）によって、入力ベクトルデータＸに対して一番尤もらしい出力値を推定することができると言える。
【０１２８】
次に、テンプレートマッチング部３０３は、各特徴点の座標の合成ベクトルである期待値ベクトルＥから入力ベクトルデータの特徴点の座標を算出し（Ｓ２３）、特徴点の座標を照合部１０７に出力する。
【０１２９】
次に、照合部１０７は、Ｓ２３で求めた特徴点の座標を用いて、オフライン処理により画像データベース１０６に登録した顔画像との照合に用いる画像である顔領域を決定する（Ｓ２４）。
【０１３０】
ここで、検出された照合用の顔領域の例を図７に示す。本実施の形態では、鼻の座標１３０３を中心として、一辺の長さが両眼の間隔ａの２倍であり、上下の辺が両眼を結ぶ直線と平行であるような正方形領域を、照合に用いる領域である照合領域１３０２として決定する。
【０１３１】
次に、照合部１０７は、顔領域１３０２の画像と予めオフライン処理により登録した画像データベース１０６との画像を、例えば統計的手法である主成分分析を利用した固有顔（Ｅｉｇｅｎｆａｃｅ）法などの照合手法を用いて照合し（Ｓ２５）、結果を出力部１０８に出力する（Ｓ２６）。
【０１３２】
なお、固有顔法は、参照画像のサイズを正規化して全画素の濃淡値をＮ次元ベクトルとし、全ての参照画像から０＜Ｆ＜ＮであるＦ次元の情報として、情報量を圧縮し、顔部分空間を主成分分析という統計的手法により生成する。そして、入力画像から顔の存在確率の高い領域を正規化し、顔部分空間との直交距離を類似度とし、顔部分空間への射影先の位置により人物を認識するというものである。
【０１３３】
出力部１０８の出力例を図８に示す。ディスプレイ１５０１上には、入力された入力顔画像１５０２と、照合結果１５０４、１５０５が表示される。
【０１３４】
以上のように実施の形態１によれば、予めオフライン処理により学習した第１の電子データであるサンプルの顔画像と、サンプルの顔画像における特徴点の座標との相関を用いて、画像入力部１０１で撮影した第２の電子データである照合用の顔画像における特徴点の座標を推定することができる。
【０１３５】
そして、上述したように推定した特徴点の座標より照合用の顔領域を求め、求めた照合用の顔領域画像と、予めオフライン処理により登録した画像データベース１０６内の顔画像を比較し、照合することができる。
【０１３６】
また、入力ベクトルに対して固有なパラメタを、３回の行列演算で推定することが出来る。つまり、従来のテンプレートマッチングを用いた探索に比べ格段に小さな計算コストで推定できるため、従来法に比べ格段に少ない演算で所望のパラメタを推定し、少ない演算で顔画像の照合ができる。
【０１３７】
また、実施の形態１によれば、第１の教示ベクトルの共分散行列から自己相関情報を求め、第１の教示ベクトルと第２の教示ベクトルとの共分散行列から相互相関情報を求め、自己相関情報と相互相関情報から特徴抽出行列を求め、第１の教示ベクトルの平均ベクトル、第２の教示ベクトルの平均ベクトル、特徴抽出行列及び入力ベクトルデータから、照合用顔画像の特徴点の座標を算出することができる。このように、自己相関情報、相互相関情報、特徴抽出行列、第１の教示ベクトルの平均ベクトル、第２の教示ベクトルの平均ベクトル、照合用顔画像の特徴点の座標を、単純に行列演算だけ、つまり乗算だけで計算できるため、コンピュータの処理コストが非常に小さくなる。
【０１３８】
なお、実施の形態１では、上述した（数１）〜（数７）を用いてパラメタの推定を行ったが、予めサンプルの顔画像とサンプルの顔画像における特徴点の座標との相関を求め、求めた相関を使ってパラメタの推定を行う式であれば（数１）〜（数７）以外の式を用いても同様の効果を得ることができる。
【０１３９】
なお、サンプルの画像データである第１の電子データ以外でも、特徴点の座標の推定を行うことは可能である。
【０１４０】
（実施の形態２）
本発明の実施の形態２は、実施の形態１では教示ベクトルＶ₂を顔の各特徴点の座標から求めるのに対し、実施の形態２では顔の特徴点の座標周辺の領域画像から教示ベクトルＶ₂を求め、照合用顔画像に適したテンプレートを照合用画像から自動的に生成するようにしたものである。
【０１４１】
これにより、一枚のテンプレートでのマッチング処理が可能になるというものであり、従来のように複数のテンプレートを用いる場合に比べ各段に計算コストを減少させることができるというものである。
【０１４２】
具体的には、まず、実施の形態２は、予め画像入力部１０１で撮影した第１の電子データであるサンプルの顔画像とサンプルの顔画像における眼、鼻、口及び眉などの特徴点の座標周辺の画像である領域画像との相関を調べる。次に、実施の形態２は、この相関を用いて画像入力部１０１で撮影した第２の電子データである照合用の顔画像からマッチング処理に使う特定の領域画像をテンプレートとして求める。
【０１４３】
次に、実施の形態２は、求めたテンプレートと照合用の顔画像とのマッチングを行うことで、照合用顔画像の特徴点の座標を推定し、照合に用いる領域である顔領域画像を求める。
【０１４４】
そして、実施の形態２は、顔領域画像と予め用意してある画像データベース１０６内の画像とを比較することで、第２の電子データである顔画像の照合を行う。
【０１４５】
以下、実施の形態２によるデータ照合装置について詳細に説明する。
【０１４６】
また、実施の形態２にかかるデータ照合装置の構成を示す図は、実施の形態１のデータ照合装置と同様であるので、説明は省略する。
【０１４７】
また、実施の形態２によるデータ照合装置は、実施の形態１と同様に、オフライン処理とオンライン処理に大別される。ここで、特に、実施の形態１との差異について、図９から図１１を用いて説明する。
【０１４８】
図９は、実施の形態２によるオフライン処理の動作フローを示す図であり、図１０は、第１の電子データであるサンプルの顔画像から特徴点の座標周辺の画像である領域画像を示す図であり、図１１は、オンライン処理の動作フローを示す図である。
【０１４９】
実施の形態２のオフライン処理では、Ｓ９０からＳ９２の処理は図３に示すＳ１０からＳ１２の処理、Ｓ９４からＳ９８までの処理は、図３に示すＳ１５〜Ｓ１９の処理と同様の処理であるので、ここでの説明は省略する。
【０１５０】
次に、学習部１０３の第２の教示ベクトル計算部２０１が、Ｓ９２において選択された特徴点の座標から、図１０に示すように、特徴点の座標周辺の画像である領域画像を選択する。
【０１５１】
顔画像１４０１の原点Ｏ１４０２を基準に、右眉１４０３ａ、右目１４０４ａ、左眉１４０５ａ、左目１４０６ａ、鼻１４０７ａ及び口１４０８ａのそれぞれの座標（Ｘ座標、Ｙ座標）を特徴点の座標として、第２の教示ベクトル計算部２０１に入力される。
【０１５２】
また、第２の教示ベクトル計算部２０１は、入力された特徴点の座標１４０３ａ〜１４０８ａから顔の各特徴点の座標のテンプレート画像である右眉領域画像１４０３ｂ、右目領域画像１４０４ｂ、左眉領域画像１４０５ｂ、左目領域画像１４０６ｂ、鼻領域画像１４０７ｂ及び口領域画像１４０８ｂを作成する。
【０１５３】
各領域画像を矩形とする場合の決定方法としては、図７で、求めた両眼の間隔ａを基準として、各領域の幅と高さを決定し、特徴点の座標を中心として領域を設定する。眉の幅はａ／２であり、高さはａ／４となり、眼の幅はａ／２であり、高さはａ／４となり、鼻の幅はａ×２／３であり、高さはａ×２／３となり、口の幅はａであり、高さはａ／２となる。
【０１５４】
このように、両眼の間隔ａを基準にすることで、図１０の顔画像に表示される顔の大きさに因らずに各領域画像を決定することが可能となる。
【０１５５】
次に、第２の教示ベクトル計算部２０１が、作成した顔の各特徴点の座標のテンプレート画像１４０３ｂ〜１４０８ｂにおける各画素の値を、（数１）を用いてラスタスキャン順に並び替え、ベクトルデータ化した合成ベクトルを作成し、これを教示ベクトルＶ₂とする（Ｓ９３）。
【０１５６】
ただし、Ｓ９７において、求められる特徴抽出行列Ｃ₃は、教示ベクトルＶ₂を用いて作成されるので、領域画像を抽出するパラメタである。
【０１５７】
次に、実施の形態２にかかるオンライン処理について図１１を用いて説明する。Ｓ１１０からＳ１１２の処理は図６のＳ２０からＳ２２の処理と同様であるので、ここでの説明は省略する。
【０１５８】
Ｓ１１２おいて作成した期待値ベクトルＥは、オフライン処理Ｓ９５で求めた教示ベクトルＶ₁、Ｖ₂の平均ベクトルＭ₁、Ｍ₂と、Ｓ９７で求めた特徴抽出行列Ｃ₃と、オンライン処理のＳ１１１で求めたベクトルＸを用いた第２の電子データである照合用の顔画像における各特徴点の座標のテンプレート画像の合成ベクトルとなっている。このため、パラメタ計算部３０２は、期待値ベクトルＥより顔の各特徴点の座標のテンプレート画像を得る（Ｓ１１３）。
【０１５９】
次に、テンプレートマッチング部３０３は、Ｓ１１３において得たテンプレート画像と、Ｓ１１０において得た画像入力部１０１から入力した照合用の画像とをテンプレートマッチを行い、テンプレート画像にマッチする領域を検出し、この検出した領域から、顔の各特徴点の座標を決定する（Ｓ１１４）。
【０１６０】
なお、テンプレートマッチ処理は、図２８に示すように入力画像２００１上を探索窓２００２でスキャンし、探索窓内の画像を入力とした際に、テンプレート画像との相関が最も高い領域を選ぶ処理である。テンプレートマッチに用いる相関演算は、例えば、ＳＡＤ相関演算（数８）や正規化相関演算（数９）などを用いる。なお、（数８）及び（数９）において、Ｘは探索窓内の画像、Ｔはテンプレート画像である。
【０１６１】
【数８】

【数９】

以後の、Ｓ１１５からＳ１１７の処理は図６のＳ２４からＳ２６の処理と同様であるので、ここでの説明は省略する。
【０１６２】
以上実施の形態２によれば、サンプルの画像と、サンプルの画像の特定領域画像との相関を学習し、この相関を用いて照合用顔画像の特定領域画像を推定し、推定した照合用顔画像の特定領域画像をテンプレート画像とできる。これにより、照合用顔画像自身からテンプレート画像を自動的に生成できるため、一枚のテンプレートでのマッチング処理が可能になり、従来のようにマルチテンプレートなど複数のテンプレートを用いる場合に比べ各段に計算コストを減少させることが可能となる。
【０１６３】
更に、実施の形態２によれば、第１の教示ベクトル及び第２の教示ベクトルの自己相関情報及び相互相関情報から求めた特徴抽出行列、第１の教示ベクトルの平均ベクトル、第２の教示ベクトルの平均ベクトル、及び照合用顔画像のベクトルデータから、照合用顔画像の特徴点の座標を算出することができる。このように、単純に行列演算だけ、つまり乗算だけで固有なパラメタの計算ができるため、コンピュータの処理コストが非常に小さくなる。
【０１６４】
また、テンプレートマッチングにおけるテンプレート画像推定に本発明の実施の形態２を用いた場合、入力画像から想定される特定の画像を推定し、これをテンプレートに用いたマッチング処理が可能になる。つまり、テンプレートパターン１つで処理が行えるため、従来のテンプレートマッチングのように、入力画像の多様性に対応した、パターン及び大きさの異なる複数個のテンプレートを用意するマルチテンプレート法と比較して、計算コストの削減が可能である。
【０１６５】
（実施の形態３）
本発明の実施の形態３は、教示ベクトルＶ₂を実施の形態１では顔の各特徴点の座標から求め、実施の形態２では顔の特徴点の座標周辺の領域画像から求めるのに対して、顔の特徴点の座標及び顔の特徴点の座標周辺における領域画像の結合ベクトルから教示ベクトルＶ₂を求めるようにしたものである。
【０１６６】
そして、このように求めた教示ベクトルＶ₂から照合用顔画像の特徴点の座標を推定し、更に、照合用顔画像に適したテンプレートを照合用画像から自動的に生成し、この推定した特徴点の座標の周辺領域を自動生成したテンプレートを用いてマッチングをするものである。
【０１６７】
これにより、テンプレートの探索範囲を限定し、一枚のテンプレートでのマッチング処理が可能になるというものである。この結果、従来のように、照合用顔画像の全領域を探索したり、複数のテンプレートを使用したりするのに比べ、各段に計算コストを減少させることができるというものである。
【０１６８】
以下、実施の形態３によるデータ照合装置について詳細に説明する。
【０１６９】
また、実施の形態３にかかるデータ照合装置の構成を示す図は、実施の形態１のデータ照合装置と同様であるので、説明は省略する。
【０１７０】
また、実施の形態３によるデータ照合装置は、実施の形態１と同様に、オフライン処理とオンライン処理に大別される。ここで、特に、実施の形態１及び実施の形態２との差異について、図１２及び図１３を用いて説明する。
【０１７１】
図１２は、実施の形態３によるオフライン処理の動作フローを示す図であり、図１３は、オンライン処理の動作フローを示す図である。
【０１７２】
オフライン処理では、Ｓ１２００からＳ１２０２までの処理は，図３に示すＳ１０からＳ１２の処理と同様であり、Ｓ１２０５からＳ１２０９までの処理は図３に示すＳ１５からＳ１９までの処理と同様であるので、ここでの説明は省略する。
【０１７３】
Ｓ１２０２により、選択された特徴点の座標から、パラメタ入力部１０２が、図１０に示すように、特徴点の座標周辺の画像である領域画像を選択する（Ｓ１２０３）。
【０１７４】
次に、学習部１０３は、（数１）を用いて、顔の各特徴点の座標を順に並べて連結してひとつのベクトルデータにしたものと、作成した顔の各特徴点の座標のテンプレート画像１４０３ｂ〜１４０８ｂを構成する各画素の値をラスタスキャン順に並べることでベクトルデータ化したベクトルデータとを合成した合成ベクトルを作成し、これを教示ベクトルＶ₂とする（Ｓ１２０４）。
【０１７５】
ただし、Ｓ１２０８において、求められる特徴抽出行列Ｃ₃は、教示ベクトルＶ₂を用いて作成されるので、特徴点の座標及び領域画像を抽出するパラメタである。
【０１７６】
次に、オンライン処理について図１３を用いて説明する。Ｓ１３００からＳ１３０３の処理は図６のＳ２０からＳ２３の処理、Ｓ１３０６からＳ１３０８の処理は図６のＳ２４からＳ２６の処理と同様であるので、ここでの説明は省略する。
【０１７７】
Ｓ１３０２において作成した期待値ベクトルＥは、オフライン処理Ｓ１６で求めた教示ベクトルＶ₁、Ｖ₂の平均ベクトルＭ₁、Ｍ₂と、Ｓ１２０８で求めた特徴点の座標及び領域画像を抽出する特徴抽出行列Ｃ₃と、オンライン処理のＳ１３０１で求めたベクトルＸを用いた第２の電子データである照合用の顔画像における各特徴点の座標のテンプレート画像の合成ベクトルとなっている。このため、パラメタ計算部３０２は、期待値ベクトルＥより、Ｓ１３０３において、照合用の顔画像における特徴点の座標を得て、Ｓ１３０４において、領域画像であるテンプレート画像を得る。
【０１７８】
次に、テンプレートマッチング部３０３は、Ｓ１３０３で得られた特徴点の座標周辺において、Ｓ１３０４において得たテンプレート画像と、Ｓ１３００において得た画像入力部１０１から入力した照合用の画像とをテンプレートマッチを行い、テンプレート画像にマッチする領域を検出する。そして、テンプレートマッチング部３０３は、この検出した領域から、顔の各特徴点の座標を決定する（Ｓ１３０５）。
【０１７９】
なお、実施の形態１と同様に、テンプレートマッチに用いる相関演算は、ＳＡＤ相関演算（数８）や正規化相関演算（数９）などを用いる。
【０１８０】
以上説明したように、実施の形態３によれば、照合用の顔画像における各特徴点の座標の座標と各特徴点の座標の領域画像の合成ベクトルを推定しておき、推定した顔の各特徴点の座標における座標の周辺に探索範囲を限定できる。
【０１８１】
さらに、この限定した探索範囲に対して、推定された各特徴点の座標の領域画像をテンプレートとして用いてテンプレートマッチング探索をするといった階層的な探索を行うことができる。
【０１８２】
このように、階層的な探索により検出精度も向上するほか、探索範囲を縮小でき、かつ使用するテンプレートも推定された一枚となり、従来に比べ少ない計算コストで検出が可能となる。
【０１８３】
また、実施の形態３によれば、第１の教示ベクトルの共分散行列を求めることで自己相関情報を、第１の教示ベクトルと第２の教示ベクトルとの共分散行列を求めることで相互相関情報を求められる。これにより、単純に行列演算だけ、つまり乗算だけで自己相関情報及び相互相関情報を求められ、処理コストが非常に小さくなる。
【０１８４】
（実施の形態４）
本発明の実施の形態４は、パラメタ推定装置を、車両画像の照合に応用した車両画像照合装置である。
【０１８５】
車両画像照合装置は、画像入力部１０１により撮影された道路上の車両を含む濃淡パターンなどの画像からナンバープレート、ライト及び運転席などの特徴点の座標をベイズ推定により検出する。そして、車両画像照合装置は、得られた特徴点の座標に基づいて得られる車両全体の画像及びナンバープレート番号などの特徴が、予め登録された車両のものと一致するかを照合するものである。
【０１８６】
また、車両画像照合装置の構成を示す図は、実施の形態１による図１と同様であるので、ここでの説明は省略する。
【０１８７】
車両画像照合装置の処理は、オフライン処理とオンライン処理に大別することができる。以下、特に、実施の形態１と異なる部分について図１４から図１６を用いて説明する。
【０１８８】
図１４は、実施の形態４によるオフライン処理の動作フローを示す図である。図１４のＳ１４０１、Ｓ１４０５〜Ｓ１４０８の処理は、実施の形態１と同様で、図３のＳ１１、Ｓ１６〜Ｓ１９の処理と同様であるので、ここでの説明は省略する。
【０１８９】
まず、画像入力部１０１が、入力される第１の電子データとして、照合の対象となるＮ台の車両画像をパラメタ入力部１０２、学習部１０３及び画像データベースに入力する（Ｓ１４００）。
【０１９０】
次に、学習部１０３の第１の教示ベクトル計算部２０２が、教示ベクトルＶ₁を算出する（Ｓ１４０１）。
【０１９１】
次に、車両画像の特徴点の座標をパラメタ入力部１０２に含まれるディスプレイが、車両画像の特徴点の座標を一枚ずつ表示する。そして、使用者が、表示された車両画像からパラメタ入力部１０２に含まれるマウスを用いて手動で車両画像の特徴点の座標を選択すると、パラメタ入力部１０２が選択された特徴点の座標を学習部１０３に出力する（Ｓ１４０２）。
【０１９２】
ここで、図１５に、車両画像の特徴点の座標を示す。入力画像１７０１上の車両における特徴点の座標として、フロント部１７０８と車両の各頂点の座標Ａ１７０２〜座標Ｆ１７０７が入力される。
【０１９３】
次に、学習部１０３の第２の教示ベクトル計算部２０１が、入力された特徴点の座標から教示ベクトルＶ₂を算出する（Ｓ１４０３）。
【０１９４】
次に、パラメタ入力部１０２、学習部１０３の第２の教示ベクトル計算部２０１及び学習部１０３の第１の教示ベクトル計算部２０２は、Ｎ台分の処理が終了したかを判定し（Ｓ１４０４）、終了したらＳ１４０５に進み、終了していない場合はＳ１４００に進む。
【０１９５】
以後、Ｓ１４０５からＳ１４０８の処理は、図３に示すＳ１６からＳ１９の処理と同様であり、実施の形態１と同様の処理であるため、ここでの説明は省略する。
【０１９６】
次に、オンライン処理について、図１６に示す動作フローを用いて説明する。
【０１９７】
オンライン処理の目的は、入力された車両画像から求める特徴点の座標を推定し、この特徴点の座標を用いて車両画像、ナンバープレート及びフロント部の画像の解析・保存をすることにある。
【０１９８】
図１６のＳ１９００からＳ１９０３、Ｓ１９０７の処理は、図６のＳ２０からＳ２３、Ｓ２６の処理と同様であり、実施の形態１と同様の処理であるので、ここでの説明は省略する。
【０１９９】
照合部１０７は、Ｓ１９０３で求めた特徴点の座標から基準となる間隔を決定し、これに基づき、車両画像、ナンバープレート部領域画像及びフロント部領域画像を切り出す（Ｓ１９０４）。
【０２００】
次に、照合部１０７は、ナンバープレート部領域画像を文字読み取り装置などにかけて、車両のナンバープレート番号を読み取る（Ｓ１９０５）。
【０２０１】
更に、照合部１０７は、車両画像及びナンバープレートと、予めオフライン処理により登録した画像データベース１０６との画像を、例えば固有顔法などの照合手法を用いて照合する（Ｓ１９０６）。
【０２０２】
ここで、図１７に、入力画像１８０２、車両画像１８０４、ナンバープレート部領域画像及び読み取り結果１８０５、照合結果１８０６、及びフロント部領域画像１８０７の表示例を示す。
【０２０３】
以上説明したように実施の形態４により、入力された車両画像及び推定された特徴点の座標の座標を図示し、特徴点の座標を基準に車両全体の画像、ナンバープレート部の画像及びナンバープレート部の文字の読み取り、予め登録された車両との照合結果、運転席及び助手席の画像の拡大画像を表示することにより、車両の情報及び運転者の状況を簡易に把握することができる。
【０２０４】
なお、実施の形態４では、実施の形態１のパラメタ推定方法を用いたが、実施の形態２または実施の形態３を車両画像照合装置に適用した形態であっても良い。
【０２０５】
（実施の形態５）
本発明の実施の形態５は、サンプルの入力画像である入力ベクトルの分布にバラツキが存在する場合に、様々な分布を持つ照合用の入力画像に対応したものである。
【０２０６】
具体的には、サンプルの入力画像と、特徴点の座標との合成情報を複数の分布に分け、それぞれの分布毎に、照合用画像と、照合用画像の特徴点の座標との相関を調べ、これらの相関を用いて照合用画像の特徴点の座標を求めるものである。
【０２０７】
また、実施の形態５によるデータ照合装置は、実施の形態１と同様に、オフライン処理と、オンライン処理に大別される。ここでは、特に、実施の形態１との差異について、図１８から図２０を用いて説明する。
【０２０８】
また、データ照合装置の構成を示す図は、実施の形態１のデータ照合装置と同様であるため、ここでの説明は省略する。ただし、学習部１０３は図２０に示す構成とする。ここで、４０１に示すものは、複数の第１の教示ベクトル及び第２の教示ベクトルを確率分布に基づいた要素分布に分類するための分布要素計算部である。
【０２０９】
また、図１８は、実施の形態５によるオフライン処理の動作フローを示す図であり、図１９は、オンライン処理の動作フローを示す図である。
【０２１０】
オフライン処理における、Ｓ２１００からＳ２１０４の処理は、図３に示すＳ１０からＳ１５までの処理と同じであり、実施の形態１と同様であるので、ここでの説明は省略する。
【０２１１】
Ｓ２１０４によるＮ個分の第１の電子データの入力を確認した後に、分布要素計算部４０１は、Ｎ人分の教示ベクトルＶ₁、Ｖ₂の合成ベクトルを構成する。
【０２１２】
次に、分布要素計算部４０１は、Ｎ人分の合成ベクトルの集合における確率分布を混合ガウシアンモデル（以下ＧＭＭと呼ぶ）でＲ個の分布を持つ要素分布にモデル化する。そして、分布要素計算部４０１は、要素分布毎に第１の教示ベクトル及び第２の教示ベクトルを平均ベクトル計算部２０３ａに出力する。
【０２１３】
平均ベクトル計算部２０３ａ、平均ベクトル計算部２０３ｂ、相互相関情報計算部２０４及び自己相関情報計算部２０５は、モデル化した際のｋ（ｋ＝１・・Ｒ）番目の要素分布のパラメタを計算する。
【０２１４】
即ち、平均ベクトル計算部２０３ｂは、ｋ番目の要素分布に属するベクトルＶ₁の平均ベクトルＭ₁ ^kを計算し、平均ベクトル計算部２０３ａは、ｋ番目の要素分布に属するベクトルＶ₂の平均ベクトルＭ₂ ^kを計算し、相互相関情報計算部は、ベクトルＶ₁の共分散行列Ｃ₁ ^kの擬似逆行列であるＣ₁ ^k*を計算し、自己相関情報計算部は、ベクトルＶ₁、Ｖ₂の相互相関行列Ｃ₁₂ ^kを計算する（Ｓ２１０５）。
【０２１５】
また、平均ベクトル計算部２０３ｂは、平均ベクトルＭ₁ ^kを、自己相関情報計算部２０５及び相互相関情報計算部２０４に出力し、平均ベクトル計算部２０３ａは、平均ベクトルＭ₂ ^kを、相互相関情報計算部２０４に出力する。
【０２１６】
この計算には、通常ＥＭ（Ｅｘｐｅｃｔａｔｉｏｎ Ｍａｘｉｍｉｚａｔｉｏｎ）アルゴリズムが用いられるが、これは文献：Ｃｈｒｉｓｔｏｐｈｅｒ Ｍ．Ｂｉｓｈｏｐ， Ｏｘｆｏｒｄ出版“Ｎｅｕｒａｌ Ｎｅｔｗｏｒｋｓ ｆｏｒ Ｐａｔｔｅｒｎ Ｒｅｃｏｇｎｉｔｉｏｎ”５９〜７３頁（１９９５）に詳細に記述されている。
【０２１７】
次に、自己相関情報計算部２０６は、求めたベクトルＶ₁の共分散行列の擬似逆行列Ｃ₁ ^k*と、相互相関行列Ｃ₁₂ ^kから（数１０）に従って特徴抽出行列Ｃ₃ ^kを計算する（Ｓ２１０６）。
【０２１８】
【数１０】

次に、特徴抽出行列計算部２０６は、平均ベクトルＭ₁ ^k、Ｍ₂ ^k及び特徴抽出行列Ｃ₃ ^kを相互情報データベースに出力する（Ｓ２１０７）。
【０２１９】
次に、学習部１０３は、Ｒ個の分布の平均ベクトルＭ₁ ^k、Ｍ₂ ^k及び特徴抽出行列Ｃ₃ ^kが相互情報データベースに蓄積されている場合は、オフライン処理を終了し、Ｒ個の分布の平均ベクトルＭ₁ ^k、Ｍ₂ ^k及び特徴抽出行列Ｃ₃ ^kが相互情報データベースに蓄積されていない場合は、Ｓ２１０５からＳ２１０７を繰り返す（Ｓ２１０８）。
【０２２０】
次に、実施の形態５におけるオンライン処理について、図１９に示す動作フローを用いて説明する。
【０２２１】
実施の形態５のオンライン処理における、Ｓ２２００、Ｓ２２０１、Ｓ２２０３〜Ｓ２２０６の処理は、図６のＳ２０、Ｓ２１、Ｓ２３からＳ２６までの処理と同じであり、実施の形態１と同様であるので、ここでの説明は省略する。
【０２２２】
パラメタ計算部３０２は、Ｓ２２０１により算出した入力ベクトルデータＸに対して、オフラインで求めた特徴抽出行列Ｃ₃ ^k及び平均ベクトルＭ₁ ^k、Ｍ₂ ^kから、（数１１）に従って入力ベクトルデータＸに対する特徴点の座標の期待値ベクトルＥを計算する（Ｓ２２０２）。
【０２２３】
【数１１】

（数１１）で得られる特徴点の座標の期待値ベクトルＥは、教示ベクトルＶ₁と教示ベクトルＶ₂の関係を、２つのベクトルの合成ベクトルにおける分布が混合正規分布であるという仮定の下で、ベイズ推定を用いて学習した時の、入力ベクトルデータＸに対して得られる出力の期待値に等しい。ベイズ推定とは、母数の分布と適当な損失関数を定義し、損失関数の期待値が最小となるように推定する統計的推定法をいう。すなわち、（数１１）によって、入力ベクトルデータＸに対して一番尤もらしい出力値を推定することができると言える。
【０２２４】
以後のＳ２２０３からＳ２２０６の処理は、実施の形態１と同様であるので、ここでの説明は省略する。
【０２２５】
以上のように実施の形態５によれば、サンプルの入力画像を複数の分布に分け、それぞれの分布毎にサンプルの入力画像と、この入力画像の特徴点の座標との相関を調べるようにし、これらの相関を用いて照合用の入力画像の特徴点の座標を推定することができる。これにより、照合用の入力画像の分布、つまり特性にバラツキがあっても、正確に特徴点の座標を推定することができる。
【０２２６】
また、（数１１）に示されるように、行列演算による直接計算によって、入力ベクトルの固有なパラメタを推定することが出来る。このように、従来のテンプレートマッチングを用いた繰り返し演算による探索に比べ、格段に小さな計算コストで、かつ、混合分布モデルを用いているため非常に高精度に推定できるため、その効果は非常に大きい。
【０２２７】
また、実施の形態５によれば、単純に行列演算だけ、つまり乗算だけで要素分布毎の相互相関情報が求められる。
【０２２８】
（実施の形態６）
本発明の実施の形態６は、実施の形態５と同様に、サンプルの入力画像である入力ベクトルの分布にバラツキが存在する場合に対応するものである。実施の形態５は教示ベクトルＶ₂顔を各特徴点の座標から求めるのに対し、実施の形態６は、実施の形態２と同様に、顔の特徴点の座標周辺の領域画像から教示ベクトルＶ₂を求め、照合用顔画像に適したテンプレートを照合用画像から自動的に生成するようにしたものである。
【０２２９】
具体的には、サンプルの入力画像と、顔の特徴点の座標周辺の領域画像との合成情報を複数の分布に分け、それぞれの分布毎に、照合用画像と、照合用画像の特徴点の座標との相関を調べ、これらの相関を用いて照合用画像の特徴点の座標を求めるものである。
【０２３０】
また、実施の形態６によるデータ照合装置は、実施の形態２及び実施の形態５と同様に、オフライン処理と、オンライン処理に大別される。ここでは、特に、実施の形態１との差異について、図２１及び図２２を用いて説明する。
【０２３１】
実施の形態６のオフライン処理における、Ｓ２４００からＳ２４０２、Ｓ２４０４の処理は、図３のＳ１０からＳ１２、Ｓ１５の処理と同様であり、実施の形態１と同様であるので、ここでの説明は省略する。また、Ｓ２４０５からＳ２４０８の処理は、図１８のＳ２１０５からＳ２１０８の処理と同様であるので、説明を省略する。
【０２３２】
学習部１０３の第２の教示ベクトル計算部２０１が、Ｓ２４０２により入力した特徴点の座標から、顔の各部位における周辺領域の画像を切り出し、教示ベクトルＶ₂を算出する（Ｓ２４０３）。
【０２３３】
次にＳ２４０４において、Ｎ個分の入力の終了を確認した後に、学習部１０３は、図２１に示すＳ２４０５からＳ２４０８の処理を行い、Ｒ個の分布の平均ベクトルＭ₁ ^k、Ｍ₂ ^k及び特徴抽出行列Ｃ₃ ^kが相互情報データベースに蓄積を行う。
【０２３４】
ただし、ここでの特徴抽出行列Ｃ₃ ^kは、顔の特徴点の座標周辺の領域画像により産出されたものである。
【０２３５】
次に、実施の形態６におけるオンライン処理について、図２２に示す動作フローを用いて説明する。
【０２３６】
実施の形態６のオンライン処理における、Ｓ２５００、Ｓ２５０１、Ｓ２５０５からＳ２５０７までの処理は、図６のＳ２０、Ｓ２１、Ｓ２４からＳ２６の処理と同様であり、実施の形態１と同様であるので、ここでの説明は省略する。
【０２３７】
パラメタ計算部３０２は、Ｓ２５０１において算出されたベクトルＸに、（数１１）を用いて、期待値ベクトルＥを算出する（Ｓ２５０２）。
【０２３８】
次に、テンプレートマッチング部３０３は、Ｒ個の分布を持つ要素分布にモデル化したテンプレート画像を作成し（Ｓ２５０３）、テンプレートマッチングを実行し、第２の電子データにおける特徴点の座標を算出する（Ｓ２５０４）。
【０２３９】
以後のＳ２５０５からＳ２５０７の処理は、実施の形態１と同様であるので、ここでの説明は省略する。
【０２４０】
以上説明したように実施の形態６によれば、サンプルの画像を複数の分布に分け、それぞれの分布毎にサンプル画像とこの画像の各部位の画像との相関を調べるようにし、これらの相関を用いることにより、照合用の入力画像の各部位における画像を推定することができる。これにより、照合用の入力画像の分布、つまり特性にバラツキがあっても、正確に各部位を推定できる。そして、この推定した各部位の画像をテンプレートとしてテンプレートマッチングを行うことで、照合用の入力画像における特徴点の座標を検出できる。
【０２４１】
（実施の形態７）
本発明の実施の形態７は、実施の形態５と同様に、サンプルの入力画像である入力ベクトルの分布にバラツキが存在する場合に対応したものである。教示ベクトルＶ₂を、実施の形態５では、顔の各特徴点の座標から求め、実施の形態６では、顔の特徴点の座標周辺の領域画像から求めるのに対して、実施の形態７では、実施の形態３と同様に、顔の特徴点の座標及び顔の特徴点の座標周辺における領域画像の結合ベクトルから教示ベクトルＶ₂を求める。そして、求めた教示ベクトルＶ₂を用いて、照合用顔画像の特徴点の座標を推定し、更に、照合用顔画像に適したテンプレートを照合用画像から自動的に生成する。そして、この推定した特徴点の座標の周辺領域を自動生成したテンプレートを用いてマッチングをするものである。
【０２４２】
これにより、テンプレートの探索範囲を限定し、一枚のテンプレートでのマッチング処理が可能になるというものであり、従来のように、照合用顔画像の全領域の探索、複数のテンプレートの使用に比べ各段に計算コストを減少させることができるというものである。
【０２４３】
また、実施の形態７によるデータ照合装置は、実施の形態３及び実施の形態５と同様に、オフライン処理と、オンライン処理に大別される。ここでは、特に、実施の形態１との差異について、図２３及び図２４を用いて説明する。
【０２４４】
実施の形態７のオフライン処理における、Ｓ２６００からＳ２６０２、Ｓ２６０５の処理は、図３のＳ１０からＳ１２、Ｓ１５の処理と同様であり、実施の形態１と同様であるので、ここでの説明は省略する。
【０２４５】
学習部１０３の第２の教示ベクトル計算部２０１は、Ｓ２６０２において入力された特徴点の座標を用いて、第１の電子データにおける各部位の周辺領域の画像を切り出し（Ｓ２６０３）、Ｓ２６０２において入力された特徴点の座標と、Ｓ２６０３により切り出された画像の合成ベクトルである教示ベクトルＶ₂を算出する（Ｓ２６０４）。
【０２４６】
Ｓ２６０５において、Ｎ個分の入力の終了を確認した後に、学習部１０３は、図２１のＳ２４０５からＳ２４０８と同様の処理を行い、Ｒ個の分布の平均ベクトルＭ₁ ^k、Ｍ₂ ^k及び特徴抽出行列Ｃ₃ ^kが相互情報データベースに蓄積を行う（Ｓ２６０６からＳ２６０９）。
【０２４７】
ただし、ここでの特徴抽出行列Ｃ₃ ^kは、第１の電子データにおける特徴点の座標と特徴点の座標周辺の領域画像の合成ベクトルにより産出されたものである。
【０２４８】
次に、実施の形態７におけるオンライン処理について、図２４に示す動作フローを用いて説明する。
【０２４９】
実施の形態７のオンライン処理における、Ｓ２７００、Ｓ２７０１、Ｓ２７０５からＳ２７０７の処理は、図６のＳ２０、Ｓ２１、Ｓ２４からＳ２６までの処理と同様であり、実施の形態１と同様であるので、ここでの説明は省略する。
【０２５０】
パラメタ計算部３０２は、Ｓ２７０１において算出されたベクトルＸから、（数１１）を用いて、期待値ベクトルＥを算出する（Ｓ２７０２）。
【０２５１】
次に、テンプレートマッチング部３０３は、Ｒ個の分布を持つ要素分布にモデル化したテンプレート画像を作成し（Ｓ２７０３）、Ｓ２７０３によりベクトルＥで得られた特徴点の座標周辺でテンプレートマッチングを実行し、第２の電子データにおける特徴点の座標を算出する（Ｓ２７０４）。
【０２５２】
以後のＳ２７０５からＳ２７０７の処理は、実施の形態１と同様であるので、ここでの説明は省略する。
【０２５３】
以上説明したように実施の形態７によれば、サンプルの画像を複数の分布に分け、それぞれの分布毎にサンプル画像と、この画像の各特徴点の座標と各特徴点の座標の画像との相関を調べるようにできる。そして、調べた相関を使用して照合用の入力画像の顔における各特徴点の座標を推定できる。そして、さらに推定した特徴点の座標の周辺領域を推定されたテンプレートを用いてマッチングをするという階層的な探索処理により、照合用画像の特徴点の座標の推定ができる。
【０２５４】
これにより、特徴点の座標の推定における精度の向上が可能であり、テンプレートマッチングの範囲を限定した領域に絞り、しかも一枚のテンプレートだけで行える。この結果、従来の全探索する手法に比べ計算コストの削減が可能となる。
【０２５５】
（実施の形態８）
本発明の実施の形態８は、サンプルの入力画像が車両であり、実施の形態５と同様に、サンプルの入力画像である入力ベクトルの分布にバラツキが存在する場合に、様々な分布を持つ照合用の入力画像に対応したものである。
【０２５６】
また、実施の形態８によるデータ照合装置は、実施の形態４及び実施の形態５と同様に、オフライン処理と、オンライン処理に大別される。ここでは、特に、実施の形態１との差異について、図２５を用いて説明する。
【０２５７】
実施の形態８のオフライン処理は、実施の形態５と同様であるので、ここでの説明は省略する。ただし、Ｒ個の分布の平均ベクトルＭ₁ ^k、Ｍ₂ ^k及び特徴抽出行列Ｃ₃ ^kは、車両画像、ナンバープレート部領域画像及びフロント部領域画像からなる。
【０２５８】
次に、実施の形態８におけるオンライン処理について、図２５に示す動作フローを用いて説明する。
【０２５９】
実施の形態８のオンライン処理における、Ｓ２８００、Ｓ２８０１、Ｓ２８０３からＳ２８０７の処理は、図６のＳ２０、Ｓ２１、Ｓ２３及びＳ２６の処理と同様であり、実施の形態１と同様であるので、ここでの説明は省略する。
【０２６０】
パラメタ計算部３０２は、Ｓ２８０１において算出されたベクトルＸから、（数１１）を用いて、期待値ベクトルＥを算出する（Ｓ２８０２）。
【０２６１】
照合部１０７は、Ｓ２８０３において算出された特徴点の座標から、車両画像、ナンバープレート部領域画像及びフロント部領域画像を算出し（Ｓ２８０４）、ナンバープレートを読取り（Ｓ２８０５）、車両画像及びナンバープレートの照合を行う（Ｓ２８０６）。
【０２６２】
以上説明したように、実施の形態８によれば、サンプルの車両画像を複数の分布に分け、それぞれの分布毎にサンプルの車両画像と、この画像の各特徴点の座標の画像との相関を調べるようにし、これらの相関を用いて照合用の車両画像の各特徴点の座標における画像を推定することができる。
【０２６３】
これにより、照合用の車両画像の分布、つまり特性にバラツキがあっても、正確に各特徴点の座標を推定できる。そして、この推定した各特徴点の座標の画像をテンプレートとしてテンプレートマッチングを行うことで、照合用の車両画像における特徴点の座標を検出できる。
【０２６４】
なお、実施の形態８では、実施の形態５のデータ照合装置を車両画像照合装置に適用した形態で説明したが、実施の形態６または実施の形態７のデータ照合装置を車両画像照合装置に適用した形態であっても良い。
【０２６５】
なお、本発明は、上述の実施の形態に限定されるものではない。
【０２６６】
なお、本発明では、コンピュータがプログラムを読み込むことで実現し、これをフレキシブル等の記憶媒体に記憶して移送することにより、独立した他のコンピュータシステムで容易に実施することができる。
【０２６７】
また、本発明を実施するコンピュータをプログラムするために使用できる命令を含む記憶媒体であるコンピュータプログラム製品が本発明の範囲に含まれる。この記憶媒体は、フレキシブルディスク、光ディスク、ＣＤＲＯＭ、磁気ディスク、ＲＯＭ、ＲＡＭ、ＥＰＲＯＭ、ＥＥＰＲＯＭ、磁気光カード、メモリカード又はＤＶＤ等であるが、特にこれらに限定されるものではない。
【０２６８】
また、本発明のパラメタ推定装置を顔画像及び車両画像以外の画像の照合に使用しても良い。
【０２６９】
また、本発明では、パラメタ推定装置を画像照合に用いたが、入力データとして画像以外のデータを入力することで、本パラメタ推定装置を画像照合以外の照合装置に用いても良い。例えば、入力データとして音声データなどを入力することで、本パラメタ推定装置を音声照合装置に用いても良い。
【０２７０】
【発明の効果】
以上説明したように本発明によれば、少ない処理コストで入力画像の特徴点の座標を正確に求めることができる。
【図面の簡単な説明】
【図１】本発明の実施の形態１におけるデータ照合装置を示す図
【図２】実施の形態１における学習部を示す図
【図３】実施の形態１にかかるデータ照合装置のオフライン処理の動作フロー図
【図４】実施の形態１においてユーザにより入力される顔の特徴点の座標を示す図
【図５】実施の形態１におけるパラメタ推定部を示す図
【図６】実施の形態１にかかるデータ照合装置のオンライン処理の動作フロー図
【図７】実施の形態１において検出された顔領域を示す図
【図８】実施の形態１においてディスプレイへ出力される図
【図９】本発明の実施の形態２にかかるデータ照合装置のオフライン処理の動作フロー図
【図１０】実施の形態２においてユーザにより入力される顔の特徴点の座標を示す図
【図１１】実施の形態２にかかるデータ照合装置のオンライン処理の動作フロー図
【図１２】本発明の実施の形態３にかかるデータ照合装置のオフライン処理の動作フロー図
【図１３】実施の形態３にかかるデータ照合装置のオンライン処理の動作フロー図
【図１４】本発明の実施の形態４にかかる車両画像照合装置のオフライン処理の動作フロー図
【図１５】実施の形態４においてユーザにより入力される車両画像の特徴点の座標を示す図
【図１６】実施の形態４にかかる車両画像照合装置のオンライン処理の動作フロー図
【図１７】実施の形態４においてディスプレイへ出力される図
【図１８】本発明の実施の形態５にかかるデータ照合装置のオフライン処理の動作フロー図
【図１９】実施の形態５にかかるデータ照合装置のオンライン処理の動作フロー図
【図２０】実施の形態５にかかる学習部を示す図
【図２１】本発明の実施の形態６にかかるデータ照合装置のオフライン処理の動作フロー図
【図２２】実施の形態６にかかるデータ照合装置のオンライン処理の動作フロー図
【図２３】本発明の実施の形態７にかかるデータ照合装置のオフライン処理の動作フロー図
【図２４】実施の形態７にかかるデータ照合装置のオンライン処理の動作フロー図
【図２５】実施の形態８にかかる車両画像照合装置のオンライン処理の動作フロー図
【図２６】従来の顔の特徴抽出方法の動作フロー図
【図２７】目のテンプレートを示す図
【図２８】テンプレートによる探索を示す図
【符号の説明】
１０１ 画像入力部
１０２ パラメタ入力部
１０３ 学習部
１０４ 相関情報データベース
１０５ パラメタ推定部
１０６ 画像データベース
１０７ 照合部
１０８ 出力部
２０１ 第２の教示ベクトル計算部
２０２ 第１の教示ベクトル計算部
２０３ａ、２０３ｂ 平均ベクトル計算部
２０４ 相互相関情報計算部
２０５ 自己相関情報計算部
２０６ 特徴抽出行列計算部
３０１ ベクトル計算部
３０２ パラメタ計算部
３０３ テンプレートマッチング部
４０１ 分布要素計算部
１３０２ 照合領域[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a parameter estimation device that extracts a specific parameter from an input pattern, and a data collation device using the parameter estimation device.
[0002]
[Prior art]
The process of extracting a specific parameter from the input pattern is a very general process in pattern information processing. For example, processing for extracting the positions of eyes and nose from a human face image and processing for extracting the position of a license plate from a vehicle image correspond to processing for extracting parameters.
[0003]
Conventionally, the most popular method for such processing is called a collation filter method as summarized below, and many examples of use have been proposed. An example of a facial feature extraction method will be described with reference to FIG.
[0004]
As shown in the operation flow of FIG. 26, first, templates for the eyes and nose are prepared in the template database 1601. The template database 1601 stores a plurality of eye templates 1701 as shown in FIG.
[0005]
First, when an input image is given from the camera (S81), one template image 1701 is obtained from the template database 1601 (S82). Next, as shown in FIG. 28, the input image 2001 is searched in the search window 2002, and the similarity between the image in the search window 2002 and the template 1701 is obtained (S83). For the calculation of the similarity, a normalized correlation between the image in the search window 2002 and the template 1701 is often used.
[0006]
Then, it is determined whether or not this process has been performed for the entire input image 2001 (S84). Until the entire input image 2001 is performed, the input image 2001 is scanned with the search window 2002 (S85), and the process of S83 is performed.
[0007]
Next, it is determined whether or not the above-described search has been performed for all templates 1701 included in the template database 1601 (S86). If processing has not been performed for all templates 1701, the template 1701 to be processed is changed (S87). ), The process proceeds to the process of S83, and the processes of S83 to S85 are performed for all templates.
[0008]
Next, based on the similarity between the image in the search window 2002 and the template 1701 obtained by the processing of S83 to S87, the position of the local area (search window 2002 region) most similar to the template 1701 is found from the input image 2001. The position corresponding to the local area is output (S88).
[0009]
As a method based on such a method, for example, there is one described in Non-Patent Document 1.
[0010]
[Non-Patent Document 1]
R. Brunelli, T .; Poggio, “Face Recognition: Features versus Template”, IEEE Trans. Patt. Anal. Machine Intel. , Vol. PAMI-8, pp. 34-43, 1993
[0011]
[Problems to be solved by the invention]
The problem with the conventional method as described above is the processing cost of the computer. If the size of the input image to be searched is S, the template size is T, and normalized correlation is used as the similarity criterion, the time calculation amount requires 2 × T × S operations as the unit operation. To do. For example, in the coordinate extraction of the feature points of a typical face image, even when considering the case of T = 50 × 20 = 1000 (pel) and S = 150 × 150 = 22,500 (pel), 2 × 1000 is obtained only by multiplication. × 22500 = 45 × 1 million times = 45 million times. This means that the computation speed of the computer is improved, but enormous computation costs are required.
[0012]
Many templates used for processing use typical data such as an average of all learning data, and matching often fails depending on the environment. For this reason, there is a method in which a plurality of templates are prepared according to an input pattern, and similarity calculation is performed using a plurality of templates. However, since this method increases the number of processes according to the number of templates, the load becomes large even in view of the processing cost of the computer.
[0013]
The present invention has been made in view of such a point, and an object thereof is to accurately obtain the coordinates of feature points of input data at a low processing cost.
[0014]
[Means for Solving the Problems]
The present invention learns in advance the correlation between the sample data and the coordinates of the feature points in the data, and uses the correlation obtained by this learning to determine where the coordinates of the feature points of the input data are. This is what we estimated.
[0015]
Since the same kind of data and the coordinates of the feature points in this data have a certain correlation, the processing cost can be reduced by using the correlation obtained as described above to determine where the coordinates of the feature points of the data are. And can be obtained accurately.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
A parameter estimation device according to a first aspect of the present invention includes an image input unit that converts optical data into electronic data, and the coordinates of a first feature point from the first electronic data input from the image input unit. Calculating the autocorrelation information from a plurality of the first electronic data, calculating the cross-correlation information from the plurality of first electronic data and the plurality of coordinates of the first feature points, A feature extraction matrix for estimating the coordinates of the second feature point of the second electronic data input from the image input unit from the autocorrelation information and the cross-correlation information is calculated, and a plurality of the first electronic data A learning unit that outputs the first average vector calculated from the second average vector calculated from the coordinates of the plurality of first feature points and the feature extraction matrix, and the first input from the learning unit. Average vector of , The correlation information database storing the second average vector and the feature extraction matrix, the second electronic data, the first average vector, the second average vector, and the feature extraction matrix. And a parameter estimation unit for estimating the coordinates of the two feature points.
[0017]
Thereby, it is comprised by calculation of an intrinsic parameter only by matrix calculation, and has the effect | action that the coordinate of a feature point can be calculated | required correctly with little processing cost.
[0018]
According to a second aspect of the present invention, in the parameter estimation device according to the first aspect, the learning unit calculates a first teaching vector that is a vector pattern in which the first electronic data is arranged in a scanning order. A teaching vector calculation unit, a second teaching vector calculation unit that calculates a second teaching vector that is a vector pattern in which the coordinates of the first feature points are arranged in order, and a plurality of the first teaching vectors are averaged An average vector calculation unit that calculates a first average vector and calculates a second average vector obtained by averaging a plurality of the second teaching vectors, and inputs the plurality of the first teaching vectors and the first average vector An autocorrelation information calculation unit that calculates a pseudo inverse matrix of a first covariance matrix that is autocorrelation information of the first teaching vector, a plurality of the first teaching vectors, and the first average vector A plurality of the second teaching vectors and the second average vector, and calculating a second covariance matrix that is cross-correlation information between the first teaching vector and the second teaching vector. A correlation information calculation unit, a pseudo inverse matrix of the first covariance matrix, and the second covariance matrix are input, and a feature is obtained from the pseudo inverse matrix of the first covariance matrix and the second covariance matrix. And a feature extraction matrix calculator that calculates an extraction matrix.
[0019]
Thereby, since the feature extraction matrix for obtaining the coordinates of the feature points can be calculated by matrix calculation, there is an effect that the processing cost can be reduced.
[0020]
According to a third aspect of the present invention, in the parameter estimation device according to the first aspect or the second aspect, the parameter estimation unit inputs the second electronic data, and the second electronic data is scanned in order. A vector calculation unit for calculating an input vector, which is an arrayed vector pattern, and an expectation including coordinates of the second feature point from the input vector, the first average vector, the second average vector, and the feature extraction matrix A configuration including a parameter calculation unit that calculates a value vector and a template matching unit that calculates the coordinates of the second feature point from the expected value vector is adopted.
[0021]
Thereby, it has the effect | action that the coordinate of a feature point can be calculated by matrix calculation, without using a some template.
[0022]
A parameter estimation device according to a fourth aspect of the present invention includes an image input unit that converts optical data into electronic data, and the first electronic data input from the image input unit around the coordinates of the first feature point. An autocorrelation information is calculated from a plurality of the first electronic data and a parameter input unit for inputting an image, and a cross-correlation is performed from the plurality of the first electronic data and images around the coordinates of the plurality of the first feature points. A feature extraction matrix for calculating information, and further estimating an image around the coordinates of the second feature point of the second electronic data input from the image input unit from the autocorrelation information and the cross-correlation information Calculating and outputting a first average vector calculated from the plurality of first electronic data, a second average vector calculated from images around the coordinates of the plurality of first feature points, and the feature extraction matrix. With the learning department A correlation information database for storing the first average vector, the second average vector, and the feature extraction matrix input from the learning unit; the second electronic data; the first average vector; And a parameter estimation unit that estimates the coordinates of the second feature point using the average vector and the feature extraction matrix.
[0023]
Thereby, since the coordinates of the feature points are obtained from one template, the processing cost can be reduced.
[0024]
According to a fifth aspect of the present invention, in the parameter estimation device according to the fourth aspect, the learning unit calculates a first teaching vector that is a vector pattern in which the first electronic data is arranged in a scanning order. A teaching vector calculation unit, a second teaching vector calculation unit that calculates a second teaching vector that is a vector pattern in which images around the coordinates of the first feature point are arranged in order, and a plurality of the first teaching vectors An average vector calculator that calculates a first average vector obtained by averaging, a second average vector obtained by averaging a plurality of the second teaching vectors, and a plurality of the first teaching vectors and the first average An autocorrelation information calculation unit that inputs a vector and calculates a pseudo inverse matrix of a first covariance matrix that is autocorrelation information of the first teaching vector; a plurality of the first teaching vectors; An average vector, a plurality of the second teaching vectors, and the second average vector are input, and a second covariance matrix that is cross-correlation information between the first teaching vector and the second teaching vector is calculated. A cross-correlation information calculation unit, a pseudo inverse matrix of the first covariance matrix and the second covariance matrix are input, and from the pseudo inverse matrix of the first covariance matrix and the second covariance matrix A feature extraction matrix calculation unit for calculating a feature extraction matrix.
[0025]
Thereby, the calculation for creating the template can be configured only by matrix calculation, and the processing cost can be reduced.
[0026]
According to a sixth aspect of the present invention, in the parameter estimation device according to the fourth aspect or the fifth aspect, the parameter estimation unit inputs the second electronic data, and the second electronic data is scanned in order. A vector calculation unit for calculating an input vector that is a vector pattern arranged; and an image around a coordinate of a second feature point from the input vector, the first average vector, the second average vector, and the feature extraction matrix. A parameter calculation unit for calculating an expected value vector including the image, and calculating an image around the coordinates of the second feature point from the expected value vector, and matching the image around the coordinate of the second feature point A template matching unit that searches for a collation area that is an image area of electronic data and calculates coordinates of the second feature point from the collation area.
[0027]
Thus, since a template is created from an image to be collated, collation can be performed with one template, and processing cost can be reduced.
[0028]
A parameter estimation device according to a seventh aspect of the present invention includes an image input unit that converts optical data into electronic data, coordinates of a first feature point and first coordinates from the first electronic data input from the image input unit. A parameter input unit that inputs an image around the coordinates of one feature point; and autocorrelation information is calculated from the plurality of first electronic data, and the plurality of first electronic data and the plurality of first feature points are calculated. And the second electron input from the image input unit from the autocorrelation information and the cross-correlation information. Calculating a feature extraction matrix for estimating the coordinates of the second feature point of the data and an image around the coordinates of the second feature point, and a first average vector calculated from a plurality of the first electronic data; A plurality of said joint vectors A learning unit that outputs the second average vector and the feature extraction matrix to be calculated, and correlation information that accumulates the first average vector, the second average vector, and the feature extraction matrix that are input from the learning unit. A database, and a parameter estimation unit that estimates the coordinates of the second feature point using the second electronic data, the first average vector, the second average vector, and the feature extraction matrix.
[0029]
Thereby, it has the effect | action that the coordinate of a feature point and a template can be calculated | required simultaneously.
[0030]
According to an eighth aspect of the present invention, in the parameter estimation device according to the seventh aspect, the learning unit calculates a first teaching vector that is a vector pattern in which the first electronic data is arranged in a scanning order. A teaching vector calculation unit, a second teaching vector calculation unit that calculates a second teaching vector that is a vector pattern in which combined vectors are arranged in order, and a first average vector that averages a plurality of the first teaching vectors An average vector calculation unit that calculates a second average vector obtained by averaging a plurality of the second teaching vectors, a plurality of the first teaching vectors and a first average vector are input, and the first An autocorrelation information calculation unit that calculates a pseudo inverse matrix of a first covariance matrix that is autocorrelation information of the teaching vectors of the plurality of teaching vectors, a plurality of the first teaching vectors, the first average vector, The cross-correlation information for inputting the second teaching vector and the second average vector, and calculating a second covariance matrix that is cross-correlation information between the first teaching vector and the second teaching vector. A calculation unit; and a feature extraction matrix calculation unit that calculates a feature extraction matrix from the pseudo inverse matrix of the first covariance matrix and the second covariance matrix.
[0031]
As a result, the coordinates of the feature points and the template can be obtained only by matrix calculation, and the processing cost can be reduced.
[0032]
According to a ninth aspect of the present invention, in the parameter estimation device according to the seventh aspect or the eighth aspect, the parameter estimation unit inputs the second electronic data, and the second electronic data is scanned in order. A vector calculation unit for calculating an input vector that is a vector pattern arranged; a coordinate of a second feature point from the input vector, the first average vector, the second average vector, and the feature extraction matrix; A parameter calculation unit for calculating an expected value vector including an image around the coordinates of the feature point, and estimating the coordinates of the second feature point and the image around the coordinate of the second feature point from the expected value vector; A collation area that is an image area of the second electronic data that coincides with an image around the coordinates of the second feature point estimated around the coordinates of the second feature point is searched, and the second area is searched from the collation area. And a template matching unit for calculating the feature coordinates.
[0033]
As a result, the vicinity of the estimated feature point coordinates is searched with the template created from the image to be collated, so that the search processing cost can be reduced.
[0034]
A parameter estimation device according to a tenth aspect of the present invention is an image input unit that converts optical data into electronic data, and a parameter that inputs coordinates of a first feature point from the first electronic data input from the image input unit. A set of combined information of the input unit, the first electronic data and the coordinates of the first feature point is divided into a plurality of distributions, and autocorrelation information and a plurality of the first electronic data are Cross-correlation information is calculated from the plurality of first electronic data and the coordinates of the plurality of first feature points, and further, a second input from the image input unit from the auto-correlation information and the cross-correlation information. Calculating a feature extraction matrix for estimating the coordinates of the second feature point of the electronic data for each distribution, and a first average vector calculated from the plurality of first electronic data obtained for each distribution; The compound obtained for each distribution A learning unit that outputs a second average vector calculated from the coordinates of the first feature points and the feature extraction matrix obtained for each distribution, and the learning unit that is obtained for each distribution A correlation information database storing one average vector, the second average vector, and the feature extraction matrix; the second electronic data; the first average vector; the second average vector; and the feature extraction matrix. And a parameter estimation unit that estimates the coordinates of the second feature points for each distribution.
[0035]
Thereby, even when the characteristics of the first electronic data are dispersed, the coordinates of the feature points of the second electronic data can be obtained.
[0036]
According to an eleventh aspect of the present invention, in the parameter estimation device according to the tenth aspect, the learning unit calculates a first teaching vector that is a vector pattern in which the first electronic data is arranged in a scanning order. A teaching vector calculation unit, a second teaching vector calculation unit that calculates a second teaching vector that is a vector pattern in which the coordinates of the first feature points are arranged in order, a plurality of the first teaching vectors and a plurality of the teaching vectors A distribution element calculation unit that creates a plurality of combined vectors from the second teaching vector, and divides the first teaching vector and the second teaching vector into a plurality of distributions based on a probability distribution of the set of the combined vectors; A first average vector obtained by averaging a plurality of the first teaching vectors for each distribution is calculated, and a second average vector obtained by averaging the plurality of the second teaching vectors for each distribution A plurality of first teaching vectors and a first average vector are input for each distribution, and a first covariance matrix that is autocorrelation information of the first teaching vectors is calculated. An autocorrelation information calculation unit for calculating a pseudo inverse matrix for each distribution, a plurality of the first teaching vectors, the first average vector, a plurality of the second teaching vectors, and the second average vector; A cross-correlation information calculation unit that inputs each distribution, calculates a second covariance matrix that is cross-correlation information between the first teaching vector and the second teaching vector for each distribution, and for each distribution, The pseudo-inverse matrix of the first covariance matrix and the second covariance matrix are input, and a feature extraction matrix is extracted from the pseudo-inverse matrix of the first covariance matrix and the second covariance matrix for each distribution. Feature extraction matrix calculation And, including the.
[0037]
Thereby, it has the effect | action that the characteristic extraction matrix suitable for the characteristic of the distributed 1st electronic data can be calculated | required.
[0038]
According to a twelfth aspect of the present invention, in the parameter estimation device according to the tenth aspect or the eleventh aspect, the parameter estimation unit inputs the second electronic data, and the second electronic data is scanned in order. A vector calculation unit for calculating an input vector which is a vector pattern arranged; the input vector; a first average vector classified for each distribution; a second average vector classified for each distribution; and a feature extraction matrix classified for each distribution A parameter calculation unit that calculates an expected value vector including the coordinates of the second feature point, and a template matching unit that calculates the coordinates of the second feature point from the expected value vector.
[0039]
As a result, even if the characteristics of the second electronic data are dispersed, the feature extraction matrix suitable for the dispersed characteristics is used, so that the processing cost can be reduced.
[0040]
A parameter estimation device according to a thirteenth aspect of the present invention includes an image input unit that converts optical data into electronic data, and a first feature point around the coordinates of the first feature point that is input from the image input unit. A set of combined information of a parameter input unit for inputting an image, the first electronic data and an image around the coordinates of the first feature point is divided into a plurality of distributions, and a plurality of the first information is divided for each distribution. Calculate cross-correlation information from autocorrelation information from electronic data and a plurality of the first electronic data and images around the coordinates of the plurality of first feature points, and further, from the autocorrelation information and the cross-correlation information, A feature extraction matrix for estimating an image around the coordinates of the second feature point of the second electronic data input from the image input unit is calculated for each of the distributions, and the plurality of first values obtained for each of the distributions are calculated. Calculated from electronic data A learning unit that outputs a first average vector, a second average vector calculated from images around the coordinates of the plurality of first feature points obtained for each distribution, and the feature extraction matrix obtained for each distribution; A correlation information database that stores the first average vector, the second average vector, and the feature extraction matrix that are input from the learning unit obtained for each distribution, the second electronic data, the first electronic data, A parameter estimation unit that estimates the coordinates of the second feature points for each of the distributions using the average vector, the second average vector, and the feature extraction matrix.
[0041]
Thus, even when the characteristics of the first electronic data are dispersed, the coordinates of the feature points of the second electronic data can be obtained with one template.
[0042]
According to a fourteenth aspect of the present invention, in the parameter estimation device according to the thirteenth aspect, the learning unit calculates a first teaching vector that is a vector pattern in which the first electronic data is arranged in a scanning order. A teaching vector calculation unit, a second teaching vector calculation unit that calculates a second teaching vector that is a vector pattern in which images around the coordinates of the first feature point are arranged in order, and a plurality of the first teaching vectors And a plurality of combined vectors from the plurality of second teaching vectors, and a distribution element calculation unit that divides the first teaching vector and the second teaching vector into a plurality of distributions based on a probability distribution of the set of combined vectors A first average vector obtained by averaging a plurality of the first teaching vectors for each distribution, and a second average obtained by averaging the plurality of the second teaching vectors for each distribution. An average vector calculation unit for calculating a vector, a plurality of the first teaching vectors and the first average vector for each distribution, and a first covariance which is autocorrelation information of the first teaching vector An autocorrelation information calculation unit for calculating a pseudo inverse matrix of each matrix for each distribution, a plurality of the first teaching vectors, the first average vector, a plurality of the second teaching vectors, and the second average vector; For each distribution, a cross-correlation information calculation unit that calculates a second covariance matrix that is cross-correlation information between the first teaching vector and the second teaching vector for each distribution, and for each distribution The pseudo-inverse matrix of the first covariance matrix and the second covariance matrix are input, and the feature extraction matrix is obtained from the pseudo-inverse matrix of the first covariance matrix and the second covariance matrix. Feature extraction calculated for each distribution And a matrix calculator.
[0043]
Thereby, it has the effect | action that the characteristic extraction matrix suitable for the characteristic of the distributed 1st electronic data can be calculated | required.
[0044]
According to a fifteenth aspect of the present invention, in the parameter estimation device according to the thirteenth aspect or the fourteenth aspect, the parameter estimation unit inputs the second electronic data, and the second electronic data is scanned in order. A vector calculation unit for calculating an input vector which is a vector pattern arranged; the input vector; a first average vector classified for each distribution; a second average vector classified for each distribution; and a feature extraction matrix classified for each distribution A parameter calculation unit that calculates an expected value vector including an image around the coordinates of the second feature point, an image around the coordinates of the second feature point is calculated from the expected value vector, and the second feature A template area is searched for a collation area that is an image area of the second electronic data that coincides with an image around the coordinates of the point, and the coordinates of the second feature point are calculated from the collation area. And a packaging unit.
[0045]
Thereby, even if the characteristics of the second electronic data are dispersed, the coordinates of the second feature point can be obtained with a single template.
[0046]
A parameter estimation device according to a sixteenth aspect of the present invention includes an image input unit that converts optical data into electronic data, coordinates of a first feature point and first coordinates from the first electronic data input from the image input unit. A parameter input unit that inputs an image around the coordinates of one feature point; and a plurality of sets of synthesis information of the first electronic data, the coordinates of the first feature point, and the images around the coordinates of the first feature point For each distribution, autocorrelation information, a plurality of the first electronic data, a plurality of coordinates of the first feature points, and a coordinate of the first feature points are distributed for each of the distributions. The cross-correlation information is calculated from the surrounding image, and the coordinates of the second feature point and the second feature point of the second electronic data input from the image input unit from the auto-correlation information and the cross-correlation information. For estimating the image around the coordinates of An extraction matrix is calculated for each distribution, the first average vector calculated from the plurality of first electronic data obtained for each distribution, and the coordinates of the plurality of first feature points obtained for each distribution And a learning unit that outputs the second average vector calculated from the image around the coordinates of the first feature point and the feature extraction matrix obtained for each distribution, and the learning unit obtained for each distribution. A correlation information database for storing the first average vector, the second average vector, and the feature extraction matrix; the second electronic data; the first average vector; the second average vector; and the feature extraction. And a parameter estimation unit that estimates the coordinates of the second feature points for each distribution using a matrix.
[0047]
Thereby, even when the characteristics of the first electronic data are dispersed, the coordinates of the second feature point and the template can be estimated, so that the processing cost of the search using the template can be reduced.
[0048]
According to a seventeenth aspect of the present invention, in the parameter estimation device according to the sixteenth aspect, the learning unit calculates a first teaching vector that is a vector pattern in which the first electronic data is arranged in a scanning order. A second teaching vector calculation unit that calculates a second teaching vector that is a vector pattern in which the coordinates of the first feature point and the images around the coordinates of the first feature point are sequentially arranged; A plurality of synthesized vectors are created from a plurality of the first teaching vectors and a plurality of the second teaching vectors, and a plurality of the first teaching vectors and the second teaching vectors are determined by a probability distribution of the set of the synthesized vectors. And a first average vector obtained by averaging a plurality of the first teaching vectors for each distribution, and a plurality of the second teaching vectors for each distribution. An average vector calculation unit for calculating a second average vector obtained by averaging the data, and a plurality of the first teaching vectors and the first average vector for each of the distributions, and autocorrelation of the first teaching vector An autocorrelation information calculation unit that calculates a pseudo inverse matrix of the first covariance matrix, which is information, for each distribution, a plurality of the first teaching vectors, the first average vector, and a plurality of the second teachings A cross-correlation in which a vector and the second average vector are input for each distribution, and a second covariance matrix that is cross-correlation information between the first teaching vector and the second teaching vector is calculated for each distribution. For each distribution, the information calculation unit inputs the pseudo inverse matrix of the first covariance matrix and the second covariance matrix, and the pseudo inverse matrix of the first covariance matrix and the second covariance matrix. The feature extraction matrix from the variance matrix And a feature extraction matrix calculating section for calculating for each fabric.
[0049]
Thereby, it has the effect | action that the characteristic extraction matrix suitable for the characteristic of the distributed 1st electronic data can be calculated | required.
[0050]
According to an eighteenth aspect of the present invention, in the parameter estimation device according to the sixteenth aspect or the seventeenth aspect, the parameter estimation unit inputs the second electronic data, and the second electronic data is scanned in order. A vector calculation unit for calculating an input vector which is a vector pattern arranged; the input vector; a first average vector classified for each distribution; a second average vector classified for each distribution; and a feature extraction matrix classified for each distribution A parameter calculation unit that calculates an expected value vector including the coordinates of the second feature point and an image around the coordinates of the second feature point, and the coordinates of the second feature point and the second from the expected value vector An image around the coordinates of the feature point is calculated, a matching area that is an image area of the second electronic data that matches the image around the coordinates of the second feature point is searched, and the matching area is searched from the matching area. And a template matching unit for calculating a second feature coordinates.
[0051]
As a result, even if the characteristics of the second electronic data are dispersed, the area around the estimated feature point coordinates is searched with the template created from the image to be collated, so that the search processing cost can be reduced. .
[0052]
According to a nineteenth aspect of the present invention, an image input unit that converts optical data into electronic data, a parameter input unit that inputs coordinates of a first feature point from the first electronic data input from the image input unit, Calculating autocorrelation information from a plurality of the first electronic data, calculating cross-correlation information from a plurality of the first electronic data and a plurality of coordinates of the first feature points, and calculating the autocorrelation information and the A feature extraction matrix for estimating the coordinates of the second feature point of the second electronic data input from the image input unit from the cross-correlation information is calculated, and the first calculated from a plurality of the first electronic data. An average vector of 1, a second average vector calculated from the coordinates of the plurality of first feature points, and a learning unit that outputs the feature extraction matrix; the first average vector input from the learning unit; Second average vector And the correlation information database for storing the feature extraction matrix, the coordinates of the second feature point using the second electronic data, the first average vector, the second average vector, and the feature extraction matrix A parameter estimation unit for estimating the image, an image database for storing the first electronic data, a collation area that is an image used for collation from the coordinates of the second electronic data and the second feature points, A data collating apparatus including a collation area and a collation unit that collates a match between the first electronic data stored in the image database.
[0053]
Thereby, the collation area can be calculated from the coordinates of the feature points estimated by the matrix operation, and the processing cost can be reduced.
[0054]
According to a twentieth aspect of the present invention, there is provided an image input unit for converting optical data into electronic data, and a parameter for inputting an image around the coordinates of the first feature point from the first electronic data input from the image input unit. Autocorrelation information is calculated from the input unit and the plurality of first electronic data, and cross-correlation information is calculated from the plurality of first electronic data and images around the coordinates of the first feature points, Further, a feature extraction matrix for estimating an image around the coordinates of the second feature point of the second electronic data input from the image input unit is calculated from the autocorrelation information and the cross-correlation information, and a plurality of feature extraction matrices are calculated. A learning unit that outputs a first average vector calculated from the first electronic data, a second average vector calculated from images around the coordinates of the plurality of first feature points, and the feature extraction matrix; Input from the learning unit A correlation information database for storing the first average vector, the second average vector, and the feature extraction matrix; the second electronic data; the first average vector; the second average vector; and the feature. From the parameter estimation unit that estimates the coordinates of the second feature points using an extraction matrix, the image database that stores the first electronic data, the second electronic data, and the coordinates of the second feature points A data collating apparatus including a collation unit that calculates a collation area that is an image used for collation and collates the collation area with the first electronic data stored in the image database.
[0055]
Thereby, the collation region can be calculated from the coordinates of the feature points obtained from one template, and the processing cost can be reduced.
[0056]
According to a twenty-first aspect of the present invention, there is provided an image input unit that converts optical data into electronic data, the coordinates of the first feature points and the first feature points from the first electronic data input from the image input unit. A parameter input unit that inputs an image around the coordinates; and autocorrelation information is calculated from the plurality of first electronic data, the coordinates of the plurality of first electronic data and the plurality of first feature points, and a plurality of Cross-correlation information is calculated from a coupling vector with an image around the coordinates of the first feature point, and a second electronic data input from the image input unit is input from the auto-correlation information and the cross-correlation information. A feature extraction matrix for estimating an image around the coordinates of the feature points and the coordinates of the second feature points is calculated, a first average vector calculated from the plurality of first electronic data, and a plurality of the combined vectors 2nd average calculated from A learning unit that outputs a vector and the feature extraction matrix; a correlation information database that stores the first average vector, the second average vector, and the feature extraction matrix that are input from the learning unit; A parameter estimation unit that estimates the coordinates of the second feature point using the electronic data, the first average vector, the second average vector, and the feature extraction matrix; and an image that stores the first electronic data A collation area which is an image used for collation is calculated from the database, the second electronic data and the coordinates of the second feature point, and the collation area and the first electronic data stored in the image database A data collating apparatus including a collation unit for collating the match.
[0057]
As a result, by searching around the coordinates of the estimated feature points using a single template, the search processing cost can be reduced, and the matching area can be calculated from the coordinates of the feature points obtained by template matching, thereby reducing processing costs. Has the effect of being able to
[0058]
According to a twenty-second aspect of the present invention, an image input unit that converts optical data into electronic data, and a parameter input unit that inputs coordinates of a first feature point from the first electronic data input from the image input unit, , Dividing a set of combined information of the first electronic data and the coordinates of the first feature points into a plurality of distributions, and for each distribution, autocorrelation information and a plurality of the plurality of the first electronic data Cross-correlation information is calculated from the first electronic data and the coordinates of the plurality of first feature points, and further, second electronic data input from the image input unit is calculated from the auto-correlation information and the cross-correlation information. A feature extraction matrix for estimating the coordinates of the second feature point is calculated for each distribution, and a first average vector calculated from the plurality of first electronic data obtained for each distribution, for each distribution The plurality of first characteristics obtained in A learning unit that outputs a second average vector calculated from the coordinates and the feature extraction matrix obtained for each distribution, the first average vector input from the learning unit obtained for each distribution, The second feature using the correlation information database storing the two mean vectors and the feature extraction matrix, the second electronic data, the first mean vector, the second mean vector, and the feature extraction matrix. It is an image used for collation from the parameter estimation part which estimates the coordinate of a point for every said distribution, the image database which accumulate | stores said 1st electronic data, and the coordinate of said 2nd electronic data and said 2nd feature point A data collation apparatus including a collation unit that calculates a collation area and collates the collation area with the first electronic data stored in the image database.
[0059]
Thereby, even when the characteristics of the first electronic data are dispersed, the collation region can be calculated from the coordinates of the feature points estimated by the matrix calculation, and the processing cost can be reduced.
[0060]
  A twenty-third aspect of the present invention converts optical data into electronic data.Conversion step toThe aboveConverted in the conversion stepThe coordinates of the first feature point are input from the first electronic data, and autocorrelation information is calculated from the plurality of the first electronic data.Steps to doCalculating cross-correlation information from a plurality of the first electronic data and a plurality of coordinates of the first feature pointsSteps to do, From the autocorrelation information and the cross-correlation informationConverted in the conversion stepCalculate feature extraction matrix for estimating coordinates of second feature point of second electronic dataSteps to doOutputting a first average vector calculated from a plurality of the first electronic data, a second average vector calculated from the coordinates of the plurality of first feature points, and the feature extraction matrix.outputStep and saidOutput in the output stepThe first average vector, the second average vector, and the feature extraction matrix are stored in a correlation information database, and the second electronic data, the first average vector, the second average vector, and the feature extraction are stored. Estimate the coordinates of the second feature point using a matrixSteps to doThe first electronic data is stored in an image database, and a collation area, which is an image used for collation, is calculated from the second electronic data and the coordinates of the second feature points.Steps to do, Verifying a match between the verification area and the first electronic data stored in the image databaseA matching stepThis is a characteristic data matching method.
[0061]
Thereby, the collation area can be calculated from the coordinates of the feature points estimated by the matrix operation, and the processing cost can be reduced.
[0062]
  A twenty-fourth aspect of the present invention converts optical data into electronic data.Conversion step toThe aboveConverted in the conversion stepAn image around the coordinates of the first feature point is input from the first electronic data, and autocorrelation information is calculated from the plurality of the first electronic data.Steps to doCalculating cross-correlation information from images around the coordinates of the plurality of first electronic data and the plurality of first feature points, and further calculating the cross-correlation information from the auto-correlation information and the cross-correlation information.Converted in the conversion stepCalculate a feature extraction matrix for estimating an image around the coordinates of the second feature point of the second electronic dataSteps to doA first average vector calculated from a plurality of the first electronic data, a second average vector calculated from images around the coordinates of the plurality of first feature points, and the feature extraction matrixOutput step to,Output in the output stepThe first average vector, the second average vector, and the feature extraction matrix are stored in a correlation information database, and the second electronic data, the first average vector, the second average vector, and the feature extraction are stored. Estimate the coordinates of the second feature point using a matrixSteps to doThe first electronic data is stored in an image database, and a collation area, which is an image used for collation, is calculated from the second electronic data and the coordinates of the second feature points.Steps to do, Verifying a match between the verification area and the first electronic data stored in the image databaseA matching stepThis is a data collating method characterized by the above.
[0063]
As a result, by searching around the coordinates of the estimated feature points using a single template, the search processing cost can be reduced, and the matching area can be calculated from the coordinates of the feature points obtained by template matching, thereby reducing processing costs. Has the effect of being able to
[0064]
  According to a twenty-fifth aspect of the present invention, optical data is converted into electronic data.Conversion step toThe aboveConverted in the conversion stepInput the coordinates of the first feature point and the image around the coordinates of the first feature point from the first electronic data, and calculate autocorrelation information from the plurality of the first electronic dataSteps to doCalculating cross-correlation information from a plurality of the first electronic data and a combination vector of coordinates of the plurality of first feature points and images around the plurality of coordinates of the first feature points.Steps to doFrom the autocorrelation information and the cross-correlation informationConverted in the converting stepCalculate a feature extraction matrix for estimating the coordinates of the second feature point of the second electronic data and an image around the coordinates of the second feature pointSteps to doA first average vector calculated from a plurality of the first electronic data, a second average vector calculated from a plurality of the combined vectors, and the feature extraction matrixOutput step toThe aboveOutput in output stepStoring the first average vector, the second average vector, and the feature extraction matrix in a correlation information database, the second electronic data, the first average vector, the second average vector, and the Estimating the coordinates of the second feature point using a feature extraction matrixSteps to doThe first electronic data is stored in an image database, and a collation area, which is an image used for collation, is calculated from the second electronic data and the coordinates of the second feature points.Steps to do, Verifying a match between the verification area and the first electronic data stored in the image databaseAnd having stepsThis is a data collating method characterized by the above.
[0065]
As a result, by searching around the coordinates of the estimated feature points using a single template, the search processing cost can be reduced, and the matching area can be calculated from the coordinates of the feature points obtained by template matching, thereby reducing processing costs. Has the effect of being able to
[0066]
  According to a twenty-sixth aspect of the present invention, optical data is converted into electronic data.Conversion step toThe aboveConverted in the conversion stepThe coordinates of the first feature points are input from the first electronic data, and a set of combined information of the first electronic data and the coordinates of the first feature points is divided into a plurality of distributions. Cross-correlation information is calculated from autocorrelation information from a plurality of the first electronic data, a plurality of the first electronic data and coordinates of the plurality of first feature points, and further the autocorrelation information and the cross-correlation From informationConverted in the converting stepA feature extraction matrix for estimating the coordinates of the second feature point of the second electronic data is calculated for each distribution.Steps to doA first average vector calculated from the plurality of first electronic data obtained for each distribution, a second average vector calculated from the coordinates of the plurality of first feature points obtained for each distribution, and Output the feature extraction matrix obtained for each distributionOutput step to,It calculated | required for every said distribution output in the said output step.The first average vector, the second average vector, and the feature extraction matrix are stored in a correlation information database, and the second electronic data, the first average vector, the second average vector, and the feature extraction are stored. Estimate the coordinates of the second feature point for each distribution using a matrixSteps to doThe first electronic data is stored in an image database, and a collation area, which is an image used for collation, is calculated from the second electronic data and the coordinates of the second feature points.Steps to do, Verifying a match between the verification area and the first electronic data stored in the image databaseAnd having stepsThis is a characteristic data matching method.
[0067]
Thereby, even when the characteristics of the first electronic data are dispersed, the collation region can be calculated from the coordinates of the feature points estimated by the matrix calculation, and the processing cost can be reduced.
[0068]
  According to a twenty-seventh aspect of the present invention, a computer converts optical data into electronic data.Conversion step to,Converted by the converting stepInput the coordinates of the first feature point from the first electronic dataShiCalculating autocorrelation information from a plurality of the first electronic dataSteps to doCalculating cross-correlation information from a plurality of the first electronic data and a plurality of coordinates of the first feature pointsShiCalculating a feature extraction matrix for estimating the coordinates of the second feature point of the second electronic data input from the autocorrelation information and the cross-correlation informationSteps to doA first average vector calculated from the plurality of first electronic data, a second average vector calculated from the coordinates of the plurality of first feature points, and the feature extraction matrixOutput step toThe aboveOutput in the output stepAccumulating the first average vector, the second average vector, and the feature extraction matrix in a correlation information databaseShiThe coordinates of the second feature point are estimated using the second electronic data, the first average vector, the second average vector, and the feature extraction matrix.Steps to do, Storing the first electronic data in an image databaseShiThe collation area which is an image used for collation is calculated from the coordinates of the second electronic data and the second feature point.Steps to do, Stored in the collation area and the image databaseShiMatch with the first electronic dataAnd step to doIt is a program characterized by this.
[0069]
Thereby, the collation area can be calculated from the coordinates of the feature points estimated by the matrix operation, and the processing cost can be reduced.
[0070]
  According to a twenty-eighth aspect of the present invention, a computer converts optical data into electronic data.Conversion steps and,Converted in the converting stepInput an image around the coordinates of the first feature point from the first electronic dataShiCalculating autocorrelation information from a plurality of the first electronic dataSteps to doCalculating cross-correlation information from a plurality of the first electronic data and images around the coordinates of the plurality of first feature pointsShiFurther, from the autocorrelation information and the cross-correlation informationConverted in the converting stepCalculate a feature extraction matrix for estimating an image around the coordinates of the second feature point of the second electronic dataSteps to doA first average vector calculated from a plurality of the first electronic data, a second average vector calculated from images around the coordinates of the plurality of first feature points, and the feature extraction matrixOutput step toThe aboveOutput in the output stepAccumulating the first average vector, the second average vector, and the feature extraction matrix in a correlation information databaseShiThe coordinates of the second feature point are estimated using the second electronic data, the first average vector, the second average vector, and the feature extraction matrix.Steps to do, Storing the first electronic data in an image databaseShiThe collation area which is an image used for collation is calculated from the coordinates of the second electronic data and the second feature point.Steps to doMatching the matching area with the first electronic data stored in the image databaseAnd step to doIt is a program characterized by this.
[0071]
As a result, by searching around the coordinates of the estimated feature points using a single template, the search processing cost can be reduced, and the matching area can be calculated from the coordinates of the feature points obtained by template matching, thereby reducing processing costs. Has the effect of being able to
[0072]
  According to a twenty-ninth aspect of the present invention, a computer converts optical data into electronic data.Conversion steps and,Converted in the converting stepInput the coordinates of the first feature point and the image around the coordinates of the first feature point from the first electronic dataShiCalculating autocorrelation information from a plurality of the first electronic dataSteps to doCalculating cross-correlation information from a plurality of the first electronic data and a combination vector of coordinates of the plurality of first feature points and images around the plurality of coordinates of the first feature points.Steps to do, From the autocorrelation information and the cross-correlation informationConverted in the conversion stepCalculate a feature extraction matrix for estimating the coordinates of the second feature point of the second electronic data and an image around the coordinates of the second feature pointSteps to doA first average vector calculated from a plurality of the first electronic data, a second average vector calculated from a plurality of the combined vectors, and the feature extraction matrixOutput step toThe aboveOutput in the output stepAccumulating the first average vector, the second average vector, and the feature extraction matrix in a correlation information databaseShiThe coordinates of the second feature point are estimated using the second electronic data, the first average vector, the second average vector, and the feature extraction matrix.Steps to do, Storing the first electronic data in an image databaseShiThe collation area which is an image used for collation is calculated from the coordinates of the second electronic data and the second feature point.Steps to doMatching the matching area with the first electronic data stored in the image databaseAnd step to doIt is a program characterized by this.
[0073]
As a result, by searching around the coordinates of the estimated feature points using a single template, the search processing cost can be reduced, and the matching area can be calculated from the coordinates of the feature points obtained by template matching, thereby reducing processing costs. Has the effect of being able to
[0074]
  According to a thirtieth aspect of the present invention, a computer converts optical data into electronic data.Conversion steps andThe aboveConverted in the conversion stepInput the coordinates of the first feature point from the first electronic dataShi, Dividing a set of combined information of the first electronic data and the coordinates of the first feature points into a plurality of distributions, and for each distribution, autocorrelation information and a plurality of the plurality of the first electronic data Cross-correlation information is calculated from the first electronic data and the coordinates of the plurality of first feature pointsShiFurther, from the autocorrelation information and the cross-correlation informationConverted in the converting stepA feature extraction matrix for estimating the coordinates of the second feature point of the second electronic data is calculated for each distribution.Steps to doA first average vector calculated from the plurality of first electronic data obtained for each distribution, a second average vector calculated from the coordinates of the plurality of first feature points obtained for each distribution, and Output the feature extraction matrix obtained for each distributionOutput step toThe aboveObtained for each distribution output in the output stepAccumulating the first average vector, the second average vector, and the feature extraction matrix in a correlation information databaseShiThe coordinates of the second feature point are estimated for each distribution using the second electronic data, the first average vector, the second average vector, and the feature extraction matrix.Steps to do, Storing the first electronic data in an image databaseShiThe collation area which is an image used for collation is calculated from the coordinates of the second electronic data and the second feature point.Steps to doVerifying a match between the verification region and the first electronic data stored in the image database;,It is a program characterized by causing
[0075]
  Thereby, even when the characteristics of the first electronic data are dispersed, the collation region can be calculated from the coordinates of the feature points estimated by the matrix calculation, and the processing cost can be reduced.
  According to a thirty-first aspect of the present invention, sample face image input means for inputting a sample face image as a sample, feature point input means for inputting feature points of the sample face image, and collation for newly estimating the feature points Collation face image input means for inputting a target face image, synthesis information of the sample face image and the feature point is divided into a plurality of face image distributions, face image autocorrelation information of the sample face image, and the sample face image And face image cross-correlation information between the feature image and the facial image learning means for each facial image distribution, and the facial image auto-correlation information and the facial image cross-correlation information obtained for each facial image distribution. A face image parameter estimating means for estimating the feature point of the collation target face image, an image database storing the sample face image, coordinates of the collation target face image and the feature point of the collation target face image Calculating a matching region that is an image to be used for al matching is data matching apparatus comprising, a collating means for collating a match with the stored sample face images in the image database and the verification area.
  In a thirty-second aspect of the present invention, the face image learning means is configured to combine a face image of a first teaching vector obtained by vectorizing the sample face image and a second teaching vector obtained by vectorizing feature points of the sample face image. A face image mixture distribution model that comprises a plurality of face image element distributions of the face image synthesis vector is constructed, and the face image autocorrelation information belonging to each face image element distribution is determined as the first teaching vector. The face image cross-correlation information is obtained by obtaining a covariance matrix of the first teaching vector and the second teaching vector, and the face image parameter estimating means The feature point of the face image to be collated is estimated using the face image autocorrelation information and the face image cross-correlation information belonging to each face image element distribution. 1 is a data verification apparatus according.
[0076]
(Embodiment 1)
In the first embodiment of the present invention, a case where the parameter estimation device is applied to a data collating device will be described.
[0077]
FIG. 1 is a diagram showing a data collating apparatus according to Embodiment 1 of the present invention.
[0078]
The image input unit 101 converts optical data into electronic data and outputs it.
[0079]
The parameter input unit 102 inputs the feature point coordinates from the first electronic data input from the image input unit 101.
[0080]
The learning unit 103 calculates information regarding the first electronic data from the first electronic data input from the image input unit 101 and the coordinates of the feature points input from the parameter input unit 102.
[0081]
The correlation information database 104 stores information on the first electronic data calculated by the learning unit 103.
[0082]
The parameter estimation unit 105 estimates the coordinates of feature points in the second electronic data from the information about the second electronic data input from the image input unit 101 and the first electronic data stored in the correlation information database 104.
[0083]
The image database 106 stores first electronic data.
[0084]
The collation unit 107 collates the first electronic data that matches the coordinates of the feature points in the second electronic data estimated by the parameter estimation unit 105.
[0085]
The output unit 108 displays the collation result collated by the collation unit 107.
[0086]
The data collating apparatus according to the first embodiment will be described below. In the face image collation, it is important to accurately obtain the position where the face exists from the electronic data input from the image input unit 101.
[0087]
Therefore, in the first embodiment, attention is paid to the fact that there is a correlation between the face image that is input electronic data and the coordinates of feature points such as eyes, nose, eyebrows, and mouth that are unique parameters in the face image. It is what. In other words, if the correlation between the coordinates of the unique parameters of the face image is obtained in advance, it is possible to obtain the feature point coordinates that are unique parameters even if an unknown face image is input. It is what.
[0088]
That is, in the first embodiment, the correlation between the face image that is the sample and the coordinates of the feature points in the sample face image is learned in advance, and the image input unit 101 uses the previously learned correlation. The coordinate of the feature point in the face image for collation which is the second electronic data is estimated, and the face area which is the area used for the collation is obtained using the coordinate of the estimated feature point. In the first embodiment, the face image is compared by comparing the obtained face area image with an image in the face image database 106 prepared in advance.
[0089]
Further, in the first embodiment, by estimating the correlation between the face images that are a plurality of samples and the coordinates of the feature points in the plurality of face images, the coordinates of the feature points in the matching face image are estimated. The accuracy has been improved.
[0090]
Specifically, the processing of the data collating apparatus according to the first embodiment is roughly divided into offline processing and online processing.
[0091]
In the offline processing, the correlation between the face images of a plurality of samples and the coordinates of the feature points of the face image is calculated in advance.
[0092]
The online processing uses the correlation calculated in the offline processing to estimate the feature point coordinates from the matching face image input from the image input unit 101, and the face region image determined from the feature point coordinates in advance Collation processing with the face image of the image database 106 registered by offline processing is performed.
[0093]
Next, offline processing will be described. The offline processing is processing using the image input unit 101, the learning unit 103, the correlation information database 104, and the image database 106.
[0094]
Next, offline processing performed by the data collating apparatus will be described in detail with reference to FIGS. FIG. 2 is a diagram illustrating a configuration of the learning unit 103. FIG. 3 is an operation flowchart of offline processing of the data collating apparatus according to the first embodiment.
[0095]
First, the configuration of the learning unit 103 will be described.
[0096]
The learning unit 103 is provided with a second teaching vector calculation unit 201 that calculates and outputs a second teaching vector obtained from the coordinates of the feature points in the face image that is the first electronic data.
[0097]
Further, the learning unit 103 is provided with a first teaching vector calculation unit that calculates and outputs a first teaching vector obtained from a face image that is first electronic data.
[0098]
The average vector calculation unit 203a calculates the average vector of the second teaching vector obtained by the second teaching vector calculation unit 201. The average vector calculation unit 203b calculates the average vector of the first teaching vectors obtained by the first teaching vector calculation unit 202.
[0099]
The cross-correlation information calculation unit 204 is a covariance matrix that is cross-correlation information between the first teaching vector calculated by the first teaching vector calculation unit 202 and the second teaching vector calculated by the second teaching vector calculation unit 201. Is calculated.
[0100]
The autocorrelation information calculation unit 205 calculates a pseudo inverse matrix of a covariance matrix that is autocorrelation information of the first teaching vector obtained by the first teaching vector calculation unit 202.
[0101]
The feature extraction matrix calculator 206 calculates a feature extraction matrix from the pseudo inverse matrix of the covariance matrix of the autocorrelation information calculated by the autocorrelation information calculator 205 and the covariance matrix of the cross correlation information calculated by the cross correlation information calculator 204. Is calculated.
[0102]
Next, the offline processing operation of the data collating apparatus will be described with reference to FIG.
[0103]
First, the image input unit 101 inputs a face image as a sample which is first electronic data, and inputs the sample face image to the parameter input unit 102, the learning unit 103, and the image database 106 (S10).
[0104]
However, electronic data different from the first electronic data can be input to the image database 106.
[0105]
Next, the first teaching vector calculation unit 202 of the learning unit 103 converts the value of each pixel in the input sample face image into a vector pattern arranged in raster scan order using (Equation 1). Teach vector V₁(S11) and output to the average vector calculation unit 203b.
[0106]
[Expression 1]

Next, a display included in the parameter input unit 102 displays a sample face image. When the user selects the coordinates of the facial feature points from the face image using the mouse included in the parameter input unit 102, the parameter input unit 102 learns the facial feature point coordinates selected by the user from the learning unit. It outputs to 103 (S12).
[0107]
Here, the coordinates of the feature points of the input face are shown in FIG. Based on the origin O1208 of the face image 1201, the coordinates (X coordinate, Y coordinate) of the right eyebrow 1202, the right eye 1203, the left eyebrow 1204, the left eye 1205, the nose 1206, and the mouth 1207 are coordinated by the user as feature points. Selected.
[0108]
Next, the second teaching vector calculation unit 201 of the learning unit 103 uses (Equation 1) as the coordinates of the input feature points, and sets the coordinates of the input feature points as a data matching device. For example, the right eyebrow, the right eye, the left eyebrow, the left eye, the nose, and the mouth are arranged in order to create one vector data, which is used as the teaching vector V₂(S13) and output to the average vector calculation unit 203a.
[0109]
Next, the parameter input unit 102, the second teaching vector calculation unit 201 of the learning unit 103, and the first teaching vector calculation unit 202 of the learning unit 103 perform the processes from S10 to S13 for the necessary N pieces. If the process is repeated, the process proceeds to (S16). If the process is not completed, S11 to S14 are repeated (S15).
[0110]
Next, the average vector calculation units 203a and 203b perform N teaching vectors V₁, V₂Set {V₁}, {V₂} Using (Equation 2) and (Equation 3), the teaching vector V₁Mean vector M which is the mean vector of₁And teaching vector V₂Mean vector M which is the mean vector of₂Is calculated (S16).
[0111]
The average vector calculation unit 203b then calculates the average vector M₁Are output to the autocorrelation information calculation unit 205 and the cross correlation information calculation unit 204, and the average vector calculation unit 203a₂Is output to the cross-correlation information calculation unit 204.
[0112]
[Expression 2]

[Equation 3]

Next, the autocorrelation information calculation unit 205 generates N teaching vectors V.₁Set {V₁} And mean vector M₁Is substituted into (Equation 4) and the teaching vector V₁The covariance matrix C₁To calculate the distribution of the N face images input from the image input unit 101. In addition, the autocorrelation information calculation unit 205 generates a covariance matrix C₁, Pseudo inverse matrix C₁ ^*And is output to the feature extraction matrix calculation unit 206.
[0113]
[Expression 4]

Further, the cross-correlation information calculation unit 204 includes N teaching vectors V₁And teaching vector V₂, Mean vector M₁And mean vector M₂Is substituted into (Equation 5) and the teaching vector V₁And V₂The covariance matrix C₂Is calculated, and the correlation between the N face images and the feature point coordinates in the face image is calculated and output to the feature extraction matrix calculation unit 206 (S17).
[0114]
[Equation 5]

Next, the feature extraction matrix calculation unit 206 calculates the pseudo inverse matrix C obtained in S17.₁ ^*And covariance matrix C₂And a feature extraction matrix C which is a parameter for extracting the coordinates of the feature points from the matching face image input by online processing_ThreeIs calculated according to (Equation 6) (S18).
[0115]
[Formula 6]

Then, the feature extraction matrix calculation unit 206 calculates the obtained average vector M₁, M₂And feature extraction matrix C_ThreeIs output to the correlation information database 104 (S19). In this way, the offline process ends.
[0116]
Next, online processing will be described with reference to FIGS. FIG. 5 is a diagram illustrating a configuration of the parameter estimation unit 105. FIG. 6 is an operation flowchart of online processing of the data collating apparatus according to the first embodiment.
[0117]
The parameter estimation unit 105 includes a vector calculation unit 301 that calculates and outputs an input vector X from the second electronic data.
[0118]
The parameter calculation unit 302 also includes an input vector X and an average vector M calculated by the parameter estimation unit 105.₁, M₂And feature extraction matrix C_ThreeThe expected value vector including the coordinates of the feature points is calculated from and output.
[0119]
Further, the template matching unit 303 calculates the coordinates of the feature points from the expected value vector calculated by the parameter calculation unit 302 and outputs it.
[0120]
The purpose of online processing is to estimate the feature point coordinates, which are parameters specific to the input, from the input face image, and obtain the face region that is the face region image used for matching from the estimated feature point coordinates. In other words, the obtained face area is collated with an image registered in the image database 106.
[0121]
First, the image input unit 101 inputs a collation image that is second electronic data (S20), and outputs it to the vector calculation unit 301 of the parameter estimation unit 105. Next, the vector calculation unit 301 arranges the values of the pixels in the second electronic data in the raster scan order, converts them into input vector data X (S21), and outputs the input vector data X to the parameter calculation unit 302.
[0122]
Next, the parameter calculation unit 302 inputs the input vector data X and the feature extraction matrix C obtained by offline processing._Three, Mean vector M₁And mean vector M₂Is substituted into (Equation 7), and an expected value vector E including the coordinates of the feature point with respect to the input vector data X is calculated (S22) and output to the template matching unit 303.
[0123]
[Expression 7]

Here, (Equation 7) is an expression for calculating the coordinates of the feature points of the matching image using the cross-correlation between the N face images and the coordinates of the feature points of the face image obtained by the offline processing.
[0124]
Here, the calculation cost of the present invention is calculated in the same example as the calculation cost calculation by template matching. A case where the input image size is 150 × 150 pixels × 22,500 pixels, and the coordinates of both eyes (right eye X coordinate, right eye Y coordinate, left eye X coordinate, left eye Y coordinate) are estimated for a total of four dimensions with respect to the input. Suppose. In this case, when a vector obtained by converting a 150 × 150 input image into a vertical column is substituted into (Equation 7) and multiplication is used as a unit operation, C_ThreeIs a matrix of 22,500 vertical x 4 horizontal, (X-M₁) Is a 22500-dimensional vector, the number of multiplications is 4 × 22500 = 90000 (pel). In other words, the multiplication alone results in a calculation cost that is 1/500 of the template match, and the effect is great.
[0125]
Here, the reason why the coordinates of the feature point can be estimated by calculation using (Equation 7) will be described.
[0126]
The expected value vector E indicating the coordinates of the feature points obtained by (Equation 7) is the teaching vector V₁And teaching vector V₂Is the same as the expected value of the output obtained for the input vector data X when learning using Bayesian estimation under the assumption that the distribution of the two vectors is a normal distribution.
[0127]
Bayesian estimation is a statistical estimation method in which a parameter distribution and an appropriate loss function are defined and estimation is performed so that the expected value of the loss function is minimized. That is, it can be said that the most likely output value for the input vector data X can be estimated by (Equation 7).
[0128]
Next, the template matching unit 303 calculates the coordinates of the feature points of the input vector data from the expected value vector E, which is a combined vector of the coordinates of the feature points (S23), and outputs the feature point coordinates to the matching unit 107. .
[0129]
Next, the collation unit 107 determines a face area that is an image used for collation with the face image registered in the image database 106 by offline processing using the coordinates of the feature points obtained in S23 (S24).
[0130]
Here, an example of the detected face area for comparison is shown in FIG. In the present embodiment, a square region whose center is the nose coordinate 1303 and whose length of one side is twice the distance a between both eyes and whose upper and lower sides are parallel to a straight line connecting both eyes is collated. It is determined as a collation area 1302 which is an area used for.
[0131]
Next, the collation unit 107 collates an image of the face region 1302 and an image of the image database 106 registered in advance by offline processing, for example, an eigenface method using a principal component analysis that is a statistical method. (S25), and the result is output to the output unit 108 (S26).
[0132]
The eigenface method normalizes the size of the reference image, sets the gray value of all the pixels as an N-dimensional vector, compresses the information amount as F-dimensional information where 0 <F <N from all the reference images, The face subspace is generated by a statistical method called principal component analysis. Then, a region having a high face existence probability is normalized from the input image, the orthogonal distance from the face partial space is set as the similarity, and the person is recognized by the position of the projection destination on the face partial space.
[0133]
An output example of the output unit 108 is shown in FIG. On the display 1501, an input face image 1502 that has been input and matching results 1504 and 1505 are displayed.
[0134]
As described above, according to the first embodiment, using the correlation between the sample face image, which is the first electronic data previously learned by offline processing, and the coordinates of the feature points in the sample face image, the image input unit The coordinates of the feature points in the collation face image that is the second electronic data photographed in 101 can be estimated.
[0135]
Then, a face area for collation is obtained from the coordinates of the estimated feature points as described above, and the face area image for collation obtained is compared with the face image in the image database 106 registered in advance by offline processing and collated. be able to.
[0136]
In addition, a unique parameter for the input vector can be estimated by three matrix operations. In other words, since it can be estimated at a much lower calculation cost than a search using conventional template matching, a desired parameter can be estimated with much fewer calculations than with the conventional method, and face images can be matched with fewer calculations.
[0137]
According to the first embodiment, the autocorrelation information is obtained from the covariance matrix of the first teaching vector, the cross correlation information is obtained from the covariance matrix of the first teaching vector and the second teaching vector, A feature extraction matrix is obtained from the correlation information and the cross-correlation information, and the coordinates of the feature points of the face image for matching are obtained from the average vector of the first teaching vector, the average vector of the second teaching vector, the feature extraction matrix and the input vector data. Can be calculated. In this way, autocorrelation information, cross-correlation information, feature extraction matrix, average vector of the first teaching vector, average vector of the second teaching vector, and coordinates of feature points of the face image for matching are simply calculated by matrix calculation. In other words, since the calculation can be performed only by multiplication, the processing cost of the computer becomes very small.
[0138]
In the first embodiment, the parameters are estimated using the above-described (Equation 1) to (Equation 7), but the correlation between the sample face image and the coordinates of the feature points in the sample face image is obtained in advance. The same effect can be obtained even if an expression other than (Expression 1) to (Expression 7) is used as long as it is an expression for estimating a parameter using the obtained correlation.
[0139]
It should be noted that it is possible to estimate the coordinates of feature points other than the first electronic data which is sample image data.
[0140]
(Embodiment 2)
The second embodiment of the present invention is different from the first embodiment in the teaching vector V.₂In the second embodiment, the teaching vector V is obtained from the area image around the coordinates of the facial feature points.₂And a template suitable for the face image for verification is automatically generated from the image for verification.
[0141]
As a result, matching processing with a single template becomes possible, and the calculation cost can be reduced at each stage as compared with the case of using a plurality of templates as in the prior art.
[0142]
Specifically, in the second embodiment, feature points such as eyes, nose, mouth, and eyebrows in the sample face image and the sample face image, which are first electronic data previously captured by the image input unit 101, are described. The correlation with the region image that is an image around the coordinates is examined. Next, in the second embodiment, a specific region image used for matching processing is obtained as a template from the face image for matching, which is the second electronic data photographed by the image input unit 101, using this correlation.
[0143]
Next, the second embodiment estimates the coordinates of the feature points of the face image for collation by matching the obtained template with the face image for collation, and obtains a face area image that is an area used for collation. .
[0144]
In the second embodiment, the face image as the second electronic data is collated by comparing the face area image with an image in the image database 106 prepared in advance.
[0145]
Hereinafter, the data collating apparatus according to the second embodiment will be described in detail.
[0146]
Moreover, since the figure which shows the structure of the data collation apparatus concerning Embodiment 2 is the same as that of the data collation apparatus of Embodiment 1, description is abbreviate | omitted.
[0147]
Further, the data collating apparatus according to the second embodiment is roughly divided into offline processing and online processing, as in the first embodiment. Here, in particular, differences from the first embodiment will be described with reference to FIGS.
[0148]
FIG. 9 is a diagram illustrating an operation flow of offline processing according to the second embodiment, and FIG. 10 is a diagram illustrating a region image that is an image around the coordinates of feature points from a sample face image that is first electronic data. FIG. 11 is a diagram showing an operation flow of online processing.
[0149]
In the offline processing of the second embodiment, the processing from S90 to S92 is the processing from S10 to S12 shown in FIG. 3, and the processing from S94 to S98 is the same as the processing from S15 to S19 shown in FIG. The description here is omitted.
[0150]
Next, as shown in FIG. 10, the second teaching vector calculation unit 201 of the learning unit 103 selects a region image that is an image around the feature point coordinates, as shown in FIG.
[0151]
With reference to the origin O1402 of the face image 1401, the coordinates of the right eyebrow 1403a, right eye 1404a, left eyebrow 1405a, left eye 1406a, nose 1407a and mouth 1408a are used as the feature point coordinates. Input to the teaching vector calculation unit 201.
[0152]
The second teaching vector calculation unit 201 also outputs a right eyebrow region image 1403b, a right eye region image 1404b, and a left eyebrow region image, which are template images of the coordinates of the feature points of the face from the input feature point coordinates 1403a to 1408a. 1405b, a left eye region image 1406b, a nose region image 1407b, and a mouth region image 1408b are created.
[0153]
As a determination method when each region image is rectangular, the width and height of each region are determined based on the obtained distance b between the eyes in FIG. 7, and the region is set around the coordinates of the feature points. To do. The width of the eyebrows is a / 2, the height is a / 4, the width of the eye is a / 2, the height is a / 4, the width of the nose is a × 2/3, the height Is a × 2/3, the width of the mouth is a, and the height is a / 2.
[0154]
Thus, by using the distance a between the eyes as a reference, it is possible to determine each region image regardless of the size of the face displayed in the face image of FIG.
[0155]
Next, the second teaching vector calculation unit 201 rearranges the values of the pixels in the template images 1403b to 1408b of the coordinates of the feature points of the created face in the raster scan order using (Equation 1), and vector data A synthesized vector is created and this is used as a teaching vector V₂(S93).
[0156]
However, in S97, the obtained feature extraction matrix C_ThreeIs the teaching vector V₂This is a parameter for extracting a region image.
[0157]
Next, online processing according to the second embodiment will be described with reference to FIG. Since the processing from S110 to S112 is the same as the processing from S20 to S22 in FIG. 6, a description thereof is omitted here.
[0158]
The expected value vector E created in S112 is the teaching vector V obtained in the offline process S95.₁, V₂Mean vector M₁, M₂And the feature extraction matrix C obtained in S97_ThreeAnd the combined vector of the template image of the coordinates of each feature point in the face image for matching, which is the second electronic data using the vector X obtained in S111 of the online processing. Therefore, the parameter calculation unit 302 obtains a template image of the coordinates of each feature point of the face from the expected value vector E (S113).
[0159]
Next, the template matching unit 303 performs template matching between the template image obtained in S113 and the matching image input from the image input unit 101 obtained in S110, and detects a region matching the template image. From the detected area, the coordinates of each feature point of the face are determined (S114).
[0160]
Note that the template matching process is a process of selecting a region having the highest correlation with the template image when the input window 2001 is scanned with the search window 2002 and an image in the search window is input as shown in FIG. is there. Correlation calculation used for template matching uses, for example, SAD correlation calculation (Equation 8), normalized correlation calculation (Equation 9), or the like. In (Equation 8) and (Equation 9), X is an image in the search window, and T is a template image.
[0161]
[Equation 8]

[Equation 9]

Since the subsequent processing from S115 to S117 is the same as the processing from S24 to S26 in FIG. 6, the description thereof is omitted here.
[0162]
According to the second embodiment, the correlation between the sample image and the specific area image of the sample image is learned, the specific area image of the matching face image is estimated using this correlation, and the estimated matching face is estimated. A specific area image of the image can be used as a template image. As a result, a template image can be automatically generated from the matching face image itself, so matching processing with a single template becomes possible, and compared to the case of using a plurality of templates such as a multi-template as in the past, It is possible to reduce the calculation cost.
[0163]
Further, according to the second embodiment, the feature extraction matrix obtained from the autocorrelation information and the cross-correlation information of the first teaching vector and the second teaching vector, the average vector of the first teaching vector, the second teaching vector The coordinate of the feature point of the face image for collation can be calculated from the average vector and the vector data of the face image for collation. In this way, the unique parameters can be calculated simply by matrix operation, that is, by multiplication, so that the processing cost of the computer becomes very small.
[0164]
Further, when the second embodiment of the present invention is used for template image estimation in template matching, a specific image assumed from an input image is estimated, and matching processing using this as a template becomes possible. In other words, since processing can be performed with one template pattern, as compared with the multi-template method that prepares a plurality of templates with different patterns and sizes corresponding to the diversity of input images, as in conventional template matching, Calculation cost can be reduced.
[0165]
(Embodiment 3)
In the third embodiment of the present invention, the teaching vector V₂In the first embodiment, the coordinates of the facial feature points are obtained from the coordinates of the facial feature points. In the second embodiment, the coordinates of the facial feature points and the facial feature points are obtained. A teaching vector V from a combined vector of region images around the coordinates₂Is to ask for.
[0166]
The teaching vector V thus obtained is₂The coordinates of the feature point of the face image for matching are estimated from the template, and the template suitable for the face image for matching is automatically generated from the image for matching, and the peripheral area of the coordinates of the estimated feature point is automatically generated. Is used for matching.
[0167]
Thus, the template search range is limited, and matching processing with a single template becomes possible. As a result, the calculation cost can be reduced at each stage as compared with the conventional method of searching the entire region of the face image for verification or using a plurality of templates.
[0168]
Hereinafter, the data collating apparatus according to the third embodiment will be described in detail.
[0169]
Moreover, since the figure which shows the structure of the data collation apparatus concerning Embodiment 3 is the same as that of the data collation apparatus of Embodiment 1, description is abbreviate | omitted.
[0170]
Further, the data collating apparatus according to the third embodiment is roughly divided into offline processing and online processing as in the first embodiment. Here, in particular, differences from the first embodiment and the second embodiment will be described with reference to FIGS.
[0171]
FIG. 12 is a diagram illustrating an operation flow of offline processing according to the third embodiment, and FIG. 13 is a diagram illustrating an operation flow of online processing.
[0172]
In the offline processing, the processing from S1200 to S1202 is the same as the processing from S10 to S12 shown in FIG. 3, and the processing from S1205 to S1209 is the same as the processing from S15 to S19 shown in FIG. The description in is omitted.
[0173]
As shown in FIG. 10, the parameter input unit 102 selects a region image that is an image around the coordinates of the feature point from the coordinates of the feature point selected in S1202 (S1203).
[0174]
Next, the learning unit 103 uses (Equation 1) to arrange the coordinate of each feature point of the face in order and combine them into one vector data, and a template image of the coordinates of each feature point of the created face A combined vector is generated by combining vector data obtained by arranging the values of the pixels constituting 1403b to 1408b in the raster scan order, and this is used as the teaching vector V.₂(S1204).
[0175]
However, the feature extraction matrix C obtained in S1208_ThreeIs the teaching vector V₂Since it is created using, the coordinates of the feature points and the parameters for extracting the region image are used.
[0176]
Next, online processing will be described with reference to FIG. The processing from S1300 to S1303 is the same as the processing from S20 to S23 in FIG. 6, and the processing from S1306 to S1308 is the same as the processing from S24 to S26 in FIG.
[0177]
The expected value vector E created in S1302 is the teaching vector V obtained in the offline process S16.₁, V₂Mean vector M₁, M₂And a feature extraction matrix C for extracting the feature point coordinates and area image obtained in S1208._ThreeAnd the combined vector of the template image of the coordinates of each feature point in the face image for matching, which is the second electronic data using the vector X obtained in S1301 of the online processing. For this reason, the parameter calculation unit 302 obtains the coordinates of the feature points in the face image for collation from the expected value vector E in S1303, and obtains a template image that is a region image in S1304.
[0178]
Next, the template matching unit 303 performs template matching between the template image obtained in S1304 and the image for matching input from the image input unit 101 obtained in S1300 around the coordinates of the feature points obtained in S1303. A region matching the template image is detected. Then, the template matching unit 303 determines the coordinates of each feature point of the face from the detected area (S1305).
[0179]
As in the first embodiment, the correlation calculation used for template matching uses SAD correlation calculation (Formula 8), normalized correlation calculation (Formula 9), and the like.
[0180]
As described above, according to the third embodiment, the coordinate vector of each feature point in the face image for matching and the combined vector of the area image of the coordinate of each feature point are estimated, and each estimated face The search range can be limited around the coordinates of the feature points.
[0181]
Furthermore, it is possible to perform a hierarchical search such that a template matching search is performed on the limited search range using a region image of the estimated coordinates of each feature point as a template.
[0182]
As described above, the hierarchical search improves the detection accuracy, the search range can be reduced, and the template to be used is also estimated, so that detection can be performed with a lower calculation cost than in the past.
[0183]
Further, according to the third embodiment, the autocorrelation information is obtained by obtaining the covariance matrix of the first teaching vector, and the cross correlation is obtained by obtaining the covariance matrix of the first teaching vector and the second teaching vector. You are asked for information. As a result, autocorrelation information and cross-correlation information can be obtained simply by matrix operation, that is, by multiplication, and the processing cost is greatly reduced.
[0184]
(Embodiment 4)
Embodiment 4 of the present invention is a vehicle image collation device in which a parameter estimation device is applied to vehicle image collation.
[0185]
The vehicle image collation device detects the coordinates of feature points such as license plates, lights, and driver's seats from Bayesian estimation from an image such as a light and shade pattern including a vehicle on a road photographed by the image input unit 101. The vehicle image collation device collates whether the image of the entire vehicle and the features such as the license plate number obtained based on the coordinates of the obtained feature points match those of the vehicle registered in advance. .
[0186]
Moreover, since the figure which shows the structure of a vehicle image collation apparatus is the same as that of FIG. 1 by Embodiment 1, description here is abbreviate | omitted.
[0187]
The processing of the vehicle image matching device can be broadly divided into offline processing and online processing. In the following, portions different from the first embodiment will be described with reference to FIGS.
[0188]
FIG. 14 is a diagram illustrating an operation flow of offline processing according to the fourth embodiment. The processes in S1401 and S1405 to S1408 in FIG. 14 are the same as those in the first embodiment, and are the same as the processes in S11 and S16 to S19 in FIG.
[0189]
First, the image input unit 101 inputs N vehicle images to be collated as first electronic data to be input to the parameter input unit 102, the learning unit 103, and the image database (S1400).
[0190]
Next, the first teaching vector calculation unit 202 of the learning unit 103 performs the teaching vector V₁Is calculated (S1401).
[0191]
Next, a display including the coordinates of the feature points of the vehicle image in the parameter input unit 102 displays the coordinates of the feature points of the vehicle image one by one. Then, when the user manually selects the coordinates of the feature point of the vehicle image from the displayed vehicle image using the mouse included in the parameter input unit 102, the parameter input unit 102 learns the coordinate of the selected feature point. The data is output to the unit 103 (S1402).
[0192]
Here, FIG. 15 shows the coordinates of the feature points of the vehicle image. As the coordinates of the feature points in the vehicle on the input image 1701, the front part 1708 and the coordinates A1702 to coordinates F1707 of each vertex of the vehicle are input.
[0193]
Next, the second teaching vector calculation unit 201 of the learning unit 103 calculates the teaching vector V from the input feature point coordinates.₂Is calculated (S1403).
[0194]
Next, the parameter input unit 102, the second teaching vector calculation unit 201 of the learning unit 103, and the first teaching vector calculation unit 202 of the learning unit 103 determine whether the processing for N units has been completed (S1404). If completed, the process proceeds to S1405. If not completed, the process proceeds to S1400.
[0195]
Thereafter, the processing from S1405 to S1408 is the same as the processing from S16 to S19 shown in FIG. 3, and is the same as that of the first embodiment, and thus the description thereof is omitted here.
[0196]
Next, online processing will be described using the operation flow shown in FIG.
[0197]
The purpose of the online processing is to estimate the coordinates of the feature points to be obtained from the input vehicle image, and to analyze and store the vehicle image, the license plate, and the image of the front part using the coordinates of the feature points.
[0198]
The processes from S1900 to S1903 and S1907 in FIG. 16 are the same as the processes from S20 to S23 and S26 in FIG. 6 and are the same as those in the first embodiment, and thus description thereof is omitted here.
[0199]
The collating unit 107 determines a reference interval from the coordinates of the feature points obtained in S1903, and cuts out the vehicle image, the license plate region image, and the front region image based on this (S1904).
[0200]
Next, the collation unit 107 reads the license plate number of the vehicle by applying the license plate area image to a character reader or the like (S1905).
[0201]
Further, the collation unit 107 collates the vehicle image and the license plate with the image database 106 registered in advance by offline processing using a collation method such as the eigenface method (S1906).
[0202]
FIG. 17 shows a display example of the input image 1802, the vehicle image 1804, the license plate area image and the reading result 1805, the matching result 1806, and the front area image 1807.
[0203]
As described above, according to the fourth embodiment, the input vehicle image and the coordinates of the estimated feature points are illustrated, and the image of the entire vehicle, the license plate image and the license plate based on the feature point coordinates. It is possible to easily grasp the vehicle information and the driver's situation by reading the characters of the part, displaying the collation result with the vehicle registered in advance, and displaying enlarged images of the driver seat and passenger seat images.
[0204]
In the fourth embodiment, the parameter estimation method of the first embodiment is used. However, the second embodiment or the third embodiment may be applied to a vehicle image matching device.
[0205]
(Embodiment 5)
The fifth embodiment of the present invention corresponds to a collation input image having various distributions when there is variation in the distribution of input vectors that are sample input images.
[0206]
Specifically, the composite information of the sample input image and the feature point coordinates is divided into a plurality of distributions, and the correlation between the matching image and the feature point coordinates of the matching image is examined for each distribution. The coordinates of the feature points of the verification image are obtained using these correlations.
[0207]
Further, the data collating apparatus according to the fifth embodiment is roughly divided into offline processing and online processing, as in the first embodiment. Here, in particular, differences from the first embodiment will be described with reference to FIGS.
[0208]
Further, since the diagram showing the configuration of the data collating apparatus is the same as that of the data collating apparatus according to the first embodiment, description thereof is omitted here. However, the learning unit 103 has the configuration shown in FIG. Here, what is indicated by 401 is a distribution element calculation unit for classifying a plurality of first teaching vectors and second teaching vectors into element distributions based on probability distributions.
[0209]
FIG. 18 is a diagram showing an operation flow of offline processing according to the fifth embodiment, and FIG. 19 is a diagram showing an operation flow of online processing.
[0210]
The processing from S2100 to S2104 in the offline processing is the same as the processing from S10 to S15 shown in FIG.
[0211]
After confirming the input of the first electronic data for N pieces in S2104, the distribution element calculation unit 401 reads the teaching vectors V for N persons.₁, V₂Of the composite vector.
[0212]
Next, the distribution element calculation unit 401 models the probability distribution in the set of N combined vectors into an element distribution having R distributions using a mixed Gaussian model (hereinafter referred to as GMM). Then, the distribution element calculation unit 401 outputs the first teaching vector and the second teaching vector for each element distribution to the average vector calculation unit 203a.
[0213]
The average vector calculation unit 203a, average vector calculation unit 203b, cross-correlation information calculation unit 204, and autocorrelation information calculation unit 205 calculate parameters of the k (k = 1... R) -th element distribution when modeled. .
[0214]
That is, the average vector calculation unit 203b calculates the vector V belonging to the kth element distribution.₁Mean vector M₁ ^kThe average vector calculation unit 203a calculates the vector V belonging to the kth element distribution.₂Mean vector M₂ ^kThe cross-correlation information calculation unit calculates the vector V₁Covariance matrix C₁ ^kC which is a pseudo inverse matrix of₁ ^{k *}And the autocorrelation information calculation unit calculates the vector V₁, V₂Cross-correlation matrix C₁₂ ^kIs calculated (S2105).
[0215]
In addition, the average vector calculation unit 203b generates an average vector M₁ ^kAre output to the autocorrelation information calculation unit 205 and the cross correlation information calculation unit 204, and the average vector calculation unit 203a₂ ^kIs output to the cross-correlation information calculation unit 204.
[0216]
For this calculation, an EM (Expectation Maximization) algorithm is usually used. Bishop, Oxford, “Neural Networks for Pattern Recognition”, pages 59-73 (1995).
[0217]
Next, the autocorrelation information calculation unit 206 calculates the calculated vector V₁Pseudo-inverse matrix C of the covariance matrix₁ ^{k *}And the cross-correlation matrix C₁₂ ^kTo the feature extraction matrix C according to (Equation 10)_Three ^kIs calculated (S2106).
[0218]
[Expression 10]

Next, the feature extraction matrix calculator 206 calculates the average vector M₁ ^k, M₂ ^kAnd feature extraction matrix C_Three ^kIs output to the mutual information database (S2107).
[0219]
Next, the learning unit 103 calculates an average vector M of R distributions.₁ ^k, M₂ ^kAnd feature extraction matrix C_Three ^kIs stored in the mutual information database, the offline processing is terminated, and an average vector M of R distributions is terminated.₁ ^k, M₂ ^kAnd feature extraction matrix C_Three ^kAre not stored in the mutual information database, S2105 to S2107 are repeated (S2108).
[0220]
Next, online processing in the fifth embodiment will be described using the operation flow shown in FIG.
[0221]
The processes of S2200, S2201, S2203 to S2206 in the online process of the fifth embodiment are the same as the processes from S20, S21, S23 to S26 of FIG. 6, and are the same as those of the first embodiment. Description of is omitted.
[0222]
The parameter calculation unit 302 applies the feature extraction matrix C obtained offline to the input vector data X calculated in S2201._Three ^kAnd mean vector M₁ ^k, M₂ ^kThen, the expected value vector E of the coordinates of the feature point with respect to the input vector data X is calculated according to (Equation 11) (S2202).
[0223]
## EQU11 ##

The expected value vector E of the feature point coordinates obtained by (Equation 11) is the teaching vector V₁And teaching vector V₂Is equal to the expected value of the output obtained for the input vector data X when learning using Bayesian estimation under the assumption that the distribution of the two vectors in the combined vector is a mixed normal distribution. Bayesian estimation is a statistical estimation method that defines a distribution of parameters and an appropriate loss function, and estimates such that the expected value of the loss function is minimized. That is, it can be said that the most likely output value for the input vector data X can be estimated by (Equation 11).
[0224]
Since the subsequent processing from S2203 to S2206 is the same as that in the first embodiment, description thereof is omitted here.
[0225]
As described above, according to the fifth embodiment, the sample input image is divided into a plurality of distributions, and the correlation between the sample input image and the coordinates of the feature points of the input image is examined for each distribution. Using these correlations, the coordinates of the feature points of the input image for verification can be estimated. Thereby, even if the distribution of the input image for collation, that is, the characteristics vary, the coordinates of the feature points can be estimated accurately.
[0226]
Further, as shown in (Equation 11), a unique parameter of the input vector can be estimated by direct calculation by matrix operation. In this way, compared to the conventional search by repeated operation using template matching, the effect is very large because it can be estimated with a very low calculation cost and with a very high accuracy because of using a mixed distribution model. .
[0227]
Further, according to the fifth embodiment, the cross-correlation information for each element distribution can be obtained simply by matrix operation, that is, by multiplication.
[0228]
(Embodiment 6)
The sixth embodiment of the present invention corresponds to the case where there is a variation in the distribution of input vectors, which are sample input images, as in the fifth embodiment. In the fifth embodiment, the teaching vector V₂Whereas a face is obtained from the coordinates of each feature point, the sixth embodiment is similar to the second embodiment in that the teaching vector V is obtained from the area image around the coordinates of the face feature points.₂And a template suitable for the face image for verification is automatically generated from the image for verification.
[0229]
Specifically, the composite information of the sample input image and the region image around the coordinates of the facial feature points is divided into a plurality of distributions, and for each distribution, the matching image and the feature point of the matching image The correlation with the coordinates is examined, and the coordinates of the feature points of the verification image are obtained using these correlations.
[0230]
In addition, the data collating apparatus according to the sixth embodiment is roughly divided into offline processing and online processing, as in the second and fifth embodiments. Here, in particular, differences from the first embodiment will be described with reference to FIGS.
[0231]
In the offline processing of the sixth embodiment, the processing from S2400 to S2402 and S2404 is the same as the processing from S10 to S12 and S15 in FIG. . Also, the processing from S2405 to S2408 is the same as the processing from S2105 to S2108 in FIG.
[0232]
The second teaching vector calculation unit 201 of the learning unit 103 cuts out the image of the peripheral region in each part of the face from the coordinates of the feature points input in S2402, and the teaching vector V₂Is calculated (S2403).
[0233]
Next, in S2404, after confirming the end of N inputs, the learning unit 103 performs the processing from S2405 to S2408 shown in FIG.₁ ^k, M₂ ^kAnd feature extraction matrix C_Three ^kAccumulates in the mutual information database.
[0234]
However, the feature extraction matrix C here_Three ^kIs produced by an area image around the coordinates of the facial feature points.
[0235]
Next, online processing in the sixth embodiment will be described with reference to the operation flow shown in FIG.
[0236]
The processing from S2500, S2501, S2505 to S2507 in the online processing of the sixth embodiment is the same as the processing of S20, S21, S24 to S26 in FIG. 6, and is the same as that of the first embodiment. Description of is omitted.
[0237]
The parameter calculation unit 302 calculates an expected value vector E using (Equation 11) for the vector X calculated in S2501 (S2502).
[0238]
Next, the template matching unit 303 creates a template image modeled into an element distribution having R distributions (S2503), executes template matching, and calculates the coordinates of feature points in the second electronic data ( S2504).
[0239]
Since the subsequent processing from S2505 to S2507 is the same as that in the first embodiment, description thereof is omitted here.
[0240]
As described above, according to the sixth embodiment, the sample image is divided into a plurality of distributions, and the correlation between the sample image and each part of the image is examined for each distribution. By using this, it is possible to estimate an image at each part of the input image for verification. Thereby, even if the distribution of the input image for verification, that is, the characteristics vary, each part can be estimated accurately. Then, by performing template matching using the estimated image of each part as a template, it is possible to detect the coordinates of the feature points in the input image for verification.
[0241]
(Embodiment 7)
The seventh embodiment of the present invention corresponds to the case where there is a variation in the distribution of input vectors, which are sample input images, as in the fifth embodiment. Teaching vector V₂In the fifth embodiment, the coordinates are obtained from the coordinates of each feature point of the face, and in the sixth embodiment, the area around the coordinates of the facial feature points is obtained. 3, the teaching vector V is calculated from the coordinates of the facial feature points and the combined vectors of the region images around the facial feature point coordinates.₂Ask for. And the obtained teaching vector V₂Is used to estimate the coordinates of the feature points of the matching face image, and a template suitable for the matching face image is automatically generated from the matching image. Then, matching is performed using a template that automatically generates a peripheral area of the coordinates of the estimated feature points.
[0242]
As a result, the template search range is limited and matching processing can be performed with a single template. Compared to the conventional search for the entire region of the face image for matching and the use of multiple templates. The calculation cost can be reduced at each stage.
[0243]
Further, the data collating apparatus according to the seventh embodiment is roughly divided into an offline process and an online process as in the third and fifth embodiments. Here, differences from the first embodiment will be described with reference to FIGS.
[0244]
In the offline processing according to the seventh embodiment, the processing from S2600 to S2602 and S2605 is the same as the processing from S10 to S12 and S15 in FIG. .
[0245]
The second teaching vector calculation unit 201 of the learning unit 103 uses the coordinates of the feature points input in S2602 to cut out the image of the peripheral region of each part in the first electronic data (S2603), and is input in S2602 The teaching vector V, which is a combined vector of the coordinates of the feature points and the image cut out in S2603₂Is calculated (S2604).
[0246]
In S2605, after confirming the end of N inputs, the learning unit 103 performs the same processing as in S2405 to S2408 in FIG.₁ ^k, M₂ ^kAnd feature extraction matrix C_Three ^kAccumulates in the mutual information database (S2606 to S2609).
[0247]
However, the feature extraction matrix C here_Three ^kIs generated by the combined vector of the feature point coordinates in the first electronic data and the region image around the feature point coordinates.
[0248]
Next, online processing in the seventh embodiment will be described using the operation flow shown in FIG.
[0249]
In the online processing of the seventh embodiment, the processing from S2700, S2701, S2705 to S2707 is the same as the processing from S20, S21, S24 to S26 in FIG. Description of is omitted.
[0250]
The parameter calculation unit 302 calculates an expected value vector E from the vector X calculated in S2701 using (Equation 11) (S2702).
[0251]
Next, the template matching unit 303 creates a template image modeled as an element distribution having R distributions (S2703), executes template matching around the coordinates of the feature points obtained by the vector E in S2703, The coordinates of the feature points in the second electronic data are calculated (S2704).
[0252]
Since the subsequent processing from S2705 to S2707 is the same as that of the first embodiment, description thereof is omitted here.
[0253]
As described above, according to the seventh embodiment, the sample image is divided into a plurality of distributions, and for each distribution, the sample image, the coordinates of the feature points of the image, and the images of the coordinates of the feature points are You can check the correlation. Then, the coordinates of each feature point in the face of the input image for verification can be estimated using the examined correlation. Further, the coordinate of the feature point of the matching image can be estimated by a hierarchical search process in which matching is performed using the estimated template in the peripheral area of the estimated feature point coordinate.
[0254]
As a result, the accuracy in estimating the coordinates of feature points can be improved, and the template matching range can be narrowed down to a limited region, and only one template can be used. As a result, the calculation cost can be reduced as compared with the conventional full search method.
[0255]
(Embodiment 8)
In the eighth embodiment of the present invention, the input image of the sample is a vehicle, and in the same way as in the fifth embodiment, when there is variation in the distribution of the input vector that is the input image of the sample, collation having various distributions is performed. This corresponds to the input image.
[0256]
Further, the data collating apparatus according to the eighth embodiment is roughly divided into an offline process and an online process as in the fourth and fifth embodiments. Here, differences from the first embodiment will be particularly described with reference to FIG.
[0257]
Since the off-line processing of the eighth embodiment is the same as that of the fifth embodiment, description thereof is omitted here. However, an average vector M of R distributions₁ ^k, M₂ ^kAnd feature extraction matrix C_Three ^kConsists of a vehicle image, a license plate area image and a front area image.
[0258]
Next, online processing according to Embodiment 8 will be described using the operation flow shown in FIG.
[0259]
In the online processing of the eighth embodiment, the processing from S2800, S2801, S2803 to S2807 is the same as the processing of S20, S21, S23, and S26 in FIG. Description is omitted.
[0260]
The parameter calculation unit 302 calculates an expected value vector E from the vector X calculated in S2801 using (Equation 11) (S2802).
[0261]
The collation unit 107 calculates a vehicle image, a license plate region image, and a front region image from the coordinates of the feature points calculated in S2803 (S2804), reads the license plate (S2805), and reads the vehicle image and the license plate. Collation is performed (S2806).
[0262]
As described above, according to the eighth embodiment, the sample vehicle image is divided into a plurality of distributions, and the correlation between the sample vehicle image and the image of the coordinates of each feature point of each image is calculated for each distribution. It is possible to estimate the image at the coordinates of each feature point of the vehicle image for verification using these correlations.
[0263]
As a result, the coordinates of each feature point can be accurately estimated even if the distribution of the vehicle image for verification, that is, the characteristics vary. Then, by performing template matching using the estimated image of the coordinates of each feature point as a template, the coordinates of the feature points in the vehicle image for verification can be detected.
[0264]
In the eighth embodiment, the data collating apparatus according to the fifth embodiment is applied to the vehicle image collating apparatus. However, the data collating apparatus according to the sixth embodiment or the seventh embodiment is applied to the vehicle image collating apparatus. It may be in the form.
[0265]
The present invention is not limited to the embodiment described above.
[0266]
In the present invention, it is realized by a computer reading a program, and this can be easily implemented in another independent computer system by storing it in a flexible storage medium and transferring it.
[0267]
Also included within the scope of the invention are computer program products that are storage media containing instructions that can be used to program a computer that implements the invention. The storage medium is a flexible disk, an optical disk, a CDROM, a magnetic disk, a ROM, a RAM, an EPROM, an EEPROM, a magnetic optical card, a memory card, a DVD, or the like, but is not particularly limited thereto.
[0268]
Moreover, you may use the parameter estimation apparatus of this invention for collation of images other than a face image and a vehicle image.
[0269]
In the present invention, the parameter estimation device is used for image collation. However, the parameter estimation device may be used for a collation device other than image collation by inputting data other than images as input data. For example, the parameter estimation apparatus may be used as a voice collation apparatus by inputting voice data or the like as input data.
[0270]
【The invention's effect】
As described above, according to the present invention, the coordinates of the feature points of the input image can be accurately obtained with a small processing cost.
[Brief description of the drawings]
FIG. 1 is a diagram showing a data collating apparatus according to a first embodiment of the present invention.
FIG. 2 is a diagram showing a learning unit in the first embodiment
FIG. 3 is an operation flow diagram of offline processing of the data collating apparatus according to the first embodiment;
4 is a diagram showing coordinates of facial feature points input by a user in Embodiment 1. FIG.
FIG. 5 is a diagram showing a parameter estimation unit in the first embodiment.
FIG. 6 is an operation flowchart of online processing of the data collating apparatus according to the first embodiment;
7 is a diagram showing a face area detected in the first embodiment; FIG.
FIG. 8 is a diagram output to the display in the first embodiment
FIG. 9 is an operation flowchart of offline processing of the data collating apparatus according to the second embodiment of the present invention;
FIG. 10 is a diagram showing coordinates of facial feature points input by a user in the second embodiment
FIG. 11 is an operation flowchart of online processing of the data collating apparatus according to the second embodiment;
FIG. 12 is an operation flowchart of offline processing of the data collating apparatus according to the third embodiment of the present invention;
FIG. 13 is an operation flowchart of online processing of the data collating apparatus according to the third embodiment;
FIG. 14 is an operation flowchart of offline processing of the vehicle image matching device according to the fourth embodiment of the present invention;
FIG. 15 is a diagram illustrating coordinates of feature points of a vehicle image input by a user in the fourth embodiment.
FIG. 16 is an operation flow diagram of online processing of the vehicle image matching device according to the fourth embodiment;
FIG. 17 is a diagram output to the display in the fourth embodiment
FIG. 18 is an operation flowchart of offline processing of the data collating apparatus according to the fifth embodiment of the present invention;
FIG. 19 is an operation flowchart of online processing of the data collating apparatus according to the fifth embodiment;
FIG. 20 is a diagram showing a learning unit according to the fifth embodiment;
FIG. 21 is an operation flowchart of offline processing of the data collating apparatus according to the sixth embodiment of the present invention;
FIG. 22 is an operation flowchart of online processing of the data collating apparatus according to the sixth embodiment;
FIG. 23 is an operation flowchart of offline processing of the data collating apparatus according to the seventh embodiment of the present invention;
FIG. 24 is an operation flow diagram of online processing of the data collating apparatus according to the seventh embodiment;
FIG. 25 is an operation flow diagram of online processing of the vehicle image matching device according to the eighth embodiment;
FIG. 26 is an operation flowchart of a conventional facial feature extraction method.
FIG. 27 shows an eye template
FIG. 28 is a diagram showing search by template.
[Explanation of symbols]
101 Image input unit
102 Parameter input section
103 Learning Department
104 Correlation information database
105 Parameter estimation unit
106 Image database
107 Verification unit
108 Output section
201 Second teaching vector calculation unit
202 1st teaching vector calculation part
203a, 203b Average vector calculator
204 Cross correlation information calculator
205 Autocorrelation information calculator
206 Feature extraction matrix calculator
301 Vector calculator
302 Parameter calculator
303 Template matching section
401 Distribution element calculator
1302 Verification area

Claims (32)

  An image input unit that converts optical data into electronic data, a parameter input unit that inputs the coordinates of the first feature point from the first electronic data input from the image input unit, and a plurality of the first electronic data Autocorrelation information is calculated from the plurality of first electronic data and coordinates of the plurality of first feature points, and from the image input unit from the autocorrelation information and the crosscorrelation information A feature extraction matrix for estimating the coordinates of the second feature points of the input second electronic data is calculated, a first average vector calculated from a plurality of the first electronic data, and a plurality of the first electronic data. A learning unit that outputs a second average vector calculated from the coordinates of one feature point and the feature extraction matrix, and the first average vector, the second average vector, and the feature extraction that are input from the learning unit queue A correlation information database to be accumulated; and a parameter estimation unit that estimates coordinates of the second feature point using the second electronic data, the first average vector, the second average vector, and the feature extraction matrix; A parameter estimation device including:   The learning unit calculates a first teaching vector that is a vector pattern in which the first electronic data is arranged in the scanning order, and a vector pattern in which the coordinates of the first feature points are arranged in order. A second teaching vector calculation unit for calculating the second teaching vector, a first average vector obtained by averaging the plurality of first teaching vectors, and a plurality of the second teaching vectors being averaged An average vector calculation unit that calculates a second average vector, a plurality of the first teaching vectors and the first average vector, and a first covariance matrix that is autocorrelation information of the first teaching vectors An autocorrelation information calculation unit for calculating a pseudo inverse matrix of the plurality of first teaching vectors, the first average vector, the plurality of second teaching vectors, and the second average vector , A cross-correlation information calculation unit that calculates a second covariance matrix that is cross-correlation information between the first teaching vector and the second teaching vector, a pseudo inverse matrix of the first covariance matrix, and the first 2. A parameter estimation according to claim 1, further comprising: a feature extraction matrix calculation unit that inputs a covariance matrix of 2 and calculates a feature extraction matrix from the pseudo inverse matrix of the first covariance matrix and the second covariance matrix. apparatus.   The parameter estimation unit receives the second electronic data, calculates a vector pattern that is a vector pattern in which the second electronic data is arranged in scanning order, the input vector, the first average A parameter calculator for calculating an expected value vector including the coordinates of the second feature point from the vector, the second average vector, and the feature extraction matrix; and calculating the coordinates of the second feature point from the expected value vector The parameter estimation device according to claim 1, further comprising: a template matching unit that performs processing.   An image input unit that converts optical data into electronic data, a parameter input unit that inputs an image around the coordinates of the first feature point from the first electronic data input from the image input unit, and a plurality of the first Autocorrelation information is calculated from the plurality of first electronic data and images around the coordinates of the plurality of first feature points, and further, the autocorrelation information and the cross correlation information are calculated. A feature extraction matrix for estimating an image around the coordinates of the second feature point of the second electronic data input from the image input unit from the correlation information is calculated, and calculated from a plurality of the first electronic data. A first average vector, a second average vector calculated from images around the coordinates of the plurality of first feature points and the feature extraction matrix, and the first input from the learning unit Average vector of , The correlation information database storing the second average vector and the feature extraction matrix, the second electronic data, the first average vector, the second average vector, and the feature extraction matrix. A parameter estimation device including a parameter estimation unit that estimates the coordinates of two feature points.   The learning unit sequentially arranges a first teaching vector calculation unit that calculates a first teaching vector that is a vector pattern in which the first electronic data is arranged in the scanning order, and images around the coordinates of the first feature point. A second teaching vector calculation unit that calculates a second teaching vector that is a vector pattern, a first average vector obtained by averaging the plurality of first teaching vectors, and a plurality of the second teaching vectors An average vector calculation unit for calculating a second average vector obtained by averaging the first teaching vector and a plurality of the first teaching vectors and the first average vector, and the first teaching vector is autocorrelation information of the first teaching vector. An autocorrelation information calculation unit for calculating a pseudo inverse matrix of a covariance matrix, a plurality of the first teaching vectors, the first average vector, a plurality of the second teaching vectors, and the second average vector A cross-correlation information calculation unit that calculates a second covariance matrix that is cross-correlation information between the first teaching vector and the second teaching vector, and a pseudo inverse of the first covariance matrix And a feature extraction matrix calculation unit that inputs a matrix and the second covariance matrix, and calculates a feature extraction matrix from the pseudo inverse matrix of the first covariance matrix and the second covariance matrix. 4. The parameter estimation device according to 4.   The parameter estimation unit receives the second electronic data, calculates a vector pattern that is a vector pattern in which the second electronic data is arranged in scanning order, the input vector, the first average A parameter calculator for calculating an expected value vector including an image around the coordinates of the second feature point from the vector, the second average vector, and the feature extraction matrix; and the coordinates of the second feature point from the expected value vector A peripheral image is calculated, a collation area that is an image area of the second electronic data that matches the peripheral image of the second feature point is searched, and the coordinates of the second feature point are retrieved from the collation area. The parameter estimation device according to claim 4, further comprising: a template matching unit that calculates   An image input unit for converting optical data into electronic data, and parameter input for inputting the coordinates of the first feature point and the image around the coordinates of the first feature point from the first electronic data input from the image input unit And autocorrelation information from a plurality of the first electronic data, a plurality of the first electronic data, a plurality of coordinates of the first feature points, and a plurality of coordinates of the first feature points The cross-correlation information is calculated from the combined vector with the image, and the coordinates of the second feature point and the second feature of the second electronic data input from the image input unit from the auto-correlation information and the cross-correlation information A feature extraction matrix for estimating an image around the coordinates of the point, a first average vector calculated from the plurality of first electronic data, a second average vector calculated from the plurality of the combined vectors, and Feature extraction A learning unit that outputs a column; a correlation information database that stores the first average vector, the second average vector, and the feature extraction matrix that are input from the learning unit; the second electronic data; And a parameter estimation unit that estimates the coordinates of the second feature point using the average vector of 1, the second average vector, and the feature extraction matrix.   The learning unit calculates a first teaching vector that is a vector pattern in which the first electronic data is arranged in scanning order, and a second teaching pattern that is a vector pattern in which combined vectors are arranged in order. A second teaching vector calculation unit for calculating a teaching vector; a first average vector obtained by averaging a plurality of the first teaching vectors; and a second average vector obtained by averaging the plurality of second teaching vectors An average vector calculation unit for calculating the first teaching vector and the first average vector, and a pseudo inverse matrix of the first covariance matrix, which is autocorrelation information of the first teaching vector, An autocorrelation information calculation unit to calculate, a plurality of the first teaching vectors, the first average vector, a plurality of the second teaching vectors, and the second average vector are input, A cross-correlation information calculation unit for calculating a second covariance matrix that is cross-correlation information between the first teaching vector and the second teaching vector; a pseudo inverse matrix of the first covariance matrix; The parameter estimation device according to claim 7, further comprising: a feature extraction matrix calculation unit that calculates a feature extraction matrix from the covariance matrix.   The parameter estimation unit receives the second electronic data, calculates a vector pattern that is a vector pattern in which the second electronic data is arranged in scanning order, the input vector, the first average A parameter calculation unit for calculating an expected value vector including coordinates of the second feature point and an image around the coordinates of the second feature point from the vector, the second average vector, and the feature extraction matrix; An image around the coordinates of the second feature point and an image around the coordinate of the second feature point; an image around the coordinate of the second feature point estimated; 9. A template matching unit that searches a matching area that is an image area of the second electronic data that matches and calculates a coordinate of the second feature point from the matching area. Parameter estimation device.   An image input unit that converts optical data into electronic data; a parameter input unit that inputs coordinates of a first feature point from the first electronic data input from the image input unit; the first electronic data; A set of composite information with the coordinates of the first feature points is divided into a plurality of distributions, and for each distribution, autocorrelation information, a plurality of the first electronic data, and a plurality of the plurality of the first electronic data Cross-correlation information is calculated from the coordinates of the first feature point, and further, the coordinates of the second feature point of the second electronic data input from the image input unit are estimated from the auto-correlation information and the cross-correlation information. A feature extraction matrix for each distribution, a first average vector calculated from the plurality of first electronic data obtained for each distribution, and a plurality of the first vectors obtained for each distribution Second calculated from the coordinates of the feature point A learning unit that outputs an average vector and the feature extraction matrix obtained for each distribution, and the first average vector, the second average vector, and the feature extraction that are input from the learning unit obtained for each distribution Using the correlation information database for storing the matrix, the second electronic data, the first average vector, the second average vector, and the feature extraction matrix, the coordinates of the second feature points for each distribution A parameter estimation device including a parameter estimation unit for estimation.   The learning unit calculates a first teaching vector that is a vector pattern in which the first electronic data is arranged in the scanning order, and a vector pattern in which the coordinates of the first feature points are arranged in order. A second teaching vector calculation unit for calculating the second teaching vector, a plurality of the first teaching vectors and a plurality of the second teaching vectors, and a set of the combining vectors A distribution element calculation unit that divides the first teaching vector and the second teaching vector into a plurality of distributions, and a first average vector obtained by averaging the plurality of first teaching vectors for each distribution. An average vector calculation unit that calculates and averages a plurality of the second teaching vectors for each distribution, and a plurality of the first teaching vectors for each distribution. And an autocorrelation information calculation unit that calculates a pseudo inverse matrix of a first covariance matrix, which is autocorrelation information of the first teaching vector, for each distribution, A first teaching vector, the first average vector, a plurality of the second teaching vectors, and the second average vector are inputted for each distribution, and the first teaching vector and the second teaching vector A cross-correlation information calculation unit that calculates a second covariance matrix that is cross-correlation information for each distribution, and a pseudo inverse matrix of the first covariance matrix and the second covariance matrix for each distribution. The parameter estimation device according to claim 10, further comprising: a feature extraction matrix calculation unit that inputs and calculates a feature extraction matrix for each distribution from the pseudo inverse matrix of the first covariance matrix and the second covariance matrix. .   The parameter estimation unit is configured to input the second electronic data and calculate an input vector, which is a vector pattern in which the second electronic data is arranged in a scanning order, and to classify the input vector and distribution. A parameter calculator for calculating an expected value vector including coordinates of the second feature point from a first average vector, a second average vector classified for each distribution, and a feature extraction matrix classified for each distribution; and the expected value The parameter estimation apparatus according to claim 10, further comprising a template matching unit that calculates coordinates of the second feature point from a vector.   An image input unit that converts optical data into electronic data, a parameter input unit that inputs an image around the coordinates of the first feature point from the first electronic data input from the image input unit, and the first electronic A set of combined information of the data and an image around the coordinates of the first feature point is divided into a plurality of distributions, and autocorrelation information and a plurality of the first electrons are obtained from the plurality of first electronic data for each distribution. Cross-correlation information is calculated from the data and images around the coordinates of the plurality of first feature points, and the second electronic data of the second electronic data input from the image input unit is calculated from the auto-correlation information and the cross-correlation information. A feature extraction matrix for estimating an image around the coordinates of two feature points for each distribution, a first average vector calculated from the plurality of first electronic data obtained for each distribution, Multiple previous obtained for each distribution A learning unit that outputs a second average vector calculated from an image around the coordinates of the first feature point and the feature extraction matrix obtained for each distribution, and the learning unit that is obtained for each distribution is input from the learning unit A correlation information database for storing the first average vector, the second average vector, and the feature extraction matrix; the second electronic data; the first average vector; the second average vector; and the feature extraction matrix. A parameter estimation unit including a parameter estimation unit that estimates the coordinates of the second feature point for each of the distributions.   The learning unit sequentially arranges a first teaching vector calculation unit that calculates a first teaching vector that is a vector pattern in which the first electronic data is arranged in the scanning order, and images around the coordinates of the first feature point. A second teaching vector calculation unit that calculates a second teaching vector that is a vector pattern, a plurality of the first teaching vectors and a plurality of the second teaching vectors, and a plurality of combining vectors. A distribution element calculation unit that divides the first teaching vector and the second teaching vector into a plurality of distributions according to a probability distribution of a set of vectors; and a first that averages the plurality of first teaching vectors for each of the distributions. An average vector calculation unit that calculates a second average vector obtained by averaging a plurality of second teaching vectors for each distribution, and a plurality of the first vectors for each distribution. An autocorrelation information calculation unit that inputs a teaching vector and a first average vector and calculates a pseudo inverse matrix of a first covariance matrix that is autocorrelation information of the first teaching vector for each distribution; The first teaching vector, the first average vector, a plurality of the second teaching vectors, and the second average vector are input for each distribution, and the first teaching vector and the second teaching vector are input. A cross-correlation information calculation unit that calculates a second covariance matrix that is cross-correlation information for each distribution, and a pseudo inverse matrix of the first covariance matrix and the second covariance matrix for each distribution The parameter estimation device according to claim 13, further comprising: a feature extraction matrix calculation unit that calculates a feature extraction matrix for each distribution from the pseudo inverse matrix of the first covariance matrix and the second covariance matrix. .   The parameter estimation unit is configured to input the second electronic data and calculate an input vector, which is a vector pattern in which the second electronic data is arranged in a scanning order, and to classify the input vector and distribution. A parameter calculation unit for calculating an expected value vector including an image around the coordinates of the second feature point from a first average vector, a second average vector classified for each distribution, and a feature extraction matrix classified for each distribution; An image around the coordinates of the second feature point is calculated from the expected value vector, and a collation area that is an image area of the second electronic data that matches the image around the coordinate of the second feature point is searched. The parameter estimation apparatus according to claim 13, further comprising: a template matching unit that calculates coordinates of the second feature point from the collation region.   An image input unit for converting optical data into electronic data, and parameter input for inputting the coordinates of the first feature point and the image around the coordinates of the first feature point from the first electronic data input from the image input unit And a set of composite information of the first electronic data, the coordinates of the first feature points and the image around the coordinates of the first feature points, are divided into a plurality of distributions. Cross-correlation information is calculated from autocorrelation information from one electronic data, a plurality of the first electronic data, a plurality of coordinates of the first feature points, and an image around the coordinates of the first feature points; A feature extraction matrix for estimating the coordinates of the second feature point of the second electronic data input from the image input unit and the image around the coordinate of the second feature point from the autocorrelation information and the cross-correlation information Calculate for each distribution and find for each distribution A second average calculated from a first average vector calculated from the plurality of first electronic data, a plurality of coordinates of the first feature points obtained for each distribution, and an image around the coordinates of the first feature points. A learning unit that outputs the average extraction vector and the feature extraction matrix obtained for each distribution, and the first average vector, the second average vector, and the feature that are input from the learning unit obtained for each distribution The coordinates of the second feature points for each distribution using the correlation information database for storing the extraction matrix, the second electronic data, the first average vector, the second average vector, and the feature extraction matrix. A parameter estimation device including a parameter estimation unit for estimating the parameter estimation unit.   A first teaching vector calculating unit that calculates a first teaching vector that is a vector pattern in which the first electronic data is arranged in the scanning order; a coordinate of the first feature point; and a first feature point A second teaching vector calculation unit that calculates a second teaching vector that is a vector pattern in which images around the coordinates of the two are sequentially arranged; a plurality of the first teaching vectors and a plurality of the second teaching vectors; A distribution element calculation unit that creates a combined vector and divides the first teaching vector and the second teaching vector into a plurality of distributions according to a probability distribution of the set of the combined vectors; and a plurality of the first teaching vectors for each distribution An average vector calculation unit that calculates a first average vector that averages the teaching vectors, calculates a second average vector that averages a plurality of the second teaching vectors for each of the distributions; Each time, a plurality of the first teaching vector and the first average vector are input, and a pseudo inverse matrix of the first covariance matrix, which is autocorrelation information of the first teaching vector, is calculated for each distribution. An autocorrelation information calculation unit, a plurality of the first teaching vectors, the first average vector, a plurality of the second teaching vectors, and the second average vector are input for each distribution, and the first A cross-correlation information calculation unit that calculates, for each distribution, a second covariance matrix that is cross-correlation information between the teaching vector and the second teaching vector, and a pseudo inverse of the first covariance matrix for each distribution A feature extraction matrix calculation unit that inputs a matrix and the second covariance matrix, and calculates a feature extraction matrix for each distribution from the pseudo inverse matrix of the first covariance matrix and the second covariance matrix; Parameter estimation according to claim 16, including Location.   The parameter estimation unit is configured to input the second electronic data and calculate an input vector, which is a vector pattern in which the second electronic data is arranged in a scanning order, and to classify the input vector and distribution. Expected value vector including the first average vector, the second average vector classified for each distribution, and the feature extraction matrix classified for each distribution, the coordinates of the second feature point and the image around the coordinates of the second feature point A parameter calculation unit that calculates the coordinates of the second feature point and an image around the coordinate of the second feature point from the expected value vector, and coincides with the image around the coordinate of the second feature point 18. A template matching unit that searches for a collation area that is an image area of the second electronic data and calculates coordinates of the second feature point from the collation area. Parameter estimation device.   An image input unit that converts optical data into electronic data, a parameter input unit that inputs the coordinates of the first feature point from the first electronic data input from the image input unit, and a plurality of the first electronic data Autocorrelation information is calculated from the plurality of first electronic data and coordinates of the plurality of first feature points, and from the image input unit from the autocorrelation information and the crosscorrelation information A feature extraction matrix for estimating the coordinates of the second feature points of the input second electronic data is calculated, a first average vector calculated from a plurality of the first electronic data, and a plurality of the first electronic data. A learning unit that outputs a second average vector calculated from the coordinates of one feature point and the feature extraction matrix, and the first average vector, the second average vector, and the feature extraction that are input from the learning unit queue A correlation information database to be accumulated; and a parameter estimation unit that estimates coordinates of the second feature point using the second electronic data, the first average vector, the second average vector, and the feature extraction matrix; A collation area that is an image used for collation is calculated from the image database for storing the first electronic data, the second electronic data, and the coordinates of the second feature point, and the collation area and the image database are stored in the collation area and the image database. A data collating apparatus comprising: a collation unit that collates with the accumulated first electronic data.   An image input unit that converts optical data into electronic data, a parameter input unit that inputs an image around the coordinates of the first feature point from the first electronic data input from the image input unit, and a plurality of the first Autocorrelation information is calculated from the plurality of first electronic data and images around the coordinates of the plurality of first feature points, and further, the autocorrelation information and the cross correlation information are calculated. A feature extraction matrix for estimating an image around the coordinates of the second feature point of the second electronic data input from the image input unit from the correlation information is calculated, and calculated from a plurality of the first electronic data. A first average vector, a second average vector calculated from images around the coordinates of the plurality of first feature points and the feature extraction matrix, and the first input from the learning unit Average vector of , The correlation information database storing the second average vector and the feature extraction matrix, the second electronic data, the first average vector, the second average vector, and the feature extraction matrix. A parameter estimation unit that estimates the coordinates of two feature points; an image database that stores the first electronic data; and a collation that is an image used for collation from the second electronic data and the coordinates of the second feature points A data collating apparatus comprising: a collation unit that calculates a region and collates a match between the collation region and the first electronic data stored in the image database.   An image input unit for converting optical data into electronic data, and parameter input for inputting the coordinates of the first feature point and the image around the coordinates of the first feature point from the first electronic data input from the image input unit And autocorrelation information from a plurality of the first electronic data, a plurality of the first electronic data, a plurality of coordinates of the first feature points, and a plurality of coordinates of the first feature points The cross-correlation information is calculated from the combined vector with the image, and the coordinates of the second feature point and the second feature of the second electronic data input from the image input unit from the auto-correlation information and the cross-correlation information A feature extraction matrix for estimating an image around the coordinates of the point, a first average vector calculated from the plurality of first electronic data, a second average vector calculated from the plurality of the combined vectors, and Feature extraction A learning unit that outputs a column; a correlation information database that stores the first average vector, the second average vector, and the feature extraction matrix that are input from the learning unit; the second electronic data; A parameter estimation unit that estimates the coordinates of the second feature point using the average vector of 1, the second average vector, and the feature extraction matrix; an image database that stores the first electronic data; The collation which calculates the collation area | region which is an image used for collation from the electronic data of 2 and the coordinate of the said 2nd feature point, and collates the match with the said 1st electronic data accumulate | stored in the said image database A data collating device including a section.   An image input unit that converts optical data into electronic data; a parameter input unit that inputs coordinates of a first feature point from the first electronic data input from the image input unit; the first electronic data; A set of composite information with the coordinates of the first feature points is divided into a plurality of distributions, and for each distribution, autocorrelation information, a plurality of the first electronic data, and a plurality of the plurality of the first electronic data Cross-correlation information is calculated from the coordinates of the first feature point, and further, the coordinates of the second feature point of the second electronic data input from the image input unit are estimated from the auto-correlation information and the cross-correlation information. A feature extraction matrix for each distribution, a first average vector calculated from the plurality of first electronic data obtained for each distribution, and a plurality of the first vectors obtained for each distribution Second calculated from the coordinates of the feature points A learning unit that outputs an average vector and the feature extraction matrix obtained for each distribution, and the first average vector, the second average vector, and the feature extraction that are input from the learning unit obtained for each distribution Using the correlation information database for storing the matrix, the second electronic data, the first average vector, the second average vector, and the feature extraction matrix, the coordinates of the second feature points for each distribution A parameter estimation unit to be estimated; an image database for storing the first electronic data; a collation region that is an image used for collation from the coordinates of the second electronic data and the second feature point; A data collating apparatus including a collation unit that collates a match between a region and the first electronic data stored in the image database. A conversion step for converting optical data into electronic data, and the coordinates of the first feature point from the first electronic data converted in the conversion step are input, and autocorrelation information is calculated from the plurality of first electronic data. the method comprising, converted calculating a cross-correlation information from the plurality of first electronic data and a plurality of the first feature point coordinates, in said converting step from said autocorrelation information and the cross-correlation information Calculating a feature extraction matrix for estimating the coordinates of the second feature point of the second electronic data, a first average vector calculated from the plurality of first electronic data, and the plurality of the first second average vector and an output step of outputting the feature extraction matrix, the first average vector output in the output step of calculating the coordinates of the feature points of The second average vector and the feature extraction matrix in a correlation information database, and using the second electronic data, the first average vector, the second average vector, and the feature extraction matrix A step of estimating the coordinates of the second feature point; and a collation region which is an image used for collation from the second electronic data and the coordinates of the second feature point, wherein the first electronic data is accumulated in an image database. steps and data verification method characterized by having a matching step of matching a match with the stored first electronic data to the image database and the collation region is calculated. A converting step of converting the optical data into electronic data, and inputs the image coordinates around the first of the first from the electronic data of the feature points that have been converted in the conversion step, the autocorrelation from the plurality of first electronic data calculated calculating information, the cross-correlation information from the plurality of first electronic data and a plurality of said first coordinate neighborhood of the feature point image, further, the from the autocorrelation information and the cross-correlation information Calculating a feature extraction matrix for estimating an image around the coordinates of the second feature point of the second electronic data converted in the conversion step; and a first calculated from the plurality of first electronic data. mean vector, and the second average vector and the output step of outputting the feature extraction matrix calculating from image coordinates around the plurality of the first feature point, the output step of Wherein is output had first average vector, the second average vector and the feature extraction matrix accumulates the correlation information database, the second electronic data, the first average vector, the second average vector and estimating the coordinates of the second feature point using the feature extraction matrix, the first electronic data stored in the image database, the second electronic data and the second feature coordinates data characterizing a step of calculating a matching region that is an image to be used for matching, and matching step of matching a match with the stored first electronic data to the image database and the verification area, to have from Matching method. A conversion step of converting the optical data into electronic data; a first feature point coordinate and an image around the first feature point coordinate from the first electronic data converted in the conversion step; calculating an autocorrelation information from the first electronic data, a plurality of the first electronic data and a plurality of the first feature point coordinates and a plurality of said first coordinate neighborhood of the feature point image A step of calculating cross-correlation information from a combination vector, and coordinates of second feature points and second feature points of the second electronic data converted in the conversion step from the auto-correlation information and the cross-correlation information the calculated and calculating a feature extraction matrix for estimating an image around a first average vector calculated from the plurality of first electronic data, a plurality of the combining vector And an output step of outputting the average vector and the feature extraction matrix, the first average vector output in the output step, and storing said second average vector and the feature extraction matrix to the correlation information database, the first 2 electronic data, the first average vector, and estimating the coordinates of the second feature point using the second average vector and the feature extraction matrix, the first electronic data to the image database A step of accumulating and calculating a collation area, which is an image used for collation, from the coordinates of the second electronic data and the second feature point; and the first electronic data accumulated in the collation area and the image database. And a step of collating a match with the data collating method. A converting step of converting the optical data into electronic data, from the first electronic data converted in the conversion step to enter the coordinates of the first feature points of the first electronic data and the first characteristic point A set of information combined with coordinates is divided into a plurality of distributions, and for each distribution, autocorrelation information, a plurality of the first electronic data, and a plurality of the first feature points are extracted from the plurality of first electronic data. A feature extraction matrix for calculating the cross-correlation information from the coordinates, and further estimating the coordinates of the second feature point of the second electronic data converted in the conversion step from the auto-correlation information and the cross-correlation information From the step of calculating for each distribution, the first average vector calculated from the plurality of first electronic data obtained for each distribution, and the coordinates of the plurality of first feature points obtained for each distribution Calculation That an output step of outputting the feature extraction matrix obtained in the second mean vector and each of the distribution, the first average vector calculated for each of the distribution output in the output step, the second mean vector And the feature extraction matrix is stored in a correlation information database, and the coordinates of the second feature point are obtained using the second electronic data, the first average vector, the second average vector, and the feature extraction matrix. A step of estimating for each distribution; a step of storing the first electronic data in an image database and calculating a collation region which is an image used for collation from the coordinates of the second electronic data and the second feature points; When, data collating method of the steps of checking a match between the stored first electronic data to the image database and the verification area, characterized with A conversion step for converting optical data into electronic data, and the coordinates of the first feature point from the first electronic data converted by the conversion step are input to a computer, and autocorrelation is performed from a plurality of the first electronic data. calculating information, a plurality of the first electronic data and a plurality of the first from the coordinates of the feature points to calculate the cross-correlation information, a second input from the autocorrelation information and the cross-correlation information Calculating a feature extraction matrix for estimating coordinates of second feature points of electronic data, a first average vector calculated from a plurality of the first electronic data, and a plurality of the first feature points an output step of outputting a second calculated from the coordinates of the mean vector and the feature extraction matrix, the first average vector output in the output step, the second average base The torque and the feature extraction matrix accumulates the correlation information database, the second electronic data, the first average vector, the second average vector and the coordinate of the second point feature with the feature extraction matrix estimating the steps of: said first electronic data stored in the image database, calculating a matching region that is an image for use in matching from the second electronic data and the second feature coordinates, the a program characterized by causing a step of checking a match between the accumulated collation region in the image database first electronic data. A computer, a conversion step Ru to convert the optical data into electronic data, and inputs the image coordinates around the first of the first from the electronic data of the feature points that have been converted in the conversion step, a plurality of the first electronic calculating an autocorrelation information from the data, calculates the cross-correlation information from the plurality of first electronic data and a plurality of said first coordinate neighborhood of the feature point image, further, the self-correlation information and the cross Calculating a feature extraction matrix for estimating an image around the coordinates of the second feature point of the second electronic data converted in the conversion step from the correlation information; and calculating from a plurality of the first electronic data first average vector is, a second average vector and the output step of outputting the feature extraction matrix calculating from image coordinates around the plurality of the first feature point, Said first average vector output at serial output step, and storing said second average vector and the feature extraction matrix to the correlation information database, the second electronic data, the first average vector, the second mean vector and estimating the coordinates of the second feature point using the feature extraction matrix, the first electronic data stored in the image database, the second electronic data and the second feature of the wherein calculating a matching region that is an image to be used for matching from the point of coordinates, the steps of checking a match between the stored first electronic data and the verification area the image database, that cause the Program. A computer, an optical data input and the conversion step of Ru is converted into electronic data, an image of the coordinates and the coordinates near the first feature point of the first first from the electronic data of the feature points that have been converted in the conversion step , step a, a plurality of said first coordinates and a plurality of coordinates around the first feature points of the electronic data and a plurality of the first feature points to calculate the autocorrelation information from the plurality of first electronic data Calculating a cross-correlation information from a combined vector with the image of the image, coordinates of a second feature point of the second electronic data converted in the conversion step from the autocorrelation information and the cross-correlation information, and a second a step of calculating a feature extraction matrix for estimating an image coordinate neighborhood of the feature point, a first average vector calculated from the plurality of first electronic data, a plurality of the binding vector Second average vector and an output step of outputting the feature extraction matrix, the first average vector output in the output step, the second average vector and correlation information database the feature extraction matrix calculating Le accumulated in the second electronic data, the first average vector, and estimating the coordinates of the second feature point using the second average vector and the feature extraction matrix, the first accumulate electronic data in the image database, calculating a matching region that is an image for use in matching from the second electronic data and the second feature coordinates, stored in the image database and the verification area And a step of collating a match with the first electronic data. A computer, a conversion step Ru to convert the optical data into electronic data, enter the first feature coordinates from a first electronic data converted in said converting step, the said first electronic data first A set of composite information with the coordinates of the feature points is divided into a plurality of distributions, and for each distribution, autocorrelation information, a plurality of the first electronic data, and a plurality of the first electronic data from the plurality of first electronic data. from the coordinates of the feature points to calculate the cross-correlation information, further, for estimating the second feature point coordinates of the second electronic data converted in said converting step from said autocorrelation information and the cross-correlation information step a first average vector calculated from the plurality of first electronic data obtained for each of the distribution, the plurality of the first Japanese determined for each of the distribution for calculating a feature extraction matrix for each of the distribution Second average vector and the output step of outputting the feature extraction matrix obtained for each distribution, the first average vector calculated on the basis distribution output in the output step of calculating from the point of coordinates, the A second average vector and the feature extraction matrix are stored in a correlation information database, and the second electronic data, the first average vector, the second average vector, and the feature extraction matrix are used to store the second average vector and the feature extraction matrix. estimating the coordinates of the feature points for each of the distribution, the first electronic data stored in the image database, said a second electronic data and images used in the matching from the second feature point coordinates match possible to perform the steps of calculating a region, a step of checking the coincidence between the stored first electronic data to the image database and the verification area, the Program which is characterized. Sample face image input means for inputting a sample face image as a sample, feature point input means for inputting feature points of the sample face image, and a collation face image for newly inputting a collation target face image for estimating the feature points The input means, the composite information of the sample face image and the feature point is divided into a plurality of face image distributions, the face image autocorrelation information of the sample face image, and the face image of the sample face image and the feature point The face image learning means for obtaining correlation information for each face image distribution, and the feature of the face image to be collated using the face image autocorrelation information and the face image cross-correlation information obtained for each face image distribution. A face image parameter estimating means for estimating a point, an image database for storing the sample face image, and an image used for matching from the coordinates of feature points of the matching target face image and the matching target face image. Calculating a matching area, the data matching apparatus comprising, a collating means for collating a match with the stored sample face images in the image database and the verification area. The face image learning means constructs a face image composition vector of a first teach vector obtained by vectorizing the sample face image and a face image composition vector of a second teach vector obtained by vectorizing feature points of the sample face image. A face image mixture distribution model in which a plurality of face image element distributions of vectors are aggregated is estimated, and the face image autocorrelation information belonging to each face image element distribution is obtained by obtaining a covariance matrix of the first teaching vector. The face image cross-correlation information is obtained by obtaining a covariance matrix of the first teaching vector and the second teaching vector,
  The face image parameter estimation means estimates the feature points of the face image to be collated using the face image autocorrelation information and the face image cross-correlation information belonging to each face image element distribution. Item 32. The data verification device according to Item 31.

Images