JP3594889B2 - Image processing method, image processing apparatus, and recording medium storing program for executing image processing method - Google Patents

Description

を用いる。この場合、平滑化処理部１５と微分処理部１６の処理を１つのフィルタで処理するフィルタの形状は、ガウス関数を微分した形で書き表すことができる。例えば、ｘ方向の偏微分は、
【数２】

であり、これが画像処理におけるフィルタ形状となる。ここでは、２方向の偏微分処理をグラジエントとして、以降の画像処理方法を述べる。
以下、ガウス関数のグラジエントは、
【数３】

であらわす。ここでフィルタは
【数４】

をそのまま使うのではなく、これを
【数５】

倍した
【数６】

を用いる。これは、単独のステップエッジについて、その出力値を一定にするためである。（定数倍を行わないと、平滑化が進む、すなわちσが大きくなるにつれ、エッジはつぶれていってしまう。）
この場合、ガウス関数による平滑化処理部１５と、２方向の一次偏微分処理による微分処理部１６の処理を組み合わせた画像処理方法は以下の式で示され、平滑化処理と微分処理とを同時に実行する。また、画像処理結果１８は、次式の計算結果として得られる。
【数７】

ここで、
【数８】

はディジタル画像であり、すでにｇ＝ＣＲＩというように反射率Ｒ、照度Ｉ、定数Ｃで表現されているとする。また「＊」は、畳み込み処理を意味する記号である。積分領域は対象とする画像の領域である。
ただし、以上の数式による説明は、画像を連続関数で置き換えて考えたものである。実際のディジタル画像の処理においては、画像はサンプリングされているため、信号は離散値であり、積分は総和Σの形で示される。
【００２２】
次に、パラメータ決定処理部１７におけるパラメータの値の具体的計算手順を示す。ここでは、処理のはじめの手順を対数変換（画素値変換）とする。この場合、与えられた画像（解析対象）をｇとすると、各σについてその結果のノルムを計算する。この例では、例えば
【数９】

を計算する。この結果は、ホワイトノイズや一様に一方方向に明るさが増す画像など、ある特殊な画像では、単純減少、単純増加となるが、これは例外的であり、ごく普通の物体が撮影されている場合は、必ず極大値をもつ。この時、最も大きな値を示すσを平滑化処理部１５における最適なパラメータとして選ぶ。
なお、以上の議論は、ガウス関数以外であってもパラメータにより平滑化を記述できる関数を用いれば、同じ議論が成り立つ。また、対数変換を行わない場合や順序が異なる場合も、同様に式を構成することができるので、同じ議論で必要なパラメータ値を得ることができる。これにより、最適な平滑化処理を定量的に得ることができる。
ここで、以上のような画像処理が、実際に何を行っていることになるのかを示しておく。これは、周波数帯域の特徴として考えるのが妥当である。はじめの対数変換は、物体の表面反射率と照度の積を和の形に変換するものである。次の平滑化処理、微分処理は、それぞれ高周波制限、低周波制限処理にあたる。よって２つの処理を行うことは、中程度の周波数への帯域制限処理（バンドパスフィルタ）である。この時、それぞれノイズ（主に高周波成分）、照明変動（主に低周波成分）を除去する。そして残る物体の表面の特徴をできるだけ残すことが可能となる。実際には、上述の方法で得られるパラメータの値は、物体の表面反射率の特徴を最も良く抽出する値であり、最も強調して利用する周波数帯域を定めていることになる。定量的にもこの値が、一般的な照明変動およびホワイトノイズに対する耐久性を最も得やすい。
【００２３】
また、上述の説明では、画素値変換処理部１４を設け、対数変換処理によりディジタル信号化された画像を、画像に記録された物体の表面反射率と照度の和で表される信号へ変換する場合を示したが、ガンマ補正値γ＜１なる係数を乗算されディジタル信号化された画像を画像処理する場合には、対数変換処理を近似的には省略できることを述べる。
ガンマ補正値γ係数は、入力の輝度に対し、出力を非線型にする時の係数である。ここで、出力画素値が輝度のγ乗（オフセットなど有り）と設定される。通例、画像を生成する際のγは１より小さいので、定性的にはあたかも対数変換と同様に輝度が大きくなるにつれ、その増加分に対する画素値の増加を減少させるという効果が得られる。従って、対数変換よりも軽微な演算量で同様な効果が得られる。
例えばテレビ映像では、コントラストが不足する部分をγ係数を用いて疑似的に広い帯域を表現しようとする方法を利用する。つまり、良く使われるγ＝１／２．２は、暗い部分のコントラストはあまり変化させず、明るい部分のコントラストを下げる効果を持つ。これは、近似的には対数変換と同じであり、従ってこのような場合には対数変換を省略しても、結果はほぼ同じものが得られる。具体的には、図１の画像変換処理部１４を省略することにあたる。
【００２４】
（第２の実施の形態）
次に、図２を用いて、第１の実施の形態による画像処理方法を用いた画像比較方法を、第２の実施の形態として説明する。
図２は、第２の実施の形態による画像比較装置の構成を説明するブロック図である。
図２において、符号２１は、画像比較装置へ入力された画像Ａである。符号２２は画像Ｂである。このうち、画像Ａ２１を辞書画像、画像Ｂ２２を入力画像と考える。すなわち、与えられた画像Ａ２１に対し、どれだけ画像Ｂ２２が類似しているかを計算することが目的である。
符号２３は、第１の実施の形態で説明した画素値変換処理部であって、画像Ａ２１に対する画素値変換処理部である。符号２４は、第１の実施の形態で説明した画素値変換処理部であって、画像Ｂ２２に対する画素値変換処理部である。
同様に、符号２５は、第１の実施の形態で説明した平滑化処理部であって、画素値変換処理部２３の出力に対する平滑化処理部である。符号２６は、第１の実施の形態で説明した平滑化処理部であって、画素値変換処理部２４の出力に対する平滑化処理部である。
同様に、符号２７は、第１の実施の形態で説明した微分処理部であって、平滑化処理部２５の出力に対する微分処理部である。符号２８は、第１の実施の形態で説明した微分処理部であって、平滑化処理部２６の出力に対する微分処理部である。
また、符号２９と符号３０は、それぞれ画像Ａ２１と画像Ｂ２２の画像処理結果であって、画像処理結果Ａ２９は微分処理部２７の、画像処理結果Ｂ３０は微分処理部２８の出力結果である。
符号３１は、画像処理結果Ａ２９と画像処理結果Ｂ３０とを比較する、画像比較処理部である。
符号３２は、画像比較処理部３１における画像比較結果である。
【００２５】
なお、図２においても第１の実施の形態と同様に、符号２１から符号３０における構成要素、あるいは信号は、順序を変えたり簡略化することが可能である。また、上述の画素値変換処理部２３と２４、平滑化処理部２５と２６、微分処理部２７と２８、画像比較処理部３１も第１の実施の形態と同様に、それぞれ専用のハードウェアにより実現されるものであってもよく、また、メモリおよびＣＰＵ（中央演算装置）により構成され、上記の各部の機能を実現するためのプログラムをメモリにロードして実行することにより、その機能を実現させるものであってもよい。
【００２６】
次に、本実施の形態の動作を説明する。
本実施の形態では、画像Ａ２１を基準としておき、画素値変換処理部２３と２４、平滑化処理部２５と２６、微分処理部２７と２８を用いて、第１の実施の形態で説明した画像処理方法により、画像Ａ２１の特徴を最も良く抽出するパラメータで画像Ａ２１、画像Ｂ２２の処理を行っておく。
画像比較処理部３１では、すでに画像１の特徴が最も良く得られているため、単純な方法で正確な比較を実現できる。
画像比較処理部３１における画像比較方法は、例えば、両画像の画素値の差分の和や、両画像の画素値の差の絶対値の２乗値、両画像の画素値の相関値等を計算し、その大小により、両画像の一致度を判断する方法が有効である。
また、画素値の相関値計算は、たとえ２枚の比較する画像が異なるＡ／Ｄ変換系により得られ、γ係数が異なっている場合でも、画素値変換処理部２３と２４を、簡単な対数変換処理とすれば良いという特徴を持つ。これは、相関の計算途中で、異なるγ係数の値の影響は分数の分母と分子に１回づつ現われるため、完全に除去されることによる。このため、次の第３の実施の形態で述べる画像マッチング方法の辞書画像と、比較される入力画像との系の違いを意識する必要がないことになる。これは、１つの「辞書」という画像を他の系を用いて撮影された画像へ適用できることを意味する。
【００２７】
（第３の実施の形態）
次に、図３を用いて、第２の実施の形態による画像比較方法を用いた画像マッチング方法を、第３の実施の形態として説明する。
図３は、第３の実施の形態による画像マッチング方法を説明する模式図である。
図３において、符号４１は、予め与えられる辞書画像である。符号４２は、入力画像であって、辞書画像４１より大きな画像である。図３における細かい一マスは、それぞれの画像の１ドットの大きさを示している。
辞書画像４１は、画像のずらしマッチングを行うための元画像であり、入力画像４２の一部と辞書画像４１との画像比較を行い、最も類似する結果が得られる位置を見つけるものである。これをここでは画像マッチングと呼ぶことにする。符号４３はマッチング位置Ａ、符号４４はマッチング位置Ｂを、それぞれ示す。
【００２８】
次に、本実施の形態の動作を説明する。
本実施の形態で説明する画像マッチング方法は、入力画像４２の中から辞書画像４１と同じ大きさの画像領域を順次選び（これを「ずらし」と表現する）、選ばれた領域と辞書画像４１とを用いて、第２の実施の形態で説明した画像比較方法により、画像比較を行う。この時、できるだけ細かいステップでマッチング位置を選ぶと、見落としがないマッチングが実現できる。しかし、これは処理量の増加を招くため、実際にはで見落としが無い程度できるだけ荒いステップでマッチング位置を選ぶ必要が生じる。このため、多くの場合、ステップを１ドットとしてマッチングが行なわれている。経験的に１ドットで不足する場合は、１ドットより細かいステップ（すなわちサブピクセル）を用いたマッチングが行われている。
微分処理および平滑化処理を経た結果の画像比較では、位置ずれが比較結果に与える影響を平滑化処理の度合いによって計算することができる。この結果は平滑化処理の度合いのみに依存し、画像のパターンそのものには依存しない。例えば平滑化処理をガウス関数で行い、かつ相関値０．８程度を十分な類似と判断する場合、σと位置ずれの許容度ｄにはおおよそｄ＜０．４４６σの関係が成り立つ。例えば、σが１．０以上であれば、サブピクセルを用いたずらしマッチングは不用で、１ドット単位のずらしマッチングで良いということが言える。これは、１ドット単位でずらしマッチングを行う場合のずれ量の最大値０．５ドットでも、相関値を概ね０．８以上を得ることができるという意味である。また逆にσが２．０以上の場合、すべてのドットを用いたずらしマッチングも不用である。これは見落としをなくすためという意味であり、勿論、より正確な位置決めが必要な場合には、細かなずらしマッチングを行うことを否定するものではない。
以上の直接的な効果は、σが大きい場合には画像を縮小した上でずらしマッチングを精度を落とさずに行うことができることである。たとえばσ＝３．０であれば、画像を１／３に縮小し、σ＝１．０を用いて１ドット単位のずらしマッチングを行えばよいことがわかる。これにより、無駄のない高速なマッチングが実現できる。
【００２９】
更に、第１の実施の形態で、既に示したように、σは最も強調して利用する周波数帯域を定めている。従って、利用しようとするσによって強調される周波数以上の周波数帯域は除去しても、画像比較には大きな影響を与えない。そのため、σが大きい場合、この値に従って不要な高周波成分の除去、すなわち画像の縮小を行っても十分な画像比較が実現できる。
縮小率は、例えばσ＝３．０の場合、画像を１／３に縮小してσ＝１．０を利用することができる。縮小率は、常に、
【数１０】

という関係が成り立てば良い。（上の例では、縮小率＝１／３である）
例えば、処理アルゴリズムの中に第１の実施の形態で説明した方法で選ばれた最適なσを適用して、縮小後のσを１．０に近い値とするようにすることができる。この場合、画像を縮小により、画像ずらしマッチングのサーチステップを常に１に保つことができるという意味を持つ。
例えばガウス関数では、画像をｎ倍に拡大してσを同じｎ倍にすれば、結果的にはじめと同じ平滑化の効果が得られる。これはガウス関数に限らず、一般的な平滑化で成り立つ。従って、決めたσが大きい場合、画像を縮小しても同じ効果が得られることになる。
【００３０】
なお、上述の第１から第３の実施の形態で説明した、画像から必要情報を取り出す画像処理、２つの画像の比較処理、画像のずらしマッチング処理は、それぞれ、その機能を実現するためのプログラムを、コンピュータ読みとり可能な記録媒体に記録して、この記録媒体に記録されたプログラムをコンピュータシステムに読み込ませ、実行することにより、上述の各画像処理における機能を実現しても良い。
【００３１】
ここで、上記「コンピュータシステム」とは、ＯＳや周辺機器等のハードウェアを含み、さらにＷＷＷ（Ｗｏｒｌｄ Ｗｉｄｅ Ｗｅｂ）システムを利用している場合であれば、ホームページ提供環境（あるいは表示環境）も含むものとする。また、「コンピュータ読みとり可能な記録媒体」とは、フロッピーディスク、光磁気ディスク、ＲＯＭ、ＣＤ−ＲＯＭ等の可搬媒体、コンピュータシステムに内蔵されるハードディスク等の記憶装置のことをいう。更に、「コンピュータ読みとり可能な記録媒体」とは、インターネット等のコンピュータネットワークや電話回線等の通信回線を介してプログラムを送出する場合のように、短時間の間、動的にプログラムを保持するもの（伝送媒体もしくは伝送波）、その場合のサーバやクライアントとなるコンピュータシステム内部の揮発性メモリのように、一定時間プログラムを保持しているものも含むものとする。
【００３２】
また、上記プログラムは、前述した機能の一部を実現するためのものであっても良く、更に前述した機能をコンピュータシステムに既に記憶されているプログラムとの組み合わせで実現できるもの、いわゆる差分ファイル（差分プログラム）であっても良い。
【００３３】
【発明の効果】
以上の如く本発明によれば、ディジタル信号化された画像の画像処理方法において、ディジタル信号化された画像を、物体の表面反射率と照度の和で表される信号へ変換する画素値変換処理と、画素値変換処理結果に、予め定められた平滑化を行う平滑化処理、及び低域の信号を軽減する微分処理と、平滑化処理における処理精度を、定量的に制御するパラメータ決定処理とを含む構成とした。
これにより、これまでのようなノウハウや実験により各種パラメータを設定するのはなく、定量的な画像処理、画像比較および画像ずらしマッチングを実現することが可能となる。
【００３４】
また、本発明は、パラメータ決定処理を、与えられるディジタル画像、もしくは画素値変換処理後の画像信号のノルムを計算する処理を含み、該ノルムが極大になるパラメータを、平滑化処理の処理精度を決定するパラメータの最適値とする構成とした。
これにより、画像の比較用パラメータを定量的に決定することで、各種外乱（照明変動、ノイズなどの影響）を除去し、物体そのものが持つ特徴を抽出することが可能となる。
更に、物体そのものが持つ特徴を抽出することにより、正確な（物体そのものの特徴を利用した）画像比較を実現することが可能となる。
同時に、画像の比較用パラメータを計算した結果、高周波成分が不要であるとの結論が出た場合、画像を縮小し、高速な画像比較を実現することが可能となる。
【００３５】
また、本発明は、ディジタル化された２枚の画像を比較して類似度を計算する画像比較方法において、２枚の画像に対し、それぞれ上記画像処理方法による画像処理と、画像処理後の結果を比較する画像比較処理とを含む構成とした。
これにより、外乱による画像の位置ずれ耐久性を予め予測できるため、どれだけ細かい位置ずれを考慮した画像比較を行う必要があるのかを予め決定でき、従って画像を利用した物体抽出を定量的に設計することが可能となる。
また、画像比較に用いる辞書の作成時にγ係数を考慮する必要がないため、異なるＡ／Ｄ変換系を経た画像を用いて、別のＡ／Ｄ変換系を利用した画像の画像比較も正確に実現することが可能となる。よって、画像マッチングを行う時場合、辞書画像として登録する画像の撮影系を意識する必要がない。
従って、与えられた画像の中から物体固有の特徴を効率的に取り出し、結果的に照明変動、ノイズなどの一般的な外乱に対する耐久性を持つ前処理（画像処理）方法を定量的な方法で得ると同時に、画像処理方法において、各種外乱に対する耐久性を持つバンドパスフィルタの設計方法を定量化することができるという効果が得られる。
【図面の簡単な説明】
【図１】第１の実施の形態による画像処理装置の構成を説明するブロック図である。
【図２】第２の実施の形態による画像比較装置の構成を説明するブロック図である。
【図３】第３の実施の形態による画像マッチング方法を説明する模式図である。
【符号の説明】
１１ シーン情報
１２ ディジタル画像生成処理部
１３ 入力画像
１４ 画素値変換処理部
１５ 平滑化処理部
１６ 微分処理部
１７ パラメータ決定処理部
１８ 画像処理結果
２１ 画像Ａ
２２ 画像Ｂ
２３ 画素値変換処理部
２４ 画素値変換処理部
２５ 平滑化処理部
２６ 平滑化処理部
２７ 微分処理部
２８ 微分処理部
２９ 画像処理結果Ａ
３０ 画像処理結果Ｂ
３１ 画像比較処理部
３２ 画像比較結果
４１ 辞書画像
４２ 入力画像
４３ マッチング位置Ａ
４４ マッチング位置Ｂ[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to pre-processing of an image, pre-processing for performing image comparison, or pre-processing for performing image shift matching and a design method thereof. In particular, the present invention relates to a method for controlling illumination fluctuation durability and noise durability and extracting necessary information from an image.
[0002]
[Prior art]
Conventionally, many techniques for comparing and matching images have been proposed.
The greatest challenge of the technology is to obtain durability against disturbances, for which a number of methods have been proposed.
[0003]
[Problems to be solved by the invention]
However, until now, only a method of setting various parameters is known experimentally, and a processing method and a comparison method after quantitatively setting parameters have not been known. For this reason, there is a problem that much know-how is required for designing image processing.
In particular, the smoothing part is almost always performed for minimum noise removal. Conventionally, these have been embedded as know-how in the processing, so that the specific processing method and the optimal parameter are used. There was a problem that the method of finding was not clear.
[0004]
The present invention has been made in view of the above problems, and has an image processing method capable of obtaining image processing having durability against disturbance, efficient image comparison, and image shift matching.And image processing equipmentThe purpose is to provide. More specifically, features specific to an object are efficiently extracted from a given image, and consequently a preprocessing (image processing) method that is durable against general disturbances such as illumination fluctuation and noise is quantified. Image processing method that quantifies the design method of a band-pass filter that is durable against various disturbances at the same timeImage processing equipment,And a recording medium on which a program for executing the image processing method is recorded.
[0005]
[Means for Solving the Problems]
In order to solve the above problems, the present invention provides a digital signalsignalAn image processing method for digitally processing the digitally converted image.signalIs converted into a signal represented by the sum of the surface reflectance and illuminance of the object recorded in the image, pixel value conversion processing,By specifying the degree of smoothing with a parameter,Given theimagesignalControl of the degree of removal of high frequency componentsSmoothing process and givenimageSignalLow frequency componentDifferential processing to reduce theDetermine the parametersAnd a parameter determination process to be performed.Converted image signalToOn the other hand,SmoothedlaterPerformed the differentiation process or performed the differentiation processlaterEither perform the smoothing process, or perform the smoothing process and the differentiation process collectively.A process for further restricting the frequency band is performed, and the parameter determining process determines a parameter at which the norm of the image signal after the process for restricting the frequency band is maximized as an optimal value.It is characterized by the following.
With the above configuration, it is possible to realize quantitative image processing, image comparison, and image shift matching without setting various parameters based on know-how and experiments as in the past.Further, by quantitatively determining the comparison parameter of the image, various disturbances (effects of illumination fluctuation, noise, and the like) can be removed, and the feature of the object itself can be extracted. Further, by extracting the features of the object itself, it is possible to realize accurate image comparison (using the features of the object itself). At the same time, if it is concluded that high-frequency components are unnecessary as a result of calculating the comparison parameter of the image, the image can be reduced and high-speed image comparison can be realized.
[0006]
The present invention provides the image processing method, wherein the pixel value conversion processing uses logarithmic conversion to convert the digital signal into an image.signalIs converted into a signal represented by the sum of the surface reflectance and the illuminance of the object recorded in the image.
[0007]
The present inventionA pixel value is set to the γ power of luminance using a gamma correction value γ where γ <1.Digital signal imagesignalIs an image processing method for digitally processingBy specifying the degree of smoothing with a parameter,Given theimagesignalControl of the degree of removal of high frequency componentsSmoothing process and givenimageSignalLow frequency componentDifferential processing to reduce theDetermine the parametersThe digitalized imagesignalToOn the other hand,SmoothedlaterPerformed the differentiation process or performed the differentiation processlaterEither perform the smoothing process, or perform the smoothing process and the differentiation process collectively.A process for further restricting the frequency band is performed, and the parameter determining process determines a parameter at which the norm of the image signal after the process for restricting the frequency band is maximized as an optimal value.It is characterized by the following.
[0008]
According to the present invention, in the above image processing method, the differentiation processing is independent two-dimensional primary partial differentiation processing.
[0009]
According to the present invention, in the image processing method described above, the smoothing process is performed by using a convolution process of a Gaussian function.
[0010]
The present inventionAn image comparison method for calculating a degree of similarity between images based on a comparison result obtained by comparing image signals of two digitalized images, wherein the method is performed on the image signals of the two images. It is characterized by including an image processing by an image processing method and an image comparison process of comparing image signals of two images after the image processing to obtain a comparison result.
With the above-described configuration, the durability of image displacement due to disturbance can be predicted in advance, so that it is possible to determine in advance how fine image displacement needs to be considered in order to perform image comparison. It is possible to design.
In addition, since it is not necessary to consider the γ coefficient when creating a dictionary used for image comparison, it is possible to accurately compare an image using another A / D conversion system using an image that has passed through a different A / D conversion system. It can be realized.
[0011]
The present inventionIn the above-described image comparison method, the image comparison process is characterized in that a comparison result of two digitized images is obtained as a magnitude of a correlation value of pixel values of the two images.
[0012]
The present invention is characterized in that in the above-described image comparison method, the image comparison processing obtains a comparison result of the two digitized images as a sum of pixel value differences between the two images.
[0013]
The present inventionIn the above-described image comparison method, the image comparison processing is characterized in that a comparison result of the two digitized images is obtained as a magnitude of a square value of an absolute value of a pixel value difference between the two images. .
[0014]
The present inventionAn image matching method for searching for an area similar to a predetermined dictionary image from an image larger than the dictionary image while shifting its position. The comparison process between the image obtained for each position and the dictionary image is performed by the image comparison method. The method is characterized by using a method.
[0015]
The present inventionA computer-readable recording medium in which a program for causing a computer to execute any of the above-described methods is recorded.
[0016]
The present inventionAn image processing apparatus for digitally processing an image signal converted into a digital signal, comprising: converting the image signal converted into a digital signal into a signal represented by a sum of surface reflectance and illuminance of an object recorded on an image. A pixel value converting means for performing the control, a smoothing means for quantitatively controlling the degree of removal of the high frequency component of the given image signal by designating the degree of smoothing with a parameter, A differential unit for reducing a frequency component, and a parameter determining unit for determining the parameter used in the smoothing unit. The image signal converted by the pixel value converting unit is processed by the smoothing unit. After performing the processing by the differentiating means, the processing by the smoothing means after performing the processing by the differentiating means, or by the smoothing means Or performing the processing by the differentiating means collectively, or performing the processing of limiting the frequency band, and the parameter determining means maximizes the norm of the image signal after the processing of limiting the frequency band It is characterized in that parameters are determined as optimal values.
The present invention relates to an image processing apparatus for digitally processing an image signal that has been converted into a digital signal with a pixel value set to the luminance raised to the power of γ using a gamma correction value γ satisfying γ <1. A smoothing means for quantitatively controlling the degree of removal of the high frequency components of the given image signal, a differentiating means for reducing the low frequency components of the given image signal, and Means for determining the parameters used by the means, wherein the image signal converted into a digital signal is processed by the smoothing means and then processed by the differentiating means, After performing the processing by the smoothing means, or by performing the processing by the smoothing means and the processing by the differentiating means collectively. Performs a process of limiting the number band, the parameter determining means is characterized by determining a parameter norm is maximum of the processed image signal to limit the frequency band as the optimum value.
The present invention is an image comparison device that calculates an image similarity based on a comparison result obtained by comparing image signals of two digitalized images, wherein the image signals of the two images are The image processing apparatus includes an image processing unit that performs image processing by the image processing method, and an image comparison unit that compares image signals of the two images after the image processing and obtains a comparison result.
[0017]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(First Embodiment)
First, with reference to FIG. 1, an image processing method for controlling illumination fluctuation durability and noise durability and extracting necessary information from an image will be described as a first embodiment.
FIG. 1 is a block diagram illustrating the configuration of the image processing apparatus according to the first embodiment.
In FIG. 1, reference numeral 11 denotes scene information of a given image, and the scene information obtained when photographed by a camera or the like is a function of the product of the illuminance on the scene and the object surface reflectance.
Reference numeral 12 denotes a digital image generation processing unit that generates a digital image, and includes a process of converting light into an electric signal, such as a CCD, and an A / D conversion process of an electric signal. Reference numeral 13 denotes an input image obtained by being digitized by the digital image generation processing unit 12.
Reference numeral 14 denotes a pixel value conversion processing unit that converts a digital signal image into a signal represented by the sum of the surface reflectance and the illuminance of an object recorded in the image. A signal represented by the product of the surface reflectance of the object and the illuminance recorded on the digital signalized image is converted into a signal represented by the sum of the surface reflectance of the object and the illuminance. This means that if the image is digitized in a general way, the pixel values of the image are already a function of the product of the surface reflectance of the object and the illuminance.
[0018]
Reference numeral 15 denotes a smoothing processing unit that smoothes the result converted by the pixel value conversion processing unit 14. In this embodiment, for the sake of simplicity, a case will be described in which a Gaussian function convolution process is used for the smoothing process. However, the present invention is not limited to this. May be used. However, when the pixel value conversion processing unit 14 described later is omitted, the condition is that the degree of smoothing can be parameterized. In the case of high-frequency cut, if the specified frequency or higher is completely cut, the specified frequency can also be a parameter. Reference numeral 16 denotes a differential processing unit. In this embodiment, a case will be described where partial orthogonal differentiation processing is performed in two orthogonal directions. However, the operation to be performed is not limited to this. Alternatively, higher order differentiation processing may be performed.
Reference numeral 17 denotes a parameter determination processing unit that determines a parameter used in the smoothing processing unit 15. The parameters may be determined in advance, and the norm of the image signal after the pixel value conversion processing of the given digital image is calculated, and the parameter at which the norm becomes a maximum is determined by the processing accuracy of the smoothing processing unit 15. May be determined as the optimal value of the parameter for determining
Reference numeral 18 denotes the image processing result obtained by performing the above processing.
[0019]
The order of the pixel value conversion processing unit 14 and the smoothing processing unit 15 is the same even if this processing is reversed.
In the above description, the smoothing processing unit 15 and the differentiation processing unit 16 have been described as a series of separate components. However, in image processing, changing the order of these two processes may also be performed as a single process. It is possible to end. This is because both the smoothing and the differentiation are mathematically convolutions, and the convolution can be performed in a reversible manner, so that the order can be changed. It is also possible to process as.
[0020]
The above-described digital image generation processing unit 12, pixel value conversion processing unit 14, smoothing processing unit 15, differentiation processing unit 16, and parameter determination processing unit 17 are each realized by dedicated hardware. It is also configured with a memory and a CPU (Central Processing Unit), and realizes the functions by loading the programs for realizing the functions of the above-described units into the memory and executing the programs. You may.
[0021]
Next, the operation of the present embodiment will be described using a case where convolution processing of a Gaussian function is used for the smoothing processing as an example, and a method of determining the value of the smoothing parameter will be described.
First, when the scene information 11 having a function of the product of the illuminance of the given image to the scene and the reflectance of the object surface is input to the image processing apparatus of the present embodiment, , The input image 13 is input to the pixel value conversion processing unit 14.
The pixel value conversion processing unit 14 converts the digital signal image into a signal represented by the sum of the surface reflectance and the illuminance of the object recorded in the image by a logarithmic conversion process.
The output of the pixel value conversion processing unit 14 is smoothed in the smoothing processing unit 15. In the case where a Gaussian function is used in the image processing, the image is two-dimensional.
(Equation 1)

Is used. In this case, the shape of a filter that performs the processing of the smoothing processing unit 15 and the differentiation processing unit 16 with one filter can be expressed in a form obtained by differentiating a Gaussian function. For example, the partial derivative in the x direction is
(Equation 2)

Which is the filter shape in the image processing. Here, the following image processing method will be described with the partial differential processing in two directions as a gradient.
In the following, the gradient of the Gaussian function is
(Equation 3)

It represents. Where the filter is
(Equation 4)

Instead of using
(Equation 5)

Multiplied
(Equation 6)

Is used. This is to make the output value of a single step edge constant. (If the constant multiplication is not performed, the edges are crushed as the smoothing proceeds, that is, as σ increases.)
In this case, an image processing method combining the processing of the smoothing processing unit 15 by the Gaussian function and the processing of the differentiation processing unit 16 by the two-dimensional primary partial differentiation processing is represented by the following equation. Execute. The image processing result 18 is obtained as a calculation result of the following equation.
(Equation 7)

here,
(Equation 8)

Is a digital image, which is already represented by the reflectance R, the illuminance I, and the constant C as g = CRI. “*” Is a symbol indicating convolution processing. The integration area is the area of the target image.
However, the description using the above mathematical expressions is based on the assumption that images are replaced with continuous functions. In actual digital image processing, since the image is sampled, the signal is a discrete value, and the integral is shown in the form of a sum 総.
[0022]
Next, a specific calculation procedure of the parameter value in the parameter determination processing unit 17 will be described. Here, the first procedure of the process is logarithmic conversion (pixel value conversion). In this case, assuming that a given image (analysis target) is g, the norm of the result is calculated for each σ. In this example, for example
(Equation 9)

Is calculated. The result is a simple decrease or increase in some special images, such as white noise or an image with uniformly increasing brightness in one direction. If they do, they always have a local maximum. At this time, σ indicating the largest value is selected as an optimal parameter in the smoothing processing unit 15.
Note that the above discussion holds true even if a function other than the Gaussian function can be used to describe smoothing using parameters. Further, even when the logarithmic conversion is not performed or the order is different, the formula can be similarly configured, so that the necessary parameter value can be obtained by the same discussion. Thereby, the optimal smoothing process can be obtained quantitatively.
Here, what the above image processing is actually doing is shown. It is appropriate to consider this as a characteristic of the frequency band. The first logarithmic transformation is to transform the product of the surface reflectance of the object and the illuminance into a sum form. The following smoothing processing and differentiation processing correspond to high frequency limiting processing and low frequency limiting processing, respectively. Therefore, performing the two processes is a band limiting process (bandpass filter) for a medium frequency. At this time, noise (mainly high frequency components) and illumination fluctuation (mainly low frequency components) are removed. Then, it is possible to leave as much as possible the surface features of the remaining object. Actually, the value of the parameter obtained by the above method is a value that best extracts the characteristics of the surface reflectance of the object, and determines the frequency band to be used with the most emphasis. Quantitatively, this value is most likely to provide durability against general illumination fluctuation and white noise.
[0023]
Further, in the above description, the pixel value conversion processing unit 14 is provided, and the image converted into a digital signal by the logarithmic conversion processing is converted into a signal represented by the sum of the surface reflectance and the illuminance of the object recorded in the image. Although a case has been described, it is described that when an image converted into a digital signal by multiplying by a coefficient satisfying a gamma correction value γ <1 is subjected to image processing, logarithmic conversion processing can be omitted approximately.
The gamma correction value γ coefficient is a coefficient for making the output non-linear with respect to the input luminance. Here, the output pixel value is set to the γ-th power of the luminance (there is an offset or the like). Usually, since γ at the time of generating an image is smaller than 1, qualitatively, the effect of decreasing the increase in pixel value with respect to the increase as the luminance increases as in logarithmic conversion is obtained. Therefore, a similar effect can be obtained with a smaller calculation amount than the logarithmic conversion.
For example, in the case of a television image, a method of expressing a portion where the contrast is insufficient by using a γ coefficient to express a pseudo wide band is used. That is, γ = 1 / 2.2, which is often used, has an effect of lowering the contrast of a bright part without changing the contrast of a dark part so much. This is approximately the same as the logarithmic transformation, so that in such a case, omitting the logarithmic transformation will yield almost the same result. Specifically, it corresponds to omitting the image conversion processing unit 14 in FIG.
[0024]
(Second embodiment)
Next, an image comparison method using the image processing method according to the first embodiment will be described as a second embodiment with reference to FIG.
FIG. 2 is a block diagram illustrating the configuration of the image comparison device according to the second embodiment.
In FIG. 2, reference numeral 21 denotes an image A input to the image comparison device. Reference numeral 22 denotes an image B. Among them, the image A21 is considered as a dictionary image and the image B22 is considered as an input image. That is, the purpose is to calculate how similar the image B22 is to the given image A21.
Reference numeral 23 denotes the pixel value conversion processing unit described in the first embodiment, which is a pixel value conversion processing unit for the image A21. Reference numeral 24 denotes the pixel value conversion processing unit described in the first embodiment, which is a pixel value conversion processing unit for the image B22.
Similarly, reference numeral 25 denotes the smoothing processing unit described in the first embodiment, which is a smoothing processing unit for the output of the pixel value conversion processing unit 23. Reference numeral 26 denotes the smoothing processing unit described in the first embodiment, which is a smoothing processing unit for the output of the pixel value conversion processing unit 24.
Similarly, reference numeral 27 denotes the differential processing unit described in the first embodiment, which is a differential processing unit for the output of the smoothing processing unit 25. Reference numeral 28 denotes the differential processing unit described in the first embodiment, which is a differential processing unit for the output of the smoothing processing unit 26.
Reference numerals 29 and 30 denote image processing results of the images A21 and B22, respectively. The image processing result A29 is an output result of the differentiation processing unit 27, and the image processing result B30 is an output result of the differentiation processing unit 28.
Reference numeral 31 denotes an image comparison processing unit that compares the image processing result A29 and the image processing result B30.
Reference numeral 32 denotes an image comparison result in the image comparison processing unit 31.
[0025]
Note that, in FIG. 2, as in the first embodiment, the order of components or signals denoted by reference numerals 21 to 30 can be changed or simplified. The pixel value conversion processing units 23 and 24, the smoothing processing units 25 and 26, the differentiation processing units 27 and 28, and the image comparison processing unit 31 are also implemented by dedicated hardware similarly to the first embodiment. The functions may be realized by a memory and a CPU (Central Processing Unit), and a program for realizing the function of each unit described above is loaded into the memory and executed. May be performed.
[0026]
Next, the operation of the present embodiment will be described.
In the present embodiment, the image A21 is used as a reference, and the pixel value conversion processing units 23 and 24, the smoothing processing units 25 and 26, and the differentiation processing units 27 and 28 are used for the image described in the first embodiment. According to the processing method, the processing of the image A21 and the image B22 is performed in advance by using the parameters that best extract the characteristics of the image A21.
In the image comparison processing unit 31, since the characteristics of the image 1 have already been best obtained, accurate comparison can be realized by a simple method.
The image comparison method in the image comparison processing unit 31 calculates, for example, the sum of the pixel values of the two images, the square value of the absolute value of the difference between the pixel values of the two images, the correlation value of the pixel values of the two images, and the like. However, a method of judging the degree of coincidence between the two images based on the magnitude is effective.
In addition, the pixel value correlation value calculation is performed by a simple logarithmic calculation even if two images to be compared are obtained by different A / D conversion systems and the γ coefficients are different. There is a feature that conversion processing may be performed. This is because during the calculation of the correlation, the influence of the different γ coefficient values appears once in the denominator and the numerator of the fraction, and is completely removed. For this reason, it is not necessary to be aware of the difference between the dictionary image of the image matching method described in the third embodiment and the input image to be compared. This means that one “dictionary” image can be applied to an image captured using another system.
[0027]
(Third embodiment)
Next, an image matching method using the image comparison method according to the second embodiment will be described as a third embodiment with reference to FIG.
FIG. 3 is a schematic diagram illustrating an image matching method according to the third embodiment.
In FIG. 3, reference numeral 41 is a dictionary image given in advance. Reference numeral 42 denotes an input image, which is an image larger than the dictionary image 41. Each fine square in FIG. 3 indicates the size of one dot of each image.
The dictionary image 41 is an original image for performing image shift matching, and performs image comparison between a part of the input image 42 and the dictionary image 41 to find a position where the most similar result is obtained. This is referred to herein as image matching. Reference numeral 43 indicates a matching position A, and reference numeral 44 indicates a matching position B.
[0028]
Next, the operation of the present embodiment will be described.
In the image matching method described in the present embodiment, an image area having the same size as the dictionary image 41 is sequentially selected from the input image 42 (this is expressed as “shift”), and the selected area and the dictionary image 41 are selected. , Image comparison is performed by the image comparison method described in the second embodiment. At this time, if the matching position is selected in as small steps as possible, matching without oversight can be realized. However, this leads to an increase in the amount of processing, so that it is necessary to select a matching position in steps as coarse as possible without actually overlooking. Therefore, in many cases, matching is performed with one step as one dot. If one dot is empirically insufficient, matching using steps smaller than one dot (that is, sub-pixels) is performed.
In the image comparison of the result after the differentiation processing and the smoothing processing, the influence of the positional shift on the comparison result can be calculated based on the degree of the smoothing processing. This result depends only on the degree of the smoothing process, not on the image pattern itself. For example, when the smoothing process is performed using a Gaussian function and a correlation value of about 0.8 is determined to be sufficiently similar, a relationship of approximately d <0.446σ is established between σ and the tolerance d of positional deviation. For example, if σ is 1.0 or more, it can be said that the shift matching using the sub-pixel is unnecessary and the shift matching in units of one dot is sufficient. This means that a correlation value of about 0.8 or more can be obtained even with a maximum value of 0.5 dots of the shift amount when performing shift matching in units of one dot. On the other hand, when σ is 2.0 or more, it is unnecessary to perform the shift matching using all the dots. This means that oversight is eliminated, and, of course, when more accurate positioning is required, it is not denied that fine shift matching is performed.
The direct effect described above is that, when σ is large, the image can be reduced and the shift matching can be performed without lowering the accuracy. For example, if σ = 3.0, it can be seen that the image should be reduced to 1/3 and shifted matching in units of one dot should be performed using σ = 1.0. Thus, high-speed matching without waste can be realized.
[0029]
Further, in the first embodiment, as already described, σ determines the frequency band to be used with the most emphasis. Therefore, even if the frequency band higher than the frequency emphasized by σ to be used is removed, the image comparison is not largely affected. Therefore, when σ is large, sufficient image comparison can be realized even if unnecessary high-frequency components are removed according to this value, that is, the image is reduced.
When the reduction ratio is, for example, σ = 3.0, the image can be reduced to １ ／ and σ = 1.0 can be used. The reduction ratio is always
(Equation 10)

It is only necessary that the relationship be established. (In the above example, the reduction ratio is 1/3)
For example, by applying the optimum σ selected by the method described in the first embodiment to the processing algorithm, σ after reduction can be set to a value close to 1.0. In this case, by reducing the image, it means that the search step of the image shift matching can always be kept at 1.
For example, in the case of a Gaussian function, if the image is enlarged n times and σ is made the same n times, the same smoothing effect as the first is obtained. This is not limited to the Gaussian function, but is achieved by general smoothing. Therefore, when the determined σ is large, the same effect can be obtained even if the image is reduced.
[0030]
The image processing for extracting necessary information from an image, the comparison processing of two images, and the image shift matching processing described in the first to third embodiments are programs for realizing the functions. May be recorded on a computer-readable recording medium, and a program recorded on the recording medium may be read and executed by a computer system to realize the functions of the above-described image processing.
[0031]
Here, the “computer system” includes hardware such as an OS and peripheral devices, and further includes a homepage providing environment (or display environment) if a WWW (World Wide Web) system is used. Shall be considered. The “computer-readable recording medium” refers to a portable medium such as a floppy disk, a magneto-optical disk, a ROM, a CD-ROM, and a storage device such as a hard disk built in a computer system. Further, a “computer-readable recording medium” refers to a medium that dynamically stores a program for a short time, such as when a program is transmitted through a computer network such as the Internet or a communication line such as a telephone line. (Transmission medium or transmission wave), in which case a program holding a program for a certain period of time, such as a volatile memory in a computer system serving as a server or a client, is also included.
[0032]
Further, the program may be for realizing a part of the above-mentioned functions, and may be a program for realizing the above-mentioned functions in combination with a program already stored in the computer system, that is, a so-called difference file ( Difference program).
[0033]
【The invention's effect】
As described above, according to the present invention, in the image processing method for a digital signal image, a pixel value conversion process for converting the digital signal image into a signal represented by the sum of the surface reflectance and the illuminance of the object And a pixel value conversion processing result, a smoothing processing for performing a predetermined smoothing, a differentiation processing for reducing low-frequency signals, and a parameter determination processing for quantitatively controlling the processing accuracy in the smoothing processing. .
As a result, it is possible to realize quantitative image processing, image comparison, and image shift matching without setting various parameters based on know-how and experiments as in the past.
[0034]
In addition, the present invention includes a process of calculating a norm of a given digital image or an image signal after a pixel value conversion process in the parameter determination process. The configuration is such that the parameters to be determined are optimal values.
This makes it possible to quantitatively determine the comparison parameter of the image, remove various disturbances (effects of illumination fluctuation, noise, and the like), and extract a feature of the object itself.
Further, by extracting the features of the object itself, it is possible to realize accurate image comparison (using the features of the object itself).
At the same time, if it is concluded that the high-frequency component is unnecessary as a result of calculating the comparison parameter of the image, the image can be reduced and high-speed image comparison can be realized.
[0035]
According to the present invention, there is provided an image comparison method for comparing two digitized images to calculate a degree of similarity, wherein each of the two images is subjected to image processing by the image processing method, and a result after the image processing. And an image comparison process for comparing.
This makes it possible to predict in advance the durability of image misalignment due to external disturbances, so that it is possible to determine in advance how small image misalignment needs to be considered, and therefore to quantitatively design object extraction using images. It is possible to do.
In addition, since it is not necessary to consider the γ coefficient when creating a dictionary used for image comparison, it is possible to accurately compare an image using another A / D conversion system using an image that has passed through a different A / D conversion system. It can be realized. Therefore, when performing image matching, it is not necessary to be conscious of the photographing system of an image registered as a dictionary image.
Therefore, features unique to an object are efficiently extracted from a given image, and consequently a preprocessing (image processing) method having durability against general disturbances such as illumination fluctuation and noise is quantitatively performed. At the same time, in the image processing method, there is obtained an effect that a design method of a bandpass filter having durability against various disturbances can be quantified.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus according to a first embodiment.
FIG. 2 is a block diagram illustrating a configuration of an image comparison device according to a second embodiment.
FIG. 3 is a schematic diagram illustrating an image matching method according to a third embodiment.
[Explanation of symbols]
11 Scene information
12 Digital image generation processing unit
13 Input image
14 Pixel value conversion processing unit
15 Smoothing processing unit
16 Differential processing unit
17 Parameter determination processing unit
18 Image processing results
21 Image A
22 Image B
23 Pixel value conversion processing unit
24 Pixel value conversion processing unit
25 Smoothing processing unit
26 Smoothing processing unit
27 Differential processing unit
28 Differential processing unit
29 Image processing result A
30 Image processing result B
31 Image comparison processing unit
32 Image comparison results
41 Dictionary image
42 Input image
43 Matching position A
44 Matching position B

Claims (14)

An image processing method for digitally processing an image signal converted into a digital signal ,
The digital signal of image signals, the pixel value conversion process for converting into a signal represented by the sum of the surface reflectance and the illuminance of the recorded object image,
A smoothing process for quantitatively controlling the degree of removal of high frequency components of a given image signal by designating the degree of smoothing with a parameter ,
Differential processing for reducing low frequency components of a given image signal,
A parameter determination process for determining the parameter in the smoothing process,
Including
For the converted image signal in the pixel value conversion process, whether to perform the differential processing after performing the smoothing process, or performs the smoothing process after performing the differential process, and the smoothing process It performs a process of limiting the more frequency bands to the or collectively performing differentiation processing, either,
The image processing method according to claim 1, wherein in the parameter determining process, a parameter at which the norm of the image signal after the process of limiting the frequency band is maximized is determined as an optimum value . The pixel value conversion process includes:
Using logarithmic transformation, an image according to the digital signal of image signals, to claim 1, wherein the conversion to the signal represented by the sum of the surface reflectance and the illuminance of the recorded object in the image Processing method. An image processing method for digitally processing an image signal that has been converted into a digital signal by setting a pixel value to a power of γ using a gamma correction value γ satisfying γ <1 ;
A smoothing process for quantitatively controlling the degree of removal of high frequency components of a given image signal by designating the degree of smoothing with a parameter ,
Differential processing for reducing low frequency components of a given image signal,
A parameter determination process for determining the parameter in the smoothing process,
Including
For the digital signal of image signals, wherein whether to perform differential processing after the smoothing processing, whether to perform the smoothing process after performing the differential processing, the differential processing and the smoothing processing or collectively performed, performs a process of limiting the more frequency bands in any one of
The image processing method according to claim 1, wherein in the parameter determining process, a parameter at which the norm of the image signal after the process of limiting the frequency band is maximized is determined as an optimum value . The differentiation processing is
The image processing method according to any one of claims 1 to 3, wherein the first partial differentiation processing is performed in two independent directions. The smoothing process includes:
The image processing method according to any one of claims 1 to 4, wherein the smoothing is performed using a convolution process of a Gaussian function. An image comparison method for calculating a degree of similarity between images based on a comparison result of comparing image signals of two images converted into a digital signal ,
6. Image processing by the image processing method according to claim 1, which is performed on image signals of the two images.
An image comparison process of comparing image signals of the two images after the image processing to obtain a comparison result;
An image comparison method comprising: The image comparison process includes:
The image comparison method according to claim 6 , wherein a comparison result of the two digitized images is obtained as a magnitude of a correlation value between pixel values of the two images. The image comparison process includes:
7. The image comparison method according to claim 6 , wherein a comparison result of the two digitized images is obtained as a magnitude of a sum of pixel value differences between the two images. The image comparison process includes:
7. The image comparison method according to claim 6 , wherein a comparison result of the two digitized images is obtained as a magnitude of a square value of an absolute value of a pixel value difference between the two images. . An image matching method for searching for a region similar to a predetermined dictionary image while shifting its position from an image larger than the dictionary image,
The comparison processing between the image obtained for each position and the dictionary image includes:
An image matching method using the image comparison method according to claim 6 . A computer-readable recording medium on which a program for causing a computer to execute the method according to claim 1 is recorded. An image processing device for digitally processing an image signal converted into a digital signal,
Pixel value conversion means for converting the digitalized image signal into a signal represented by the sum of the surface reflectance and illuminance of the object recorded in the image,
A smoothing means for quantitatively controlling a degree of removal of a high frequency component of a given image signal by designating a degree of smoothing with a parameter,
Differentiating means for reducing low-frequency components of a given image signal;
Parameter determining means for determining the parameters used in the smoothing means,
With
For the image signal converted by the pixel value converting means, the processing by the differentiating means is performed after the processing by the smoothing means, or the processing by the smoothing means is performed after the processing by the differentiating means. Perform, or perform the processing by the smoothing means and the processing by the differentiating means collectively, perform the processing of limiting the frequency band by either,
The parameter determining means determines, as an optimal value, a parameter at which the norm of the image signal after the processing for limiting the frequency band is maximized.
An image processing apparatus comprising: An image processing apparatus for digitally processing an image signal that has been converted into a digital signal with a pixel value set to the power of γ using a gamma correction value γ satisfying γ <1,
A smoothing means for quantitatively controlling a degree of removal of a high frequency component of a given image signal by designating a degree of smoothing with a parameter,
Differentiating means for reducing low-frequency components of a given image signal;
Parameter determining means for determining the parameters used in the smoothing means,
With
For the digitalized image signal, whether to perform the processing by the differentiating unit after performing the processing by the smoothing unit, or to perform the processing by the smoothing unit after performing the processing by the differentiating unit, Either the processing by the smoothing means and the processing by the differentiating means are performed collectively, or the processing of limiting the frequency band is performed,
The parameter determining means determines, as an optimal value, a parameter at which the norm of the image signal after the processing for limiting the frequency band is maximized.
An image processing apparatus comprising: An image comparison device that calculates a similarity between images based on a comparison result obtained by comparing image signals of two images that have been converted into digital signals,
An image processing unit that performs image processing on the image signals of the two images by the image processing method according to any one of claims 1 to 5,
Image comparing means for comparing the image signals of the two images after the image processing to obtain a comparison result;
An image comparison device comprising:

Images