JP4231279B2 - Digital image processing device - Google Patents

Description

【００４２】
上式を重みとして、例えば境界線として直線 （未定係数a, b）を仮定した場合には、次のような重み付き残差平方和を求める。
【００４３】
【数２】

【００４４】
上式で求められた重み付き残差平方和を最小とする未定係数(a, b)を決定することにより、境界線である直線が得られる。Ｓ２０６は、上記Ｓ２０４からＳ２０５、までの境界線確定作業により、画像座標を確定したいＩＣＰもしくはサブＩＣＰに隣接する境界線を全て確定したか否かを判定し、まだ全ての境界線を確定していなければ、全て確定するまでＳ２０４〜Ｓ２０５を繰り返す。
【００４５】
Ｓ２０６において、座標を求めたいＩＣＰもしくはサブＩＣＰに隣接する全境界線が確定できているか否かを判定する。確定ができていれば、どの隣接する境界線も求めたいＩＣＰ、サブＩＣＰを必ず通過するので、Ｓ２０４及びＳ２０５で求めた隣接する境界線の１交点をＩＣＰ（あるいは、サブＩＣＰ）として決定する（Ｓ２０７）。図１２は、建物の隅に設定したサブＧＣＰに対応するサブＩＣＰを、S205で求めた建物の壁面に相当する境界線１３（白い破線で示される）の交点（隅）をサブＩＣＰ１４として決定した様子を示す図である。図１２に示すように、白い破線で明示された建物のエッジが明瞭に抽出されていることがわかる。
【００４６】
S208において、幾何歪み補正を行いたい画像に含まれている全てのＩＣＰと、サブＩＣＰと、の画像座標を求めたか否かを判定し、まだ求められていないＩＣＰあるいはサブＩＣＰがあればＳ２０４に戻り、同じ手順により座標を求める。Ｓ２０７のように、画像基準点の画像座標(u, v)を画素ではなく境界線の交点として求めることにより、サブ画素レベル（０．１ピクセル以下）で座標を求めることが可能となり、得られる基準点の位置精度は、画素サイズよりも高い精度で求めることができる。Ｓ２００の過程において、画像基準点の画像座標(u, v)を求めることにより、得られる基準点の位置座標の誤差は、Ｓ２０２の画素分割時に分割数の逆数に比例して、例えば図１０においては、位置誤差は１／２に減少する。Ｓ２０７において基準点を交点として求めることにより、誤差は１／１０以下に減少する。したがってＳ２００処理全体を通して、基準点の位置精度は、オリジナル画像により基準点を求めた場合の少なくとも１０倍以上（図１１の場合２０倍程度）まで向上させることができる。
【００４７】
図８に示すＳ３００の第３段階「幾何補正式（座標変換式）の決定」では、Ｓ２００で求めたＩＣＰ及びサブＩＣＰ群を利用して衛星画像座標系を地図座標系に変換する変換式を求め、最終的に画像を地図座標系に重ね合わせる処理を行う。まず処理Ｓ３０１で、Ｓ２００において座標を求めたＩＣＰのうちから１点を選び出し、そのＩＣＰに関連付けられるサブＩＣＰ群の画像座標情報と、各々に対応すべきサブＧＣＰ群の地図座標情報（緯度経度及び高度）と、に基づいて、画像座標(u, v)と地図座標(x, y)、もしくは、標高zを考慮に入れた座標(x, y, z)を対応付ける変換式を求める。
【００４８】
図１３に、座標変換系の概念図を示す。サブＩＣＰの画像座標(u, v)、対応するサブＧＣＰの地図座標(x, y, z)とし、画像カラム方向の変換式をF(x, y, z)、ライン方向の変換式をG(x, y,z)とすると、地図座標に対応する画像座標は以下の式で表される。
u = F(x, y, z)
v = G(x, y, z)
【００４９】
上記の関係式に基づいて、地図座標系１７のｘ−ｙ座標上の点が座標変換され、画像座標系１６のｕ−ｖ座標上の点として求められる。したがって、この逆変換を行う式を求める事で、画像座標系１６のｕ−ｖ座標上の点を地図座標系１７のｘ−ｙ座標上に正確に戻す事が出来る。
【００５０】
上記文献などによれば、座標変換式として一般的に用いられるには、▲１▼ヘルマート変換（縮尺歪みや回転歪みの補正、原点移動に用いる）、▲２▼アフィン変換（スキュー歪みの補正、線形変換に用いる）、▲３▼疑似アフィン変換（線形ねじれ変換に用いる）、▲４▼二／三次元射影変換（直行型・放物型歪みの補正に用いる）、▲５▼二次元射影変換（四辺形歪みの補正に用いる）、▲６▼三次元射影変換（射影歪みの補正に用いる）、▲７▼高次多項式変換（高次歪みの補正に用いる）等がある。これらの変換式を以下に示す。
【００５１】
▲１▼ヘルマート変換（未知変数a, b, c, d）
【数３】

【００５２】
▲２▼アフィン変換（未知変数a, b, c, d, e, f）
【数４】

【００５３】
▲３▼疑似アフィン変換（未知変数a1, a2, a3, a4, b1, b2, b3, b4）
【数５】

【００５４】
▲４▼二次元等角変換（未知変数a, b, c, d, e, f）
【数６】

【００５５】
▲５▼二次元射影変換（未知変数a1, a2, a3, a4, a5, a6, a7, a8）
【数７】

【００５６】
▲６▼三次元射影変換（未知変数a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11）
【数８】

【００５７】
▲７▼高次多項式変換（未知変数aij, bij）
【数９】

【００５８】
単純な画像の平行移動や回転・拡大縮小しか必要がなく、変換したい局所において変換の前後で像の形が相似であるような場合にはヘルマート変換が用いられ、線形移動と平行移動が合成されたようにx, ｙを独立に線形変換させたい場合にはアフィン変換が用いられる。また、より複雑な歪み、例えばxy項のような捩れを伴う変換が必要な場合には、疑似アフィン変換が用いられる（例えば、大林成行、小島尚人編著「実務者のためのリモートセッシング」、フジ・テクノシステム、２００２年、ｐ．１２２参照）。
この文献によれば、システム補正済みの衛星画像を入力データとして幾何学的歪み補正を行う場合、ヘルマート変換とアフィン変換による補正だけで良い場合が多い。
【００５９】
画像の歪みの形態から判断して最も適当だと予想される変換式を用い、サブＩＣＰ群の画像上の座標値と測定で求められたサブＧＣＰ群の緯度経度・高度から、最小二乗法により仮定した変換式の未定係数を求める。例えば、地表上では正方形の物体が衛星画像上では菱形に歪んで写っている場合、座標変換式としてアフィン変換を仮定して、サブＧＣＰの地表座標(xi, yi)、(i = 1, …, n)とそれに対応するサブＩＣＰの画像座標値(ui, vi)とすると、理論式と観測値との差の二乗二乗を加え合わせた次式で、Q1及びQ2が最小となるように(a, b, c, d, e, f)を決めれば良い。Q1及びQ2のそれぞれについて、(a, b, c)、(d, e, f)で偏微分をした結果が0に等しいとして以下の式により求めることが出来る。
【００６０】
【数１０】

【００６１】
Ｓ３０１において、最小二乗法を用いて未定係数を決定する際、極端に二乗二乗誤差が大きくなるサブＩＣＰ(um, vm)−サブＧＣＰ (xm, ym) (mは、1〜nの内の任意の値)の組については、サブＧＣＰの測定エラーやサブＩＣＰの選定ミスの可能性が高いため、未定係数を求める際に標本から除外し、(ui, vi, xi, yi) ( i = 1, …, m-1, m+1, …, n)のn-1個の標本から再度未知変数を求めることにより、未定係数の推定精度を高めることが出来る。
【００６２】
Ｓ３０２で示すように、局所的な座標系変換式でＩＣＰを地図座標系に変換する。Ｓ３０３において、Ｓ３０１で求めた座標系変換式が十分な精度か否かをＩＣＰの画像座標とＧＣＰの地図座標及び高度情報とを用いて判定する。Ｓ３０１で求めた変換式を用いて、画像座標系のＩＣＰ(u, v)を地図座標系へ変換した座標(x’, y’)と、実際のＧＣＰの地図座標(x, y)と、でどれだけの距離が離れているかを誤差として求め、その誤差が期待する変換精度（例えば、画素分割処理後の画像で１．０ピクセル以下など）に到達している否かを基準にして行う。
【００６３】
Ｓ３０３において、座標変換に伴う誤差が期待値より悪い場合には、Ｓ３０１において仮定した変換式が適切でなかったと推定できるため、Ｓ３０１に戻り、再度座標変換式を検討し未定係数を決定する。設定した変換精度を満たすまでＳ３０１からＳ３０３までの過程を繰り返す。
【００６４】
いずれの変換式を用いても設定した精度が得られない場合には、設定した精度が高すぎるか、或いは、設定した精度を得るためには用意したサブＧＣＰ群が適切ではなかったかのいずれかであると推定され、再度Ｓ１００のＧＣＰ・サブＧＣＰ群の測量の過程に立ち戻ってＧＣＰの取得計画から練り直し、再度処理を行う。
【００６５】
Ｓ３０３で座標変換に伴う誤差が期待値を満たしていた場合には、Ｓ３０１で求めた地図座標系変換の逆変換を行うことで、ＧＣＰが画像座標系上に投影される地点 (u’, v’)をＧＣＰに対応する真のＩＣＰとして扱う。
【００６６】
以上のように、Ｓ３０１からＳ３０４までの処理を全ＧＣＰ／ＩＣＰに対して行うことにより、各ＧＣＰ／ＩＣＰ周辺の局所的な座標変換式とＧＣＰに対応する真のＩＣＰの座標とが求められる。Ｓ３０５において、Ｓ３０１からＳ３０４までの処理が全ＧＣＰ／ＩＣＰに対して行われたか否かを判定し、まだ処理の行われていないＧＣＰが存在する場合には、Ｓ３０１からＳ３０４までの処理を行い、ＧＣＰとそれに対応するＩＣＰとの座標を求める。
【００６７】
全てのＧＣＰに対応し採用された真のＩＣＰの座標を求めた後、Ｓ３０６において、これらの全ＧＣＰと真のＩＣＰとの座標情報を用いて、Ｓ３０１と同様の処理により最終的な画像全体の座標変換式を求める。ここで、全体的な座標変換式はＳ３０１で求めた局所的変換式を基に変換式を推定し、Ｓ３０１において最小二乗法を用いて未定係数を決定した時と同様に、極端に二乗誤差が大きくなる真ＩＣＰ−ＧＣＰの組については、未定係数を求める際に標本から除外することにより、未定係数の推定精度を高めることができる。
【００６８】
Ｓ３０７において、全ブロックＳ３０６で求めた画像全体に対する座標変換式が適切なものか否かを判定し、Ｓ３０１において局所的な座標変換式を求める際に標本として使用したサブＧＣＰ−サブＩＣＰの座標値データを変換式に代入して、Ｓ３０５と同程度の変換精度が得られたか否かを調べる。この際、Ｓ３０１で局所的変換式を求める際に使用しなかった標本は用いない。Ｓ３０７において、もしも期待していた座標変換精度を満たさなかった場合には、再びＳ３０６に戻り、仮定した座標変換式を別の変換式に変更し、再度、座標変換式と未定係数とを求める。
【００６９】
次に、Ｓ３００で求めた座標変換式を用いて、画像データが、希望する地図座標系に対応するように幾何補正（座標変換）処理を行う(第４段階：Ｓ４００)。このような座標変換式で求められる幾何歪み補正済み画像における入力画像の各画素に対応する位置は、一般に整数にならない。従って、幾何補正後、画像の濃度値を求める際には、Ｓ２０１で画素分割時に行った場合と同様に内挿法による補間処理を行う。
【００７０】
以上説明したように、Ｓ１００からＳ４００までの一連の処理を行うことにより、ＧＣＰ周辺の座標変換前後の位置対応精度が大幅に向上し、また全ＧＣＰから得られる座標変換式は、式の次数および未定係数がより適切な値になるため、画像全体についても、より精度の高い変換処理が可能となる。
【００７１】
以上説明したように、本実施の形態によるデジタル画像処理装置によれば、解像度が高くないデジタル画像を用いても、高い精度で幾何学的補正を行うことが出来る。また、画素分割を用いると、粗いサンプリング間隔で取得されたデジタル画像であっても、デジタル画像上の特定地点の位置精度を画像取得時のサンプリング間隔に拘束されることなく、より精確な位置を割り出すことが出来る。
【００７２】
統計的に地上基準点に対応する画基準点を選定することにより、選定処理に必要な処理時間を短縮化でき、選定される画像基準点の位置精度の向上のみではなく、一連の基準点選定処理において処理施行者の基準点選定に伴う際の個人的な癖などが入り込まないため、画像処理における高い再現性を得ることが出来る。
【００７３】
さらに、複数のサブＩＣＰ−サブＧＣＰから求めた局所的な座標変換式に基づいて、ＧＣＰに対応するＩＣＰの画像座標を求めるため、粗いサンプリング間隔の画像上でもより高い精度でＩＣＰの座標を求めることが可能となる。
【００７４】
ＧＣＰのみで座標変換式を求めた場合に比べて、より位置精度の高いＩＣＰを用いて変換式を求めることができるため、より高次な幾何学的歪みをより高い精度での除去が可能とする。
【００７５】
尚、本実施の形態により用いられた（Ｓ１００〜Ｓ４００までで示される）幾何学的歪み補正技術は、高分解能衛星画像を対象とする場合に限られるものではなく、高分解能衛星画像よりも地上分解能の高い航空写真などを用いた空中測量においても、例えば粗い画素間隔で取得されているデジタル写真を用いた画像データを対象にする場合には有効である。
【００７６】
以上、実施の形態に沿って説明したが、本発明はこれらに制限されるものではない。その他、種々の変更、改良、組合せが可能なことは当業者に自明であろう。
【００７７】
【発明の効果】
以上説明したように、本発明のデジタル画像処理技術によれば、デジタル画像を高い精度で幾何学的補正することができるという利点がある。
【図面の簡単な説明】
【図１】本発明の一実施の形態によるデジタル画像処理技術の原理を簡単に説明するための図である。
【図２】本発明の一実施の形態によるデジタル画像処理システムに関連する高分解能衛星システムの全体構成を示す概略図である。
【図３】本発明の一実施の形態によるデジタル画像処理システムで行われるシステム補正処理の流れを示すフローチャート図である、
【図４】本発明の一実施の形態によるデジタル画像処理装置の構成例を示す機能ブロック図である。
【図５】本発明の一実施の形態による画像処理システムにおける全体の処理の流れを示すフローチャート図である。
【図６】図５に示す全体処理のうちの実際の測量による座標値の取得処理の流れを示すフローチャート図である。
【図７】図５に示す全体処理のうちの画像参照点の取得処理の流れを示すフローチャート図である。
【図８】図５に示す全体処理のうちの幾何補正式の決定処理の流れを示すフローチャート図である。
【図９】本発明の一実施の形態による画像処理技術のうち、実測を行うＧＣＰ及びサブＧＣＰ群を設定する例を示す図である。
【図１０】本発明の一実施の形態による画像処理技術のうち、ＧＣＰ及びサブＧＣＰのデータ記録状態の例を示す図である。
【図１１】本発明の一実施の形態による画像処理技術のうち、サブピクセル化処理とエッジ抽出処理との例を示す図である。
【図１２】本発明の一実施の形態による画像処理技術のうち、エッジ抽出処理後の画像に対して境界線推定を行い、サブＩＣＰを決定した例を示す図である。
【図１３】ｕ−ｖ画像座標系とｘ−ｙ地図座標系との間の座標変換処理の概念を示す図である。
【図１４】一般的な衛星画像の例である。
【符号の説明】
１…人工衛星（高分解能撮像衛星）、２…地上受信局、３…データ解析センター（もしくは衛星運用センター）、４…画像処理システム、５…高分解能衛星画像の一部切り抜き、６…サブＧＣＰ群、７…地上基準点（ＧＣＰ）、８…道路、９…建物、１０…高分解能衛星画像の拡大図（オリジナル）、１１…サブＧＣＰ群の高分解能衛星画像をサブピクセル化（1画素から２×２画素）した画像、１２…地上基準点（ＧＣＰ）に対して空間一次微分処理によるエッジ検出処理を行った後の画像、１３…推定境界線、１４…サブＩＣＰ、１５…境界線候補画素、１６…画像座標系、１７…地図座標系。[0001]
BACKGROUND OF THE INVENTION
The present invention relates to digital image processing technology, and more particularly to aerial photogrammetry, for example, a technology for correcting geometric distortion for a digital image having a low resolution (ground resolution) such as a digital image taken by an aircraft or an artificial satellite. .
[0002]
[Prior art]
In recent years, digital images have come to be used in place of analog images as various images such as surveying means for general photography and map production. This is presumably because the digital image does not deteriorate with time and can be easily processed by an information device such as a computer.
[0003]
Accordingly, opportunities to use digital images are increasing in aerial photogrammetry technology. Aerial photogrammetry is a technology that uses a surveying camera attached to a flying object such as an aircraft or an artificial satellite to photograph a target area to be measured, and based on this, creates a map and analyzes and interprets features. It is. The term “aerial photograph” generally means a vertical aerial photograph taken vertically downward from an aircraft, but recently, a high-resolution satellite image obtained by photographing the ground from an artificial satellite is used. Attempts have been made to conduct photogrammetry.
[0004]
When performing aerial photogrammetry, the size varies depending on the height of the land. Furthermore, the aerial photograph, which is the center projected image with the incorrect projected position, is corrected by geometric processing by post-processing after shooting, and back-projected to the instantaneous state in which the photograph was shot, and projected. Therefore, it is necessary to convert the position and shape of the feature and the terrain into an orthographic projection map correctly projected on the map (see, for example, Non-Patent Document 1).
[0005]
According to Non-Patent Document 1, correction of geometric distortion is performed using a ground control point (GCP) on a map, a coordinate system in an image having geometric distortion, and a geometric distortion. This is realized by obtaining a coordinate conversion formula between the coordinate system (map coordinate system) with corrected distortion. A general method is to remove geometric distortion by converting each point on the image to a correct position on the map coordinate system based on this coordinate conversion formula.
[0006]
Non-patent literature 1 According to the above, for example, the coordinates of the image system coordinate to be removed and the map coordinate system to be referred to when performing correction, such as the center of intersections of roads and railways, the center of refracted parts / bent parts, or the corners and centers of buildings. An object that can be easily confirmed and can accurately grasp the position and shape is used as the GCP. By arranging such GCPs uniformly in an image where the geometric distortion is desired to be removed (the actual number depends on the required correction accuracy), it becomes possible to remove the distortion of the entire image.
[0007]
There has also been proposed a method for correcting geometric distortion of aerial photographs using a map of the shooting area. In this method, aerial photographs, which are analog data, and pre-processed maps are captured by a scanner, image transformation processing such as affine transformation, enlargement, and rotation is performed on the photograph image so that the photograph and map match, and the map is overlaid. It is a combination. However, in this method, since distortion of the captured image is removed using an existing map created in the past as a reference, it is not possible to correct the distortion of the image using a current object such as a road or a building ( For example, see Patent Document 1).
[0008]
Furthermore, a method of performing geometric correction by matching an aerial photograph having a large geometric distortion with a satellite image having no distortion has been proposed (for example, see Patent Document 2). This method treats satellite images as having very little positional distortion and providing very high positional accuracy.
[0009]
[Non-Patent Document 1]
Supervised by Mikio Takagi and Yoshihisa Shimoda, “Image Analysis Handbook”, The University of Tokyo Press, 1998, p. 423-429.
[Patent Document 1]
JP-A-5-165402
[Patent Document 2]
JP 2000-276045 A
[0010]
[Problems to be solved by the invention]
However, in reality, in satellite images, for example, research and commercial high-resolution satellite images, the system has been corrected to eliminate internal distortion caused by the sensor mechanism and external distortion caused by the position and orientation of the satellite at the time of shooting. However, the geometric distortion caused by the geographical feature of the object cannot be completely removed.
[0011]
Aerial photographs that have been used for aerial surveys in the past are analog images with continuous changes in shading and color, and are considered to have extremely small particle size and virtually infinite resolution (ground resolution). In contrast, a satellite image is a digital image obtained by dividing an analog image into small cells (pixels) of the same shape and converting the average luminance value of each cell to an integer value as the representative luminance value in that cell. Since the obtained image is quantized at a rough sampling (sampling) interval determined by the pixel size of the sensor, a structure finer than the size of the ground corresponding to one pixel (ground resolution) is not reproduced on the image.
[0012]
Since the accuracy of geometric distortion correction is greatly influenced by the accuracy of the selected GCP, in order to obtain a corrected image having a position accuracy equivalent to a large-scale map, the alignment accuracy by GCP is 1 pixel or less. It is necessary to suppress. Japanese Patent Laid-Open No. 2000-276045 describes that the positional accuracy is significantly higher than the pixel size of the image. However, since an actual satellite image is strongly influenced by quantization error, at least when selecting GCP, Positional errors of about 2 to 8 pixels around will occur.
[0013]
FIG. 14 shows an example of a satellite image. In the satellite image, the corresponding size of one pixel in the high-resolution satellite image on the actual ground is very rough, about 0.6 to 1.0 m, and as shown in FIG. It is difficult to accurately detect corners, roads, and bridge boundaries. When the boundary (edge) cannot be detected accurately, when removing geometric distortion, appropriately select a reference point (ICP) on the satellite image corresponding to the GCP on the topographic map. Is difficult. If it is difficult to determine an appropriate corresponding point, the correction accuracy is naturally poor. Na Therefore, if the conventional technique is applied to the satellite image as it is, it is difficult to realize distortion correction with sufficient accuracy. Therefore, it becomes difficult to match the satellite image to the map.
[0014]
An object of the present invention is to provide a technique for correcting a digital image easily and accurately and matching it with a map with high accuracy.
[0015]
[Means for Solving the Problems]
According to one aspect of the invention, a procedure for correcting a digital image comprising:
On the map corresponding to the digital image to be corrected, a plurality of first reference points on the map whose map coordinate information is known and a plurality of second reference points selected for each of the first reference points. Obtaining a third reference point and a fourth reference point on the image respectively corresponding to the reference points;
A procedure for obtaining a fifth reference point having high local accuracy with respect to the third reference point based on the fourth reference point;
Correcting geometric distortion in the digital image based on the plurality of fifth reference points and the plurality of first reference points corresponding thereto;
A program for causing a computer to execute is provided. As the second reference point, it is preferable to select a point that can be easily recognized as an object.
[0016]
By performing edge recognition on the digital image and performing weighted linear approximation on the recognized edge, for example, by luminance, the third reference point and the fourth reference point (estimated to be in the vicinity of the straight line) It is preferred to include a procedure for determining.
[0017]
Since geometric correction is performed based on a plurality of sub-reference points, the use of the above program improves geometric distortion correction accuracy. By selecting a point that can be easily recognized as an object as the second reference point, the correction accuracy can be further improved.
[0018]
According to another aspect of the present invention, there is provided a digital image processing device that corrects a digital image, and on a map corresponding to the digital image to be corrected, a plurality of map coordinates information each having a known map coordinate information. Based on a first storage unit that stores a first reference point and a plurality of second reference points selected for one of the first reference points, and the first reference point and the second reference point, A detection unit that detects a corresponding third reference point and a fourth reference point on the digital image to be corrected, and the third reference point and the fourth reference point detected by the detection unit are stored. Based on the second storage unit and the fourth reference point, a fifth reference point having high local accuracy is obtained with respect to the third reference point, and a plurality of the fifth reference points and a plurality of corresponding fifth reference points are obtained. Based on one reference point in the digital image A geometric correction formula determination unit that corrects geometric distortion; and a geometric correction unit that corrects geometric distortion in the digital image based on the geometric correction formula determined by the geometric correction formula determination unit. Digital image processing device
Is provided.
[0019]
For example, the quantization error caused by rough sampling of the original image is minimized, and the position accuracy on the image of the point ICP on the image corresponding to GCP is made higher than the pixel size of the original image. For this purpose, in a method of correcting geometric distortion of an image, a digital image for which geometric distortion is to be removed is input, and latitude / longitude / altitude information of GCP and sub-GCP group included in the digital image is acquired / input. Next, a sub ICP group corresponding to the sub GCP group is obtained for the input image using a pixel division processing method, an edge extraction processing method, and a weighted least square method. Next, a relational expression (coordinate system conversion expression) between local image system coordinates and map system coordinates is estimated from the sub ICP group and the sub GCP group by the least square method. Subsequently, an ICP adjacent to the sub ICP group is obtained from the obtained local coordinate system transformation formula, and a coordinate system transformation formula for correcting the geometric distortion of the entire image is obtained from the ICP on the image corresponding to the GCP. Thereby, geometric correction with high accuracy can be performed.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
In this specification, the term “digital image” refers to a wide concept including, in addition to a digital image taken by a digital camera or the like, for example, an image obtained by digitizing an analog image taken by an analog camera using a scanner or the like, or digital image data itself. Indicates.
[0021]
Before describing embodiments of the present invention, considerations made by the inventors will be described first. The inventor often uses a reference in a digital image when positioning such a digital image and a reference (such as a map) when there are many cases where an absolute reference point is not found in the image, such as a digital image. In addition to the points (ICP) and the reference points (GCP) on the map, sub-reference points (sub-reference points on the digital image (hereinafter referred to as “sub-ICP”) associated with these reference points and the map. The upper sub-reference point (hereinafter referred to as “sub-GCP”) is set, and it has been devised to use these sub-ICP and sub-GCP as auxiliary reference points.
[0022]
For example, as the reference point, the center position of an intersection (for example, a crossroad) is often used, but as the sub-reference point, corners of a building built at the four corners of the crossroad are used. Conceivable. This is because the corner of a building is often easier to identify than the middle position of an intersection. Since it is considered that there are few opportunities to take a satellite photograph or the like, it is preferable that a target for ICP and a sub ICP is taken to a certain extent before photographing. Based on the above consideration, a digital image processing technique according to an embodiment of the present invention will be described with reference to the drawings.
[0023]
FIG. 1 is a schematic diagram showing the principle of digital image processing according to the present embodiment. FIG. 2 is a schematic diagram showing the overall configuration of a high-resolution satellite system according to an embodiment of the present invention. FIG. 3 is a flowchart showing the flow of system correction processing performed in the digital image processing system. FIG. 4 is a functional block diagram showing a configuration example of the digital image processing apparatus according to the embodiment of the present invention.
[0024]
As shown in FIG. 1, a plurality of second reference points are determined corresponding to the first reference points on the map. On the other hand, on the image, a third reference point corresponding to the first reference point and a plurality of fourth reference points corresponding to the plurality of second reference points are determined. Based on the plurality of fourth reference points, a fifth reference point having a higher geometric accuracy than the third reference point is determined. The entire digital image is geometrically corrected based on the entire first and fifth reference points.
[0025]
As shown in FIG. 2, the high-resolution satellite image is taken from the artificial satellite 1 that is several hundred kilometers above the ground, and the taken data is compressed and sent to the ground receiving station 2. The data received by the receiving station 2 is sent to the analysis center 3 as it is, and the image processing system 4 installed therein performs several stages of data processing shown in FIG. Become.
[0026]
FIG. 3 is a diagram showing a procedure for processing raw data sent from a satellite. The raw data S1 received at the receiving station is first decompressed (expanded) from the compressed data (S2). Next, sensitivity correction due to the sensitivity characteristics of the optical system and photoelectric conversion sensor used, correction of luminance change due to solar altitude, topographical inclination, etc., and the atmosphere existing between the satellite and the object So-called radiation amount correction is performed to correct the attenuation of reflected light due to the absorption or scattering of sunlight by (S3). Thereafter, a sensor correction process (S4) is performed to remove the internal distortion caused by the sensor mechanism and the external distortion caused by the attitude of the artificial satellite at the time of photographing or the rotation of the earth. The correction process including the radiation amount correction S3 and the sensor correction S4 is referred to as system correction. The obtained image is generally a system-corrected image S5 that has undergone the processing up to S4.
[0027]
As shown in FIG. 4, an image processing system according to an embodiment of the present invention includes a processing device 101, a hard disk (storage device) 105, a display (display device) 102, an input unit, and a keyboard 103 and a mouse. And an input unit 104. The processing apparatus 101 includes a pixel division processing unit 111, an ICP / sub ICP detection unit 112, a geometric correction point determination unit 113, and a geometric correction unit 114. The hard disk 15 includes an image data storage file 151, a GCP / sub GCP data storage file 152, and an ICP / sub ICP data storage file 153.
[0028]
The pixel division processing unit 111 performs processing for dividing the pixels constituting the digital image. The ICP / sub ICP detection unit 112 performs processing for detecting ICP and sub ICP. The geometric correction formula determination unit 113 determines a geometric correction formula using the reference point detected in 112. The geometric correction unit 114 performs actual geometric correction.
[0029]
FIG. 5 to FIG. 8 are flowcharts showing the flow of digital image correction processing according to the present embodiment. FIG. 5 is a diagram showing the flow of the entire process, and FIGS. 6 to 8 are diagrams showing details of the process of each step of FIG.
[0030]
As shown in FIG. 5, the image distortion correction processing according to the present embodiment includes first to fourth correction processing. The first-stage correction process S100 selects a GCP and sub-GCP group to be used for correction from a topographic map including a region to be corrected and performs a survey on site or a method of reading from a topographic map. By using it, the latitude / longitude and altitude information of each reference point is acquired. Next, in the second step S200, reference points ICP and sub ICP groups on the image corresponding to each GCP and sub GCP group on the image to be corrected are obtained semi-automatically, for example. In the third step S300, an ICP point with high local accuracy is obtained using the sub ICP group obtained in the second step S200, and an image coordinate conversion formula is obtained from GCP / ICP of the entire image. As will be described later, in the final step 307 (FIG. 8) of S300, the conversion accuracy is verified by converting the sub-GCP group to the image coordinate shape. If the criterion is satisfied, the process proceeds to the final step S400 and actually Perform coordinate transformation of the entire image.
[0031]
FIG. 6 shows the flow of processing in the first-stage processing S100 “acquisition of actual coordinate values”. Here, the coordinates of the reference point used when obtaining the conversion formula by actual route surveying or the like are obtained. FIG. 9 is a diagram illustrating a setting example of a GCP and a sub GCP group to be actually measured. First, a topographic map including an image region to be converted is obtained (S101), and in addition to GCP6 (in this case, the middle point of the crossroad) that is generally used, the topography is also displayed on the GCP image as well as the GCP. The user arbitrarily sets a plurality of auxiliary reference points (sub GCP groups) 7 that are easy to observe on the figure, in this case, the edges of the buildings built at the four corners of the crossroads (S102). Both GCP6 and sub-GCP7 are easy to see on the image as actual intersections that can be clearly read on the image and the map, such as intersections of roads 8 and four corners of building 9. It is preferable to be able to discriminate between. In S103, in addition to the survey of GCP6, the sub-GCP group 7 also performs an actual route survey on the exact latitude and longitude and altitude of each point, or calculates the latitude and longitude information of the corresponding point from the topographic map. Ask. The GCP and sub-GCP group obtained in this way holds measurement data in a format in which GCP and sub-GCP are associated with each other as shown in FIG. FIG. 10 is data in which a unique ID for identifying a GCP or a sub-GCP (SGCP), its longitude and latitude (both in decimal units), altitude (m units), and IDs of adjacent GCPs are associated with each other. is there.
[0032]
The number of reference points required for the GCP and the sub GCP group is the minimum or the minimum number of unknown variables included in the coordinate system conversion formula applied in the geometric distortion correction processing step S300 in the entire or partial region of the image from which distortion is to be removed. However, 1/2 or more is necessary, and more than the number of unknown variables is necessary. When geometric distortion correction is performed using system-corrected satellite images as input data, it is often only necessary to perform correction by Helmat transform (unknown variable 4) and affine transform (unknown variable 6). Therefore, six or more GCPs may be selected. Similarly, the number of sub GCPs set around each GCP also needs to be about the number of undetermined coefficients included in the coordinate system conversion formula for matching each point of the sub ICP group to the sub GCP group. What is necessary is just to select the same number (six or more).
[0033]
In FIG. 9, 12 points that can be seen from the GCP are set as sub GCPs. The sub-GCP does not need to be a neighboring point that can be seen from the GCP. If the point is not geographically significantly different from the GCP point, it is easy to measure slightly away from the neighboring blocks (or check on the image) (Easy) point may be used.
[0034]
However, it is desirable that the sub-GCP group related to the GCP selects a point that is as close to the same altitude as possible. If a reference point with extremely different altitudes is included, even if a satellite photo with a high altitude is taken, it is taken from a finite altitude, so the position of the image on the photo moves depending on the altitude difference of land and buildings There is a high possibility that a phenomenon called “falling” will occur. Therefore, when GCP or sub-GCP with a large difference in altitude is used, the average ground altitude of the region including all GCP groups or sub-GCP groups used in advance from the topographic map is obtained as the reference plane altitude. When using the image coordinates (u, v) of ICP or sub ICP, the coordinates (u ', v') when only the altitude is projected to the reference plane altitude are obtained without changing the latitude and longitude, and this is processed later. Used in.
[0035]
As shown in FIG. 7, S200 is a flowchart showing the flow of the second-stage process “image reference point acquisition process”. In this process, each reference point GCP for which the coordinates in the map coordinate system are obtained in S100, and For the sub GCP group, ICP and sub ICP corresponding to each are determined on the satellite image. First, as shown in S202, a combination of pixel division (pixel division) processing and rearrangement (resampling) processing is performed on the original input image image S201 sampled at a coarse pixel interval of the detector. Then, one pixel of the image to be subjected to distortion correction is divided into a plurality of pixels, and the pixel size of one pixel is reduced. Since the position accuracy on the digital image is determined by the pixel size, the position of the corresponding point can be obtained with high accuracy by appropriately performing this process.
[0036]
In S202, since one pixel of the original image is divided into a plurality of pixels, it is necessary to estimate pixel data to be obtained from pixel values of surrounding pixels. Such rearrangement processing is called an interpolation method, and (1) nearest neighbor interpolation method, (2) co-primary interpolation method, (3) cubic convolution interpolation method, etc. are used (for example, Mikio Takagi, (See “Image Analysis Handbook” supervised by Y. Shihida, University of Tokyo Press, 1998, p.441-444). By using an appropriate interpolation method, the effect of smoothing the image density value distribution can also be obtained, and the sudden density change between adjacent pixels that was noticeable in the original image is the peak position of the density value distribution on the image. For smooth edges, the shape is preserved as it is, and smooth density value changes can be obtained without impairing important information such as edges.
[0037]
An edge (line element) extraction process is subsequently performed on the image that has undergone the pixel division process in S202, and a boundary line element or boundary line candidate pixel that can be subjected to a sudden luminance change that is likely to be a boundary line on the image is obtained. Extract. The GCP and sub GCP group set in S103 are both set at points that are easy to interpret and recognize as objects on the image, and by extracting line and contour elements based on the image, the corresponding ICP The sub ICP group can be easily identified. Typical edge extraction processes include a method that uses the spatial first derivative (gradient) of each shade change in both column and line directions, a method that uses the spatial second derivative (Laplacian), and other edge detection. (For example, “Image Analysis Handbook” supervised by Mikio Takagi and Yoshihisa Shimoda, University of Tokyo Press, 1998, p. 550-564).
[0038]
FIG. 11 is a diagram illustrating an example of an image obtained by performing S202 to S203. By the image dividing process of S202, the divided image 11 obtained by dividing one pixel of the original image 10 into 2 × 2 pixels is obtained. In this example, a cubic convolution interpolation method is used as an interpolation method when dividing the image. The edge-extracted image 12 is obtained as a result of further performing the edge extraction processing on the image 11 after the division processing. Here, spatial first-order differentiation is used as edge extraction processing. A portion corresponding to the wall surface of the building on the image 11 is detected as a pixel (boundary line candidate pixel) having a large luminance change in the image 12 after edge extraction, and it can be seen that it looks bright.
[0039]
Next, with respect to the edge extraction image 12, the boundary line necessary for obtaining the ICP or the sub ICP is determined by the processing from S204 to S207, and the image system coordinates of the ICP or the sub ICP group are obtained. More specifically, first, in S204, the ICP or sub ICP whose coordinates are to be obtained and a plurality of boundary lines adjacent to the reference point are estimated by visual observation, and boundary line candidates that will constitute each boundary line. The user himself / herself designates the area surrounding the pixel using an input device such as the mouse 14 (FIG. 4) while viewing the screen. Next, in the area designated in S204, by using a weighted least square method that weights the luminance of each pixel as a weight, fitting to a straight line or curve makes the likelihood of the reference line candidate pixel in the designated area smooth. A realistic optimum boundary line is determined (S205).
[0040]
In the weighted least square method performed in S205, the luminance value li of the pixel (ui, vi) (i = 1, 2,..., N) in the estimated boundary area specified in S204 is the luminance value in the same area. The following equation obtained from the ratio with the sum is used.
[0041]
[Expression 1]

[0042]
If the above equation is used as a weight, for example, a straight line (undefined coefficient a, b) is assumed as a boundary line, the following weighted residual sum of squares is obtained.
[0043]
[Expression 2]

[0044]
By determining the undetermined coefficient (a, b) that minimizes the weighted residual sum of squares obtained by the above equation, a straight line as a boundary line is obtained. In S206, it is determined whether or not all the boundary lines adjacent to the ICP or sub ICP for which the image coordinates are to be determined are determined by the boundary line determination operations from S204 to S205, and all the boundary lines are still determined. If not, S204 to S205 are repeated until all of them are confirmed.
[0045]
In S206, it is determined whether or not all boundary lines adjacent to the ICP or sub ICP whose coordinates are to be obtained have been determined. If it is confirmed, any adjacent boundary line always passes through the ICP and sub ICP to be obtained. Therefore, one intersection of the adjacent boundary lines obtained in S204 and S205 is determined as ICP (or sub ICP) ( S207). In FIG. 12, the sub ICP corresponding to the sub GCP set in the corner of the building is determined as the sub ICP 14 at the intersection (corner) of the boundary line 13 (indicated by a white broken line) corresponding to the wall surface of the building obtained in S205. It is a figure which shows a mode. As shown in FIG. 12, it can be seen that the edge of the building clearly indicated by the white broken line is clearly extracted.
[0046]
In S208, it is determined whether or not the image coordinates of all ICPs and sub ICPs included in the image to be subjected to geometric distortion correction have been obtained. If there are ICPs or sub ICPs that have not yet been obtained, the process proceeds to S204. Return and obtain coordinates using the same procedure. As in S207, by obtaining the image coordinates (u, v) of the image reference point as the intersection of the boundary lines instead of the pixels, the coordinates can be obtained at the sub-pixel level (0.1 pixel or less). The position accuracy of the reference point can be obtained with higher accuracy than the pixel size. By obtaining the image coordinates (u, v) of the image reference point in the process of S200, the error of the position coordinates of the reference point obtained is proportional to the reciprocal of the division number at the time of pixel division in S202, for example, in FIG. The position error is reduced to 1/2. By obtaining the reference point as an intersection in S207, the error is reduced to 1/10 or less. Accordingly, throughout the processing of S200, the position accuracy of the reference point can be improved to at least 10 times or more (about 20 times in the case of FIG. 11) when the reference point is obtained from the original image.
[0047]
In the third stage “determination of geometric correction formula (coordinate conversion formula)” of S300 shown in FIG. 8, a conversion formula for converting the satellite image coordinate system to the map coordinate system using the ICP and the sub ICP group obtained in S200 is provided. Finally, the process of superimposing the image on the map coordinate system is performed. First, in step S301, one point is selected from the ICPs whose coordinates have been obtained in step S200, and the image coordinate information of the sub ICP group associated with the ICP and the map coordinate information (latitude, longitude and latitude) of the sub GCP group to be associated with each point. Based on (altitude), a conversion formula that correlates image coordinates (u, v) and map coordinates (x, y), or coordinates (x, y, z) taking altitude z into consideration is obtained.
[0048]
FIG. 13 shows a conceptual diagram of the coordinate conversion system. The image coordinate (u, v) of the sub ICP, the map coordinate (x, y, z) of the corresponding sub GCP, the conversion formula in the image column direction is F (x, y, z), and the conversion formula in the line direction is G. Assuming (x, y, z), the image coordinates corresponding to the map coordinates are expressed by the following equations.
u = F (x, y, z)
v = G (x, y, z)
[0049]
Based on the above relational expression, a point on the xy coordinate of the map coordinate system 17 is coordinate-transformed and obtained as a point on the uv coordinate of the image coordinate system 16. Therefore, by obtaining an expression for performing this inverse transformation, a point on the uv coordinate of the image coordinate system 16 can be accurately returned to the xy coordinate of the map coordinate system 17.
[0050]
According to the above documents and the like, in order to be generally used as a coordinate transformation equation, (1) Helmart transformation (used for correction of scale distortion and rotational distortion, origin movement), (2) affine transformation (correction of skew distortion, Used for linear transformation), (3) pseudo-affine transformation (used for linear torsion transformation), (4) two- and three-dimensional projective transformation (used for correction of direct and parabolic distortion), and (5) two-dimensional projective transformation. (6) Three-dimensional projective transformation (used for correcting projection distortion), (7) High-order polynomial transformation (used for correcting high-order distortion), and the like. These conversion equations are shown below.
[0051]
(1) Helmart transform (unknown variables a, b, c, d)
[Equation 3]

[0052]
(2) Affine transformation (unknown variables a, b, c, d, e, f)
[Expression 4]

[0053]
(3) Pseudo-affine transformation (unknown variables a1, a2, a3, a4, b1, b2, b3, b4)
[Equation 5]

[0054]
(4) Two-dimensional conformal transformation (unknown variables a, b, c, d, e, f)
[Formula 6]

[0055]
(5) Two-dimensional projective transformation (unknown variables a1, a2, a3, a4, a5, a6, a7, a8)
[Expression 7]

[0056]
(6) Three-dimensional projective transformation (unknown variables a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11)
[Equation 8]

[0057]
(7) Higher order polynomial transformation (unknown variables aij, bij)
[Equation 9]

[0058]
If only simple translation and rotation / enlargement / reduction of the image are required, and the image shape is similar before and after the transformation in the local area to be transformed, the Helmat transform is used, and the linear translation and translation are combined. As described above, affine transformation is used when it is desired to linearly transform x and y independently. In addition, when more complicated distortion, for example, transformation with twist such as xy term is necessary, pseudo-affine transformation is used (for example, “Remote Seching for Practitioners” edited by Naoyuki Obayashi and Naoto Kojima) Fuji Techno System, 2002, p. 122).
According to this document, when geometric distortion correction is performed using system-corrected satellite images as input data, it is often only necessary to perform correction by Helmat transform and affine transform.
[0059]
Using the transformation formula that is expected to be most appropriate as judged from the form of distortion of the image, the coordinate value on the image of the sub ICP group and the latitude / longitude / altitude of the sub GCP group obtained by measurement are calculated by the least square method. Find the undetermined coefficient of the assumed conversion formula. For example, if a square object is distorted in a diamond shape on a satellite image on the ground surface, assuming the affine transformation as a coordinate transformation formula, the ground coordinates (xi, yi), (i = 1,... , n) and the corresponding image coordinate values (ui, vi) of the sub ICP, the following formula is obtained by adding the square of the difference between the theoretical formula and the observed value so that Q1 and Q2 are minimized ( a, b, c, d, e, f) should be determined. For each of Q1 and Q2, assuming that the result of partial differentiation at (a, b, c) and (d, e, f) is equal to 0, it can be obtained by the following equation.
[0060]
[Expression 10]

[0061]
In S301, when the undetermined coefficient is determined using the least square method, the sub-square ICP (um, vm) −sub-GCP (xm, ym) (m is an arbitrary one of 1 to n). For the group of (values), since there is a high possibility of sub-GCP measurement errors and sub-ICP selection mistakes, the undetermined coefficients are excluded from the sample and (ui, vi, xi, yi) (i = 1 ,..., M−1, m + 1,..., N), by obtaining unknown variables again from n−1 samples, it is possible to improve the estimation accuracy of the undetermined coefficients.
[0062]
As shown in S302, the ICP is converted into a map coordinate system by a local coordinate system conversion formula. In S303, it is determined whether or not the coordinate system conversion formula obtained in S301 is sufficiently accurate using the image coordinates of ICP, the map coordinates of GCP, and altitude information. Using the conversion formula obtained in S301, coordinates (x ′, y ′) obtained by converting ICP (u, v) of the image coordinate system into the map coordinate system, and map coordinates (x, y) of the actual GCP, This is performed based on whether or not the distance is as an error and whether the error has reached the expected conversion accuracy (for example, 1.0 pixel or less in the image after pixel division processing). .
[0063]
In S303, if the error associated with the coordinate conversion is worse than the expected value, it can be estimated that the conversion formula assumed in S301 is not appropriate, so the process returns to S301 and the coordinate conversion formula is examined again to determine the undetermined coefficient. The processes from S301 to S303 are repeated until the set conversion accuracy is satisfied.
[0064]
If the set accuracy cannot be obtained using any of the conversion formulas, either the set accuracy is too high or the prepared sub GCP group is not appropriate for obtaining the set accuracy. It is estimated that there is, and the process returns to the surveying process of the GCP / sub-GCP group in S100 again, refines the GCP acquisition plan, and performs the process again.
[0065]
If the error due to the coordinate transformation satisfies the expected value in S303, the point (u ′, v) where the GCP is projected on the image coordinate system is performed by performing the inverse transformation of the map coordinate system transformation obtained in S301. ') Is treated as a true ICP corresponding to GCP.
[0066]
As described above, by performing the processing from S301 to S304 on all the GCP / ICPs, the local coordinate conversion formula around each GCP / ICP and the true ICP coordinates corresponding to the GCP are obtained. In S305, it is determined whether or not the processing from S301 to S304 has been performed on all GCP / ICPs. If there is a GCP that has not yet been processed, the processing from S301 to S304 is performed. The coordinates of the GCP and the corresponding ICP are obtained.
[0067]
After obtaining the coordinates of the true ICP adopted corresponding to all the GCPs, in S306, using the coordinate information of these all GCPs and the true ICPs, the same processing as in S301 is performed to complete the final image. Find the coordinate transformation formula. Here, the overall coordinate conversion formula is estimated based on the local conversion formula obtained in S301, and in the same way as when the undetermined coefficient is determined using the least square method in S301, the square error is extremely large. For the true ICP-GCP pair that increases, the estimation accuracy of the undetermined coefficient can be increased by excluding the undetermined coefficient from the sample.
[0068]
In S307, it is determined whether or not the coordinate transformation formula for the entire image obtained in all blocks S306 is appropriate, and the coordinate value of the sub GCP-sub ICP used as a sample when obtaining the local coordinate transformation formula in S301. By substituting the data into the conversion formula, it is checked whether or not the conversion accuracy equivalent to S305 is obtained. At this time, a sample that is not used when the local conversion formula is obtained in S301 is not used. In S307, if the expected coordinate conversion accuracy is not satisfied, the process returns to S306 again, the assumed coordinate conversion formula is changed to another conversion formula, and the coordinate conversion formula and the undetermined coefficient are obtained again.
[0069]
Next, using the coordinate conversion formula obtained in S300, geometric correction (coordinate conversion) processing is performed so that the image data corresponds to the desired map coordinate system (fourth stage: S400). In general, the position corresponding to each pixel of the input image in the geometric distortion corrected image obtained by such a coordinate conversion formula is not an integer. Therefore, when the density value of the image is obtained after geometric correction, interpolation processing by interpolation is performed in the same manner as when performing pixel division in S201.
[0070]
As described above, by performing a series of processing from S100 to S400, the position correspondence accuracy before and after the coordinate conversion around the GCP is greatly improved, and the coordinate conversion formula obtained from all GCPs is the order of the formula and Since the undetermined coefficient becomes a more appropriate value, a more accurate conversion process can be performed for the entire image.
[0071]
As described above, according to the digital image processing apparatus according to the present embodiment, geometric correction can be performed with high accuracy even when a digital image with a high resolution is used. In addition, when pixel division is used, even in a digital image acquired at a rough sampling interval, the position accuracy of a specific point on the digital image is not restricted by the sampling interval at the time of image acquisition, and a more accurate position can be obtained. Can be determined.
[0072]
By selecting the image reference points that correspond to the ground reference points statistically, the processing time required for the selection process can be shortened, not only improving the position accuracy of the selected image reference points, but also selecting a series of reference points. Since the process does not include personal habits associated with the selection of the processing operator's reference point, high reproducibility in image processing can be obtained.
[0073]
Further, since the ICP image coordinates corresponding to the GCP are obtained based on the local coordinate conversion formula obtained from the plurality of sub ICP-sub GCPs, the ICP coordinates are obtained with higher accuracy even on an image with a rough sampling interval. It becomes possible.
[0074]
Compared with the case where the coordinate conversion equation is obtained only by GCP, the conversion equation can be obtained using ICP having higher positional accuracy, so that higher-order geometric distortion can be removed with higher accuracy. To do.
[0075]
Note that the geometric distortion correction technique (shown in S100 to S400) used in the present embodiment is not limited to the case of targeting a high-resolution satellite image, but rather than the high-resolution satellite image. Even in aerial surveying using high-resolution aerial photographs and the like, it is effective, for example, when image data using digital photographs acquired at coarse pixel intervals is targeted.
[0076]
As mentioned above, although demonstrated along embodiment, this invention is not restrict | limited to these. It will be apparent to those skilled in the art that other various modifications, improvements, and combinations can be made.
[0077]
【The invention's effect】
As described above, according to the digital image processing technique of the present invention, there is an advantage that the digital image can be geometrically corrected with high accuracy.
[Brief description of the drawings]
FIG. 1 is a diagram for briefly explaining the principle of a digital image processing technique according to an embodiment of the present invention;
FIG. 2 is a schematic diagram showing an overall configuration of a high-resolution satellite system related to a digital image processing system according to an embodiment of the present invention.
FIG. 3 is a flowchart showing a flow of system correction processing performed in the digital image processing system according to the embodiment of the present invention;
FIG. 4 is a functional block diagram illustrating a configuration example of a digital image processing apparatus according to an embodiment of the present invention.
FIG. 5 is a flowchart showing an overall processing flow in the image processing system according to the embodiment of the present invention.
6 is a flowchart showing a flow of coordinate value acquisition processing by actual surveying in the entire processing shown in FIG. 5. FIG.
7 is a flowchart showing a flow of image reference point acquisition processing in the overall processing shown in FIG. 5. FIG.
FIG. 8 is a flowchart showing a flow of a geometric correction formula determination process in the overall process shown in FIG. 5;
FIG. 9 is a diagram illustrating an example of setting a GCP and a sub GCP group for actual measurement in the image processing technique according to the embodiment of the present invention.
FIG. 10 is a diagram showing an example of data recording states of GCP and sub GCP in the image processing technique according to the embodiment of the present invention.
FIG. 11 is a diagram illustrating an example of sub-pixel conversion processing and edge extraction processing in the image processing technology according to the embodiment of the present invention.
FIG. 12 is a diagram illustrating an example in which a sub-ICP is determined by performing boundary line estimation on an image after edge extraction processing in the image processing technique according to the embodiment of the present invention.
FIG. 13 is a diagram illustrating a concept of a coordinate conversion process between a uv image coordinate system and an xy map coordinate system.
FIG. 14 is an example of a general satellite image.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Artificial satellite (high resolution imaging satellite), 2 ... Ground receiving station, 3 ... Data analysis center (or satellite operation center), 4 ... Image processing system, 5 ... Partial clipping of high resolution satellite image, 6 ... Sub GCP Group 7 ... ground reference point (GCP), 8 ... road, 9 ... building, 10 ... enlarged view of high resolution satellite image (original), 11 ... sub resolution of sub GCP group high resolution satellite image (from 1 pixel) 2 × 2 pixels), 12... Image after performing edge detection processing by spatial first-order differential processing on the ground reference point (GCP), 13... Estimated boundary line, 14... Sub ICP, 15. Pixel, 16 ... image coordinate system, 17 ... map coordinate system.

Claims (5)

A procedure for correcting a digital image,
On the map corresponding to the digital image to be corrected, each of a plurality of first reference points which are ground reference points (reference points GCP) on the map whose map coordinate information is known and one of the first reference points A plurality of second reference points (sub GCPs) on the map that are selected around each of the second reference point and a third reference point and a fourth reference point on the image corresponding to the map reference coordinates A procedure for determining an ICP and a sub ICP corresponding to each reference point GCP and sub GCP group for which coordinates in the system are obtained, and for an original input image sampled at a coarse pixel interval The pixel division (pixel division) process and the rearrangement (resampling) process are combined to divide one pixel of an image to be corrected for distortion into a plurality of pixels and reduce the pixel size of one pixel. Determined, on the basis of the third reference point and said fourth reference point, the procedure for obtaining the high fifth reference point of the local precision on an image with respect to the third reference point,
An ICP adjacent to the sub ICP group is obtained from a local coordinate system conversion equation obtained based on the plurality of fifth reference points and the plurality of first reference points corresponding thereto, and from the ICP on the image corresponding to GCP. A procedure for obtaining a coordinate system conversion formula for correcting geometric distortion of the entire image,
The program for making a computer perform the procedure which correct | amends the geometric distortion in the said digital image by the said coordinate transformation type | formula . Procedure for correcting the geometrical distortion in the previous SL digital image,
Based on the third reference point or the plurality of fourth reference points and the plurality of first reference points corresponding thereto, the image coordinate information of the fourth reference point and the map coordinates of the corresponding second reference point A local coordinate system conversion formula is obtained based on the information, and the corrected fifth reference point corresponding to the third reference point is obtained using the obtained local coordinate system conversion formula. The program according to claim 1 . And a step of correcting geometric distortion in the digital image based on the plurality of fifth reference points and the plurality of first reference points corresponding to the fifth reference points.
A coordinate system conversion formula for the entire digital image is obtained based on the image coordinate information of the fifth reference point and the map coordinate information of the corresponding first reference point, and the coordinate system of the obtained digital image is obtained. program according to claim 1 or 2, characterized in that it comprises a corrected procedure of obtaining the first reference point corresponding to the fifth reference point by using a conversion equation. A digital image processing apparatus for correcting a digital image,
On the map corresponding to the digital image to be corrected, each of a plurality of first reference points which are ground reference points (reference points GCP) on the map whose map coordinate information is known and one of the first reference points A first storage unit that stores a second reference point that is a plurality of sub-reference points (sub-GCPs) on a map selected in the vicinity thereof ,
Based on the first reference point and the second reference point , the coordinates of the corresponding third reference point and fourth reference point on the digital image to be corrected are obtained in the map coordinate system. A procedure for determining an ICP and a sub ICP corresponding to each of the reference point GCP and the sub GCP group on the image, and pixel division (pixel division) with respect to the original input image sampled at a coarse pixel interval A detection unit that performs processing and rearrangement (resampling) processing in combination, divides one pixel of an image to be subjected to distortion correction into a plurality of pixels, reduces the pixel size of one pixel, and detects it;
A second storage unit for storing the third reference point and the fourth reference point detected by the detection unit;
Based on the third reference point and the fourth reference point , a fifth reference point having high local accuracy with respect to the third reference point is obtained on the image, and a plurality of the fifth reference points and corresponding to the fifth reference point are obtained. The ICP adjacent to the sub ICP group is obtained from the local coordinate system conversion formula obtained based on the plurality of first reference points, and the geometric distortion of the entire image is corrected from the ICP on the image corresponding to the GCP. A geometric correction formula determination unit for obtaining a coordinate system conversion formula to be
A geometric correction unit that corrects geometric distortion in the digital image based on the geometric correction formula determined by the geometric correction formula determination unit;
A digital image processing apparatus. A method of correcting geometric distortion of an image using ground surface coordinates using a plurality of small area images and latitude / longitude / altitude measurement values of points included therein,
Inputting digital images taken from flying objects;
A plurality of first reference points that are ground reference points (reference points GCP) in the digital image and auxiliary reference points (sub GCPs) on a plurality of maps that surround the plurality of first reference points. Obtaining latitude and longitude information and altitude information of the second reference point group;
Inputting latitude and longitude information and altitude information of the first reference point and the second reference point;
In the digital image, obtaining an ICP on the digital image corresponding to the reference point GCP based on a third reference point group on the digital image corresponding to the second reference point group , the map coordinates A procedure for determining an ICP and a sub ICP corresponding to each reference point GCP and sub GCP group for which coordinates in the system are obtained, and for an original input image sampled at a coarse pixel interval The pixel division (pixel division) process and the rearrangement (resampling) process are performed in combination, and one pixel of the image to be corrected for distortion is divided into a plurality of pixels, and the pixel size of one pixel is reduced. Steps,
Obtaining , on the image, a fifth reference point having high local accuracy with respect to the third reference point based on the third reference point and the fourth reference point;
An ICP adjacent to the sub ICP group is obtained from a local coordinate system conversion equation obtained based on the plurality of the fifth reference points and the plurality of first reference points corresponding thereto, and on the image corresponding to the reference point GCP. Obtaining a coordinate system conversion equation for correcting geometric distortion of the entire image from the ICP;
Correcting a geometric distortion in the digital image by the coordinate transformation formula .

Images