JP3846245B2

JP3846245B2 - Three-dimensional shape measuring apparatus and method

Info

Publication number: JP3846245B2
Application number: JP2001280283A
Authority: JP
Inventors: 研東海
Original assignee: Fuji Xerox Co Ltd; Fujifilm Business Innovation Corp
Current assignee: Fujifilm Business Innovation Corp
Priority date: 2001-09-14
Filing date: 2001-09-14
Publication date: 2006-11-15
Anticipated expiration: 2021-09-14
Also published as: JP2003083729A

Description

【０００１】
【発明の属する技術分野】
本発明は、測定対象に対してパターン光を照射することによって得られるパターン投影像を、複数の撮像手段で異なる方向から撮像し、パターンの変化に基づいて距離情報を得る三角測量法に基づく３次元形状計測技術に関する。
【０００２】
【従来の技術】
３次元形状を取得する手法には、アクティブ手法（Ａｃｔｉｖｅｖｉｓｉｏｎ）とパッシブ手法（Ｐａｓｓｉｖｅｖｉｓｉｏｎ）がある。アクティブ手法は、（１）レーザ光や超音波等を発して、対象物からの反射光量や到達時間を計測し、奥行き情報を抽出するレーザ手法や、（２）スリット光などの特殊なパターン光源を用いて、対象表面パターンの幾何学的変形等の画像情報より対象形状を推定するパターン投影方法や、（３）光学的処理によってモアレ縞により等高線を形成させて、３次元情報を得る方法などがある。一方、パッシブ手法は、対象物の見え方、光源、照明、影情報等に関する知識を利用して、一枚の画像から３次元情報を推定する単眼立体視、三角測量原理で各画素の奥行き情報を推定する二眼立体視等がある。
【０００３】
本発明は、アクティブ手法の３次元計測技術であるパターン投影法に関するものである。パターン投影法では、対象とする物体に基準となるパターン光を投影し、基準となるパターン光が投影された方向とは異なる方向から撮影を行う。撮影されたパターンは、物体の形状によって変形を受けたものとなる。観測された変形パターンと投影したパターンとの対応づけを行うことで、物体の３次元計測を行える。パターン投影法では、高速かつ高精度な測定が可能であるが、変形パターンと投影したパターンの対応づけにおいていかに誤対応を少なくするかが課題となる。
【０００４】
そこで、本出願と同一出願人に係る特願平１０−１９１０６３号（特開２０００−９４４２号公報）、特願平１０−２４７７９６号（特開２０００−６５５４２号公報）では投光されたパターンをフィードバックし新たなコードを生成することで対象物に依存しない３次元画像撮影装置を提案した。
【０００５】
【発明が解決しようとする課題】
上記の特願平１０−１９１０６３号（特開２０００−９４４２号公報）、特願平１０−２４７７９６号（特開２０００−６５５４２号公報）における３次元画像撮影装置は、投光パターンを複数の強度や複数の波長によって符号化されたものを投影して実現する。その際に、従来では、投光パターンが被写体の輝度情報、素材などの影響によって変化し、３次元形状を算出する際に、エラーとなって適切な３次元形状が計測できない。そのために、上記３次元撮像装置は、投光素子と同光軸に配置をし、被写体情報による投光パターンの変化分をモニタし、再符号化を実施し、３次元形状を計測している。
【０００６】
しかしながら、上記構成は、一方のカメラで得られた新しい符号が全く同じ新しい符号として別のカメラでも発生していることを前提としている。これは、装置中のすべてのカメラが同じ出力特性を有しており、対象物体上でのある明度に対して同じ信号を出力することを必要とする。この条件を厳密に満たす複数のカメラを準備することはコスト的に難しい。また、仮に準備したとしても、経時変化あるいは使用時の動作環境の変動などによりばらつきが生じてしまうことは避け難い。
【０００７】
さらに、対象物体上のテクスチャによって生じた新しい符号は、一意であるとは限らない。これは、対象物体上の別の点における別のテクスチャによって、同じ新しい符号が発生する可能性があるためである。すなわち、点ｐ（ｘ，ｙ）における反射率をｒ（ｘ，ｙ）、投影された符号の輝度をｉ（ｘ，ｙ）としたときに、対象物体上の２つの異なる点Ｐ_１（ｘ_１，ｙ_１）と点Ｐ_２（ｘ_２，ｙ_２）の反射率と、投影された輝度との関係が、下記式（１）の関係にあるとする。
【数１】
ｒ（ｘ_１，ｙ_１）／ｒ（ｘ_２，ｙ_２）≒ｉ（ｘ_２，ｙ_２）／ｉ（ｘ_１，ｙ_１）…（１）
対象物体上の２つの異なる点Ｐ_１（ｘ_１，ｙ_１）と点Ｐ_２（ｘ_２，ｙ_２）が上記式（１）の関係にあるとすると、下記式（２）の関係を満足することになる。
【数２】
ｒ（ｘ_１，ｙ_１）×ｉ（ｘ_１，ｙ_１）≒ｒ（ｘ_２，ｙ_２）×ｉ（ｘ_２，ｙ_２）…（２）
この結果、カメラによって撮影された像の持つ輝度は、対象物体上の２つの異なる点Ｐ_１（ｘ_１，ｙ_１）と点Ｐ_２（ｘ_２，ｙ_２）においてほぼ同じものとなってしまう。従って、各カメラの撮影像の輝度に基づいて対応付けを行なおうとした場合、一方のカメラで撮影された点と、他方のカメラで撮影された点の対応付けが困難になる。特に、複数のカメラを使用した撮影画像を用いて対応付け処理を実行する構成では、カメラ各々の特性によりカメラ毎に出力値が異なる場合がある。このようにカメラの特性により出力値が異なる場合、上述のような投光パターンを投影した画像を複数のカメラで撮影し、一方のカメラで撮影された点と、他方のカメラで撮影された点の対応付けを行なおうとしても対応付けが困難、あるいは不正確になることがある。
【０００８】
本発明は、上記問題点を解決することを目的としてなされたものであり、出力特性にばらつきがあるようなカメラ群を使用した場合でも正確な対応付けのもとに再符号化を行うことができ、かつ、対象物体のテクスチャへの依存を少なくすることで誤対応が生じにくく、信頼性の高い３次元形状計測方法および装置を提供することにある。
【０００９】
【課題を解決するための手段】
本発明によれば、上述の目的を達成するために特許請求の範囲に記載のとおりの構成を採用している。
【００１０】
ここでは、まず、特許請求の範囲の記載内容について補充的に説明を行う。
【００１１】
本発明の第１の側面によれば、上述の目的を解決するために、ストライプパターンを投影する投光手段と、前記投光手段と略同主点および略同光軸を有する第１のカメラと、前記投光手段と異なる主点を有する第２のカメラとを備える３次元形状計測装置において：参照物体に対して前記パターンを投影した場合と計測物体に対して前記パターンを投影した場合とに関して、前記第１のカメラおよび第２のカメラにより撮像したパターン投影画像のエッジ間の出力値を記憶する出力値記憶手段と；参照物体に関する前記第１のカメラの前記出力値と計測物体に関する前記第１のカメラの前記出力値とに基づいて、計測物体の反射係数を演算する反射係数演算手段と；計測物体の前記反射係数と参照物体に関する前記第２のカメラの前記出力値とを演算して、前記第２のカメラの出力予測値を算出する出力値予測手段と；前記第２のカメラの前記出力予測値と計測物体に関する前記第２のカメラの前記出力値とを用いて不一致度を算出する不一致度算出手段と；不一致度がもっとも小さくなるエッジを抽出して、前記第１のカメラの出力値に対する対応点として選択する対応づけ手段とを有することを特徴とする。
【００１２】
とくに、輝度値差の絶対値と隣接ストライプにおける輝度値比の差の絶対値との重みづけ和を算出することにより不一致度を算出することができる。
【００１３】
この構成においては、第１のカメラおよび第２のカメラによる参照物体上の投影パターンを撮像して得たエッジ間出力値情報および第１のカメラによる計測物体上の投影パタンを撮像して得たエッジ間出力値情報に基づいて、第２のカメラから得ることが予測されるエッジ間出力予測値情報を算出し、このエッジ間出力予測値情報と第２のカメラから計測物体に関して実際に得られるエッジ間出力値情報とにより第１のカメラおよび第２のカメラのパターン投影画像のエッジの対応づけを確実に実行することができる。
【００１４】
参照物体のパターン画像を用いているので第１のカメラと第２のカメラの間の品質のバラツキにより生じるエラーを抑制できる。輝度値差のみでなく、隣接ストライプの輝度値比の差を用いれば、局所的に輝度値の比が保存されているので、より一層確実なエッジの対応づけが可能となる。
【００１５】
また、本発明の第２の側面によれば、上述の目的を達成するために、ストライプパターンを投影する投光手段と、前記投光手段と略同主点および略同光軸を有する第１のカメラと、前記投光手段と異なる主点を有する第２のカメラとを備える３次元形状計測装置において：参照物体に対して前記パターンを投影した場合と計測物体に対して前記パターンを投影した場合とに関して、前記第１のカメラおよび第２のカメラにより撮像したパターン投影画像のエッジ間の出力値を記憶する出力値記憶手段と；参照物体に関する前記第１のカメラの前記出力値と計測物体に関する前記第１のカメラの前記出力値とに基づいて、計測物体の反射係数を演算する反射係数演算手段と；計測物体の前記反射係数と参照物体に関する前記第２のカメラの前記出力値とを演算して、前記第２のカメラの出力予測値を算出する出力値予測手段と；前記出力予測値に対してある補正値を用いてさらに補正を行った補正出力予測値を算出する補正手段と；計測物体に関する前記第２のカメラの前記出力値から、前記補正手段が算出した前記第２のカメラの補正出力予測値にもっとも近い出力値を持つエッジを抽出する対応候補抽出手段と；前記対応候補抽出手段により抽出した前記エッジに関する前記第２のカメラの前記出力値と、前記補正手段が算出した前記第２のカメラの補正出力予測値との差分を算出し、前記差分と所定の閾値とを比較して対応候補が正しい対応かどうかを判定する判定手段と；正しい対応の場合には前記補正値を前記第２のカメラの前記出力値および前記第２のカメラの前記出力予測値に基づいて更新し前記閾値を前記差分に基づいて更新し、誤った対応の場合には予め定められた値に更新する更新手段と；誤った対応の場合には前記第２のカメラの前記出力値から、前記第２のカメラの前記出力予測値にもっとも近い出力値を持つエッジを抽出する対応候補再抽出手段とを有することを特徴とする。
【００１６】
この構成においても、参照物体のパターン画像を用いているので第１のカメラと第２のカメラの間の品質のバラツキにより生じるエラーを抑制できる。しかも、計測物体の反射が必ずしも理想的な完全拡散反射でないことも考慮して徐々に補正を行い、しかも、表面の反射状態が急激に変化する場合には補正をリセットするようにしている。この結果、計測対象の表面状態に合致したエッジの対応づけが可能となる。前記補正値および前記閾値の更新は、双方を行うのでなく、一方のみを行うようにしてもよい。
【００１７】
なお、本発明は、装置やシステムとして実現するほかに方法の対応でも実施可能であり、少なくともその一部をコンピュータプログラムとして実装できることももちろんである。
【００１８】
【発明の実施の形態】
以下、実施例を用いて本発明を詳細に説明する。
【００１９】
［実施例１］
まず、本発明の実施例１を説明する。
【００２０】
図１は、実施例１が適用される３次元形状計測装置を模式的に示しており、図２は、測定対象および投影パターンの例を示しており、図３は、測定原理を示している。図１において、３次元形状計測装置１００は、３次元計測用にパターンを投影するパターン投影装置（投光手段。プロジェクタとも呼ぶ）１０１、ほぼ同主点、同光軸でパターンをモニタするモニタ用撮像装置（第１のカメラ。カメラ１とも呼ぶことがある）１０２、三角測量用撮像装置（第２のカメラ。カメラ２と呼ぶこともある）１０３を含んで構成される。基本的な構成は、特開２０００−９４４２号公報および特開２０００−６５５４２号公報の構成と同様である。投影装置１０１は、プロジェクタもしくはレーザスリット投影系を用いる。パターン投影装置１０１は、あらかじめ定められたコードに対応する輝度値によって投影パターンを形成し、投影を行う。投影パターンは、図２に示すような濃淡のあるスリットパターンを用い、例えば、図２右側に図示されている物体にパターン投影する。投影パターンはハーフミラー１０４を介して測定対象（物体）に投影され、その反射光がハーフミラー１０４で反射されてモニタ用撮像装置１０２に入射する。測定対象からの反射光はモニタ用撮像装置１０２とは異なる光路を経て三角測量用撮像装置１０３に入射する。
【００２１】
図３にその模様を示す。モニタ用撮像装置１０２と投影装置１０１をハーフミラー１０４などで、ほぼ同主点、同光軸に配置する。三角計測用に別光軸上に三角測量用撮像装置１０３を用意する。そして、図２に示すようなストライプパターンを投影する。ほぼ同主点、同光軸のモニタ用撮像装置（カメラ１）１０２で観測された画像（カメラ１撮像画像）から再符号化を実施し、測定用撮像装置（カメラ２）１０３で観測された画像（カメラ２撮像画像）とで距離画像を算出する。
【００２２】
再符合化について、図４のフローチャートを参照して詳しく説明する。本実施例では画像中のエッジの両側に位置する２つの輝度値を利用して符合化を行うが、３つ以上の輝度値を用いて符合化を行っても良く、限定するものではない。
【００２３】
まず、参照物体を用いた処理を行う（Ｓ１０）。カメラ１およびカメラ２はまず、図６に示すような、既知の距離に配置した参照物体上に投影された投影パターンを撮像する。この状態を標準状態と呼ぶこととする。参照物体としては、均一な反射係数を有する平板を用いることが望ましい。参照物体は既知距離に配置された既知形状の物体であるため、カメラ１画像とカメラ２画像におけるストライプの対応を容易に行うことができる。そこで、撮像画像から、各コードに対応する出力値を算出し、各カメラごとに記憶しておく。出力値の算出については、エッジ間に存在する複数画素について輝度値の平均値を算出するなどによって行うことができる。このとき、ｎ番目のエッジ位置をＥｎ、その左側および右側のコードをそれぞれＣｎ＿ｌｅｆｔ、Ｃｎ＿ｒｉｇｈｔとする。また、図７に示すように、Ｃｎ＿ｌｅｆｔおよびＣｎ＿ｒｉｇｈｔのコードに対するカメラ１の出力値をＯｎ＿ｌｅｆｔ＿１、Ｏｎ＿ｒｉｇｈｔ＿１とし、カメラ２の出力値をＯｎ＿ｌｅｆｔ＿２、Ｏｎ＿ｒｉｇｈｔ＿２とする。具体的な動作としては、Ｏｎ＿ｌｅｆｔ＿１、Ｏｎ＿ｒｉｇｈｔ＿１、Ｏｎ＿ｌｅｆｔ＿２、Ｏｎ＿ｒｉｇｈｔ＿２を各カメラ、各コードごとに記憶しておくこととなる。以上の動作により、標準状態におけるコードと出力値の対応を各カメラごとにつけたことになる。なお、この動作は計測前にあらかじめ行っておくことができ、計測ごとに繰り返す必要はない。
【００２４】
次に測定対象物に対する処理を行う（Ｓ１１）。測定対象物に対してパターンを投影し、各カメラで撮像を行う。カメラ１において撮像された画像に対してエッジ検出オペレータを作用させることでエッジを抽出する。カメラ１はパターン投影装置とほぼ同主点、同光軸を有するので、図８に示すように前述した標準状態とほぼ同じ位置にエッジ位置を検出することができる。すなわち、測定対象物における撮像画像について、標準状態におけるＥｎ、Ｃｎ＿ｌｅｆｔ、Ｃｎ＿ｒｉｇｈｔを対応させることができる。ただし、図９に示すように、カメラの出力値は対象物体のテクスチャによって変化させられているため、Ｏｎ’＿ｌｅｆｔ＿１、Ｏｎ’＿ｒｉｇｈｔ＿１となっている。Ｏｎ’＿ｌｅｆｔ＿１、Ｏｎ’＿ｒｉｇｈｔ＿１の算出については、標準状態においてＯｎ＿ｌｅｆｔ＿１、Ｏｎ＿ｒｉｇｈｔ＿１を算出したときと同じ手順を踏めば良い。このとき、
【数３】
Ｒｎ＿ｌｅｆｔ＝Ｏｎ’＿ｌｅｆｔ＿１／Ｏｎ＿ｌｅｆｔ＿１
Ｒｎ＿ｒｉｇｈｔ＝Ｏｎ’＿ｒｉｇｈｔ＿１／Ｏｎ＿ｒｉｇｈｔ＿１
とすると、Ｒｎ＿ｌｅｆｔ、Ｒｎ＿ｒｉｇｈｔは対象物体上での各ストライプにおける反射率と見なすことができる。カメラ１の各エッジにおいてＲｎを算出し、カメラ２においても同様にエッジ位置を検出し、Ｏｎ’＿ｌｅｆｔ＿２、Ｏｎ’＿ｒｉｇｈｔ＿２を算出する。
【００２５】
次に、カメラ１での計測物体パターン投影画像におけるエッジ間輝度情報と、カメラ２での計測物体パターン投影画像におけるエッジ間輝度情報、さらに、算出した反射係数：Ｒｎを用いて、エピポーララインに基づく対応付け処理を実行し（Ｓ１２）、その後距離計算を行う（Ｓ１３）。
【００２６】
カメラ１において算出しておいたＲｎ＿ｌｅｆｔ、Ｒｎ＿ｒｉｇｈｔをもちいれば、カメラ２が出力するであろう値（出力予測値）Ｏｎ’＿ｌｅｆｔ＿２＿ｐｒｅ、Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅを推定することができる。カメラ１とカメラ２が出力の線形性を有しており、さらに測定対象の表面性が完全拡散反射面と近似できる場合、以下の式で推定することができる。
【数４】
Ｏｎ’＿ｌｅｆｔ＿２＿ｐｒｅ＝Ｒｎ＿ｌｅｆｔ×Ｏｎ＿ｌｅｆｔ＿２
Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ＝Ｒｎ＿ｒｉｇｈｔ×Ｏｎ＿ｒｉｇｈｔ＿２
なお、カメラの出力特性が非線形であるときはその特性方程式を用いたものに、あるいは測定対象の表面性に関する完全拡散以外のモデルを適用する場合にはそのモデルを用いて予測づけを行うものであり、上式は一意なものではなく、説明のために簡単なモデルを使用しているに過ぎないものである。
【００２７】
対応付けについて、図５のフローチャートを用いて、説明する。対応づけは、エピポーラ線上においてＯｎ’＿ｌｅｆｔ＿２＿ｐｒｅ、Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅとＯｎ’＿ｌｅｆｔ＿２、Ｏｎ’＿ｒｉｇｈｔ＿２を用いた次式の不一致度算出関数を用いて、もっとも小さい不一致度（Ｄｉｓｃｒｅｐａｎｃｙ）をもつエッジを探索することによって行う（Ｓ２０、Ｓ２１、Ｓ２３）。
【数５】
Ｄｉｓｃｒｅｐａｎｃｙ＝ＩｎｔＤｉｆ＋α×ＢａｓｅＶａｌ×ＲａｔｉｏＤｉｆ
ただし、
【数６】
ＩｎｔＤｉｆ＝Ａｂｓ（Ｏｎ’＿ｌｅｆｔ＿２＿ｐｒｅ−Ｏｎ’＿ｌｅｆｔ＿２）＋Ａｂｓ（Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ−Ｏｎ’＿ｒｉｇｈｔ＿２）
また、
【数７】
ｉｆＯｎ’＿ｌｅｆｔ＿２＿ｐｒｅ＜Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ
ＲａｔｉｏＤｉｆ＝Ａｂｓ（Ｏｎ’＿ｌｅｆｔ＿２＿ｐｒｅ／Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ−Ｏｎ’＿ｌｅｆｔ＿２／Ｏｎ’＿ｒｉｇｈｔ＿２）
ＢａｓｅＶａｌ＝Ｏｎ’＿ｒｉｇｈｔ＿２
また、
【数８】
ｉｆＯｎ’＿ｌｅｆｔ＿２＿ｐｒｅ＞Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ
ＲａｔｉｏＤｉｆ＝Ａｂｓ（Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ／Ｏｎ’＿ｌｅｆｔ＿２＿ｐｒｅ−Ｏｎ’＿ｒｉｇｈｔｔ＿２／Ｏｎ’＿ｌｅｆｔ＿２）
ＢａｓｅＶａｌ＝Ｏｎ’＿ｌｅｆｔ＿２
【００２８】
不一致度算出関数は輝度値差の絶対値であるＩｎｔＤｉｆと隣接ストライプの比の差であるＲａｔｉｏＤｉｆとの重みつき和で構成されており、αは重みづけの定数である。ＲａｔｉｏＤｉｆを利用しているのは、輝度値自体は変化しても、局所的な領域においては隣接ストライプの輝度値の比は保存されることが予想されるためである。
【００２９】
カメラ２の測定画像において、あるエッジＥ＿ｗｒｏｎｇにおける輝度値をＯｎ’＿ｌｅｆｔ＿２＿ｗｒｏｎｇ、Ｏｎ’＿ｒｉｇｈｔ＿２＿ｗｒｏｎｇとし、別のエッジＥ＿ｃｏｒｒにおける輝度値をＯｎ’＿ｌｅｆｔ＿２＿ｃｏｒｒ、Ｏｎ’＿ｒｉｇｈｔ＿２＿ｃｏｒｒとする。このとき、測定された輝度値がたとえば以下のようであったとする。
【数９】
Ｏｎ’＿ｌｅｆｔ＿２＿ｗｒｏｎｇ＝６０
Ｏｎ’＿ｒｉｇｈｔ＿２＿ｗｒｏｎｇ＝７０
Ｏｎ’＿ｌｅｆｔ＿２＿ｃｏｒｒ＝４０
Ｏｎ’＿ｒｉｇｈｔ＿２＿ｃｏｒｒ＝７０
【００３０】
また、出力予測値は以下のような値であったとする。
【数１０】
Ｏｎ’＿ｌｅｆｔ＿２＿ｐｒｅ＝５０
Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ＝８０
このとき、
【数１１】
ＩｎｔＤｉｆ＿ｗｒｏｎｇ＝２０
ＩｎｔＤｉｒ＿ｃｏｒｒ＝２０
となり、輝度値差の絶対値だけでは対応づけの判定ができなくなるが、
【数１２】
ＲａｔｉｏＤｉｆｆ＿ｗｒｏｎｇ＝１６．２５
ＲａｔｉｏＤｉｆｆ＿ｃｏｒｒ＝３．７５
となり、α＝１としたときには
【数１３】
Ｄｉｓｃｒｅｐａｎｃｙ＿ｗｒｏｎｇ＝３６．２５
Ｄｉｓｃｒｅｐａｎｃｙ＿ｃｏｒｒ＝２３．７５
という値が得られる。
【００３１】
ここで、Ｅ＿ｗｒｏｎｇにおいては左側ストライプは予測値よりも大きな値であるが、右側ストライプは予測値よりも小さな値となっている。その一方で、Ｅ＿ｃｏｒｒにおいては、どちらのストライプも予測値よりも小さな値となっている。局所的な領域においてＥ＿ｗｒｏｎｇのような予測値との変化の逆転現象が発生することは考えにくいため、Ｅ＿ｃｏｒｒが正しい対応のエッジ点であると考えられる。すなわち、不一致度算出関数Ｄｉｓｃｒｅｐａｎｃｙは正しい対応を導きだすことができていることになる。
【００３２】
このように、カメラ１とカメラ２が同じ出力特性を持たない、すなわち、Ｏｎ’＿ｌｅｆｔ（ｒｉｇｈｔ）＿１とＯｎ’＿ｌｅｆｔ（ｒｉｇｈｔ）＿２が一致しない場合でもカメラ２において確実に対応点を検出することができるようになる。対応点が決定された後は一般的な三角測量の原理により距離が測定できる。
【００３３】
［実施例２］
つぎに本発明の実施例２について説明する。
【００３４】
この実施例は実施例１と同様に参照物体上の投影パターンを用いる。実施例１では、隣接ストライプの輝度比が局所的にも保存されるという性質を利用してその隣接ストライプの輝度比の差を用いて不一致度を算出した。実施例２では、多くの表面では反射の特性が連続的に変化することを利用して、反射モデル（モデルずれ量、補正値Ｏｆｆ）を導入しこれを徐々に変化させ、最適化を図るようにしている。しかも、反射の特性が急激に変化する場合には、その時点で補正値をリセットするようにしている。
【００３５】
以下、実施例２の３次元形状計測装置について説明する。ただし、エッジの対応づけ以外は実施例１と同様であるので、エッジの対応づけについてのみ説明する。
【００３６】
対応付けについて、図１０のフローチャートを用いて説明する。対応づけは、エピポーラ線上においてＯｎ’＿ｌｅｆｔ＿２＿ｐｒｅ、Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅにほぼ等しいＯｎ’＿ｌｅｆｔ＿２、Ｏｎ’＿ｒｉｇｈｔ＿２をもつエッジを探索することによって行う。
【００３７】
初期状態においては、予め定められた対応付けに関する閾値ｔｈを設定するとともに、モデルずれ比Ｏｆｆを１に設定する（Ｓ３０、Ｓ３１）。Ｏｎ’＿ｌｅｆｔ＿２＿ｐｒｅ×Ｏｆｆ、Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ×Ｏｆｆにほぼ等しいＯｎ’＿ｌｅｆｔ＿２、Ｏｎ’＿ｒｉｇｈｔ＿２をもつエッジを探索した後、次式によって算出される予測値との相違値であるＤｉｆｆを用いてその対応付けが正しいかどうかの判定を行う（Ｓ３２、Ｓ３３）。
【数１４】
Ｄｉｆｆ＝Ａｂｓ（Ｏｎ’＿ｌｅｆｔ＿２＿ｐｒｅ×Ｏｆｆ−Ｏｎ’＿ｌｅｆｔ＿２）＋Ａｂｓ（Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ×Ｏｆｆ−Ｏｎ’＿ｒｉｇｈｔ＿２）
Ｄｉｆｆ＜ｔｈのときこの対応付けは正しいと判定する。正しいと判定された対応の場合には、以下のようにＯｆｆの更新を行う（Ｓ３４）。
【数１５】
Ｏｆｆ_ｎｅｗ＝（Ｏｆｆ＋（Ｏｆｆ_ｎｏｗ）×α）／（１＋α）
ただし、
【数１６】
ｉｆＯｎ’＿ｌｅｆｔ＿２＿ｐｒｅ，Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ＞ｖＴｈ
Ｏｆｆ_ｎｏｗ＝（Ｏｎ’＿ｌｅｆｔ＿２／Ｏｎ’＿ｌｅｆｔ＿２＿ｐｒｅ＋Ｏｎ’＿ｒｉｇｈｔ＿２／Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ）／２
【数１７】
ｉｆＯｎ’＿ｌｅｆｔ＿２＿ｐｒｅ＞ｖＴｈ，Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ＜ｖＴｈ
Ｏｆｆ_ｎｏｗ＝Ｏｎ’＿ｌｅｆｔ＿２／Ｏｎ’＿ｌｅｆｔ＿２＿ｐｒｅ
【数１８】
ｉｆＯｎ’＿ｌｅｆｔ＿２＿ｐｒｅ＜ｖＴｈ，Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ＞ｖＴｈ
Ｏｆｆ_ｎｏｗ＝Ｏｎ’＿ｒｉｇｈｔ＿２／Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ
【数１９】
Ｏｎ’＿ｌｅｆｔ＿２＿ｐｒｅ＜ｖＴｈ，Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ＜ｖＴｈ
の場合はＯｆｆの更新を行わない。
【００３８】
上式でｖＴｈは予め定められた定数で、ｖＴｈ以下の値の場合は対応付けられた点と予測値との比が誤差を含みやすくなるためＯｆｆの更新の際に使用しないようにするための値である。予測値の算出においては、対象物体の表面性状について何らかのモデル化（たとえば完全拡散反射）を行っているが、実際の測定対象における反射はそのモデルでは完全に記述することができない場合が多い。モデルと実現象のずれを吸収するために上述のようなモデルずれ量Ｏｆｆの更新を行う。αはＯｆｆの更新の際における緩和係数であり、予め定められた定数である。また、このように実現象とのずれを吸収した後は、予測値と観測値との差異が小さくなることが期待されるため、対応付けが正しいかどうかの判定に用いるｔｈも次式のように更新を行う（Ｓ３４）。
【数２０】
Ｔｈ_ｎｅｗ＝（ｔｈ＋（Ｄｉｆｆ＋Ｍａｒｇｉｎ）×β）／（１＋β）
【００３９】
上式によって、新しい閾値は現在の相違値であるＤｉｆｆに近いものに更新することができる。βは予め定められた定数であり、ｔｈの更新の際における緩和係数である。また、Ｍａｒｇｉｎも予め定められた定数で、ｔｈが極端に小さくなることを防ぐためのものである。
【００４０】
Ｄｉｆｆ＞ｔｈとなり、誤った対応付けと判定したときは、その領域に関してそれまでに対応付けを行った領域と測定対象の表面性状が変化したことが予想されるため、Ｔｈを予め定められた元の値にリセットし、またＯｆｆも１にリセットする（Ｓ３５）。その後再度Ｏｎ’＿ｌｅｆｔ＿２＿ｐｒｅ×Ｏｆｆ、Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ×Ｏｆｆ（すなわち元の予測値であるＯｎ’＿ｌｅｆｔ＿２＿ｐｒｅ、Ｏｎ’＿ｒｉｇｈｔ＿２＿ｐｒｅ）にほぼ等しいＯｎ’＿ｌｅｆｔ＿２、Ｏｎ’＿ｒｉｇｈｔ＿２をもつエッジを探索し、正しい対応付けかどうかの判定を行う（Ｓ３６、Ｓ３７、Ｓ３８）。すべての対応づけが終了した場合には処理を終了する（Ｓ３９）。
【００４１】
この実施例によれば、参照物体の投影パターンを用いてカメラ１とカメラ２との間のバラツキを吸収でき、しかも、計測物体の表面反射に適応して補正を行うことにより、一層確実なエッジの対応づけが可能となる。
【００４２】
このように、カメラ１とカメラ２が同じ出力特性を持たない、すなわち、Ｏｎ’＿ｌｅｆｔ（ｒｉｇｈｔ）＿１とＯｎ’＿ｌｅｆｔ（ｒｉｇｈｔ）＿２が一致しない場合でもカメラ２において確実に対応点を検出することができるようになる。対応点が決定された後は一般的な三角測量の原理により距離が測定できる。
【００４３】
【発明の効果】
以上述べたように、本発明の３次元形状計測装置および３次元形状計測方法は、出力特性にばらつきがあるような、あるいは元々は出力特性がそろっていたにもかかわらず環境、経時変化などによりばらつきが生じたようなカメラ群を使用した場合でも再符号化を行うことができ、かつ、対象物体のテクスチャへの依存を少なくすることで誤対応が生じにくく、信頼性の高い３次元形状計測を行うことが可能になる。
【図面の簡単な説明】
【図１】本発明に係わる３次元形状計測装置のシステム概略図である。
【図２】本発明の３次元形状計測装置の投影装置の投影パターンの一例と読みとりオブジェクト例を示す図である。
【図３】図２のプロジェクトパターン及びオブジェクトにおいて、図１の構成で画像読みとりを実施したときの概略図である。
【図４】本発明の実施例１の動作を説明するフローチャートである。
【図５】図４のフローチャートの要部を説明するフローチャートである。
【図６】上述実施例の動作を説明する図である。
【図７】上述実施例の動作を説明する図である。
【図８】上述実施例の動作を説明する図である。
【図９】上述実施例の動作を説明する図である。
【図１０】本発明の実施例２の動作を説明するフローチャートである。
【符号の説明】
１００３次元形状計測装置
１０１パターン投影装置
１０１投影装置（プロジェクタ）
１０２モニタ用撮像装置（カメラ１）
１０３三角測量用撮像装置（カメラ２）
１０４ハーフミラー[0001]
BACKGROUND OF THE INVENTION
The present invention is based on a triangulation method in which a pattern projection image obtained by irradiating pattern light to a measurement object is imaged from different directions by a plurality of imaging means, and distance information is obtained based on the pattern change. Dimensional shape measurement technology.
[0002]
[Prior art]
There are an active method (Active vision) and a passive method (Passive vision) as a method for acquiring a three-dimensional shape. The active method includes (1) a laser method that emits laser light, ultrasonic waves, etc., measures the amount of reflected light and the arrival time from an object, and extracts depth information; and (2) a special pattern light source such as slit light. A pattern projection method for estimating the target shape from image information such as geometric deformation of the target surface pattern using (3), a method for obtaining three-dimensional information by forming contour lines with moire fringes by optical processing, etc. There is. On the other hand, the passive method uses knowledge about the appearance of an object, light source, illumination, shadow information, etc., and uses monocular stereoscopic vision to estimate 3D information from a single image, depth information of each pixel by triangulation principle And binocular stereopsis for estimating
[0003]
The present invention relates to a pattern projection method that is a three-dimensional measurement technique using an active method. In the pattern projection method, reference pattern light is projected onto a target object, and photographing is performed from a direction different from the direction in which the reference pattern light is projected. The photographed pattern is deformed depending on the shape of the object. By associating the observed deformation pattern with the projected pattern, three-dimensional measurement of the object can be performed. In the pattern projection method, high-speed and high-precision measurement is possible, but the problem is how to reduce miscorrespondence in associating the deformed pattern with the projected pattern.
[0004]
Therefore, in Japanese Patent Application No. 10-191063 (Japanese Patent Laid-Open No. 2000-9442) and Japanese Patent Application No. 10-247796 (Japanese Patent Laid-Open No. 2000-65542) related to the same applicant as the present application, the projected pattern is We proposed a 3D image capturing device that does not depend on the target object by generating a new code by feedback.
[0005]
[Problems to be solved by the invention]
The three-dimensional image photographing apparatus in Japanese Patent Application Nos. 10-191063 (Japanese Patent Application Laid-Open No. 2000-9442) and Japanese Patent Application No. 10-247796 (Japanese Patent Application Laid-Open No. 2000-65542) has a plurality of intensity patterns. Or by projecting what is encoded by multiple wavelengths. At that time, conventionally, the projection pattern changes due to the influence of the luminance information of the subject, the material, and the like, and when calculating the three-dimensional shape, an error occurs and an appropriate three-dimensional shape cannot be measured. Therefore, the three-dimensional imaging apparatus is arranged on the same optical axis as the light projecting element, monitors the amount of change in the light projection pattern according to the subject information, performs re-encoding, and measures the three-dimensional shape. .
[0006]
However, the above configuration is based on the premise that a new code obtained by one camera is generated in another camera as the same new code. This requires that all cameras in the device have the same output characteristics and output the same signal for a certain lightness on the target object. It is difficult in terms of cost to prepare a plurality of cameras that strictly satisfy this condition. Even if it is prepared, it is unavoidable that variations occur due to changes over time or fluctuations in the operating environment during use.
[0007]
Furthermore, the new code generated by the texture on the target object is not always unique. This is because the same new code can be generated by different textures at different points on the target object. That is, when the reflectance at the point p (x, y) is r (x, y) and the luminance of the projected code is i (x, y), two different points P on the target object ₁ (X ₁ , Y ₁ ) And point P ₂ (X ₂ , Y ₂ ) And the projected luminance are assumed to be in the relationship of the following formula (1).
[Expression 1]
r (x ₁ , Y ₁ ) / R (x ₂ , Y ₂ ) ≒ i (x ₂ , Y ₂ ) / I (x ₁ , Y ₁ ) ... (1)
Two different points P on the target object ₁ (X ₁ , Y ₁ ) And point P ₂ (X ₂ , Y ₂ ) Satisfies the relationship of the above formula (1), the relationship of the following formula (2) is satisfied.
[Expression 2]
r (x ₁ , Y ₁ ) X i (x ₁ , Y ₁ ) ≒ r (x ₂ , Y ₂ ) X i (x ₂ , Y ₂ ) ... (2)
As a result, the brightness of the image photographed by the camera has two different points P on the target object. ₁ (X ₁ , Y ₁ ) And point P ₂ (X ₂ , Y ₂ ) Is almost the same. Therefore, when the association is attempted based on the brightness of the captured image of each camera, it is difficult to associate the point captured by one camera with the point captured by the other camera. In particular, in the configuration in which the association process is executed using captured images using a plurality of cameras, the output value may differ from camera to camera depending on the characteristics of each camera. In this way, when the output value varies depending on the characteristics of the camera, an image obtained by projecting the projection pattern as described above is shot with a plurality of cameras, and the point shot with one camera and the point shot with the other camera Even if the association is attempted, the association may be difficult or inaccurate.
[0008]
The present invention has been made for the purpose of solving the above-described problems, and re-encoding can be performed based on accurate association even when using a group of cameras whose output characteristics vary. An object of the present invention is to provide a highly reliable three-dimensional shape measurement method and apparatus that can be made less susceptible to erroneous correspondence by reducing the dependence on the texture of the target object.
[0009]
[Means for Solving the Problems]
According to the present invention, in order to achieve the above-mentioned object, the configuration as described in the claims is adopted.
[0010]
Here, first, supplementary explanation will be given on the contents of the claims.
[0011]
According to the first aspect of the present invention, in order to solve the above-described object, a light projecting unit that projects a stripe pattern, and a first camera having substantially the same principal point and substantially the same optical axis as the light projecting unit. And a second camera having a different principal point from the light projecting means: when the pattern is projected onto a reference object and when the pattern is projected onto a measurement object The output value storage means for storing the output value between edges of the pattern projection image captured by the first camera and the second camera; the output value of the first camera related to a reference object and the measurement object related to the measurement object Reflection coefficient calculation means for calculating a reflection coefficient of a measurement object based on the output value of the first camera; and the output value of the second camera related to the reflection coefficient of the measurement object and a reference object The output value predicting means for calculating the predicted output value of the second camera by using the output predicted value of the second camera and the output value of the second camera related to the measurement object. A mismatch degree calculating means for calculating a mismatch degree; and an association means for extracting an edge having the smallest mismatch degree and selecting it as a corresponding point for the output value of the first camera.
[0012]
In particular, the degree of inconsistency can be calculated by calculating the weighted sum of the absolute value of the luminance value difference and the absolute value of the luminance value ratio difference in adjacent stripes.
[0013]
In this configuration, the output value information between edges obtained by imaging the projection pattern on the reference object by the first camera and the second camera and the projection pattern on the measurement object by the first camera are obtained. Based on the inter-edge output value information, the inter-edge output predicted value information predicted to be obtained from the second camera is calculated, and the inter-edge output predicted value information and the measured object are actually obtained from the second camera. The correspondence between the edges of the pattern projection images of the first camera and the second camera can be reliably executed based on the output value information between the edges.
[0014]
Since the pattern image of the reference object is used, errors caused by quality variations between the first camera and the second camera can be suppressed. By using not only the luminance value difference but also the luminance value ratio difference between adjacent stripes, since the luminance value ratio is stored locally, it is possible to associate edges more reliably.
[0015]
According to the second aspect of the present invention, in order to achieve the above-mentioned object, a light projecting means for projecting a stripe pattern, a first light projector having substantially the same principal point and substantially the same optical axis as the light projecting means. And a second camera having a principal point different from that of the light projecting means: when projecting the pattern onto a reference object and projecting the pattern onto a measurement object Output value storage means for storing an output value between edges of a pattern projection image captured by the first camera and the second camera; and the output value of the first camera related to a reference object and a measurement object A reflection coefficient calculating means for calculating a reflection coefficient of a measurement object based on the output value of the first camera relating to; the reflection coefficient of the measurement object; and the second camera relating to a reference object An output value predicting means for calculating an output predicted value of the second camera by calculating a force value; and calculating a corrected output predicted value obtained by further correcting the output predicted value using a correction value. Correcting means for extracting, from the output values of the second camera related to the measurement object, correspondence candidate extracting means for extracting an edge having an output value closest to the corrected output predicted value of the second camera calculated by the correcting means Calculating a difference between the output value of the second camera related to the edge extracted by the correspondence candidate extracting unit and the corrected output predicted value of the second camera calculated by the correcting unit; Determining means for comparing a predetermined threshold value to determine whether the correspondence candidate is a correct correspondence; in the case of a correct correspondence, the correction value is the output value of the second camera and the output of the second camera. Updating means for updating based on the measured value, updating the threshold based on the difference, and updating to a predetermined value in the case of an incorrect response; and in the case of an incorrect response, the second camera Corresponding candidate re-extraction means for extracting an edge having an output value closest to the predicted output value of the second camera from the output value.
[0016]
Even in this configuration, since the pattern image of the reference object is used, errors caused by quality variations between the first camera and the second camera can be suppressed. In addition, the correction is gradually performed in consideration of the fact that the reflection of the measurement object is not necessarily an ideal perfect diffuse reflection, and the correction is reset when the reflection state of the surface changes abruptly. As a result, it is possible to associate edges that match the surface state of the measurement target. The correction value and the threshold value may be updated only instead of both.
[0017]
It should be noted that the present invention can be implemented not only as an apparatus or a system but also as a method, and of course, at least a part thereof can be implemented as a computer program.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, the present invention will be described in detail using examples.
[0019]
[Example 1]
First, Example 1 of the present invention will be described.
[0020]
FIG. 1 schematically illustrates a three-dimensional shape measuring apparatus to which the first embodiment is applied, FIG. 2 illustrates an example of a measurement target and a projection pattern, and FIG. 3 illustrates a measurement principle. . In FIG. 1, a three-dimensional shape measuring apparatus 100 is a pattern projecting apparatus (light projecting means, also called a projector) 101 that projects a pattern for three-dimensional measurement, for monitoring that monitors a pattern with substantially the same principal point and the same optical axis. An imaging apparatus (first camera; sometimes referred to as camera 1) 102 and a triangulation imaging apparatus (second camera; sometimes referred to as camera 2) 103 are configured. The basic configuration is the same as the configuration of Japanese Patent Application Laid-Open Nos. 2000-9442 and 2000-65542. The projector 101 uses a projector or a laser slit projection system. The pattern projection apparatus 101 forms a projection pattern with a luminance value corresponding to a predetermined code and performs projection. As the projection pattern, a slit pattern with shading as shown in FIG. 2 is used. For example, the pattern is projected onto the object shown on the right side of FIG. The projection pattern is projected onto the measurement target (object) via the half mirror 104, and the reflected light is reflected by the half mirror 104 and enters the monitor imaging device 102. Reflected light from the measurement object enters the triangulation image pickup device 103 through an optical path different from that of the monitor image pickup device 102.
[0021]
FIG. 3 shows the pattern. The monitor imaging device 102 and the projection device 101 are arranged at substantially the same principal point and the same optical axis by a half mirror 104 or the like. An imaging device 103 for triangulation is prepared on another optical axis for triangulation measurement. Then, a stripe pattern as shown in FIG. 2 is projected. Re-encoding was performed from an image (camera 1 captured image) observed by the monitor imaging device (camera 1) 102 having substantially the same principal point and the same optical axis, and was observed by the measurement imaging device (camera 2) 103. A distance image is calculated from the image (camera 2 captured image).
[0022]
The re-encoding will be described in detail with reference to the flowchart of FIG. In this embodiment, encoding is performed using two luminance values located on both sides of the edge in the image. However, encoding may be performed using three or more luminance values, and is not limited.
[0023]
First, processing using a reference object is performed (S10). First, the camera 1 and the camera 2 capture an image of a projection pattern projected on a reference object arranged at a known distance as shown in FIG. This state is called a standard state. It is desirable to use a flat plate having a uniform reflection coefficient as the reference object. Since the reference object is an object having a known shape arranged at a known distance, the correspondence between stripes in the camera 1 image and the camera 2 image can be easily performed. Therefore, an output value corresponding to each code is calculated from the captured image and stored for each camera. The output value can be calculated by calculating an average value of luminance values for a plurality of pixels existing between edges. At this time, the n-th edge position is En, and the left and right codes are Cn_left and Cn_right, respectively. Also, as shown in FIG. 7, the output values of the camera 1 for the codes Cn_left and Cn_right are On_left_1 and On_right_1, and the output values of the camera 2 are On_left_2 and On_right_2. As a specific operation, On_left_1, On_right_1, On_left_2, and On_right_2 are stored for each camera and each code. With the above operation, the correspondence between the code and the output value in the standard state is set for each camera. This operation can be performed in advance before the measurement, and does not need to be repeated for each measurement.
[0024]
Next, processing for the measurement object is performed (S11). A pattern is projected onto the measurement object, and an image is taken with each camera. Edges are extracted by applying an edge detection operator to an image captured by the camera 1. Since the camera 1 has substantially the same principal point and the same optical axis as the pattern projection apparatus, the edge position can be detected at substantially the same position as in the standard state described above, as shown in FIG. That is, En, Cn_left, and Cn_right in the standard state can be associated with the captured image on the measurement object. However, as shown in FIG. 9, since the output value of the camera is changed by the texture of the target object, On′_left_1 and On′_right_1 are obtained. About calculation of On'_left_1 and On'_right_1, what is necessary is just to follow the same procedure as when On_left_1 and On_right_1 were calculated in the standard state. At this time,
[Equation 3]
Rn_left = On'_left_1 / On_left_1
Rn_right = On′_right — 1 / On_right — 1
Then, Rn_left and Rn_right can be regarded as the reflectance in each stripe on the target object. Rn is calculated at each edge of the camera 1, and the edge position is similarly detected at the camera 2, and On′_left — 2 and On′_right — 2 are calculated.
[0025]
Next, based on the epipolar line using the edge-to-edge luminance information in the measured object pattern projection image by the camera 1, the edge-to-edge luminance information in the measurement object pattern projection image by the camera 2, and the calculated reflection coefficient Rn Correlation processing is executed (S12), and then distance calculation is performed (S13).
[0026]
If Rn_left and Rn_right calculated in the camera 1 are used, values (output predicted values) On′_left_2_pre and On′_right_2_pre that the camera 2 will output can be estimated. When the cameras 1 and 2 have output linearity and the surface property of the measurement object can be approximated to a perfect diffuse reflection surface, it can be estimated by the following equation.
[Expression 4]
On'_left_2_pre = Rn_left × On_left_2
On'_right_2_pre = Rn_right × On_right_2
When the output characteristics of the camera are non-linear, predictions are made using the characteristic equation, or when applying a model other than complete diffusion related to the surface property of the object to be measured. Yes, the above equation is not unique, it just uses a simple model for illustration.
[0027]
The association will be described with reference to the flowchart of FIG. The correspondence is to search for an edge having the smallest inconsistency (Discrepany) using an inconsistency calculation function of the following equation using On'_left_2_pre, On'_right_2_pre and On'_left_2, On'_right_2 on the epipolar line. (S20, S21, S23).
[Equation 5]
Discrepancy = IntDif + α × BaseVal × RatioDif
However,
[Formula 6]
IntDif = Abs (On'_left_2_pre-On'_left_2) + Abs (On'_right_2_pre-On'_right_2)
Also,
[Expression 7]
if On'_left_2_pre <On'_right_2_pre
RatioDif = Abs (On'_left_2_pre / On'_right_2_pre-On'_left_2 / On'_right_2)
BaseVal = On'_right_2
Also,
[Equation 8]
if On'_left_2_pre>On'_right_2_pre
RatioDif = Abs (On'_right_2_pre / On'_left_2_pre-On'_righttt_2 / On'_left_2)
BaseVal = On'_left_2
[0028]
The discrepancy calculation function is composed of a weighted sum of IntDif which is the absolute value of the luminance value difference and RatioDif which is the difference of the ratio of adjacent stripes, and α is a weighting constant. RatioDif is used because the ratio of the luminance values of adjacent stripes is expected to be preserved in a local area even if the luminance value itself changes.
[0029]
In the measurement image of the camera 2, the luminance values at a certain edge E_wlong are On'_left_2_long and On'_right_2_long, and the luminance values at another edge E_corr are On'_left_2_corr and On'_right_2_corr. At this time, it is assumed that the measured luminance value is as follows, for example.
[Equation 9]
On'_left_2_wlong = 60
On'_right_2_wlong = 70
On'_left_2_corr = 40
On'_right_2_corr = 70
[0030]
Further, it is assumed that the predicted output value is as follows.
[Expression 10]
On'_left_2_pre = 50
On'_right_2_pre = 80
At this time,
[Expression 11]
IntDif_long = 20
IntDir_corr = 20
Therefore, it is not possible to determine the association only with the absolute value of the luminance value difference,
[Expression 12]
RatioDiff_long = 16.25
RatioDiff_corr = 3.75
When α = 1
[Formula 13]
Discrepancy_long = 36.25
Discrepancy_corr = 23.75
Is obtained.
[0031]
Here, in E_long, the left stripe is larger than the predicted value, but the right stripe is smaller than the predicted value. On the other hand, in E_corr, both stripes are smaller than predicted values. Since it is unlikely that a reversal phenomenon of a change from a predicted value such as E_long occurs in a local region, it is considered that E_corr is a correct corresponding edge point. That is, the discrepancy degree calculation function Discrepancy can derive a correct correspondence.
[0032]
As described above, even when the camera 1 and the camera 2 do not have the same output characteristics, that is, when the On'_left (right) _1 and the On'_left (right) _2 do not match, the corresponding point is reliably detected in the camera 2. Will be able to. After the corresponding points are determined, the distance can be measured by a general triangulation principle.
[0033]
[Example 2]
Next, a second embodiment of the present invention will be described.
[0034]
In this embodiment, the projection pattern on the reference object is used as in the first embodiment. In Example 1, the degree of inconsistency was calculated using the difference in luminance ratio between adjacent stripes by utilizing the property that the luminance ratio between adjacent stripes is also stored locally. In the second embodiment, a reflection model (model deviation amount, correction value Off) is introduced by utilizing the fact that reflection characteristics continuously change on many surfaces, and this is gradually changed to achieve optimization. I have to. In addition, when the reflection characteristic changes abruptly, the correction value is reset at that time.
[0035]
Hereinafter, the three-dimensional shape measuring apparatus according to the second embodiment will be described. However, other than the edge association, the process is the same as in the first embodiment, and therefore only the edge association will be described.
[0036]
The association will be described with reference to the flowchart of FIG. The association is performed by searching for an edge having On'_left_2 and On'_right_2 that is substantially equal to On'_left_2_pre and On'_right_2_pre on the epipolar line.
[0037]
In the initial state, a threshold value th related to a predetermined association is set, and the model deviation ratio Off is set to 1 (S30, S31). After searching for an edge having On'_left_2, On'_right_2 that is approximately equal to On'_left_2_pre x Off and On'_right_2_pre x Off, the correlation is performed using Diff, which is a difference value from the predicted value calculated by the following equation: It is determined whether or not is correct (S32, S33).
[Expression 14]
Diff = Abs (On′_left_2_pre × Off−On′_left_2) + Abs (On′_right_2_pre × Off−On′_right_2)
When Diff <th, it is determined that this association is correct. If the response is determined to be correct, Off is updated as follows (S34).
[Expression 15]
Off _new = (Off + (Off _now ) × α) / (1 + α)
However,
[Expression 16]
if On'_left_2_pre, On'_right_2_pre> vTh
Off _now = (On'_left_2 / On'_left_2_pre + On'_right_2 / On'_right_2_pre) / 2
[Expression 17]
if On'_left_2_pre> vTh, On'_right_2_pre <vTh
Off _now = On'_left_2 / On'_left_2_pre
[Formula 18]
if On'_left_2_pre <vTh, On'_right_2_pre> vTh
Off _now = On'_right_2 / On'_right_2_pre
[Equation 19]
On'_left_2_pre <vTh, On'_right_2_pre <vTh
In the case of, Off is not updated.
[0038]
In the above equation, vTh is a predetermined constant. If the value is less than vTh, the ratio between the associated point and the predicted value tends to include an error, so that it is not used when updating Off. Value. In the calculation of the predicted value, some modeling (for example, complete diffuse reflection) is performed on the surface property of the target object. However, in many cases, the reflection in the actual measurement target cannot be described completely. In order to absorb the deviation between the model and the actual phenomenon, the model deviation amount Off is updated as described above. α is a relaxation coefficient when Off is updated, and is a predetermined constant. In addition, after absorbing the deviation from the actual phenomenon in this way, it is expected that the difference between the predicted value and the observed value will be small. Therefore, th used for determining whether the correspondence is correct is as follows: (S34).
[Expression 20]
Th _new = (Th + (Diff + Margin) × β) / (1 + β)
[0039]
With the above formula, the new threshold can be updated to be close to the current difference value Diff. β is a predetermined constant and is a relaxation coefficient when th is updated. Further, Margin is a predetermined constant for preventing th from becoming extremely small.
[0040]
When Diff> th and it is determined that there is an incorrect association, it is expected that the surface property of the measurement target and the region that has been associated with the region have changed so far. Is reset to 1 and Off is also reset to 1 (S35). After that, search again on On__left_2, On'_right_2 that has On'_left_2, On'_right_2 that is approximately equal to On'_left_2_pre × Off, On'_right_2_pre × Off (ie, On'_left_2_pre, On'_right_2_pre, which is the original predicted value) It is determined whether or not (S36, S37, S38). If all the associations are completed, the process is terminated (S39).
[0041]
According to this embodiment, the variation between the camera 1 and the camera 2 can be absorbed using the projection pattern of the reference object, and the edge can be more surely corrected by performing the correction in accordance with the surface reflection of the measurement object. Can be associated.
[0042]
As described above, even when the camera 1 and the camera 2 do not have the same output characteristics, that is, when the On'_left (right) _1 and the On'_left (right) _2 do not match, the corresponding point is reliably detected in the camera 2. Will be able to. After the corresponding points are determined, the distance can be measured by a general triangulation principle.
[0043]
【The invention's effect】
As described above, the three-dimensional shape measurement apparatus and the three-dimensional shape measurement method of the present invention have variations in output characteristics, or the environment and time-dependent changes even though the output characteristics were originally gathered. Re-encoding can be performed even when a group of cameras with variations is used, and by reducing the dependence on the texture of the target object, erroneous correspondence is less likely to occur, and highly reliable three-dimensional shape measurement It becomes possible to do.
[Brief description of the drawings]
FIG. 1 is a system schematic diagram of a three-dimensional shape measuring apparatus according to the present invention.
FIG. 2 is a diagram showing an example of a projection pattern and an example of a reading object of the projection device of the three-dimensional shape measuring apparatus of the present invention.
3 is a schematic diagram when the image reading is performed with the configuration of FIG. 1 in the project pattern and object of FIG. 2;
FIG. 4 is a flowchart for explaining the operation of the first embodiment of the present invention.
FIG. 5 is a flowchart for explaining a main part of the flowchart of FIG. 4;
FIG. 6 is a diagram for explaining the operation of the embodiment.
FIG. 7 is a diagram for explaining the operation of the embodiment.
FIG. 8 is a diagram for explaining the operation of the embodiment.
FIG. 9 is a diagram for explaining the operation of the embodiment.
FIG. 10 is a flowchart for explaining the operation of the second embodiment of the present invention.
[Explanation of symbols]
100 3D shape measuring device
101 pattern projector
101 Projector
102 Imaging device for monitor (camera 1)
103 Triangulation imaging device (Camera 2)
104 half mirror

Claims

3 comprising a light projecting means for projecting a stripe pattern, a first camera having substantially the same principal point and substantially the same optical axis as the light projecting means, and a second camera having a principal point different from the light projecting means. In the three-dimensional shape measuring device,
Stores output values between edges of pattern projection images captured by the first camera and the second camera with respect to a case where the pattern is projected onto a reference object and a case where the pattern is projected onto a measurement object. Output value storage means for
A reflection coefficient calculation means for calculating a reflection coefficient of the measurement object based on the output value of the first camera related to the reference object and the output value of the first camera related to the measurement object;
An output value prediction means for calculating the output value of the second camera by calculating the reflection coefficient of the measurement object and the output value of the second camera related to the reference object;
Using the output predicted value of the second camera and the output value of the second camera related to the measurement object, an absolute value of a difference between the luminance value of the output value and the luminance value of the output predicted value; A mismatch degree calculating means for calculating a mismatch degree by calculating a weighted sum of a difference between a luminance value ratio in an adjacent stripe of the output value and a luminance value ratio in an adjacent stripe of the output predicted value;
A three-dimensional shape measuring apparatus comprising: an association unit that extracts an edge having the smallest mismatch degree and selects it as a corresponding point for the output value of the first camera.

3 comprising a light projecting means for projecting a stripe pattern, a first camera having substantially the same principal point and substantially the same optical axis as the light projecting means, and a second camera having a principal point different from the light projecting means. In the three-dimensional shape measuring device,
Stores output values between edges of pattern projection images captured by the first camera and the second camera with respect to a case where the pattern is projected onto a reference object and a case where the pattern is projected onto a measurement object. Output value storage means for
A reflection coefficient calculation means for calculating a reflection coefficient of the measurement object based on the output value of the first camera related to the reference object and the output value of the first camera related to the measurement object;
An output value prediction means for calculating the output value of the second camera by calculating the reflection coefficient of the measurement object and the output value of the second camera related to the reference object;
A mismatch degree calculation means for calculating a mismatch degree using the predicted output value of the second camera and the output value of the second camera related to a measurement object;
A three-dimensional shape measuring apparatus comprising: an association unit that extracts an edge having the smallest mismatch degree and selects it as a corresponding point for the output value of the first camera.

3 comprising a light projecting means for projecting a stripe pattern, a first camera having substantially the same principal point and substantially the same optical axis as the light projecting means, and a second camera having a principal point different from the light projecting means. In the three-dimensional shape measuring device,
First edge-to-edge output value information obtained by imaging the projection pattern on the reference object with the first camera, and between the second edges obtained by imaging the projection pattern on the measurement object with the first camera Based on the output value information and the third edge output value information obtained by imaging the projection pattern on the reference object by the second camera, the projection pattern on the measurement object is imaged by the second camera. Calculate output value prediction information between edges that are predicted to be obtained,
The first camera and the second camera by comparing the output value prediction information between edges and the output value information between the fourth edges obtained by imaging the projection pattern on the measurement object by the second camera. A three-dimensional shape measuring apparatus characterized by associating edges of a pattern projection image picked up by (1).

The difference between the luminance value of the inter-edge output value prediction information and the luminance value of the fourth inter-edge output value information, the ratio of the luminance values of adjacent stripes of the inter-edge output value prediction information, and the fourth edge The three-dimensional shape measurement apparatus according to claim 3, wherein edges of the pattern projection image are associated with each other based on a difference between luminance value ratios of adjacent stripes in the inter-output value information.

Using a light projecting means for projecting a stripe pattern, a first camera having substantially the same principal point and substantially the same optical axis as the light projecting means, and a second camera having a principal point different from the light projecting means In a 3D shape measurement method for measuring a 3D shape,
Stores output values between edges of pattern projection images captured by the first camera and the second camera with respect to a case where the pattern is projected onto a reference object and a case where the pattern is projected onto a measurement object. An output value storing step,
A reflection coefficient calculation step of calculating a reflection coefficient of the measurement object based on the output value of the first camera related to the reference object and the output value of the first camera related to the measurement object;
An output value prediction step of calculating an output predicted value of the second camera by calculating the reflection coefficient of the measurement object and the output value of the second camera related to a reference object;
Using the output predicted value of the second camera and the output value of the second camera related to the measurement object, an absolute value of a difference between the luminance value of the output value and the luminance value of the output predicted value; A mismatch degree calculation step of calculating a mismatch degree by calculating a weighted sum of a difference between a luminance value ratio in an adjacent stripe of the output value and a luminance value ratio in an adjacent stripe of the output predicted value;
A three-dimensional shape measuring method comprising: an associating step of extracting an edge having the smallest mismatch degree and selecting it as a corresponding point for the output value of the first camera.

3 comprising a light projecting means for projecting a stripe pattern, a first camera having substantially the same principal point and substantially the same optical axis as the light projecting means, and a second camera having a principal point different from the light projecting means. In the three-dimensional shape measuring device,
Stores output values between edges of pattern projection images captured by the first camera and the second camera with respect to a case where the pattern is projected onto a reference object and a case where the pattern is projected onto a measurement object. Output value storage means for
A reflection coefficient calculation means for calculating a reflection coefficient of the measurement object based on the output value of the first camera related to the reference object and the output value of the first camera related to the measurement object;
An output value prediction means for calculating the output value of the second camera by calculating the reflection coefficient of the measurement object and the output value of the second camera related to the reference object;
Correction means for calculating a corrected output predicted value obtained by further correcting the output predicted value using a certain correction value;
Correspondence candidate extraction means for extracting an edge having an output value closest to the corrected output predicted value of the second camera calculated by the correction means from the output value of the second camera related to a measurement object;
Calculating a difference between the output value of the second camera related to the edge extracted by the correspondence candidate extracting unit and the corrected output predicted value of the second camera calculated by the correcting unit; A determination means for comparing the threshold value to determine whether the correspondence candidate is a correct correspondence;
In the case of correct correspondence, the correction value is updated based on the output value of the second camera and the predicted output value of the second camera, and the threshold is updated based on the difference. Update means for updating to a predetermined value in the case,
Corresponding candidate re-extracting means for extracting an edge having an output value closest to the predicted output value of the second camera from the output value of the second camera in the case of incorrect correspondence. A three-dimensional shape measuring apparatus.

Using a light projecting means for projecting a stripe pattern, a first camera having substantially the same principal point and substantially the same optical axis as the light projecting means, and a second camera having a principal point different from the light projecting means In a 3D shape measurement method for measuring a 3D shape,
Stores output values between edges of pattern projection images captured by the first camera and the second camera with respect to a case where the pattern is projected onto a reference object and a case where the pattern is projected onto a measurement object. An output value storing step,
A reflection coefficient calculation step of calculating a reflection coefficient of the measurement object based on the output value of the first camera related to the reference object and the output value of the first camera related to the measurement object;
An output value prediction step of calculating an output predicted value of the second camera by calculating the reflection coefficient of the measurement object and the output value of the second camera related to a reference object;
A correction step of calculating a corrected output predicted value obtained by further correcting the output predicted value using a certain correction value;
A candidate extraction step for extracting an edge having an output value closest to the corrected output predicted value of the second camera calculated by the correction unit from the output value of the second camera related to the measurement object;
Calculating a difference between the output value of the second camera related to the edge extracted by the correspondence candidate extracting unit and the corrected output predicted value of the second camera calculated by the correcting unit; A determination step of comparing the threshold value to determine whether the correspondence candidate is a correct correspondence;
In the case of correct correspondence, the correction value is updated based on the output value of the second camera and the predicted output value of the second camera, and the threshold is updated based on the difference. An update step for updating to a predetermined value in some cases;
A correspondence candidate re-extracting step of extracting an edge having an output value closest to the output predicted value of the second camera from the output value of the second camera in the case of incorrect correspondence. A three-dimensional shape measuring method.