JP4155071B2 - Ray direction measuring device and measuring method - Google Patents

Description

【００４０】
【数２】

【００４１】
と表すことができる。
【００４２】
この数式（１）数式（２）から
【００４３】
【数３】

【００４４】
【数４】

【００４５】
が得られる。
【００４６】
以上の条件より、数式（３）、数式（４）（ｉ＝１，２，・・・，Ｉ）における既知パラメータ｛ｐｉ｝、｛ｐ´ｉ｝から、未知パラメータ｛ｍｉ｝、｛α´ｉ｝、｛Ｒ´｝、｛ｂ｝、｛ｂ´｝を推定する。数式（３）（４）より｛ｍｉ｝を消去すると
【００４７】
【数５】

【００４８】
が得られる。
【００４９】
なおここで、測定方法の定義より、各投影点｛ｐｉ｝、｛ｐ´ｉ｝ はそれぞれ同一の平面上にある。そこで数式（５）をこの条件を考慮したものに書き換える。
【００５０】
いま、図６における２つの平面｛Π（１）｝、｛Π（２）｝を、それぞれ法線ベクトル｛ｈ｝、｛ｈ´｝と、各座標原点から平面までの距離｛ｄ｝、｛ｄ´｝で表すことにする。平面上に存在する３つ以上の点（｛ｐ１（１）｝、｛ｐ２（１）｝、｛ｐ３（１）｝、｛ｐ４（１）｝）、（｛ｐ１（２）｝、｛ｐ２（２）｝、｛ｐ３（２）｝、｛ｐ４（２）｝）の３次元位置がそれぞれ既知なので、各平面のパラメータ｛ｈ｝、｛ｄ｝、｛ｈ´｝、｛ｄ´｝はそこから一意に算出可能である。なお、｛ｈ｝、｛ｄ｝は｛Ｕ（１）｝座標系で、｛ｈ´｝、｛ｄ´｝は｛Ｕ（２）｝座標系で表されたものである。
【００５１】
平面｛Π（１）｝上に存在する任意の点｛ｐｉ｝と、平面｛Π（２）｝に存在する任意の点｛ｐ´ｉ｝の満たすべき数式は
【００５２】
【数６】

【００５３】
【数７】

【００５４】
である。これらの式で、（・，・）はベクトルの内積を示す。
【００５５】
数式（６）（７）を数式（５）に代入して
【００５６】
【数８】

【００５７】
ここで、
【００５８】
【数８１】

【００５９】
で表されるテンソル積の関係を用いると、数式（８）は
【００６０】
【数８２】

【００６１】
と書き換えられ、これより
【００６２】
【数９】

【００６３】
と置いたときに｛ｐｉ｝と｛ｐ´ｉ｝の関係を
【００６４】
【数１０】

【００６５】
の形式で表すことができる。
【００６６】
この数式（１０）を本問題における基本方程式と定義する。
【００６７】
数式（１０）における変換行列｛Ａ´｝を、観測パラメータより求めていくことにする。数式（１０）では両辺にスカラー倍の自由度があるので、
【００６８】
【数１０１】

【００６９】
とおくことができる。これより、
【００７０】
【数１０２】

【００７１】
として数式（１０）に代入すると、
【００７２】
【数１１】

【００７３】
なる２式が得られる。未知パラメータ（｛ａｉｊ｝）の数は８つなので、４組（ｉ＝１，２，３，４）の数式（１０）より行列｛Ａ´｝は一意に求まる。
【００７４】
ある光軸ｉが２つの異なる平面上に投影され、それぞれ個別の座標系による座標表現で、｛ｐ｝、｛ｐ´ｉ｝として観測されたとき、｛ｐｉ｝と｛ｐ´ｉ｝の関係が数式（１０）の基本方程式で表されることがわかった。また、そこで用いられる未知パラメータ｛Ａ´｝が、４組（ｉ＝１、２、３、４）の観測点｛ｐ１｝、｛ｐ２｝、｛ｐ３｝、｛ｐ４｝、｛ｐ´１｝、｛ｐ´２｝、｛ｐ´３｝、｛ｐ´４｝から求めることができることがわかった。
【００７５】
数式（３）（４）における｛ｍｉ｝（ｉ＝１、２、・・・，Ｉ）を求めるには、｛Ａ´｝を分解することにより数式（５）における未知パラメータ｛ｂ｝、｛ｂ´｝、｛Ｒ´｝を求める必要がある。
【００７６】
｛Ａ´｝は前述のように実質的に８つのパラメータで表現され、これに対して、｛ｂ｝、｛ｂ´｝、｛Ｒ´｝には合計９つのパラメータがある。
【００７７】
図５に示すように観測回数を増やし、例えば３回の観測を行うと、第１回の観測と第２回の観測、および、第１回の観測と第３回の観測、の組み合わせにより、２つの変換行列、｛Ａ´｝、｛Ａ´´｝が得られる。よってここでの拘束パラメータ数は、８×２＝１６に増える。これに対して未知パラメータは、｛ｂ｝、｛ｂ´｝、｛Ｒ´｝に｛ｂ｝、｛ｂ´´｝、｛Ｒ´´｝が加わるが、ここで｛ｂ｝は共通なので結局、｛ｂ｝、｛ｂ´｝、｛Ｒ´｝、｛ｂ´´｝、｛Ｒ´´｝の合計１５となる。これにより未知パラメータ数は拘束数より少なくなるので、それぞれの未知パラメータを求めることができる。
【００７８】
まず第１回の観測と第２回の観測の組み合わせによる変換行列｛Ａ´｝において、未知パラメータ｛ｂ｝、｛ｂ´｝、｛Ｒ´｝のなかの｛ｂ｝が取るべき値に関する条件（｛ｂ｝に関する条件式）を求める。
【００７９】
いま、ある光線方向｛ｍｖ｝を考える。この光線は、平面｛Π（１）｝上のある無限遠の位置に投影され、また、同時に平面｛Π（２）｝上のある無限遠の位置に投影されるものとする。このような光線｛ｍｖ｝の条件は、図７に示すように光線方向が平面｛Π（１）｝と平面｛Π（２）｝の交線方向｛ｌ｝に等しくなる場合のみ成り立つ。
【００８０】
以上の条件を満たす方向｛ｍｖ｝を実際に求める。
【００８１】
平面上の無限遠点を、方向を示す単位ベクトル｛ｃ｝で表現すると、実際の無限遠点は
【００８２】
【数１１１】

【００８３】
となるが、数式（１０）におけるホモグラフイーでは、定数倍の自由度があり、２平面間の変換も問題なく行えるので、投影点を考える上でスケール定数｛ｚ｝については省略することができる。
【００８４】
このような考えから、光線｛ｍｖ｝の平面｛Π（１）｝上の無限遠投影点と、平面｛Π（２）｝上の無限遠投影点を、それぞれ単位方向ベクトル｛ｐｖ｝、｛ｐ´ｖ｝と表現する。
図７に示すように｛ｐｖ｝、｛ｐ´ｖ｝は方向｛ｍｖ｝、すなわち２平面の交線｛ｌ｝の方向を表すことになる。なお｛ｐｖ｝、｛ｐ´ｖ｝はそれぞれ別の座標系（｛Ｕ（１）｝、｛Ｕ（２）｝座標系）で表現されていることに注意する。
【００８５】
このような条件を満たす｛ｐｖ｝、｛ｐ´ｖ｝を求めると、平面｛Π（２）｝の法線が｛ｈ´｝だから、平面｛Π（２）｝の無限遠方向｛ｐ´ｖ｝は、
【００８６】
【数１２】

【００８７】
を満たす。
【００８８】
同様に、平面｛Π（１）｝の法線が｛ｈ｝であるから｛ｐｖ｝は、（｛ｈ｝，｛ｐｖ｝）＝０を満たす。
数式（１０）より｛ｐｖ｝と｛ｐ´ｖ｝の関係は
【００８９】
【数１２１】

【００９０】
だから、
【００９１】
【数１３】

【００９２】
を満たす。
【００９３】
数式（１２）、数式（１３）より｛ｐ´ｖ｝は
【００９４】
【数１４】

【００９５】
によって求まる。なおここで normalize［・］はベクトルのノルムが１となるような正規化操作を、×はベクトルの外積を表す。
【００９６】
以上より、２平面の交線方向が、｛Ｕ（２）｝座標系によって表されるベクトル｛ｐ´ｖ｝として求まった。
【００９７】
次に図８に示すように｛ｐ´ｖ｝（交線｛ｌ｝の方向）に直交し、平面｛Π（２）｝上の無限遠点に投影する光線方向を｛ｎｖ｝とする。この無限遠投影点を｛ｑ´ｖ｝と表す。
【００９８】
平面｛Π（２）｝の法線が｛ｈ´｝だから、
【００９９】
【数１５】

【０１００】
である。よって｛ｑ´ｖ｝は｛ｐ´ｖ｝と｛ｈ´｝の両方に直交することになる。｛ｐ´ｖ｝は数式（１４）で求められるので、
【０１０１】
【数１６】

【０１０２】
となる。
【０１０３】
光線｛ｎｖ｝が平面｛Π（１）｝上に投影する点を｛ｑｖ｝とおく。数式（１０）より
【０１０４】
【数１６１】

【０１０５】
となる。ここで｛ｑｖ｝は無限遠点ではないので、単位ベクトルではなく、平面｛Π（１）｝上の実際の点として求める必要がある。よってスケール係数を特定する必要がある。平面の式より（｛ｈ｝,｛ｑｖ｝）＝｛ｄ｝だから、
【０１０６】
【数１７】

【０１０７】
となる。
【０１０８】
以上より、｛ｑｖ｝が特定された。図８に示すように照射位置｛ｂ｝から｛ｑｖ｝に向かう方向（｛ｑｖ｝−｛ｂ｝）は｛ｌ｝（＝｛ｐｖ｝）に直交する。よって、｛ｂ｝は点｛ｑｖ｝を通り、｛ｌ｝に直交する平面｛Π（ｑ）｝上になければならないことが分かる。
【０１０９】
次に図９に示すように、平面｛Π（２）｝上の無限遠点に投影点を投影する２つの光線｛ｕｑｖ｝，｛ｕｖ｝を考えて、さらに条件を絞り込んでいくことにする。なお｛ｕｑｖ｝は前述点｛ｑｖ｝に投影点を投影する光線とする。
【０１１０】
２つの光線の平面｛Π（１）｝上の投影点｛ｑｖ｝，｛ｃｖ｝と照射点｛ｂ｝のつくる角度｛θ｝は、それらの光線が平面｛Π（２）｝上に投影する点｛ｑ´ｖ｝、｛ｃ´ｖ｝と照射点｛ｂ´｝がつくる角度｛θ´｝となることに注意する。
【０１１１】
図９からわかるように、光線｛ｕｖ｝が、平面｛Π（１）｝に投影する点｛ｃｖ｝は、｛ｑｖ｝をとおり方向｛ｌ｝（＝｛ｐｖ｝）に平行な直線｛Ｌ｝上にあることから、この点を
【０１１２】
【数１８】

【０１１３】
と表すことができる。同じ光線｛ｕｖ｝が平面｛Π（２）｝上に投影する無限遠点｛ｃ´ｖ｝は数式（１０）より
【０１１４】
【数１９】

【０１１５】
である。
【０１１６】
ここで、数式（１２１）かつ｜｛ｐｖ｝｜＝１だから
【０１１７】
【数１９１】

【０１１８】
であり、また数式（１７）より
【０１１９】
【数１９２】

【０１２０】
だから、これを数式（１９）に代入すると
【０１２１】
【数２０】

【０１２２】
となる。
【０１２３】
さて、ここで、照射装置から投影点｛ｑｖ｝に向かう光線｛ｕｑｖ｝、すなわち数式（１８）において、｛λ｝＝０とおいた場合の光線と、数式（１８）において｛λ｝＝０以外の場合での光線｛ｕｖ｝のつくる角度｛θ｝に着目する。
【０１２４】
照射位置｛ｂ｝から｛ｑｖ｝までの距離は｜｛ｂ｝−｛ｑｖ｝｜だから
【０１２５】
【数２１】

【０１２６】
となる。
【０１２７】
一方、これらの２つの光線は平面｛Π（２）｝上の無限遠点に投影されるが、それらを｛ｑ´ｖ｝、｛ｃ´ｖ｝と方向ベクトルであらわし、これらがつくる角度｛θ´｝をもとめると、
【０１２８】
【数２２】

【０１２９】
となる。｛θ｝＝｛θ´｝となるので、数式（２１）と数式（２２）をひとつにまとめて、数式（２０）を代入すると、
【０１３０】
【数２３】

【０１３１】
が得られる。
【０１３２】
任意の｛λ｝について数式（２３）が満たされるためには
【０１３３】
【数２３１】

【０１３４】
でなければならない。これは照射位置｛ｂ｝と点｛ｑｖ｝との距離｛ｒ｝が常に、
【０１３５】
【数２４】

【０１３６】
すなわち点｛ｑｖ｝を中心とした半径｛ｒ｝の球上に｛ｂ｝が存在しなければならないことを示している。
【０１３７】
なおここで｛ｐ´ｖ｝、｛ｑ´ｖ｝、｛ｑｖ｝はそれぞれ数式（１４）（１６）（１７）より求められるものである。
【０１３８】
以上をまとめると、照射位置ｂは図１０（ａ）に示すように、平面｛Π（ｑ）｝上の、点｛ｑｖ｝を中心とする半径｛ｒ｝の円｛Ω（１）｝上に存在しなければいけないことがわかった。
【０１３９】
以上、ホモグラフィー｛Ａ´｝が与えられたときに照射位置｛ｂ｝が取るべき条件について考えてきたので、上記の結論は｛ｂ｝の位置に関する必要条件ということができる。証明は省略するが、これは十分条件にもなっている。
【０１４０】
以上より、任意の２つの平面｛Π（１）｝、｛Π（２）｝に光線を照射したとき、それぞれの投影位置から互いの変換行列｛Ａ´｝が得られ、その際に、照射位置｛ｂ｝が空間内のある円｛Ω（１）｝に存在することが必要十分条件となることがわかった。
しかしながら、これだけでは｛ｂ｝を一意に特定できず、目的となる光軸方向｛ｍｉ｝を求めることができない。
【０１４１】
そこで、もう一つの平面｛Π（３）｝を考えることにする。この第３の平面｛Π（３）｝と、第１の平面｛Π（１）｝の組み合わせにより、別の変換行列｛Ａ´´｝を得ることができ、これから｛Ａ´｝のときと全く同じように、図１０（ｂ)に示すように照射位置ｂの存在する円｛Ω（２）｝を求めることができる。
【０１４２】
円｛Ω（１）｝、円｛Ω（２）｝のどちらも平面｛Π（１）｝を表す座標系、すなわち｛Ｕ｝座標系によって表現されているので、図１０（ｃ）に示すように、この２つの円が交わる点を直接計算すればこれが照射位置｛ｂ｝の存在点となる。
【０１４３】
ここにおいて、目的となる光軸方向｛ｍｉ｝を一意に算出することが可能になった。具体的には｛ｂ｝を数式（５）に代入し、順次未知パラメータを求めていった後にこれを数式（３）に代入すれば計算される。
【０１４４】
なお２つの円はそれぞれ平面｛Π（１）｝に直交する平面上にあり、その中心も平面｛Π（１）｝上にあるので、どちらの円も３次元空間内で平面｛Π（１）｝を中心とした面対称の位置にある。よって、その交点も平面｛Π（１）｝の表側と裏側の２つ存在するが、どちらを考えても平面｛Π（１）｝との相対的な関係は変わらない。
【０１４５】
以上の計算は、例えば図１４に示した自動計算装置（３００）によって実現される。自動計算装置は、データ転送を行うシステムバス（３３０）、あらかじめ決められたプログラムやその時の状況に応じて、各種演算処理を行うCPU等の中央演算装置（３４０）、プログラムや数値データを保存するメモリ等の主記憶装置（３５０）、同じくプログラムや数値データを保存するハードディスク等の外部記憶装置（３６０）から構成される。本装置には光線方向算出プログラム（３８５）が記録されており、このプログラムがいったん主記憶装置（３５０）に読み出された後に、光線投影位置計測手段から入力される各投影点の位置情報を用いて計算を行い、各光線方向を算出し、外部記憶装置（３６０）に計算結果（３９０）を記録する。
【０１４６】
以上述べてきた本発明の実施例における処理の手順を図１４に示す。任意の位置、方向に光線照射装置を動かす（１５１０）。次に平面スクリーンに光線を照射する（１５２０）。次に平面スクリーン上の投影点位置を計測する（１５３０）。これを任意の回数（ただし３回以上）繰り返す。次に光線方向を算出する。光線方向算出部（１５４０）では、複数回の投影からある組み合わせを選びそれから得られた投影点位置情報から各光線に関する拘束上件を求め（１５５０）、これを異なる組み合わせで複数回行い、その結果得られる複数の拘束条件を統合し、光線方向を求める（１５６０）。
【０１４７】
以上の手順をとることで、複数光線の照射方向が算出される。
【０１４８】
【発明の効果】
本発明によれば、１つのスクリーンを用いて測定が行われるため、高精度・大掛かりな専用装置を用意することなく、光線照射装置の光線方向を測定することが可能となる。
【図面の簡単な説明】
【図１】本発明の一実施例の形態を示す構成図
【図２】カメラの自己位置同定を用いた３次元位置姿勢センサ（ａ）と光線照射装置を用いた３次元位置姿勢センサ（ｂ）
【図３】光線照射装置と各光線の方向
【図４】２つの投影点の絶対位置測定による光線方向推定方向
【図５】光線方向測定方法（ａ）（ｂ）（ｃ）（ｄ）
【図６】２回の測定における光線照射装置の位置および姿勢を一致させた場合の各投影平面の関係
【図７】ある照射光線｛ｍｖ｝の２平面｛Π（１）｝、｛Π（２）｝上の無限遠点に向かったときの、無限遠方向を示すベクトル｛ｐｖ｝、｛ｐ´ｖ｝と｛ｍｖ｝の関係
【図８】ベクトル｛ｌ｝、点｛ｑｖ｝と｛ｂ｝の存在する平面｛Π（ｑ）｝の関係
【図９】平面｛Π（２）｝上の無限遠点方向に投影点を持つ２つの光線｛ｕｑｖ｝、｛ｕｖ｝のつくる角度｛θ｝と各投影点の関係
【図１０】（ａ）は｛ｂ｝の存在する円｛Ω（１）｝、（ｂ）は｛ｂ｝の存在する円｛Ω（２）｝、（ｃ）は３つの平面｛Π（１）｝、｛Π（２）｝、｛Π（３）｝を用いて｛ｂ｝を一意に決定していく過程。
【図１１】（ａ）カメラの自己位置推定方法と（ｂ）光線照射装置の位置・姿勢推定方法
【図１２】平面スクリーン２を４台のカメラで撮影するときの状況
【図１３】光線方向算出手段を実現する装置構成例
【図１４】実施例における処理手順
【符号の説明】
１．光線照射装置
２．平面スクリーン
３．光線投影位置計測手段
４．光線方向算出手段
５１．〜５４．光線投影位置計測撮影用カメラ
６．マーカ撮影カメラ
７．光線照射装置
９．９１.平行な平面スクリーン
１０．カメラ
１１．平面スクリーン
１２．光線照射装置
１３．平面スクリーン[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a light beam direction measuring device and a light beam direction measuring method for measuring the position and posture of a light beam irradiation device and for self-position identification used in various fields such as robot control, measurement, and live-action CG synthesis.
[0002]
[Prior art]
Camera self-position identification that estimates the position and orientation of a camera from a screen that captures a marker as a feature point is a basic technology used in various fields such as robot control, measurement, and live-action CG synthesis.
[0003]
One of the basic methods is called the perspective three-point problem.
It is known that when the internal parameters of the camera are known, a quaternary equation is derived from the three-dimensional positions of the three feature points and the shooting screen thereof, and the camera position and orientation can be calculated by solving them.
[0004]
The number of feature points is three, which is a necessary and sufficient number of observations for this estimation, but can simplify the calculation, uniquely determine the solution of the multi-order equation, obtain the camera internal parameters at the same time, or error For the purpose of reducing the number of feature points, a method of increasing the number of feature points beyond that to provide redundancy [see Non-Patent Document 1] has also been proposed.
[0005]
These principles are applicable not only to camera self-location identification but also to other applications.
[0006]
In the self-position identification of the camera, as shown in FIG. 11A, the three-dimensional position of the subject and each optical axis direction (straight direction connecting the viewpoint and the subject: {m0}, {m1}, { When m2}) is known, the position / posture of the camera 10 can be obtained.
[0007]
On the other hand, as shown in FIG. 3, consider a light beam irradiation apparatus 1 that irradiates a plurality of light beams in a direction fixed in advance to the irradiation apparatus.
[0008]
The light irradiation device 1 is moved to an arbitrary position and direction, and light rays (ray irradiation direction vectors: {m1}, {m2}, {m3}, {m4}) irradiated from here are projected to a certain place. If the three-dimensional position of the projection point is measured by any means, as shown in FIG. 11B, the information obtained and the geometric constraints thereof are exactly the same as those in FIG. The position / posture of the irradiation device 12 can be obtained by the same method as the self-position identification.
[0009]
As a specific example, an attempt has been made to apply the irradiation apparatus to a three-dimensional position and orientation sensor in VR (virtual reality) [see Non-Patent Document 2].
[0010]
FIG. 2B shows an application implementation example.
[0011]
The experience person who headed for the super-large screen can move the device (light irradiation device 7) freely toward the screen 2 to input a total of six parameters of position and orientation to the system.
[0012]
For example, by attaching the device 7 to the experience person's head and using non-visible light as a light beam, the experience person's viewpoint position and line-of-sight direction are automatically detected, thereby dynamically changing the displayed image. Application to VR applications such as.
[0013]
A method using this principle as a pointing device in a large display is disclosed in the following Patent Document 1. Now, in the above-described application, as shown in FIG. 3, the relative direction of each light beam irradiated from the irradiation apparatus 1 is known in advance. Therefore, the measurement method becomes a problem.
[0014]
The most easily understood method is the method shown in FIG.
[0015]
Two projection planes (9, 91) whose positional relationship is known, such as parallel planes, are set, and the positions of two projection points (for example, {p1}, {p′1}) projected onto the projection plane are measured. Then, the direction of the light beam can be obtained by connecting the two. However, this method requires a large-scale device with two projection planes (9, 9 ′) with high accuracy so that the two projection points can be read as values in the same coordinate system, and calibration of the device itself is also required. Required.
[0016]
[Non-Patent Document 1]
L. Quan and Z. Lan, “Linear N ≧ 4-point pose determination,” Proc. ICCV, pp. 778-782, 1998
[Non-Patent Document 2]
Naoki Suzuki, Hiroki Takahashi, Masayuki Nakajima, “Active 3D Position Sensor AMUSE for Large VR System”, Journal of VR Society of Japan, Vol. 4, No. 3, pp. 511-520, 1999
[Patent Document 1]
JP 2003-36142 A [Problems to be Solved by the Invention]
As a method of measuring the relative direction of each light beam irradiated from the light beam irradiation device 1, two projection surfaces such as parallel screen surfaces such as parallel screen surfaces are known, as in the light beam direction measurement device shown in FIG. A method is conceivable in which the position of two projection points projected on this is read as a value in the same coordinate system, and the direction is obtained from this value. However, this method requires a large-scale apparatus installed with high accuracy in order to perform the two projections under a strict relationship. Furthermore, advanced calibration of the device itself is also required.
[0017]
Therefore, in the present invention, a light direction measuring means that does not require such a large-scale device is realized.
[0018]
[Means for Solving the Problems]
In order to achieve the above object, a light beam irradiating means for simultaneously irradiating light beams in a plurality of directions, a flat screen for projecting each light beam irradiated by the light beam irradiating means, and projection of each light beam projected on the flat screen A light beam projection position measuring means for measuring the position and a light beam direction calculating means for calculating each light beam direction from the projection position information of each light beam measured by the light beam projection position measuring means are configured.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
The configuration of an embodiment of the present invention is shown in FIG.
[0020]
As can be seen from the figure, a flat screen 2 for projecting each light beam irradiated by the light beam irradiation means 1, a light beam projection position measuring means 3 for measuring the projection position of each light beam projected on the flat screen 2,
It comprises light ray direction calculation means 4 for calculating each light ray direction from the measured projection position information of each light ray.
[0021]
As shown in FIG. 3, the light irradiation device 1 irradiates light in a plurality of directions at the same time, the number of irradiated light is four or more, and the irradiation direction of each light is fixed (the relative direction of each light). Is unknown).
[0022]
Each light beam irradiated by the light beam irradiation means 1 is projected onto the flat screen 2.
[0023]
The light ray projection position measuring means 3 projects the projection position information (each projection point) of each light ray from the image information obtained by photographing the projection position of each light ray projected on the flat screen 2 with a plurality of cameras (51, 52, 53, 54). 3 dimensional coordinate values) are measured.
[0024]
As an embodiment of the light projection position measuring means 3, FIG. 12 shows a situation when the flat screen 2 shown in FIG. 1 is photographed by four cameras.
[0025]
In FIG. 12, a camera (51) to a camera (54) are cameras that capture an image of the flat screen 2, and these cameras are installed on a surface on which a screen display screen can be imaged. In the case of a transmissive screen, it may be a place where the back side of the screen can be imaged.
[0026]
Each camera images a part of the screen 2, but is arranged at a position where the entire image display area can be imaged when the imaging areas of the cameras are integrated. Of course, the areas captured by these cameras may overlap each other.
[0027]
At this time, calibration between the position of the screen 2 and each camera for measuring the projection point is performed in advance and the relationship is fixed, so that each projection point can be read as a three-dimensional coordinate value.
[0028]
As a result, the light projection position measuring means 3 determines the projection point ({p1}, {p2}, {p3}, {p4}) position of each light ray of the light irradiation device 1 projected onto the flat screen 2 as a three-dimensional position. Measured as data and output as coordinate information of projection points of each ray.
[0029]
The light ray direction calculating means 4 calculates each light ray direction from the projection point position information of each light ray measured by the light ray projection position measuring means 3.
[0030]
Next, a procedure for calculating each light beam direction of the light beam irradiation device 1 is shown in FIG.
[0031]
By using the application system described in the prior art, the projection points ({p1 (1)}, {p2 (1) of the light rays when the light rays are irradiated from an arbitrary location {b (1)} in FIG. )}, {P3 (1)}, {p4 (1)}).
[0032]
Of course, since the light beam directions of the irradiation device 1 are not known at this time, the position and orientation of the irradiation point {b (1)} cannot be obtained, and the light beam direction is estimated only from the information of the projection point. I can't do that either. However, if the irradiation apparatus 1 is moved to another arbitrary position and the same measurement is performed a plurality of times (FIG. 5 (b), FIG. 5 (c),..., FIG. 5 (d)), respectively. Projected points ({p1 (1)}, {p2 (1)},..., {P1 (2)},..., {P4 (2)}, {p1 (N)},. The direction of each light beam emitted from the irradiation device can be calculated from the positional information of {p4 (N)}) using geometric constraint conditions, and at the same time, the position and orientation of the irradiation device at the time of each irradiation Can be requested.
[0033]
Here, the position information {b} and {pi} of each irradiation point and projection point are both represented by a 3-D vector indicating a three-dimensional position expressed in the world coordinate system.
[0034]
A specific calculation method will be described by taking a combination of the first measurement (FIG. 5A) and the second measurement (FIG. 5B) as an example.
[0035]
As shown in FIGS. 5A and 5B, the position and orientation of the irradiation device 1 are different in the first measurement and the second measurement, but on the same projection plane (flat screen 2). Projecting against. As shown in FIG. 6, this relationship is equivalent to projecting on different projection planes {Π (1)}, {Π (2)} with the same position / posture of the irradiation apparatus 1. .
[0036]
Here, since the relative relationship between the position and orientation of the irradiation device at each measurement time in FIG. 5 is unknown, the relative relationship between the position of each plane and the normal direction in FIG. 6 is also unknown.
[0037]
Now consider the coordinate system. In FIG. 5, since all the same projection planes are used in each measurement, each projection point ({p1 (1)}, {p2 (1)},..., {P1 (2)},..., {P4 (2)}, {p1 (N)},... {P4 (N)}) are represented in the same coordinate system (world coordinate system). On the other hand, in FIG. 6, since the irradiation apparatus and the projection plane are moved and rotated during the second measurement, the coordinate origin and coordinate axes of the coordinate system representing the projection point coordinate value {p′i} are the same. Will be moved and rotated. That is, in FIG. 6, the coordinate values {b} and {pi} used in the first measurement are different from the coordinate values {b ′} and {p′i} used in the second measurement. Must be interpreted as represented in the system (however, the positional information {b} and {b ′} of the irradiation points {b (1)} and {b (2)} indicate the same point in the space) Note that) Therefore, the coordinate system used in the first measurement is the {U (1)} coordinate system, the coordinate used in the second measurement is the {U (2)} coordinate system, and hereinafter the {U (3)} coordinate. System, called {U (N)} coordinate system.
[0038]
The relationship between the projection points on the two projection planes in FIG.
In the first measurement, the position of the irradiation device is {b}, the projection point of the light beam {mi} onto the plane 1 (Π (1)) is {pi}, and each is represented in the {U (1)} coordinate system. A three-dimensional vector to be used. Similarly, in the second measurement, the position of the irradiation device is {b ′}, the projection point of the light beam {mi} onto the plane 2 ({Π (2)}) is {p′i}, and {U ( 2)} A three-dimensional vector expressed in a coordinate system. Each relationship is [0039]
[Expression 1]

[0040]
[Expression 2]

[0041]
It can be expressed as.
[0042]
From this formula (1) and formula (2):
[Equation 3]

[0044]
[Expression 4]

[0045]
Is obtained.
[0046]
From the above conditions, from the known parameters {pi}, {p′i} in the equations (3) and (4) (i = 1, 2,..., I), the unknown parameters {mi}, {α ′ i}, {R ′}, {b}, {b ′} are estimated. When {mi} is deleted from Equations (3) and (4),
[Equation 5]

[0048]
Is obtained.
[0049]
Here, according to the definition of the measurement method, each projection point {pi}, {p′i} is on the same plane. Therefore, Equation (5) is rewritten to take this condition into consideration.
[0050]
Now, the two planes {Π (1)} and {Π (2)} in FIG. 6 are respectively converted into normal vectors {h} and {h ′} and distances {d} and {d} d ′}. Three or more points on the plane ({p1 (1)}, {p2 (1)}, {p3 (1)}, {p4 (1)}), ({p1 (2)}, {p2 (2)}, {p3 (2)}, {p4 (2)}) are known in three-dimensional positions, so the parameters {h}, {d}, {h ′}, {d ′} of each plane are It can be calculated uniquely from there. Note that {h} and {d} are represented in the {U (1)} coordinate system, and {h ′} and {d ′} are represented in the {U (2)} coordinate system.
[0051]
The mathematical formula to be satisfied by an arbitrary point {pi} existing on the plane {Π (1)} and an arbitrary point {p′i} existing on the plane {Π (2)} is:
[Formula 6]

[0053]
[Expression 7]

[0054]
It is. In these expressions, (•, •) indicates the inner product of the vectors.
[0055]
Substituting Equations (6) and (7) into Equation (5)
[Equation 8]

[0057]
here,
[0058]
[Formula 81]

[0059]
Using the tensor product relationship expressed by the following equation (8):
[Formula 82]

[0061]
It is rewritten and from this [0062]
[Equation 9]

[0063]
The relationship between {pi} and {p′i}
[Expression 10]

[0065]
Can be expressed in the form
[0066]
Equation (10) is defined as the basic equation in this problem.
[0067]
The transformation matrix {A ′} in Expression (10) is obtained from the observation parameters. In equation (10), there is a degree of freedom of scalar multiplication on both sides.
[0068]
## EQU1 ##

[0069]
It can be said. Than this,
[0070]
## EQU10 ##

[0071]
Substituting into the formula (10) as
[0072]
[Expression 11]

[0073]
The following two formulas are obtained. Since the number of unknown parameters ({aij}) is eight, the matrix {A ′} is uniquely obtained from the equation (10) of four sets (i = 1, 2, 3, 4).
[0074]
When a certain optical axis i is projected on two different planes and observed as {p} and {p′i} in coordinate representations of individual coordinate systems, the relationship between {pi} and {p′i} Is represented by the basic equation of Equation (10). Also, the unknown parameter {A ′} used there is four sets (i = 1, 2, 3, 4) of observation points {p1}, {p2}, {p3}, {p4}, {p′1} , {P′2}, {p′3}, and {p′4}.
[0075]
In order to obtain {mi} (i = 1, 2,..., I) in the equations (3) and (4), the unknown parameters {b}, {b} in the equation (5) are decomposed by decomposing {A ′}. It is necessary to obtain b ′} and {R ′}.
[0076]
{A ′} is substantially expressed by eight parameters as described above, whereas {b}, {b ′}, and {R ′} have a total of nine parameters.
[0077]
As shown in FIG. 5, when the number of observations is increased, for example, three observations are performed, the combination of the first observation and the second observation, and the first observation and the third observation, Two transformation matrices, {A ′} and {A ″} are obtained. Therefore, the number of constraint parameters here increases to 8 × 2 = 16. On the other hand, unknown parameters are {b}, {b ′}, {R ′} added with {b}, {b ″}, {R ″}. , {B}, {b ′}, {R ′}, {b ″}, {R ″}. As a result, the number of unknown parameters becomes smaller than the number of constraints, so that each unknown parameter can be obtained.
[0078]
First, in the transformation matrix {A ′}, which is a combination of the first observation and the second observation, a condition regarding the value to be taken by {b} among the unknown parameters {b}, {b ′}, {R ′} (Conditional expression regarding {b}) is obtained.
[0079]
Consider a certain ray direction {mv}. This ray is projected to a certain infinity position on the plane {Π (1)}, and at the same time, projected to a certain infinity position on the plane {Π (2)}. Such a condition for the light beam {mv} is satisfied only when the light beam direction is equal to the intersecting direction {l} between the plane {Π (1)} and the plane {Π (2)} as shown in FIG.
[0080]
A direction {mv} that satisfies the above conditions is actually obtained.
[0081]
When the infinity point on the plane is expressed by a unit vector {c} indicating the direction, the actual infinity point is
[Formula 111]

[0083]
However, in the homography in the equation (10), there is a constant multiple degree of freedom, and conversion between two planes can be performed without any problem. Therefore, the scale constant {z} may be omitted when considering the projection point. it can.
[0084]
From such an idea, an infinite projection point on the plane {Π (1)} of the ray {mv} and an infinite projection point on the plane {Π (2)} are respectively represented by unit direction vectors {pv}, {pv}, { p′v}.
As shown in FIG. 7, {pv} and {p′v} represent the direction {mv}, that is, the direction of the intersection line {l} between the two planes. Note that {pv} and {p′v} are expressed in different coordinate systems ({U (1)} and {U (2)} coordinate systems), respectively.
[0085]
When {pv} and {p′v} satisfying such conditions are obtained, the normal of the plane {Π (2)} is {h ′}, so that the infinity direction {p ′ of the plane {Π (2)} v} is
[0086]
[Expression 12]

[0087]
Meet.
[0088]
Similarly, since the normal of the plane {Π (1)} is {h}, {pv} satisfies ({h}, {pv}) = 0.
From equation (10), the relationship between {pv} and {p′v} is
[Equation 121]

[0090]
So,
[0091]
[Formula 13]

[0092]
Meet.
[0093]
From equation (12) and equation (13), {p′v} is
[Expression 14]

[0095]
It is obtained by. Here, normalize [•] represents a normalization operation in which the norm of the vector is 1, and x represents an outer product of the vectors.
[0096]
From the above, the intersecting direction of the two planes was obtained as a vector {p′v} represented by the {U (2)} coordinate system.
[0097]
Next, as shown in FIG. 8, let {nv} be the direction of light rays projected onto an infinite point on the plane {（(2)} orthogonal to {p′v} (direction of intersection line {l}). This infinite projection point is represented as {q′v}.
[0098]
Since the normal of the plane {Π (2)} is {h '},
[0099]
[Expression 15]

[0100]
It is. Therefore, {q′v} is orthogonal to both {p′v} and {h ′}. Since {p′v} is obtained by Expression (14),
[0101]
[Expression 16]

[0102]
It becomes.
[0103]
Let {qv} be the point at which the ray {nv} projects onto the plane {Π (1)}. From Equation (10)
[Formula 161]

[0105]
It becomes. Here, since {qv} is not an infinite point, it is necessary to obtain it as an actual point on the plane {Π (1)}, not a unit vector. Therefore, it is necessary to specify the scale factor. Since ({h}, {qv}) = {d} from the plane equation,
[0106]
[Expression 17]

[0107]
It becomes.
[0108]
From the above, {qv} is specified. As shown in FIG. 8, the direction ({qv}-{b}) from the irradiation position {b} toward {qv} is orthogonal to {l} (= {pv}). Thus, it can be seen that {b} must lie on a plane {Π (q)} passing through the point {qv} and orthogonal to {l}.
[0109]
Next, as shown in FIG. 9, two light rays {uqv} and {uv} for projecting projection points to an infinite point on the plane {Π (2)} are considered, and the conditions are further narrowed down. . Note that {uqv} is a light beam that projects a projection point onto the aforementioned point {qv}.
[0110]
The angle {θ} between the projection points {qv}, {cv} and the irradiation point {b} on the plane {Π (1)} of the two rays is projected on the plane {Π (2)}. Note that the angle {θ ′} is formed by the points {q′v}, {c′v} and the irradiation point {b ′}.
[0111]
As can be seen from FIG. 9, the point {cv} that the ray {uv} projects onto the plane {Π (1)} passes through {qv} and is a straight line {L} parallel to the direction {l} (= {pv}). } Because it is above, this point is [0112]
[Expression 18]

[0113]
It can be expressed as. The infinite point {c′v} at which the same ray {uv} projects onto the plane {Π (2)} is expressed by the equation (10).
[Equation 19]

[0115]
It is.
[0116]
Here, since Expression (121) and | {pv} | = 1,
[Formula 191]

[0118]
From the formula (17),
192

[0120]
Therefore, if this is substituted into the equation (19), [0121]
[Expression 20]

[0122]
It becomes.
[0123]
Now, here, the ray {uqv} from the irradiation device toward the projection point {qv}, that is, the ray when {λ} = 0 in Equation (18), and other than {λ} = 0 in Equation (18) Attention is paid to the angle {θ} formed by the light beam {uv}.
[0124]
Since the distance from the irradiation position {b} to {qv} is | {b} − {qv} |
[Expression 21]

[0126]
It becomes.
[0127]
On the other hand, these two rays are projected to an infinite point on the plane {Π (2)}, and they are represented by {q′v}, {c′v} and a direction vector, and the angle { Finding θ ′},
[0128]
[Expression 22]

[0129]
It becomes. Since {θ} = {θ ′}, when formula (21) and formula (22) are combined into one and formula (20) is substituted,
[0130]
[Expression 23]

[0131]
Is obtained.
[0132]
In order for Equation (23) to be satisfied for any {λ},
[Expression 231]

[0134]
Must. This is because the distance {r} between the irradiation position {b} and the point {qv} is always
[0135]
[Expression 24]

[0136]
That is, {b} must exist on a sphere having a radius {r} centered at the point {qv}.
[0137]
Here, {p′v}, {q′v}, and {qv} are obtained from Equations (14), (16), and (17), respectively.
[0138]
In summary, as shown in FIG. 10A, the irradiation position b is on a circle {Ω (1)} having a radius {r} centered on a point {qv} on a plane {Π (q)}. I found that it must exist.
[0139]
As described above, since the conditions to be taken by the irradiation position {b} when the homography {A ′} is given have been considered, the above conclusion can be said to be a necessary condition regarding the position of {b}. Proof is omitted, but this is also a sufficient condition.
[0140]
As described above, when light is irradiated onto any two planes {Π (1)} and {Π (2)}, a mutual transformation matrix {A ′} is obtained from the respective projection positions. It has been found that a necessary and sufficient condition is that the position {b} exists in a certain circle {Ω (1)} in the space.
However, {b} cannot be uniquely specified by this alone, and the target optical axis direction {mi} cannot be obtained.
[0141]
Therefore, consider another plane {平面 (3)}. By combining this third plane {Π (3)} and the first plane {Π (1)}, another transformation matrix {A ″} can be obtained, and from now on when {A ′} In exactly the same manner, a circle {Ω (2)} where the irradiation position b exists can be obtained as shown in FIG.
[0142]
Since both the circle {Ω (1)} and the circle {Ω (2)} are expressed by the coordinate system representing the plane {Π (1)}, that is, the {U} coordinate system, FIG. 10 (c) shows. Thus, if the point where these two circles intersect is directly calculated, this becomes the existence point of the irradiation position {b}.
[0143]
Here, the target optical axis direction {mi} can be uniquely calculated. Specifically, {b} is substituted into Equation (5), and unknown parameters are sequentially obtained and then substituted into Equation (3) for calculation.
[0144]
Each of the two circles is on a plane orthogonal to the plane {Π (1)} and the center thereof is also on the plane {Π (1)}. Therefore, both circles have a plane {Π (1) in the three-dimensional space. )} At the center of plane symmetry. Therefore, there are two intersections, the front side and the back side of the plane {Π (1)}, but the relative relationship with the plane {Π (1)} does not change regardless of which one is considered.
[0145]
The above calculation is realized by, for example, the automatic calculation apparatus (300) shown in FIG. The automatic computing device stores a system bus (330) that performs data transfer, a central processing unit (340) such as a CPU that performs various arithmetic processing according to a predetermined program and the situation at that time, a program, and numerical data. A main storage device (350) such as a memory and an external storage device (360) such as a hard disk for storing programs and numerical data are also included. In this apparatus, a ray direction calculation program (385) is recorded. After this program is once read into the main memory (350), position information of each projection point input from the ray projection position measuring means is obtained. Calculation is performed to calculate each light beam direction, and the calculation result (390) is recorded in the external storage device (360).
[0146]
FIG. 14 shows a processing procedure in the embodiment of the present invention described above. The light irradiation device is moved to an arbitrary position and direction (1510). Next, light is irradiated onto the flat screen (1520). Next, the projection point position on the flat screen is measured (1530). This is repeated an arbitrary number of times (however, 3 times or more). Next, the light direction is calculated. In the ray direction calculation unit (1540), a certain combination is selected from a plurality of projections, a constraint on each ray is obtained from the projection point position information obtained from the combination (1550), and this is performed a plurality of times with different combinations. The obtained plurality of constraint conditions are integrated to determine the light beam direction (1560).
[0147]
The irradiation direction of a plurality of light beams is calculated by taking the above procedure.
[0148]
【The invention's effect】
According to the present invention, since the measurement is performed using one screen, it is possible to measure the light beam direction of the light beam irradiation device without preparing a dedicated device with high accuracy and large scale.
[Brief description of the drawings]
FIG. 1 is a configuration diagram showing an embodiment of the present invention. FIG. 2 is a three-dimensional position / orientation sensor (a) using self-position identification of a camera and a three-dimensional position / orientation sensor (b) using a light irradiation device. )
[Fig. 3] Ray irradiation device and direction of each ray [Fig. 4] Ray direction estimation direction by measuring absolute position of two projection points [Fig. 5] Ray direction measuring method (a) (b) (c) (d)
FIG. 6 shows the relationship between projection planes when the position and orientation of the light beam irradiation device are matched in two measurements. FIG. 7 shows two planes {Π (1)} and {Π ( 2)} Relationship between vectors {pv}, {p′v} and {mv} indicating the direction of infinity when heading to the infinity point in FIG. 8 FIG. 8: Vector {l}, points {qv} and {qv} b} exists in the plane {Π (q)}. FIG. 9 is an angle {{qv}, {uv} formed by two rays {uqv} and {uv} having projection points in the direction of the infinity point on the plane {Π (2)}. Relationship between θ} and each projection point [FIG. 10] (a) is a circle {Ω (1)} where {b} exists, and (b) is a circle {Ω (2)} where {b} exists, (c) ) Is a process of uniquely determining {b} using three planes {Π (1)}, {Π (2)}, and {Π (3)}.
11A is a self-position estimation method of a camera, and FIG. 12B is a position / posture estimation method of a light irradiation apparatus. FIG. 12 is a situation when a plane screen 2 is photographed by four cameras. Device configuration example for realizing the calculation means [FIG. 14] Processing procedure in the embodiment [Explanation of symbols]
1. 1. Light irradiation device Flat screen 3. 3. Ray projection position measuring means Ray direction calculation means 51. -54. 5. Light projection position measurement photographing camera 6. Marker photographing camera 9. Light irradiation device 9.91. Parallel flat screen Camera 11. Flat screen 12. Light irradiation device 13. Flat screen

Claims (3)

In the beam direction measuring device for measuring each beam direction of the beam irradiation means for irradiating four or more beams in a plurality of directions at the same time,
The light direction measuring device includes a flat screen for projecting each light beam irradiated by the light beam irradiation means,
Ray projection position measuring means for measuring the projection position of each ray projected on the flat screen;
Ray direction calculating means for calculating each ray direction from projection position information of each ray measured by the ray projection position measuring means ,
The light beam direction calculating means determines that the position and orientation of the light beam irradiating means with respect to the flat screen is unknown, and that each light beam direction with respect to the light beam irradiating means is also unknown, A plurality of constraint conditions related to the position of the light beam irradiating means are obtained from the projection position information of each light beam measured by the light beam projection position measuring means at the time of performing the projection more than once, and the light beam using the plurality of constraint conditions. A light direction measuring apparatus that calculates the position of each irradiation means and each light direction. Regarding a light beam direction measuring method for measuring each light beam direction of a light beam irradiation means for irradiating four or more light beams in a plurality of directions simultaneously,
The light beam direction measuring method projects each light beam irradiated by the light beam irradiation means onto a flat screen;
Measuring a projection position of each light ray projected on the flat screen;
Calculating each ray direction from the measured projection position information of each ray ,
The step of calculating the direction of each light beam includes determining the position and orientation of the light beam irradiating unit with respect to the flat screen and determining the direction of each light beam relative to the light beam irradiating unit to be different from each other. A plurality of constraint conditions relating to the position of the light beam irradiating means are obtained from the projection position information of each light beam measured by the light beam projection position measuring means when the projection is performed three times or more in the direction, and the plurality of constraint conditions are used. And calculating the position and direction of each light beam. In the light direction measuring method according to claim 2,
In the step of projecting each light beam onto the flat screen, the light beam irradiating means is moved in an arbitrary position and in an arbitrary direction to perform projection .

Images