JP3847686B2 - 3D shape measuring device - Google Patents

Description

ここで、Ｌ_A （ｘ），Ｌ_B （ｘ），Ｌ_C （ｘ）は、それぞれ区間Ａ，Ｂ，Ｃにおける変調区間幅、Ｋ_X は照射角度を変調区間幅へ変換するための係数、ｘ₀ は照射開始角度を表す。なお、説明を簡単化するために各区間の最大区間幅を１．０とするように正規化してある。以上の式を見れば分かるように、先に述べたように、３つの区間の合計光量は常に、Ｌ_A （ｘ）＋Ｌ_B （ｘ）＋Ｌ_C （ｘ）＝１．５となり、一定となっていることが解る。
【００１９】
こうして、図５に示すような輝度値Ｉと視差αの関係を撮像方向毎にあらかじめ求めて輝度−視差ＬＵＴ（輝度−視差変換手段）１６１を形成しておけば、これを用いて輝度値Ｉから視差αを検出することができる。さらに、校正データとして取得しておいた視差−距離変換用のＬＵＴを用いて変換すればリアルタイムに距離を取得することができるようになる。なお、この実施の形態１では、照射角度符合化変調信号１５１と視差抽出変調信号１５２のコードパターンを反転させたものとして説明したが、両者を同一パターンとしても構わない。その場合にも、図５のようなＩ−αの関係図が別途与えられ、それに従って視差抽出が可能となる。
【００２０】
以上のように、実施の形態１によれば、計測光の投光系と照射像の撮像系内において、同期クロック信号を共有し、異なる変調も可能な２つの変調手段を設けたことにより、従来は複数フレームに分けて照射していた変調パターンを、単一フレームで照射し、視差情報を単一フレームの受光量の変化として観測できる構成とした。したがって、従来必要であった視差情報を復元するための複雑な計算処理が不要となり、簡単なルックアップテーブルを用意するだけで、受光量（輝度）−視差情報−距離情報の変換が可能となり、その結果、距離分布のリアルタイム計測による移動物体の距離計測を可能とする効果が得られる。
【００２１】
実施の形態２．
実施の形態１では、測定対象物体は均一な完全拡散面であり、環境の背景光が存在しない場合の測定に適している。この実施の形態２では、測定対象物体の表面の反射率が不均一で、背景光も存在する場合についての測定に適している。
図６はこの発明の実施の形態２による３次元計測装置の構成を示す構成図である。図において、２０１〜２０３はＤＭＤ、２０５〜２０７はＣＣＤ撮像素子、２１１は照射光源、２１２，２１３は投光系レンズ、２１５，２１７〜２２０は撮像系レンズ、２１４，２１６はハーフミラー、２３１はクロックパルス源、２３２〜２３４はＤＭＤ駆動回路、２４１は計測対象物体である。２５１は照射角度符号化変調信号、２５２，２５３は視差抽出変調信号、２５４〜２５６は同期クロック信号、２５７〜２５９はＤＭＤ駆動信号である。なお、ＣＣＤ撮像素子２０６，２０７後の輝度−視差ＬＵＴについては省略して示す。
【００２２】
撮像系には、２つのハーフミラー（光学的分岐手段）２１４，２１６を備えることで計測対象物体２４１からの照射像を３つの方向に分岐させる構成となっている。このうち、１つの分岐方向に対しては、ＤＭＤの変調手段を介さずにＣＣＤ撮像素子（光電変換手段）２０７で分岐照射像を直接観測するようにしている。他方、２つの分岐方向に対しては、変調手段として２つのＤＭＤ（撮像系変調手段）２０２，２０３を介してＣＣＤ撮像素子（光電変換手段）２０５，２０６でそれぞれ観測する構成としている。ここで、この場合の変調手段としては必ずしもＤＭＤである必要はなく、マトリックス状に配置された複数の極小光学面を持つ変調手段であればよく、実施の形態１でも述べたような透過型アクティブマトリクス方式の液晶デバイスなどであってもよい。
【００２３】
図６において、ＤＭＤ２０１，２０２，２０３に対して、ＤＭＤ駆動回路２３２，２３３，２３４からＤＭＤ駆動信号２５７，２５８，２５９がそれぞれ与えられる。これらのＤＭＤ駆動信号は、クロックパルス源２３１からの同期クロック信号２５４，２５５，２５６によりタイミングが合わせられ、図７に示すように各ＤＭＤに対して別の変調パターンで与えられるようにしている。このことにより、ＣＣＤ撮像素子２０５，２０６，２０７はそれぞれ異なる情報を含む明度情報を得る。なお、ＣＣＤ撮像素子２０７については、式（１）の関係から一様照明光を照射した明度情報を取得する。
【００２４】
次に、図７で示した変調用の各ＤＭＤに与える変調パターンと受光後の信号処理について述べる。
まず、ＤＭＤ（投光系変調手段）２０１に与える変調信号は実施の形態１と同様であり、ＤＭＤ２０２についてはＤＭＤ２０１とちょうど位相が１８０度反転した変調パターン、ＤＭＤ２０３についてはＤＭＤ２０１と同位相とすると、これらのＤＭＤに対応した２つのＣＣＤ撮像素子２０５，２０６では、以下の光量の光が受光されることになる。
Ｉ_A ＝Ｋ_R ・ａ＋ｂ （５）
Ｉ_B ＝Ｋ_R ・（１−ａ）＋ｂ （６）
ここで、Ｉ_A はＣＣＤ撮像素子２０５の受光量、Ｉ_B はＣＣＤ撮像素子２０６の受光量、ａは視差に応じた変調区間幅、ｂは背景光、Ｋ_R は計測対象物体表面での反射率と変調区間幅から実際の計測光照射量への変換係数を乗じた係数である。なお、説明を簡単化するために視差に応じた変調区間幅ａについては、ＤＭＤ２０１で変調区間全域に渡って照射が行われたとした時の照射量を１として正規化している。また、ハーフミラー２１４，２１６による分岐後の光量は各分岐方向で同一とし、実際の光量からＣＣＤ撮像素子２０５，２０６上での光電変換係数については省略して定式化しているが、このことはここでの説明に何ら影響を及ぼさない。
【００２５】
一方、ＤＭＤを介さないＣＣＤ撮像素子２０７では、以下の光量の光が受光されることになる。
Ｉ_C ＝Ｋ_R ＋ｂ （７）
この時、以下のようにＫ_R ，ａ，ｂを求めることができる。
Ｋ_R ＝２・Ｉ_C −（Ｉ_A ＋Ｉ_B ） （８）
ａ＝（Ｉ_C −Ｉ_B ）／（２Ｉ_C −Ｉ_Ａ −Ｉ_B ） （９）
ｂ＝Ｉ_A ＋Ｉ_B −Ｉ_C （１０）
ここで、ａと視差αの関係をあらかじめ求めておくことで、視差αを求めることができる。さらに距離と視差αの関係も求めておけば、距離への変換が可能である。
【００２６】
以上のように、実施の形態２によれば、複数方向に分岐して変調可能な構成としたため、複数の変調効果を同時に得ることが可能となり、これらの複数の情報を用いて、環境の背景光や計測対象物体の表面反射率が未知であっても、その影響を除去した上で３次元情報を取得可能となる効果が得られる。また、一部は変調手段を介さずに直接観測可能な構成としたため、一様な照明を照射した計測対象物体像を３次元情報と同時に得ることができる効果がある。
【００２７】
実施の形態３．
図８はこの発明の実施の形態３による３次元計測装置の構成を示す構成図である。
図において、３０１，３０２はＤＭＤ、３０５〜３０７はＣＣＤ撮像素子、３１１は計測光源、３１２，３１３は投光系レンズ、３１４はハーフミラー、３１５〜３１８は撮像系レンズ、３３１はクロックパルス源、３３２，３３３はＤＭＤ駆動回路、３４１は計測対象物体である。３５１は照射角度符号化変調信号、３５２は視差抽出変調信号、３５４，３５５は同期クロック信号、３５６，３５７はＤＭＤ駆動信号である。なお、ＣＣＤ撮像素子３０５，３０６後の輝度−視差ＬＵＴについては省略して示す。
【００２８】
この実施の形態３では、ＤＭＤ（撮像系変調手段）３０２は、ＤＭＤ駆動回路３３３からのＤＭＤ駆動信号３５７により複数方向に反射方向を切り替える光学スイッチとしても動作する。したがって、ハーフミラー３１４を通過してＤＭＤ３０２に到達した計測対象物体３４１の照射像は、各方向に切り替えられ、それぞれの反射方向に配置されたＣＣＤ撮像素子（光電変換手段）３０５，３０６上にそれぞれ結像される。このことにより、図６の実施の形態２におけるように分岐後の系統毎に変調手段のＤＭＤ２０２，２０３をそれぞれ配置した場合に比べ、光学的分岐手段と変調手段を兼用した構成とすることができる。したがって、実施の形態２に比し、より簡素な構成とすることができる効果が得られる。
【００２９】
上記実施の形態２および３において、照射像の分岐後の１系統については変調手段を介さないでＣＣＤ撮像素子２０７または３０７に決像させる構成について示した。これに代わって、分岐後の全系統とも変調手段を介する構成とし、それぞれに独立な要素を含むように異なった変調をかけるようにしてもよい。そうすれば、上記の説明と同様に反射率Ｋ_Ｒ 、受光量ａ、背景光ｂを求めることができる。また、そのような構成にしておいて、うち１系統については、必要に応じて、変調をかけずに直接ＣＣＤ撮像素子に導くように切り替える構成しておけば、上記実施の形態２および３の両機能を備えることが可能となる。
また、その他の例として、背景光、反射率分布のいずれかが、あらかじめ分かっている場合や無視しうる場合には、照射像を分岐による２系統の構成とし、その２系統で変調を行ったり、あるいはそのうち１系統を変調し、残り１系統を非変調といった動作体系にしてもよく、使用環境に合わせてより簡素化することが可能となる。
【００３０】
【発明の効果】
以上のように、この発明によれば、投光系が、マトリックス状に配置した極小光学面を所定の変調パターンに従って切り替えることにより計測光源からの出射光を変調してパターン計測光を生成し、当該パターン計測光を計測対象物体上に照射する投光系変調手段を有し、撮像系が、パターン計測光により計測対象物体上に形成された照射像が結像され、投光系変調手段と同期して動作し、所定の変調パターンと連動した変調パターンに従ってマトリックス状に配置された極小光学面を切り替えることにより、結像された照射像を変調して仮想的に設置した計測基準点と計測対象物体までの距離に応じた視差情報を照射像の光量変化成分として抽出する撮像系変調手段と、この照射像の光量変化成分を受光して光電変換し、輝度情報を得る光電変換手段と、得られた輝度情報に基づいて視差情報を求める輝度−視差変換手段とを有するように構成したので、環境の背景光成分は存在せず、計測対象物体の表面反射率が均一であるとして、あらかじめ校正情報を取得しておけば、視差情報を単一フレームの受光量の変化として直接観測可能になり、従来必要であった視差情報復元のための複雑な計算処理が不要となり、簡単なＬＵＴを用意するだけで距離情報のリアルタイム計測を可能にする効果がある。
【００３１】
この発明によれば、投光系が、マトリックス状に配置した極小光学面を所定の変調パターンに従って切り替えることにより計測光源からの出射光を変調してパターン計測光を生成し、当該パターン計測光を計測対象物体上に照射する投光系変調手段を有し、撮像系が、パターン計測光により計測対象物体上に形成された照射像を複数方向に分岐する光学的分岐手段と、この光学的分岐手段による一つまたは複数の分岐方向に配置され、分岐された照射像がそれぞれ結像され、投光系変調手段と同期して動作し、所定の変調パターンと連動した変調パターンに従ってマトリックス状に配置された極小光学面を切り替えることにより、結像された照射像を変調して仮想的に設置した計測基準点と計測対象物体までの距離に応じた視差情報を照射像の光量変化成分として抽出する各撮像系変調手段と、これら各撮像系変調手段から得られた照射像の光量変化成分を受光して光電変換し、輝度情報をそれぞれ得る各光電変換手段と、これら各光電変換手段から得られた輝度情報に基づいて視差情報をそれぞれ求める輝度−視差変換手段とを有するように構成したので、複数の変調効果を同時に得ることが可能となり、これらの複数の情報を用いて、環境の背景光や計測対象物体の表面反射率が未知であっても、その影響を除去した上で３次元情報を取得可能とする効果がある。
【００３２】
この発明によれば、残りの分岐方向に配置され、分岐された照射像が直接結像され、この直接結像された照射像を光電変換する光電変換手段を有するように構成したので、一部は変調手段を介さずに直接観測するため、一様な照明を照射した計測対象物体像を３次元情報と同時に得ることができる効果がある。
【００３３】
この発明によれば、撮像系変調手段の少なくとも一つとして、結像された照射像を複数の反射方向に切り替える動作を兼ねる光学スイッチを用い、反射方向のそれぞれに光電変換手段を配置するように構成したので、光学的分岐手段と変調手段を兼用することにより、装置の構成を簡略化できる効果がある。
【図面の簡単な説明】
【図１】 この発明の実施の形態１による３次元計測装置の構成を示す構成図である。
【図２】 同実施の形態１に係るＤＭＤ駆動信号を示すタイムチャートである。
【図３】 同実施の形態１に係る他のＤＭＤ駆動信号を示すタイムチャートである。
【図４】 同実施の形態１に係る変調パターン光による動作を示す説明図である。
【図５】 同実施の形態１に係る輝度値Ｉと視差αの関係を示すルックアップテーブルの例を示す説明図である。
【図６】 同実施の形態２による３次元計測装置の構成を示す構成図である。
【図７】 同実施の形態２に係る変調パターン光による動作を示す説明図である。
【図８】 同実施の形態３による３次元計測装置の構成を示す構成図である。
【符号の説明】
１０１，１０２，２０１〜２０３，３０１，３０２ マイクロミラーデバイス（ＤＭＤ）、１０３，２０５〜２０７，３０５〜３０７ ＣＣＤ撮像素子、１１１，２１１，３１１ 計測光源、１１２，１１３，２１２，２１３，３１２，３１３ 投光系レンズ、１１４，１１５，２１５，２１７〜２２０，３１５〜３１８ 撮像系レンズ、１３１，２３１，３３１ クロックパルス源、１３２，１３３，２３２〜２３４，３３２，３３３ ＤＭＤ駆動回路、１４１，２４１，３４１ 計測対象物体、１５１，２５１，３５１ 照射角度符号化変調信号、１５２，２５２，２５３，３５２ 視差抽出変調信号、１５３，１５４，２５４〜２５６，３５４，３５５ 同期クロック信号、１５５，１５６，２５７〜２５９，３５６，３５７ ＤＭＤ駆動信号、１６１ 輝度−視差ＬＵＴ、１７１ 視差画像、２１４，２１６，３１４ ハーフミラー。[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a three-dimensional measurement apparatus that performs three-dimensional shape measurement in real time using a pattern light projection method.
[0002]
[Prior art]
For three-dimensional measurement, there is widely known a light cutting method in which triangulation is applied, slit-like light called a light cutting line is projected onto a measurement object, and the state in which this changes according to the object shape is observed. However, in this method, three-dimensional distance information other than the portion where the slit light hits cannot be obtained, and in order to obtain a three-dimensional distance distribution, multiple observations are performed while gradually changing the angle at which the light section line is projected. Need to do. Therefore, a very long measurement time is required.
On the other hand, there is a three-dimensional measurement method using a spatial coding method (see, for example, Patent Document 1). This method encodes the projection angle information of the light section line into a plurality of projection patterns, irradiates the measurement target object with each light pattern, images the measurement target object at different positions, and decodes the observation results. The position of the measurement target object is obtained. According to this method, it is possible to perform measurement in a shorter time compared to the light cutting method in which the projection angle is changed. For example, when observation is performed for 256 projection angles, the normal light-cutting method requires 256 observations, but the spatial encoding method requires only 8 pattern projections and observations, and 256/8. = 32 times faster.
[0003]
[Patent Document 1]
JP 60-152903 A (page 1-2, FIG. 2)
[0004]
[Problems to be solved by the invention]
Since the conventional three-dimensional measurement method based on the spatial coding method is configured as described above, the measurement time is shorter than that of the light cutting method, but it still requires multiple frames of measurement time. There is a problem that measurement is difficult when is a moving object.
[0005]
The present invention has been made to solve the above-described problems, and an object of the present invention is to obtain a three-dimensional measurement apparatus that enables real-time measurement of distance distribution and is suitable for distance measurement of a moving object.
[0006]
[Means for Solving the Problems]
A three-dimensional measuring apparatus according to the present invention is a three-dimensional shape measuring apparatus provided with a light projecting system that irradiates a measurement target object with measurement light and an imaging system that acquires three-dimensional information from an irradiation image of the measurement target object. The light projecting system generates pattern measurement light by modulating the light emitted from the measurement light source by switching the minimum optical surfaces arranged in a matrix according to a predetermined modulation pattern, and irradiates the measurement target object with the pattern measurement light The projection system has a light projection system modulation means, and the imaging system forms an irradiation image formed on the measurement target object by the pattern measurement light, operates in synchronization with the light projection system modulation means, and has a predetermined modulation pattern. By switching the minimal optical surfaces arranged in a matrix according to the interlocking modulation pattern, the imaged irradiation image is modulated and the distance between the virtual reference reference point and the object to be measured is set. Imaging system modulation means for extracting the same parallax information as a light quantity change component of the irradiation image, photoelectric conversion means for receiving the light quantity change component of the irradiation image and photoelectrically converting it to obtain luminance information, and the obtained luminance information A luminance-disparity conversion unit for obtaining parallax information based on the information;
[0007]
A three-dimensional measuring apparatus according to the present invention is a three-dimensional shape measuring apparatus provided with a light projecting system that irradiates a measurement target object with measurement light and an imaging system that acquires three-dimensional information from an irradiation image of the measurement target object. The light projecting system generates pattern measurement light by modulating the light emitted from the measurement light source by switching the minimum optical surfaces arranged in a matrix according to a predetermined modulation pattern, and irradiates the measurement target object with the pattern measurement light An optical branching means for branching an irradiation image formed on the measurement target object by the pattern measurement light in a plurality of directions, and one or a plurality of the optical branching means by the optical branching means. Are arranged in the direction of branching, and the branched irradiation images are respectively formed, operate in synchronization with the light projection system modulation means, and form a matrix according to a modulation pattern interlocked with a predetermined modulation pattern By switching the placed minimal optical surface, the imaged irradiation image is modulated, and the parallax information according to the distance from the measurement reference point virtually installed and the object to be measured is extracted as the light quantity change component of the irradiation image Each imaging system modulation means, each photoelectric conversion means for receiving and photoelectrically converting the light quantity change component of the irradiation image obtained from each imaging system modulation means, and obtaining luminance information, respectively, obtained from each photoelectric conversion means Luminance-disparity conversion means for obtaining parallax information based on the obtained luminance information.
[0008]
The three-dimensional measuring apparatus according to the present invention is arranged in the remaining branch direction, and has a photoelectric conversion means for directly forming a branched irradiation image and photoelectrically converting the directly imaged irradiation image. To do.
[0009]
The three-dimensional measuring apparatus according to the present invention uses, as at least one of the imaging system modulation means, an optical switch that also serves to switch the formed irradiation image to a plurality of reflection directions, and a photoelectric conversion means for each of the reflection directions. It is arranged.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
An embodiment of the present invention will be described below.
Embodiment 1 FIG.
1 is a block diagram showing a configuration of a three-dimensional measuring apparatus according to Embodiment 1 of the present invention. In the figure, reference numerals 101 and 102 denote micro mirror devices (Digital Micro Mirror Device: hereinafter referred to as DMD). This DMD is an element developed for video projectors by Texas Instruments Inc. in the United States. It incorporates hundreds of thousands of tiny mirrors (minimum optical surfaces) on a semiconductor substrate in a matrix, and each mirror is binary. It is a digitally controlled high-speed reflective optical switch that moves by using the pulse width modulation technology of the method and creates an image by controlling on / off of incident light. Normally, RGB modulation, which is the three primary colors of light, is performed by shifting the color filter sequentially, but in this Embodiment 1, the color filter is removed and used as a monochromatic modulation element. However, it is of course possible to employ a modulation configuration using a color filter. Reference numeral 103 denotes a CCD image pickup device, 111 denotes a measurement light source, 112 and 113 denote projection system lenses, and 114 and 115 denote image pickup system lenses. Reference numeral 131 denotes a clock pulse source, 132 and 133 denote DMD drive circuits, and 141 denotes a measurement target object. Reference numeral 151 denotes an illumination angle coded modulation signal, 152 denotes a parallax extraction modulation signal, 153 and 154 denote synchronization clock signals, 155 and 156 denote DMD drive signals, 161 denotes a luminance-parallax LUT (Look Up Table), and 171 denotes It is a parallax image.
[0011]
Next, the operation will be described.
The emitted light from the measurement light source 111 is condensed on the light receiving surface of the DMD (light projection system modulation means) 101 via the light projection system lens 112. The DMD 101 is supplied with a DMD drive signal 155 from the DMD drive circuit 132, and a plurality of mirrors forming the light receiving surface are driven by this signal so as to switch the reflection direction with respect to the incident light. That is, the incident light becomes pattern measurement light that is modulated according to the modulation pattern formed by the DMD drive signal 155. The pattern measurement light modulated by the DMD 101 is irradiated onto the measurement target object 141 via the light projection system lens 113.
[0012]
In the imaging system, the following imaging process is performed on the irradiation image formed on the measurement target object 141 by the irradiated pattern measurement light. The irradiation image is formed on the DMD (imaging system modulation means) 102 by the imaging system lens 114. A DMD drive signal 156 is given to the DMD 102 from the DMD drive circuit 133, and a plurality of mirrors forming the light receiving surface are driven by this signal so as to switch the reflection direction with respect to the incident light. Therefore, the formed irradiation image is modulated according to the modulation pattern formed by the DMD drive signal 156. The modulated irradiation image is formed on the CCD image sensor (photoelectric conversion means) 103 through the imaging system lens 115.
Here, the modulation means is not limited to DMD, but may be any element capable of high-speed modulation in a matrix. For example, a transmissive active matrix type liquid crystal device having a plurality of minimum shutters (minimum optical surfaces) ( LCD) may be used.
[0013]
DMD 101 and DMD 102 are provided with DMD drive signals 155 and 156 for generating respective modulation patterns from DMD drive circuits 132 and 133. These signals are generated as follows. The DMD driving circuit 132 is supplied with the irradiation angle coded modulation signal 151, while the DMD driving circuit 133 is supplied with the parallax extraction modulation signal 152. Further, synchronous clock signals 153 and 154 are respectively provided from the clock pulse source 131, and the two DMD drive signals 155 and 156 are synchronized for matching timing.
[0014]
2 and 3 are time charts showing DMD drive signals 155 and 156 for controlling the switch operation of the DMDs 101 and 102. FIG. (A) shows a DMD drive signal 155 that gives a modulation pattern to the DMD 101, and (b) shows a DMD drive signal 156 that gives a modulation pattern to the DMD 102. In this way, by providing a modulation pattern in which both of the DMDs 101 and 102 are interlocked, the parallax information is photoelectrically converted by the CCD image sensor 103 and directly obtained as luminance information, and three-dimensional measurement is possible in real time. .
[0015]
Here, a method of giving a modulation pattern will be described with reference to FIGS.
First, in FIG. 2, it is assumed that the horizontal axis represents a time axis and is divided into three sections A, B, and C in one frame time. Note that, here, since DMD is usually used for color modulation of the three primary colors RGB of light, for convenience, the configuration is divided into three sections of A, B, and C as described above, but the number of sections in one frame time is as follows. Arbitrary and intensity modulation in a single section may be used.
[0016]
Next, a case will be described in which the modulation patterns (a) and (b) in the DMD 101 and the DMD 102 are reversed with respect to each other in the regions divided into the three sections as shown in FIG. When a reference plane is assumed at a certain distance, the timing of light projection and light reception is just reversed with respect to the irradiation image on the measurement target object 141 existing on the reference plane, and the DMD 102 is irradiated by the DMD 101. The image cannot be received at all. Let this pair of modulation patterns be a code pair of irradiation angle coding / parallax extraction with parallax 0.
By the way, the modulated measurement light on the projection side has a pattern that changes depending on the irradiation angle (position). Therefore, when the measurement target object 141 exists at a position deviated from the above-described reference plane, and the pattern measurement light irradiated with the parallax α that is the difference in irradiation angle is irradiated onto the measurement target object 141, the pattern measurement is performed. If the luminance difference of light can be detected, the parallax α corresponding to the distance between the virtually set measurement reference point and the measurement target object can be detected. For example, if a pair of modulation patterns on the light projecting side and the light receiving side is formed as shown in FIG. 2, a light quantity proportional to the parallax α is obtained as an inner product value in the DMD 102 as shown in FIG. Here, the measurement reference point may be assumed anywhere, but usually it is often set between the light projecting side focus and the light receiving side focus.
[0017]
Here, the modulation pattern corresponding to the parallax α is given by the following equation, for example.
(L _A + K _T ・ Α, L _B -K _T ・ Β, L _C -K _T ・ Γ) (1)
However, α = β + γ, α, β, γ ≧ 0.
Where L _A , L _B , L _C Is a section width (modulation time) that serves as a reference when pattern measurement light is applied to sections A, B, and C. K _T Is a coefficient for converting the parallax α, β, γ into a section width.
If the modulation pattern as described above is given, the total light amount in all the sections A, B, and C is always constant regardless of the amount of α, and the observer can only see that illumination with a constant light amount is performed. There is an advantage that the measurement can be performed without being conscious of the measurement operation and the density image of the target can be acquired.
[0018]
In FIG. 4, as the modulation pattern according to the equation (1), the modulation pattern (corresponding to the irradiation angle encoding modulation signal 151 and the parallax extraction modulation signal 152 when the timing is slightly shifted with respect to each irradiation angle is shown. Pattern) pairs, and a luminance value proportional to the parallax in each imaging direction is directly obtained. Specifically, the code pattern of the position (x, y) where the vertical position is y in the direction corresponding to the irradiation angle x is given as the code pattern of the irradiation angle encoding modulation signal 151 as follows. In the first embodiment, the code pattern of the parallax extraction modulation signal 152 is given by inverting it. However, this is an example, and other code pattern sequences may be used as long as the code pattern is uniquely determined with respect to the irradiation angle.

Where L _A (X), L _B (X), L _C (X) is the modulation section width in sections A, B, and C, and K _X Is a coefficient for converting the irradiation angle into the modulation interval width, x ₀ Represents the irradiation start angle. In order to simplify the description, the maximum section width of each section is normalized to 1.0. As can be seen from the above equation, as described above, the total light amount in the three sections is always L _A (X) + L _B (X) + L _C (X) = 1.5, which is constant.
[0019]
Thus, if the relationship between the luminance value I and the parallax α as shown in FIG. 5 is obtained in advance for each imaging direction and the luminance-parallax LUT (luminance-parallax converting means) 161 is formed, the luminance value I is used using this. Can detect parallax α. Furthermore, if conversion is performed using a parallax-distance conversion LUT acquired as calibration data, the distance can be acquired in real time. Although the first embodiment has been described assuming that the code patterns of the irradiation angle encoding modulation signal 151 and the parallax extraction modulation signal 152 are inverted, they may be the same pattern. Also in this case, a relation diagram of I-α as shown in FIG. 5 is given separately, and the parallax can be extracted accordingly.
[0020]
As described above, according to the first embodiment, in the measurement light projecting system and the irradiated image capturing system, the synchronization clock signal is shared, and two modulation means capable of different modulations are provided. Conventionally, a modulation pattern that has been irradiated in a plurality of frames is irradiated in a single frame, and parallax information can be observed as a change in the amount of received light in a single frame. Therefore, complicated calculation processing for restoring the disparity information that has been necessary in the past is unnecessary, and it is possible to convert the received light amount (luminance) -disparity information-distance information only by preparing a simple lookup table. As a result, the effect of enabling distance measurement of a moving object by real-time measurement of distance distribution is obtained.
[0021]
Embodiment 2. FIG.
In the first embodiment, the object to be measured is a uniform complete diffusion surface, which is suitable for measurement when there is no background light of the environment. The second embodiment is suitable for measurement in the case where the reflectance of the surface of the measurement target object is non-uniform and background light is also present.
FIG. 6 is a block diagram showing a configuration of a three-dimensional measuring apparatus according to Embodiment 2 of the present invention. In the figure, 201 to 203 are DMDs, 205 to 207 are CCD image sensors, 211 is an irradiation light source, 212 and 213 are projection system lenses, 215 and 217 to 220 are imaging system lenses, 214 and 216 are half mirrors, and 231 is A clock pulse source, 232 to 234 are DMD driving circuits, and 241 are objects to be measured. Reference numeral 251 denotes an irradiation angle encoding modulation signal, 252 and 253 are parallax extraction modulation signals, 254 to 256 are synchronous clock signals, and 257 to 259 are DMD drive signals. Note that the luminance-parallax LUT after the CCD image sensors 206 and 207 is omitted.
[0022]
The imaging system includes two half mirrors (optical branching means) 214 and 216 so that an irradiation image from the measurement target object 241 is branched in three directions. Among these, for one branching direction, the branch irradiation image is directly observed by the CCD image pickup device (photoelectric conversion unit) 207 without using the DMD modulation unit. On the other hand, the two branch directions are observed by the CCD image pickup devices (photoelectric conversion means) 205 and 206 via two DMDs (image pickup system modulation means) 202 and 203 as modulation means, respectively. Here, the modulation means in this case is not necessarily DMD, and any modulation means having a plurality of minimum optical surfaces arranged in a matrix may be used, and the transmission type active as described in the first embodiment. A matrix type liquid crystal device or the like may also be used.
[0023]
In FIG. 6, DMD drive signals 257, 258, and 259 are provided to DMDs 201, 202, and 203 from DMD drive circuits 232, 233, and 234, respectively. These DMD drive signals are timed by synchronous clock signals 254, 255, and 256 from a clock pulse source 231, and are given to each DMD in a different modulation pattern as shown in FIG. As a result, the CCD imaging devices 205, 206, and 207 obtain brightness information including different information. For the CCD image sensor 207, brightness information obtained by irradiating uniform illumination light is obtained from the relationship of the expression (1).
[0024]
Next, the modulation pattern given to each DMD for modulation shown in FIG. 7 and signal processing after light reception will be described.
First, the modulation signal given to the DMD (projection system modulation means) 201 is the same as that in the first embodiment. The DMD 202 has a modulation pattern whose phase is exactly 180 degrees reversed from that of the DMD 201, and the DMD 203 has the same phase as the DMD 201. The two CCD image sensors 205 and 206 corresponding to these DMDs receive the following light amounts.
I _A = K _R ・ A + b (5)
I _B = K _R (1-a) + b (6)
Where I _A Is the amount of light received by the CCD image sensor 205, I _B Is the amount of light received by the CCD image sensor 206, a is the modulation interval width corresponding to the parallax, b is the background light, K _R Is a coefficient obtained by multiplying the reflectance on the surface of the measurement target object and the conversion interval width to the actual measurement light irradiation amount. In order to simplify the explanation, the modulation interval width a corresponding to the parallax is normalized by setting the irradiation amount when the irradiation is performed over the entire modulation interval by the DMD 201 to 1. In addition, the amount of light after branching by the half mirrors 214 and 216 is the same in each branch direction, and the photoelectric conversion coefficients on the CCD image pickup devices 205 and 206 are omitted from the actual light amount, but this is formulated. It does not affect the explanation here.
[0025]
On the other hand, the CCD image pickup device 207 that does not pass through the DMD receives the following amount of light.
I _C = K _R + B (7)
At this time, K _R , A, b can be obtained.
K _R = 2 ・ I _C -(I _A + I _B (8)
a = (I _C -I _B ) / (2I _C -I _A -I _B (9)
b = I _A + I _B -I _C (10)
Here, the parallax α can be obtained by obtaining the relationship between a and the parallax α in advance. Furthermore, if the relationship between the distance and the parallax α is also obtained, conversion to the distance is possible.
[0026]
As described above, according to the second embodiment, since the modulation can be performed by branching in a plurality of directions, a plurality of modulation effects can be obtained at the same time, and the background of the environment can be obtained using the plurality of pieces of information. Even if the surface reflectance of the light or the measurement target object is unknown, an effect is obtained in which the three-dimensional information can be acquired after removing the influence. In addition, since a part of the configuration can be directly observed without using the modulation means, there is an effect that a measurement target object image irradiated with uniform illumination can be obtained simultaneously with the three-dimensional information.
[0027]
Embodiment 3 FIG.
FIG. 8 is a block diagram showing the configuration of the three-dimensional measuring apparatus according to Embodiment 3 of the present invention.
In the figure, 301 and 302 are DMDs, 305 to 307 are CCD imaging devices, 311 is a measurement light source, 312 and 313 are projection system lenses, 314 is a half mirror, 315 to 318 are imaging system lenses, 331 is a clock pulse source, Reference numerals 332 and 333 denote DMD driving circuits, and reference numeral 341 denotes an object to be measured. Reference numeral 351 denotes an irradiation angle encoding modulation signal, 352 is a parallax extraction modulation signal, 354 and 355 are synchronization clock signals, and 356 and 357 are DMD drive signals. Note that the luminance-parallax LUT after the CCD imaging devices 305 and 306 is omitted.
[0028]
In the third embodiment, the DMD (imaging system modulation means) 302 also operates as an optical switch that switches the reflection direction in a plurality of directions by the DMD drive signal 357 from the DMD drive circuit 333. Therefore, the irradiation image of the measurement target object 341 that passes through the half mirror 314 and reaches the DMD 302 is switched in each direction, and is respectively placed on the CCD image pickup devices (photoelectric conversion means) 305 and 306 arranged in the respective reflection directions. Imaged. As a result, compared to the case where the DMDs 202 and 203 of the modulation means are arranged for each of the systems after branching as in the second embodiment in FIG. 6, a configuration in which the optical branching means and the modulation means are combined can be obtained. . Therefore, an effect that a simpler structure can be obtained as compared with the second embodiment is obtained.
[0029]
In the second and third embodiments, the configuration in which the CCD image pickup device 207 or 307 determines the image of one line after the irradiation image branching without using the modulation means has been described. Instead of this, all the systems after the branching may be configured to have modulation means, and different modulations may be applied so as to include independent elements. Then, the reflectance K as in the above description. _R The received light amount a and the background light b can be obtained. Further, in such a configuration, if one of the systems is switched so as to be guided directly to the CCD image pickup device without being modulated as necessary, the above-described second and third embodiments can be used. Both functions can be provided.
As another example, when either the background light or the reflectance distribution is known in advance or can be ignored, the irradiation image is divided into two systems and the two systems are used for modulation. Alternatively, an operation system in which one of them is modulated and the remaining one is not modulated can be simplified according to the use environment.
[0030]
【The invention's effect】
As described above, according to the present invention, the light projecting system generates the pattern measurement light by modulating the light emitted from the measurement light source by switching the minimum optical surfaces arranged in a matrix according to the predetermined modulation pattern, A projection system modulation unit that irradiates the measurement target object with the pattern measurement light, and the imaging system forms an irradiation image formed on the measurement target object by the pattern measurement light; Operates in synchronization, switches the minimal optical surfaces arranged in a matrix according to the modulation pattern linked with the predetermined modulation pattern, and modulates the imaged irradiation image and measures the measurement reference point virtually installed Imaging system modulation means for extracting parallax information according to the distance to the target object as a light quantity change component of the irradiated image, and light that receives the light quantity change component of the irradiated image and photoelectrically converts it to obtain luminance information Since it is configured to have a conversion means and a luminance-disparity conversion means for obtaining parallax information based on the obtained luminance information, there is no background light component of the environment, and the surface reflectance of the measurement target object is uniform. As long as calibration information is acquired in advance, it becomes possible to directly observe the parallax information as a change in the amount of received light in a single frame, eliminating the need for complicated calculation processing for restoring parallax information, which was necessary in the past, There is an effect that enables real-time measurement of distance information only by preparing a simple LUT.
[0031]
According to the present invention, the light projecting system generates pattern measurement light by modulating the light emitted from the measurement light source by switching the minimal optical surfaces arranged in a matrix according to the predetermined modulation pattern, and the pattern measurement light is generated. An optical branching unit having a light projection system modulation unit for irradiating the measurement target object, and an imaging system for branching an irradiation image formed on the measurement target object by pattern measurement light in a plurality of directions, and this optical branching Arranged in one or a plurality of branch directions by the means, the branched irradiation images are respectively formed, operate in synchronization with the light projection system modulation means, and are arranged in a matrix according to the modulation pattern interlocked with the predetermined modulation pattern The parallax information corresponding to the distance from the measurement reference point virtually installed by modulating the formed irradiation image and the object to be measured is switched by switching the minimal optical surface. Each imaging system modulation means that extracts as a light quantity change component, each photoelectric conversion means that receives and photoelectrically converts the light quantity change component of the irradiation image obtained from each imaging system modulation means, and obtains luminance information, respectively, Since it is configured to have a luminance-disparity conversion unit that obtains parallax information based on luminance information obtained from the photoelectric conversion unit, a plurality of modulation effects can be obtained at the same time. Thus, even if the background light of the environment or the surface reflectance of the measurement target object is unknown, there is an effect that three-dimensional information can be acquired after removing the influence.
[0032]
According to the present invention, it is arranged in the remaining branching direction, and the branched irradiation image is directly formed, and the photoelectric conversion means for photoelectrically converting the directly imaged irradiation image is provided. Since the direct observation is performed without using the modulation means, the measurement target object image irradiated with uniform illumination can be obtained simultaneously with the three-dimensional information.
[0033]
According to this invention, as at least one of the imaging system modulation means, an optical switch that also serves to switch the formed irradiation image to a plurality of reflection directions is used, and the photoelectric conversion means is arranged in each of the reflection directions. Since it is configured, there is an effect that the configuration of the apparatus can be simplified by combining the optical branching unit and the modulating unit.
[Brief description of the drawings]
FIG. 1 is a configuration diagram showing a configuration of a three-dimensional measurement apparatus according to Embodiment 1 of the present invention.
FIG. 2 is a time chart showing DMD drive signals according to the first embodiment.
FIG. 3 is a time chart showing another DMD drive signal according to the first embodiment.
FIG. 4 is an explanatory diagram showing an operation by modulation pattern light according to the first embodiment.
FIG. 5 is an explanatory diagram showing an example of a look-up table showing the relationship between the luminance value I and the parallax α according to the first embodiment.
FIG. 6 is a configuration diagram showing a configuration of a three-dimensional measurement apparatus according to the second embodiment.
FIG. 7 is an explanatory diagram showing an operation by modulation pattern light according to the second embodiment.
FIG. 8 is a configuration diagram showing a configuration of a three-dimensional measurement apparatus according to the third embodiment.
[Explanation of symbols]
101, 102, 201-203, 301, 302 Micromirror device (DMD), 103, 205-207, 305-307 CCD imaging device, 111, 211, 311 Measuring light source, 112, 113, 212, 213, 312, 312 Projection lens, 114, 115, 215, 217 to 220, 315 to 318 Imaging lens, 131, 231 and 331 Clock pulse source, 132, 133, 232 to 234, 332 and 333 DMD drive circuit, 141, 241, 341 Measurement object, 151, 251 and 351 Irradiation angle coded modulation signal, 152, 252, 253 and 352 Parallax extraction modulation signal, 153, 154, 254 to 256, 354, 355 Synchronous clock signal, 155, 156 and 257 to 259, 356, 357 DMD drive signal, 161 Luminance-parallax LUT, 171 parallax image, 214, 216, 314 half mirror.

Claims (4)

In a three-dimensional shape measuring apparatus including a light projecting system that irradiates a measurement target object with measurement light and an imaging system that acquires three-dimensional information from an irradiation image of the measurement target object.
The light projecting system is
A projection system for generating pattern measurement light by modulating light emitted from a measurement light source by switching the minimum optical surfaces arranged in a matrix according to a predetermined modulation pattern, and irradiating the measurement target object on the measurement target object Having modulation means,
The imaging system is
An irradiation image formed on the object to be measured is formed by the pattern measurement light, operates in synchronization with the light projection system modulation means, and is arranged in a matrix according to a modulation pattern interlocked with the predetermined modulation pattern. By switching the minimal optical surface, the parallax information corresponding to the distance from the measurement reference point virtually installed by modulating the imaged irradiation image and the object to be measured is extracted as the light quantity change component of the irradiation image Imaging system modulation means for
Photoelectric conversion means for receiving the light quantity change component of the irradiated image and photoelectrically converting it to obtain luminance information;
A three-dimensional shape measuring apparatus comprising: luminance-disparity conversion means for obtaining the parallax information based on the obtained luminance information. In a three-dimensional shape measuring apparatus including a light projecting system that irradiates a measurement target object with measurement light and an imaging system that acquires three-dimensional information from an irradiation image of the measurement target object.
The light projecting system is
A projection system for generating pattern measurement light by modulating light emitted from a measurement light source by switching the minimum optical surfaces arranged in a matrix according to a predetermined modulation pattern, and irradiating the measurement target object on the measurement target object Having modulation means,
The imaging system is
Optical branching means for branching an irradiation image formed on the measurement target object in a plurality of directions by the pattern measurement light;
Arranged in one or a plurality of branching directions by the optical branching means, the branched irradiation images are respectively formed, operate in synchronism with the light projecting modulation means, and modulate in conjunction with the predetermined modulation pattern By switching the minimal optical surfaces arranged in a matrix according to the pattern, the imaged irradiation image is modulated to irradiate parallax information according to the distance to the measurement target point virtually installed and the object to be measured Each imaging system modulation means for extracting as a light quantity change component of the image,
Each photoelectric conversion means for receiving and photoelectrically converting the light quantity change component of the irradiation image obtained from each of these imaging system modulation means to obtain luminance information, and
A three-dimensional shape measuring apparatus comprising: a luminance-disparity conversion unit that obtains the parallax information based on luminance information obtained from each of the photoelectric conversion units. 3. The three-dimensional shape measurement according to claim 2, further comprising photoelectric conversion means arranged in the remaining branching direction and directly forming the branched irradiation image and photoelectrically converting the directly imaged irradiation image. apparatus. The at least one imaging system modulation unit uses an optical switch that also serves to switch an imaged irradiation image to a plurality of reflection directions, and a photoelectric conversion unit is arranged in each of the reflection directions. The three-dimensional measuring apparatus according to claim 2 or claim 3.

Images