JP4324403B2 - Visibility device for moving objects - Google Patents

Description

【００３７】
また、カメラ間隔計算部２４は、データベースに記憶された複数のカメラ間隔データの中から最適なカメラ間隔データを選択する選択部を有している。識別モードが、前記した（１）「距離指定モード」に設定されている場合には、予め指定された距離までの間に存在する物体を全て認識することができるように、カメラ間隔データを選択する。例えば、（１）「距離指定モード」において予め指定された距離が「５５０ｍ」である場合には、データベースに記憶された表１の関係を用いて、カメラ間隔データとして「０．９ｍ」を選択する。
【００３８】
また、識別モードが、前記した（２）「近距離優先モード」に設定されている場合には、最も遠い位置に存在する物体（すなわち物体が３個ある場合には、３番目に近い位置に存在する物体）を認識することができるように、カメラ間隔データを選択する。例えば、最も遠い位置に存在する対象物までの距離が「３００ｍ」である場合には、データベースに記憶された表１の関係を用いて、カメラ間隔データとして「０．５ｍ」を選択する。
【００３９】
また、カメラ間隔計算部２４は、選択して決定したカメラ間隔データに基づいてカメラ駆動装置１５を駆動するための駆動信号を、前記した画像取得装置１のカメラ駆動装置１５に出力する。カメラ駆動装置１５は、出力された駆動信号を受けて可動式ステレオカメラの可動カメラ１４を移動させてカメラ間隔を変更する。すなわち、カメラ間隔計算部２４及びカメラ駆動装置１５は、本発明におけるカメラ間隔変更手段である。
【００４０】
詳細画像情報生成部２５は、物体を識別する形状データを記憶しており、ステレオ画像処理部２２から送られてきた距離画像情報から物体の形状情報を生成し、この形状情報と予め記憶していた形状情報とを比較して、可動式ステレオカメラで撮像した物体を識別する。また、詳細画像情報生成部２５は、詳細画像指定部２３から送られてきた画像の識別モードに従って詳細画像情報を作成して画像表示装置３に送る。ここでは、詳細画像情報として、詳細画像指定部２３で指定した指定対象物の形状情報と距離情報とを含むものを採用している。
【００４１】
画像表示装置３は、画像表示処理部２１によって送られたステレオカメラの画像を立体画像（立体視界情報）として表示する。すなわち、画像表示装置３は、本発明における立体視界情報表示手段である。本実施の形態においては、画像表示装置３としてＨＭＤ（Head Mounted Display）を採用しており、パイロットの左右の目には、各々左右のカメラからの画像が入ることとなる。
【００４２】
また、画像表示装置３に詳細画像情報生成部２５から距離情報及び形状情報が送られると、立体視界情報に形状情報を重ねて画面に表示する。そして、正確な距離をパイロットに認識させるために定期的に距離を示す数字を形状データの近傍に表示する。さらに、進行方向の物体が所定距離以内に近付いたり進路上に位置したりした場合には、その物体の形状データを点滅させ、警報を鳴らしてパイロットに報知する。
【００４３】
次に、本実施の形態に係る航空機の飛行時における画像表示動作について説明する。
【００４４】
通常の飛行状態においては、画像取得装置１の固定式ステレオカメラによって取得された外界の画像情報に基づいて生成された立体視界情報が、画像表示装置３に表示される。パイロットは、画像表示装置３に表示された立体視界情報を見て操縦を行う。
【００４５】
一方、通常飛行中に、所定の物体をパイロットが認識し、その物体についての詳細な画像を必要とした場合には、パイロットがその物体（指定対象物）を含む領域を識別モードで指定する。識別モードが「近距離優先モード」に設定されている場合には、画像処理装置２の詳細画像指定部２３が、画像取得装置１の固定式ステレオカメラによる撮像情報に基づいて、一番近い指定対象物を確認した後、３番目に近い指定対象物を求め、その距離をステレオ画像処理部２２のデータから呼び出す。３番目に近い指定対象物が存在しない場合には、２番目に近い指定対象物までの距離を呼び出し、２番目に近い指定対象物が存在しない場合には、１番近い指定対象物までの距離を呼び出す。
【００４６】
すなわち、固定式ステレオカメラを構成する固定カメラ１２、１３の間隔やこれら固定カメラ１２、１３の焦点距離等を用いて三角測量の原理により算出された指定対象物までの距離情報を、ステレオ画像処理部２２の記憶部から読み込む。その他のモードでは、指定された距離を記憶していた情報から読み込む。
【００４７】
詳細画像指定部２３によって呼び出された指定対象物までの距離の情報が、カメラ間隔計算部２４に出力されると、カメラ間隔計算部２４の選択部は、指定対象物までの距離に対応したカメラ間隔データを、記憶部であるデータベースから選択する。そして、選択して決定したカメラ間隔データに基づいてカメラ駆動装置１５を駆動するための駆動信号を画像取得装置１に出力する。
【００４８】
カメラ駆動装置１５は、駆動信号を受けて可動式ステレオカメラの可動カメラ１４を移動させてカメラ間隔を変更する。そして、カメラ間隔変更後の可動式ステレオカメラによって取得された画像の距離情報を、画像処理装置２のステレオ画像処理部２２が求めて詳細画像情報生成部２５に出力する。
【００４９】
詳細画像情報生成部２５には、詳細画像指定部２３から識別モードが送られてきており、ステレオ画像処理部２２から送られてきた距離情報を処理して指定対象物の形状情報を生成する。そして、記憶していた形状情報と比較して指定対象物を識別し、この結果と識別モードとに従って詳細画像情報を生成し、この詳細画像情報を画像表示装置３に出力する。このとき、指定対象物の形状情報として記憶していた前記形状情報を画像表示装置３に出力する。
【００５０】
さらに、詳細画像情報生成部２５は、算出された指定対象物までの距離情報を参照して、指定対象物が障害物であるか否かの識別を行う。指定対象物を障害物と認識した場合に、この指定対象物（障害物）をパイロットに報知するための障害物表示情報を生成する。画像表示装置３は、詳細画像情報生成部２４によって生成された障害物表示情報を受けて、かかる障害物表示情報を立体視界情報とともに表示する。
【００５１】
本実施の形態に係る航空機に搭載された移動体の視界装置においては、画像取得装置１の固定式ステレオカメラ、可動式ステレオカメラ及びカメラ駆動装置１５と、画像処理装置２のステレオ画像処理部２２、詳細画像指定部２３及びカメラ間隔計算部２４と、によって、指定対象物までの距離を測定できる。
【００５２】
具体的には、固定式ステレオカメラからの画像情報に基づいて、詳細画像指定部２３で所定の物体（指定対象物）の画像を取得することを指示し、固定式ステレオカメラによる物体の画像情報に基づいて測定された距離に応じて可動式ステレオカメラのカメラ間隔をカメラ間隔変更手段（カメラ間隔計算部２４及びカメラ駆動装置１５）によって変更する。そして、カメラ間隔変更後の可動式ステレオカメラによって取得された物体の画像情報をステレオ画像処理部２２によってステレオ処理する。
【００５３】
すなわち、固定式ステレオカメラ及びステレオ画像処理部２２によって測定した物体までの距離をフィードバックして可動式ステレオカメラのカメラ間隔を変更し、カメラ間隔変更後の可動式ステレオカメラ及びステレオ画像処理部２２によって物体を含む正確な画像情報を取得する。そして、可動式ステレオカメラによる的確な視界情報を、指定された識別モードに従ってリアルタイムで提供させ続けることができる。このため、物体までの距離が大きく頻繁に変化する飛行中においても、物体を的確に識別するとともにこの物体までの正確な距離を提供することができる。この結果、変化する視界の状況を的確に把握することができ、飛行時の安全性を向上させることができる。
【００５４】
また、本実施の形態に係る航空機に搭載された移動体の視界装置においては、カメラ間隔計算部２４のデータベースに複数のカメラ間隔データを予め記憶させておき、詳細画像指定部２３から送られてきた物体までの距離に対応するカメラ間隔データを、これら複数のカメラ間隔データの中から選択する。そして、選択したカメラ間隔データに基づいてカメラ間隔を変更する。すなわち、詳細画像指定部２３から送られてきた物体までの距離を利用して、可動式ステレオカメラのカメラ間隔を簡易な方式で適切に決定することができる。
【００５５】
また、本実施の形態に係る航空機に搭載された移動体の視界装置においては、人の両眼の間隔に略等しい間隔で配置された２台の固定カメラ１２、１３からなる固定式ステレオカメラにより取得した安定的な外界の画像情報に基づいて、操縦者に疲労感を与えない自然な立体視界情報を生成し、この生成した立体視界情報を画像表示装置３に表示することができる。また、可動式ステレオカメラによって取得した画像情報に基づいて所定の物体までの正確な距離を求め、これに基づいて詳細画像情報を生成し、この詳細画像情報（又は識別し易い適切な形状情報に置き換えた画像）を立体視界情報とともに画像表示装置３に表示することができる。すなわち、自然な立体視界情報の提示と、正確な距離情報に基づいた正確な画像情報の提示と、の双方を同時に実現させることができる。
【００５６】
なお、以上の実施の形態においては、画像取得装置１に３台のカメラ（固定カメラ１２、１３、可動カメラ１４）を搭載して、これら３台のカメラによって固定式ステレオカメラ及び可動式ステレオカメラを構成した例を示したが、４台のカメラによって固定式ステレオカメラ及び可動式ステレオカメラを構成することもできる。
【００５７】
すなわち、図２に示すように、画像取得装置１に固定式ステレオカメラを構成する２台の固定カメラ１６、１７と、可動式ステレオカメラを構成する２台の可動カメラ１８、１９と、を搭載し、可動カメラ１８、１９のカメラ間隔をカメラ駆動装置１５で変更することもできる。
【００５８】
また、以上の実施の形態においては、パイロットが正確に確認したい物体（指定対象物）を含む領域を指定し、この指定した領域にある指定対象物が特定の物体に該当するか否かを識別する方式を採用したが、航空機の高度や飛行方向に応じて自動的に障害物の識別を行う方式を採用することもできる。
【００５９】
すなわち、航空機が特定の高度を特定の方向に向けて飛行する場合に、予定した飛行経路上に、固定式ステレオカメラにより物体が撮像された場合には、可動式ステレオカメラにより障害物に成り得るか否かを自動的に識別するようにすることもできる。また、識別モードを選択するとき、ステレオ画像処理部２２に記憶させている色分けした画像情報を一時的に表示して距離を確認できるようにしてもよい。
【００６０】
また、以上の実施の形態においては、本発明に係る移動体の視界装置を搭載する航空機の例として有人の回転翼航空機を示したが、固定翼航空機や飛行船等の他の航空機に本発明に係る移動体の視界装置を搭載してもよい。また、有人操縦の他に、無人操縦や遠隔操縦にも適用できる。また、航空機に限らず、自動車、鉄道車両、船舶等の各種移動体に本発明に係る移動体の視界装置を搭載することもできる。
【００６１】
【発明の効果】
請求項１に記載の発明によれば、固定式ステレオカメラによる画像情報をステレオ画像処理して測定した所定の物体までの距離をフィードバックしてカメラ間隔を変更し、カメラ間隔変更後の可動式ステレオカメラで取得した画像情報をステレオ画像処理して、その物体を含む正確な画像情報を取得し続けることができる。従って、物体までの距離が大きくかつ頻繁に変化する運用環境においても、その物体を含む視界情報を正確に把握することができる。この結果、航空機等に好適に搭載して使用することができる。
また、人の両眼の間隔に略等しい間隔に配置された固定式ステレオカメラによる画像情報に基づいて立体視界情報を生成するので、操縦者又は運転者に疲労感を与えない自然な立体視界情報を表示することができる。また、可動式ステレオカメラの画像情報をステレオ画像処理して測定した所定の対象物までの正確な距離の情報に基づいて的確な画像情報を生成し、この画像情報を立体視界情報とともに表示する。すなわち、自然な立体視界情報の提示と、正確な距離情報に基づいた的確な画像情報の提示と、の双方を同時に実現させることができる。
【００６２】
請求項２に記載の発明によれば、複数のカメラ間隔データを記憶する手段と、距離に対応してカメラ間隔データを選択する手段と、を備えるので、固定式ステレオカメラの画像情報に基づいて測定された距離を利用して、可動式ステレオカメラのカメラ間隔を簡易な方式で適切に決定することができる。
【図面の簡単な説明】
【図１】本発明の実施の形態に係る航空機の主要部（移動体の視界装置）の構成を説明するためのブロック図である。
【図２】本発明の実施の形態に係る航空機の画像取得装置の構成変更例を示すブロック図である。
【符号の説明】
３ 画像表示装置（立体画像情報表示手段）
１２、１３ 固定カメラ
１４ 可動カメラ
１５ カメラ駆動装置（駆動手段、カメラ間隔変更手段）
１６、１７ 固定カメラ
１８、１９ 可動カメラ
２１ 画像表示処理部（立体視界情報生成手段）
２２ ステレオ画像処理部（測距手段）
２３ 詳細画像指定部
２４ カメラ間隔計算部（記憶部、選択部、カメラ間隔変更手段）
２５ 詳細画像情報生成部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a mobile visual field device, and in particular, to accurately measure a distance to a predetermined object and display stereoscopic visual field information and information on an object to be monitored so as to be easily visible. About.
[0002]
[Prior art]
In recent years, a technique has been proposed in which pseudo-stereoscopic field information is generated from image information of the outside world acquired by an image sensor such as an infrared camera or a laser radar, and this stereoscopic field information is presented to an automobile driver or an aircraft operator. Has been. In addition, a technology that improves safety while driving a car or flying an aircraft by detecting obstacles in front using a stereo camera, etc., and presenting such obstacle information to the driver or operator is developed. Has been.
[0003]
For example, as a technique for presenting both the stereoscopic view information and the obstacle information, the fusion view information is generated from the stereoscopic view information and the obstacle information obtained from a pair of images captured using a pair of stereo cameras, A fusion visual field device that presents generated fusion visual field information to a passenger as a visible image has been proposed (see, for example, Patent Document 1).
[0004]
In addition, as a technique for presenting stereoscopic field information, stereoscopic image shooting that eliminates problems that occur when the distance to the object to be photographed is short and long by changing the distance between the two cameras constituting the stereo camera. An apparatus has been proposed (see, for example, Patent Document 2). Further, as a technique for detecting an obstacle, an out-of-vehicle monitoring device that detects a wide range of three-dimensional objects by simultaneously using a long-distance stereo camera and a short-distance stereo camera has been proposed (see Patent Document 3). ).
[0005]
[Patent Document 1]
JP 2001-344597 A (2nd page, FIG. 1)
[Patent Document 2]
JP-A-6-276552 (first page, FIG. 1)
[Patent Document 3]
Japanese Patent Laid-Open No. 11-39596 (first page, FIG. 2)
[0006]
[Problems to be solved by the invention]
By the way, in the fusion field of view device described in Patent Document 1, in order to present natural stereoscopic field information that does not give the driver or driver fatigue, the interval between the two cameras constituting the stereo camera is narrowed. On the other hand, it is necessary to increase the distance between the two cameras in order to detect an obstacle present in the distance. For this reason, since it is necessary to determine the interval between the two cameras so as to satisfy the conflicting requirements at the same time, the obstacle detection range is limited.
[0007]
In addition, in the stereoscopic image capturing apparatus described in Patent Document 2, in order to determine the interval between two cameras constituting the stereo camera, an apparatus that measures the distance from the camera to the object to be captured using ultrasonic waves or the like. (Distance measuring device) is used. Accordingly, it is necessary to prepare a distance measuring device using ultrasonic waves or the like separately from the stereo camera and to construct a complicated electric system and mechanism for linking the distance measuring device and the stereo camera. For this reason, there is a possibility that the operation environment in which the distance to the object to be photographed frequently changes cannot be handled.
[0008]
Moreover, since the outside monitoring apparatus described in Patent Document 3 cannot change the camera interval of the long-distance and short-distance stereo cameras, in an operation environment in which the distance to the obstacle is large and frequently changes. There is a problem that it can not cope.
[0009]
It is an object of the present invention to accurately measure the distance to an object to which attention should be paid because the distance to a predetermined object may be an obstacle in an operating environment where the distance to the predetermined object is large and frequently changes. It is an object of the present invention to provide a moving object visual field device capable of accurately presenting shape information of an object.
[0010]
[Means for Solving the Problems]
  In order to solve the above problems, the invention described in claim 1Arranged at approximately equal spacing between human eyesA fixed stereo camera having two fixed cameras and a movable stereo camera having two cameras with variable camera intervals are provided, and field of view information is presented to a driver or driver of a predetermined moving body In the moving field of view device, the fixed stereo cameraLaImage information of a predetermined object obtained fromAnd beforeFixed stereo turtleLa'sCamera interval andOn the basis of theMeasure the distance to the objectShi,TheThe camera interval of the movable stereo camera is changed according to the measured distance to the object.HandStep andThe distance to the object is measured based on the image information of the object acquired by the movable stereo camera whose camera interval is changed and the camera interval of the movable stereo camera, and the measured distance to the object Generating the distance information based on the image, the means generating the stereoscopic view information based on the image information acquired by the fixed stereo camera, the stereoscopic view information and the distance information as the view information in the display deviceShowHandAnd a step.
[0011]
  According to the first aspect of the present invention, the distance to the object is determined by the principle of triangulation based on the image information of the predetermined object acquired by the fixed stereo camera, the interval between the two fixed cameras, and the like.MeasureAnd the camera distance of the movable stereo camera according to this measured distanceChangeChange. Then, as with the fixed stereo camera, the accurate distance to this object based on the image information of the object acquired by the movable stereo camera after changing the camera interval, the camera interval after changing, etc.MeasureDetermine.
[0012]
  That is, fixed stereo cameraBased on the image information of the predetermined object acquired by the camera and the camera interval of the fixed stereo cameraA movable stereo camera after changing the camera interval by feeding back the distance to the pre-measured object.Based on the image information of the object acquired by the camera and the camera interval of the movable stereo cameraThe distance to the object can be measured again and accurately. Therefore, it is possible to accurately generate object shape information using this accurate distance information.
[0013]
  For example, when measuring the distance to a distant object (distant object), a fixed stereo cameraBased on the image information of the predetermined object acquired by the camera and the camera interval of the fixed stereo cameraIncrease the camera interval of the movable stereo camera according to the measured distance. And this movable stereo camera with this widened camera intervalBased on the image information of the object acquired by the camera and the camera interval of the movable stereo cameraThe distance to the far target object can be measured again and the shape information of the far target object can be accurately generated using the distance information.
[0014]
  In addition, when measuring the distance to an object (a nearby target object) present at a relatively close position, a fixed stereo cameraBased on the image information of the predetermined object acquired by the camera and the camera interval of the fixed stereo cameraThe camera interval of the movable stereo camera is reduced according to the measured distance. And this movable stereo camera with this narrowed camera intervalBased on the image information of the object acquired by the camera and the camera interval of the movable stereo cameraIt is possible to accurately measure the distance to the near target object again, and to accurately generate the shape information of the near target object using the distance information.
[0015]
  Therefore, even in an operating environment where the distance to the object is large and changes frequently, the distance to the object can be measured accurately, and the shape information of the object can be accurately generated using this accurate distance information. ,displayapparatusCan be displayed. As a result, it can be suitably mounted and used on transportation equipment, particularly aircraft.
  In addition, since the stereoscopic field information is generated based on the image information of the outside world acquired by the fixed stereo camera arranged at an interval substantially equal to the interval between both eyes of the person, the fixed stereo camera having two fixed cameras Based on the acquired stable external image information, it is possible to generate natural stereoscopic field information that does not give the passenger a feeling of fatigue, and to display the generated stereoscopic field information.
  Further, since the distance information about the predetermined object is generated from the information captured by the movable stereo camera, the predetermined object can be accurately identified and displayed together with the stereoscopic field information. That is, both the presentation of natural stereoscopic field information and the presentation of accurate image information based on accurate distance information can be realized simultaneously.
[0016]
  According to a second aspect of the present invention, there is provided a visual field device for a moving body according to the first aspect.,in frontThe camera interval of the movable stereo camera and the frontRecordingStores multiple camera interval data that defines the correspondence between the distance range measured by the camera interval.meansAnd beforeNoteFrom the stored multiple camera interval data,Based on the image information of the object acquired by a fixed stereo camera and the camera interval of the fixed stereo cameraMeasuredUp to the objectSelect the camera interval data corresponding to the distancemeansAnd.
[0017]
  According to the second aspect of the present invention, a plurality of camera interval data in which a correspondence relationship between the distance to the object and the camera interval is determined so that the object to be imaged can be accurately captured.NoteRemember. And from these multiple camera interval data, fixed stereo cameraBased on the image information of the predetermined object acquired by the camera and the camera interval of the fixed stereo cameraCamera interval data corresponding to the measured distance is selected, and the camera interval is changed based on the selected camera interval data. That is, fixed stereo cameraBased on the image information of the predetermined object acquired by the camera and the camera interval of the fixed stereo cameraBy using the measured distance to appropriately determine the camera interval of the movable stereo camera by a simple method and capturing an image, accurate information can be acquired.
[0021]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, an aircraft provided with a moving field of view device according to the present invention will be described as an example.
[0022]
First, the configuration of the aircraft according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram for explaining a configuration of a main part (moving object visual field device) of an aircraft according to the present embodiment. The aircraft according to the present embodiment is a manned rotorcraft that is piloted by a pilot.
[0023]
A rotary wing aircraft according to the present embodiment includes various components necessary for flight such as a fuselage, rotor, and tail, a cockpit where a pilot controls, and an image acquisition device 1 as shown in FIG. An image processing device 2 and an image display device 3 are provided. The image processing device 2 and the image display device 3 are both provided in the cockpit, and the image acquisition device 1 and the image processing device 2, and the image processing device 2 and the image display device 3 are electrically connected. Further, the image display device 3 is provided in a form worn on the pilot's head so that the pilot can visually recognize the image displayed on the image display device 3.
[0024]
The image acquisition device 1 acquires image information of the outside world. As shown in FIG. 1, the image acquisition device 1 has a camera frame 11, fixed cameras 12 and 13 fixed to the camera frame 11, and the camera frame 11 on the camera frame 11. The movable camera 14 disposed so as to be movable in the directions of arrows A and B and a camera driving device 15 are provided. The fixed cameras 12 and 13 constitute a fixed stereo camera, and the fixed camera 12 and the movable camera 14 constitute a movable stereo camera.
[0025]
The fixed stereo camera is composed of two fixed cameras 12 and 13 arranged at an interval substantially equal to the interval between both eyes of a person, and stably acquires image information of the outside world. Based on the image information continuously acquired by the fixed cameras 12 and 13 whose camera interval is not changed, the image processing device 2 generates natural stereoscopic field information that does not give the passenger a feeling of fatigue, and generates the generated natural three-dimensional image. Visibility information is displayed on the image display device 3. Further, upon receiving an instruction from the pilot, the image processing apparatus 2 instructs the image acquisition apparatus 1 to acquire detailed image information of an object instructed by the pilot (hereinafter referred to as “designated object”). The distance information based on the fixed stereo camera is used to determine the camera interval of the movable stereo camera.
[0026]
The movable stereo camera is composed of a fixed camera 12 and a movable camera 14 and changes the camera interval in accordance with a signal from the image processing device 2. Image information including a region designated by a pilot or a designated object is obtained. get. Then, the image information acquired by the movable stereo camera is also sent to the image processing device 2 to generate distance information and shape information of the designated object.
[0027]
The movable camera 14 is driven by a camera driving device 15. The camera driving device 15 receives the driving signal from the image processing device 2 and moves the movable camera 14 and is a driving means in the present invention. When the movable camera 14 is moved in the direction of the arrow A in FIG. 1 by the camera driving device 15, the camera interval of the movable stereo camera becomes narrower, and when the movable camera 14 is moved in the direction of the arrow B in FIG. The camera interval of the stereo camera is increased.
[0028]
The image processing device 2 receives input of various image information acquired by the image acquisition device 1, generates stereoscopic view information and object display information from the image information, and generates the generated stereoscopic view information and object display information. This is output to the image display device 3. As shown in FIG. 1, the image processing apparatus 2 includes an image display processing unit 21, a stereo image processing unit 22, a detailed image designating unit 23, a camera interval calculation unit 24, a detailed image information generation unit 25, and the like.
[0029]
The image display processing unit 21 adjusts the brightness and contrast of the image information based on the image information of the outside world acquired by the fixed stereo camera of the image acquisition device 1 and generates stereoscopic field information. . That is, the image display processing unit 21 is a stereoscopic field information generation unit in the present invention. The stereoscopic field information generated by the image display processing unit 21 is output to the image display device 3 and displayed. The image information of the outside world acquired by the fixed stereo camera is also sent to the stereo image processing unit 22.
[0030]
The stereo image processing unit 22 is a distance measuring unit in the present invention, and stereo image processing of the image information of the object acquired by the fixed and movable stereo cameras of the image acquisition device 1 (the distance to the object is determined by the principle of triangulation). To calculate). Then, the distance to the object determined from the image information obtained by the fixed stereo camera is stored in a storage unit (not shown) as image information (distance image information) color-coded according to the distance. In the present embodiment, an object existing in a relatively close position is displayed in red, an object existing in a distant position is displayed in blue, and an object existing in an intermediate position is displayed in an intermediate color between red and blue. Yes. Further, the distance image information obtained from the image information obtained by the movable stereo camera is output to the detailed image information generation unit 25.
[0031]
The detailed image designating unit 23 stores the distance image information obtained from the image information obtained by the fixed stereo camera in the stereo image processing unit 22 based on the information of the designated identification method (identification mode). And output to the camera interval calculation unit 24. Alternatively, the designated distance is output to the camera interval calculation unit 24. At the same time, information regarding the identification mode is output to the detailed image information generation unit 25.
[0032]
As the identification mode, (1) “distance designation mode” for recognizing all objects existing up to a predetermined distance as designated objects, and (2) the third closest position to the object existing at the closest position. “Short distance priority mode” for recognizing up to an existing object as a designated object, (3) “Object limitation mode” for presetting an object that exists between a predetermined distance and is recognized as a designated object , Etc.
[0033]
The types of objects recognized as designated objects in the “object limitation mode” in (3) are (a) steel tower, (b) building, (c) vehicle, (d) person, (e) combinations thereof (For example, a combination of a building and a vehicle, a combination of a building and a person, a combination of a vehicle and a person) and the like. The identification mode in the detailed image designating unit 23 is stored in advance, and the setting is performed by the pilot using a switch or the like not shown.
[0034]
The camera interval calculation unit 24 is a movable stereo camera that can measure the detailed information of an object within the range of the distance or the point of the distance more precisely from the distance information sent from the detailed image designating unit 23. The camera interval is determined, and the determined camera interval is output to the image acquisition device 1.
[0035]
The camera interval calculation unit 24 stores the camera interval data (m) of the movable stereo camera and the object detection permissible range data (m) that is data of a distance range measurable by each camera interval data. It has a database as a storage unit. Table 1 shows an example of the camera interval data and the object detection allowable range data stored in the database. The camera interval data and the object detection allowable range data stored in the database are calculated based on the lens focal length of the camera to be used, the number of pixels of the camera, and the like.
[0036]
[Table 1]

[0037]
The camera interval calculation unit 24 has a selection unit that selects optimal camera interval data from among a plurality of camera interval data stored in the database. When the identification mode is set to (1) “Distance designation mode” as described above, the camera interval data is selected so that all objects existing up to a predetermined distance can be recognized. To do. For example, (1) When the distance designated in advance in “distance designation mode” is “550 m”, “0.9 m” is selected as the camera interval data using the relationship of Table 1 stored in the database. To do.
[0038]
In addition, when the identification mode is set to (2) “Short distance priority mode” as described above, the object located at the farthest position (that is, when there are three objects, the position closest to the third position). Camera interval data is selected so that an existing object can be recognized. For example, when the distance to the object existing at the farthest position is “300 m”, “0.5 m” is selected as the camera interval data using the relationship of Table 1 stored in the database.
[0039]
Further, the camera interval calculation unit 24 outputs a drive signal for driving the camera drive device 15 to the camera drive device 15 of the image acquisition device 1 described above based on the camera interval data selected and determined. The camera drive device 15 receives the output drive signal and moves the movable camera 14 of the movable stereo camera to change the camera interval. That is, the camera interval calculation unit 24 and the camera driving device 15 are camera interval changing means in the present invention.
[0040]
The detailed image information generation unit 25 stores shape data for identifying an object, generates object shape information from the distance image information sent from the stereo image processing unit 22, and stores the shape information in advance. The object captured by the movable stereo camera is identified by comparing the obtained shape information. The detailed image information generation unit 25 creates detailed image information according to the image identification mode sent from the detailed image designation unit 23 and sends the detailed image information to the image display device 3. Here, as the detailed image information, information including the shape information and distance information of the specified object specified by the detailed image specifying unit 23 is adopted.
[0041]
The image display device 3 displays the image of the stereo camera sent by the image display processing unit 21 as a stereoscopic image (stereoscopic view information). That is, the image display device 3 is a stereoscopic field information display unit in the present invention. In the present embodiment, an HMD (Head Mounted Display) is adopted as the image display device 3, and images from the left and right cameras respectively enter the left and right eyes of the pilot.
[0042]
Further, when distance information and shape information are sent from the detailed image information generation unit 25 to the image display device 3, the shape information is superimposed on the stereoscopic field information and displayed on the screen. Then, a number indicating the distance is periodically displayed in the vicinity of the shape data in order to make the pilot recognize the accurate distance. Further, when an object in the traveling direction approaches within a predetermined distance or is positioned on the course, the shape data of the object is blinked, and an alarm is sounded to notify the pilot.
[0043]
Next, an image display operation during flight of the aircraft according to the present embodiment will be described.
[0044]
In a normal flight state, stereoscopic view information generated based on image information of the outside world acquired by the fixed stereo camera of the image acquisition device 1 is displayed on the image display device 3. The pilot operates by looking at the stereoscopic field information displayed on the image display device 3.
[0045]
On the other hand, when a pilot recognizes a predetermined object during normal flight and needs a detailed image of the object, the pilot designates an area including the object (designated object) in the identification mode. When the identification mode is set to “short distance priority mode”, the detailed image designation unit 23 of the image processing apparatus 2 designates the closest designation based on the imaging information by the fixed stereo camera of the image acquisition apparatus 1. After confirming the target object, the third closest target object is obtained, and the distance is called from the data of the stereo image processing unit 22. When there is no third closest specified object, the distance to the second closest specified object is called, and when there is no second closest specified object, the distance to the closest specified object Call.
[0046]
That is, the distance information to the designated object calculated by the principle of triangulation using the distance between the fixed cameras 12 and 13 constituting the fixed stereo camera, the focal length of these fixed cameras 12 and 13, and the like is obtained by stereo image processing. Read from the storage unit of the unit 22. In other modes, the specified distance is read from the stored information.
[0047]
When the information on the distance to the specified object called by the detailed image specifying unit 23 is output to the camera interval calculating unit 24, the selection unit of the camera interval calculating unit 24 selects the camera corresponding to the distance to the specified object. The interval data is selected from a database that is a storage unit. Then, a drive signal for driving the camera drive device 15 is output to the image acquisition device 1 based on the selected camera interval data.
[0048]
The camera drive device 15 receives the drive signal and moves the movable camera 14 of the movable stereo camera to change the camera interval. Then, the distance information of the image acquired by the movable stereo camera after changing the camera interval is obtained by the stereo image processing unit 22 of the image processing device 2 and output to the detailed image information generating unit 25.
[0049]
An identification mode is sent from the detailed image designating unit 23 to the detailed image information generating unit 25, and the distance information sent from the stereo image processing unit 22 is processed to generate shape information of the designated object. Then, the designated object is identified in comparison with the stored shape information, detailed image information is generated according to the result and the identification mode, and the detailed image information is output to the image display device 3. At this time, the shape information stored as the shape information of the designated object is output to the image display device 3.
[0050]
Furthermore, the detailed image information generation unit 25 refers to the calculated distance information to the designated object and identifies whether or not the designated object is an obstacle. When the designated object is recognized as an obstacle, obstacle display information for informing the pilot of the designated object (obstacle) is generated. The image display device 3 receives the obstacle display information generated by the detailed image information generation unit 24 and displays the obstacle display information together with the stereoscopic field information.
[0051]
In the moving field of view device mounted on the aircraft according to the present embodiment, the fixed stereo camera, the movable stereo camera, and the camera driving device 15 of the image acquisition device 1 and the stereo image processing unit 22 of the image processing device 2 are used. The detailed image designation unit 23 and the camera interval calculation unit 24 can measure the distance to the designated object.
[0052]
Specifically, based on the image information from the fixed stereo camera, the detailed image designating unit 23 is instructed to acquire an image of a predetermined object (designated object), and the image information of the object by the fixed stereo camera is obtained. The camera interval of the movable stereo camera is changed by the camera interval changing means (camera interval calculating unit 24 and camera driving device 15) according to the distance measured based on the above. Then, the stereo image processing unit 22 performs stereo processing on the object image information acquired by the movable stereo camera after the camera interval is changed.
[0053]
That is, the distance to the object measured by the fixed stereo camera and the stereo image processing unit 22 is fed back to change the camera interval of the movable stereo camera, and the movable stereo camera and the stereo image processing unit 22 after the camera interval change is performed. Accurate image information including an object is acquired. And it can continue providing the exact visual field information by a movable stereo camera in real time according to the designated identification mode. For this reason, even during a flight in which the distance to the object is large and frequently changes, the object can be accurately identified and an accurate distance to the object can be provided. As a result, it is possible to accurately grasp the changing visual field situation and improve the safety during flight.
[0054]
Further, in the mobile field of view device mounted on the aircraft according to the present embodiment, a plurality of camera interval data is stored in advance in the database of the camera interval calculation unit 24 and sent from the detailed image designation unit 23. Camera interval data corresponding to the distance to the object is selected from the plurality of camera interval data. Then, the camera interval is changed based on the selected camera interval data. That is, by using the distance to the object sent from the detailed image specifying unit 23, the camera interval of the movable stereo camera can be appropriately determined by a simple method.
[0055]
Moreover, in the mobile field of view device mounted on the aircraft according to the present embodiment, a fixed stereo camera composed of two fixed cameras 12 and 13 arranged at an interval substantially equal to the interval between human eyes. Based on the acquired stable external image information, natural stereoscopic field information that does not give the driver a feeling of fatigue can be generated, and the generated stereoscopic field information can be displayed on the image display device 3. Further, an accurate distance to a predetermined object is obtained based on the image information acquired by the movable stereo camera, and detailed image information is generated based on this, and the detailed image information (or appropriate shape information that is easy to identify) is generated. The replaced image) can be displayed on the image display device 3 together with the stereoscopic field information. That is, both the presentation of natural stereoscopic vision information and the presentation of accurate image information based on accurate distance information can be realized simultaneously.
[0056]
In the above embodiment, the image acquisition device 1 is equipped with three cameras (fixed cameras 12, 13 and a movable camera 14), and these three cameras are used for a fixed stereo camera and a movable stereo camera. Although the example which comprised this was shown, a fixed stereo camera and a movable stereo camera can also be comprised by four cameras.
[0057]
That is, as shown in FIG. 2, the image acquisition apparatus 1 includes two fixed cameras 16 and 17 constituting a fixed stereo camera and two movable cameras 18 and 19 constituting a movable stereo camera. The camera interval between the movable cameras 18 and 19 can be changed by the camera driving device 15.
[0058]
In the above embodiment, the pilot designates an area including the object (designated object) that the pilot wants to confirm accurately, and identifies whether the designated object in the designated area corresponds to a specific object. However, it is also possible to adopt a method of automatically identifying obstacles according to the altitude and flight direction of the aircraft.
[0059]
That is, when an aircraft flies at a specific altitude in a specific direction, if an object is imaged by a fixed stereo camera on a planned flight path, it can become an obstacle by a movable stereo camera. It is also possible to automatically identify whether or not. Further, when selecting the identification mode, the color-coded image information stored in the stereo image processing unit 22 may be temporarily displayed so that the distance can be confirmed.
[0060]
In the above embodiment, a manned rotary wing aircraft has been shown as an example of an aircraft equipped with the mobile field of view device according to the present invention. However, the present invention is applied to other aircraft such as fixed wing aircraft and airships. Such a moving field of view device may be mounted. In addition to manned maneuvering, it can also be applied to unmanned maneuvering and remote maneuvering. In addition, the visual field device for a moving body according to the present invention can be mounted not only on an aircraft but also on various moving bodies such as automobiles, railway vehicles, and ships.
[0061]
【The invention's effect】
  According to the first aspect of the present invention, the distance to a predetermined object measured by stereo image processing of image information obtained by a fixed stereo camera is fed back to change the camera interval, and the movable stereo after the camera interval is changed. Stereo image processing is performed on the image information acquired by the camera, and accurate image information including the object can be continuously acquired. Therefore, even in an operating environment where the distance to an object is large and changes frequently, the field-of-view information including the object can be accurately grasped. As a result, it can be suitably mounted and used on an aircraft or the like.
  In addition, since the stereoscopic view information is generated based on the image information obtained by the fixed stereo camera arranged at a distance approximately equal to the distance between both eyes of the person, the natural stereoscopic view information that does not give the driver or driver fatigue. Can be displayed. In addition, accurate image information is generated based on information on an accurate distance to a predetermined object measured by performing stereo image processing on the image information of the movable stereo camera, and this image information is displayed together with the stereoscopic field information. That is, both the presentation of natural stereoscopic field information and the presentation of accurate image information based on accurate distance information can be realized simultaneously.
[0062]
  According to invention of Claim 2,, DoubleMemorize the number of camera interval datameansAnd select the camera interval data corresponding to the distance.meansWhenTheSince the distance is measured based on the image information of the fixed stereo camera, the camera interval of the movable stereo camera can be appropriately determined by a simple method.
[Brief description of the drawings]
FIG. 1 is a block diagram for explaining a configuration of a main part (a moving object field of view device) of an aircraft according to an embodiment of the present invention.
FIG. 2 is a block diagram showing a configuration change example of the aircraft image acquisition apparatus according to the embodiment of the present invention.
[Explanation of symbols]
3. Image display device (stereoscopic image information display means)
12, 13 Fixed camera
14 Movable camera
15 Camera drive device (drive means, camera interval change means)
16, 17 Fixed camera
18, 19 Movable camera
21 Image display processing unit (stereoscopic view information generating means)
22 Stereo image processing unit (ranging means)
23 Detailed image specification part
24 camera interval calculation unit (storage unit, selection unit, camera interval changing means)
25 Detailed image information generator

Claims (2)

A fixed stereo camera having two fixed cameras arranged at a distance approximately equal to the distance between both eyes of a human and a movable stereo camera having two cameras having variable camera intervals, and having a predetermined movement In a mobile field of view device that presents field of view information to a driver or driver of a body,
Wherein a distance to the object is measured based on the image information of the predetermined object more acquired stationary stereo camera and the camera interval before Symbol stationary stereo camera, the distance to the object that is the measuring and hand-stage to change the camera interval of the movable stereo camera according,
The distance to the object is measured based on the image information of the object acquired by the movable stereo camera whose camera interval is changed and the camera interval of the movable stereo camera, and the measured distance to the object Means for generating distance information based on
Means for generating stereoscopic field information based on image information acquired by the fixed stereo camera;
Hand stage that displays and said distance information and the stereoscopic viewing information on the display device as said viewing information,
A moving field of view device comprising: Means for storing the camera interval before Symbol movable stereo camera, before and range of the distance measuring at hear camera intervals, a plurality of camera separation data defining the correspondence between,
From the previous crisis憶plurality of camera separation data, the distance to the object is measured based on the camera interval of the fixed image information and the stationary stereo camera of the object acquired by a stereo camera Means for selecting corresponding camera interval data;
The visual field device for a moving body according to claim 1, comprising:

Images