JP4152800B2 - Road falling object detection device - Google Patents

Description

【００５４】
次いで、処理部２は、ステップＳ２５で算出した当該小領域の相関値が閾値ａより小さいか否かを判定する（ステップＳ２６）。閾値ａより小さければ（すなわち、現画像の当該小領域と背景画像の当該小領域との相関性が所定程度より低ければ）、ステップＳ２７へ移行し、一方、閾値ａより小さくなければ、ステップＳ２９へ移行する。当該小領域が路上落下物に相当する領域であれば、当該小領域の相関値は閾値ａより小さくなり、ステップＳ２７へ移行することになる。
【００５５】
ステップＳ２７において、処理部２は、ステップＳ２３で得た２値化背景差分画像における当該小領域（ステップＳ２４で最新に選択された小領域）が、白画素（現画像の画素と背景画像の画素との差が大きい画素）を含むか否かを、判定する。白画素を含む場合は、処理部２は、当該小領域を一次候補小領域（路上落下物の一次的な候補となる小領域）であると判別した（ステップＳ２８）後に、ステップＳ２９へ移行する。一方、ステップＳ２８において白画素を含まないと判定されると、処理部２は、当該小領域を一次候補小領域であると判別することなく、ステップＳ２９へ移行する。
【００５６】
ステップＳ２９において、処理部２は、既に説明したようにして予め設定された複数の小領域の全てについてステップＳ２４の選択が終了した否かを判定し、終了していなければステップＳ２４へ戻り、終了していればステップＳ３０へ移行する。
【００５７】
本実施の形態のように、一次候補小領域の判別に際して、ステップＳ２２，Ｓ２３の処理を行い、ステップＳ２７でＹＥＳの場合にのみ当該小領域を一次候補小領域であると判別することが、一次候補小領域の抽出の精度をより高めるために好ましい。しかしながら、本発明では、ステップＳ２２，Ｓ２３，２７を除去してステップＳ２１の後に直ちにステップＳ２４へ移行し、ステップＳ２６でＹＥＳの場合にステップＳ２８へ移行してもよい。
【００５８】
ステップＳ３０において、処理部２は、既に説明したようにして予め設定された複数の小領域のうち、当該ステップＳ３０で今回未だ選択されていない１つの小領域を選択する。
【００５９】
次に、処理部２は、ステップＳ３０で最新に選択された小領域がステップＳ２８で一次候補小領域であると判別されたものか否かを、判定する（ステップＳ３１）。当該小領域が一次候補小領域でなければ、処理部２は、当該小領域に関連づけられたカウント値をゼロにリセットし（ステップＳ３２）、ステップＳ３６へ移行する。一方、当該小領域が一次候補小領域であれば、処理部２は、当該小領域に関連づけられたカウント値を１だけインクリメントし（ステップＳ３３）、ステップＳ３４へ移行する。
【００６０】
ステップＳ３４において、処理部２は、当該小領域のカウント値が閾値ｂ以上であるか否かを判定する。閾値ｂ以上であれば、処理部２は、当該小領域を二次候補小領域であると判別し（ステップＳ３５）、ステップＳ３６へ移行する。
【００６１】
ステップＳ３１〜Ｓ３５によって、ステップＳ２１でサンプリングされたｂ枚の連続する画像の全てについて、ある小領域が一次候補小領域となった場合に、初めて、当該小領域が二次候補小領域であると判別されることになる。路上落下物は静止物体であるので、当該小領域が路上落下物に相当する領域であれば、やがて当該領域は二次候補小領域と判別されることになる。
【００６２】
本発明では、ステップＳ３２において、当該小領域に関連づけられたカウント値をゼロにリセットする代わりに、当該小領域に関連づけられたカウント値を、ゼロを最小値として１だけデクリメントしてもよい。この場合には、路上落下物が自動車の陰に隠れて見え隠れするような事態が頻繁に起きても、より迅速に二次候補小領域が判別されるため、より迅速に路上落下物を検出することができる。
【００６３】
ステップＳ３６において、処理部２は、既に説明したようにして予め設定された複数の小領域の全てについてステップＳ３０の選択が終了した否かを判定し、終了していなければステップＳ３０へ戻り、終了していればステップＳ３７へ移行する。
【００６４】
ステップＳ３７において、処理部２は、ステップＳ３５で二次候補小領域であると判別された小領域についてラベリング処理を行う。これによるラベリング領域（ラベル付けされた領域）には、隣接する複数の二次候補小領域が存在する場合のそれらの連結領域と、他の隣接しない単独の二次候補小領域がある場合のその単独の二次候補小領域とが含まれることになる。
【００６５】
次に、処理部２は、ステップＳ３７でラベル付けされた各領域（各ラベリング領域）についてその大きさが所定大きさ以上であるか否かを判定し、所定大きさ以上のラベリング領域のみをそれぞれ落下物候補領域として判別する（ステップＳ３８）。小領域の大きさを比較的小さく設定した場合には、本実施の形態のように、ラベリング領域の大きさで落下物候補領域を選別することが好ましい。もっとも、処理部２は、ステップＳ３８を行う代わりに、ラベリング領域の大きさに拘わらず、全てのラベリング領域をそれぞれ落下物候補領域として判別してもよい。特に、小領域の大きさを比較的大きく設定した場合などには、ラベリング領域の大きさで落下物候補領域を選別しなくても、落下物候補領域の選別精度が落ちるようなことはない。
【００６６】
ここで、背景画像が図２に示す画像であるとともに、各ループにおけるステップＳ２１でのサンプリング画像が、図２に示す画像からある時点で図３に示す画像となり、その後も繰り返して図３に示す画像となる場合を例に挙げて、説明する。サンプリング画像が最初に図３に示す画像となったループのステップＳ３３で、図１０（ａ）に示すように、４つの小領域のカウント値が１となる。その後、サンプリング画像が最初に図３に示す画像となってからｎ回目のループのステップＳ３３で、図１０（ｂ）に示すように、前記４つの小領域のカウント値がｎとなる。そして、サンプリング画像が最初に図３に示す画像となってからｂ回目のループのステップＳ３３で、前記４つの小領域のカウント値がｂとなると、ステップＳ３５でこれらの４つの小領域がそれぞれ二次候補小領域と判別され、図１０（ｃ）に示すように、これらの４つの小領域がステップＳ３７でラベリングされて４つの小領域分の連結領域（図１０（ｃ）のハッチング領域）、この１つの連結領域が所定大きさ以上であれば、ステップＳ３８で落下物候補領域として判別される。
【００６７】
ステップＳ３９以降の処理は、このようにして得られる落下物候補領域に関して行われる。なお、ステップＳ３８で判別される落下物候補領域が１つに限られるものでないことは、言うまでもない。
【００６８】
なお、本発明では、ステップＳ３７及びＳ３８を除去して、ステップＳ３６でＹＥＳの場合には、ステップＳ３５で判別された各二次候補小領域をそのまま落下物候補領域として判別し、この落下物候補領域に関してステップＳ３９以降の処理を行ってもよい。
【００６９】
ステップＳ３８が終了すると、処理部２は、最新のステップＳ３８で落下物候補領域として判別されたラベリング領域があるか否かを判定する（ステップＳ３９）。最新のステップＳ３８で判別された落下物候補領域がない場合は、ステップＳ４の処理を終了してステップＳ５へ移行する。一方、最新のステップＳ３８で判別された落下物候補領域がある場合は、処理部２は、最新のステップＳ３８で判別された落下物候補領域のうち、今回未だステップＳ３９で選択されていない１つの落下物候補領域を選択する（ステップＳ４０）。
【００７０】
次に、処理部２は、ステップＳ３９で選択された落下物候補領域について、当該落下物候補領域が検出処理対象として、既に、処理部２の内部メモリ（図示せず）に登録されているか否かを判定する（ステップＳ４１）。既に登録されていれば、ステップＳ４３へ移行する。一方、未登録であれば、処理部２は、ステップＳ３９で選択された落下物候補領域を前記メモリに検出処理対象として登録し（ステップＳ４２）、ステップＳ４３へ移行する。なお、ステップＳ３９で選択された落下物候補領域が既に登録されている落下物候補領域と完全に同一でなければ、両者の領域の一部が重複していても、ステップＳ３９で選択された落下物候補領域は、ステップＳ４２で検出処理対象として登録される。一旦検出処理対象として登録された落下物候補領域は、後述するステップＳ５５で削除されない限り、検出処理対象として登録され続ける。
【００７１】
ステップＳ４３において、処理部２は、最新のステップＳ３８で判別された落下物候補領域の全てについてステップＳ４０の選択が終了した否かを判定し、終了していなければステップＳ４０へ戻り、終了していればステップＳ４の処理を終了してステップＳ５へ移行する。
【００７２】
次に、ステップＳ５の落下物候補領域の追跡・落下物判定処理の詳細について、図８を参照して説明する。
【００７３】
ステップＳ４の処理が終了すると、処理部２は、その内部メモリに、現在、落下物候補領域が検出処理対象として登録されているか否かを判定する（ステップＳ５１）。登録されていなければ、ステップＳ５の処理を終了してステップＳ６へ移行する。一方、登録されていれば、ステップＳ５２へ移行する。
【００７４】
ステップＳ５２において、処理部２は、その内部メモリに検出処理対象として現在登録されている落下物候補領域のうち、当該ステップＳ５２で未だ今回選択されていない１つの落下物候補領域を選択する。
【００７５】
次に、処理部２は、ステップＳ５２で選択された落下物候補領域ついて、現画像と背景画像との間の相関値を算出する（ステップＳ５３）。次いで、処理部２は、ステップＳ５３で算出した当該落下物候補領域の相関値が閾値ｃより小さいか否かを判定する（ステップＳ５４）。閾値ｃより小さければ（すなわち、現画像の当該落下物候補領域と背景画像の当該落下物候補領域との相関性が所定程度より低ければ）、ステップＳ５７へ移行し、一方、閾値ｃより小さくなければ、ステップＳ５５へ移行する。当該落下物候補領域が路上落下物に相当する領域であれば、当該落下物候補領域の相関値は閾値ｃより小さくなり、ステップＳ５７へ移行することになる。
【００７６】
本実施の形態のように、ステップＳ５３，Ｓ５４の処理を行うことが、路上落下物の検出精度をより高めるために好ましい。しかしながら、本発明では、ステップＳ５３，５４を除去してステップＳ５２の後に直ちにステップＳ５７へ移行してもよい。
【００７７】
ステップＳ５５において、処理部２は、当該落下物候補領域（ステップＳ５２で最新に選択された落下物候補領域）の検出処理対象としての登録を、内部メモリから削除し、ステップＳ５６へ移行する。なお、ステップＳ５５において、当該落下物候補領域の検出処理対象としての登録を削除するだけでなく、ステップＳ３２と同様に、当該落下物候補領域に相当する小領域に関連づけられているカウント値をリセット又はデクリメントしてもよい。
【００７８】
ステップＳ５７において、処理部２は、当該落下物候補領域（ステップＳ５２で最新に選択された落下物候補領域）について、現画像と前回ステップＳ２１でサンプリングされた画像との間（ただし、１ループ目のステップ５７では、現画像とステップＳ３でサンプリングされた画像との間）の相関値を算出する。次いで、処理部２は、ステップＳ５７で算出した相関値が閾値ｄより大きいか否かを判定する（ステップＳ５８）。閾値ｄより大きければ（すなわち、現画像の当該落下物候補領域と前回ステップＳ２１でサンプリングされた画像（ただし、１ループ目では、ステップＳ３でサンプリングされた画像）の当該落下物候補領域との間の相関性が所定の程度より大きければ）、ステップＳ５９へ移行し、一方、閾値ｄより大きくなければ、ステップＳ５５へ移行する。当該落下物候補領域がが路上落下物に相当する領域であれば、当該落下物候補領域の相関値は閾値ｄより大きくなり、ステップＳ５９へ移行することになる。
【００７９】
ステップＳ５９において、処理部２は、当該落下物候補領域（ステップＳ５２で最新に選択された落下物候補領域）に関連づけられた存在時間カウント値を、１だけインクリメントする。存在時間カウント値は、ステップＳ４１で落下物候補領域が検出処理対象が登録される際に、ゼロに初期設定されて当該落下物候補領域に関連づけて、処理部２の内部メモリに記憶されている。したがって、存在時間カウント値は、これに関連づけられた落下物候補領域が検出処理対象として登録されている時間に対応している。
【００８０】
次に、処理部２は、当該落下物候補領域（ステップＳ５２で最新に選択された落下物候補領域）に関連づけられた存在時間カウント値が閾値ｅ以上であるか否かを判定する（ステップＳ６０）。すなわち、処理部２は、当該落下物候補領域が検出処理対象として登録されている時間が所定時間以上であるか否かを判定する。閾値ｅ以上でなければステップＳ５６へ移行する。一方、閾値ｅ以上であれば、処理部２は、路上落下物が存在すると判定し、落下物検出信号を出力し、全ての処理を終了する。当該落下物候補領域が路上落下物に相当する領域であれば、当該落下物候補領域の存在時間カウント値はやがて閾値以上となり、ステップＳ６１へ移行することになる。
【００８１】
なお、ステップＳ４１で落下物候補領域が検出処理対象が登録される際に、当該落下物候補領域と関連づけてその登録時刻を記憶し、ステップＳ５９を削除して、ステップＳ５８でＹＥＳの場合に、ステップＳ６０において、当該落下物候補領域の登録時刻と現在時刻とを比較することで当該落下物候補領域の登録されている時間を求め、その時間が所定時間以上であるか否かを判定してもよい。
【００８２】
ステップＳ５６において、処理部２は、検出処理対象として現在登録されている全ての落下物候補領域の全てについてステップＳ５２の選択が終了したか否かを判定し、終了していなければステップＳ５２へ戻り、終了していればステップＳ５の処理を終了してステップＳ６へ移行する。
【００８３】
次に、ステップＳ６の背景画像の更新処理の詳細について、図９を参照して説明する。
【００８４】
ステップＳ５からステップＳ６へ移行すると、まず、処理部２は、背景更新条件を満たしているか否かを判定する（ステップＳ７１）。背景更新条件を満たしていなければ、背景画像を更新することなくステップＳ４へ戻り、満たしていれば、背景画像を更新した（ステップＳ７２）後にステップＳ４へ戻る。ステップＳ７２の背景画像の更新は、例えば、現画像をｇ（ｔ）とするとともに現在の背景画像をｆ（ｔ−１）として前述した数１に従って画像ｆ（ｔ）を算出し、この画像ｆ（ｔ）を更新後の背景画像とすることによって、行うことができる。
【００８５】
ここで、背景更新条件としては、例えば、下記の（ａ）〜（ｆ）の条件のうちの任意の１つ以上の条件の組み合わせとすることができる。また、下記の（ｆ）の条件を満たす場合には、（ａ）〜（ｅ）の条件を満たすか否かに拘わらずに、強制的に背景画像を更新するようにしてもよい。このとき、背景更新処理として、新たに背景画像を作成し直すこととし、ステップＳ２と同じ処理を行ってもよい。
【００８６】
（ａ）外部から停止車両の検知信号を受けることとし、当該検知信号を受けていないこと。
【００８７】
（ｂ）外部から渋滞情報を受けることとし、当該渋滞情報を受けていないこと。
【００８８】
（ｃ）最新にステップＳ２５で算出された各小領域についての現画像と背景画像との間の相関値に基づいて、これらの相関値の平均値及び分散値のうちの少なくとも一方を算出することとし、前記平均値が所定値よりも大きいか、前記分散値が所定値よりも小さいか、あるいは、前記平均値が所定値よりも大きくかつ前記分散値が所定値よりも小さいこと。前記平均値が大きいければ、渋滞などが発生していないと推測できることが判明した。また、前記分散値が小さければ、渋滞などが発生していないと推測できることが判明した。
【００８９】
（ｄ）現在、検出処理対象として落下物候補領域が登録されていないこと。
【００９０】
（ｅ）最新にステップＳ３７でラベル付けされた領域の大きさが所定値以下であること。
【００９１】
（ｆ）監視領域がトンネル内の領域である場合において、トンネル内の照明モードを示す信号を受けることとし、照明モードが変更されたこと。
【００９２】
本実施の形態では、小領域ごとに求めたサンプリング画像と背景画像との間の相関値（ステップＳ２５）に基づいて路上落下物候補領域（ステップＳ３８）を得、この落下物候補領域のうち現画像と前回のサンプリング画像との間の相関値が所定値より大きいもの（ステップＳ５８）について検出処理対象とし、当該検出処理対象の落下物候補領域が所定時間以上存在する場合に路上落下物が存在すると判定している（ステップＳ６０，Ｓ６１）。このように、本実施の形態によれば、小領域ごとのサンプリング画像と背景画像との相関による路上落下物候補領域の抽出と、現画像と前回のサンプリング画像との間の相関による路上落下物候補領域の絞り込みとの、いわば２段階の選抜を経た領域に関して、時間的に継続して存在するという条件下で、路上落下物を検出している。したがって、本実施の形態によれば、前記従来の路上落下物検出装置に比べて、より高い精度で路上落下物を検出することができる。
【００９３】
また、本実施の形態では、路上落下物候補領域の抽出の際の相関値演算（ステップＳ２５）は小領域ごとに行うので、その演算量は全領域について一括して行う場合に比べて少なくてすむ。さらに、路上落下物候補領域の絞り込みの際の相関値演算（ステップＳ５３，Ｓ５７）は路上落下物候補領域についてのみ行うので、その演算量も少なくてすむ。したがって、本実施の形態によれば、全体的に見ても、演算量は比較的少なくてすむ。
【００９４】
以上、本発明の実施の形態について説明したが、本発明はこの実施の形態に限定されるものではない。
【００９５】
【発明の効果】
以上説明したように、本発明によれば、比較的少ない演算量ですむとともに、より高い精度で路上落下物を検出することができる路上落下物検出装置を提供することができる。
【図面の簡単な説明】
【図１】本発明の一実施の形態による路上落下物検出装置を示す概略ブロック図である。
【図２】撮像された画像の例を模式的に示す概略図である。
【図３】撮像された画像の他の例を模式的に示す概略図である。
【図４】図１に示す路上落下物検出装置の動作の全体を示す概略フローチャートである。
【図５】図４中のステップＳ２の詳細を示す概略フローチャートである。
【図６】図４中のステップＳ４の一部の詳細を示す概略フローチャートである。
【図７】図４中のステップＳ４の残りの部分の詳細を示す概略フローチャートである。
【図８】図４中のステップＳ５の詳細を示す概略フローチャートである。
【図９】図４中のステップＳ６の詳細を示す概略フローチャートである。
【図１０】図１に示す路上落下物検出装置の所定の処理過程を模式的に示す説明図である。
【符号の説明】
１ テレビカメラ
２ 処理部
３ 画像メモリ[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a road falling object detection device that detects a falling object on a road.
[0002]
[Prior art]
Conventionally, an apparatus disclosed in Patent Document 1 below is known as this type of road falling object detection apparatus. In this device, an image sampled from a moving image is segmented into small regions, and a normalized correlation value with a corresponding region of a background image that does not include a moving object prepared in advance for each small region and does not have an obstacle is calculated. Is repeatedly performed on an image sampled at regular time intervals, and a change in the correlation value is determined with a constant threshold value to detect an obstacle on the road.
[0003]
In such a conventional road fall object detection device, a correlation value between a sampling image and a background image is obtained for each small area, and road fall objects are detected based on these correlation values. The correlation value between the sampled image and the background image can be obtained for all regions at once, and the processing speed of the correlation value calculation can be increased as compared with the case where a falling object on the road is detected based on this correlation value. In addition, it is possible to improve the detection accuracy of falling objects on the road.
[0004]
[Patent Document 1]
JP 2002-24808 A
[0005]
[Problems to be solved by the invention]
However, in the conventional road falling object detection device, the temporal change state of the correlation value between the sampling image and the background image obtained for each small area is obtained with a certain threshold value, so that sufficient detection accuracy is not necessarily obtained. It was difficult to get.
[0006]
The present invention has been made in view of such circumstances, and an object of the present invention is to provide a road falling object detection apparatus that can detect a road falling object with higher accuracy while requiring a relatively small amount of calculation. .
[0007]
[Means for Solving the Problems]
In order to solve the above-described problem, the road falling object detection device according to the first aspect of the present invention includes an imaging unit that images a monitoring area including a road area, and a plurality of small areas obtained by dividing the image area according to the monitoring area. Each of the first correlation value calculating means for obtaining a correlation value between an image obtained from the imaging means according to the passage of time and sequentially sampled and a background image, a current image that is the latest sampled image, and Road fall object candidates that are candidates for road fall objects related to the current image based on the correlation values of the small areas obtained by the first correlation value calculation means for the image sampled in the past. Road fall object candidate area obtaining means for obtaining an area, means for registering the road fall object candidate area obtained in association with the current image as a detection process target, and current detection object as the detection process target Obtained by the second correlation value calculating means for obtaining a correlation value between the current image and the previously sampled image, and the second correlation value calculating means for the registered falling object candidate area on the road Means for deleting the falling object candidate area on the road whose correlation value is less than or equal to a predetermined value from the registration as the detection processing target, and at least of the falling object candidate area currently registered as the detection processing target Determination means for determining that there is a fallen object on the road when a value corresponding to a time registered as the detection processing target of one area indicates a predetermined time or more. is there.
[0008]
In this first aspect, a road falling object candidate area is obtained based on the correlation value between the sampling image and the background image obtained for each small area, and the current image and the previous sampling image of the falling object candidate area are obtained. If the correlation value between the two is larger than a predetermined value, it is determined as a detection processing target, and if there is a falling object candidate area for the detection processing target for a predetermined time or more, it is determined that there is a falling object on the road. As described above, according to the first aspect, the extraction of the road fall object candidate area by the correlation between the sampling image and the background image for each small area and the road fall by the correlation between the current image and the previous sampling image are performed. On the condition that the object candidate area is narrowed down, that is, the area that has been selected in two stages, the falling object on the road is detected under the condition that it exists in time. Therefore, it is possible to detect a falling object on the road with higher accuracy than the conventional falling object detection apparatus on the road.
[0009]
Further, in the first aspect, the correlation value calculation at the time of extracting the falling object candidate area on the road is performed for each small area, so that the amount of calculation can be reduced compared with the case where the calculation is performed for all areas at once. Furthermore, since the correlation value calculation at the time of narrowing down the road fall object candidate area is performed only for the road fall object candidate area, the calculation amount can be reduced. Therefore, according to the first aspect, the calculation amount is relatively small even when viewed as a whole.
[0010]
The road falling object detection device according to a second aspect of the present invention is the road falling object candidate area obtaining means according to the first aspect, wherein: (a) the current correlation image is obtained by the first correlation value calculating means; In addition, based on the correlation value of each of the small areas, it is determined whether each of the plurality of small areas in the current image is a primary candidate small area that is a small area that is a primary candidate for the falling object on the road. Primary candidate small region determination means for performing (b) a count associated with each of the plurality of small regions associated with the small region determined to be the primary candidate small region for the current image Increment means for incrementing a value; and (c) a small area that is determined not to be the primary candidate small area for the current image among the count values associated with each of the plurality of small areas A reset means for resetting the associated count value; and (d) a secondary candidate in which, after the increment by the increment means, the associated count value is a small area having a predetermined value or more after the increment. And secondary candidate small region discriminating means for discriminating the small region.
[0011]
In this second aspect, by introducing the count value for each small area and adopting the increment means and the reset means, secondary candidate small areas are extracted on condition of continuity of the primary candidate small areas. . Therefore, according to the second aspect, the falling object candidate area can be more appropriately extracted, and consequently, the falling object on the road can be detected with higher accuracy.
[0012]
The road falling object detection device according to a third aspect of the present invention, in the first aspect, the road falling object candidate area acquisition means is obtained by (a) the first correlation value calculation means for the current image. In addition, based on the correlation value of each of the small areas, it is determined whether each of the plurality of small areas in the current image is a primary candidate small area that is a small area that is a primary candidate for the falling object on the road. Primary candidate small region determination means for performing (b) a count associated with each of the plurality of small regions associated with the small region determined to be the primary candidate small region for the current image Increment means for incrementing a value; and (c) a small area that is determined not to be the primary candidate small area for the current image among the count values associated with each of the plurality of small areas A decrement means for decrementing the associated count value; and (d) a secondary candidate whose associated count value is a small area equal to or greater than a predetermined value among the plurality of small areas after the increment by the increment means. And secondary candidate small region discriminating means for discriminating the small region.
[0013]
The third mode is basically the same as the second mode, but the decrementing unit is employed in place of the reset unit employed in the second mode. Therefore, even if a situation in which a fallen object on the road is hidden behind the automobile and frequently appears, the fallen object on the road can be detected more quickly.
[0014]
In the road falling object detection device according to a fourth aspect of the present invention, in the second or third aspect, the primary candidate small region determination means is obtained by the first correlation value calculation means for the current image. A small area having a correlation value determining means for determining whether or not the correlation value of each of the small areas is equal to or greater than a predetermined value, wherein the correlation value is determined to be equal to or greater than the predetermined value by the correlation value determining means; Is determined to be the primary candidate small region.
[0015]
This fourth aspect is an example of a primary candidate small region determination method. According to the fourth aspect, since the primary candidate small area is simply determined based only on the magnitude of the correlation value, the calculation amount can be reduced.
[0016]
According to a fifth aspect of the present invention, in the second or third aspect, in the second or third aspect, the primary candidate small region determining means is obtained by the first correlation value calculating means for the current image. Correlation value determination means for determining whether or not the correlation value of each of the small regions is equal to or greater than a predetermined value; and means for obtaining a difference image between the current image and the background image, and the correlation value determination Among the small areas determined by the means that the correlation value is determined to be equal to or greater than the predetermined value, a small area including pixels having a difference value equal to or greater than the predetermined value in the difference image is determined to be the primary candidate small area. Is.
[0017]
The fifth aspect is another example of the primary candidate small region determination method. According to the fifth aspect, not only the primary candidate small area is determined based on the magnitude of the correlation value, but also a small area including a pixel having a difference value equal to or larger than a predetermined value in the difference image among the small areas having a large correlation value. The area is determined to be a primary candidate small area. Therefore, although the calculation amount is slightly increased as compared with the fourth aspect, the primary candidate small region can be extracted with higher accuracy, and consequently, the falling object on the road can be detected with higher accuracy.
[0018]
In the road falling object detection device according to a sixth aspect of the present invention, in any one of the second to fifth aspects, the road falling object candidate area acquisition means includes a labeling means for labeling the secondary candidate small area. And having the labeled area as the road fall object candidate area.
[0019]
In the sixth aspect, an example in which a labeling area of a secondary candidate small area is adopted as a road falling object candidate area.
[0020]
The road falling object detection device according to a seventh aspect of the present invention is the road falling object detection area acquiring means according to any one of the second to fifth aspects, wherein the road falling object candidate area obtaining means is a labeling means for labeling the secondary candidate small area. And a size determination unit that determines whether or not the size of the labeled area is equal to or greater than a predetermined size, and the labeling region that is determined to be greater than or equal to the predetermined size by the size determination unit Is obtained as the on-road fallen object candidate region.
[0021]
As in the sixth aspect, the seventh aspect is an example in which the labeling area of the secondary candidate small area is adopted as the road fall object candidate area. In the seventh aspect, since the road fall object candidate area having a labeling area of a predetermined size or larger is used as the road fall object candidate area, the road fall object candidate area can be extracted with higher accuracy, and therefore with higher accuracy. A falling object on the road can be detected.
[0022]
The road falling object detection device according to an eighth aspect of the present invention, in the sixth or seventh aspect, for the road falling object candidate area currently registered as the detection processing target, the current image and the background image. A third correlation value calculating means for obtaining a correlation value between the road fall object candidate area on the road whose correlation value obtained by the third correlation value calculating means is greater than or equal to a predetermined value; And means for deleting from the registration.
[0023]
According to the eighth aspect, for the labeling area as the falling object candidate area, a correlation value between the current image and the background image is obtained, and an area where the correlation value is equal to or greater than a predetermined value is excluded from the detection processing target area. Therefore, a falling object on the road can be detected with higher accuracy.
[0024]
According to a ninth aspect of the present invention, in the road falling object detection device according to any one of the second to fifth aspects, the road falling object candidate area acquisition unit uses the secondary candidate small area as the road falling object candidate. It is obtained as a region.
[0025]
The ninth aspect is an example in which the secondary candidate small area is directly adopted as the road fall object candidate area.
[0026]
According to a tenth aspect of the present invention, the road falling object detection device according to any one of the first to ninth aspects includes update means for updating the background image under a predetermined condition based on the sampled image. It is provided.
[0027]
According to the tenth aspect, since the update means for updating the background image under a predetermined condition is provided, the surrounding conditions (such as outdoor weather conditions, lighting conditions in the tunnel, vehicle stoppage and traffic jams, etc.) ) Changes, it is possible to detect a fallen object on the road with higher accuracy.
[0028]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, a road falling object detection device according to the present invention will be described with reference to the drawings.
[0029]
FIG. 1 is a schematic block diagram showing a road falling object detection apparatus according to an embodiment of the present invention.
[0030]
As shown in FIG. 1, the on-road falling object detection device according to the present embodiment performs A / D conversion on a television camera 1 as an imaging means for imaging a monitoring area including a road area and an image signal from the television camera 1. The image memory 3 is provided with a processing unit 2 that performs sampling based on the sampled image and detects a fallen object on the road. Although not shown in the drawings, the processing unit 2 includes, for example, a microcomputer and other electronic circuits so as to realize an operation described later in addition to the A / D converter.
[0031]
In the present invention, the monitoring area may be an area in a tunnel or an outdoor area.
[0032]
2 and 3 are schematic views schematically showing examples of images taken by the television camera 1. FIG. 2 shows a state where there is no moving object such as a falling object on the road or an automobile, and FIG. 3 shows a state where a falling object exists on the road. 2 and 3, 11 is an image of the lane on one side of the road, 12 is an image of the center line, and 13 is an image of the lane on the other side. In FIG. 3, reference numeral 14 denotes an image of a fallen object on the road.
[0033]
In this embodiment, a plurality of small areas obtained by dividing an image area corresponding to the monitoring area are set in advance with respect to the image, as indicated by broken lines in FIGS. 2 and 3. In the present embodiment, each small area is set to the same size on the image area, but in accordance with the actual size, the small area of the near side area is set to be large, and the back side area is set. The small area may be set small. In addition, the number and shape of each small area can be appropriately set in consideration of the size of the road fallen object to be detected. Each small area is associated with a count value used in steps S32 to S34 described later, and the count value is initially set to zero. These count values are stored in an internal memory (not shown) of the processing unit.
[0034]
In the present embodiment, the entire image captured by the television camera 1 is assumed to be a monitoring area. However, an area where a falling object on the road cannot exist is excluded from the monitoring area. Needless to say, it may be excluded from the target of the processing described later by masking. In this case, the small area may be set only for a desired monitoring area.
[0035]
Next, the operation of the road falling object detection device according to the present embodiment will be described with reference to FIGS.
[0036]
FIG. 4 is a schematic flowchart showing the overall operation of the road surface state determination device according to the present embodiment. FIG. 5 is a schematic flowchart showing details of step S2 in FIG. FIG. 6 is a schematic flowchart showing details of a part of step S4 in FIG. FIG. 7 is a schematic flowchart showing details of the remaining part of step S4 in FIG. FIG. 8 is a schematic flowchart showing details of step S5 in FIG. FIG. 9 is a schematic flowchart showing details of step S6 in FIG.
[0037]
First, the general operation of the road falling object detection device according to the present embodiment will be described with reference to FIG.
[0038]
First, by operating an input unit (not shown), various parameters necessary for processing (for example, the cumulative number N used in step S16 described later, the threshold value a used in step S26, the threshold value b used in step S34, and used in step S54). Threshold c, threshold d used in step S58, threshold e used in step S60, etc.) are set (step S1). These parameters are stored in an internal memory (not shown) of the processing unit. Of course, these parameters may be set in advance and cannot be changed.
[0039]
Next, the processing unit 2 creates an initial background image (step S2). Next, the processing unit 2 samples the current image from the television camera 1 as the previous image required in step S5 in the first one-loop process, and fetches it into the image memory 3 (step S3). Thereafter, the processing unit 2 extracts / registers a fallen object candidate area (step S4), and performs tracking processing on the registered fallen object candidate area to determine whether there is a fallen object on the road (step S5). . When determining that there is a falling object on the road, the processing unit 2 outputs a falling object detection signal to the outside, and ends the process. On the other hand, if the processing unit determines that there is no falling object on the road, the processing unit updates the background image under a predetermined condition (step S6), and returns to step S4.
[0040]
Next, details of the initial background image creation processing in step S2 will be described with reference to FIG.
[0041]
When step S1 ends, the processing unit 2 sets a count value n indicating the cumulative number of times to zero (step S11). Next, the processing unit 2 initializes the previous accumulated image f (t−1) to an image in which all pixels are zero (step S12). Thereafter, the processing unit 2 samples the current image from the television camera 1 as the current image g (t) and loads it into the image memory 3 (step S13).
[0042]
Next, the processing unit 2 creates the current cumulative image f (t) according to the following equation 1 based on the previous cumulative image f (t−1) and the image g (t) sampled in step S13. (Step S14). In Equation 1, α is a weighting coefficient that satisfies 0 <α <1, and is set to a value that reduces the influence of a moving body such as an automobile.
[0043]
[Expression 1]
f (t) = αf (t−1) + (1−α) g (t)
[0044]
Thereafter, the processing unit 2 increments the count value n stored in its internal memory (not shown) by 1 (step S15), and then determines whether or not the count value n has reached a predetermined cumulative number N. (Step S16). If the predetermined cumulative number N has not been reached, the processing unit 2 sets the current cumulative image f (t) as the previous cumulative image f (t-1) (step S17), and returns to step S13. On the other hand, if it is determined in step S16 that the predetermined cumulative number N has been reached, the processing unit 2 stores the current cumulative image f (t) in the image memory 3 as an initial background image (step S18).
[0045]
By creating an initial background image in this way, an initial background image as shown in FIG. 2 can be created. Of course, it goes without saying that an image picked up by the television camera 1 in the absence of a moving body such as an automobile may be sampled and used as an initial background image.
[0046]
In the following description, the background image means the initial background image stored in step S18 until it is updated in step S6. After the update in step S6, the background image updated in step S6 is the latest updated background image. Means an image.
[0047]
Next, details of the extraction / registration process of the falling object candidate area in step S4 will be described with reference to FIGS.
[0048]
When the process of step S3 is completed, the processing unit 2 samples the current image from the television camera 1 and loads it into the image memory 3 (step S21). The image sampled most recently in step S21 is called the current image.
[0049]
Next, the processing unit 2 creates a difference image (background difference image) between the current image and the background image (step S22), and binarizes the background difference image with a predetermined threshold (step S23). Here, in the background difference image after binarization, a pixel having a large difference between the pixel of the current image and the pixel of the background image is a white pixel. Therefore, the pixel at the place where the falling object exists on the road is a white pixel. Note that the threshold value used for binarization of the background difference image may be a fixed threshold value or a variable threshold value based on, for example, a discriminant analysis method.
[0050]
Thereafter, the processing unit 2 selects the plurality of small areas set in advance as described above (for example, individual areas separated by broken lines in FIGS. 2 and 3) at this step S24. One small area that has not been selected is selected (step S24).
[0051]
Next, the processing unit 2 calculates a correlation value between the current image and the background image for the small region most recently selected in step S24 (step S25). For example, if the small area most recently selected in step S24 is the lower left small area in FIGS. 2 and 3, the processing unit 2 calculates the lower left small area of the current image and the lower left small area of the background image. A correlation value between them is calculated. The correlation value is also called a correlation coefficient, and the higher the value is, the higher the correlation between both regions is.
[0052]
As a specific calculation formula for obtaining the correlation value, various publicly known formulas can be adopted. As an example, the following formula 2 can be mentioned. In Equation 2, r indicates a correlation value. Equation 2 is an equation for calculating the correlation value r in the p × q pixel region of the image f (x, y) and the image g (x, y). In Equation 2, N is the total number of pixels in the area.
[0053]
[Expression 2]

[0054]
Next, the processing unit 2 determines whether or not the correlation value of the small region calculated in step S25 is smaller than the threshold value a (step S26). If it is smaller than the threshold value a (that is, if the correlation between the small area of the current image and the small area of the background image is lower than a predetermined level), the process proceeds to step S27. Migrate to If the small area corresponds to a falling object on the road, the correlation value of the small area becomes smaller than the threshold value a, and the process proceeds to step S27.
[0055]
In step S27, the processing unit 2 determines that the small region (the small region selected most recently in step S24) in the binarized background difference image obtained in step S23 is a white pixel (the pixel of the current image and the pixel of the background image). It is determined whether or not the pixel includes a pixel having a large difference. If a white pixel is included, the processing unit 2 determines that the small area is a primary candidate small area (a small area that is a primary candidate for a fallen object on the road) (step S28), and then proceeds to step S29. . On the other hand, if it is determined in step S28 that the white pixel is not included, the processing unit 2 proceeds to step S29 without determining that the small area is the primary candidate small area.
[0056]
In step S29, the processing unit 2 determines whether or not the selection in step S24 has been completed for all of the plurality of preset small areas as described above. If not completed, the processing unit 2 returns to step S24 and ends. If so, the process proceeds to step S30.
[0057]
As in the present embodiment, when determining the primary candidate small area, the processes of steps S22 and S23 are performed, and only when YES is determined in step S27, it is determined that the small area is the primary candidate small area. This is preferable in order to further improve the accuracy of extraction of candidate small regions. However, in the present invention, steps S22, S23, and 27 may be removed, and the process may proceed to step S24 immediately after step S21. If YES in step S26, the process may proceed to step S28.
[0058]
In step S30, the processing unit 2 selects one small region that has not yet been selected in step S30 from among a plurality of small regions set in advance as described above.
[0059]
Next, the processing unit 2 determines whether or not the small region selected most recently in Step S30 is determined to be the primary candidate small region in Step S28 (Step S31). If the small area is not the primary candidate small area, the processing unit 2 resets the count value associated with the small area to zero (step S32), and proceeds to step S36. On the other hand, if the small area is the primary candidate small area, the processing unit 2 increments the count value associated with the small area by 1 (step S33), and proceeds to step S34.
[0060]
In step S34, the processing unit 2 determines whether or not the count value of the small area is equal to or greater than the threshold value b. If it is equal to or greater than the threshold value b, the processing unit 2 determines that the small region is a secondary candidate small region (step S35), and proceeds to step S36.
[0061]
When a certain small area becomes a primary candidate small area for all the b consecutive images sampled in step S21 by steps S31 to S35, the small area is a secondary candidate small area for the first time. It will be determined. Since the falling object on the road is a stationary object, if the small area corresponds to the falling object on the road, the area is eventually determined as the secondary candidate small area.
[0062]
In the present invention, in step S32, instead of resetting the count value associated with the small region to zero, the count value associated with the small region may be decremented by 1 with zero being the minimum value. In this case, even if a situation where the fallen object on the road is hidden behind the car frequently occurs, the secondary candidate small area is identified more quickly, so the fallen object on the road is detected more quickly. be able to.
[0063]
In step S36, the processing unit 2 determines whether or not the selection in step S30 has been completed for all of the plurality of preset small areas as described above. If not completed, the processing unit 2 returns to step S30 and ends. If so, the process proceeds to step S37.
[0064]
In step S37, the processing unit 2 performs a labeling process on the small area determined to be the secondary candidate small area in step S35. In this labeling area (labeled area), when there are a plurality of adjacent secondary candidate small areas, those connected areas, and when there are other non-adjacent single secondary candidate small areas, A single secondary candidate small region is included.
[0065]
Next, the processing unit 2 determines whether or not the size of each region (each labeling region) labeled in step S37 is greater than or equal to a predetermined size, and only the labeling regions that are greater than or equal to the predetermined size. It is determined as a fallen object candidate area (step S38). When the size of the small region is set to be relatively small, it is preferable to select the fallen object candidate region based on the size of the labeling region as in the present embodiment. However, instead of performing step S38, the processing unit 2 may determine all the labeling areas as fallen object candidate areas regardless of the size of the labeling area. In particular, when the size of the small area is set to be relatively large, the falling object candidate area selection accuracy does not deteriorate even if the falling object candidate area is not selected based on the size of the labeling area.
[0066]
Here, the background image is the image shown in FIG. 2, and the sampling image in step S21 in each loop becomes the image shown in FIG. 3 at a certain point in time from the image shown in FIG. 2, and then repeatedly shown in FIG. The case where an image is used will be described as an example. In step S33 of the loop in which the sampled image first becomes the image shown in FIG. 3, the count values of the four small regions become 1, as shown in FIG. Thereafter, in step S33 of the n-th loop after the sampling image first becomes the image shown in FIG. 3, the count values of the four small regions become n, as shown in FIG. 10B. Then, when the count value of the four small areas becomes b in step S33 of the b-th loop after the sampling image becomes the image shown in FIG. 3 for the first time, each of the four small areas becomes two in step S35. As shown in FIG. 10 (c), these four small areas are labeled in step S37, and connected areas corresponding to the four small areas (hatched areas in FIG. 10 (c)), If this one connected area is larger than the predetermined size, it is determined as a fallen object candidate area in step S38.
[0067]
The processing after step S39 is performed on the fallen object candidate region obtained in this way. Needless to say, the number of fallen object candidate areas determined in step S38 is not limited to one.
[0068]
In the present invention, steps S37 and S38 are removed, and in the case of YES in step S36, each secondary candidate small area determined in step S35 is determined as it is as a falling object candidate area, and this falling object candidate is determined. You may perform the process after step S39 regarding an area | region.
[0069]
When step S38 ends, the processing unit 2 determines whether or not there is a labeling area determined as the falling object candidate area in the latest step S38 (step S39). If there is no fallen object candidate area determined in the latest step S38, the process of step S4 is terminated and the process proceeds to step S5. On the other hand, when there is a fallen object candidate area determined in the latest step S38, the processing unit 2 selects one of the fallen object candidate areas determined in the latest step S38 that has not yet been selected in step S39. A falling object candidate area is selected (step S40).
[0070]
Next, regarding the falling object candidate area selected in step S39, the processing unit 2 has already registered the falling object candidate area in the internal memory (not shown) of the processing unit 2 as a detection processing target. Is determined (step S41). If already registered, the process proceeds to step S43. On the other hand, if not registered, the processing unit 2 registers the falling object candidate region selected in step S39 as a detection processing target in the memory (step S42), and proceeds to step S43. Note that if the fallen object candidate area selected in step S39 is not completely identical to the already registered fallen object candidate area, the fall selected in step S39 is performed even if a part of both areas overlap. The object candidate area is registered as a detection processing target in step S42. The fallen object candidate area once registered as the detection processing target continues to be registered as the detection processing target unless deleted in step S55 described later.
[0071]
In step S43, the processing unit 2 determines whether or not the selection in step S40 has been completed for all the fallen object candidate areas determined in the latest step S38. If not, the process unit 2 returns to step S40 and ends. Then, the process of step S4 is terminated and the process proceeds to step S5.
[0072]
Next, details of the tracking / falling object determination process of the falling object candidate area in step S5 will be described with reference to FIG.
[0073]
When the process of step S4 ends, the processing unit 2 determines whether or not the falling object candidate area is currently registered as a detection process target in the internal memory (step S51). If not registered, the process of step S5 is terminated and the process proceeds to step S6. On the other hand, if registered, the process proceeds to step S52.
[0074]
In step S52, the processing unit 2 selects one fallen object candidate area that has not yet been selected in step S52 from among the fallen object candidate areas currently registered as detection processing targets in the internal memory.
[0075]
Next, the processing unit 2 calculates a correlation value between the current image and the background image for the falling object candidate region selected in step S52 (step S53). Next, the processing unit 2 determines whether or not the correlation value of the falling object candidate area calculated in step S53 is smaller than the threshold value c (step S54). If it is smaller than the threshold value c (that is, if the correlation between the falling object candidate area of the current image and the falling object candidate area of the background image is lower than a predetermined level), the process proceeds to step S57, while it must be smaller than the threshold value c. Then, the process proceeds to step S55. If the falling object candidate area is an area corresponding to a falling object on the road, the correlation value of the falling object candidate area becomes smaller than the threshold value c, and the process proceeds to step S57.
[0076]
As in the present embodiment, it is preferable to perform the processing of steps S53 and S54 in order to further increase the accuracy of detecting a fallen object on the road. However, in the present invention, steps S53 and S54 may be removed and the process may proceed to step S57 immediately after step S52.
[0077]
In step S55, the processing unit 2 deletes the registration as the detection processing target of the falling object candidate area (the falling object candidate area most recently selected in step S52) from the internal memory, and proceeds to step S56. In step S55, not only the registration of the falling object candidate area as the detection processing target is deleted, but also the count value associated with the small area corresponding to the falling object candidate area is reset as in step S32. Or you may decrement.
[0078]
In step S57, the processing unit 2 determines between the current image and the image sampled in the previous step S21 for the fallen object candidate area (the latest fallen object candidate area selected in step S52) (however, in the first loop). In step 57, a correlation value between the current image and the image sampled in step S3 is calculated. Next, the processing unit 2 determines whether or not the correlation value calculated in step S57 is larger than the threshold value d (step S58). If it is larger than the threshold value d (that is, between the falling object candidate area of the current image and the falling object candidate area of the image sampled in the previous step S21 (however, in the first loop, the image sampled in step S3)). If the correlation is greater than a predetermined level), the process proceeds to step S59, and if not, the process proceeds to step S55. If the falling object candidate area is an area corresponding to a falling object on the road, the correlation value of the falling object candidate area becomes larger than the threshold value d, and the process proceeds to step S59.
[0079]
In step S59, the processing unit 2 increments the existence time count value associated with the fallen object candidate area (the fallen object candidate area most recently selected in step S52) by one. The existence time count value is initially set to zero and stored in the internal memory of the processing unit 2 when the falling object candidate area is registered as a detection processing target in step S41 and is associated with the falling object candidate area. . Therefore, the existence time count value corresponds to the time when the fallen object candidate area associated therewith is registered as the detection processing target.
[0080]
Next, the processing unit 2 determines whether or not the existence time count value associated with the falling object candidate area (the falling object candidate area selected most recently in Step S52) is equal to or greater than the threshold e (Step S60). ). That is, the processing unit 2 determines whether or not the time during which the falling object candidate region is registered as a detection processing target is a predetermined time or more. If it is not more than the threshold value e, it will transfer to step S56. On the other hand, if it is more than the threshold value e, the process part 2 will determine with the fallen object on a road existing, will output a fallen object detection signal, and will complete | finish all the processes. If the falling object candidate area is an area corresponding to a falling object on the road, the existence time count value of the falling object candidate area will eventually become equal to or greater than the threshold value, and the process proceeds to step S61.
[0081]
In addition, when a detection target for a fallen object candidate area is registered in step S41, the registration time is stored in association with the fallen object candidate area, step S59 is deleted, and if YES in step S58, In step S60, the registered time of the falling object candidate area is compared with the current time to obtain the registered time of the falling object candidate area, and it is determined whether or not the time is a predetermined time or more. Also good.
[0082]
In step S56, the processing unit 2 determines whether or not the selection in step S52 has been completed for all of the fallen object candidate regions currently registered as detection processing targets. If not, the processing unit 2 returns to step S52. If completed, the process of step S5 is terminated and the process proceeds to step S6.
[0083]
Next, details of the background image update processing in step S6 will be described with reference to FIG.
[0084]
When the process proceeds from step S5 to step S6, first, the processing unit 2 determines whether or not the background update condition is satisfied (step S71). If the background update condition is not satisfied, the process returns to step S4 without updating the background image. If the background update condition is satisfied, the background image is updated (step S72), and then the process returns to step S4. The background image is updated in step S72 by, for example, calculating the image f (t) according to the above-described equation 1 with the current image g (t) and the current background image f (t-1). This can be done by setting (t) as the updated background image.
[0085]
Here, as the background update condition, for example, any one or more of the following conditions (a) to (f) can be combined. When the following condition (f) is satisfied, the background image may be forcibly updated regardless of whether the conditions (a) to (e) are satisfied. At this time, as the background update process, a new background image may be newly created, and the same process as step S2 may be performed.
[0086]
(A) The detection signal of the stopped vehicle is received from the outside, and the detection signal is not received.
[0087]
(B) The traffic information is received from the outside, and the traffic information is not received.
[0088]
(C) calculating at least one of an average value and a variance value of these correlation values based on the correlation value between the current image and the background image for each small region calculated in step S25 most recently. The average value is larger than a predetermined value, the variance value is smaller than a predetermined value, or the average value is larger than a predetermined value and the variance value is smaller than a predetermined value. It has been found that if the average value is large, it can be assumed that there is no traffic jam. It was also found that if the variance value is small, it can be assumed that no traffic jam has occurred.
[0089]
(D) The falling object candidate area is not currently registered as a detection processing target.
[0090]
(E) The size of the area labeled in the latest step S37 is not more than a predetermined value.
[0091]
(F) When the monitoring area is an area in the tunnel, a signal indicating the illumination mode in the tunnel is received and the illumination mode is changed.
[0092]
In the present embodiment, a road falling object candidate area (step S38) is obtained based on the correlation value (step S25) between the sampling image and the background image obtained for each small area, and the present falling object candidate area is selected from the current falling object candidate areas. If the correlation value between the image and the previous sampled image is greater than the predetermined value (step S58), the object is to be detected, and there is a fallen object on the road when the object to be detected is present for more than a predetermined time. Then, it is determined (steps S60 and S61). As described above, according to the present embodiment, the falling object candidate area on the road is extracted based on the correlation between the sampling image and the background image for each small area, and the falling object on the road is determined based on the correlation between the current image and the previous sampling image. On the condition that the candidate area is narrowed down, that is, the area that has been selected in two stages, the fallen object on the road is detected under the condition that it exists in time. Therefore, according to the present embodiment, it is possible to detect a falling object on the road with higher accuracy than the conventional falling object detection apparatus.
[0093]
Further, in this embodiment, the correlation value calculation (step S25) at the time of extracting the falling object candidate area on the road is performed for each small area, so that the amount of calculation is less than the case where the calculation is performed collectively for all areas. I'm sorry. Further, since the correlation value calculation (steps S53 and S57) when narrowing down the road fall object candidate area is performed only on the road fall object candidate area, the calculation amount can be reduced. Therefore, according to the present embodiment, the calculation amount is relatively small even when viewed as a whole.
[0094]
As mentioned above, although embodiment of this invention was described, this invention is not limited to this embodiment.
[0095]
【The invention's effect】
As described above, according to the present invention, it is possible to provide a road falling object detection apparatus that can detect a road falling object with higher accuracy while requiring a relatively small amount of calculation.
[Brief description of the drawings]
FIG. 1 is a schematic block diagram showing a falling object detection apparatus on a road according to an embodiment of the present invention.
FIG. 2 is a schematic diagram schematically showing an example of a captured image.
FIG. 3 is a schematic diagram schematically illustrating another example of a captured image.
4 is a schematic flowchart showing the overall operation of the road falling object detection device shown in FIG. 1; FIG.
FIG. 5 is a schematic flowchart showing details of step S2 in FIG. 4;
6 is a schematic flowchart showing details of a part of step S4 in FIG. 4; FIG.
FIG. 7 is a schematic flowchart showing details of the remaining part of step S4 in FIG.
FIG. 8 is a schematic flowchart showing details of step S5 in FIG. 4;
FIG. 9 is a schematic flowchart showing details of step S6 in FIG. 4;
10 is an explanatory view schematically showing a predetermined processing process of the road falling object detection device shown in FIG. 1; FIG.
[Explanation of symbols]
1 TV camera
2 processing section
3 Image memory

Claims (12)

Imaging means for imaging a monitoring area including a road area;
First correlation value calculation for obtaining a correlation value between a sequentially sampled image obtained from the imaging means and sequentially sampled for each of a plurality of small areas obtained by dividing an image area corresponding to the monitoring area Means,
Based on the correlation value of each of the small regions obtained by the first correlation value calculation means for the current image that is the latest sampled image and the image sampled in the past, the current image is related to the current image. Road falling object candidate area acquisition means for obtaining a road falling object candidate area that is a candidate for road falling object,
Means for registering the falling object candidate area on the road obtained in relation to the current image as a detection processing target;
Second correlation value calculation means for obtaining a correlation value between the current image and the previously sampled image for the road falling object candidate region currently registered as the detection processing target;
Means for deleting the road fall object candidate area whose correlation value obtained by the second correlation value calculating means is not more than a predetermined value from registration as the detection processing target;
When the value corresponding to the time registered as the detection processing target of at least one of the fallen object candidate regions currently registered as the detection processing target indicates a predetermined time or more Determining means for determining that there is a fallen object on the road;
Equipped with a,
The on-road falling object candidate area acquisition means includes: (a) the plurality of small areas for the current image based on the correlation values of the small areas obtained by the first correlation value calculation means for the current image; Primary candidate small region determination means for determining whether or not each is a primary candidate small region that is a small region that is a primary candidate for the fallen object on the road, and (b) is associated with each of the plurality of small regions. And (c) an increment means for incrementing a count value associated with the small area determined to be the primary candidate small area for the current image, and (c) associated with each of the plurality of small areas. A decrement means for decrementing a count value associated with a small area that is determined not to be the primary candidate small area for the current image among the count values; A secondary candidate small region discriminating unit that discriminates a secondary candidate small region that is a small region in which the associated count value is equal to or larger than a predetermined value among the plurality of small regions after the increment by the increment unit; ,
A falling object detection device on the road. The primary candidate small area determining means includes correlation value determining means for determining whether or not the correlation value of each small area obtained by the first correlation value calculating means for the current image is greater than or equal to a predetermined value. 2. The on-road falling object detection according to claim 1, further comprising: determining, as the primary candidate small region, a small region in which the correlation value is determined to be equal to or greater than the predetermined value by the correlation value determining unit. apparatus. The primary candidate small area determining means includes correlation value determining means for determining whether or not the correlation value of each small area obtained by the first correlation value calculating means for the current image is greater than or equal to a predetermined value. And a means for obtaining a difference image between the current image and the background image, in the difference image among the small areas determined by the correlation value determination means that the correlation value is equal to or greater than the predetermined value. The on-road falling object detection device according to claim 1, wherein a small area including a pixel having a difference value equal to or greater than a predetermined value is determined as the primary candidate small area. Imaging means for imaging a monitoring area including a road area;
First correlation value calculation for obtaining a correlation value between a sequentially sampled image obtained from the imaging means and sequentially sampled for each of a plurality of small areas obtained by dividing an image area corresponding to the monitoring area Means,
Based on the correlation value of each of the small regions obtained by the first correlation value calculation means for the current image that is the latest sampled image and the image sampled in the past, the current image is related to the current image. Road falling object candidate area acquisition means for obtaining a road falling object candidate area that is a candidate for road falling object,
Means for registering the falling object candidate area on the road obtained in relation to the current image as a detection processing target;
Second correlation value calculation means for obtaining a correlation value between the current image and the previously sampled image for the road falling object candidate region currently registered as the detection processing target;
Means for deleting the road fall object candidate area whose correlation value obtained by the second correlation value calculating means is not more than a predetermined value from registration as the detection processing target;
When the value corresponding to the time registered as the detection processing target of at least one of the fallen object candidate regions currently registered as the detection processing target indicates a predetermined time or more Determining means for determining that there is a fallen object on the road;
With
The on-road falling object candidate area acquisition means includes: (a) the plurality of small areas for the current image based on the correlation values of the small areas obtained by the first correlation value calculation means for the current image; Primary candidate small region determination means for determining whether or not each is a primary candidate small region that is a small region that is a primary candidate for the fallen object on the road, and (b) is associated with each of the plurality of small regions. And (c) an increment means for incrementing a count value associated with the small area determined to be the primary candidate small area for the current image, and (c) associated with each of the plurality of small areas. Resetting means for resetting a count value associated with a small area that is determined not to be the primary candidate small area for the current image among the count values; A secondary candidate small region discriminating unit for discriminating a secondary candidate small region which is a small region having an associated count value equal to or larger than a predetermined value among the plurality of small regions after the increment by the increment means; ,
The primary candidate small area determining means includes correlation value determining means for determining whether or not the correlation value of each small area obtained by the first correlation value calculating means for the current image is greater than or equal to a predetermined value. And a means for obtaining a difference image between the current image and the background image, in the difference image among the small areas determined by the correlation value determination means that the correlation value is equal to or greater than the predetermined value. A small area including a pixel having a difference value equal to or greater than a predetermined value is determined as the primary candidate small area;
A falling object detection device on the road. The road fall object candidate area acquisition means has a labeling means for labeling the secondary candidate small area, and obtains the labeled area as the road fall object candidate area. Road falling object detection apparatus in any one. The on-road falling object candidate area acquisition means includes a labeling means for labeling the secondary candidate small area, and a size determination means for determining whether or not the size of the labeled area is greater than or equal to a predetermined size. 5. The road falling according to claim 1, wherein a labeling area that is determined by the size determination means to be equal to or larger than the predetermined size is obtained as the road falling object candidate area. Object detection device. Third correlation value calculation means for obtaining a correlation value between the current image and the background image for the falling object candidate area currently registered as the detection processing target;
Means for deleting from the registration as the detection processing target the road fall object candidate area in which the correlation value obtained by the third correlation value calculating means is equal to or greater than a predetermined value;
The on-road falling object detection device according to claim 5 or 6. The road fall object detection device according to any one of claims 1 to 4, wherein the road fall object candidate area obtaining unit obtains the secondary candidate small area as the road fall object candidate area. Imaging means for imaging a monitoring area including a road area;
First correlation value calculation for obtaining a correlation value between a sequentially sampled image obtained from the imaging means and sequentially sampled for each of a plurality of small areas obtained by dividing an image area corresponding to the monitoring area Means,
Based on the correlation values of the small regions obtained by the first correlation value calculation means for the current image that is the latest sampled image and the image sampled in the past. A road fall object candidate area obtaining means for obtaining a road fall object candidate area that is a candidate for a road fall object in relation to the current image;
Means for registering the falling object candidate area on the road obtained in relation to the current image as a detection processing target;
Second correlation value calculation means for obtaining a correlation value between the current image and the previously sampled image for the road falling object candidate region currently registered as the detection processing target;
Means for deleting the road fall object candidate area whose correlation value obtained by the second correlation value calculating means is not more than a predetermined value from registration as the detection processing target;
When the value corresponding to the time registered as the detection processing target of at least one of the fallen object candidate regions currently registered as the detection processing target indicates a predetermined time or more Determining means for determining that there is a fallen object on the road;
With
The on-road falling object candidate area acquisition means includes: (a) the plurality of small areas for the current image based on the correlation values of the small areas obtained by the first correlation value calculation means for the current image; Primary candidate small region determination means for determining whether or not each is a primary candidate small region that is a small region that is a primary candidate for the fallen object on the road, and (b) is associated with each of the plurality of small regions. And (c) an increment means for incrementing a count value associated with the small area determined to be the primary candidate small area for the current image, and (c) associated with each of the plurality of small areas. Resetting means for resetting a count value associated with a small area that is determined not to be the primary candidate small area for the current image among the count values; A secondary candidate small region discriminating unit for discriminating a secondary candidate small region which is a small region having an associated count value equal to or larger than a predetermined value among the plurality of small regions after the increment by the increment means; ,
The on-road falling object candidate area acquisition means has a labeling means for labeling the secondary candidate small area, and obtains the labeled area as the on-road falling object candidate area,
Third correlation value calculation means for obtaining a correlation value between the current image and the background image for the falling object candidate area currently registered as the detection processing target;
Means for deleting from the registration as the detection processing target the road fall object candidate area in which the correlation value obtained by the third correlation value calculating means is equal to or greater than a predetermined value;
Further comprising
A falling object detection device on the road. Imaging means for imaging a monitoring area including a road area;
First correlation value calculation for obtaining a correlation value between a sequentially sampled image obtained from the imaging means and sequentially sampled for each of a plurality of small areas obtained by dividing an image area corresponding to the monitoring area Means,
Based on the correlation value of each of the small regions obtained by the first correlation value calculation means for the current image that is the latest sampled image and the image sampled in the past, the current image is related to the current image. Road falling object candidate area acquisition means for obtaining a road falling object candidate area that is a candidate for road falling object,
Means for registering the falling object candidate area on the road obtained in relation to the current image as a detection processing target;
Second correlation value calculation means for obtaining a correlation value between the current image and the previously sampled image for the road falling object candidate region currently registered as the detection processing target;
Means for deleting the road fall object candidate area whose correlation value obtained by the second correlation value calculating means is not more than a predetermined value from registration as the detection processing target;
When the value corresponding to the time registered as the detection processing target of at least one of the fallen object candidate regions currently registered as the detection processing target indicates a predetermined time or more Determining means for determining that there is a fallen object on the road;
With
The on-road falling object candidate area obtaining means (a) calculates the first correlation value for the current image. Based on the correlation value of each of the small areas obtained by the means, each of the plurality of small areas for the current image is a primary candidate small area that is a small area that is a primary candidate for the falling object on the road. Primary candidate small region determining means for determining whether or not, and (b) out of the count values associated with each of the plurality of small regions, the small region determined to be the primary candidate small region for the current image Incrementing means for incrementing the associated count value; and (c) out of the count values associated with each of the plurality of small areas, a small area that is determined not to be the primary candidate small area for the current image. Reset means for resetting the associated count value; and (d) after the increment by the increment means, the associated count value is associated with the associated count value. It said count value has a secondary candidate small area determining means for determining secondary candidate small area is a small area of more than a predetermined value,
The on-road falling object candidate area acquisition means includes a labeling means for labeling the secondary candidate small area, and a size determination means for determining whether or not the size of the labeled area is greater than or equal to a predetermined size. Having a labeling area determined by the size determination means as being equal to or larger than the predetermined size as the road fall object candidate area,
Third correlation value calculation means for obtaining a correlation value between the current image and the background image for the falling object candidate area currently registered as the detection processing target;
Means for deleting from the registration as the detection processing target the road fall object candidate area in which the correlation value obtained by the third correlation value calculating means is equal to or greater than a predetermined value;
Further comprising
A falling object detection device on the road. The primary candidate small area determining means includes correlation value determining means for determining whether or not the correlation value of each small area obtained by the first correlation value calculating means for the current image is greater than or equal to a predetermined value. 11. The road fall according to claim 9, further comprising: determining, as the primary candidate small region, a small region in which the correlation value is determined to be equal to or greater than the predetermined value by the correlation value determining unit. Object detection device. The road falling object detection apparatus according to any one of claims 1 to 11 , further comprising updating means for updating the background image under a predetermined condition based on the sampled image.

Images