JP3679958B2 - License plate recognition device - Google Patents

Description

ここで、abs()は絶対値、&&は論理積、||は論理和をそれぞれ意味する。また、lag₁,lag_２,lag_３,は、ずらし量を意味するパラメータで、画像上でのプレートの大きさによって決定されるが、大体５〜９の値が適切である。θは２値化閾値であり、１５〜２０位が適当である。
【００２１】
（２）上記（１）の処理を、全てのｍ（０≦ｍ≦Ｘ／ｋ₁ −１）について行なう。
（３）上記（１）、（２）の処理を、全てのｎ（０≦ｎ≦Ｙ／ｋ₂ −１）について行なう。
上記疑似ずらし相関画像生成処理により、図４（ａ）に示す縮小画像から同図（ｂ）に示すような疑似ずらし相関画像が得られる。
【００２２】
３．疑似論理和画像生成処理（ステップＢ３）
（１）次式により、点（ｍ，ｎ）における疑似論理和画像Ｉ_or（ｍ，ｎ）を生成する。

上式中の各記号並びにパラメータの意味は、前記疑似ずらし相関画像生成処理の各記号、パラメータと同一であり、用いるパラメータ値も同一のものとする。
【００２３】
（２）上記（１）の処理を、全てのｍ（０≦ｍ≦Ｘ／ｋ₁ −１）について行なう。
（３）上記（１）、（２）の処理を、全てのｎ（０≦ｎ≦Ｙ／ｋ₂ −１）について行なう。
上記疑似論理和画像生成処理により、図５（ａ）に示す縮小画像から同図（ｂ）に示すような疑似論理和画像が得られる。
【００２４】
４．プレート候補領域探索処理（ステップＢ４）
（１）小領域毎の２値化面積計算
ステップＢ２により得られた疑似ずらし相関画像を、図６に示すような複数の小領域に分割し、小領域の中で２値化されている部分（＝輝度値が１に設定されている画素）の面積を各小領域毎に計算する。１小領域当りのｘ及びｙ方向サイズは、画面上のプレートｘ，ｙサイズの半分程度が目安である。
【００２５】
（２）プレート候補小領域選択
以下の手順により、プレートの前部あるいは一部を含んでいるプレート候補小領域を選択する。
i．各小領域の面積の大きい順にソートする。
ii．ソートした各領域の上位Ｎ_cand個について、
ａ．（第ｎ位の小領域の値）／（第１位の小領域の値）＞ｒ₁
ｂ．（選択された候補数）＜Ｎ_select
の何れかを満たす小領域を全て選択する。ここで、Ｎ_candは５０個、Ｎ_selectは２０個位が適当である。また、ｒ₁ の値は、０．５程度が適当である。
【００２６】
iii．上記（２）で選択したプレート候補小領域を、画像上で下から順に（＝ｙ座標の大きい順に）なるようにソートする。
【００２７】
５．プレート切り出し処理（ステップＢ５）
上記プレート候補領域探索処理で求めたプレート候補小領域について、１位の候補領域から順に以下の手順によってプレート領域としての適合性をチェックし、プレート領域の切り出しを行なう。
【００２８】
（１）周辺小領域マージ処理
以下の手順により、注目するプレート候補小領域の周辺の小領域をマージする。
i．１位候補領域から順に、注目するプレート小領域を１つ選ぶ。
ii．注目するプレート小領域に隣接する小領域のうち、上記プレート候補領域探索処理においてプレート候補小領域として選ばれたものがあるかどうかを調べる。
【００２９】
iii．上記ii．で該当する隣接小領域が見つかった場合、以下に示すａ、ｂの条件を満たすかどうかを調べて、満たす場合にはこれをマージし、注目するプレート小領域のｘ，ｙ方向範囲を図７に示すように修正する。マージされたプレート小領域は以後無効とし、処理対象としない。
ａ．ｘ方向の通算マージ回数がＮ_merge 回以下である（Ｎ_merge は４程度）。
ｂ．他の小領域によってマージされていない。
【００３０】
iv．上記ii．、iii．をマージできる隣接小領域がなくなるまで繰り返す。
【００３１】
（２）プレート領域仮設定処理
上記（１）の処理によって得られた、周辺小領域をマージしたプレート候補小領域に対して左・右・上・下それぞれに関して余裕を見て（１０〜１５画素程度）、プレート領域の仮設定を行なう。
【００３２】
（３）ｘ方向領域設定処理
仮設定された上記プレートｘ領域について、以下の手順によりプレートｘ方向の領域設定を行なう。
i．疑似論理和画像上のプレート仮設定領域に対して、ｘ軸への射影ヒストグラムを求める。
ii．上記i．で求めたｘ軸射影ヒストグラムデータに対して、移動平均長ｌmaxの移動平均処理を行なう。移動平均長ｌmaxは、５０〜５５程度に設定する。
【００３３】
iii．以下の手順により、移動平均処理を施したｘ軸射影ヒストグラムデータのピーク点及び移動平均データの最大値を求める。
Ａ．移動平均処理を施したｘ軸射影ヒストグラムの値が最大となるｘ座標、及びその値を求め、その求めた最大値を「max_x_hist」とする。
Ｂ．以下の計算式によって、参照値「ref_x_hist」を計算する。
ref_x_hist ＝ max_x_hist * r₂
r₂ は、０．８程度の値が適当である。
Ｃ．図８に示すように移動平均処理を施したｘ軸射影ヒストグラムデータを左右から調べて、その値が「ref_x_hist」を初めて越えるｘ座標を左右各々について求め、２点の中点ｘ_c を計算する。
Ｄ．移動平均処理を施したｘ軸射影ヒストグラムデータの重心ｘ_g を計算する。
Ｅ．図９に示すように上記中点ｘ_c と重心ｘ_g を比較して、ヒストグラムデータ全体の中心座標により近い方をｘ軸射影ヒストグラムデータのピーク点とする。図９では、中点ｘ_c をピーク点として選択した状態を示している。
【００３４】
vi．プレート左右端探索処理
以下の手順により、ｘ軸射影ヒストグラムデータと、iii．で求めたピーク点ｘ座標及び移動平均データの最大値「max_x_hist」を用いて、プレート左右端座標を求める。
Ａ．以下の計算式によって、参照値「ref_hist」を計算する。
ref_hist＝max_x_hist * r₃
r₃ は、０．２程度の値が適当である。
Ｂ．図１０に示すように、以下の範囲において、ｘ軸射影ヒストグラムデータをピーク点に近い方から調べ、その値が初めて参照値「ref_hist」以下となるところを求めてプレート左右端とする。
左端：（ピーク点）− x_srch_area1〜（ピーク点）− x_srch_area2
右端：（ピーク点）＋ x_srch_area2〜（ピーク点）＋ x_srch_area1
上記「x_srch_area1」は７５〜８０、「x_srch_area2」は４５〜５０程度の値が適当である。
【００３５】
（４）ｙ方向領域設定処理
仮設定されたプレートｙ領域について、以下の処理によりプレートｙ方向の領域設定を行なう。
i．疑似論理和画像上のプレート仮設定領域に対して、ｙ軸への射影ヒストグラムを求める。
ii．上記i．で求めたｙ軸射影ヒストグラムデータに対して、移動平均長ｌ_may の移動平均処理を行なう。移動平均長ｌ_may は、１５〜２０程度に設定する。
【００３６】
iii．以下の手順によりプレート上下端座標を求める。
Ａ．移動平均処理を施したｙ軸射影ヒストグラムデータの最大値「max_y_hist」を求め、以下の計算式によって、参照値「ref_hist」を計算する。
ref_hist＝max_y_hist * r₄
r₄ は、０．２程度の値が適当である。
Ｂ．図１１に示すように、移動平均処理を施したｙ軸射影ヒストグラムデータを上下端から探索して、初めて参照値「ref_hist」以上となる座標をそれぞれについて求め、プレート上下端とする。但し、上下端は、上端→下端の順に求め、下端を求める際には上端からの距離（＝高さ）がＨ_plate より小さくならないように座標を設定する。Ｈ_plate は、３５〜４０程度の値が適当である。
【００３７】
（５）ギャップラインチェック処理
ｘ，ｙ方向設定処理で設定したプレート領域に対して、ギャップライン（ナンバープレートの大文字と小文字の間にある、水平方向のすき間）のチェックを行なう。
i．ギャップライン捕捉
疑似論理和画像を用いて、設定したプレート領域に対して、ギャップライン捕捉する。
Ａ．疑似論理和画像上のプレート設定領域において、ｙ軸射影ヒストグラム求め、その求めたヒストグラムデータ列を「proj_hist［］」とする。
Ｂ．設定したプレート領域において、各横ライン毎に最大ゼロラン長（２値化画像で“０”が連続して続くところ）となる領域を求め、その求めた最大ゼロラン長データ列を「zero_run［］」とする。
【００３８】
Ｃ．以下の計算式により、ギャップライン探索範囲を設定する。
（探索開始点）＝（始点ｙ座標）＋（マージン）
（探索終端点）＝（始点ｙ座標）＋（プレートｙサイズ）×r₅ −（マージン）
r₅ は、０．７、マージンは５〜１０程度の値が適当である。
【００３９】
Ｄ．上記Ｃ．で求めたギャップライン探索範囲において、ヒストグラムデータ列「proj_hist［］」及び最大ゼロラン長データ列「zero_run［］」を用いて、以下の条件に最も良く当てはまるｙ座標を求め、ギャップラインｙ座標とする。
【００４０】
ａ．最大ゼロラン長の長い行がｙ方向に数行（２，３行程度）連続する。
ｂ．ギャップライン領域にｙ軸射影ヒストグラムの小さい部分が現れる。
ｃ．ギャップライン領域の上下数行に、ｙ軸射影の大きい部分が現れる。
上記のようにして求めたギャップラインｙ座標をｙ_gap とする。
【００４１】
ii．ギャップライン評価
上記の処理で求めたギャップライン座標ｙ_gap 、及びｘ，ｙ方向プレート領域の情報を元に、ナンバープレートのギャップラインとして正当であるかどうかを評価する。
［始点ｙ座標−ギャップラインｙ座標間の大きさチェック］
始点ｙ座標−ギャップラインｙ座標間の大きさがｈ₁ より大きい場合は、ナンバープレートでないと判定する。ｈ₁ は２０程度の値が適当である。
［ギャップライン、小文字部高さチェック］
Ａ．プレート領域において、各ｙ座標毎に（プレート幅−ｙ軸射影ヒストグラム値）を計算し、求めたデータ列を「sub_data［］」とする。
【００４２】
Ｂ．次式により、参照ゼロ長さ「ref_zero_len」を計算する。
ref_zero_len＝sub_data［y_gap］* r₆
r₆ は、０．８〜０．８５程度の値が適当である。
【００４３】
Ｃ．ｙ軸射影ヒストグラムデータと「ref_zero_len」とを用いて、
gap_h（ギャップライン高さ）：ギャップライン近傍で、「sub_data［］」が連続して「ref_zero_len」以上となっている高さ
zero_h（ゼロ領域高さ）：ギャップライン上側で、「sub_data［］」が「ref_zero_len」以上となっている高さ
char_h（小文字部高さ）：ギャップライン上側で、「sub_data［］」が「ref_zero_len」以下となっている高さ
をそれぞれ求める。
【００４４】
Ｄ．以下の何れかの条件に当てはまる場合は、ナンバープレートでないと判定する。
gap_h ＞ max_gap_h
char_h ＜ zero_h
max_gap_h は、４〜５程度の値が適当である。
【００４５】
［ギャップライン位置、長さチェック］
以下の何れかの条件に当てはまる場合は、ナンバープレートでないと判定する。
y_gap ＜＝（プレート高さ）＊ r₇ ＋（プレート始点ｙ座標）
y_gap ＞＝（プレート高さ）＊ r₈ ＋（プレート始点ｙ座標）
（ギャップライン長さ）＜＝min_gap_len
r₇ は０．２程度、r₈ は０．５程度、「min_gap_len」は２０〜２５程度の値がそれぞれ適当である。
以上の全てのチェックにパスした場合は、切り出した領域はナンバープレートであると判定する。
【００４６】
６．２位候補以下のプレート切り出し処理
上記１．〜５．の処理を、以下の何れかの条件に当てはまるまで続ける。
【００４７】
ａ．全てのプレート候補小領域をチェックし終える。
ｂ．１．〜５．の処理をＮ_try 回実行した後で、１つ以上のプレート領域が切り出されている（Ｎ_try は５〜８程度）。
【００４８】
７．プレート領域ソート
上記１．〜６．までの処理の結果、複数のプレート領域が見つかった場合は、終点ｙ座標の大きい順（＝下にあるプレート領域から順）になるようにソートする。
【００４９】
次に本発明に係るナンバープレート切り出し処理と、従来手法との比較について説明する。
本発明における疑似ずらし相関画像は、従来手法の正・負高周波２値画像のずらし相関画像とほぼ同様の、プレート領域“１”、それ以外が“０”となるような２値化画像となっている。
また、疑似ずらし相関画像と疑似論理和画像を用いた小領域毎の２値化領域面積によるプレート候補領域の選定処理によって、従来手法のずらし相関画像に対する２次元テンプレートマッチング処理による、マッチング度の高い領域を求める処理とほぼ同等の効果を得ることができる。
従って、本発明に係るナンバープレート切り出し処理によるナンバープレート切り出し性能は、従来手法による切り出し性能とほぼ同等であると言える。
【００５０】
そして、本発明に係るナンバープレート切り出し処理によれば、従来手法に比べてフィルタリング処理が非常に少なく、加算、乗算の演算量が大幅に減るため、汎用の演算装置でも高速にプレート切り出し処理を行なうことが可能である。また、本発明では、使用する演算装置のアーキテクチャに依存しない汎用の言語のみでアルゴリズムを記述できるものであり、移植性が高い。更に、専用の演算装置を開発する必要がなく、従来に比べて開発期間の短縮及びコストの低減化が可能である。
なお、本発明によるナンバープレート認識装置は、有料道路の料金収受処理において進入車両のナンバープレートを認識する場合だけでなく、その他、例えば駐車場において車両のナンバープレートを認識する場合等に適用し得るものである。
【００５１】
【発明の効果】
以上詳記したように本発明によれば、専用の演算装置を新規に開発することなく、汎用の演算装置を用いて高速かつ高精度にプレート領域を切り出すことができ、開発期間の短縮及びコストの低減化を図ることができる。
【図面の簡単な説明】
【図１】本発明の実施の形態において対象とするディジタル画像の座標系を定義した図。
【図２】本発明の実施の形態に係るナンバープレート認識装置の構成を示す図。
【図３】同実施の形態におけるナンバープレート切り出し処理の手順を示すフローチャート。
【図４】同実施の形態における疑似ずらし相関画像の例を示す図。
【図５】同実施の形態における疑似論理和画像の例を示す図。
【図６】同実施の形態における疑似ずらし相関画像上の小領域毎の２値化面積計算処理を説明する図。
【図７】同実施の形態における疑似ずらし相関画像上の小領域のマージ処理を説明する図。
【図８】同実施の形態におけるｘ方向領域設定処理の一処理を説明する図。
【図９】同実施の形態におけるｘ方向領域設定処理の一処理を説明する図。
【図１０】同実施の形態におけるプレート左右端設定処理を説明する図。
【図１１】同実施の形態におけるプレート上下端設定処理を説明する図。
【図１２】従来例に係るナンバープレート認識装置における認識処理手順を示すフローチャート。
【符号の説明】
１…車両撮像装置
２…画像入力装置
３…演算装置[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a license plate recognition device that recognizes a license plate of a vehicle, and more particularly to a technology for cutting out a license plate portion.
[0002]
[Prior art]
Conventionally, in a toll road toll collection system, a license plate recognition device is installed at a toll gate to determine the type of a passing vehicle, and a vehicle imaging device such as a TV camera is used to image the vehicle front including the vehicle license plate. The captured image is processed to recognize the license plate.
[0003]
FIG. 12 is a flowchart showing a recognition processing procedure in a conventional license plate recognition apparatus. As shown in FIG. 12, the license plate recognition process includes a pre-process for removing signal noise and the like on the input vehicle image (step S1), and a plate cut-out process for cutting out a license plate portion from the vehicle image (step S2). The character cutout process (step S3) for cutting out individual character areas from the plate area, and the character recognition process (step S4) for identifying the cutout character as the most appropriate character type are mainly comprised of four processes.
[0004]
In the license plate recognition process, as a typical example of the conventional method related to the plate cut-out process shown in step S2, as shown below,
1. Generation of a horizontal moving average filter image for the original image.
[0005]
2. Generation of a difference image (positive high frequency image = original image−moving average filter image, negative high frequency image = moving average filter image−original image) between the horizontal moving average filter image and the original image.
[0006]
3. Generation of a binary image (positive / negative high-frequency binary image) of the two-difference image (positive / negative high-frequency image).
[0007]
4). Generation of a shifted correlation image of the positive and negative high-frequency binary images.
[0008]
5. A two-dimensional template matching process for the shifted correlation image.
[0009]
6). In the above two-dimensional template matching process, a region with a high degree of matching is selected as a plate region.
[0010]
In some cases, a plate cut-out process is performed through the above process.
[0011]
[Problems to be solved by the invention]
However, in the plate cut-out process, addition and multiplication are repeatedly performed many times in processes involving “filtering process” such as a process for generating a moving average filter image and a two-dimensional template matching process for a shifted correlation image. For this reason, in order to perform recognition processing at the speed required as a product in the license plate recognition device, it is necessary to use a dedicated arithmetic device, and it takes a lot of time and cost to develop and manufacture the product. was there.
[0012]
In order to solve the above problem, as a license plate recognition device capable of cutting out a plate area at high speed using a general-purpose arithmetic device, the present applicant previously applied for a patent in Japanese Patent Application No. 10-148618. There is. The license plate cut-out process in this license plate recognition device is such that the brightness image changes from dark to bright on the edge image (horizontal second-order differential binarized image) used for plate search, such as road surface unevenness and vehicle front grille. There is a problem that a portion that is continuously generated at a high cycle is easily binarized and may be cut out as a plate region by mistake.
[0013]
The present invention has been made to solve the above-described problems, and can be constructed at low cost by using a general-purpose arithmetic device without newly developing a dedicated arithmetic device, and at high speed and high accuracy. It is an object to provide a license plate recognition device that can be cut out.
[0014]
[Means for Solving the Problems]
The license plate recognition device according to the present invention includes a reduction unit that reduces a captured vehicle image in a horizontal direction and a vertical direction, and an image reduced by the reduction unit, based on a value of each pixel in a positive or negative direction from the pixel. The pixel value at a position shifted by a predetermined amount is subtracted, and the subtraction result is compared with a binarization threshold value to be binarized, and the logical product of the value shifted in the positive direction and the value shifted in the negative direction A pseudo-shift correlation image generation unit that generates a pseudo-shift correlation image by calculation, and a pixel value at a position shifted by a predetermined amount in the positive or negative direction from the value of each pixel with respect to the image reduced by the reduction unit. Subtraction is performed, and the subtraction result is compared with a binarization threshold value to be binarized, and a pseudo-OR image is generated by a logical OR operation of the value shifted in the positive direction and the value shifted in the negative direction. OR image generation means and The pseudo-shift correlation image generated by the pseudo-shift correlation image generation unit is divided into a plurality of small areas, and the area of a pixel whose luminance value is set to “1” in each small area is calculated, License plate candidate area searching means for selecting a license plate candidate small area based on the area value of the small area, and the pseudo-OR image generating means based on the license plate candidate small area obtained by the license plate candidate area searching means A license plate section for temporarily setting a license plate area on the pseudo-OR image generated by the above, checking the suitability of the license plate area as the license plate area, and cutting out the license plate area. Features.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, a license plate recognition apparatus according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram defining a coordinate system of a target digital image in the present embodiment. In the following description, as shown in FIG. 1, a coordinate system is used in which the origin of the target digital image I is taken at the upper left corner, the x-axis is taken to the right in the horizontal direction, and the y-axis is taken vertically downward. In addition, the luminance value of the point (i, j) on the image I is represented as I (i, j). In addition, it is assumed that the vehicle image including the target license plate is captured at a resolution necessary for recognizing each character of the license plate. First, the positioning of the license plate cutout process in the license plate recognition apparatus will be described.
[0016]
FIG. 2 is a diagram showing a configuration of the license plate recognition apparatus. In FIG. 2, reference numeral 1 denotes a vehicle image pickup device such as a TV camera, which picks up an image of a front portion including a license plate of a vehicle that has entered a toll gate, for example, and inputs the image (analog signal) to the image input device 2. The image input device 2 converts an analog image signal picked up by the vehicle image pickup device 1 into a digital image signal and inputs the digital image signal to the arithmetic device 3.
[0017]
When the vehicle image is input from the image input device 2, the arithmetic device 3 performs pre-processing (step A 1) for removing signal noise from the image, and license plate cut-out processing (step A 1) for cutting out a license plate portion from the image. A2), a character cutout process (step A3) for cutting out individual character areas from the plate area, and a character recognition process (step A4) for identifying the cutout character as the most appropriate character type are sequentially executed.
[0018]
The license plate cutout process according to the present invention shown in step A2 is executed according to the flowchart shown in FIG. This license plate cut-out processing includes input image reduction processing (step B1), pseudo-shift correlation image generation processing (step B2), pseudo-OR image generation processing (step B3), plate candidate area search processing (step B4), plate cut-out. It consists of processing (step B5).
[0019]
Next, specific processing of each of the above steps B1 to B5 will be described.
1. Input image reduction processing (step B1)
In this processing, the captured vehicle image is reduced to 1 / k ₁ and 1 / k ₂ in x and y (vertical and horizontal), respectively. The reduction of the image is performed by taking out each pixel of the vehicle image by skipping (k ₁ −1) pixels and (k ₂ −1) pixels for x and y, respectively. If the original image is I, the reduced image is I _sub , the horizontal size of the original image is X, and the vertical size is Y,
I _sub (m, n) = I (k ₁ m, k ₂ n)
However, 0 ≦ m ≦ X / k ₁ −1, 0 ≦ n ≦ Y / k ₂ −1
And then, when applicable, as the possible values of k _1, k _2, both about 1, 2, and 3 is appropriate. In addition, the specific value of each parameter illustrated in the following description is based on k ₁ = k ₂ = 2.
[0020]
2. Pseudo-shifted correlation image generation process (step B2)
(1) A pseudo shifted correlation image I _cor (m, n) at the point (m, n) is generated by the following equation.

Here, abs () means an absolute value, && means a logical product, and || means a logical sum. Also, lag ₁ , lag ₂ , lag ₃ , are parameters that mean the shift amount, and are determined by the size of the plate on the image, but values of about 5 to 9 are appropriate. θ is a binarization threshold value, and 15 to 20 is appropriate.
[0021]
(2) The process of (1) is performed for all m (0 ≦ m ≦ X / k ₁ −1).
(3) The processes (1) and (2) are performed for all n (0 ≦ n ≦ Y / k ₂ −1).
By the pseudo shifted correlation image generation process, a pseudo shifted correlation image as shown in FIG. 4B is obtained from the reduced image shown in FIG.
[0022]
3. Pseudo OR image generation process (step B3)
(1) Generate a pseudo-OR image I _or (m, n) at the point (m, n) by the following equation.

The meanings of the symbols and parameters in the above equation are the same as the symbols and parameters of the pseudo-shift correlation image generation process, and the parameter values used are also the same.
[0023]
(2) The process of (1) is performed for all m (0 ≦ m ≦ X / k ₁ −1).
(3) The processes (1) and (2) are performed for all n (0 ≦ n ≦ Y / k ₂ −1).
By the pseudo-OR image generation process, a pseudo-OR image as shown in FIG. 5B is obtained from the reduced image shown in FIG.
[0024]
4). Plate candidate area search process (step B4)
(1) A portion that is obtained by dividing the pseudo shifted correlation image obtained by the binarized area calculation step B2 for each small region into a plurality of small regions as shown in FIG. The area of (= pixel whose luminance value is set to 1) is calculated for each small region. As a guideline, the size in the x and y directions per small area is about half of the size of the plate x and y on the screen.
[0025]
(2) Selection of plate candidate small region The plate candidate small region including the front or part of the plate is selected by the following procedure.
i. Sort in ascending order of area of each small area.
ii. For the top N _{cand of} each sorted area,
a. (Value of nth subregion) / (value of first subregion)> r ₁
b. (Number of selected candidates) <N _select
All small regions satisfying any of the above are selected. Here, it is appropriate that N _cand is 50 and N _select is about 20. Further, the value of r ₁ is suitably about 0.5.
[0026]
iii. The plate candidate small areas selected in the above (2) are sorted on the image in order from the bottom (= y coordinate in descending order).
[0027]
5. Plate cutting process (step B5)
The plate candidate small areas obtained by the above-described plate candidate area search process are checked for suitability as a plate area by the following procedure in order from the first candidate area, and the plate area is cut out.
[0028]
(1) Peripheral small region merging process The small regions around the target plate candidate small region are merged by the following procedure.
i. In order from the first candidate area, one plate small area of interest is selected.
ii. It is checked whether there is a small region adjacent to the target small plate region that has been selected as a candidate plate small region in the plate candidate region search process.
[0029]
iii. Ii. If a corresponding adjacent small region is found in step (b), it is checked whether or not the following conditions a and b are satisfied. If they are satisfied, these are merged, and the x and y direction ranges of the target plate small region are shown in FIG. Modify as shown in. The merged small plate area is invalidated and not processed.
a. The total number of merges in the x direction is N _merge times or less (N _merge is about 4).
b. Not merged by other subregions.
[0030]
iv. Ii. Iii. Repeat until there are no adjacent small areas that can be merged.
[0031]
(2) Temporary setting process for plate area With respect to the left, right, upper, and lower margins for the plate candidate small areas obtained by merging the peripheral small areas obtained by the process (1) above (10 to 15 pixels) About)), the plate area is temporarily set.
[0032]
(3) X-direction area setting process For the plate x area that has been provisionally set, the area in the plate x direction is set according to the following procedure.
i. A projection histogram on the x-axis is obtained for the temporary plate setting area on the pseudo-OR image.
ii. I. The moving average processing of the moving average length lmax is performed on the x-axis projection histogram data obtained in step (1). The moving average length lmax is set to about 50 to 55.
[0033]
iii. According to the following procedure, the peak point of the x-axis projection histogram data subjected to the moving average process and the maximum value of the moving average data are obtained.
A. The x coordinate at which the value of the x-axis projection histogram subjected to the moving average processing is maximized and its value are obtained, and the obtained maximum value is set as “max_x_hist”.
B. The reference value “ref_x_hist” is calculated by the following calculation formula.
ref_x_hist = max_x_hist * r ₂
r ₂ is a suitable value of about 0.8.
C. As shown in FIG. 8, the x-axis projection histogram data subjected to the moving average process is examined from the left and right, and the x coordinate whose value exceeds “ref_x_hist” for the first time is obtained for each of the left and right, and the midpoint x _{c of the} two points is calculated. .
D. The centroid _xg of the x-axis projection histogram data subjected to the moving average process is calculated.
E. As shown in FIG. 9, the midpoint _xc and the center of gravity _xg are compared, and the one closer to the center coordinates of the entire histogram data is set as the peak point of the x-axis projection histogram data. 9 shows a state in which selects the midpoint x _c as a peak point.
[0034]
vi. Plate left and right edge search processing x-axis projection histogram data according to the following procedure, and iii. The plate left and right end coordinates are obtained using the peak point x coordinate obtained in step 1 and the maximum value “max_x_hist” of the moving average data.
A. The reference value “ref_hist” is calculated by the following calculation formula.
ref_hist = max_x_hist * r ₃
r ₃ is suitably a value of about 0.2.
B. As shown in FIG. 10, in the following range, the x-axis projection histogram data is examined from the side closer to the peak point, and the position where the value is below the reference value “ref_hist” for the first time is obtained and set as the left and right edges of the plate.
Left edge: (Peak point)-x_srch_area1 ~ (Peak point)-x_srch_area2
Right edge: (peak point) + x_srch_area2 to (peak point) + x_srch_area1
The value of “x_srch_area1” is appropriately 75 to 80, and the value of “x_srch_area2” is approximately 45 to 50.
[0035]
(4) Y-direction area setting process For the temporarily set plate y area, area setting in the plate y direction is performed by the following process.
i. A projection histogram on the y-axis is obtained for the temporary plate setting area on the pseudo-OR image.
ii. I. The moving average process of the moving average length l _may is performed on the y-axis projection histogram data obtained in (1). The moving average length l _may is set to about 15-20.
[0036]
iii. The upper and lower end coordinates of the plate are obtained by the following procedure.
A. The maximum value “max_y_hist” of the y-axis projection histogram data subjected to the moving average process is obtained, and the reference value “ref_hist” is calculated by the following calculation formula.
ref_hist = max_y_hist * r ₄
r ₄ is suitably a value of about 0.2.
B. As shown in FIG. 11, the y-axis projection histogram data subjected to the moving average process is searched from the upper and lower ends, and the coordinates that are equal to or higher than the reference value “ref_hist” are obtained for the first time to be the upper and lower ends of the plate. However, the upper and lower ends are obtained in the order of upper end → lower end, and when obtaining the lower end, coordinates are set so that the distance from the upper end (= height) is not smaller than H _plate . A value of about 35 to 40 is appropriate for H _plate .
[0037]
(5) Gap line check process The gap line (horizontal gap between upper and lower case letters of the license plate) is checked for the plate area set in the x and y direction setting processes.
i. Gap line capture is performed on the set plate area using the gap line capture pseudo-OR image.
A. In the plate setting area on the pseudo-OR image, a y-axis projection histogram is obtained, and the obtained histogram data string is set to “proj_hist []”.
B. In the set plate area, an area having a maximum zero run length (a place where “0” continues in the binarized image continues) is obtained for each horizontal line, and the obtained maximum zero run length data string is “zero_run []”. And
[0038]
C. The gap line search range is set by the following formula.
(Search start point) = (start point y coordinate) + (margin)
(Search end point) = (start point y coordinate) + (plate y size) × r ₅ − (margin)
It is appropriate that r ₅ is 0.7 and the margin is about 5 to 10.
[0039]
D. C. above. Using the histogram data string “proj_hist []” and the maximum zero-run length data string “zero_run []” in the gap line search range obtained in step 1, the y coordinate that best meets the following conditions is obtained and used as the gap line y coordinate. .
[0040]
a. A long line with a maximum zero run length continues several lines (about a few lines) in the y direction.
b. A small portion of the y-axis projection histogram appears in the gap line region.
c. A large portion of the y-axis projection appears in several lines above and below the gap line region.
The gap line y coordinate obtained as described above is _defined as y _gap .
[0041]
ii. Gap Line Evaluation Based on the gap line coordinates y _gap obtained in the above processing and the information on the x and y direction plate regions, it is evaluated whether or not it is valid as a license plate gap line.
[Size check between start point y coordinate and gap line y coordinate]
If the size between the starting point y coordinate and the gap line y coordinate is larger than h _1, it is determined that it is not a license plate. A value of about 20 is appropriate for h ₁ .
[Gap line, lower case height check]
A. In the plate area, (plate width-y-axis projection histogram value) is calculated for each y coordinate, and the obtained data string is defined as “sub_data []”.
[0042]
B. The reference zero length “ref_zero_len” is calculated by the following equation.
ref_zero_len = sub_data [y _gap ] * r ₆
r _6, the value of the order of 0.8 to 0.85 is appropriate.
[0043]
C. Using y-axis projection histogram data and “ref_zero_len”,
gap_h (gap line height): height where “sub_data []” is continuously “ref_zero_len” or more near the gap line
zero_h (zero area height): The height above the gap line where “sub_data []” is greater than or equal to “ref_zero_len”
char_h (lowercase part height): Each height above which “sub_data []” is equal to or lower than “ref_zero_len” is obtained above the gap line.
[0044]
D. If any of the following conditions is met, it is determined that the license plate is not used.
gap_h> max_gap_h
char_h <zero_h
A value of about 4 to 5 is appropriate for max_gap_h.
[0045]
[Gap line position and length check]
If any of the following conditions is met, it is determined that the license plate is not used.
y _gap <= (plate height) * r ₇ + (plate start point y coordinate)
y _gap > = (plate height) * r ₈ + (plate start point y coordinate)
(Gap line length) <= min_gap_len
It is appropriate that r ₇ is about 0.2, r ₈ is about 0.5, and “min_gap_len” is about 20 to 25.
When all the above checks are passed, it is determined that the cut-out area is a license plate.
[0046]
6. Plate cut-out process below the second candidate ~ 5. This process is continued until one of the following conditions is satisfied.
[0047]
a. Finish checking all the plate candidate subregions.
b. 1. ~ 5. After executing the above process N _try times, one or more plate regions are cut out (N _try is about 5 to 8).
[0048]
7. Plate area sorting ~ 6. If a plurality of plate areas are found as a result of the processes up to this point, sorting is performed so that the end point y-coordinate is in descending order (= from the lower plate area).
[0049]
Next, a comparison between the license plate cutout processing according to the present invention and the conventional method will be described.
The pseudo shifted correlation image in the present invention is a binarized image in which the plate region is “1” and the others are “0”, which is almost the same as the shifted correlation image of the positive / negative high-frequency binary image of the conventional method. ing.
In addition, by selecting the plate candidate area based on the binarized area for each small area using the pseudo shifted correlation image and the pseudo OR image, the matching degree is high by the two-dimensional template matching process for the shifted correlation image of the conventional method. It is possible to obtain substantially the same effect as the processing for obtaining the region.
Therefore, it can be said that the license plate cut-out performance by the license plate cut-out process according to the present invention is almost equivalent to the cut-out performance by the conventional method.
[0050]
In addition, according to the license plate cutout process according to the present invention, the filtering process is very small compared to the conventional method, and the amount of calculation of addition and multiplication is greatly reduced. It is possible. In the present invention, an algorithm can be described only with a general-purpose language that does not depend on the architecture of a computing device to be used, so that portability is high. Furthermore, it is not necessary to develop a dedicated arithmetic device, and the development period and cost can be reduced compared to the conventional case.
Note that the license plate recognition device according to the present invention can be applied not only when the license plate of the approaching vehicle is recognized in the toll collection process on the toll road, but also when the license plate of the vehicle is recognized in a parking lot, for example. Is.
[0051]
【The invention's effect】
As described above in detail, according to the present invention, a plate area can be cut out at high speed and with high accuracy using a general-purpose arithmetic device without newly developing a dedicated arithmetic device, and the development period can be shortened and the cost can be reduced. Can be reduced.
[Brief description of the drawings]
FIG. 1 is a diagram defining a coordinate system of a target digital image in an embodiment of the present invention.
FIG. 2 is a diagram showing a configuration of a license plate recognition device according to an embodiment of the present invention.
FIG. 3 is a flowchart showing the procedure of a license plate cutout process in the embodiment.
FIG. 4 is a diagram showing an example of a pseudo-shift correlation image in the embodiment.
FIG. 5 is a diagram showing an example of a pseudo-OR image in the embodiment.
FIG. 6 is a view for explaining a binarized area calculation process for each small region on the pseudo shifted correlation image in the embodiment;
FIG. 7 is a view for explaining merge processing of small areas on the pseudo shifted correlation image in the embodiment;
FIG. 8 is a view for explaining one process of an x-direction area setting process in the embodiment.
FIG. 9 is a view for explaining one process in an x-direction area setting process in the embodiment.
FIG. 10 is a view for explaining plate left and right end setting processing in the embodiment;
FIG. 11 is a view for explaining plate upper and lower end setting processing in the embodiment;
FIG. 12 is a flowchart showing a recognition processing procedure in a license plate recognition apparatus according to a conventional example.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Vehicle imaging device 2 ... Image input device 3 ... Arithmetic device

Claims (1)

A reduction means for reducing the captured vehicle image horizontally and vertically;
For the image reduced by the reduction means, the pixel value at a position shifted by a predetermined amount in the positive or negative direction from the pixel value is subtracted from the pixel value, and the subtraction result is compared with the binarization threshold value to obtain a binary value. A pseudo-shift correlation image generating means for generating a pseudo-shift correlation image by AND operation of the value when shifted in the positive direction and the value when shifted in the negative direction;
For the image reduced by the reduction means, the pixel value at a position shifted by a predetermined amount in the positive or negative direction from the pixel value is subtracted from the pixel value, and the subtraction result is compared with the binarization threshold value to obtain a binary value. A pseudo-OR image generation means for generating a pseudo-OR image by a logical OR operation of the value when shifted in the positive direction and the value when shifted in the negative direction;
The pseudo-shift correlation image generated by the pseudo-shift correlation image generation means is divided into a plurality of small regions, and the area of the pixel whose luminance value is set to “1” in each small region is calculated, License plate candidate area searching means for selecting a license plate candidate small area based on the area value of the small area;
Based on the license plate candidate small area obtained by the license plate candidate area searching means, a license plate area is provisionally set on the pseudo-OR image generated by the pseudo-OR image generating means, and the temporary setting area And a license plate cutting means for checking the compatibility as a license plate area and cutting out the license plate area.

Images