JP3706431B2 - Part shape recognition device - Google Patents

Description

【００３３】
【数２】

【００３４】
そして、画素Ｅの場所におけるエッジ強度Ｓは次の数式３で与えられ、グラディエント方向αは次の数式４で与えられる。
【００３５】
【数３】

【００３６】
【数４】

【００３７】
ただし、通常arctanは、−π／２〜π／２の範囲で計算されるが、ここでは、極座標系の方向θに合わせて０〜２πの範囲で計算される。
【００３８】
ラジアルスキャン方向θごとに、エッジ強度Ｓが最大となる点（ｒ，θ）が、部品のエッジ（輪郭）として抽出される。また、この点のグラディエント方向αがそのエッジ点でのエッジベクトルとされる。このようにして、部品画像の各エッジ点のｒ、θ、αが抽出される。
【００３９】
（３） 次に、エッジベクトルの補正処理が行なわれる（ステップ３）。つまり、図５に示すように、注目エッジ点Ａから互いに反対方向に所定距離離れた２つのエッジ点Ｂ、Ｃが選択され、選択された２つのエッジ点Ｂ、Ｃを結ぶ直線の方向に直交する方向（内側から外側に向かう方向）が、注目エッジ点Ａのエッジベクトルαとされる。
【００４０】
ただし、注目エッジ点から互いに反対方向に所定距離離れた２つのエッジ点であっても、その点のグラディエント方向αが注目エッジ点のグラディエント方向αとほぼ等しくない場合には、その点は選択されない。このような場合には、グラディエント方向αが注目エッジ点のグラディエント方向αと等しくないエッジ点の代わりに、そのエッジ点から注目エッジ点により近いエッジ点であって、グラディエント方向αが注目エッジ点のグラディエント方向αとほぼ等しいエッジ点が選択される。
【００４１】
このようなエッジベクトルαの補正が全てのエッジ点に対して行なわれる。ステップ２で抽出された各エッジ点のグラディエント方向αにノイズ誤差があっても、上記補正処理により精度の高いグラディエント方向（エッジベクトル）αが得られる。なお、ステップ２で求められた各エッジ点のグラディエント方向αを補正前のグラディエント方向αといい、ステップ３で求められた各エッジ点のグラディエント方向αを補正後のグラディエント方向αということにする。
【００４２】
次に、部品画像を対局的に四角形とみなし、その四角形の４辺（以下、モールド直線という）に対する法線ベクトルα１〜α４と、原点Ｇから各モールド直線までの距離ρ１〜ρ４が求められる（ステップ４、５）。
【００４３】
まず、直線の方程式について説明する。図５において、重心Ｇを原点とするＸＹ座標を想定すると、原点Ｇからθ方向に距離ｒのエッジ点ＤのＸＹ座標値は、ｒとθとを用いて次の数式５で表される。
【００４４】
【数５】

【００４５】
また、電子部品の上辺の直線の方程式は、原点Ｇからその直線までの距離をρとし、その直線の法線ベクトルをαとすると、ハフ変換により数式６で与えられる。
【００４６】
【数６】

【００４７】
数式６から、ｘ、ｙを消去すると、数式７が得られる。
【００４８】
【数７】

【００４９】
数式７で得られた直線の方程式のパラメータには、法線ベクトルαと原点Ｇから直線までの距離ρとがある。
【００５０】
（４） そこで、まず、各モールド直線に対する法線ベクトルα１〜α４が求められる（ステップ４）。
【００５１】
図７（ａ）は、横軸にエッジベクトルα（補正前のエッジベクトルαおよび補正後のエッジベクトルαの両方を含む）をとり、縦軸にラジアルスキャン方向θをとって、各エッジ点の分布を表したものである。この分布に基づいて、エッジベクトルαに対するエッジ点の度数を表すヒストグラムが生成される。エッジベクトルαに対するエッジ点のヒストグラムを図７（ｂ）に示す。
【００５２】
電子部品は大局的に四角形とみなすことができるので、その４辺に対応して４箇所に、エッジベクトルαの度数のピークＰ１〜Ｐ４ができる。四角形の各辺の方向は９０°ずつ異なっているので、エッジベクトルαの度数のピークＰ１〜Ｐ４の間隔も９０°ごとに現れる。
【００５３】
図７（ｂ）のヒストグラムが９０°単位に４分割され、各９０°単位のヒストグラムが足し合わされる。これにより、図７（ｃ）に示すヒストグラムが得られる。図７（ｃ）のヒストグラムには、１つの大きなピークＰが現れる。このピークＰに対応するエッジベクトルが、１つのモールド直線に対する法線ベクトルα１として抽出される。
【００５４】
抽出された１つのモールド直線に対する法線ベクトルα１に、９０°、１８０°および２７０°がそれぞれ加算されることにより、他の３つのモールド直線に対する法線ベクトルα２、α３およびα４が求められる。このような法線ベクトルα１〜α４の求め方を第１方法ということにする。
【００５５】
ところで、実際の部品には、微小な形状の歪みがあるため、対辺どうしが平行になるとは限らない。そこで、微小な形状の歪みを吸収するために、４つのモールド直線に対する法線ベクトルα１〜α４を、次のようにして求めてもよい。ここでは、法線ベクトルα２についてのみ説明する。
【００５６】
図７（ｂ）のヒストグラムの横軸上において、第１方法によって求められた法線ベクトルα２を中心とする幅±Δαの範囲を設定する。Δαとしては、たとえば５°が設定される。そして、その範囲内の各αごとに、αの値とそれに対するエッジ点の度数との積を求め、求められた積を加算する。つまり、αの値とそれに対するエッジ点の度数との積の累積値を求める。得られた累積値を、上記範囲内の度数の累積値で除算することにより、その範囲内でのαの平均値を算出する。そして、得られた平均値を法線ベクトルα２とする。このような法線ベクトルα１〜α４の求め方を第２方法ということにする。
【００５７】
また、部品の中には、１辺が隣り合う２辺と直角でない台形状のものがある。このような部品においては、隣り合う２辺と直角でない傾斜辺において、その辺に沿ったモールド直線を検出することが好ましい。
【００５８】
そこで、４つのモールド直線に対する法線ベクトルα１〜α４を、次のようにして求めてもよい。ここでは、法線ベクトルα２についてのみ説明する。
【００５９】
図７（ｂ）のヒストグラムにおいて、図７（ｃ）に基づいて上記のようにして求められた法線ベクトルα２の付近で度数がピークとなるαを局所的に探索し、探索されたαを法線ベクトルα２とする。この方法では、図７（ｂ）のＰ１、Ｐ２、Ｐ３、Ｐ４が法線ベクトルα１、α２、α３、α４となる。このような法線ベクトルα１〜α４の求め方を第３方法ということにする。
【００６０】
なお、第２方法と同様に、第３方法によって求められた法線ベクトルα１、α２、α３、α４付近のαの平均値を法線ベクトルα１、α２、α３、α４としてもよい。
【００６１】
（５） 次に、原点Ｇから各モールド直線までの距離ρ１〜ρ４が求められる（ステップ５）。
【００６２】
つまり、ステップ４で求められた４個の法線ベクトルα１〜α４のうち、まず、法線ベクトルα１に似通ったエッジベクトル（補正前のエッジベクトルαおよび補正後のエッジベクトルαの両方を含む）を持つエッジ点が選択される。そして、選択された各エッジ点ごとに、数式７により、ρが計算される。この際、数式７のαとしては、α１が用いられる。そして、得られたρに基づいて、ρに対するエッジ点の度数を表すヒストグラムを作成し、度数の最も大きいρが、法線ベクトルα１に対応するモールド直線に対するρ（＝ρ１）として抽出される。これにより、法線ベクトルがα１である直線のうちの最も長い直線と、原点Ｇとの距離がρ１として選択される。
【００６３】
同様にして、法線ベクトルα２、α３およびα４に対応するモールド直線のρ２、ρ３およびρ４が抽出される。
【００６４】
通常のハフ変換では、全ての検出エッジ点に対してその点のグラディエント方向αをパラメータとしてρが計算される。そして、各エッジ点のグラディエント方向αおよびρがα−ρ空間に投票され、票の多いα、ρの組み合わせを用いて直線が決定される。しかしながら、この方法では角チップのように４本の直線しか存在しない部品では問題はないが、異形部品、ＩＣ等のリード部品ではモールドの他、コーナ部分、モールドから突出した部分にも直線部分があるので、票の多い順に４本の直線を選択すると、モールドの４直線以外の直線が選択されてしまうことがある。
【００６５】
上記実施の形態では、モールド直線の法線ベクトルが先に決定され、この法線ベクトルに似通ったエッジベクトルを持つエッジ点に対してのみρの計算を行なっているので、モールド直線とは別の法線ベクトルを持つ直線が検出されるといったことがない。
【００６６】
（６） 次に、各エッジ点が、四角形の４辺ごとにクラス分けされる（ステップ６）。つまり、隣り合う２本のモールド直線の交点に対する極座標系の方向θ１、θ２、θ３およびθ４が求められる。隣り合う２本のモールド直線の交点を、コーナ位置と定義する。各エッジ点のθにより、各エッジ点がどの辺（モールド直線）に属するかがクラス分けされる。
【００６７】
隣り合う２本のモールド直線の交点に対する極座標系の方向θ１、θ２、θ３およびθ４は、次のようにして求められる。
【００６８】
法線ベクトルがα１のモールド直線に対するρをρ１とし、法線ベクトルがα２のモールド直線に対するρをρ２とし、これらのモールド直線の交点（コーナ位置）のＸＹ座標を（ｘｏ，ｙｏ）とすると、数式６から次の数式８が成り立つ。
【００６９】
【数８】

【００７０】
数式８から、上記両モールド直線の交点のＸＹ座標（ｘｏ，ｙｏ）は、次の数式９によって求められる。
【００７１】
【数９】

【００７２】
このようにして、４つのコーナ位置のＸＹ座標が求められると、各コーナ位置が原点Ｇからどの方向に位置しているかが計算されることにより、隣り合う２本のモールド直線の交点に対する極座標系の方向θ１、θ２、θ３およびθ４が求められる。
【００７３】
（７） 次に、各クラス毎に、各エッジ点からそのエッジ点が属するクラスの辺（モールド直線）までの距離Δρが算出される（ステップ７）。
【００７４】
つまり、まず、数式７を用いて各エッジ点ごとにρが求められる。この際、法線ベクトルαとしては、当該エッジ点が属するクラスのモールド直線の法線ベクトルα１、α２、α３、α４が用いられる。つまり、エッジ点を通りかつそのエッジ点が属するモールド直線と平行な直線と、原点Ｇとの距離ρが算出される。
【００７５】
次に、各エッジ点に対して求められたρと、そのエッジ点が属するモールド直線のρ１、ρ２、ρ３またはρ４との差Δρ、すなわち、エッジ点からモールド直線までの距離が求められる。図８は、モールド直線Ｌに対応するクラスに属するエッジ点と、これらのエッジ点ごとに求められたΔρを示している。図８において、Ｒ１およびＲ２は、コーナ位置を示している。エッジ点がモールド直線上にある場合には、そのΔρは０となる。エッジ点がモールド直線より外側にある場合には、そのΔρは正の値（Δρ＞０）となる。エッジ点がモールド直線より内側にある場合には、そのΔρは負の値（Δρ＜０）となる。
【００７６】
（８）次に、各エッジ点が、エッジ部分とコーナ部分とにクラス分けされる（ステップ８）。
【００７７】
まず、コーナ位置の方向θ付近にあるエッジ点からモールド直線の中央部に向かって、各エッジ点のΔρが調べられ、最初にΔρが０となるエッジ点がエッジ端点とされる。図８の例では、点Ｑ１、Ｑ２、Ｑ３、Ｑ８がエッジ端点である。モールド直線上の２つのエッジ端点Ｑ１、Ｑ２の間部分がエッジ部分とされる。また、２つのモールド直線のコーナ点付近にある２のエッジ端点の間部分、すなわち、図８の点Ｑ１とＱ８との間部分、点Ｑ２とＱ３との間部分がコーナ部分とされる。
【００７８】
（９）次に、４つのエッジ部分におけるΔρの分布に基づいてエッジ形状が認識され、４つのコーナ部分におけるΔρの分布に基づいてコーナ形状が認識される。
【００７９】
エッジ形状認識は、たとえば、次のようにして行なわれる。まず、エッジ点がモールド直線上に位置しているか否かを判定するしきい値を、モールド直線に直交する方向の距離として設定する。そして、エッジ部分に属するエッジ点のうち、そのΔρが＋方向および−方向にそのしきい値を越えているエッジ点が存在しない場合には、そのエッジ部分の形状を”直線形状”と判定する。
【００８０】
エッジ部分のエッジ点のうち、そのΔρが＋方向にそのしきい値を越えているエッジ点が存在する場合には、原則的にそのエッジ部分の形状を”突起あり”と判定する。また、エッジ部分のエッジ点のうち、そのΔρが−方向にそのしきい値を越えているエッジ点が存在する場合には、原則的にそのエッジ部分の形状を”陥没あり”と判定する。ただし、突起または陥没が３個以上存在する場合には、そのエッジ部分の形状を”不良形状”と判定する。
【００８１】
コーナ形状認識は、コーナ部分のΔρの分布だけでなく、コーナ部分の２つのエッジ端点を結ぶ直線を一方の対角線とする正方形または長方形領域内の部品部分の面積と背景部分の面積との比やコーナ部分の各エッジ点のベクトルの方向の変化に基づいて判定してもよい。
【００８２】
図９は部品画像の一例を示している。図１０は、図９の画像から求められたΔρの分布を示している。図９において、Ｒ１〜Ｒ４はコーナ位置を示し、Ｑ１〜Ｑ８はエッジ端点を示している。この例では、各エッジ部分のΔρの分布に基づいて、左側エッジ部ＥＬは”突起あり”と認識され、下側エッジ部ＥＤは”直線形状”と認識され、右側エッジ部ＥＲは”不良形状”と認識され、上側エッジ部ＥＵは”直線形状”と認識される。
【００８３】
また、各コーナ部分のΔρの分布に基づいて、右上コーナ部および右下コーナ部は”Ｃ面取り”と認識され、左上コーナ部および左下コーナ部は”不良形状”と認識される。左上コーナ部および左下コーナ部が”Ｒ面取り”と認識されず、”不良形状”と認識されているのは、コーナ部が突起に連続している場合には、”不良形状”と認識するようにしたことによる。
【００８４】
コーナ形状認識を、コーナ部分の２つのエッジ点を結ぶ直線を一方の対角線とする正方形領域内の部品部分の面積と背景部分の面積との比に基づいて判定する方法について説明する。
【００８５】
図１１の（ａ）、（ｂ）、（ｃ）は、コーナ部分の２つのエッジ端点を結ぶ直線を一方の対角線とする正方形領域を示している。図１１（ａ）はコーナ形状が”Ｃ面取り”の例を示し、図１１（ｂ）はコーナ形状が”Ｒ面取り”の例を示し、図１１（ｃ）はコーナ形状が”不良形状”の例を示している。
【００８６】
正方形領域内の背景部分の面積Ｓ１と部品部分の面積Ｓ２との比（Ｓ１／Ｓ２）がほぼ１であれば、そのコーナ形状は”Ｃ面取り”と判定される。正方形領域内の背景部分の面積Ｓ１と部品部分の面積Ｓ２との比（Ｓ１／Ｓ２）が｛（４−π）／π｝＝０．２７にほぼ等しければ、そのコーナ形状は”Ｒ面取り”と判定される。正方形領域内の背景部分の面積Ｓ１と部品部分の面積Ｓ２との比（Ｓ１／Ｓ２）が、１または０．２７にほぼ等しくない場合には、そのコーナ形状は”不良形状”と判定される。
【００８７】
なお、エッジ形状の認識およびコーナ形状の認識を、ニューラルネットワークを用いて行なうようにしてもよい。ニューラルネットワークとしては、たとえば、図１２に示すように、入力層、中間層および出力層からなる階層型ニューラルネットワークが用いられる。
【００８８】
このようなニューラルネットワークを用いてエッジ形状の認識を行なう場合について説明する。複数の電子部品画像から得られた複数のエッジ部分におけるモールド直線との差分Δρの分布を入力パターンとし、入力パターンに対応するエッジ形状を表す信号を教師信号として、ニューラルネットワークを学習させる。出力層のユニット数が４である場合には、図１３に示すように、”直線形状”に対応する教師信号としては（１，０，０，０）が、”突起あり”に対応する教師信号としては（０，１，０，０）が、”陥没あり”に対応する教師信号としては（０，０，１，０）が、”不良形状”に対応する教師信号としては（０，０，０，１）が用いられる。
【００８９】
学習後のニューラルネットワークに、形状認識対象である電子部品画像から得られたエッジ部分におけるΔρの分布を入力信号として与えると、そのエッジ部分の形状に応じた出力が得られる。なお、エッジ部分の長さが異なる場合には、たとえば写像変換により、当該エッジ部分におけるΔρの数を、入力層のユニット数に合わせればよい。
【００９０】
ニューラルネットワークを用いてコーナ形状の認識を行なう場合は、複数の電子部品画像から得られた複数のコーナ部分におけるΔρの分布を入力パターンとし、入力パターンに対応するコーナ形状を表す信号を教師信号として、ニューラルネットワークを学習させる。
【００９１】
学習後のニューラルネットワークに、形状認識対象である電子部品画像から得られたコーナ部分におけるΔρの分布を入力信号として与えると、そのコーナ部分の形状に応じた出力が得られる。
【００９２】
【発明の効果】
この発明によれば、部品の局所形状を自動的に認識できるようになる。
【図面の簡単な説明】
【図１】電子部品自動装着システムの全体的な構成を示すブロック図である。
【図２】エッジ形状の種類を示す模式図である。
【図３】コーナ形状の種類を示す模式図である。
【図４】局所形状認識処理の手順を示すフローチャートである。
【図５】部品画像の一例を示す模式図である。
【図６】微分オペレータの重み付けを示す模式図である。
【図７】各モールド直線に対する法線ベクトルα１〜α４を求める方法を説明するためのグラフである。
【図８】モールド直線Ｌに対応するクラスに属するエッジ点と、これらのエッジ点ごとに求められたΔρを示す模式図である。
【図９】部品画像の一例を示す模式図である。
【図１０】図９の画像から求められたΔρの分布を示すグラフである。
【図１１】コーナ部分の２つのエッジ端点を結ぶ直線を一方の対角線とする正方形領域を示す模式図である。
【図１２】ニューラルネットワークの構成例を示す模式図である。
【図１３】教師信号の一例を示す説明図である。
【符号の説明】
１ ホストコンピュータ
２ 電子部品装着装置
３ 画像処理装置
４ データベース
３１ 位置認識手段
３２ 標準データ計測手段[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a component shape recognition apparatus that recognizes the shape of a component by applying an image processing technique.
[0002]
[Prior art]
As electronic products become smaller and lighter, mounting of chip-shaped electronic components on a printed circuit board requires higher density. In order to realize mounting accuracy that satisfies the requirements, positioning processing of electronic components using image processing technology is indispensable.
[0003]
Since the positioning process is performed by detecting the corner position of the part and the edge position of the contour of the part, information on the local shape of the part such as the corner shape and the edge shape is required as well as various sizes of the part. .
[0004]
The registration of information related to the local shape of such parts is currently performed by manual input by the operator, and there is a problem that the burden on the operator is heavy.
[0005]
[Problems to be solved by the invention]
An object of this invention is to provide the component shape recognition apparatus which can recognize the local shape of components automatically.
[0006]
[Means for Solving the Problems]
An apparatus for recognizing a part shape according to the present invention is an edge detection means for detecting a point on a contour of a part image as an edge point based on the part image. The part image is generally regarded as a quadrangle, and every four sides of the square A clustering means for classifying edge points, a distribution creating means for obtaining a distance distribution between the edge corresponding to the class and each edge point belonging to the class, and a component based on the obtained distribution. It has a recognition means for recognizing the local shape.
[0007]
As the edge detection means, for example, by performing differential processing while scanning radially from the center of gravity position of the component image, the position of the edge point on the contour of the component image is determined in a polar coordinate system having the center of gravity position of the component image as the origin. What detects is used to detect the gradient direction of each edge point.
[0008]
As the clustering means, for example, a part image is generally regarded as a quadrangle, a normal vector calculation means for obtaining a normal vector for each side of the quadrangle, an equation obtained by a normal vector for each side and a Hough transform formula Based on the distance calculation means for obtaining the distance from the center of gravity of the component image to each side, and specifying the four sides based on the normal vector for each side and the distance from the center of gravity of the component image to each side, A means for classifying to which side each edge point belongs is provided on the basis of the direction of the polar coordinate system corresponding to the intersection position of adjacent sides in the specified four sides and the direction of the polar coordinate system of each edge point Is used.
[0009]
As the normal vector calculation means, for example, one that obtains a normal vector for each side of a quadrilateral based on the frequency distribution of edge points with respect to the gradient direction is used.
[0010]
The frequency distribution of edge points with respect to the gradient direction is preferably created in consideration of both the first gradient direction and the second gradient direction of each edge point. The first gradient direction of each edge point is a gradient direction obtained by performing differentiation processing while scanning radially from the barycentric position of the component image. The second gradient direction of each edge point is two edge points that are opposite to each other about the target edge point, and the first gradient direction is substantially equal to the first gradient direction of the target edge point. This is a normal vector for a straight line connecting edge points.
[0011]
The distance calculation means calculates the distance from the center of gravity of the component image to the target side for each side of the quadrangle. For example, the distance calculation unit resembles the normal vector for the target side determined by the normal vector calculation unit. Select an edge point with a gradient direction, and for each selected edge point, use the normal vector for the target side and the equation obtained by the Hough transform formula, A means for calculating the distance between the straight line having the same normal vector and the gravity center position of the component image, and the frequency distribution of the edge points with respect to the calculated distance are created, and the distance with the highest frequency is calculated from the gravity center position of the component image. What is provided with the means to select as a distance to an attention side is used.
[0012]
An edge point having a gradient direction similar to the normal vector for the target side is preferably selected in consideration of both the first gradient direction and the second gradient direction of each edge point. The first gradient direction of each edge point is a gradient direction obtained by performing differentiation processing while scanning radially from the barycentric position of the component image. The second gradient direction of each edge point is two edge points that are opposite to each other about the target edge point, and the first gradient direction is substantially equal to the first gradient direction of the target edge point. This is a normal vector for a straight line connecting edge points.
[0013]
As the distribution creation means, for example, based on the equation obtained by the Hough transform equation and the normal vector for each side, a straight line that passes through the edge point and is parallel to the side of the class to which the edge point belongs, and a component image Means for determining the distance from the center of gravity position of each edge point, and the distance between the center of gravity of the component image and the distance between the straight line passing through the edge point and parallel to the side to which the edge point belongs, The difference between the distance to the edge of the class to which the edge point belongs is obtained for each edge point, and the distance between the edge corresponding to the class and each edge point belonging to the class for each class based on the obtained difference. A device having a means for creating a distribution of is used.
[0014]
Examples of the recognition means include, for example, a means for classifying each edge point into four corner parts and four edge parts, and an edge shape for recognizing an edge shape based on a distance distribution for each edge part created by the distribution creation means. A device provided with a recognizing means and a corner shape recognizing means for recognizing a corner shape based on a distance distribution for each corner portion created by the distribution creating means is used.
[0015]
As the edge shape recognition means, for example, a threshold value for determining whether or not an edge point belonging to a target edge portion is located on a side corresponding to the target edge portion is set as a distance in a direction orthogonal to the side. The one that recognizes the edge shape of the target edge portion is set by comparing the distance between each edge point belonging to the target edge portion and the above side with a threshold value.
[0016]
As an edge shape recognition means, the distance distribution for a plurality of edge portions obtained from a plurality of component images is used as an input pattern, and a signal representing the edge shape corresponding to the input pattern is used as a teacher signal to the learned neural network. You may use what recognizes the edge shape of a target edge part by inputting the distance distribution with respect to the target edge part produced by the distribution creation means.
[0017]
As a corner shape recognition means, a distance distribution for a plurality of corner portions obtained from a plurality of component images is used as an input pattern, and a signal representing a corner shape corresponding to the input pattern is used as a teacher signal to the learned neural network. You may use what recognizes the corner shape of a target corner part by inputting the distance distribution with respect to the target corner part created by the distribution creation means.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention applied to an electronic component automatic mounting system will be described below with reference to the drawings.
[0019]
FIG. 1 shows the overall configuration of an electronic component automatic mounting system.
[0020]
The electronic component automatic mounting system includes a host computer 1 that controls the entire system, an electronic component mounting device 2 that mounts electronic components on a printed circuit board, and an image processing device 3 that performs component position recognition, shape recognition, and the like. The host computer 1 includes a database 4 in which standard data used for position recognition of electronic components is stored. A monitor 5 and a user interface 6 are connected to the image processing apparatus 3.
[0021]
The electronic component mounting apparatus 2 includes a component image input unit 21 that captures an image of an electronic component sucked by a nozzle with a CCD camera and acquires a component image.
[0022]
The image processing apparatus 3 takes the component image obtained by the component image input means 21 into an image memory as, for example, a 512 × 512 pixel, 8-bit, 256-gradation grayscale image, recognizes the position of the electronic component by image processing, and standard data Measure and so on. The monitor 5 displays component images, processing results, and the like.
[0023]
The image processing apparatus 3 includes a position recognition unit 31 and a standard data measurement unit 32. The position recognition unit 31 detects the position of the center of gravity, the tilt angle, and the like of the electronic component based on the electronic component image and standard data such as the type, size, and shape of the electronic component supplied from the host computer 1. The standard data measuring means 32 obtains standard data such as the type, size, and shape information of the electronic component based on the electronic component image.
[0024]
The user interface 6 corrects the standard data obtained by the standard data measuring means 32 as necessary. In addition to controlling each device, the host computer performs statistical processing of standard data, registration, update, and reading of standard data in a database.
[0025]
2 and 3 show local shapes of electronic components used as standard data. FIG. 2 shows an example of an edge shape, and FIG. 3 shows an example of a corner shape.
[0026]
Edge shapes include “linear shape” (FIG. 2 (a)), “with protrusion” (FIG. 2 (b)), “with depression” (FIG. 2 (c)), and other shapes (“defective shape”) (FIG. 2 (d)).
[0027]
The corner shape includes “R chamfer” (FIG. 3A), “C chamfer” (FIG. 3B), and other shapes (“defective shape”) (FIG. 3C).
[0028]
FIG. 4 shows a local shape recognition processing procedure performed by the standard data measuring means 32.
[0029]
(1) First, as shown in FIG. 5, the center-of-gravity position G of the component image is obtained by blob analysis or the like (step 1).
[0030]
(2) A point (edge point) on the contour of the component is obtained by performing differentiation while using the gravity center position G as the origin of the polar coordinate system (r, θ) and accessing (radial scanning) the image data radially. Is detected (step 2). That is, the position (r, θ) of the edge point and the gradient direction (edge vector) α of the point are detected.
[0031]
For this edge detection, for example, a sobel differential operator is used. FIG. 6 (a) shows the weighting of the sobel differential operator in the x direction. FIG. 6B shows the weighting of the differential operator in the y direction of sobel. As shown in FIG. 6C, when the positions and values of the pixels in the local region are expressed by A to H, the horizontal differential value Sx is expressed by Formula 1, and the vertical differential value Sy is expressed by Formula 2. Is done.
[0032]
[Expression 1]

[0033]
[Expression 2]

[0034]
The edge intensity S at the location of the pixel E is given by the following formula 3, and the gradient direction α is given by the following formula 4.
[0035]
[Equation 3]

[0036]
[Expression 4]

[0037]
However, arctan is usually calculated in the range of −π / 2 to π / 2, but here it is calculated in the range of 0 to 2π in accordance with the direction θ of the polar coordinate system.
[0038]
For each radial scan direction θ, the point (r, θ) at which the edge strength S is maximum is extracted as the edge (contour) of the part. Also, the gradient direction α of this point is the edge vector at that edge point. In this way, r, θ, and α of each edge point of the component image are extracted.
[0039]
(3) Next, edge vector correction processing is performed (step 3). That is, as shown in FIG. 5, two edge points B and C that are separated from the target edge point A by a predetermined distance in the opposite direction are selected, and orthogonal to the direction of the straight line connecting the two selected edge points B and C. The direction (direction from the inside toward the outside) is the edge vector α of the target edge point A.
[0040]
However, even if two edge points are separated from the target edge point by a predetermined distance in the opposite direction, the point is not selected if the gradient direction α of the point is not substantially equal to the gradient direction α of the target edge point. . In such a case, instead of the edge point where the gradient direction α is not equal to the gradient direction α of the target edge point, the edge point is closer to the target edge point, and the gradient direction α is the target edge point. Edge points that are approximately equal to the gradient direction α are selected.
[0041]
Such an edge vector α is corrected for all edge points. Even if there is a noise error in the gradient direction α of each edge point extracted in step 2, a highly accurate gradient direction (edge vector) α can be obtained by the correction process. Note that the gradient direction α of each edge point obtained in step 2 is referred to as a gradient direction α before correction, and the gradient direction α of each edge point obtained in step 3 is referred to as a gradient direction α after correction.
[0042]
Next, the part image is considered as a square, and normal vectors α1 to α4 with respect to four sides of the square (hereinafter referred to as mold straight lines) and distances ρ1 to ρ4 from the origin G to each mold straight line are obtained ( Steps 4, 5).
[0043]
First, the linear equation will be described. In FIG. 5, assuming an XY coordinate having the center of gravity G as the origin, the XY coordinate value of the edge point D at a distance r in the θ direction from the origin G is expressed by the following Equation 5 using r and θ.
[0044]
[Equation 5]

[0045]
Further, the equation of the straight line on the upper side of the electronic component is given by Equation 6 by the Hough transform, where ρ is the distance from the origin G to the straight line and α is the normal vector of the straight line.
[0046]
[Formula 6]

[0047]
When x and y are deleted from Expression 6, Expression 7 is obtained.
[0048]
[Expression 7]

[0049]
The parameters of the straight line equation obtained by Expression 7 include the normal vector α and the distance ρ from the origin G to the straight line.
[0050]
(4) Therefore, first, normal vectors α1 to α4 for each mold straight line are obtained (step 4).
[0051]
In FIG. 7A, the horizontal axis represents the edge vector α (including both the uncorrected edge vector α and the corrected edge vector α), and the vertical axis represents the radial scan direction θ. It represents the distribution. Based on this distribution, a histogram representing the frequency of edge points with respect to the edge vector α is generated. FIG. 7B shows a histogram of edge points with respect to the edge vector α.
[0052]
Since the electronic component can be generally regarded as a quadrangle, peaks P1 to P4 of the frequency of the edge vector α are formed at four locations corresponding to the four sides. Since the directions of the sides of the square are different by 90 °, the intervals between the peaks P1 to P4 of the frequency of the edge vector α also appear every 90 °.
[0053]
The histogram of FIG. 7B is divided into four 90 ° units, and the histograms of each 90 ° unit are added. Thereby, the histogram shown in FIG. 7C is obtained. One large peak P appears in the histogram of FIG. An edge vector corresponding to this peak P is extracted as a normal vector α1 for one mold line.
[0054]
By adding 90 °, 180 °, and 270 ° to the normal vector α1 for one extracted mold line, normal vectors α2, α3, and α4 for the other three mold lines are obtained. Such a method of obtaining the normal vectors α1 to α4 is referred to as a first method.
[0055]
By the way, an actual part has a minute shape distortion, so that the opposite sides are not always parallel to each other. Therefore, in order to absorb the distortion of a minute shape, normal vectors α1 to α4 for the four mold straight lines may be obtained as follows. Here, only the normal vector α2 will be described.
[0056]
On the horizontal axis of the histogram of FIG. 7B, a range of width ± Δα centering on the normal vector α2 obtained by the first method is set. For example, 5 ° is set as Δα. Then, for each α in the range, a product of the value of α and the frequency of the edge point is obtained, and the obtained products are added. That is, the cumulative value of the product of the value of α and the frequency of the edge point corresponding thereto is obtained. The average value of α within the range is calculated by dividing the obtained cumulative value by the cumulative value of the frequency within the range. The obtained average value is defined as a normal vector α2. Such a method of obtaining the normal vectors α1 to α4 is referred to as a second method.
[0057]
Further, some parts have a trapezoidal shape in which one side is not perpendicular to two adjacent sides. In such a component, it is preferable to detect a mold straight line along an inclined side that is not perpendicular to two adjacent sides.
[0058]
Therefore, the normal vectors α1 to α4 for the four mold straight lines may be obtained as follows. Here, only the normal vector α2 will be described.
[0059]
In the histogram of FIG. 7 (b), a search is made locally for α having a frequency peak in the vicinity of the normal vector α2 obtained as described above based on FIG. 7 (c). The normal vector is α2. In this method, P1, P2, P3, and P4 in FIG. 7B become normal vectors α1, α2, α3, and α4. Such a method of obtaining the normal vectors α1 to α4 is referred to as a third method.
[0060]
Similar to the second method, the average values of α around the normal vectors α1, α2, α3, and α4 obtained by the third method may be used as the normal vectors α1, α2, α3, and α4.
[0061]
(5) Next, distances ρ1 to ρ4 from the origin G to each mold straight line are obtained (step 5).
[0062]
That is, among the four normal vectors α1 to α4 obtained in step 4, first, an edge vector similar to the normal vector α1 (including both the edge vector α before correction and the edge vector α after correction). Edge points with are selected. Then, ρ is calculated by Equation 7 for each selected edge point. At this time, α1 is used as α in Expression 7. Then, based on the obtained ρ, a histogram representing the frequency of the edge point with respect to ρ is created, and ρ having the highest frequency is extracted as ρ (= ρ1) for the mold line corresponding to the normal vector α1. Thereby, the distance between the longest straight line among the straight lines having the normal vector α1 and the origin G is selected as ρ1.
[0063]
Similarly, ρ2, ρ3, and ρ4 of the mold straight lines corresponding to the normal vectors α2, α3, and α4 are extracted.
[0064]
In normal Hough transform, ρ is calculated for all detected edge points with the gradient direction α of the points as a parameter. Then, the gradient directions α and ρ of each edge point are voted on the α-ρ space, and a straight line is determined using a combination of α and ρ with many votes. However, with this method, there is no problem with a part that has only four straight lines, such as a square chip. However, with a deformed part or a lead part such as an IC, a straight part is also present in the corner part and the part protruding from the mold. Therefore, if four straight lines are selected in descending order of votes, straight lines other than the four straight lines of the mold may be selected.
[0065]
In the above embodiment, the normal vector of the mold straight line is determined first, and ρ is calculated only for edge points having edge vectors similar to this normal vector. A straight line having a normal vector is not detected.
[0066]
(6) Next, each edge point is classified into four sides of the quadrangle (step 6). That is, the polar coordinate system directions θ1, θ2, θ3, and θ4 with respect to the intersection of two adjacent mold straight lines are obtained. The intersection of two adjacent mold straight lines is defined as the corner position. Which edge (mold straight line) each edge point belongs to is classified according to θ of each edge point.
[0067]
The directions θ1, θ2, θ3, and θ4 of the polar coordinate system with respect to the intersection of two adjacent mold straight lines are obtained as follows.
[0068]
Ρ for a mold line with a normal vector α1 is ρ1, ρ for a mold line with a normal vector α2 is ρ2, and the XY coordinates of the intersection (corner position) of these mold lines are (xo, yo). From Expression 6, the following Expression 8 holds.
[0069]
[Equation 8]

[0070]
From Equation 8, the XY coordinates (xo, yo) of the intersection of both mold straight lines are obtained by the following Equation 9.
[0071]
[Equation 9]

[0072]
When the XY coordinates of the four corner positions are obtained in this way, the polar coordinate system for the intersection of two adjacent mold straight lines is calculated by calculating in which direction each corner position is located from the origin G. Directions θ1, θ2, θ3, and θ4 are obtained.
[0073]
(7) Next, for each class, the distance Δρ from each edge point to the side of the class to which the edge point belongs (mold straight line) is calculated (step 7).
[0074]
That is, first, ρ is obtained for each edge point using Equation 7. At this time, the normal vectors α1, α2, α3, α4 of the mold straight lines of the class to which the edge point belongs are used as the normal vectors α. That is, the distance ρ between the origin G and a straight line passing through the edge point and parallel to the mold straight line to which the edge point belongs is calculated.
[0075]
Next, the difference Δρ between ρ obtained for each edge point and ρ1, ρ2, ρ3 or ρ4 of the mold line to which the edge point belongs, that is, the distance from the edge point to the mold line is obtained. FIG. 8 shows edge points belonging to the class corresponding to the mold straight line L, and Δρ obtained for each of these edge points. In FIG. 8, R1 and R2 indicate corner positions. When the edge point is on the mold straight line, the Δρ is zero. In the case where the edge point is outside the mold straight line, the Δρ is a positive value (Δρ> 0). When the edge point is inside the mold straight line, Δρ is a negative value (Δρ <0).
[0076]
(8) Next, each edge point is classified into an edge portion and a corner portion (step 8).
[0077]
First, Δρ of each edge point is examined from the edge point in the vicinity of the direction θ of the corner position toward the center of the mold straight line, and the edge point where Δρ is first 0 is set as the edge end point. In the example of FIG. 8, points Q1, Q2, Q3, and Q8 are edge end points. A portion between two edge end points Q1 and Q2 on the mold straight line is an edge portion. Further, a portion between the two edge end points in the vicinity of the corner points of the two mold straight lines, that is, a portion between the points Q1 and Q8 and a portion between the points Q2 and Q3 in FIG.
[0078]
(9) Next, the edge shape is recognized based on the distribution of Δρ in the four edge portions, and the corner shape is recognized based on the distribution of Δρ in the four corner portions.
[0079]
Edge shape recognition is performed as follows, for example. First, a threshold value for determining whether or not the edge point is located on the mold straight line is set as a distance in a direction orthogonal to the mold straight line. If there is no edge point whose Δρ exceeds the threshold value in the + and − directions among the edge points belonging to the edge portion, the shape of the edge portion is determined as a “linear shape”. .
[0080]
In the case where there is an edge point whose Δρ exceeds the threshold value in the + direction among the edge points of the edge portion, in principle, the shape of the edge portion is determined as “protrusion”. If there is an edge point whose Δρ exceeds the threshold value in the − direction among the edge points of the edge portion, in principle, the shape of the edge portion is determined to be “depressed”. However, when there are three or more protrusions or depressions, the shape of the edge portion is determined as a “defective shape”.
[0081]
Corner shape recognition is not only the distribution of Δρ in the corner part, but also the ratio of the area of the part part in the square or rectangular area with the straight line connecting the two edge endpoints of the corner part to the area of the background part. You may determine based on the change of the direction of the vector of each edge point of a corner part.
[0082]
FIG. 9 shows an example of a component image. FIG. 10 shows the distribution of Δρ obtained from the image of FIG. In FIG. 9, R1 to R4 indicate corner positions, and Q1 to Q8 indicate edge end points. In this example, based on the distribution of Δρ of each edge portion, the left edge portion EL is recognized as “protruded”, the lower edge portion ED is recognized as “straight shape”, and the right edge portion ER is “defective shape”. ", And the upper edge EU is recognized as a" linear shape ".
[0083]
Further, based on the distribution of Δρ of each corner portion, the upper right corner portion and the lower right corner portion are recognized as “C chamfering”, and the upper left corner portion and the lower left corner portion are recognized as “defective shapes”. The upper left corner and the lower left corner are not recognized as “R chamfer”, but are recognized as “defective shape”. When the corner is continuous with the protrusion, it is recognized as “defective shape”. It depends on what you did.
[0084]
A method of determining the corner shape recognition based on the ratio of the area of the part portion in the square area and the area of the background portion with a straight line connecting two edge points of the corner portion as one diagonal line will be described.
[0085]
(A), (b), and (c) of FIG. 11 show a square region having one straight line connecting two edge end points of the corner portion as one diagonal line. 11A shows an example in which the corner shape is “C chamfering”, FIG. 11B shows an example in which the corner shape is “R chamfering”, and FIG. 11C shows that the corner shape is “bad shape”. An example is shown.
[0086]
If the ratio (S1 / S2) of the area S1 of the background portion to the area S2 of the component portion in the square region is approximately 1, the corner shape is determined as “C chamfering”. If the ratio (S1 / S2) of the area S1 of the background portion to the area S2 of the part in the square region is substantially equal to {(4-π) / π} = 0.27, the corner shape is “R chamfer”. It is determined. When the ratio (S1 / S2) of the area S1 of the background portion and the area S2 of the component portion in the square region is not substantially equal to 1 or 0.27, the corner shape is determined as “defective shape”. .
[0087]
Note that edge shape recognition and corner shape recognition may be performed using a neural network. As the neural network, for example, as shown in FIG. 12, a hierarchical neural network including an input layer, an intermediate layer, and an output layer is used.
[0088]
A case where edge shape recognition is performed using such a neural network will be described. The neural network is trained using the distribution of the difference Δρ from the mold straight line at a plurality of edge portions obtained from a plurality of electronic component images as an input pattern and a signal representing the edge shape corresponding to the input pattern as a teacher signal. When the number of units in the output layer is 4, as shown in FIG. 13, the teacher signal corresponding to “straight shape” is (1, 0, 0, 0) as the teacher signal corresponding to “protrusion”. (0, 1, 0, 0) is a signal, (0, 0, 1, 0) is a teacher signal corresponding to “with depression”, and (0, 1, 0, 0) is a teacher signal corresponding to “defective shape”. 0, 0, 1) is used.
[0089]
When the Δρ distribution in the edge portion obtained from the electronic component image that is the shape recognition target is given as an input signal to the learned neural network, an output corresponding to the shape of the edge portion is obtained. When the lengths of the edge portions are different, the number of Δρ in the edge portions may be matched with the number of units in the input layer, for example, by mapping conversion.
[0090]
When recognizing a corner shape using a neural network, the distribution of Δρ in a plurality of corner portions obtained from a plurality of electronic component images is used as an input pattern, and a signal representing the corner shape corresponding to the input pattern is used as a teacher signal. , Learn neural networks.
[0091]
When the distribution of Δρ in the corner portion obtained from the electronic component image that is the shape recognition target is given as an input signal to the learned neural network, an output corresponding to the shape of the corner portion is obtained.
[0092]
【The invention's effect】
According to the present invention, the local shape of a part can be automatically recognized.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an overall configuration of an electronic component automatic mounting system.
FIG. 2 is a schematic diagram showing types of edge shapes.
FIG. 3 is a schematic diagram showing types of corner shapes.
FIG. 4 is a flowchart illustrating a procedure of local shape recognition processing.
FIG. 5 is a schematic diagram illustrating an example of a component image.
FIG. 6 is a schematic diagram showing weighting of a differential operator.
FIG. 7 is a graph for explaining a method of obtaining normal vectors α1 to α4 for each mold straight line.
FIG. 8 is a schematic diagram showing edge points belonging to a class corresponding to a mold straight line L and Δρ obtained for each of these edge points.
FIG. 9 is a schematic diagram illustrating an example of a component image.
10 is a graph showing a distribution of Δρ obtained from the image of FIG. 9;
FIG. 11 is a schematic diagram showing a square region in which a straight line connecting two edge end points of a corner portion is one diagonal line.
FIG. 12 is a schematic diagram illustrating a configuration example of a neural network.
FIG. 13 is an explanatory diagram showing an example of a teacher signal.
[Explanation of symbols]
1 Host computer
2 Electronic component mounting device
3 Image processing device
4 Database
31 Position recognition means
32 Standard data measurement means

Claims (11)

Edge detection means for detecting a point on the contour of the component image as an edge point based on the component image;
A clustering means that regards a part image as a square and classifies edge points for each of the four sides of the square;
For each class, a distribution creating means for obtaining a distribution of distances between the edge corresponding to the class and each edge point belonging to the class, and a recognition means for recognizing the local shape of the component based on the obtained distribution,
A component shape recognition device. The edge detection means detects the position of the edge point on the contour of the component image in a polar coordinate system having the origin of the centroid position of the component image by performing differential processing while scanning radially from the centroid position of the component image. The component shape recognition apparatus according to claim 1, wherein the gradient direction of each edge point is detected. The clustering means regards the part image as a quadrangle globally, and calculates a normal vector calculation means for obtaining a normal vector for each side of the quadrangle.
Based on the normal vector for each side and the equation obtained by the Hough transform formula, distance calculation means for obtaining the distance from the center of gravity of the part image to each side, and from the normal vector for each side and the center of gravity of the part image Four sides are identified based on the distance to each side, and based on the direction of the polar coordinate system corresponding to the intersection position of adjacent sides in the identified four sides and the direction of the polar coordinate system of each edge point, A means of classifying which side an edge point belongs to,
3. The component shape recognition apparatus according to claim 1, further comprising: 4. The component shape recognition apparatus according to claim 3, wherein the normal vector calculating means calculates a normal vector for each side of the quadrangle based on a frequency distribution of edge points with respect to the gradient direction. The gradient direction of each edge point has a first gradient direction and a second gradient direction,
The first gradient direction of each edge point is a gradient direction obtained by performing differentiation processing while scanning radially from the center of gravity position of the component image,
The second gradient direction of each edge point is two edge points that are opposite to each other about the target edge point, and the first gradient direction is substantially equal to the first gradient direction of the target edge point. It is a normal vector for a straight line connecting edge points,
The component shape recognition apparatus according to claim 4, wherein a frequency distribution of the edge points with respect to the gradient direction is created in consideration of both the first gradient direction and the second gradient direction of each edge point. The distance calculation means obtains the distance from the center of gravity position of the component image to the attention side for each side of the rectangle.
An edge point having a gradient direction similar to the normal vector for the target side obtained by the normal vector calculation means is selected, and for each selected edge point, the normal vector for the target side and the Hough transform formula are used. Means for calculating the distance between the straight line passing through the point and having a normal vector equal to the normal vector for the side of interest and the gravity center position of the component image, and the frequency distribution of edge points with respect to the calculated distance Means for selecting the distance with the highest frequency as the distance from the center of gravity of the part image to the target side,
The part shape recognition apparatus of Claim 3 provided with. The gradient direction of each edge point has a first gradient direction and a second gradient direction,
The first gradient direction of each edge point is a gradient direction obtained by performing differentiation processing while scanning radially from the center of gravity position of the component image,
The second gradient direction of each edge point is two edge points that are opposite to each other about the target edge point, and the first gradient direction is substantially equal to the first gradient direction of the target edge point. It is a normal vector for a straight line connecting edge points,
The part shape recognition according to claim 6, wherein an edge point having a gradient direction similar to a normal vector with respect to the target side is selected in consideration of both the first gradient direction and the second gradient direction of each edge point. apparatus. Distribution creation means
Based on the equation obtained by the Hough transform equation and the normal vector for each side, the distance between the straight line passing through the edge point and parallel to the side of the class to which the edge point belongs, and the centroid position of the component image, Means to find for each edge point, distance between the straight line passing through the edge point and parallel to the side to which the edge point belongs, and the centroid position of the component image, and from the centroid position of the component image to the side of the class to which the edge point belongs A means for creating a distribution of distances between edges corresponding to the class and edge points belonging to the class for each class based on the obtained difference, for each edge point
The part shape recognition apparatus according to claim 1, further comprising: The recognition means is
Means for classifying each edge point into four corner parts and four edge parts;
Edge shape recognition means for recognizing edge shape based on distance distribution for each edge portion created by distribution creation means, and corner shape for recognizing corner shape based on distance distribution for each corner portion created by distribution creation means Recognition means,
The component shape recognition device according to claim 1, comprising: The edge shape recognition means sets a threshold value for determining whether or not an edge point belonging to the target edge part is located on a side corresponding to the target edge part as a distance in a direction orthogonal to the side, The component shape recognition apparatus according to claim 9, wherein the edge shape of the target edge portion is recognized by comparing a distance between each edge point belonging to the target edge portion and the side with a threshold value. The edge shape recognition means uses a distance distribution for a plurality of edge portions obtained from a plurality of component images as an input pattern, a signal representing an edge shape corresponding to the input pattern as a teacher signal, and a learned neural network, By inputting the distance distribution for the target edge portion created by the distribution creation means, the edge shape of the target edge portion is recognized,
The corner shape recognition means uses a distance distribution for a plurality of corner portions obtained from a plurality of component images as an input pattern, and a signal representing the corner shape corresponding to the input pattern as a teacher signal to the learned neural network, The component shape recognition apparatus according to claim 9, wherein the corner shape of the target corner portion is recognized by inputting a distance distribution with respect to the target corner portion created by the distribution creating means.

Images