JP4123587B2 - Image information processing apparatus and method - Google Patents

Description

【００３２】
式（３）においてｎは動きクラスタップ数であり、例えばｎ＝６と設定することができる。そして、ｐａｒａｍの値と、予め設定されたしきい値とを比較することによって動きの指標である動きクラス値が決定される。例えば、ｐａｒａｍ≦２の場合には動きクラス値０、２＜ｐａｒａｍ≦４の場合には動きクラス値１、４＜ｐａｒａｍ≦８の場合には動きクラス値２、ｐａｒａｍ＞８の場合には動きクラス値３というように動きクラス値が生成される。動きクラス値０が動きが最小（静止）であり、動きクラス値１、２、３となるに従って動きが大きいものと判断される。なお、動きベクトルを検出し、検出した動きベクトルに基づいて動きクラスを検出しても良い。
【００３３】
次に、予測係数を生成するための処理について説明する。図４に、予測係数の算出に係る処理系の構成の一例を示す。出力画像信号と同じ信号形式を有する既知の信号（例えば５２５ｐ信号）が間引きフィルタ２０、および正規方程式加算回路２７に供給される。間引きフィルタ５１は、水平方向および垂直方向で画素数がそれぞれ１／２とされ、全体として供給される信号の１／４の画素数を有するＳＤ信号（例えば５２５ｉ信号）を生成する。
【００３４】
かかる処理として、例えば、入力する画像信号について垂直方向の周波数が１／２になるように垂直間引きフィルタによって画素を間引き、さらに、水平方向の周波数が１／２になるように水平間引きフィルタによって画素を間引く等の処理が行われる。間引きフィルタ５１が生成するＳＤ信号がタップ選択回路２１、２２、２３に供給される。間引きフィルタ２０の特性を変えることによって学習の特性を変え、それによって、変換して得られる画像の画質を制御することができる。
【００３５】
タップ選択回路２１は、予測タップを選択し、選択した予測タップを正規方程式加算回路２７に供給する。また、タップ選択回路２２は、予測タップを選択し、選択した予測タップを空間クラス検出回路２４に供給する。一方、タップ選択回路２３は、予測タップを選択し、選択した予測タップを動きクラス検出回路２５に供給する。
【００３６】
空間クラス検出回路２４は、供給される空間クラスタップに基づいて空間クラス値を検出し、検出した空間クラス値をクラス合成回路２６に供給する。また、動きクラス検出回路２４は、供給される動きクラスタップに基づいて動きクラス値を検出し、検出した動きクラス値をクラス合成回路２６に供給する。クラス合成回路２６は、供給される空間クラス値および動きクラス値を合成し、合成したクラス値を正規方程式加算回路２７に供給する。
【００３７】
正規方程式加算回路２７は、予測係数を解とする正規方程式を解くための計算処理に使用されるデータを算出する。すなわち、正規方程式加算回路２７は、入力ＳＤ信号、タップ選択回路２１の出力、およびクラス合成回路２６の出力に基づいて加算処理を行うことにより、ラインデータｙ１，ｙ２の予測生成に使用される予測係数を解とする正規方程式を解くために必要なデータを算出する。
【００３８】
正規方程式加算回路２７が算出したデータが予測係数決定回路２８に供給される。予測係数決定回路２８は、供給されるデータに基づいて正規方程式を解くための計算処理を行い、ラインデータｙ１，ｙ２の予測生成に使用される予測係数を算出する。算出される予測係数が係数メモリ２９に供給され、記憶される。
【００３９】
ここで、正規方程式について説明する。上述したように、ｎ個の予測タップを使用して、ラインデータｙ１，ｙ２を構成する各画素は、上述の式（１）によって順次予測生成される。式（１）において、学習前は予測係数ｗ₁ ，‥‥，ｗ_n が未定係数である。学習は、クラス毎に複数の入力画像信号データに対して行う。かかる入力画像信号データの総数をｍと表記する場合、式（１）に従って、以下の式（４）が設定される。
【００４０】
ｙ_k ＝ｗ₁ ×ｘ_k1＋ｗ₂ ×ｘ_k2＋‥‥＋ｗ_n ×ｘ_kn （４）
（ｋ＝１，２，‥‥，ｍ）
ｍ＞ｎの場合、予測係数ｗ₁ ，‥‥，ｗ_n は一意に決まらないので、誤差ベクトルｅの要素ｅ_k を以下の式（５）で定義して、式（６）によって定義される誤差ベクトルｅを最小とするように予測係数を定めるようにする。すなわち、いわゆる最小２乗法によって予測係数を一意に定める。
【００４１】
ｅ_k ＝ｙ_k −｛ｗ₁ ×ｘ_k1＋ｗ₂ ×ｘ_k2＋‥‥＋ｗ_n ×ｘ_kn｝ （５）
（ｋ＝１，２，‥‥ｍ）
【００４２】
【数２】

【００４３】
式（６）のｅ² を最小とする予測係数を求めるための実際的な計算方法としては、ｅ² を予測係数ｗ_i (i=1,2‥‥）で偏微分し（式（７））、ｉの各値について偏微分値が０となるように各予測係数ｗ_i を定めれば良い。
【００４４】
【数３】

【００４５】
式（７）から各予測係数ｗ_i を定める具体的な手順について説明する。式（８）、（９）のようにＸ_ji，Ｙ_i を定義すると、式（７）は、式（１０）の行列式の形に書くことができる。
【００４６】
【数４】

【００４７】
【数５】

【００４８】
【数６】

【００４９】
式（１０）が一般に正規方程式と呼ばれるものである。正規方程式加算回路２７は、クラス合成回路２６から供給されたクラス情報、予測タップ選択回路２１から供給される予測タップ、および入力画像信号に基づいて、正規方程式データ、すなわち、式（８）、（９）に従うＸ_ji，Ｙ_i の値を算出する。そして、算出した正規方程式データを予測係数決定部２８に供給する。予測係数決定部２８は、正規方程式データに基づいて、掃き出し法等の一般的な行列解法に従って正規方程式を解くための計算処理を行って予測係数ｗ_i を算出する。
【００５０】
上述したような一般的な画像情報変換処理において、より精度の高い予測生成を行うためには、空間クラス分類を行うために使用されるタップの数を増やし、より細かな空間パターンの分類を行えば良い。しかしながら、そのようにすると、クラス数が大幅に増大してしまうという問題が生じる。そこで、この発明は、空間対称性を利用することにより、クラスを統合してクラス数を削減するようにしたものである。
【００５１】
この発明の一実施形態の全体的な構成について説明する。図５に、一実施形態における画像情報処理系の構成の一例を示す。以下の説明は、入力ＳＤ信号としての５２５ｉ信号を、出力画像信号としての５２５ｐ信号に変換する画像情報変換処理を行う場合を前提としたものであるが、他の画像信号形式の入力ＳＤ信号／出力画像信号間の変換を行う場合にも、この発明を適用することができる。図４において、図１等を参照して上述した一般的な画像情報処理系中の構成要素と同様な構成要素には、同一の符号を付した。
【００５２】
ここでは、タップ選択回路１の後段にタップ位置組み替え回路１０１が設けられている。また、空間クラス検出回路１０２の後段に対称性判定・クラス値変換回路１０２が設けられている。対称性判定・クラス値変換回路１０２は、空間クラス検出回路８から供給される空間クラス値に基づいて、後述するような空間対称性および／または振幅対称性を判定し、判定結果をタップ位置組み替え回路１０１に供給すると共に、判定結果に対応するクラス値変換処理を行う。
【００５３】
クラス値変換処理された空間クラス値がクラス合成回路１０に供給される。これにより、クラス合成回路１０の出力は、空間対称性および／または振幅対称性を利用したクラス統合の結果を反映するものとなる。後述するように、かかるクラス値変換処理は、空間対称性および／または振幅対称性を利用したクラス統合に対応する処理であり、従って、かかるクラス値変換処理によってクラス数が削減される。
【００５４】
タップ位置組み替え回路１０１は、対称性判定・クラス値変換回路１０２から供給される空間対称性および／または振幅対称性についての判定結果に基づいて、クラスタップ位置を組み替える。そして、組み替えたクラスタップ位置に対応する予測タップを推定予測演算回路４に供給する。タップ位置組み替え回路１０１の構成の一例を図６に示す。
【００５５】
後述するように、対称性判定・クラス値変換回路１０２から、空間対称性として水平対称性（Ｈ対称性）および垂直対称性（Ｖ対称性）、並びに時間対称性（Ｔ対称性）の各対称要素についての反転の有無を示すＨ反転情報信号、Ｖ反転情報信号、Ｔ反転情報信号が対称性判定・クラス値変換回路１０２から供給される。これらの信号に従って、それぞれ、Ｈタップ位置反転回路１５１、Ｖタップ位置反転回路１５２，Ｔタップ位置反転回路１５３がタップ位置反転処理を行う。
【００５６】
次に、一実施形態において予測係数を生成するための処理について説明する。図７に、かかる処理系の構成の一例を示す。ここで、図４等を参照して上述した、予測係数を生成するための一般的な処理系中の構成要素と同様な構成要素には同一の符号を付した。空間クラス検出回路２４の後段に対称性判定・クラス値変換回路１０３が設けられる。
【００５７】
対称性判定・クラス値変換回路１０３は、空間クラス検出回路２４から供給される空間クラス情報に基づいて空間対称性および／または振幅対称性を判定し、判定結果をタップ位置組み替え回路１０４に供給すると共に、判定結果に対応するクラス値変換処理を行う。そして、クラス値変換処理によって生成したクラス値を、クラス合成回路２６に供給する。これにより、クラス合成回路２６の出力は、空間対称性および／または振幅対称性を利用したクラス統合の結果を反映するものとなる。
【００５８】
タップ位置組み替え回路１０４は、例えば図６を参照して上述したタップ位置組み替え回路１０１と同様の構成を有し、対称性判定・クラス値変換回路１０３から供給される空間対称性に関する情報に基づいてクラスタップ位置を組み替える。そして、組み替えたクラスタップ位置に対応する予測タップを正規方程式加算回路２７に供給する。
【００５９】
これにより、正規方程式加算回路２７、予測係数決定回路２８の出力は、空間対称性および／または振幅対称性を利用したクラス統合の結果を反映するものとなる。従って、係数メモリ２９には、空間対称性および／または振幅対称性を利用したクラス統合の結果を反映する予測係数データが記憶されることになる。図５を参照して上述した画像情報変換処理系の動作に先立って、係数メモリ２９の記憶内容が係数メモリ１１にロードされる。
【００６０】
なお、タップ選択回路２１が出力する予測タップの個数は、画像情報変換処理系において使用される予測タップの個数より大きいものとされる。従って、図７中の予測係数決定部２８では、クラス毎により多くの係数が求まる。このようにして求まった予測係数の中で、絶対値が大きいものから順に使用する数の予測係数が選択される。
【００６１】
なお、図５に示したような画像信号変換処理系、図７に示したような予測係数の決定に係る処理系は、各々別個に設けても良いし、スイッチ等を適宜設けることによって構成要素を共通に使用するように構成しても良い。但し、これらの処理系を別個に設ける場合には、学習の有効性を担保するため、同一の機能を有する構成要素は、同一の動作特性を有するものを使用する必要がある。例えばタップ選択回路１、２１は、同一の動作特性を有する必要がある。構成要素を共通に使用する場合には、スイッチング等に起因して装置全体の処理が複雑化するが、装置全体に関わる回路規模、コスト等の削減に寄与することができる。
【００６２】
次に、ｙ１の空間クラス分類に使用されるタップ配置の一例を図８Ａに示す。また、ｙ１内の画素の予測生成に使用されるタップ配置の一例を図８Ｂに示す。さらに、ｙ２の空間クラス分類に使用されるタップ配置の一例を図９Ａに示し、ｙ２内の画素の予測生成に使用されるタップ配置の一例を図９Ｂに示す。以下の説明は、図２等を参照して上述した画像情報変換処理系中の空間クラス検出回路８において、図８、図９に示すようなタップ配置によって規定されるクラス分割点を使用して１ビットＡＤＲＣを行うことによってクラス分類を行う場合を前提として行う。
【００６３】
図８、図９における座標軸のとり方は以下のようなものである。すなわち、フレーム内の水平方向を、図中の水平方向の座標軸（以下、ｈ軸と表記する）に対応させる。また、フレーム内の垂直方向を、図中の垂直方向の座標軸（以下、ｖ軸と表記する）に対応させる。さらに、時間方向、すなわちフレームの時間的順序を示す方向を、図中の手前から奥に向かう座標軸（以下、ｔ軸と表記する）に対応させる。これ以降の説明において参照する、空間対称性の説明に係る各図においても、同様な座標軸を用いる。図８Ａ、図８Ｂにおいて、クラス分類用タップａ，ｂ，ｅ，ｈ，ｉは、いずれもｖ−ｔ面上に位置している。また、図９Ａ、図９Ｂにおいて、クラス分類用タップｂ，ｃ，ｄ，ｉ，ｊ，ｋは、いずれもｈ−ｔ面上に位置している。
【００６４】
対称性判定・クラス値変換回路１０２においてなされる、空間対称性を利用したクラス統合について説明する。以下の説明は、ｙ１の空間クラス分類に使用される空間クラスタップ配置の一例（図８Ａ参照）を前提として行うが、他の空間クラスタップ配置を用いる場合にも適用できる。Ｈ対称性、すなわちｖ−ｔ面に関する対称性を例として、空間対称性を利用した空間クラスの統合について説明する。空間クラス検出回路４の出力が図１０Ａに示す空間パターンに分類される場合の予測係数を図１０Ｂに示す。
【００６５】
ここで、図１０Ａにおける各タップａ〜ｉにおいては、白丸が例えば'1' 、黒丸が例えば'0' に対応する。また、ａ〜ｉに対応する、図１０Ｂ中の予測係数は、それぞれｋ_a 〜ｋ_i である。一方、図１１Ａに、図１０ＡをＨ反転した空間パターンの一例を示す。ここで、Ｈ反転は、Ｈ対称性における対称要素であるｖ−ｔ面に関して反転する（鏡像の関係とする）空間反転操作である。ｖ−ｔ面上に無いタップｄ，ｆの符号が図１０Ａと図１１Ａでは反転している。図１１Ａに示す空間パターンに対応する予測係数ｋ_a ’〜ｋ_i ’を図１１Ｂに示す。
【００６６】
図４等を参照して上述したように、出力画像信号と同じ信号形式を有する既知の複数個の画像信号に基づいて、線形予測推定式を解く演算処理を行うことによって予測係数が算出される。このような算出方法のため、ある任意の空間パターンとかかる任意の空間パターンをＨ反転して得られる空間パターンに分類されるクラスに対応して使用されるべき予測係数は、かかる任意の空間パターンに分類されるクラスに対応して使用されるべき予測係数と、ｖ−ｔ面に関して鏡像の関係となる。
【００６７】
上述の例においては、図１１Ａに示した空間パターンに分類されるクラスに対応して使用されるべき予測係数は、図１０Ａに示した空間パターンに分類されるクラスに対応して使用されるべき予測係数に対して、ｖ−ｔ面に関して鏡像の関係となる。従って、図１１Ｂに示す予測係数ｋ’_a 〜ｋ’_i は、図１０Ｂに示した予測係数ｋ_a 〜ｋ_i をＨ反転することによって得ることができる。すなわち、図１０Ａおよび図１１Ａの空間パターンに分類されるクラスについては、共通の係数値を状況に応じてＨ反転することにより、予測係数を得ることができる。より一般的には、以下のことがわかる。すなわち、空間パターン分割に使用されるタップの位置関係が互いにＨ対称の関係を成す空間パターンに分類されるクラスは、統合することができる。
【００６８】
上述したような、Ｈ対称性を利用したクラス統合と同様にして、Ｖ対称性すなわちｈ−ｔ面に関する対称性，およびＴ対称性すなわちｖ−ｈ面に関する対称性を利用してクラス統合を行うことも可能である。このことについて図１２を参照して説明する。図１２Ａが空間パターンの一例を示す。また、図１２Ｂ，図１２Ｃ，図１２Ｄに、かかる一例に対して、それぞれ、Ｈ反転、Ｖ反転、Ｔ反転が施されてなる空間パターンの例を示す。
【００６９】
図１２Ａに示す空間パターンに分類されるクラスと、図１２Ｂに示す空間パターンに分類されるクラスは、互いにＨ対称の関係にあるので、上述したようにクラス統合することができる。同様に、図１２Ａに示す空間パターンに分類されるクラスと、図１２Ｃに示す空間パターンに分類されるクラスとは、互いにＶ対称の関係にあるのでクラス統合することができ、さらに、図１２Ａに示す空間パターンに分類されるクラスと、図１２Ｄに示す空間パターンに分類されるクラスとは、互いにＶ対称の関係にあるのでクラス統合することができる。
【００７０】
また、ｖ−ｔ面、ｈ−ｔ面、ｈ−ｖ面に関する対称性を組合わせてなる対称性を有する空間パターンに分類されるクラスについても、クラス統合が可能である。図１２Ａに示した空間パターンの一例に対して、ＨＶ対称性（Ｈ対称性とＶ対称性との組合わせ）を有する空間パターンの例を図１３Ａに示す。また、ＴＨ対称（Ｔ対称性とＨ対称性との組合わせ）を図１３Ｂに示す。さらに、ＶＴ対称（Ｖ対称性とＴ対称性との組合わせ）を有する空間パターンの例を図１３Ｃに示す。一方、ＨＶＴ対称（Ｈ対称性、Ｖ対称性、Ｔ対称性の組合わせ）を有する空間パターンの例を図１３Ｄに示す。
【００７１】
例えば、図１３Ａに示した空間パターンに分類されるクラスに対応して使用されるべき予測係数は、図１２Ａに示した空間パターンに分類されるクラスに対応して使用されるべき予測係数に対して、ＨＶ対称性における対称要素、すなわちｖ−ｔ面およびｈ−ｔ面に関して鏡像の関係となる。従って、図１３Ａに示す予測係数は、図１２Ａに示した空間パターンに分類されるクラスに対応して使用されるべき予測係数をｖ−ｔ面およびｈ−ｔ面に関して反転することによって得ることができる。同様に、例えば、図１３Ｄに示す予測係数は、図１２Ａに示した空間パターンに分類されるクラスに対応して使用されるべき予測係数をＨＶＴ対称における対称要素、すなわちｖ−ｔ面，ｈ−ｔ面およびｈ−ｖ面に関して反転することによって得ることができる。
【００７２】
次に、振幅対称性、すなわち振幅方向の対称性を利用したクラス統合について説明する。まず、図１４Ａに示す空間パターンに対応する予測係数を図１４Ｂに示す。一方、図１４Ａに示した空間パターンに対して、振幅反転した、すなわち各入力画素の符号が反対である空間パターンを図１５Ａに示す。図１５Ａに示した空間パターンは、図１４Ａに示した空間パターンを有する入力画像信号を反転させることによって、より具体的には入力画像信号の最大値から当該入力画像信号を引くことによって生成することができる。
【００７３】
上述したように、予測係数は、入力画像信号における各々の空間パターン毎に、入力画像信号と予測生成される画素の間の不変的な関係に対応するものとして求められる。このため、任意の空間パターンを振幅方向に反転させることによって得られる空間パターンから予測生成される画素値は、かかる任意の空間パターンから予測生成される画素値を反転した値となる。従って、図１４Ａに示した空間パターンを有する入力画像信号を反転させ、図１４Ｂに示した予測係数を用いて式（１）による予測生成演算を行うことにより、図１５Ａに示した空間パターンを有する入力画像信号に対応する画素値の予測生成を行うことができる。
【００７４】
また、式（１）は１次線形結合式であり、また、学習時に入力される、出力画像信号と同一の信号形式を有する画像信号と推定画素データの振幅とは対等であることから、予測係数の総和は１となる。このため、以下の式（１１）に示すように入力画像信号の反転および推定画素の反転は相殺されることになり、結果として、図１５Ａに示した空間パターンに対応する予測係数として、図１４Ａに示した空間パターンに対応する予測係数をそのまま用いることができる。すなわち、図１５Ｂ中のｋ’’_a 〜ｋ’’_i は、図１４Ｂ中のｋ_a 〜ｋ_i と一致することになる。
【００７５】
【数７】

【００７６】
以上の説明からわかるように、互いに振幅反転の関係にある空間パターンに分類されるクラスは、統合することができる。
【００７７】
また、図１２等を参照して上述した、Ｈ対称性、Ｖ対称性，Ｔ対称性の内の１つ、または２つ以上を組合わせてなる対称性と、振幅対称性とを組合わせてなる対称性を有する空間パターンに分類されるクラスについても、クラス統合が可能である。まず、Ｈ対称性、Ｖ対称性，Ｔ対称性の内の１つと、振幅対称とを組合わせてなる対称性を有する空間パターンの例についてを図１６を参照して説明する。図１６Ａには、図１４Ａに示した空間パターンの一例に振幅反転が施されてなる空間パターンの例を示す。
【００７８】
また、図１６Ｂには、図１４Ａに示した空間パターンの一例に対して、振幅反転とＨ反転とが施されてなる空間パターンの例を示す。さらに、図１４Ａに示した空間パターンの一例に対して振幅反転とＶ反転とが施されてなる空間パターンの例、および図１４Ａに示した空間パターンの一例に対して振幅反転とＴ反転とが施されてなる空間パターンの例を、それぞれ、図１６Ｃおよび図１６Ｄに示す。
【００７９】
さらに、Ｈ対称性、Ｖ対称性，Ｔ対称性の内の２つ以上と、振幅対称とを組合わせてなる対称性を有する空間パターンの例について図１７を参照して説明する。図１７Ａには、図１４Ａに示した空間パターンの一例に対して、振幅反転とＨＶ反転とが施されてなる空間パターンの例を示す。また、図１７Ｂには、図１４Ａに示した空間パターンの一例に対して、振幅反転とＴＨ反転とが施されてなる空間パターンの例を示す。さらに、図１７Ｃには、図１４Ａに示した空間パターンの一例に対して振幅反転とＶＴ反転とが施されてなる空間パターンの例を示す。よりさらに、図１７Ｄに、図１４Ａに示した空間パターンの一例に対して、振幅反転とＶＴ反転とが施されてなる空間パターンの例を示す。
【００８０】
上述したような各対称性に係る空間クラス値変換処理について、図１８、図１９を参照して説明する。図１８Ａには、入力画像信号の空間パターンの一例を示す。また、図１８Ａに示す空間パターンの一例に、Ｈ反転，Ｖ反転，Ｔ反転が施されてなる空間パターンを、それぞれ、図１８Ｂ，図１８Ｃ，図１８Ｄに示す。ここで、黒塗りを付したタップの値が例えば'1' 、白のタップの値が例えば'0' を示す。
【００８１】
これらの各タップの値の総体として検出される空間クラス値は、例えば各タップＰ，Ｑ，Ｒ，Ｓ，Ｔの値をこの順に並べた５ビットの値として定義することができる。かかる定義によれば、図１８Ａに示した一例についての空間クラス値＝００１１１となり、１０進表現では７となる。また、図１８Ｂに示した一例についての空間クラス値＝０１１０１となり、１０進表現では１３となる。同様に、図１８Ｃ、図１８Ｄに示した空間パターンについての空間クラス値は、それぞれ、１０１１０（１０進表現では２２）、１１１００（１０進表現では２８）となる。勿論、空間クラス値の定義は、上述したものに限定されない。
【００８２】
同様に、入力画像信号の空間パターンの他の例を図１９Ａに示す。さらに、かかる空間パターンの他の例に、Ｈ反転，Ｖ反転，Ｔ反転が施されてなる空間パターンを、それぞれ、図１９Ｂ，図１９Ｃ，図１９Ｄに示す。ここでも図１８の場合と同様に空間クラス値が検出されるものとすると、空間クラス値は、図１８Ａ，図１９Ｂ，図１９Ｃ，図１９Ｄに示した空間パターンについて、それぞれ１１１００（１０進表現では２８）、１０１１０（１０進表現では２２）、０１１０１（１０進表現では１３）、００１１１（１０進表現では７）となる。
【００８３】
次に、空間クラス検出回路８（図５参照）から供給される空間クラス値から、上述したような空間対称性および／または振幅対称性を検出する、対称性判定・クラス値変換回路１０２について、図２０を参照して詳細に説明する。空間クラス値は、最小値判定回路２１６、振幅反転回路２０８，およびＨ反転回路２０１、Ｖ反転回路２０２、Ｔ反転回路２０３、ＨＶ反転回路２０４、ＶＴ反転回路２０５、ＴＨ反転回路２０６、ＨＴＶ反転回路２０７に供給される。
【００８４】
最小値判定回路２１６、振幅反転回路２０８については後述する。Ｈ反転回路２０１、Ｖ反転回路２０２、Ｔ反転回路２０３、ＨＶ反転回路２０４、ＶＴ反転回路２０５、ＴＨ反転回路２０６、ＨＴＶ反転回路２０７は、供給されるクラス値に対して、それぞれ、Ｈ反転処理、Ｖ反転処理、Ｔ反転処理、ＨＶ反転処理、ＶＴ反転処理、ＴＨ反転処理、ＨＴＶ反転処理を行う。これら７種類の空間反転処理の結果として得られるクラス値は、それぞれ、最小値判定回路２１６と、振幅反転処理を行う構成とに供給される。
【００８５】
振幅反転処理を行う構成として、Ｈ反転回路２０１、Ｖ反転回路２０２、Ｔ反転回路２０３、ＨＶ反転回路２０４、ＶＴ反転回路２０５、ＴＨ反転回路２０６、ＨＴＶ反転回路２０７の後段にそれぞれ振幅反転回路２０９、２１０、２１１、２１２、２１３、２１４、２１５が設けられている。また、上述した振幅反転回路２０８は、空間クラス検出回路８から供給されるクラス値に対して振幅反転処理を行う。振幅反転回路２０８、２０９、２１０、２１１、２１２、２１３、２１４、２１５の出力が最小値判定回路に供給される。
【００８６】
以上のような構成により、７種類の空間反転処理の結果として得られるクラス値および空間クラス検出回路８から供給されるクラス値の計８種類のクラス値と、それら８種類のクラス値に振幅反転処理が施されてなる８種類のクラス値、従って全部で１６種類のクラス値が最小値判定回路２１６に供給される。最小値判定回路２１６は、供給される１６種類のクラス値の内で最小のものを判定し、判定結果である最小のクラス値を、代表クラス値として再アドレッシング処理回路２１７に供給する。なお、代表クラス値は、必ずしも最小のクラス値には限定されず、例えば最大のクラス値としても良い。但し、最小のクラス値を用いる場合には、代表クラス値を表現するための情報量を最小にすることができるという利点がある。
【００８７】
また、最小値判定回路２１６は、空間クラス検出回路８から供給されるクラス値から最小のクラス値を得るために施される反転処理を示す情報を生成する。このような情報としては、最小のクラス値を得るための処理において、Ｈ反転処理、Ｖ反転処理、Ｔ反転処理の各々が施されたか否かを示す、例えば各１ビットのＨ反転情報信号、Ｖ反転情報信号、Ｔ反転情報信号を生成するようにすれば良い。これらの信号がタップ位置組み替え回路１０１（図５参照）に供給される。
【００８８】
上述したようにして生成される代表クラス値は、空間クラス検出回路８から順次供給される各クラス値に対して離散的な値をとる。このため、再アドレッシング処理回路２１７は、空間クラス検出回路８から順次供給される各クラス値の内で欠落する値をスキップして連続したクラス値に変換する処理を行う。再アドレッシング処理回路２１７の出力が対称性判定・クラス値変換回路１０２による変換処理の結果として生成される最終的な空間クラス値として、クラス合成回路１０（図５参照）に供給される。従って、クラス合成回路１０の出力は、対称性判定・クラス値変換回路１０２による、空間対称性を利用したクラス値変換処理の結果に基づくものとなる。
【００８９】
以上のようなクラス値変換により、空間クラス値に関しては、１個の代表クラス値によって特定される係数メモリ１１内の予測係数データが１６種類の空間パターンに対応することになる。従って、１６種類の空間パターンが１個のクラスに統合されることになる。
【００９０】
また、タップ位置組み替え回路１０１は、供給されるＨ反転情報信号、Ｖ反転情報信号、Ｔ反転情報信号に基づいて、推定予測演算回路４においてなされる推定予測演算に使用されるタップデータの位置を組み替える。これにより、１個のクラスに類別される１６種類の空間パターンに対応して、タップデータの位置が的確に組み替えられる。従って、対称性判定・クラス値変換回路１０２による変換処理の結果に基づいて（すなわち、各代表クラス値に基づいて）生成されるクラス合成回路１０の出力に従って係数メモリ１１から出力される予測係数データと、タップデータとの推定予測演算における整合性が担保される。従って、ラインデータｙ１，ｙ２の予測生成を的確に行うことができる。
【００９１】
次に、線順序変換回路６が行うライン倍速処理について説明する。上述したようにして推定予測演算回路４、５が生成する５２５ｐ信号の水平周期は、画像情報変換処理がなされる前の５２５ｉ信号の水平周期と同一である。線順序変換回路６は、水平周期を２倍とするライン倍速処理を行う。図２１は、ライン倍速処理をアナログ波形を用いて示すものである。推定予測演算回路４、５によって、ラインデータｙ１およびｙ２が同時に予測生成される。
【００９２】
ラインデータｙ１には、順にａ１，ａ２，ａ３‥‥のラインが含まれ、ラインデータｙ２には、順にｂ１，ｂ２，ｂ３‥‥のラインが含まれる。線順序変換回路５００内には、ラインダブラ（図示せず）とスイッチング回路（図示せず）とが設けられている。ラインダブラは、各ラインのデータを時間軸方向に１／２に圧縮し、圧縮されたデータをスイッチング回路に供給する。スイッチング回路は、供給されるデータを交互に選択して出力する。このようにして、線順次出力（ａ０，ｂ０，ａ１，ｂ１，‥‥）が形成される。
なお、この一実施形態では、現存ライン上の出力画素値（ラインデータＬ１）と、作成ライン上の出力画素値（ラインデータＬ２）とを並列構成でもって作成するようにしている。これに対し、メモリを追加し、回路の動作速度が速ければ時分割処理でラインデータｙ１およびｙ２を順に生成すると共に、ライン倍速処理を行うように構成しても良い。
【００９３】
上述したこの発明の一実施形態は、対称性判定とクラス値変換とを一括して行うようにしたものである。これに対し、対称性判定およびクラス値変換を、対称要素毎に行うようにした、この発明の他の実施形態も可能である。図２２に、他の実施形態における画像情報変換処理系の一例を示す。ここで、図５中の構成要素と同様な構成要素には、同一の符号を付した。
【００９４】
空間クラス検出回路８が生成する空間クラス値は、対称性判定回路１０５と、クラス値変換回路１０６とに供給される。対称性判定回路１０５は、例えば上述した一実施形態中の対称性判定・クラス値変換回路１０２と同様な構成を有し、Ｈ反転情報信号、Ｖ反転情報信号，Ｔ反転情報信号および振幅反転情報信号を出力する。ここで、振幅反転の有無は、タップ位置組み替えを行うためには不必要な情報なので、上述した一実施形態中の対称性判定・クラス値変換回路１０２（図５参照）においては振幅反転情報信号は出力されない。しかし、クラス値変換処理を対称性判定と別個に行う他の実施形態においては、対称性判定回路１０５が振幅反転情報信号をも出力するようになされる。
【００９５】
対称性判定回路１０５の出力の内、Ｈ反転情報信号、Ｖ反転情報信号，Ｔ反転情報信号および振幅反転情報信号がクラス値変換処理回路１０６に供給される。また、Ｈ反転情報信号、Ｖ反転情報信号，Ｔ反転情報信号がタップ位置組み替え回路１０１に供給される。タップ位置組み替え回路１０１は、供給される各反転情報信号を参照してタップ位置組み替え処理を行う。
【００９６】
クラス値変換処理回路１０６の構成の一例を図２３に示す。空間クラス検出回路８から供給されるクラス値がＨ対称クラス値変換回路１１０に供給される。Ｈ対称クラス値変換回路１１０の後段には、Ｖ対称クラス値変換回路１１１、Ｔ対称クラス値変換回路１１２、振幅対称クラス値変換回路１１３が順に設けられる。Ｈ対称クラス値変換回路１１０、Ｖ対称クラス値変換回路１１１、Ｔ対称クラス値変換回路１１２、振幅対称クラス値変換回路１１３には、それぞれ、Ｈ反転情報信号、Ｖ反転情報信号，Ｔ反転情報信号および振幅反転情報信号が供給される。
【００９７】
Ｈ対称クラス値変換回路１１０、Ｖ対称クラス値変換回路１１１、Ｔ対称クラス値変換回路１１２、振幅対称クラス値変換回路１１３は、各々に供給される反転情報信号を参照して、クラス値変換処理を行う。これらのクラス値変換処理の総体により、１６種類の空間パターン中で例えば最小のクラス値が代表クラス値として生成される。
【００９８】
振幅対称クラス値変換回路１１３の後段には、再アドレッシング処理回路１１４が設けられる。再アドレッシング処理回路１１４は、上述した一実施形態中の再アドレッシング処理回路２１７（図２０参照）と同様に、空間クラス検出回路８から順次供給される各クラス値の内で欠落する値をスキップして連続したクラス値に変換する処理を行う。再アドレッシング処理回路１１４の出力が対称性判定・クラス値変換回路１０５による変換処理の結果としての最終的な空間クラス値として出力される。
【００９９】
また、この発明の一実施形態および他の実施形態は、５２５ｉ信号を５２５ｐ信号に変換するものであるが、５２５本のライン数は、一例であって、他のライン数であってもこの発明を適用することができる。例えば、入力ＳＤ信号における水平方向の画素数に対して、２倍以外の画素数を含むラインからなる出力画像信号を予測生成するようにしても良い。より具体的には、出力画像信号として１０５０ｉ信号、すなわち走査線数が１０５０本でインターレス方式の画像信号を予測生成する画像情報変換処理を行う場合に、この発明を適用することができる。
【０１００】
【発明の効果】
上述したように、この発明は、入力画像信号から走査線構造の異なる出力画像信号を形成する画像情報変換処理を行うに際して、画像データが有する振幅対称性および／または空間対称性に基づいてなされるクラス統合に対応するクラス値変換処理を施して、クラス数を削減するようにしたものである。
【０１０１】
このため、クラス分類適応処理による画像情報変換処理のきめ細かさを損なうこと無く、処理系内で扱われるクラスの数を削減することができるので、より少ないクラス数で効率的な処理を行うことができる。
【０１０２】
従って、装置内のメモリ資産の記憶容量が小さい場合にも的確な処理を行うことが可能となるので、メモリ資産の記憶容量を削減することができる。このため、装置全体に関する、小型化、低コスト化等に寄与することができる。
【図面の簡単な説明】
【図１】この発明の一実施形態によってなされる画像情報変換処理における画素の配置の一例を示す略線図である。
【図２】この発明の一実施形態における、空間クラスタップ配置の一例を示す略線図である。
【図３】一般的な画像情報変換処理系の構成の一例を示すブロック図である。
【図４】一般的な予測係数算出処理系の構成の一例を示すブロック図である。
【図５】この発明の一実施形態における画像情報変換処理系の構成の一例を示すブロック図である。
【図６】この発明の一実施形態における画像情報変換処理系の構成の一例中の一部の構成を示すブロック図である。
【図７】この発明の一実施形態における予測係数算出処理系の構成の一例を示すブロック図である。
【図８】ラインデータｙ１についての空間クラスタップの配置の一例および予測タップの配置の一例を示す略線図である。
【図９】ラインデータｙ２についての空間クラスタップの配置の一例および予測タップの配置の一例を示す略線図である。
【図１０】空間パターンの一例と、それに対応する予測係数の一例とについて説明するための略線図である。
【図１１】図１１に示した空間パターンがＨ反転された場合の、空間パターンと予測係数とについて説明するための略線図である。
【図１２】Ｈ対称性、Ｖ対称性、Ｔ対称性について説明するための略線図である。
【図１３】Ｈ対称性、Ｖ対称性、Ｔ対称性の内の２つまたは３つを組合わせてなる対称性について説明するための略線図である。
【図１４】空間パターンの一例と、それに対応する予測係数の一例とについて説明するための略線図である。
【図１５】図１４に示した空間パターンが振幅反転された場合の、空間パターンと予測係数とについて説明するための略線図である。
【図１６】Ｈ対称性、Ｖ対称性，Ｔ対称性の内の１つと、振幅対称とを組合わせてなる対称性を有する空間パターンについて説明するための略線図である。
【図１７】Ｈ対称性、Ｖ対称性，Ｔ対称性の内の２つ以上と、振幅対称とを組合わせてなる対称性を有する空間パターンについて説明するための略線図である。
【図１８】対称性判定の一例について説明するための略線図である。
【図１９】対称性判定の他の例について説明するための略線図である。
【図２０】この発明の一実施形態における画像情報変換処理系の構成の一例中の他の一部の構成を示すブロック図である。
【図２１】ライン倍速処理について説明するための略線図である。
【図２２】この発明の他の実施形態における画像情報変換処理系の構成の一例を示すブロック図である。
【図２３】この発明の他の実施形態における画像情報変換処理系の構成の一例中の一部の構成を示すブロック図である。
【符号の説明】
１０１・・・タップ位置組み替え回路、１０２・・・対称性判定・クラス値変換回路、２１７・・・再アドレッシング処理回路[0001]
BACKGROUND OF THE INVENTION
The present invention provides image information suitable for use in, for example, a television receiver. processing apparatus And methods About.
[0002]
[Prior art]
For the purpose of obtaining an image signal having a higher resolution than the input image signal, an image information conversion process for forming an output image signal having a scanning line structure different from that of the input image signal is performed. As the image information conversion process, there is a class classification adaptive process in which class division is performed according to the three-dimensional (spatio-temporal) distribution of the signal level of the input image signal, and pixel prediction is performed by referring to the class value obtained thereby. Proposed.
[0003]
In this process, in order to obtain a higher resolution, the number of pixels used for class division may be increased to classify the spatial pattern more finely. However, if this is done, the number of classes increases, so that there is a problem that a memory having a large storage capacity is required, and that arithmetic processing is complicated. If the image information conversion process is performed using a smaller number of classes without impairing the image quality of the output image, the above-described problems can be avoided.
[0004]
[Problems to be solved by the invention]
Accordingly, an object of the present invention is to provide image information capable of performing image information conversion processing using a smaller number of classes. processing apparatus And methods Is to provide.
[0005]
[Means for Solving the Problems]
According to a first aspect of the present invention, there is provided an image information conversion apparatus configured to form an output image signal having a different scanning line structure from an input image signal.
First image data selection means for selecting, from an input image signal, a pixel located in the vicinity of a predetermined attention point in time or space;
A space class detection unit that detects a level distribution pattern from image data selected by the first image data selection unit and determines a first space class value indicating a space class to which the point of interest belongs based on the detected pattern. When,
Amplitude symmetry and / or spatial symmetry of image data selected by the first image data selection means is determined, and a plurality of first spatial class values having the same amplitude symmetry and / or spatial symmetry are determined. By flipping to a position with amplitude symmetry and / or spatial symmetry Integrated , Using the smallest class value of the first class values to be integrated Class value conversion means for converting to a second class value;
Second image data selection means for selecting, from the input image signal, a pixel that includes the point of interest and is located near the point of interest in terms of time or space;
Storage means for storing prediction coefficient data determined in advance corresponding to the class indicated by the second class value;
The position of the image data selected by the second image data selection means , At a position having the same amplitude symmetry and / or spatial symmetry as inversion processing of the first space class value made by the class value conversion means An arithmetic operation for predicting and generating a pixel at the target point by linear linear combination of the inverted image data and the prediction coefficient data selected from the storage means corresponding to the second class value Processing means,
The prediction factor is
The difference between the true pixel value in the predetermined image signal having the same signal format as the output image signal and the calculated value of the linear linear combination between the image data selected by the second image data selection means is minimized. It is an image information processing apparatus characterized by being defined as follows.
[0006]
According to a second aspect of the present invention, there is provided an image information conversion apparatus configured to form an output image signal having a different scanning line structure from an input image signal.
First image data selection means for selecting, from an input image signal, a pixel located in the vicinity of a predetermined attention point in time or space;
Space class detecting means for detecting a level distribution pattern from image data selected by the first image data selecting means and determining a space class value indicating a space class to which the target point belongs based on the detected pattern;
Second image data selection means for selecting, from the input image signal, a pixel located in the vicinity of the point of interest temporally or spatially in a plurality of frames in the input image signal;
Motion class detection means for determining a motion class value representing motion based on the inter-frame difference from the image data selected by the second image data selection means;
Class combining means for generating a first class value by combining the spatial class value and the motion class value;
Amplitude symmetry and / or spatial symmetry of image data selected by the first image data selection means is determined, and a plurality of first spatial class values having the same amplitude symmetry and / or spatial symmetry are determined. By flipping to a position with amplitude symmetry and / or spatial symmetry Integrated , Using the smallest class value of the first class values to be integrated Class value conversion means for converting to a second class value;
Third image data selection means for selecting, from the input image signal, a pixel that includes the point of interest and is located near the point of interest in terms of time or space;
Storage means for storing prediction coefficient data determined in advance corresponding to the class indicated by the second class value;
The position of the image data selected by the third image data selection means , At a position having the same amplitude symmetry and / or spatial symmetry as inversion processing of the first space class value made by the class value conversion means An arithmetic operation for predicting and generating a pixel at the target point by linear linear combination of the inverted image data and the prediction coefficient data selected from the storage means corresponding to the second class value Processing means,
The prediction factor is
The difference between the true pixel value in the predetermined image signal having the same signal format as the output image signal and the calculated value of the linear linear combination between the image data selected by the third image data selection means is minimized. It is an image information processing apparatus characterized by being defined as follows.
[0007]
The invention of claim 5 is an image information conversion method in which an output image signal having a different scanning line structure is formed from an input image signal.
A first image data selection step of selecting, from the input image signal, a pixel located in the vicinity of a predetermined point of interest temporally or spatially;
A space class detecting step of detecting a level distribution pattern from the image data selected in the first image data selecting step and determining a first space class value indicating a space class to which the target point belongs based on the detected pattern. When,
Amplitude symmetry and / or spatial symmetry of image data selected by the first image data selection means is determined, and a plurality of first spatial class values having the same amplitude symmetry and / or spatial symmetry are determined. By flipping to a position with amplitude symmetry and / or spatial symmetry Integrated , Using the smallest class value of the first class values to be integrated A class value conversion step for converting to a second spatial class value;
A second image data selection step for selecting, from the input image signal, a pixel that includes the point of interest and is located near the point of interest in terms of time or space;
A storage step of storing predetermined prediction coefficient data corresponding to the class indicated by the second class value;
The position of the image data selected by the second image data selection step , At a position having the same amplitude symmetry and / or spatial symmetry as the first spatial class value inversion processing performed in the class value conversion step An arithmetic operation for predicting and generating a pixel at the target point by linear linear combination of the inverted image data and the prediction coefficient data selected from the storage means corresponding to the second class value Processing steps,
The prediction factor is
The difference between the true pixel value in the predetermined image signal having the same signal format as the output image signal and the calculated value of the linear linear combination between the image data selected by the second image data selection step is minimized. It is an image information processing method characterized by being defined as follows.
[0008]
The invention of claim 6 is an image information conversion method in which an output image signal having a different scanning line structure is formed from an input image signal.
A first image data selection step of selecting, from the input image signal, a pixel located in the vicinity of a predetermined point of interest temporally or spatially;
A space class detection step of detecting a level distribution pattern from the image data selected in the first image data selection step, and determining a space class value indicating a space class to which the target point belongs based on the detected pattern;
A second image data selection step of selecting, from the input image signal, a pixel located in the vicinity of the point of interest temporally or spatially in a plurality of frames in the input image signal;
A motion class detection step for determining a motion class value representing motion based on the inter-frame difference from the image data selected by the second image data selection step;
A class synthesis step of synthesizing the space class value and the motion class value to generate a first class value;
Amplitude symmetry and / or spatial symmetry of the image data selected by the first image data selection step is determined, and a plurality of first spatial class values having the same amplitude symmetry and / or spatial symmetry are determined. By flipping to a position with amplitude symmetry and / or spatial symmetry Integrated , Using the smallest class value of the first class values to be integrated A class value conversion step for converting to a second class value;
A third image data selection step for selecting, from the input image signal, a pixel that includes the point of interest and is located near the point of interest in terms of time or space;
A storage step of storing predetermined prediction coefficient data corresponding to the class indicated by the second class value;
The position of the image data selected by the third image data selection means , At a position having the same amplitude symmetry and / or spatial symmetry as the first spatial class value inversion processing performed in the class value conversion step An arithmetic operation for predicting and generating a pixel at the target point by linear linear combination of the inverted image data and the prediction coefficient data selected from the storage means corresponding to the second class value Processing steps,
The prediction factor is
The difference between the true pixel value in the predetermined image signal having the same signal format as the output image signal and the calculated value of the linear linear combination between the image data selected by the third image data selection step is minimized. It is an image information processing method characterized by being defined as follows.
[0011]
According to the invention as described above, based on the amplitude symmetry and / or spatial symmetry of the image data, the classes handled in the processing system without impairing the fineness of the image information conversion processing by the class classification adaptive processing. The number of can be reduced.
[0012]
For this reason, efficient conversion can be performed with a smaller number of classes.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
Prior to the description of an embodiment of the present invention, image information conversion processing that is the premise thereof will be described below. In this process, a standard resolution digital image signal (hereinafter referred to as an SD signal) is converted into a high resolution image signal (sometimes referred to as an HD signal) and output. As the SD signal at this time, for example, an interlaced image signal (hereinafter referred to as a 525i signal) having 525 lines is used. As the high-resolution image signal, for example, a progressive-type output video signal (hereinafter referred to as a 525p signal) having 525 lines is used. Further, the number of pixels in the horizontal direction in the output image signal is twice the number of pixels in the horizontal direction in the input image signal.
[0014]
In such image information conversion processing, resolution is being improved by class classification adaptation processing according to the proposal of the present applicant. The class classification adaptive processing is different from that in which high resolution signals are formed by conventional interpolation processing. In other words, the class classification adaptive processing performs class division according to the three-dimensional (spatio-temporal) distribution of the signal level of the input SD signal, and stores the predicted count value obtained by learning in advance for each class in a predetermined storage unit. This is a process of outputting an optimum estimated value by a calculation based on the prediction formula. With the class classification adaptive processing, it becomes possible to obtain a resolution higher than the resolution of the input SD signal.
[0015]
The image signal obtained by such processing will be described in detail. FIG. 1 shows an example of pixel arrangement in image information conversion processing for converting a 525i signal as an input SD signal into a 525p signal as an output image signal by enlarging a part of an image of one field. is there. Here, a large dot indicates a pixel of the 525i signal, and a small dot indicates a pixel of the 525p signal.
[0016]
Line data y1 (shown as small black dots) at the same position as the line of the 525i signal and line data y2 (shown as white small dots) between the upper and lower lines of the 525i signal are formed. As a result, a 525p signal is predicted and generated. FIG. 1 shows a pixel arrangement in an odd field of a certain frame. In the other field (even field), each line of the 525i signal and the 525p signal is spatially shifted by 0.5 lines.
[0017]
The line data y1 and y2 are predicted and generated by class classification based on pixels in the 525i signal and linear linear combination operation. An example of the pixel arrangement in the 525i signal and the positional relationship between y1 and y2 is shown in FIG. In FIG. 2, when time advances in the horizontal direction (left to right), the pixels in each field are shown side by side in the vertical direction.
[0018]
The illustrated fields are expressed as F-1 / o, F-1 / e, F / o, and F / e in order from the left. F-1 / o is a notation indicating that the field is composed of an odd-numbered (odd) th scanning line of the F-1th frame. Similarly, F-1 / e means "a field made up of even-numbered (even) scan lines of the F-1th frame", and F / o means "odd-numbered scanlines of the Fth frame". "F / e" means "a field consisting of even-numbered scanning lines of the F-th frame". From FIG. 2, it can be seen that y1 is in the same position as the line of the 525i signal, and y2 is in the middle position of the line of the 525i signal.
[0019]
An example of a general configuration for performing the image information conversion processing as described above is shown in FIG. In the following description, a processing system that converts a 525i signal as an input SD signal into a 525p signal as an output image signal will be described as an example. However, image information conversion processing between an input SD signal / output image signal having another image signal format can also be performed by a similar configuration. An input SD signal (525i signal) is supplied to the tap selection circuits 1, 6, and 7.
[0020]
The tap selection circuit 1 cuts out an area including a plurality of pixels required for calculation processing (calculation processing according to formula (1) described later) for predicting and estimating the line data y1 and y2, and performs line processing from the cut-out area. SD pixels (hereinafter referred to as prediction taps) necessary for predicting and estimating the data y1 and y2 are selected. The selected prediction tap is supplied to the estimated prediction calculation circuit 4. Here, as a prediction tap, the thing similar to the space class tap mentioned later can be used. However, in order to improve the prediction accuracy, it is selected by the prediction tap position information corresponding to the class.
[0021]
The estimated prediction calculation circuit 4 is supplied with prediction coefficients necessary for predicting and estimating the line data y1 and y2 from a coefficient memory 11 described later. Based on the prediction tap supplied from the tap selection circuit 1 and the prediction coefficient supplied from the coefficient memory 11, the estimated prediction calculation circuits 4 and 5 sequentially predict and generate pixel values y according to the following equation (1).
[0022]
y = w ₁ X ₁ + W ₂ X ₂ + ... + w _n X _n (1)
Where x ₁ , ..., x _n Is each prediction tap, w ₁ , ..., w _n Is each prediction coefficient. That is, Expression (1) is an expression for predicting and generating the pixel value y using n prediction taps. Line data y1 and y2 are predicted and generated as a column of pixel values y.
[0023]
The estimated prediction calculation circuit 4 supplies the line data y1 and y2 to the line order conversion circuit 5. The line order conversion circuit 5 performs line double speed processing on the supplied line data to generate a high resolution signal. This high resolution signal is used as a final output image signal. Although not shown, an output image signal is supplied to the CRT display. The CRT display has a synchronous system so that an output image signal (525p signal) can be displayed. As the input SD signal, a broadcast signal or a reproduction signal of a reproduction device such as a VTR is supplied. That is, this embodiment can be incorporated in a television receiver or the like.
[0024]
On the other hand, the tap selection circuit 6 selects an SD pixel (hereinafter referred to as a space class tap) necessary for detecting a space class from the input SD signal. The output of the tap selection circuit 6 is supplied to the space class detection circuit 8. The tap selection circuit 7 selects an SD pixel (hereinafter referred to as a motion class tap) necessary for detecting a motion class from the input SD signal. The output of the tap selection circuit 7 is supplied to the motion class detection circuit 9.
[0025]
The space class detection circuit 8 detects a space class value based on the supplied space class tap and supplies the detected space class value to the class synthesis circuit 10. The motion class detection circuit 9 detects a motion class value based on the supplied motion class tap and supplies the detected space class value to the class synthesis circuit 10.
[0026]
The class synthesis circuit 10 synthesizes the space class value and the motion class value, and supplies the synthesized class value to the coefficient memory 11. The coefficient memory 11 stores a prediction coefficient determined in advance by learning described later. Then, the prediction coefficient data specified by the synthesized class value supplied from the class synthesis circuit 10 is output to the estimated prediction calculation circuit 4. In order to perform such output, a method of storing prediction coefficient data in the coefficient memory 11 along an address specified by the synthesized class information may be used.
[0027]
Here, the space class detection will be described in more detail. In general, the spatial class detection circuit detects a spatial pattern of the level distribution of image data based on the level distribution pattern of the spatial class tap, and generates a spatial class value based on the detected spatial pattern. In this case, in order to prevent the number of classes from becoming enormous, processing for compressing 8-bit input pixel data into data having a smaller number of bits is performed for each pixel. As an example of such information compression processing, ADRC (Adaptive Dynamic Range Coding) can be used. Moreover, DPCM (predictive coding), VQ (vector quantization), etc. can also be used as information compression processing.
[0028]
ADRC is an adaptive requantization method originally developed for high-efficiency coding for VTR (Video Tape Recoder), but it can efficiently express local patterns of signal level with a short word length. In this form, ADRC is used to generate codes for spatial classification. In ADRC, the maximum value MAX and the minimum value are expressed by the following equation (2), where DR is the dynamic range of the space class tap, n is the bit allocation, L is the data level of the pixel of the space class tap, and Q is the requantization code. Requantization is performed by equally dividing the MIN with a specified bit length.
[0029]
DR = MAX-MIN + 1
Q = {(L−MIN + 0.5) × 2 / DR} (2)
However, {} means a truncation process.
[0030]
Next, motion class detection will be described in more detail. The motion class detection circuit 21 calculates the average value param of the inter-frame difference absolute value according to the following equation (3) based on the supplied motion class tap. Then, the motion class value is detected based on the calculated param value.
[0031]
[Expression 1]

[0032]
In Equation (3), n is the number of motion class taps, and can be set to n = 6, for example. Then, a motion class value that is a motion index is determined by comparing the value of param with a preset threshold value. For example, motion class value 0 when param ≦ 2, motion class value 1 when 2 <param ≦ 4, motion class value 2 when 4 <param ≦ 8, motion when param> 8 A motion class value is generated, such as class value 3. It is determined that the motion class value 0 is the minimum (still) motion, and the motion class values 1, 2, and 3 increase the motion. Note that a motion vector may be detected, and a motion class may be detected based on the detected motion vector.
[0033]
Next, processing for generating a prediction coefficient will be described. FIG. 4 shows an example of the configuration of a processing system related to calculation of prediction coefficients. A known signal (for example, a 525p signal) having the same signal format as the output image signal is supplied to the thinning filter 20 and the normal equation adding circuit 27. The thinning filter 51 generates an SD signal (for example, a 525i signal) having the number of pixels halved in the horizontal direction and the vertical direction and having a ¼ pixel number of the signal supplied as a whole.
[0034]
As such processing, for example, the pixels are thinned out by a vertical thinning filter so that the frequency in the vertical direction of the input image signal becomes 1/2, and further, the pixels by the horizontal thinning filter so that the frequency in the horizontal direction becomes 1/2. Processing such as thinning out is performed. The SD signal generated by the thinning filter 51 is supplied to the tap selection circuits 21, 22, and 23. By changing the characteristic of the thinning filter 20, the characteristic of learning can be changed, and thereby the image quality of the image obtained by conversion can be controlled.
[0035]
The tap selection circuit 21 selects a prediction tap and supplies the selected prediction tap to the normal equation addition circuit 27. The tap selection circuit 22 selects a prediction tap and supplies the selected prediction tap to the space class detection circuit 24. On the other hand, the tap selection circuit 23 selects a prediction tap and supplies the selected prediction tap to the motion class detection circuit 25.
[0036]
The space class detection circuit 24 detects a space class value based on the supplied space class tap and supplies the detected space class value to the class synthesis circuit 26. The motion class detection circuit 24 detects a motion class value based on the supplied motion class tap, and supplies the detected motion class value to the class synthesis circuit 26. The class synthesis circuit 26 synthesizes the supplied space class value and motion class value, and supplies the synthesized class value to the normal equation addition circuit 27.
[0037]
The normal equation adding circuit 27 calculates data used for calculation processing for solving a normal equation having a prediction coefficient as a solution. In other words, the normal equation addition circuit 27 performs an addition process based on the input SD signal, the output of the tap selection circuit 21, and the output of the class synthesis circuit 26, so that the prediction used for the prediction generation of the line data y1 and y2 is performed. Data necessary for solving a normal equation with a coefficient as a solution is calculated.
[0038]
Data calculated by the normal equation addition circuit 27 is supplied to the prediction coefficient determination circuit 28. The prediction coefficient determination circuit 28 performs a calculation process for solving a normal equation based on the supplied data, and calculates a prediction coefficient used for prediction generation of the line data y1 and y2. The calculated prediction coefficient is supplied to the coefficient memory 29 and stored therein.
[0039]
Here, the normal equation will be described. As described above, using n prediction taps, the pixels constituting the line data y1 and y2 are sequentially predicted and generated by the above equation (1). In equation (1), the prediction coefficient w before learning ₁ , ..., w _n Is an undetermined coefficient. Learning is performed on a plurality of input image signal data for each class. When the total number of input image signal data is expressed as m, the following equation (4) is set according to equation (1).
[0040]
y _k = W ₁ X _k1 + W ₂ X _k2 + ... + w _n X _kn (4)
(K = 1, 2,..., M)
If m> n, prediction coefficient w ₁ , ..., w _n Is not uniquely determined, the element e of the error vector e _k Is defined by the following equation (5), and the prediction coefficient is determined so as to minimize the error vector e defined by equation (6). That is, the prediction coefficient is uniquely determined by a so-called least square method.
[0041]
e _k = Y _k -{W ₁ X _k1 + W ₂ X _k2 + ... + w _n X _kn } (5)
(K = 1, 2, ... m)
[0042]
[Expression 2]

[0043]
E in equation (6) ² As a practical calculation method for obtaining the prediction coefficient that minimizes ² Prediction coefficient w _i (i = 1, 2...) is partially differentiated (formula (7)), and each prediction coefficient w _i Should be determined.
[0044]
[Equation 3]

[0045]
From Equation (7), each prediction coefficient w _i A specific procedure for determining the above will be described. X as in equations (8) and (9) _ji , Y _i (7) can be written in the form of the determinant of equation (10).
[0046]
[Expression 4]

[0047]
[Equation 5]

[0048]
[Formula 6]

[0049]
Equation (10) is generally called a normal equation. The normal equation addition circuit 27 is based on the class information supplied from the class synthesis circuit 26, the prediction tap supplied from the prediction tap selection circuit 21, and the input image signal, that is, normal equation data, that is, equations (8), ( 9) according to X _ji , Y _i Is calculated. Then, the calculated normal equation data is supplied to the prediction coefficient determination unit 28. Based on the normal equation data, the prediction coefficient determination unit 28 performs a calculation process for solving the normal equation in accordance with a general matrix solution method such as a sweep-out method and the prediction coefficient w _i Is calculated.
[0050]
In general image information conversion processing as described above, in order to perform more accurate prediction generation, the number of taps used to perform space class classification is increased, and finer spatial pattern classification is performed. Just do it. However, if it does so, the problem that the number of classes will increase significantly arises. Therefore, the present invention uses the space symmetry to integrate the classes and reduce the number of classes.
[0051]
An overall configuration of an embodiment of the present invention will be described. FIG. 5 shows an example of the configuration of an image information processing system in one embodiment. The following description is based on the assumption that image information conversion processing is performed to convert a 525i signal as an input SD signal into a 525p signal as an output image signal. The present invention can also be applied when converting between output image signals. In FIG. 4, the same components as those in the general image information processing system described above with reference to FIG.
[0052]
Here, a tap position rearrangement circuit 101 is provided at the subsequent stage of the tap selection circuit 1. Further, a symmetry determination / class value conversion circuit 102 is provided after the space class detection circuit 102. The symmetry determination / class value conversion circuit 102 determines spatial symmetry and / or amplitude symmetry as will be described later based on the space class value supplied from the space class detection circuit 8, and rearranges the determination result into tap positions. A class value conversion process corresponding to the determination result is performed while being supplied to the circuit 101.
[0053]
The space class value subjected to the class value conversion process is supplied to the class synthesis circuit 10. As a result, the output of the class synthesis circuit 10 reflects the result of class integration utilizing spatial symmetry and / or amplitude symmetry. As will be described later, such class value conversion processing is processing corresponding to class integration using spatial symmetry and / or amplitude symmetry, and therefore the number of classes is reduced by such class value conversion processing.
[0054]
The tap position rearrangement circuit 101 rearranges the class tap positions based on the determination result regarding the spatial symmetry and / or amplitude symmetry supplied from the symmetry determination / class value conversion circuit 102. Then, the prediction tap corresponding to the rearranged class tap position is supplied to the estimated prediction calculation circuit 4. An example of the configuration of the tap position rearrangement circuit 101 is shown in FIG.
[0055]
As will be described later, from the symmetry determination / class value conversion circuit 102, Sky Symmetry Of horizontal symmetry (H symmetry) and vertical symmetry (V symmetry), and time symmetry (T symmetry) as An H inversion information signal, a V inversion information signal, and a T inversion information signal indicating the presence or absence of inversion for each symmetric element are supplied from the symmetry determination / class value conversion circuit 102. In accordance with these signals, the H tap position inversion circuit 151, the V tap position inversion circuit 152, and the T tap position inversion circuit 153 perform tap position inversion processing, respectively.
[0056]
Next, a process for generating a prediction coefficient in one embodiment will be described. FIG. 7 shows an example of the configuration of such a processing system. Here, the same components as those in a general processing system for generating a prediction coefficient described above with reference to FIG. A symmetry determination / class value conversion circuit 103 is provided following the space class detection circuit 24.
[0057]
The symmetry determination / class value conversion circuit 103 determines space symmetry and / or amplitude symmetry based on the space class information supplied from the space class detection circuit 24 and supplies the determination result to the tap position rearrangement circuit 104. At the same time, class value conversion processing corresponding to the determination result is performed. Then, the class value generated by the class value conversion process is supplied to the class synthesis circuit 26. As a result, the output of the class synthesis circuit 26 reflects the result of class integration utilizing spatial symmetry and / or amplitude symmetry.
[0058]
The tap position rearrangement circuit 104 has the same configuration as that of the tap position rearrangement circuit 101 described above with reference to FIG. 6, for example, and is based on the information on the spatial symmetry supplied from the symmetry determination / class value conversion circuit 103. Change the class tap position. Then, the prediction tap corresponding to the rearranged class tap position is supplied to the normal equation adding circuit 27.
[0059]
As a result, the outputs of the normal equation addition circuit 27 and the prediction coefficient determination circuit 28 reflect the result of class integration using spatial symmetry and / or amplitude symmetry. Accordingly, the coefficient memory 29 stores prediction coefficient data that reflects the result of class integration using spatial symmetry and / or amplitude symmetry. Prior to the operation of the image information conversion processing system described above with reference to FIG. 5, the content stored in the coefficient memory 29 is loaded into the coefficient memory 11.
[0060]
Note that the number of prediction taps output from the tap selection circuit 21 is larger than the number of prediction taps used in the image information conversion processing system. Accordingly, the prediction coefficient determination unit 28 in FIG. 7 obtains more coefficients for each class. Of the prediction coefficients obtained in this way, the number of prediction coefficients to be used is selected in descending order of the absolute value.
[0061]
It should be noted that the image signal conversion processing system as shown in FIG. 5 and the processing system for determining the prediction coefficient as shown in FIG. 7 may be provided separately, or components may be provided by appropriately providing switches and the like. May be used in common. However, when these processing systems are provided separately, in order to ensure the effectiveness of learning, it is necessary to use constituent elements having the same function as those having the same operation characteristics. For example, the tap selection circuits 1 and 21 need to have the same operating characteristics. When the components are used in common, the processing of the entire apparatus is complicated due to switching or the like, but it can contribute to reduction of the circuit scale, cost, etc. relating to the entire apparatus.
[0062]
Next, FIG. 8A shows an example of the tap arrangement used for the space class classification of y1. An example of the tap arrangement used for predictive generation of pixels in y1 is shown in FIG. 8B. Furthermore, FIG. 9A shows an example of tap arrangement used for the space class classification of y2, and FIG. 9B shows an example of tap arrangement used for predictive generation of the pixels in y2. In the following description, the class division points defined by the tap arrangement as shown in FIGS. 8 and 9 are used in the spatial class detection circuit 8 in the image information conversion processing system described above with reference to FIG. This is performed on the premise that classification is performed by performing 1-bit ADRC.
[0063]
The coordinate axes in FIGS. 8 and 9 are as follows. That is, the horizontal direction in the frame is made to correspond to the horizontal coordinate axis (hereinafter referred to as the h-axis) in the drawing. Further, the vertical direction in the frame is made to correspond to the vertical coordinate axis (hereinafter referred to as the v-axis) in the drawing. Furthermore, the time direction, that is, the direction indicating the temporal order of the frames is made to correspond to a coordinate axis (hereinafter referred to as a t-axis) from the front to the back in the drawing. The same coordinate axes are used also in each figure related to the explanation of the spatial symmetry referred to in the following explanation. 8A and 8B, the class classification taps a, b, e, h, i are all located on the vt plane. 9A and 9B, the class classification taps b, c, d, i, j, and k are all located on the ht plane.
[0064]
A description will be given of class integration performed in the symmetry determination / class value conversion circuit 102 using spatial symmetry. The following description will be made on the assumption of an example of the space class tap arrangement (see FIG. 8A) used for the space class classification of y1, but it can also be applied when other space class tap arrangements are used. Using H symmetry, that is, symmetry with respect to the vt plane as an example, integration of space classes using spatial symmetry will be described. FIG. 10B shows prediction coefficients when the output of the space class detection circuit 4 is classified into the space pattern shown in FIG. 10A.
[0065]
Here, in each of the taps a to i in FIG. 10A, the white circle corresponds to “1”, for example, and the black circle corresponds to “0”, for example. Also, the prediction coefficients in FIG. 10B corresponding to a to i are k respectively. _a ~ K _i It is. On the other hand, FIG. 11A shows an example of a spatial pattern in which FIG. Here, the H inversion is a space inversion operation (with a mirror image relationship) that is inverted with respect to the vt plane that is a symmetric element in H symmetry. The signs of taps d and f not on the vt plane are reversed in FIGS. 10A and 11A. Prediction coefficient k corresponding to the spatial pattern shown in FIG. 11A _a '~ K _i 'Is shown in FIG. 11B.
[0066]
As described above with reference to FIG. 4 and the like, a prediction coefficient is calculated by performing arithmetic processing for solving a linear prediction estimation formula based on a plurality of known image signals having the same signal format as the output image signal. . Because of such a calculation method, a prediction coefficient to be used corresponding to a class classified into a certain spatial pattern and a spatial pattern obtained by H-inversion of the spatial pattern is the arbitrary spatial pattern. The prediction coefficient to be used in correspondence with the class classified into (2) and a mirror image relationship with respect to the vt plane.
[0067]
In the above example, the prediction coefficient to be used corresponding to the class classified into the spatial pattern shown in FIG. 11A should be used corresponding to the class classified into the spatial pattern shown in FIG. 10A. The prediction coefficient has a mirror image relation with respect to the vt plane. Therefore, the prediction coefficient k ′ shown in FIG. _a ~ K ' _i Is the prediction coefficient k shown in FIG. _a ~ K _i Can be obtained by inverting H. That is, for the classes classified into the spatial patterns of FIG. 10A and FIG. 11A, the prediction coefficient can be obtained by inverting the common coefficient value H according to the situation. More generally, the following can be seen. That is, classes classified into spatial patterns in which the positional relationship between taps used for spatial pattern division forms an H-symmetric relationship with each other can be integrated.
[0068]
Similar to class integration using H symmetry as described above, class integration is performed using V symmetry, ie, symmetry with respect to the ht plane, and T symmetry, ie, symmetry with respect to the vh plane. It is also possible. This will be described with reference to FIG. FIG. 12A shows an example of a spatial pattern. 12B, 12C, and 12D show examples of spatial patterns obtained by performing H inversion, V inversion, and T inversion, respectively, for such an example.
[0069]
Since the class classified into the spatial pattern shown in FIG. 12A and the class classified into the spatial pattern shown in FIG. 12B are in an H-symmetric relationship with each other, the classes can be integrated as described above. Similarly, since the class classified into the spatial pattern shown in FIG. 12A and the class classified into the spatial pattern shown in FIG. 12C are in a V-symmetric relationship with each other, the classes can be integrated. The class classified into the spatial pattern shown and the class classified into the spatial pattern shown in FIG. 12D are in a V-symmetric relationship with each other, so that the classes can be integrated.
[0070]
Moreover, class integration is also possible for classes classified into spatial patterns having symmetry formed by combining symmetry with respect to the vt plane, the ht plane, and the hv plane. FIG. 13A shows an example of a spatial pattern having HV symmetry (combination of H symmetry and V symmetry) with respect to the example of the spatial pattern shown in FIG. 12A. Further, TH symmetry (combination of T symmetry and H symmetry) is shown in FIG. 13B. Furthermore, FIG. 13C shows an example of a spatial pattern having VT symmetry (combination of V symmetry and T symmetry). On the other hand, FIG. 13D shows an example of a spatial pattern having HVT symmetry (a combination of H symmetry, V symmetry, and T symmetry).
[0071]
For example, the prediction coefficient to be used corresponding to the class classified into the spatial pattern shown in FIG. 13A is the same as the prediction coefficient to be used corresponding to the class classified into the spatial pattern shown in FIG. 12A. Thus, the mirror image is related to the symmetry element in the HV symmetry, that is, the vt plane and the ht plane. Therefore, the prediction coefficient shown in FIG. 13A can be obtained by inverting the prediction coefficient to be used corresponding to the class classified into the spatial pattern shown in FIG. 12A with respect to the vt plane and the ht plane. it can. Similarly, for example, for the prediction coefficient shown in FIG. 13D, the prediction coefficient to be used corresponding to the class classified into the spatial pattern shown in FIG. 12A is the symmetrical element in the HVT symmetry, that is, the vt plane, h− It can be obtained by inversion with respect to the t plane and the hv plane.
[0072]
Next, class integration using amplitude symmetry, that is, symmetry in the amplitude direction will be described. First, the prediction coefficient corresponding to the spatial pattern shown in FIG. 14A is shown in FIG. 14B. On the other hand, FIG. 15A shows a spatial pattern in which the amplitude is inverted with respect to the spatial pattern shown in FIG. 14A, that is, the sign of each input pixel is opposite. The spatial pattern shown in FIG. 15A is generated by inverting the input image signal having the spatial pattern shown in FIG. 14A, more specifically, by subtracting the input image signal from the maximum value of the input image signal. Can do.
[0073]
As described above, the prediction coefficient is obtained for each spatial pattern in the input image signal as corresponding to the invariable relationship between the input image signal and the predicted and generated pixels. For this reason, a pixel value predicted and generated from a spatial pattern obtained by inverting an arbitrary spatial pattern in the amplitude direction is a value obtained by inverting a pixel value predicted and generated from the arbitrary spatial pattern. Accordingly, the input image signal having the spatial pattern shown in FIG. 14A is inverted, and the prediction generation calculation according to the equation (1) is performed using the prediction coefficient shown in FIG. 14B, thereby having the spatial pattern shown in FIG. 15A. Predictive generation of pixel values corresponding to the input image signal can be performed.
[0074]
Further, the expression (1) is a linear combination expression, and since the image signal having the same signal format as the output image signal input at the time of learning and the amplitude of the estimated pixel data are equal, the prediction is performed. The sum of the coefficients is 1. Therefore, the inversion of the input image signal and the inversion of the estimated pixel are canceled as shown in the following equation (11). As a result, as a prediction coefficient corresponding to the spatial pattern shown in FIG. The prediction coefficient corresponding to the spatial pattern shown in (3) can be used as it is. That is, k ″ in FIG. 15B. _a ~ K '' _i Is k in FIG. 14B. _a ~ K _i Will match.
[0075]
[Expression 7]

[0076]
As can be seen from the above description, classes classified into spatial patterns having an amplitude inversion relationship with each other can be integrated.
[0077]
Further, combining the symmetry obtained by combining one or more of H symmetry, V symmetry, and T symmetry described above with reference to FIG. 12 and the like, and amplitude symmetry. Class integration is also possible for classes that are classified into spatial patterns having symmetry. First, an example of a spatial pattern having symmetry obtained by combining one of H symmetry, V symmetry, and T symmetry and amplitude symmetry will be described with reference to FIG. FIG. 16A shows an example of a spatial pattern in which amplitude inversion is applied to the example of the spatial pattern shown in FIG. 14A.
[0078]
FIG. 16B shows an example of a spatial pattern in which amplitude inversion and H inversion are performed on the example of the spatial pattern shown in FIG. 14A. Furthermore, an example of a spatial pattern obtained by performing amplitude inversion and V inversion on the example of the spatial pattern shown in FIG. 14A, and an amplitude inversion and T inversion on the example of the spatial pattern shown in FIG. 14A. Examples of the applied spatial pattern are shown in FIGS. 16C and 16D, respectively.
[0079]
Furthermore, an example of a spatial pattern having symmetry obtained by combining two or more of H symmetry, V symmetry, and T symmetry and amplitude symmetry will be described with reference to FIG. FIG. 17A shows an example of a spatial pattern in which amplitude inversion and HV inversion are performed on the example of the spatial pattern shown in FIG. 14A. FIG. 17B shows an example of a spatial pattern in which amplitude inversion and TH inversion are performed on the example of the spatial pattern shown in FIG. 14A. Further, FIG. 17C shows an example of a spatial pattern in which amplitude inversion and VT inversion are performed on the example of the spatial pattern shown in FIG. 14A. FIG. 17D shows an example of a spatial pattern in which amplitude inversion and VT inversion are performed on the example of the spatial pattern shown in FIG. 14A.
[0080]
The space class value conversion processing related to each symmetry as described above will be described with reference to FIGS. FIG. 18A shows an example of a spatial pattern of the input image signal. In addition, FIG. 18B, FIG. 18C, and FIG. 18D show spatial patterns obtained by applying H inversion, V inversion, and T inversion to the example of the spatial pattern shown in FIG. 18A, respectively. Here, the value of the black-tapped tap is “1”, for example, and the value of the white tap is “0”, for example.
[0081]
The space class value detected as the sum of the values of these taps can be defined as, for example, a 5-bit value in which the values of the taps P, Q, R, S, and T are arranged in this order. According to such a definition, the space class value for the example shown in FIG. 18A = 00111 and 7 in decimal representation. Also, the space class value for the example shown in FIG. 18B is 01101, which is 13 in decimal representation. Similarly, the space class values for the space patterns shown in FIGS. 18C and 18D are 10110 (22 in decimal representation) and 11100 (28 in decimal representation), respectively. Of course, the definition of the space class value is not limited to that described above.
[0082]
Similarly, FIG. 19A shows another example of the spatial pattern of the input image signal. Further, other examples of such spatial patterns are shown in FIGS. 19B, 19C, and 19D, respectively, in which spatial inversion is performed by H inversion, V inversion, and T inversion. If the space class value is detected in the same manner as in FIG. 18, the space class value is 11100 (in decimal representation) for the space patterns shown in FIGS. 18A, 19B, 19C, and 19D. 28), 10110 (22 in decimal representation), 01101 (13 in decimal representation), and 00111 (7 in decimal representation).
[0083]
Next, regarding the symmetry determination / class value conversion circuit 102 that detects the spatial symmetry and / or amplitude symmetry as described above from the space class value supplied from the space class detection circuit 8 (see FIG. 5), This will be described in detail with reference to FIG. The space class value includes a minimum value determination circuit 216, an amplitude inverter circuit 208, an H inverter circuit 201, a V inverter circuit 202, a T inverter circuit 203, an HV inverter circuit 204, a VT inverter circuit 205, a TH inverter circuit 206, and an HTV inverter circuit. 207 is supplied.
[0084]
The minimum value determination circuit 216 and the amplitude inversion circuit 208 will be described later. The H inversion circuit 201, the V inversion circuit 202, the T inversion circuit 203, the HV inversion circuit 204, the VT inversion circuit 205, the TH inversion circuit 206, and the HTV inversion circuit 207 respectively perform an H inversion process on the supplied class value. , V inversion processing, T inversion processing, HV inversion processing, VT inversion processing, TH inversion processing, and HTV inversion processing. The class values obtained as a result of these seven types of spatial inversion processing are respectively supplied to the minimum value determination circuit 216 and the configuration for performing the amplitude inversion processing.
[0085]
As a configuration for performing the amplitude inversion processing, the amplitude inversion circuit 209 is provided in the subsequent stage of the H inversion circuit 201, the V inversion circuit 202, the T inversion circuit 203, the HV inversion circuit 204, the VT inversion circuit 205, the TH inversion circuit 206, and the HTV inversion circuit 207. , 210, 211, 212, 213, 214, 215 are provided. The amplitude inversion circuit 208 described above performs amplitude inversion processing on the class value supplied from the space class detection circuit 8. Outputs of the amplitude inversion circuits 208, 209, 210, 211, 212, 213, 214, and 215 are supplied to the minimum value determination circuit.
[0086]
With the configuration as described above, the class value obtained as a result of the seven types of spatial inversion processing and the class value supplied from the spatial class detection circuit 8 in total, and the amplitude inversion to these eight types of class values Eight types of class values that have been processed, and thus a total of 16 types of class values, are supplied to the minimum value determination circuit 216. The minimum value determination circuit 216 determines the minimum of the 16 types of class values to be supplied, and supplies the minimum class value, which is the determination result, to the readdressing processing circuit 217 as a representative class value. Note that the representative class value is not necessarily limited to the minimum class value, and may be the maximum class value, for example. However, when the minimum class value is used, there is an advantage that the amount of information for expressing the representative class value can be minimized.
[0087]
In addition, the minimum value determination circuit 216 generates information indicating inversion processing performed to obtain the minimum class value from the class values supplied from the space class detection circuit 8. Such information includes, for example, a 1-bit H inversion information signal indicating whether or not each of the H inversion processing, V inversion processing, and T inversion processing has been performed in the processing for obtaining the minimum class value, A V inversion information signal and a T inversion information signal may be generated. These signals are supplied to the tap position rearrangement circuit 101 (see FIG. 5).
[0088]
The representative class value generated as described above takes a discrete value for each class value sequentially supplied from the space class detection circuit 8. For this reason, the readdressing processing circuit 217 performs processing of skipping the missing values among the class values sequentially supplied from the space class detection circuit 8 and converting them into continuous class values. The output of the readdressing processing circuit 217 is supplied to the class synthesis circuit 10 (see FIG. 5) as a final space class value generated as a result of the conversion processing by the symmetry determination / class value conversion circuit 102. Therefore, the output of the class synthesis circuit 10 is based on the result of the class value conversion process using spatial symmetry by the symmetry determination / class value conversion circuit 102.
[0089]
With the class value conversion as described above, with respect to the space class value, the prediction coefficient data in the coefficient memory 11 specified by one representative class value corresponds to 16 types of space patterns. Therefore, 16 types of spatial patterns are integrated into one class.
[0090]
Further, the tap position rearrangement circuit 101 determines the position of tap data used in the estimated prediction calculation performed in the estimated prediction calculation circuit 4 based on the supplied H inverted information signal, V inverted information signal, and T inverted information signal. Rearrange. Thereby, the position of the tap data is accurately rearranged corresponding to the 16 types of spatial patterns classified into one class. Therefore, the prediction coefficient data output from the coefficient memory 11 according to the output of the class synthesis circuit 10 generated based on the result of the conversion processing by the symmetry determination / class value conversion circuit 102 (that is, based on each representative class value). And consistency in the estimated prediction calculation with the tap data is ensured. Therefore, prediction generation of the line data y1 and y2 can be accurately performed.
[0091]
Next, the line double speed process performed by the line order conversion circuit 6 will be described. As described above, the horizontal period of the 525p signal generated by the estimated prediction calculation circuits 4 and 5 is the same as the horizontal period of the 525i signal before the image information conversion process is performed. The line order conversion circuit 6 performs line double speed processing for doubling the horizontal period. FIG. 21 shows line double speed processing using an analog waveform. Line data y1 and y2 are simultaneously predicted and generated by the estimated prediction calculation circuits 4 and 5.
[0092]
The line data y1 includes lines a1, a2, a3,... In order, and the line data y2 includes lines b1, b2, b3,. In the line order conversion circuit 500, a line doubler (not shown) and a switching circuit (not shown) are provided. The line doubler compresses the data of each line by half in the time axis direction, and supplies the compressed data to the switching circuit. The switching circuit alternately selects and outputs the supplied data. In this way, line sequential outputs (a0, b0, a1, b1,...) Are formed.
In this embodiment, the output pixel value (line data L1) on the existing line and the output pixel value (line data L2) on the creation line are created in a parallel configuration. On the other hand, a memory may be added so that the line data y1 and y2 are generated in order by time division processing and the line double speed processing is performed if the operation speed of the circuit is high.
[0093]
In the embodiment of the present invention described above, symmetry determination and class value conversion are performed collectively. On the other hand, another embodiment of the present invention in which the symmetry determination and the class value conversion are performed for each symmetric element is possible. FIG. 22 shows an example of an image information conversion processing system in another embodiment. Here, the same code | symbol was attached | subjected to the component similar to the component in FIG.
[0094]
The space class value generated by the space class detection circuit 8 is supplied to the symmetry determination circuit 105 and the class value conversion circuit 106. The symmetry determination circuit 105 has the same configuration as the symmetry determination / class value conversion circuit 102 in the above-described embodiment, for example, and includes an H inversion information signal, a V inversion information signal, a T inversion information signal, and an amplitude inversion information. Output a signal. Here, the presence / absence of amplitude inversion is unnecessary information for performing the tap position rearrangement. Therefore, the amplitude inversion information signal in the symmetry determination / class value conversion circuit 102 (see FIG. 5) in the above-described embodiment. Is not output. However, in another embodiment in which the class value conversion process is performed separately from the symmetry determination, the symmetry determination circuit 105 also outputs an amplitude inversion information signal.
[0095]
Among the outputs of the symmetry determining circuit 105, the H inverted information signal, the V inverted information signal, the T inverted information signal, and the amplitude inverted information signal are supplied to the class value conversion processing circuit 106. Further, the H inverted information signal, the V inverted information signal, and the T inverted information signal are supplied to the tap position rearrangement circuit 101. The tap position rearrangement circuit 101 performs tap position rearrangement processing with reference to each supplied inversion information signal.
[0096]
An example of the configuration of the class value conversion processing circuit 106 is shown in FIG. The class value supplied from the space class detection circuit 8 is supplied to the H symmetry class value conversion circuit 110. At the subsequent stage of the H symmetric class value conversion circuit 110, a V symmetric class value conversion circuit 111, a T symmetric class value conversion circuit 112, and an amplitude symmetric class value conversion circuit 113 are provided in this order. The H symmetric class value conversion circuit 110, the V symmetric class value conversion circuit 111, the T symmetric class value conversion circuit 112, and the amplitude symmetric class value conversion circuit 113 have an H inversion information signal, a V inversion information signal, and a T inversion information signal, respectively. And an amplitude inversion information signal are supplied.
[0097]
The H symmetric class value conversion circuit 110, the V symmetric class value conversion circuit 111, the T symmetric class value conversion circuit 112, and the amplitude symmetric class value conversion circuit 113 refer to the inverted information signal supplied to each, and perform class value conversion processing. I do. The total of these class value conversion processes generates, for example, the smallest class value as a representative class value in 16 types of spatial patterns.
[0098]
A readdressing processing circuit 114 is provided following the amplitude symmetric class value conversion circuit 113. Similar to the readdressing processing circuit 217 (see FIG. 20) in the above-described embodiment, the readdressing processing circuit 114 skips missing values among the class values sequentially supplied from the space class detection circuit 8. To convert to continuous class values. The output of the re-addressing processing circuit 114 is output as the final space class value as a result of the conversion processing by the symmetry determination / class value conversion circuit 105.
[0099]
One embodiment and other embodiments of the present invention convert a 525i signal into a 525p signal. However, the number of 525 lines is an example, and the present invention can be applied to other numbers of lines. Can be applied. For example, an output image signal including a line including a number of pixels other than twice the number of pixels in the horizontal direction in the input SD signal may be predicted and generated. More specifically, the present invention can be applied to the case where image information conversion processing is performed to predict and generate an 1050i signal as an output image signal, that is, an interlaced image signal having 1050 scanning lines.
[0100]
【The invention's effect】
As described above, the present invention is based on the amplitude symmetry and / or spatial symmetry of image data when performing image information conversion processing for forming an output image signal having a different scanning line structure from an input image signal. A class value conversion process corresponding to class integration is performed to reduce the number of classes.
[0101]
For this reason, the number of classes handled in the processing system can be reduced without losing the fineness of the image information conversion processing by the class classification adaptive processing, so that efficient processing can be performed with a smaller number of classes. it can.
[0102]
Accordingly, accurate processing can be performed even when the storage capacity of the memory asset in the apparatus is small, and the storage capacity of the memory asset can be reduced. For this reason, it can contribute to size reduction, cost reduction, etc. regarding the entire apparatus.
[Brief description of the drawings]
FIG. 1 is a schematic diagram illustrating an example of an arrangement of pixels in image information conversion processing performed according to an embodiment of the present invention.
FIG. 2 is a schematic diagram illustrating an example of a space class tap arrangement according to an embodiment of the present invention.
FIG. 3 is a block diagram illustrating an example of a configuration of a general image information conversion processing system.
FIG. 4 is a block diagram illustrating an example of a configuration of a general prediction coefficient calculation processing system.
FIG. 5 is a block diagram showing an example of the configuration of an image information conversion processing system in one embodiment of the present invention.
FIG. 6 is a block diagram showing a partial configuration in an example of a configuration of an image information conversion processing system in an embodiment of the present invention.
FIG. 7 is a block diagram showing an example of a configuration of a prediction coefficient calculation processing system in one embodiment of the present invention.
FIG. 8 is a schematic diagram illustrating an example of an arrangement of space class taps and an example of an arrangement of prediction taps for line data y1.
FIG. 9 is a schematic diagram illustrating an example of an arrangement of space class taps and an example of an arrangement of prediction taps for line data y2.
FIG. 10 is a schematic diagram for explaining an example of a spatial pattern and an example of a prediction coefficient corresponding thereto.
11 is a schematic diagram for explaining a spatial pattern and a prediction coefficient when the spatial pattern shown in FIG. 11 is H-inverted. FIG.
FIG. 12 is a schematic diagram for explaining H symmetry, V symmetry, and T symmetry;
FIG. 13 is a schematic diagram for explaining symmetry formed by combining two or three of H symmetry, V symmetry, and T symmetry;
FIG. 14 is a schematic diagram for explaining an example of a spatial pattern and an example of a prediction coefficient corresponding thereto.
15 is a schematic diagram for explaining a spatial pattern and a prediction coefficient when the amplitude of the spatial pattern shown in FIG. 14 is inverted.
FIG. 16 is a schematic diagram for explaining a spatial pattern having a symmetry formed by combining one of H symmetry, V symmetry, and T symmetry and amplitude symmetry;
FIG. 17 is a schematic diagram for explaining a spatial pattern having symmetry obtained by combining two or more of H symmetry, V symmetry, and T symmetry and amplitude symmetry;
FIG. 18 is a schematic diagram for explaining an example of symmetry determination;
FIG. 19 is a schematic diagram for explaining another example of symmetry determination;
FIG. 20 is a block diagram showing another part of the configuration of an example of the configuration of the image information conversion processing system in the embodiment of the present invention.
FIG. 21 is a schematic diagram for explaining line double speed processing;
FIG. 22 is a block diagram showing an example of the configuration of an image information conversion processing system in another embodiment of the present invention.
FIG. 23 is a block diagram showing a partial configuration of an example of the configuration of an image information conversion processing system according to another embodiment of the present invention.
[Explanation of symbols]
101 ... Tap position rearrangement circuit, 102 ... Symmetry determination / class value conversion circuit, 217 ... Re-addressing processing circuit

Claims (6)

In an image information conversion apparatus configured to form an output image signal having a different scanning line structure from an input image signal,
First image data selection means for selecting, from an input image signal, a pixel located in the vicinity of a predetermined attention point in time or space;
A space in which a level distribution pattern is detected from the image data selected by the first image data selection means, and a first space class value indicating a space class to which the attention point belongs is determined based on the detected pattern. Class detection means;
A plurality of the first spatial classes that determine the amplitude symmetry and / or spatial symmetry of the image data selected by the first image data selection means, and have the same amplitude symmetry and / or spatial symmetry. values, and integrated by reversing the position with the amplitude symmetry and / or spatial symmetry, into a second class values using the minimum class value of the first class value integrating Class value conversion means;
Second image data selection means for selecting, from the input image signal, a pixel that includes the point of interest and is located near the point of interest in terms of time or space;
Storage means for storing prediction coefficient data determined in advance corresponding to the class indicated by the second class value;
The position of the image data selected by the upper Symbol second image data selection means, an inverting process and the same amplitude symmetry and / or spatial symmetry of the first spatial class values made by the class value converting means In order to predict and generate a pixel at the attention point by linear linear combination of the image data inverted to the position and the prediction coefficient data selected from the storage means corresponding to the second class value Arithmetic processing means for performing the arithmetic processing of
The above prediction coefficient is
The difference between the true pixel value in the predetermined image signal having the same signal format as that of the output image signal and the calculated value of the linear linear combination between the image data selected by the second image data selection means is minimized. An image information processing apparatus characterized by being defined as follows. In an image information conversion apparatus configured to form an output image signal having a different scanning line structure from an input image signal,
First image data selection means for selecting, from an input image signal, a pixel located in the vicinity of a predetermined attention point in time or space;
A space class detection unit that detects a level distribution pattern from the image data selected by the first image data selection unit, and determines a space class value indicating a space class to which the attention point belongs based on the detected pattern. When,
Second image data selection means for selecting, from the input image signal, a pixel located in the vicinity of the point of interest temporally or spatially in a plurality of frames in the input image signal;
Motion class detection means for determining a motion class value representing motion based on the inter-frame difference from the image data selected by the second image data selection means;
Class combining means for combining the space class value and the motion class value to generate a first class value;
A plurality of the first spatial classes that determine the amplitude symmetry and / or spatial symmetry of the image data selected by the first image data selection means, and have the same amplitude symmetry and / or spatial symmetry. values, and integrated by reversing the position with the amplitude symmetry and / or spatial symmetry, into a second class values using the minimum class value of the first class value integrating Class value conversion means;
Third image data selection means for selecting, from the input image signal, a pixel that includes the attention point and is located in the temporal or spatial vicinity of the attention point;
Storage means for storing prediction coefficient data determined in advance corresponding to the class indicated by the second class value;
The position of the image data selected by the upper Symbol third image data selection means, an inverting process and the same amplitude symmetry and / or spatial symmetry of the first spatial class values made by the class value converting means In order to predict and generate a pixel at the attention point by linear linear combination of the image data inverted to the position and the prediction coefficient data selected from the storage means corresponding to the second class value Arithmetic processing means for performing the arithmetic processing of
The above prediction coefficient is
The difference between the true pixel value in the predetermined image signal having the same signal format as that of the output image signal and the calculated value of the linear linear combination between the image data selected by the third image data selection means is minimized. An image information processing apparatus characterized by being defined as follows. In claim 2,
The input image signal is
Interlaced image signal with 525 scanning lines,
The output image signal is
An image information processing apparatus characterized by being a progressive image signal having 525 scanning lines. In claim 2,
The input image signal is
Interlaced image signal with 525 scanning lines,
The output image signal is
An image information processing apparatus characterized by being a progressive image signal having 1050 scanning lines. In an image information conversion method in which an output image signal having a different scanning line structure is formed from an input image signal,
A first image data selection step of selecting, from the input image signal, a pixel located in the vicinity of a predetermined point of interest temporally or spatially;
A space in which a level distribution pattern is detected from the image data selected in the first image data selection step, and a first space class value indicating a space class to which the attention point belongs is determined based on the detected pattern. A class detection step;
A plurality of the first spatial classes that determine the amplitude symmetry and / or spatial symmetry of the image data selected by the first image data selection means, and have the same amplitude symmetry and / or spatial symmetry. The values are integrated by inverting the position having the amplitude symmetry and / or spatial symmetry, and converted into a second spatial class value using the smallest class value among the first class values to be integrated. Class value conversion step,
A second image data selection step for selecting, from the input image signal, a pixel that includes the attention point and is located in the temporal or spatial vicinity of the attention point;
A storage step of storing predetermined prediction coefficient data corresponding to the class indicated by the second class value;
The position of the image data selected by the upper Symbol second image data selection step, having the same amplitude symmetry and inversion processing and / or spatial symmetry of the first spatial class values made in the class value conversion step In order to predict and generate a pixel at the attention point by linear linear combination of the image data inverted to the position and the prediction coefficient data selected from the storage means corresponding to the second class value An arithmetic processing step for performing the arithmetic processing of
The above prediction coefficient is
The difference between the true pixel value in the predetermined image signal having the same signal format as that of the output image signal and the calculated value of linear linear combination between the image data selected by the second image data selection step is minimized. An image information processing method characterized by being defined as follows. In an image information conversion method in which an output image signal having a different scanning line structure is formed from an input image signal,
A first image data selection step of selecting, from the input image signal, a pixel located in the vicinity of a predetermined point of interest temporally or spatially;
A space class detection step of detecting a level distribution pattern from the image data selected in the first image data selection step and determining a space class value indicating a space class to which the attention point belongs based on the detected pattern. When,
A second image data selection step of selecting, from the input image signal, a pixel located in the vicinity of the point of interest temporally or spatially in a plurality of frames in the input image signal;
A motion class detection step for determining a motion class value representing motion based on the inter-frame difference from the image data selected by the second image data selection step;
A class combining step of combining the space class value and the motion class value to generate a first class value;
A plurality of the first spatial classes that determine the amplitude symmetry and / or spatial symmetry of the image data selected by the first image data selection step, and have the same amplitude symmetry and / or spatial symmetry. The values are integrated by inverting the position having the amplitude symmetry and / or spatial symmetry, and converted into a second class value using the smallest class value among the first class values to be integrated. A class value conversion step;
A third image data selection step for selecting, from the input image signal, a pixel that includes the attention point and is located in the temporal or spatial vicinity of the attention point;
A storage step of storing predetermined prediction coefficient data corresponding to the class indicated by the second class value;
The position of the image data selected by the upper Symbol third image data selection means, have the same amplitude symmetry and inversion processing and / or spatial symmetry of the first spatial class values made in the class value conversion step In order to predict and generate a pixel at the attention point by linear linear combination of the image data inverted to the position and the prediction coefficient data selected from the storage means corresponding to the second class value An arithmetic processing step for performing the arithmetic processing of
The above prediction coefficient is
Minimize the difference between the true pixel value in the predetermined image signal having the same signal format as the output image signal and the calculated value of linear linear combination between the image data selected by the third image data selection step An image information processing method characterized by being defined as follows.

Images