JP4093413B2 - Image processing apparatus, image processing program, and recording medium recording the program - Google Patents

Description

【００１９】
なお、上記式(1)において、縮小率を定めるrは常に自然数になるようにする。原画像（I₀）の解像度が100に割り切れない場合、rは自然数に丸めた値にする。例えば、360dpiの場合、rは3とすることになり、この場合には低解像度画像（I）は、120dpiとなる。
・ 低解像度画像（I）の平滑化（Ｓ１０３）
低解像度画像（I）には、低解像度の生成過程でノイズを発生する可能性がある。また、元の原画像にも画像入力機器３０やネットワーク上で発生するノイズが含まれる可能性がある（特に外部システムから送信されてくる入力画像にその可能性が高い）。こうしたノイズは、後続の処理における障害となるので、前段で得られた低解像度画像（I）を線形フィルタで平滑化することにより、ノイズを除去する。
・ 低解像度画像（I）における特徴量計算（Ｓ１０４）
平滑化後の低解像度画像（I）に対し、その画像の特徴を表すデータ値を求めるための計算を行う。本実施形態では、各画素の周囲に所定（固定）サイズのウィンドウ（2次元配列の画素を含む）を設定し、ウィンドウ内の画素が持つ輝度（濃度）信号の平均値μと注目画素の平均値μに対する標準偏差σを計算する。これらの特徴量は、R,G,Bごとに計算される。
【００２０】
・ 低解像度画像（I）における文字領域（C）の抽出（Ｓ１０５）
次に、低解像度画像（I）に対して、カラー成分の局所適応的二値化を行うことにより、文字領域の抽出検出を行う。
具体的には、色の各成分R,G,Bにおいて、上記特徴量計算手順（Ｓ１０４）で特徴量として算出した平均値μと標準偏差σを用いて、aとbをパラメータとした閾値μ(a＋bσ)を設定し、閾値μ(a＋bσ)と低解像度画像（I）の信号値の比較を行う。背景に直接印刷された黒文字の場合、RGBのすべての成分信号において、コントラストが強くなる傾向があることに着目すると、全てのチャンネルにおいて、信号値が閾値よりも低ければ、画素 [ｉ，ｊ] を文字領域（C）の要素に設定するという方法を用いることが適当である。つまり、下記(2)式に従って、文字領域（C）の要素を求める。
【００２１】
【数２】

【００２２】
次いで、低解像度画像（I）と同じサイズで、上記で求めた文字領域（C）の要素となる画素位置では値ON、それ以外では値OFFを持つような２値画像（P）を構成する。この２値画像（P）において、横方向に背景画素のラン（画像（P）がOFFの画素を連ねた連結画素列）を取り出す。もし、取り出したランの長さが閾値τよりも短ければ、その画素を一時的にONにする。このようにして、横方向に対して、背景画素のランをスミアリング（ smearing ）する。同様に、縦方向にも、背景画素のランのスミアリングを行う。そして、横方向と縦方向のスミアリングのANDをとり、両方のスミアリングでONになるものだけを、最終的にモノクロ前景画素としての文字領域（C）として判定して、抽出する。
図６は、スキャナ入力され、本処理の対象として図５に例示した原画像（I₀）に対しここまでの処理を経て、文字領域（C）の抽出を行った結果を示す。図６に示すように、上記したスミアリングの結果として、文字を囲む矩形領域内をONとするような形（図中の領域(IV)）で文字領域（C）が抽出される。
【００２３】
・ 原画像（I₀）のブロック分割（Ｓ１０６）
次に、上記で抽出した文字領域（C）を対象にして背景を探索するプロセスを実行するが、このときに、原画像（I₀）をブロックに分割し、ブロック単位ごとに処理を行い、真の背景、即ち“地肌”を求めるようにする。
ブロックの分割は、互いに重なりのない、十分に小さいブロックとし、図６には低解像度画像（I）に行った場合を示す。図６に示すように、各ブロックのサイズと形は、例えば、辺の長さが20mm（100dpiで80画素）相当の正方形などとすればよい。
・ 分割ブロックごとに文字領域（C）内を２クラスに分類（Ｓ１０７）
次に、真の背景（原稿の地肌部分）を探索するプロセスとして、上記分割ステップで定めた各ブロック内で、上記文字領域（C）の抽出手順（Ｓ１０５）で抽出した文字領域（C）に属する画素を輝度によって２クラスに分類する。この分類には、各画素の色信号から輝度を計算し、閾値処理する方法を用いる。閾値処理には、判別分析法、モーメント保持法、エントロピ法などの既知の方法を使用することが可能である。ブロックiにおいて、分類された画素のうち、暗い方に対応する画素群を第1のクラスR_ｉ１、明るい方に対応する画素群を第2のクラスR_ｉ２とすると、R_ｉ１が文字に、R_ｉ２が背景（地肌）に対応する。
【００２４】
・ 背景（地肌）色の推定・統計量計算（Ｓ１０８）
真の背景（原稿の地肌）を探索するプロセスのステップとして、次に、上記ステップＳ１０７で得た第2のクラスR_ｉ２に属する画素数N_ｉと、R_ｉ２の平均輝度L_ｉをもとにした評価関数、例えば、N_ｉ×L_ｉのように、画素数が多く、平均輝度が高いほど大きい値をとるような関数を定義し、その値が最大になるブロックを見つけ、その中の文字領域（C）をウィンドウ(W) として設定する。
図７は、図５の入力画像に対して、ウィンドウ(W)として設定された領域を示す。ウィンドウ(W)として設定された領域の中での暗い方に対応する画素群を画素群R_Ｗ１、明るい方に対応する画素群を画素群R_Ｗ２とし、それぞれ濃いグレーと薄いグレーで示す。明るい方に対応する画素群R_Ｗ２の平均色を原画像（図５）における真の背景色（代表背景色）の推定結果として得る。
さらに、真の背景に相当する画素群R_Ｗ２において、輝度の統計量として、平均l_Ｂ及び標準偏差σを計算しておく。
【００２５】
・ 地肌領域抽出（Ｓ１０９）
前段の背景（地肌）色の推定・統計量計算手順（Ｓ１０８）で求めたウィンドウ(W)における真の背景の輝度の統計量（平均l_Ｂ及び標準偏差σ）に基づいて、背景（地肌）とみなされる領域を抽出する。低解像度画像（I）において、輝度Lが、l_Ｂ−aσ＜L＜l_Ｂ（ただし、aは正のパラメータ）であるような画素を抽出すべき背景（地肌）とみなされる画素とする。
抽出条件を示す上記した不等式で、下限値l_Ｂ−aσにおけるパラメータaは、背景の変動や裏写りの程度などにより決まり、例えば、裏写りが強ければ、調整パラメータaを大きめに設定すればよい。この下限値を裏写りが除去できる程度の値に設定すれば、裏写りよりも低輝度の領域を背景と判断しない。また、上限値をl_Ｂとすることにより、l_Ｂ以上の輝度に対する背景処理が不要になる。
図８は、図５の入力画像に対するこの処理の結果を示すもので、背景（地肌）として抽出された画素を黒で示す。図８に示すように、文字（黒と白抜きを含む）や、文字の背景であっても中間調の背景を持つ(II)部分や黒の背景を持つ(III)部分（図５参照）は、地肌背景として抽出されない。
なお、ここでは、R_Ｗ２における輝度の平均l_Ｂと標準偏差σに基づいて、背景領域を決定したが、かわりに、R_Ｗ２における輝度分布のメディアンm_Ｂやαパーセント分位点α_Ｂ（αは例えば25）を用いて、α_Ｂ＜L＜m_Ｂであるような画素を背景領域としても良い。
【００２６】
・ 地肌の色置換（Ｓ１１０）
前段で抽出した背景（地肌）に対する画質の改善処理として、抽出背景領域を本来あるべき地肌色で置換し、裏写りや汚れなどのない背景とする。手順としては、原画像（Ｉ₀）において、前段の背景領域抽出手順（Ｓ１０９）で抽出した背景領域における画素の色を、背景（地肌）色の推定・統計量計算手順（Ｓ１０８）で求めたウィンドウ(W)において計算された背景の平均色によって置換する。この色置換は、抽出領域の全ての画素について行うので、置換後の背景は平均値として算出された背景（地肌）色により一様になる。
図９は、図５の入力画像に対して、上記した色置換を施した結果の画像を示す。同図に示すように、背景(I)部分は背景（地肌）とみなされる領域として抽出されるので、その領域は色置換の対象とされ、結果として一様な背景（地肌）色により塗られて、裏写りや汚れは消失する。
【００２７】
・ 地肌以外の領域の裏写り除去と階調補正（Ｓ１１１）
この手順では、地肌以外の領域に対する画質改善処理を行う。この場合、対象とする地肌以外の領域は、上記した背景領域抽出手順（Ｓ１０９）で抽出した背景領域によって得た結果を受け、地肌領域以外の領域を特定する。図８の例によると、地肌（黒で示したON画素領域）以外の領域（白で示したOFF画素領域）、即ち中間調の背景を持つ(II)部分や黒の背景を持つ(III)部分（図５参照）を対象とする。
ここで行う画質改善処理の一つは裏移りを除去する処理である。前段の地肌の色置換手順（Ｓ１１０）では、地肌領域に対する処理であったから、地肌以外の領域の裏写りは改善されない。これは、図９の例によると、裏写りが背景(II)部分の残存することにより示される。このような裏写りを対象に、次に示すような除去処理を適用することにより、画質を改善する。
ここで適用する裏写り除去方法は、対象とした入力画像の地肌に相当する背景領域に対する改善処理に連結させて一連の処理として適用するもので、図５に例示するような入力原画像（I₀）或いはその低解像度画像（I）のみにより、即ちそれ以外の条件を不要にして、処理が可能な方法を用いることが望ましい。
【００２８】
こうした入力条件に適応する裏写り除去方法は、複写機等に適用する処理方法として既に提案された方法が存在するので、この方法を適用することにより実施することが可能であり、例えば、多重スケール処理により、局所的に背景色を推定しながら、裏写り成分をこの背景色で置換する処理（特開2001-169080号公報、参照）を用いることができる。
例示する既存の方法による裏写り除去処理手順の概要は、次の(1)〜(4)に示す通りである。
(1) カラー原画像のエッジ強度の二値化
この処理は、原画像のエッジ強度を検出し、エッジ強度画像を生成し、この画像に対して閾値処理を適用することにより、裏写りエッジを除いた二値エッジ画像を得る。
(2) 局所的カラー閾値処理による背景色推定
この処理は、カラー原画像と上記(1)で得た二値エッジ画像を入力として、二値エッジ画像のoff画素のラン（所定数以下の連続画素列）として原画像のランを抽出し、各ランの代表色を縦・横方向を総合して求め、これを背景色として推定する。
(3) 推定背景色による原画像の裏写り除去
この処理は、上記(2)で得た推定背景色により原画像の二値エッジ画像のoff画素を置き換えることにより、裏写り画像を除去する。
(4) 多重スケールのエッジ差分解析による裏写り除去画像の修正
この処理は、上記(3)で得た裏写り除去画像に対する修正処理として行うもので、裏写り除去画像に対する多重スケールのエッジ差分解析による得られる差分画像におけるON画素の周囲に背景色推定処理を施し、さらに原画像で裏写り画像部分の判定を行い、この部分を最終的に修正する。
【００２９】
上記のようにして地肌以外の領域に対する処理により裏写り除去を行った画像には、さらに階調の補正を行う。この補正は、地肌の色置換手順（Ｓ１１０）で地肌色を置換した背景領域との階調不連続が生じないようにして、画像全体を高画質に保つために行う処理である。
即ち、上記裏写り除去処理手順により裏写りが除去された地肌以外の領域（図８の例では、白で示したOFF画素領域）の画素の各色成分信号R，G，Bに、地肌色と階調が連続するように補正する変換処理を施こす。この変換処理は、具体的には下記式(3)に示す関数に従う演算により変換信号を出力する。
【００３０】
【数３】

【００３１】
図１０は、式(3)に従う入出力の関係を表し、この変換処理の特性を示す線図である。図１０に示すように、入力信号が“ l_Ｂ−aσ ”になるまでは、線形の特性で、“ l_Ｂ−aσ ”を越え最大値に達するまでは、出力信号が“ l_Ｂ”一定値となる特性を持つ。
図１１は、地肌の色置換手順（Ｓ１１０）で地肌領域の画質改善を施した後の画像（図９に示す）に対して、上記した地肌以外の領域の裏写り除去と階調補正処理（Ｓ１１１）を施した結果の画像を示す。地肌以外の領域の裏写り除去については、図１１に示すように、中間調の背景を持つ(II)部分に残存していた裏写り（図９、参照）は、この手順により除去される。
また、階調補正処理については、入力信号が“ l_Ｂ−aσ ”になるまでは、それぞれの輝度に線形の関係で対応する輝度を出力するので、中間調の領域の輝度は原画像が保存され、“ l_Ｂ−aσ ”を越えると、“ l_Ｂ”一定値となるので、真の背景（地肌を示す図９の(I)部分）は、中間調の領域との階調が連続するとともに、裏写りや地肌の劣化が改善され一様な輝度の信号を生成する。
また、変換処理に用いる特性関数として、式(3)の代わりに、ウィンドウ(W)の明るい方に対応する画素群R_Ｗ２の輝度分布のメディアンm_Ｂやαパーセント分位点α_Ｂに基づいて定める上記式(4)を用いて、式(3)におけると同様の手順により階調補正処理を施すようにしても良い。
【００３２】
「実施形態２」
実施形態１では、実施装置の基本形態を示したが、本実施形態では、実施形態１の一部の手順における処理方法を変更することにより、処理の改善を図るものである。本実施形態では、地肌背景（図５の例では(I)部分）に直接印刷された黒文字に対して有効な処理方法を提案するものである。
背景に直接印刷された黒文字の場合、RGBすべての色信号においてコントラストが強くなる傾向があり、それがエッジ量に現れるので、この点に着目し、文字領域（C）の抽出を行う。つまり、実施形態１に示した「低解像度画像（I）における特徴量計算（Ｓ１０４）」と「低解像度画像（I）における文字領域（C）の抽出（Ｓ１０５）」を注目画素のエッジ量、即ち、隣接画素に対する輝度変化とその変化量、によって行う。
具体的には、特徴量計算（Ｓ１０４）において、注目画素におけるRGBの各色信号でエッジ量を計算し、得られる各色の最小値をその画素のエッジ量（特徴量）として定める。
次の文字領域（C）の抽出（Ｓ１０５）では、まず、前のＳ１０４で特徴量として算出した各画素のエッジ量を、文字領域（C）の要素を切り出すために予め決められた閾値と比較し、閾値よりも高ければ、その画素 [ｉ，ｊ] を文字領域（C）の要素に設定する。その後、実施形態１と同様にランのsmearingを行い、最終的に文字領域（C）を抽出する。
なお、上記のように特徴量計算（Ｓ１０４）および文字領域（C）の抽出（Ｓ１０５）をエッジ量に基づいて行うが、それ以外は、実施形態１と同様の手順を用いることにより裏写りや地肌の劣化の改善を図る本発明に係わる画像処理を実施することが可能である。また、エッジ量を計算する処理は、実施形態１において説明したように、地肌以外の領域の裏写り除去処理（Ｓ１１１）でも必要とするので、計算処理の共通化を図ることができる利点もある。
【００３３】
「実施形態３」
実施形態１では、実施装置の基本形態を示したが、本実施形態では、実施形態１の一部の手順において他の処理方法を追加することにより、処理の改善を図るものである。本実施形態では、地肌背景（図５の例では(I)部分）に直接印刷された黒文字に対して有効な処理方法を提案するものである。
背景に直接印刷された黒文字の場合、RGBすべての色信号においてコントラストが強くなる傾向があり、それがエッジ量に現れるので、この点に着目した処理方法を追加して、文字領域（C）の抽出を行う。つまり、実施形態１に示した「低解像度画像（I）における特徴量計算（Ｓ１０４）」と「低解像度画像（I）における文字領域（C）の抽出（Ｓ１０５）」において、さらに注目画素のエッジ量による処理手順を加えて、漏れのない抽出を行うようにするものである。
具体的には、特徴量計算（Ｓ１０４）において、注目画素におけるRGBの各色信号でエッジ量を計算し、得られる各色の最小値をその画素のエッジ量として定め、このエッジ量を各画素の特徴量として、文字領域（C）抽出（Ｓ１０５）では、文字領域（C）の要素を切り出すために予め決められた閾値と比較し、閾値よりも高ければ、その画素 [ｉ，ｊ] を、ウィンドウ(W)の統計量を用いる方法（「実施形態１」の背景の平均輝度l_Ｂ及び標準偏差σによる方法）の結果に従って得た文字領域（C）の要素に加えて設定した後、実施形態１と同様にランのsmearingを行い，最終的に文字領域（C）を求める。
なお、上記のように特徴量計算（Ｓ１０４）および文字領域（C）の抽出（Ｓ１０５）をエッジ量に基づいて行うが、それ以外は、実施形態１と同様の手順を用いることにより裏写りや地肌の劣化の改善を図る本発明に係わる画像処理を実施することが可能である。また、エッジ量を計算する処理は、実施形態１において説明したように、地肌以外の領域の裏写り除去処理（Ｓ１１１）でも必要とするので、計算処理の共通化を図ることができる利点もある。
【００３４】
「実施形態４」
上記実施形態１〜３では、背景（地肌）色の推定や地肌領域の抽出等のステップ（Ｓ１０３〜１０９）を行う際の計算量を減らすために、原画像（I₀）を低解像度画像（I）に変換する形態で実施する例を示したが、低解像度への変換過程で不可避のエラーが生じるので、本実施形態では、エラーをできるだけ回避することが必要であり、しかも十分な能力がハード資源に用意されている場合に対応することを意図し，低解像度への変換をしないで、原画像（I₀）に対して各ステップの処理手順を行うようにする。
本実施形態は、画像処理プログラムによる処理の実行手順のフローとして、上記各実施形態で示したフローチャート（図４）において、低解像度画像（I）の生成ステップ（Ｓ１０２）を省略することによりこの実施を可能にする。このステップを省略することにより、処理フローにおける平滑化、特徴量計算、文字領域（C）の抽出、ブロック分割、文字領域の２クラス分類、代表背景色の推定、背景領域の抽出の各処理ステップを原画像に対して実行するような形態で実施することができる。
【００３５】
「実施形態５」
本実施形態は、実施装置の基本形態として示した実施形態１の一部の手順における処理方法を変更することにより、実施形態１と異なる地肌背景（図５の例では(I)部分）の改善処理を行うようにするものである。
実施形態１における地肌の色置換（Ｓ１１０）において、前段の地肌領域抽出手順（Ｓ１０９）で抽出した地肌領域における画素の色を、ウィンドウ(W)において計算された背景（地肌）の平均色（Ｓ１０８）によって置換し、この色置換により背景を一様にする。従って、この平均色が、地肌の汚れを含んだ画素によるものである場合なども起きる可能性があり、地肌そのままであることを期待しているユーザを満足させない。
そこで、本実施形態では、地肌領域における画素を“白”に置き換えるようにして、背景を地肌そのままにする処理を行う。処理手順としては、地肌の色置換（Ｓ１１０）において、地肌領域抽出手順（Ｓ１０９）で抽出した地肌領域における画素を“白”、即ち最大輝度で出力する。
また、地肌以外の領域に対する処理として、実施形態１と同様に裏写り除去処理を行い、この処理により裏写り除去を行った画像には、さらに階調の補正を行う。この補正は、地肌の色置換手順（Ｓ１１０）で地肌色を置換した背景領域との階調不連続が生じないようにして、画像全体を高画質に保つために行う処理であるという点で、実施形態１と目的は変わらないが、本実施形態では、地肌の色置換手順（Ｓ１１０）で置き換えた地肌色が“白”であるから、この色に適合した階調補正を必要とする。
即ち、上記裏写り除去処理手順により裏写りが除去された地肌以外の領域（図８の例では、白で示したOFF画素領域）の画素の各色成分信号R，G，Bに、“白”とした地肌色と階調が連続するように補正する変換処理を施こす。この変換処理は、具体的には下記式(3)に示す関数に従う演算により変換信号を出力する。
【００３６】
【数４】

【００３７】
図１２は、式(5)に従う入出力の関係を表し、この変換処理の特性を示す線図である。図１２に示すように、入力信号が“ l_Ｂ−aσ ”になるまでは、線形の特性で、“ l_Ｂ−aσ ”を越え最大値に達するまでは、出力信号が最大値となる特性を持つ。なお、“ l_Ｂ−aσ ”におけるl_Ｂ、a、σは、上記した背景（地肌）色の推定・統計量計算（Ｓ１０８），地肌領域抽出手順（Ｓ１０９）において説明したと同義である。
図１３は、地肌の色置換手順（Ｓ１１０）で地肌領域の画質改善を施した後の画像（図９に示す）に対して、上記した地肌以外の領域の裏写り除去と階調補正処理（Ｓ１１１）を施した結果の画像を示す。地肌以外の領域の裏写り除去については、図１３に示すように、中間調の背景を持つ(II)部分に残存していた裏写り（図９、参照）は、この手順により除去される。
また、階調補正処理については、入力信号が“ l_Ｂ−aσ ”になるまでは、それぞれの輝度に線形の関係で対応する輝度を出力するので、“ l_Ｂ−aσ ”に達するまでの中間調の領域の輝度は原画像が保存され、“ l_Ｂ−aσ ”を越えると、最大値一定となるので、真の背景（原稿の地肌を示す図９の(I)部分）は、中間調の領域との階調が連続するとともに、“白”になって、裏写りや地肌の劣化が現れることはなく、画質が改善される。
また、変換処理に用いる特性関数として、式(5)の代わりに、ウィンドウ(W)の明るい方に対応する画素群R_Ｗ２の輝度分布のαパーセント分位点α_Ｂに基づいて定める上記式(6)を用いて、式(5)におけると同様の手順により階調補正処理を施すようにしても良い。
【００３８】
【発明の効果】
（１） 請求項１〜７の発明に対応する効果
入力画像から文字を囲む領域を抽出し、その領域において、輝度による文字（前景）・背景の分類法を用い、さらに真の背景（地肌）を表す基準領域の評価・選択をしてそこから代表背景色（地肌色）を推定し、推定地肌色によって原画像上の地肌領域とそれ以外の領域を特定し、前者の画素に対して推定した代表背景色（地肌色）により置換する処理、後者の画素に対して局所的に背景色を推定しながら、裏写り画素を推定背景色で置換する処理、という各々に適した裏写りの除去法を適用することにより背景・地肌の色変動や裏写りなどの画像劣化を改善するようにしたので、混合分布をもつヒストグラムの解析法や入力画像機器に関する特性情報、既定のパラメータを必要とする従来法よりも、より適正に、背景や地肌の色を保持した形で画像全体の高画質化を図ることが可能になる。
また、代表背景色（地肌色）やその統計量は入力画像の特徴から統計的処理により算出するようにしたことにより、入力機器に関する知識・特性情報、既定のパラメータを一切必要とせず、背景の色変動や裏写りなどの、入力画像の特性に適応するために、ユーザが指定するパラメータも単純になる。
また、文字とその隣接画素範囲（文字領域（C））に対する輝度による文字（前景）・背景の分類処理を、ブロック分割した領域ごとに局所適応的二値化を適用する処理法を用いて行うようにしたことにより、より正しい代表背景色を得ることを可能にする。
さらに、低解像度の画像を処理対象にすることにより、各処理での計算量を減らすことが可能になり、処理の高速化を可能とし、また、ハード資源に十分な能力が用意されていない場合にも適用可能となる。
（２） 請求項８，９の発明に対応する効果
コンピュータを、請求項１乃至７のいずれかに記載された画像処理装置が具備する手段として機能させるためのプログラムを提供し、またこのプログラムをコンピュータ読み取り可能な記録媒体に記録した形態で提供することにより、上記（１）の効果を容易に具現化することが可能になる。
【図面の簡単な説明】
【図１】 本発明に係わる画像処理装置をシステム要素として構築される画像処理システムの構成例を示す。
【図２】 本発明に係わる画像処理装置を構成するＰＣのモジュール構成を示す。
【図３】 本発明の実施形態に係わる画像処理プログラムにより実現する画像処理機能のブロックを示す。
【図４】 本発明の実施形態に係わる画像処理プログラムによる処理の実行手順を示すフローチャートである。
【図５】 処理対象画像として入力される原画像（I₀）の１例を示す。
【図６】 図５に例示した原画像（I₀）に対し文字領域（C）の抽出を行った結果とブロック分割を示す。
【図７】 図５の入力画像に対して、ウィンドウ(W)として設定された領域を示す。
【図８】 図５の入力画像に対する背景（地肌）抽出処理の結果を示すもので、抽出された画素を黒で示す。
【図９】 図５の入力画像に対し推定地肌色による色置換を施した結果の画像を示す。
【図１０】 階調補正の変換処理特性（式(3)に従う入出力関係）を示す線図である。
【図１１】 図９の入力画像に対して、裏写り除去処理と図１０の特性に従う階調補正処理を施した結果の画像を示す。
【図１２】 階調補正の変換処理特性（式(5)に従う入出力関係）を示す線図である。
【図１３】 図９の入力画像に対して、裏写り除去処理と図１２の特性に従う階調補正処理を施した結果の画像を示す。
【符号の説明】
１０…ＰＣ（パーソナル コンピュータ）、
２０…低解像度画像生成手段、２１…前処理（平滑化）手段、
２２…特徴量（エッジ量、ウィンドウ統計量）計算手段、
２３…文字領域抽出手段、 ２４…紙面色の統計量推定手段、
２５…地肌領域抽出手段、 ２８…地肌の色変換手段、
２９…地肌以外の裏写り除去・階調補正手段、
３０…画像入力機器、 ５０…画像出力機器、
７０…サーバ、 ９０…MFP、
１００…画像処理システム。[0001]
BACKGROUND OF THE INVENTION
The present invention relates to color image processing when a printed document is read by an input device such as a scanner and printed / displayed from a printer, a display, or the like as an output device based on a digital image obtained. An image processing apparatus that performs image processing to improve image quality, excluding image noise (such as show-through and dirt) that occurs on the background and background that degrades the image quality of the output image, and functions as a computer as an image processing means constituting the apparatus And a recording medium on which the program is recorded.
[0002]
[Prior art]
When a printed document is read by a color image input device such as a scanner, and the resulting digital image is printed from a color printer or displayed on the display without any correction or adjustment processing, conversion by the scanner at the time of input is possible. Due to differences or fluctuations in device conditions such as characteristics, the background and background colors may not be uniform, or "back-through" will occur where the content printed on the back side is shown through, causing the image to appear dirty overall A phenomenon occurs. Image noise removal processing that impairs image quality has been performed so far, and when the background or background color is white, processing called “background removal” or “background removal” The process of replacing with white is effective. For example, a process is known in which the background level is detected based on a histogram of pixel signal values, and a luminance signal equal to or higher than the background level is output as a white level to skip the background (the following [Patent Documents 1 to 3] below). reference). By such processing, when the background or background color is white, it is possible to prevent image deterioration due to scanner fluctuation or show-through.
Patent Document 4 below discloses a process for estimating the background color of a document and changing the pixel value of the target pixel using the difference between the background color and the pixel value of the processing target. Here, the background color is determined by searching for the brightest area of the histogram of the same color, and the following four backgrounds are used: (1) near white, contone (such as white copy paper and newspaper printing paper), ( 2) Near white, halftone (magazine printing paper, etc.), (3) Far white, contone (photograph, colored paper, etc.), (4) Far white, halftone (magazine printing paper, etc.) In this method, the background reference color is determined from the color distribution according to the identification result, and the pixel color is changed based on the color difference from each target pixel.
Further, in Patent Document 5 below, by analyzing the distribution state of the peak appearing in the histogram representing the gradation distribution, a threshold value for determining the type of the region is determined as image / background / other three types, A method is disclosed in which an image region is output without processing, a background region is converted into a background color, and processing is performed in other regions according to predetermined conditions.
[0003]
[Patent Document 1]
JP 2000-22971
[Patent Document 2]
Japanese Unexamined Patent Publication No. 2000-78408
[Patent Document 3]
JP 2002-27252 A
[Patent Document 4]
JP 2000-50083
[Patent Document 5]
JP 2001-45297
[0004]
[Problems to be solved by the invention]
However, in a color document image, the color of the background or background is arbitrary, the background structure is complicated, and it is often composed of a plurality of background colors. FIG. 5 shows an example of such a case. This example shows an example in which the backgrounds of characters placed at multiple locations have different densities (color components). The (I) part showing the background of the document is the (II) part with a halftone background. It has a brighter (lower density) background than part (III) with a black background.
In the case of such a document image, simply examining the color and gradation distribution results in a “mixed distribution” in which many peaks and valleys appear in the histogram, making it difficult to estimate the true background color. Become. In Patent Documents 1, 2, and 5 described above, a method for extracting the background / background color from such a mixed distribution is described. However, since the histogram includes a lot of peaks and valleys due to noise, it is accurately desired. The background and background color cannot always be extracted, and errors increase. Note that Patent Document 3 discloses a method of restricting an image used for a histogram as a countermeasure against noise caused by a halftone dot image that erroneously estimates a true background color. However, the method is limited to adaptation to noise caused by a halftone dot. ing. In Patent Document 4, the background / background is identified as a contone and a halftone (halftone dot), but the halftone dot identification strongly depends on the frequency characteristics (MTF: Modulation Transfer Function) of the input device. Since it is difficult to apply when the input device is unknown, such as data sent from a remote place via a network, a correct identification result is not guaranteed.
[0005]
By the way, in order to avoid the above-described problems caused by the conventional methods exemplified in Patent Documents 1 to 5, the background structure as shown in FIG. A method for improving the image quality by removing the show-through and background stains occurring in the background for the structured document image has been proposed by the present applicant (Japanese Patent Application No. 2003-48834, hereinafter). This is called “preceding example”).
In this preceding example, the background pixels are extracted based on the image features of the input image, the background (background) color is estimated (that is, the character region including the character and surrounding pixels is extracted, the background pixels are classified therefrom, and the background pixels are extracted. Estimate the representative background color as the target), extract an area regarded as the background on the original image based on the estimated representative background color, and replace the color of the extracted area with the estimated representative background color or white In this way, the image quality is improved by removing the show-through of the background area and the background stain.
However, here, the background pixel of each character area is evaluated so that the estimated representative background color is the color of “true background”, that is, “background”, and the area considered as the background is extracted according to the evaluation. Therefore, the area where the image quality is improved is an area corresponding to the background. Therefore, the above-described image quality improvement processing is not performed on the background area in which the background color is applied to the background of the character, and is left as it is (for example, the improvement processing is performed on FIG. 5). As shown in FIG. 9, even if the show-through of the background (I) is removed, the show-through of the background color (II) remains). It becomes insufficient.
The present invention has been made in view of the problems in the above-described conventional method and the prior-art method for processing an input image to improve the image quality. The object of the present invention is that the structure of the background is complicated, and the background structure is plural. When a document image consisting of a background color of the image is targeted, the analysis result of the histogram composed of the mixture distribution, knowledge / characteristic information about the input image device, and the conventional method that requires default parameters are used. In addition, it is possible to apply an image quality improvement method that identifies the background (background) and other areas based on the local data of the input image, and eliminates factors that degrade image quality such as show-through and dirt, which are suitable for each. By doing so, it is to improve the image quality of the entire image more appropriately including not only the background but also the background region to which the background color is applied.
[0006]
[Means for Solving the Problems]
  The invention according to claim 1 extracts a character and its adjacent pixel range from the original digital image to be processed as a character region.Character area extractionMeans,Block dividing means for dividing the image into a plurality of blocks, and in the divided blocksPixels belonging to the extracted character areagroupThe color byA first class corresponding to the character color and a second class corresponding to the background colorClassifyClassificationMeans,SecondClassified into a classThe block selection means for selecting the block having the maximum evaluation function value based on the number of pixels and the luminance value is divided into a pixel group corresponding to the darker one and a pixel group corresponding to the brighter one in the selected block. The color of the corresponding pixel group in the original imageRepresentative background colorWhenpresumeRepresentative background color estimationAnd a background region having a color other than the representative background color on the original image based on the estimated representative background color.Background region extractionMeans and background area other than the extracted representative background colorShow-through removal to remove show-throughAn image processing apparatus.
  According to a second aspect of the present invention, in the image processing apparatus according to the first aspect, a background area extracting unit that extracts a background area that is considered to have the representative background color on the original image based on the estimated representative background color; Means for replacing the color of the background area considered to have the extracted representative background color with the estimated representative background color.
  According to a third aspect of the present invention, in the image processing apparatus according to the first aspect, background color area extracting means for extracting a background area that is regarded as having the representative background color on the original image based on the estimated representative background color; And means for replacing the color of the background area considered to have the extracted representative background color with white.
  According to a fourth aspect of the present invention, in the image processing apparatus according to any one of the first to third aspects of the present invention, the image processing apparatus further comprises means for correcting a gradation for a background region other than the representative background color. .
  According to a fifth aspect of the present invention, in the image processing apparatus according to any one of the first to fourth aspects, the character area extracting unit calculates a characteristic amount of an image from an original digital image to be processed. It is characterized by comprising.
  According to a sixth aspect of the present invention, in the image processing apparatus according to any one of the first to fifth aspects, the classification unit calculates a luminance from a color signal of each pixel and performs threshold processing on the calculated luminance. It is a means for classifying into first and second classes.
  A seventh aspect of the present invention is the image processing apparatus according to any one of the first to sixth aspects, wherein the feature amount calculating means, the character region extracting means, the block dividing means, the classification means, and the representative background color estimation. At least one of the means, the background color area extracting means, and the background area extracting means having a color other than the representative background color is a means comprising means for generating a reduced image whose resolution is lower than that of the original image. And
  The invention according to claim 8 is an image processing program for causing a computer to function as means included in the image processing apparatus according to any one of claims 1 to 7.
  The invention of claim 9 is a computer-readable recording medium on which the program according to claim 8 is recorded.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
  The present invention eliminates show-through and dirt in the background region of a document image and improves the image quality, and uses a conventional analysis result of a histogram composed of a mixture distribution (described above [Conventional] Technology] [see the problem to be solved by the invention]), and can improve the image quality not only for the background but also for the background area where colors other than the background (background color) are applied. An object of the present invention is to improve the image quality of the entire image by applying the processing method.
  In the present invention, the backgroundChichiCan be applied to areas corresponding to skinWhoSince the method cannot be applied to a region other than the background, the above-described problem of improving the image quality of the entire image by applying another improvement process to the region other than the background is solved.
  That is, for the background region corresponding to the background, a processing method according to the following principle based on the image characteristics of the input image (that is, based on the local image information) is applied to identify the background region to be improved. By capturing the representative color of the area, the problem caused by the conventional method using the analysis result of the histogram is avoided.
  The principle of this processing method will be described. When a color document image is viewed locally, it is composed of two types of areas, a foreground and a background. Reduce the quantity estimation problem into a simple two-class problem,
(1) Search for a partial area that includes the true background (background) color.
(2) Classify the area into two classes, foreground and background.
(3) Extract the background color and background representative color and degree of variation from the background color and gradation distribution.
Thus, the statistics necessary for the improvement process are obtained.
  Further, a background region corresponding to the background is extracted from the entire image based on the statistics thus obtained, and this region is replaced with the estimated representative color.
[0012]
On the other hand, since the improvement processing method used for the background area cannot be applied to an area other than the area corresponding to the background, another image quality improvement processing method (in the following embodiments, removal of show-through) is not applied to this area. In this example, “the show-through removal method” is applied. In this case, the target region receives a result obtained by the region extraction method corresponding to the above-described background, and specifies a region other than the background region.
Here, the show-through removal method applied to the area other than the background area is applied as a series of processes in conjunction with the improvement process for the background area corresponding to the background of the target input image. A versatile method that can be applied to an input image under conditions equivalent to those in processing is desirable. In other words, removal processing is possible without the need for special additional information about the target image other than the target input document image (for example, the input is only the scanner-read image of the show-through manuscript, otherwise it is unconditional) This method is desirable. Further, there is a method of removing the show-through adapted to such an input condition as a processing method that has already been proposed as a processing method applied to a copying machine or the like, and can be implemented by applying this method (details). Is exemplified later).
Further, gradation conversion is performed on the area other than the area corresponding to the background so that the gradation discontinuity with the background area does not occur. Since the target area in this case is the same as the area subjected to the show-through removal method, gradation correction is performed on the image after the show-through removal process as a series of processes linked to the show-through removal method. .
In this image quality improvement processing for the entire image, as described above, the data required for the processing is calculated by statistical processing from the image features, and processing without any knowledge / characteristic information or default parameters regarding the input image equipment is required. Enable. Therefore, this processing method can be said to be an effective method when processing data in which the input device is unknown, as in the case of transmission from a remote location via a network.
[0013]
An image processing apparatus according to the present invention will be described with reference to the following embodiments shown in the accompanying drawings. In the embodiment described below, an example in which the image processing apparatus according to the present invention is implemented in the form of an image processing system constructed as a system element is shown.
FIG. 1 shows a configuration of an image processing system according to the present embodiment. This image processing system 100 constitutes a server client system, and as shown in FIG. 1, a PC (personal computer) 10 serving as a client on a network such as a LAN, and a server 70 that centrally manages information and resources in the system. And an image input device 30 such as a scanner or a digital camera, an image output device 50 such as a printer, and the MFP 90 are connected.
The image processing apparatus according to the present invention is configured on the client PC 10. The client PC 10 requests the image input device 30, the image output device 50, and the MFP 90 managed by the server 70 to input / output a document image to be processed. Further, the client PC 10 performs processing by the image processing apparatus according to the present invention on the basis of the input image from the image input device 30 or MFP 90 that has requested processing, and outputs the image output device 50 or MFP 90 or the output unit in the PC 10. The image is output using.
Further, the image processing system 100 is linked to a server client system 100 ′ capable of image processing similar to the image processing system 100 by connecting between networks as shown in FIG. 1, or connected to an external environment via the Internet. The system is linked with an external system by a method, accepts an image generated outside, and is constructed as a system capable of outputting an image generated in the image processing system 100.
[0014]
  FIG. 2 is a module configuration diagram of a PC constituting the image processing apparatus according to the present invention.
  A PC whose module configuration is illustrated in FIG. 2 is a general-purpose computer that can be connected to a network, as shown in FIG. 2, and includes CPU 11, memory 12 such as ROM, RAM, and HDD (hard disk drive) 13 as constituent elements. A removable disk device 14 such as a CD-ROM, a display device (display) 15, a keyboard 16, a pointing device 17 such as a mouse, a network interface 18, etc. are prepared and connected by a data bus.
  In addition, a part of the storage medium used by the memory 12, the HDD 13, and the removable disk device 14 as storage means realizes the image processing function according to the present invention and realizes the image processing procedure described in the following embodiment. An image processing program (software) is recorded.
  The document document image to be processed is input by an image input device 30 such as a scanner and is stored in the HDD 13 or the like, for example. CPU11Reads out a program that realizes the above-described image processing function from a recording medium included in the storage unit, executes processing according to the program on the target document image stored in the HDD 13 or the like, and outputs the processing result or the like to the image output device 50 or the MFP 90. Alternatively, the image data is output for image output using the display device 15 in the PC 10. Note that a document image to be processed that is input / output to use the image processing function of the PC 10 may be exchanged with an external system connected to the network.
[0015]
FIG. 3 shows a block of an image processing function realized by the image processing program according to the present embodiment.
The outline of the processing by the image processing function shown in FIG. 3 is as follows.
Many characters are printed on the document image, but a normal document has a portion in which black characters are directly printed on a portion where no background is printed (background, background). So, if an area where black characters are likely to be extracted is extracted from the input image, and then the input image is divided into sufficiently small blocks, a block in which black characters are directly printed on the background (background) Can be assumed to exist. Here, the flow of processing is defined according to this assumption, and first, a character region (C) is extracted by local adaptive threshold processing and expansion processing. This process is performed by the function blocks of preprocessing (smoothing) 21, feature amount (edge amount, window statistic) calculation 22, and character region extraction 23 shown in FIG.
Next, the input image is divided into fixed-size blocks that do not overlap each other. In each block, the pixels belonging to the character area (C) are classified into two colors. Usually, a bright color (high luminance) corresponds to a background color of a character area, and a dark color corresponds to a character color. Then, a block is selected so that the number of pixels classified into the high luminance class and the value of the evaluation function determined based on the luminance value are maximized. The brighter representative color in this block is set as the average background color in the input image, and the statistics (average, standard deviation) of brightness (eg, RGB signal average: (r + g + b) / 3) are set. ). In this way, the color of the background (background) can be accurately estimated. Based on these statistics, a background (background) region is extracted. This process is performed by the functional blocks of the paper surface (background) color statistic estimation 24 and the background area extraction 25 shown in FIG. Here, in order to reduce the amount of calculation, since the processing process from the preprocessing (smoothing) 21 to the background area extraction 25 is performed with an image with reduced resolution, the low-resolution image generation 20 is applied at the beginning.
Then, the color of the extracted background (background) area is replaced with the estimated average color of the background (background) area, and for the pixels other than the background, the show-through removal processing is performed, and further the background (background) area Gradation correction is performed so as not to cause gradation discontinuity. This processing process is performed by the function blocks of the background color replacement 29 and the show-through removal / gradation correction 30 other than the background shown in FIG. 3, and is applied to the original resolution image.
[0016]
Hereinafter, embodiments of the image processing apparatus will be described in more detail as “Embodiment 1” to “Embodiment 5”. “Embodiment 1” shows a basic form of the implementation apparatus, and other embodiments show forms in which a part of the basic form is changed. Therefore, the flowchart (FIG. 4) showing the execution procedure in “Embodiment 1” is common to each embodiment.
“Embodiment 1”
FIG. 4 is a flowchart showing an execution procedure of processing by the image processing program of this embodiment.
By referring to this flowchart, the image processing of the present embodiment for improving the background image quality will be described in detail step by step.
・ Original image (I₀) Input / reception (S101)
In this example, a document image is generated by the image input device 30 such as a scanner in response to a request from the client PC 10, and the PC 10 side receives the document image from the image input device 30 and performs processing according to the present invention, and then outputs the image. In a series of processing of requesting the image output device 50, a processing procedure according to an image processing program executed on the PC 10 side is shown. Accordingly, the processing procedure of this example is as follows. First, the original document image (I, which is represented by the color image (R, G, B) signal from the image input device 30.₀) Is received and input for each of the R, G, and B channels as the processing target image, and the processing starts.
Note that the original image (I₀) Is shown in FIG. In the example shown in the figure, characters are arranged at a plurality of locations, and an example is shown in which backgrounds for characters at each location have different densities. In this example, the (I) portion showing the background of the document has a background (I) having a higher luminance (low density) than the (II) portion having a halftone background and the (III) portion having a black background. ) Part shows low-density show-through (indicated by “face-down letters”) or background noise (indicated by “dashed line”), and show-through similar to that in the background (I) part on one of the background (II) parts (Indicated by “face-down characters”). In the present embodiment, an example in which image quality is improved by applying different processing methods to the background portion of the background (I) and the background (II) (III) other than the background is shown.
[0017]
・ Generation of low resolution image (I) (S102)
In this step, in order to reduce the amount of calculation when performing steps (S103 to 109) such as estimation of the background (background) color and extraction of the background (background) area shown below, the original image (I₀) To a low resolution image (I) of about 100dpi. In the conversion to the low resolution image, the pixels of the input image are simply divided into blocks without overlapping, and the average value of the signals of the pixels included in the block is set to the signal of the corresponding pixel in the low resolution image. When the block size, that is, the reduction ratio is 1 / r, the low resolution image (I) can be calculated by the following equation (1). Here, the original color image (I₀) Is expressed by a two-dimensional pixel matrix of each color component (R, G, B), and a low resolution image (I) is obtained for each of R, G, B.
[0018]
[Expression 1]

[0019]
In the above equation (1), r that determines the reduction ratio is always a natural number. Original image (I₀) Is not divisible by 100, r is rounded to a natural number. For example, in the case of 360 dpi, r is 3, and in this case, the low resolution image (I) is 120 dpi.
・ Smoothing of low resolution image (I) (S103)
The low resolution image (I) may generate noise in the low resolution generation process. Further, the original original image may include noise generated on the image input device 30 or the network (particularly, the input image transmitted from the external system is highly likely). Since such noise becomes an obstacle in subsequent processing, the noise is removed by smoothing the low resolution image (I) obtained in the previous stage with a linear filter.
-Feature amount calculation in low resolution image (I) (S104)
For the smoothed low-resolution image (I), a calculation for obtaining a data value representing the feature of the image is performed. In the present embodiment, a predetermined (fixed) size window (including a two-dimensional array of pixels) is set around each pixel, and the average value μ of the luminance (density) signal possessed by the pixels in the window and the average of the target pixel Calculate the standard deviation σ for the value μ. These feature quantities are calculated for each of R, G, and B.
[0020]
-Extraction of character area (C) in low-resolution image (I) (S105)
Next, the character region is extracted and detected by performing local adaptive binarization of the color component on the low resolution image (I).
Specifically, for each of the color components R, G, and B, the threshold value μ using a and b as parameters using the average value μ and the standard deviation σ calculated as the feature values in the feature value calculation procedure (S104). (a + bσ) is set, and the threshold μ (a + bσ) is compared with the signal value of the low resolution image (I). In the case of black characters printed directly on the background, it is noted that the contrast tends to be strong in all RGB component signals. If the signal value is lower than the threshold value in all channels, the pixel [i, j] It is appropriate to use a method in which is set as an element of the character area (C). That is, the element of the character area (C) is obtained according to the following equation (2).
[0021]
[Expression 2]

[0022]
  Next, a binary image (P) having the same size as the low-resolution image (I) and having the value ON at the pixel positions that are the elements of the character region (C) obtained above and the value OFF otherwise is formed. . In this binary image (P), a background pixel run in the horizontal direction (a connected pixel row in which pixels in which the image (P) is OFF) is extracted. If the length of the extracted run is shorter than the threshold τ, the pixel is temporarily turned on. In this way, the background pixel runs in the horizontal directionSmearing ( smearing )To do. Similarly, in the vertical direction, the background pixel runSmearingI do. And the horizontal and vertical directionsSmearingTake AND of bothSmearingOnly those that are turned ON in the final are determined as character areas (C) as monochrome foreground pixels and extracted.
  FIG. 6 shows an original image (I) input to the scanner and exemplified in FIG.₀) Shows the result of extracting the character area (C) through the processing so far. As shown in FIG.SmearingAs a result, the character area (C) is extracted in such a form that the ON in the rectangular area surrounding the character (area (IV) in the figure) is set.
[0023]
・ Original image (I₀) Block division (S106)
Next, a process for searching for the background is performed for the character region (C) extracted above. At this time, the original image (I₀) Are divided into blocks, and processing is performed for each block unit so as to obtain a true background, that is, “background”.
The blocks are divided into sufficiently small blocks that do not overlap each other, and FIG. 6 shows a case where the low resolution image (I) is performed. As shown in FIG. 6, the size and shape of each block may be, for example, a square having a side length of 20 mm (80 pixels at 100 dpi).
-Classify the character area (C) into two classes for each divided block (S107)
Next, as a process of searching for the true background (background portion of the document), the character region (C) extracted in the character region (C) extraction procedure (S105) within each block determined in the division step. The pixel to which it belongs is classified into two classes according to luminance. For this classification, a method of calculating the luminance from the color signal of each pixel and performing threshold processing is used. For the threshold processing, a known method such as a discriminant analysis method, a moment holding method, an entropy method, or the like can be used. In block i, the pixel group corresponding to the darker one of the classified pixels is defined as the first class R._i1The second class R is the pixel group corresponding to the brighter one._i2Then R_i1Is the letter, R_i2Corresponds to the background (background).
[0024]
-Background (background) color estimation / statistics calculation (S108)
As a step of the process of searching for a true background (document background), the second class R obtained in step S107 is then used._i2Number of pixels belonging to N_iAnd R_i2Average brightness L_iAn evaluation function based on, for example, N_i× L_iDefine a function that takes a larger value as the number of pixels increases and the average luminance is higher, finds the block with the largest value, and sets the character area (C) as the window (W) To do.
FIG. 7 shows an area set as a window (W) for the input image of FIG. The pixel group corresponding to the darker side in the region set as the window (W) is the pixel group R._W1, Pixel group R corresponding to the brighter one_W2And dark gray and light gray respectively. Pixel group R corresponding to the brighter one_W2Is obtained as an estimation result of the true background color (representative background color) in the original image (FIG. 5).
Furthermore, the pixel group R corresponding to the true background_W2The average l as the luminance statistic_BAnd standard deviation σ is calculated.
[0025]
・ Background area extraction (S109)
Statistics of the luminance of the true background in the window (W) obtained in the background (background) color estimation / statistic calculation procedure (S108)_BAnd a standard deviation σ), a region regarded as a background (background) is extracted. In the low-resolution image (I), the luminance L is l_B−aσ <L <l_BIt is assumed that a pixel that is (a is a positive parameter) is regarded as a background (background) to be extracted.
In the above inequality indicating extraction conditions, lower limit l_BThe parameter a at −aσ is determined by the background fluctuation, the degree of show-through, and the like. For example, if the show-through is strong, the adjustment parameter a may be set larger. If this lower limit value is set to a value that can remove the show-through, the lower brightness area than the show-through is not determined as the background. Also, set the upper limit to l_BBy l_BBackground processing for the above luminance becomes unnecessary.
FIG. 8 shows the result of this processing for the input image of FIG. 5, and the pixels extracted as the background (background) are shown in black. As shown in FIG. 8, the character (including black and white), the (II) portion having a halftone background even if it is the background of the character, and the (III) portion having a black background (see FIG. 5) Is not extracted as a background.
Here, R_W2Average luminance at_BAnd the background deviation was determined based on the standard deviation σ._W2The median of the luminance distribution at_BAnd α percentile quantile α_B(Α is 25, for example)_B<L <m_BSuch a pixel may be used as the background region.
[0026]
・ Background color replacement (S110)
As an image quality improvement process for the background (background) extracted in the previous stage, the extracted background area is replaced with a background color that should be originally, and a background without show-through or dirt is obtained. As a procedure, the original image (I₀), The background color calculated in the window (W) obtained in the background (background) color estimation / statistics calculation procedure (S108) with the pixel color in the background region extracted in the background region extraction procedure (S109) in the previous stage. Replace with the average color. Since this color replacement is performed for all the pixels in the extraction region, the background after replacement is uniform with the background (background) color calculated as an average value.
FIG. 9 shows an image obtained as a result of performing the above-described color replacement on the input image of FIG. As shown in the figure, the background (I) part is extracted as an area considered as the background (background), so that area is subject to color replacement, and as a result, it is painted with a uniform background (background) color. Show-through and dirt disappear.
[0027]
-Removal of show-through and gradation correction in areas other than the background (S111)
In this procedure, image quality improvement processing is performed on an area other than the background. In this case, the region other than the target background is identified by receiving the result obtained by the background region extracted in the background region extraction procedure (S109) and specifying the region other than the background region. According to the example of FIG. 8, the area (OFF pixel area shown in white) other than the background (ON pixel area shown in black), that is, the (II) portion having a halftone background and the black background (III) Part (see FIG. 5) is the target.
One of the image quality improvement processes performed here is a process for removing the show-off. Since the background color replacement procedure (S110) in the previous stage is processing for the background area, the show-through of the area other than the background area is not improved. According to the example of FIG. 9, this is indicated by the show-through remaining in the background (II) portion. The image quality is improved by applying the following removal process to such show-through.
The show-through removal method applied here is applied as a series of processes in connection with the improvement process for the background region corresponding to the background of the target input image. The input original image (I₀It is desirable to use a method capable of processing only with the low-resolution image (I), that is, without any other conditions.
[0028]
The show-through removal method that adapts to such input conditions already exists as a processing method applied to a copying machine or the like, and can be implemented by applying this method. A process (see JP 2001-169080 A) that replaces the show-through component with the background color while locally estimating the background color by the process can be used.
The outline of the show-through removal processing procedure by the existing method illustrated is as shown in the following (1) to (4).
(1) Binary image edge intensity
In this process, the edge strength of the original image is detected, an edge strength image is generated, and threshold processing is applied to this image to obtain a binary edge image excluding the show-through edge.
(2) Background color estimation by local color threshold processing
In this process, the color original image and the binary edge image obtained in the above (1) are input, and the run of the original image is extracted as a run of off-pixels of the binary edge image (a predetermined pixel sequence or less), The representative color of each run is obtained in the vertical and horizontal directions, and this is estimated as the background color.
(3) Remove show-through of original image with estimated background color
In this process, the show-through image is removed by replacing the off pixel of the binary edge image of the original image with the estimated background color obtained in (2) above.
(4) Correction of show-through removal image by multi-scale edge difference analysis
This process is performed as a correction process for the show-through removed image obtained in (3) above, and background color estimation processing is performed around the ON pixels in the difference image obtained by multiscale edge difference analysis for the show-through removed image. Furthermore, the show-through image portion is determined from the original image, and this portion is finally corrected.
[0029]
As described above, gradation correction is further performed on the image from which the show-through has been removed by the processing for the area other than the background. This correction is a process performed in order to keep the entire image in high image quality so that gradation discontinuity with the background area in which the background color is replaced in the background color replacement procedure (S110) does not occur.
That is, the background color and the color of each color component signal R, G, B of the pixels other than the background from which the show-through has been removed by the show-through removal processing procedure (OFF pixel area shown in white in the example of FIG. 8) A conversion process is performed to correct the gradation to be continuous. Specifically, this conversion process outputs a converted signal by an operation according to a function shown in the following equation (3).
[0030]
[Equation 3]

[0031]
FIG. 10 is a diagram showing the input / output relationship according to the equation (3) and showing the characteristics of this conversion process. As shown in FIG. 10, the input signal is “l_BUntil “−aσ”, it is a linear characteristic, “l_BThe output signal is “l” until it reaches the maximum value exceeding −aσ_B“Has a constant value.
FIG. 11 shows the above-described background removal and gradation correction processing for an area other than the background for the image (shown in FIG. 9) after image quality improvement for the background area in the background color replacement procedure (S110). The image of the result of giving S111) is shown. As for the removal of the show-through in the area other than the background, the show-through (see FIG. 9) remaining in the (II) portion having the halftone background is removed by this procedure, as shown in FIG.
For tone correction processing, the input signal is “l_BUntil -aσ ", the brightness corresponding to each brightness is output in a linear relationship. Therefore, the brightness of the halftone area is stored as the original image._BWhen “−aσ” is exceeded, “l_B“Because of the constant value, the true background (part (I) in FIG. 9 showing the background) has continuous gradation with the halftone area, and the brightness of the show-through and background are improved and uniform brightness. Generate a signal.
Also, as a characteristic function used for the conversion process, a pixel group R corresponding to the brighter side of the window (W) instead of the equation (3)_W2The median of the luminance distribution of m_BAnd α percentile quantile α_BThe tone correction process may be performed by the same procedure as in the equation (3) using the equation (4) determined based on the above.
[0032]
“Embodiment 2”
In the first embodiment, the basic form of the implementation apparatus is shown, but in this embodiment, the processing is improved by changing the processing method in a part of the procedure of the first embodiment. The present embodiment proposes a processing method effective for black characters printed directly on the background (the (I) portion in the example of FIG. 5).
In the case of a black character printed directly on the background, the contrast tends to be strong in all RGB color signals, and this appears in the edge amount. Therefore, focusing on this point, the character region (C) is extracted. In other words, the “feature amount calculation in the low resolution image (I) (S104)” and the “character region (C) extraction in the low resolution image (I) (S105)” shown in the first embodiment are used as the edge amount of the target pixel, That is, it is performed by the luminance change and its change amount with respect to adjacent pixels.
Specifically, in the feature amount calculation (S104), the edge amount is calculated from each RGB color signal in the target pixel, and the minimum value of each color obtained is determined as the edge amount (feature amount) of the pixel.
In the extraction of the next character region (C) (S105), first, the edge amount of each pixel calculated as the feature amount in the previous S104 is compared with a predetermined threshold value for cutting out the elements of the character region (C). If it is higher than the threshold, the pixel [i, j] is set as an element of the character area (C). Thereafter, run smearing is performed in the same manner as in the first embodiment, and the character region (C) is finally extracted.
As described above, the feature amount calculation (S104) and the character region (C) extraction (S105) are performed based on the edge amount. Otherwise, show-through and It is possible to carry out image processing according to the present invention for improving the deterioration of the background. In addition, as described in the first embodiment, the processing for calculating the edge amount is also necessary for the show-through removal processing (S111) of the region other than the background, and thus there is an advantage that the calculation processing can be shared. .
[0033]
“Embodiment 3”
In the first embodiment, the basic form of the implementation apparatus is shown. However, in this embodiment, the processing is improved by adding another processing method in a part of the procedure of the first embodiment. The present embodiment proposes a processing method effective for black characters printed directly on the background (the (I) portion in the example of FIG. 5).
In the case of black characters printed directly on the background, there is a tendency for the contrast to increase in all RGB color signals, and this appears in the edge amount. Therefore, a processing method focusing on this point is added, and the character area (C) Perform extraction. That is, in the “feature amount calculation in the low resolution image (I) (S104)” and the “character region (C) extraction in the low resolution image (I) (S105)” shown in the first embodiment, the edge of the target pixel is further increased. A processing procedure based on volume is added to perform extraction without leakage.
Specifically, in the feature amount calculation (S104), the edge amount is calculated from each RGB color signal in the target pixel, the minimum value of each color obtained is determined as the edge amount of the pixel, and this edge amount is defined as the feature amount of each pixel. As a quantity, in the character area (C) extraction (S105), it is compared with a predetermined threshold value for cutting out the elements of the character area (C), and if it is higher than the threshold value, the pixel [i, j] A method using the statistic of (W) (the average luminance of the background in “Embodiment 1”)_BIn addition to the elements of the character area (C) obtained according to the result of the standard deviation σ), run smearing is performed in the same manner as in the first embodiment to finally obtain the character area (C).
As described above, the feature amount calculation (S104) and the character region (C) extraction (S105) are performed based on the edge amount. Otherwise, show-through and It is possible to carry out image processing according to the present invention for improving the deterioration of the background. In addition, as described in the first embodiment, the processing for calculating the edge amount is also necessary for the show-through removal processing (S111) of the region other than the background, and thus there is an advantage that the calculation processing can be shared. .
[0034]
“Embodiment 4”
In the first to third embodiments, in order to reduce the amount of calculation when performing steps (S103 to 109) such as background (background) color estimation and background area extraction, the original image (I₀) Is converted into a low-resolution image (I). However, inevitable errors occur during the conversion to low-resolution images. In this embodiment, it is necessary to avoid errors as much as possible. In addition, it is intended to support the case where sufficient capacity is provided in the hardware resource, and without converting to the low resolution, the original image (I₀) For each step.
In the present embodiment, the flow of the processing procedure executed by the image processing program is omitted by omitting the low-resolution image (I) generation step (S102) in the flowchart (FIG. 4) shown in the above embodiments. Enable. By omitting this step, each processing step of smoothing in the processing flow, feature amount calculation, character area (C) extraction, block division, character class two-class classification, representative background color estimation, background area extraction Can be implemented in such a manner that is performed on the original image.
[0035]
“Embodiment 5”
The present embodiment improves the background of the background (part (I) in the example of FIG. 5) different from that of the first embodiment by changing the processing method in the part of the procedure of the first embodiment shown as the basic form of the implementation apparatus. It is intended to perform processing.
In the background color replacement (S110) in the first embodiment, the pixel color in the background area extracted in the previous background area extraction procedure (S109) is used as the average color (S108) of the background (background) calculated in the window (W). ), And this color replacement makes the background uniform. Therefore, there is a possibility that this average color is caused by pixels including background stains, which does not satisfy a user who expects the background to be intact.
Therefore, in the present embodiment, a process of leaving the background as it is is performed by replacing the pixels in the background area with “white”. As a processing procedure, in the background color replacement (S110), the pixels in the background area extracted in the background area extraction procedure (S109) are output with “white”, that is, with the maximum luminance.
Further, as a process for a region other than the background, a show-through removal process is performed in the same manner as in the first embodiment, and tone correction is further performed on an image from which the show-through has been removed by this process. This correction is performed in order to prevent gradation discontinuity from the background area in which the background color is replaced in the background color replacement procedure (S110) and to maintain the entire image with high image quality. Although the purpose is not different from that of the first embodiment, in this embodiment, the background color replaced in the background color replacement procedure (S110) is “white”, so that gradation correction suitable for this color is required.
That is, “white” is added to each color component signal R, G, B of the pixel in the region other than the background from which the show-through has been removed by the show-through removal processing procedure (OFF pixel region shown in white in the example of FIG. 8). A conversion process is applied to correct the background color and gradation to be continuous. Specifically, this conversion process outputs a converted signal by an operation according to a function shown in the following equation (3).
[0036]
[Expression 4]

[0037]
FIG. 12 is a diagram showing the input / output relationship according to the equation (5) and showing the characteristics of this conversion process. As shown in FIG. 12, the input signal is “l_BUntil “−aσ”, it is a linear characteristic, “l_BThe output signal has the maximum value until it exceeds -aσ "and reaches the maximum value._B-Aσ "_B, A, and σ have the same meanings as described in the background (background) color estimation / statistical calculation (S108) and background area extraction procedure (S109).
FIG. 13 shows the above-described background removal and gradation correction processing for an area other than the background for the image (shown in FIG. 9) after image quality improvement of the background area in the background color replacement procedure (S110). The image of the result of giving S111) is shown. Regarding the show-through removal of the area other than the background, the show-through (see FIG. 9) remaining in the (II) portion having a halftone background is removed by this procedure, as shown in FIG.
For tone correction processing, the input signal is “l_BUntil “−aσ”, the corresponding luminance is output in a linear relationship with each luminance._BThe brightness of the halftone area up to -aσ "is stored as the original image._BWhen the value exceeds -aσ ", the maximum value is constant. Therefore, the true background (portion (I) in FIG. 9 showing the background of the original) has a continuous tone with the halftone area and becomes" white ". Thus, there is no show-through or deterioration of the background, and the image quality is improved.
In addition, as a characteristic function used for the conversion process, a pixel group R corresponding to the brighter of the window (W) instead of the equation (5)_W2Α percentile quantile α of the luminance distribution of_BThe tone correction processing may be performed by the same procedure as in the equation (5) using the equation (6) determined based on the above.
[0038]
【The invention's effect】
(1) Effects corresponding to the inventions of claims 1 to 7
  Extracts the area surrounding the character from the input image, and in that area the character (foreground) and background by brightnessMinutesUsing a similar method, the reference area representing the true background (background) is evaluated and selected, and the representative background color (background color) is estimated from it. The background area on the original image and other areas are estimated based on the estimated background color. The area is identified and replaced with the representative background color (background color) estimated for the former pixel. The show-through pixel is replaced with the estimated background color while locally estimating the background color for the latter pixel. Suitable for each processRemove show-throughSince image degradation such as background and background color fluctuations and show-through has been improved by applying this method, analysis methods for histograms with mixed distributions, characteristic information on input image devices, and default parameters are required. It is possible to improve the image quality of the entire image in a form that retains the background and background color more appropriately than the conventional method.
  In addition, since the representative background color (background color) and its statistics are calculated from the characteristics of the input image by statistical processing, knowledge / characteristic information about the input device and default parameters are not required at all. In order to adapt to the characteristics of the input image, such as color variation and show-through, the parameters specified by the user are also simplified.
  Also, the character (foreground) / background by brightness for the character and its adjacent pixel range (character area (C))MinutesThe class processing is performed by using a processing method that applies local adaptive binarization for each block-divided region, so that a more correct representative background color can be obtained.
  In addition, it is possible to reduce the amount of calculation in each process by using low-resolution images as processing targets, enabling high-speed processing, and when sufficient resources are not available for hardware resources It can also be applied to.
(2) Effects corresponding to the inventions of claims 8 and 9
  A program for causing a computer to function as means included in the image processing apparatus according to any one of claims 1 to 7 is provided.Computer readableBy providing the information in the form recorded on the recording medium, the effect (1) can be easily realized.
[Brief description of the drawings]
FIG. 1 shows a configuration example of an image processing system constructed using an image processing apparatus according to the present invention as a system element.
FIG. 2 shows a module configuration of a PC constituting the image processing apparatus according to the present invention.
FIG. 3 shows a block of an image processing function realized by an image processing program according to the embodiment of the present invention.
FIG. 4 is a flowchart showing an execution procedure of processing by an image processing program according to the embodiment of the present invention.
FIG. 5 shows an original image (I₀).
6 is an original image (I₀) Shows the result of extracting the character area (C) and block division.
7 shows an area set as a window (W) for the input image of FIG.
8 shows the result of background (background) extraction processing for the input image of FIG. 5, and the extracted pixels are shown in black.
FIG. 9 shows an image obtained as a result of color replacement with an estimated background color for the input image of FIG. 5;
FIG. 10 is a diagram showing conversion processing characteristics (input / output relationship according to equation (3)) for gradation correction.
11 shows an image obtained as a result of performing a show-through removal process and a gradation correction process according to the characteristics of FIG. 10 on the input image of FIG. 9;
FIG. 12 is a diagram showing conversion processing characteristics (input / output relationship according to equation (5)) for gradation correction;
13 shows an image obtained as a result of performing a show-through removal process and a gradation correction process according to the characteristics of FIG. 12 on the input image of FIG. 9;
[Explanation of symbols]
10 ... PC (personal computer),
20 ... Low-resolution image generation means, 21 ... Pre-processing (smoothing) means,
22 ... feature quantity (edge quantity, window statistic) calculation means,
23 ... Character area extraction means, 24 ... Paper color statistic estimation means,
25 ... background area extraction means, 28 ... background color conversion means,
29 ... Means for removing show-through / gradation other than background,
30 ... Image input device, 50 ... Image output device,
70 ... Server, 90 ... MFP,
100: Image processing system.

Claims (9)

Character area extracting means for extracting a character and its adjacent pixel range from the original digital image to be processed as a character area, block dividing means for dividing the image into a plurality of blocks, and character areas extracted in the divided blocks Classification means for classifying the pixel group belonging to the first class corresponding to the character color by color, the second class corresponding to the background color, and the evaluation based on the number of pixels classified into the second class and the luminance value The block selection means for selecting the block with the maximum function value is divided into a pixel group corresponding to the darker one and a pixel group corresponding to the brighter one in the selected block, and the original color of the pixel group corresponding to the brighter one is selected. a representative background color estimating means for estimating a representative background color in the image, the estimated background area having a color other than said surrogate table background color in the original image by the representative background color was extracted That a background region extracting means, the image processing apparatus to the extracted background region other than the representative background color was to and a show-through removing means for removing show-through. The image processing apparatus according to claim 1,
Based on the estimated representative background color, background area extracting means for extracting a background area regarded as having the representative background color on the original image, and the color of the background area regarded as having the extracted representative background color were estimated. An image processing apparatus comprising means for replacing with a representative background color. The image processing apparatus according to claim 1,
A background color area extracting means for extracting a background area regarded as having the representative background color on the original image based on the estimated representative background color, and a color of the background area regarded as having the extracted representative background color as white An image processing apparatus comprising: a replacing unit. The image processing apparatus according to any one of claims 1 to 3,
An image processing apparatus comprising: means for further correcting a gradation for a background region other than the representative background color. The image processing apparatus according to claim 1 ,
The image processing apparatus according to claim 1, wherein the character region extraction unit includes a feature amount calculation unit that calculates an image feature amount from an original digital image to be processed. The image processing apparatus according to any one of claims 1 to 5,
Before Symbol Classification unit, an image processing apparatus characterized by a color signal of each pixel of the luminance is calculated, and a means for classifying the first and second class by the calculated brightness thresholding. The image processing apparatus according to any one of claims 1 to 6,
The feature amount calculating means, the character region extracting means, said block division means, before Symbol classification means, the representative background color estimating means, the background color area extracting means, the background region extracting means having a color other than the representative background color At least one of the image processing apparatuses comprises means for generating a reduced image whose resolution is lower than that of the original image.   An image processing program for causing a computer to function as means included in the image processing apparatus according to claim 1. A computer-readable recording medium on which the program according to claim 8 is recorded.

Images