JP3778358B2

JP3778358B2 - Sound source separation method, apparatus and program thereof

Info

Publication number: JP3778358B2
Application number: JP2003126554A
Authority: JP
Inventors: 真理子青木; 賢一古家; 章俊片岡
Original assignee: Nippon Telegraph and Telephone Corp; NTT Inc USA
Current assignee: NTT Inc; NTT Inc USA
Priority date: 2003-05-01
Filing date: 2003-05-01
Publication date: 2006-05-24
Anticipated expiration: 2023-05-01
Also published as: JP2004336202A

Description

【０００１】
【発明の属する技術分野】
この発明は、空間的に異なる位置の複数の音源からの音響信号を、２個のマイクロホンを用いて受信し、これらマイクロホン出力信号を狭い複数の帯域に分割し、各帯域ごといずれの音源からの信号かを判別し、同一音源からの信号と判別された信号を合成して１つの音源信号を収音する音源分離方法、その装置及びプログラムに関する。
【０００２】
【従来の技術】
従来のゾーン分離収音技術には、例えば、音が持つ次のような特徴を利用したものがある。音はいくつかの周波数成分の和として表現されることが知られている。そこで、複数の音源が同時に発音している場合、これを２個以上のマイクロホンで受音し、これら各マイクロホンの出力チャネル信号を、各音源信号の周波数成分が周波数軸上で重ならない程度の帯域に分割し、チャネル信号間の音響信号パラメータ値差つまり周波数成分の到達位相差や到達レベル差を基に、各周波数成分それぞれがどの音源からのものであるかを判定し、同一音源からの成分を集めて合成することにより、各音源毎の音を個別に収音する方法が用いられていた（例えば特許文献１参照）。
【０００３】
この従来技術について図１を参照して簡単に説明する。例えば２０ｃｍ程度の間隔で設けられたマイクロホン１と２からの出力チャネル信号はチャネル間パラメータ値差検出部３において、マイクロホン１，２の位置に起因して変化するマイクロホン１，２に到達する音響信号のパラメータ値の差、つまりマイクロホン１，２に到達した音響信号のレベル差や到達時間差がチャネル間パラメータ値差として検出される。またマイクロホン１，２の各出力チャネル信号は帯域分割部４によりそれぞれ例えば離散的フーリエ変換され、更に複数の周波数帯域の信号に分割される。この各帯域の幅は各音源信号の周波数特性の差から、ひとつの音源信号の成分のみが主として存在する程度、例えば２０Ｈｚとする。帯域分割された両チャネル信号は、帯域別チャネル間パラメータ値差検出部５において、互いに同一帯域信号ごとに音響信号のパラメータ値の差が、帯域別チャネル間パラメータ値差としてそれぞれ検出される。
【０００４】
チャネル間パラメータ値差と、帯域別チャネル間パラメータ値差が音源信号判定部６Ａに入力され、音源信号判定部６Ａにおいて各帯域ごとにその帯域信号がいずれの音源からの信号であるかの判定がしきい値を用いて行われる（前記特許文献１の図９中のステップＳ３４，Ｓ３５）。その判定結果に基づき、各チャネル信号の帯域分割された信号中の同一音源信号と判定されたものが音源信号選択部６Ｂで選択され、これら同一音源信号と判定されたものが音源信号別に音源合成部７Ａ，７Ｂで合成されて、それぞれ、音源８，９からの信号として出力される。
【０００５】
音源信号判定部６Ａで判定に用いるしきい値としては、特許文献１の段落番号［００３３］及び［００３４］に次のように示されている。マイクロホン１と２を結ぶ線の２等分線に対して音源８と９が対称に位置している場合は、０をしきい値とし、音源８と９がこのような関係にない場合は、音源８の信号がマイクロホン１，２に到達する帯域別チャネル間レベル差をΔＬ_A、到達する帯域別チャネル間時間差をΔτ_A、音源９の信号がマイクロホン１，２に到達する帯域別チャネルレベル差をΔＬ_B、到達する帯域別チャネル間時間差をΔτ_Bとすると、帯域別チャネル間レベル差のしきい値ΔＬthは、
ΔＬth＝（ΔＬ_A＋ΔＬ_B）／２
とし、帯域別チャネル間時間差のしきい値Δτthは
Δτth＝（Δτ_A＋Δτ_B）／２
とする。しかし、これは一方の音源のみ発音している理想的な場合であり、実際には、音源のマイクロホンに対する方向、距離はわかっていないため、しきい値ΔＬth，Δτthを可変として、分離がよく行われるようにΔＬth，Δτthを調整することが記載されている。
【０００６】
【特許文献１】
特許第３３５５５９８号公報
【０００７】
【発明が解決しようとする課題】
従来においてはしきい値ΔＬth，Δτthを自動で計測する技術がなく、従来においては試行錯誤の上、しきい値を決定する必要があった。
また手間を掛てしきい値を決定しても、そのしきい値を長い時間用いると、音源の変動により、性能が劣化する問題があった。
【０００８】
【課題を解決するための手段】
この発明の方法によれば互いに離して配置された２個のマイクロホンの各出力チャネル信号を、複数の周波数帯域に分割し、これら分割された各出力チャネル信号の各同一帯域毎に、上記複数のマイクロホンの位置に起因して変化する、マイクロホンに到達する音響信号のパラメータの値の差を、帯域別チャネル間パラメータ値差として検出し、これら各帯域の帯域別チャネル間パラメータ値差に基づき、その帯域の上記帯域分割された各出力チャネル信号のいずれがいずれの音源から入力された信号であるかをしきい値を用いて判定し、この判定に基づき、上記帯域分割された各出力チャネル信号から、同一音源から入力された信号を少なくとも一つ選択し、この同一音源からの信号として選択された複数の帯域信号を音源信号として合成する音源分離方法において、
目的音源信号と雑音信号が混ざった混合信号のチャネル間の音響信号パラメータ値差を検出し、また、雑音信号のチャネル間音響信号パラメータ値差を検出し、これら混合信号のチャネル間パラメータ値差と、雑音信号のチャネル間パラメータ値差の値に基づき、上記しきい値を求めることを特徴とする。
【０００９】
【発明の実施の形態】
この発明の実施形態の機能構成例を図１に示す。図１から理解されるように、従来の音源分離装置に対し新たに機能が付け加えられる。
この発明では目的音源信号と雑音信号が混ざった混合信号のチャネル間の音響信号パラメータ値差と、雑音信号のチャネル間の音響信号パラメータ値差とをそれぞれ検出する。このためマイクロホン１，２の各出力チャネル信号が、目的音源信号と雑音信号が混ざった混合信号であるか、雑音信号であるかを有音判定部１１で判定する。この例ではマイクロホン１と２の各出力チャネル信号が有音判定部１１に入力され、有音判定部１１は両チャネル信号のパワーが所定レベルを所定時間継続して超えたら両チャネル信号が混合信号であり、これより所定レベル以下になれば、混合信号の状態にないと判定し、また両チャネル信号のパワーが前記所定レベルより小さい所定レベル以下で継続していれば両チャネル信号は雑音信号であると判定する。あるいは音源が例えば人間がスイッチを操作して発話するようなものである場合は、そのスイッチ操作に基づく信号を有音判定部１１に入力して、音源が発音状態、つまりその時の両チャネル信号は混合信号であり、スイッチ操作が停止されている状態の両チャネル信号は雑音信号と判定するようにしてもよい。
【００１０】
またこの例では帯域分割されたチャネル信号について帯域別チャネル間パラメータ値差を検出した場合である。つまり帯域別チャネル間パラメータ値差検出部５で検出された各帯域別チャネル間パラメータ値差は各帯域ごとに切替えることができる切替部１２へ供給され、切替部１２は有音判定部１１よりの混合信号と判定した出力により制御され、入力された各帯域別チャネル間パラメータ値差が信号用パラメータ値差保持部１３Ｓに更新格納保持される。有音判定部１１よりの出力が混合信号と判定していなければ、入力された各帯域別チャネル間パラメータ値差が雑音用パラメータ値差保持部１３Ｎに更新格納保持される。この場合、有音判定部１１が雑音信号と判定した出力により、帯域別チャネル間パラメータ値差を雑音用パラメータ値差保持部１３Ｎに更新格納保持させ、混合信号とも雑音信号とも判定していない場合は、雑音用パラメータ値差保持部１３Ｎに対する更新格納保持は行なわないようにすることが好ましい。
【００１１】
信号用パラメータ値差保持部１３Ｓに保持されている混合信号の帯域別チャネル間パラメータ値差と、雑音用パラメータ値差保持部１３Ｎに保持されている雑音信号の帯域別パラメータ値差とに基づいてしきい値決定部１４でしきい値が決定される。
例えば音源８が目的音源とし、目的音源信号をｓ（ｔ）、雑音信号をｎ（ｔ）とする。ｔは離散的時刻である。マイクロホン１の出力混合信号を（ｓ＋ｎ）_1M（ｔ）、マイクロホン２の出力混合信号を（ｓ＋ｎ）_2M（ｔ）、マイクロホン１の出力雑音信号をｎ_1N（ｔ）、マイクロホン２の出力雑音信号をｎ_2N（ｔ）とし、チャネルパラメータとしてレベルを用いる場合を例とし、各混合信号（ｓ＋ｎ）_1M（ｔ）、（ｓ＋ｎ）_2M（ｔ）の帯域分割された各帯域ｆｉ(ｉ＝１，２，…，Ｉ：Ｉは帯域分割数）の帯域信号レベルをＬ_1M［ｆｉ］，Ｌ_2M［ｆｉ］とし、各雑音信号ｎ_1N（ｔ），ｎ_2N（ｔ）の帯域分割された各帯域ｆｉの帯域信号レベルをＬ_1N［ｆｉ］，Ｌ_2N［ｆｉ］とする。この時、混合信号の帯域別チャネル間レベル差ΔＬ_M[ｆｉ]は次式で与えられる。
ΔＬ_M[ｆｉ]＝Ｌ_2M［ｆｉ］−Ｌ_1M［ｆｉ］
レベルの単位はｄＢ表示である。
【００１２】
雑音信号の帯域別チャネル間レベル差ΔＬ_N[ｆｉ]は次式で与えられる。
ΔＬ_N[ｆｉ]＝Ｌ_2N［ｆｉ］−Ｌ_1N［ｆｉ］
これら帯域別チャネル間レベル差ΔＬ_M[ｆｉ]とΔＬ_N[ｆｉ]が信号用パラメータ値差保持部１３Ｓ、雑音用パラメータ値差保持部１３Ｎにそれぞれ保持される。しきい値決定部１４ではこれら帯域別チャネル間レベル差ΔＬ_M[ｆｉ]とΔＬ_N[ｆｉ]を用いて帯域別しきい値Ｌth［ｆｉ］が例えば次式に示すように平均値として決定される。
Ｌth［ｆｉ］＝（ΔＬ_M[ｆｉ]＋ΔＬ_N[ｆｉ]）／２
最初から分離性能を良くする点からはしきい値Ｌth［ｆｉ］としては平均値がよいが、ΔＬ_M[ｆｉ]とΔＬ_N[ｆｉ]との間の値、予め決めた比率でΔＬ_M[ｆｉ]よりもΔＬ_N[ｆｉ]に近い値をしきい値Ｌth［ｆｉ］としてもよい。
【００１３】
このように決定された帯域別しきい値Ｌth［ｆｉ］はしきい値部１５に設定され、音源信号判定部６Ａにおいて、帯域ｆｉの帯域信号がいずれの音源から入力された信号であるかの判定のためのしきい値として用いられる。このしきい値としては帯域別しきい値Ｌth［ｆｉ］の代表値Ｌthを代表決定部１６で決定して、各帯域ｆｉに共通のしきい値Ｌthを用いていずれの音源から入力された信号であるかの判定を行ってもよい。代表値Ｌthの決定は例えばＬth［ｆｉ］の平均値あるいは最大のＬth［ｆｉ］などによる。
【００１４】
この音源分離装置を最初に用いる場合は、図２に示すように混合信号（ｓ＋ｎ）_1M（ｔ），（ｓ＋ｎ）_2M（ｔ），雑音信号ｎ_1N（ｔ），ｎ_2N（ｔ）を例えば３〜５秒の定めた時間、バッファ（図１中には特に示していない）に格納し（Ｓ１）、これら信号の安定したものが得られた状態で混合信号の帯域別チャネル間パラメータ値差を検出し（Ｓ２）、また雑音信号の帯域別チャネル間パラメータ値差を検出する（Ｓ３）。これらの検出は帯域分割部４及び帯域別チャネル間パラメータ値差検出部５により行う。前記バッファを省略してパラメータ値差保持部１３をバッファとして作用させても、安定した信号の帯域別チャネル間パラメータ値差がパラメータ値差保持部１３に更新保持されることになる。次にこれら混合信号の帯域別チャネル間パラメータ値差と雑音信号の帯域別チャネル間パラメータ値差に基づき、帯域別しきい値を決定しこれをしきい値設定部１５に設定する（Ｓ４）。この初期設定の後、音源分離処理を開始する。なおしきい値としては帯域別しきい値からその代表値を決定し（Ｓ５）、これを音源信号判定に用いてもよい。
【００１５】
以上のように決定したしきい値Ｌth［ｆｉ］又はＬthをそのまま採用した場合に以下に記す不具合がおきるおそれがある。すなわち、しきい値Ｌth［ｆｉ］を最も精度よく求めるためには、混合信号の帯域別チャネル間レベル差ΔＬ_M[ｆｉ]は、なるべく雑音ｎ（ｔ）が入らず、目的信号ｓ（ｔ）だけが発音している信号を使うのが望ましい。しかし、実際の環境においては、雑音が全て発生しておらず、目的音源だけが発音している状態を得ることが難しい。よって、ΔＬ_M[ｆｉ]つまり、従来技術の項で述べた帯域別チャネル間レベル差ΔＬ_Aは目的音源信号と雑音信号が混在した信号から近似的に算出することになる。よって、必ずしも最適なしきい値Ｌth［ｆｉ］になっているとは限らない。
【００１６】
そこで、前述のようにして決定したしきい値Ｌth［ｆｉ］を初期値とし、初期値を用いた場合の分離結果を評価し、つまり分離前の信号と分離後の信号とを用いて例えば分離性能を表わす評価値を分離評価部２１で計算し、その評価が所定の範囲に入るようにしきい値Ｌth［ｆｉ］を修正部２２により修正する。
【００１７】
まず、初期しきい値Ｌth［ｆｉ］を用いて、あらかじめバッファ２４に記憶してある目的音源信号と雑音信号の混合信号を分離処理する。即ち図３Ａに示すように、目的音源８が発音している状態において雑音源９も発音しているから、先に述べたようにマイクロホン１，２からの混合信号（ｓ＋ｎ）_1M(ｔ），（ｓ＋ｎ）_2M(ｔ）がこの発明による音源分離装置１０により分離処理されて、目的音源信号（ｓ＋ｎ）′_1M(ｔ）が分離出力される。あるいは図３Ｂに示すように目的音源８が発音していない状態においてはマイクロホン１，２からの雑音信号ｎ_1N(ｔ)，ｎ_2N(ｔ)が音源分離装置１０により分離処理されて雑音信号ｎ′_1N(ｔ)が分離出力される。
【００１８】
これら分離前の信号（ｓ＋ｎ）_1M(ｔ），ｎ_1N(ｔ)と分離処理後の信号（ｓ＋ｎ）′_1M(ｔ），ｎ′_1N(ｔ)とのいくつかを用いて分離性能を表わす評価値を分離評価部２１で計算する（Ｓ１、図４）。分離性能を表わす評価値としては例えば次に示す各種の信号対雑音比（ＳＮＲ）の何れかを用いることができる。
【数１】

演算子・は相関関数を表す。０≦ＳＮＲ₄≦１である。
【００１９】
評価値ＳＮＲ₁〜ＳＮＲ₄の何れかを計算し、その評価値が所定の範囲ｘ１＜ＳＮＲ＜ｘ２ｄＢに入るようにしきい値Ｌth［ｆｉ］を修正する。例えば修正判定部２３において評価値ＳＮＲが上限値ｘ２を超えているかを調べ（Ｓ２）、超えていればしきい値設定部１５内のしきい値Ｌth［ｆｉ］を所定値Δthだけ、しきい値修正部２２により減少させる（Ｓ３）。
修正判定部２３ではステップＳ２で評価値ＳＮＲが上限値ｘ２を超えていなければ下限値ｘ１より小さいかを調べ（Ｓ４）、小さければしきい値設定部１５内のしきい値Ｌth［ｆｉ］を所定値Δthだけ、しきい値修正部２２により増加させる（Ｓ５）。ステップＳ３及びＳ５の後、ステップＳ１に戻り、修正したしきい値Ｌth［ｆｉ］により再び分離処理を行って評価値ＳＮＲを求める。以下同様にして、ステップＳ４で評価値ＳＮＲが下限値より小さくなければ、修正処理を終了する。
【００２０】
例えばｘ１＝１５ｄＢ、ｘ２＝２０ｄＢ程度とする。しかし雑音が大きい場合は、大きなＳＮＲが得られないため、同一音源からの帯域信号と判定される帯域が少なくなり、分離信号の歪みが大きくなる。そのような場合はｘ１＝１０ｄＢ、ｘ２＝１５ｄＢ程度がよい。修正量Δthは例えば０．１〜０．２ｄＢ程度がよい。
評価値ＳＮＲ₂〜ＳＮＲ₄は目的音源信号成分と、雑音信号成分との両者を用いている点で実際の環境にあっている点で分離性能との対応がよいが、ＳＮＲ₂とＳＮＲ₃はＳＮＲ₄より演算量が少ない点がよい。
【００２１】
評価値としては信号対歪比ＳＤＲを用いてもよい。ＳＤＲは例えば次の何れかにより求める。
【数２】

演算子・は相関関数を表す。
【００２２】
ＳＤＲ₁とＳＤＲ₂は分離性能との対応では同一程度であるが、ＳＤＲ₁はＳＤＲ₂より計算量が少ない点がよい。この評価値ＳＤＲを用いた場合も、これが下限値ｙ１と上限値ｙ２との間に入るように、例えば図４に括弧書で示す手順でしきい値Ｌth［ｆｉ］を修正する。ＳＤＲがｙ１より小であれば、同一音源からの帯域信号と判定される帯域が少なくなり歪が大きくなるのでしきい値Ｌth［ｆｉ］を小さくし、ＳＤＲがｙ２より大であれば雑音の混入が多くなるのでしきい値Ｌth［ｆｉ］を大とする。例えば下限値ｙ１＝８ｄＢ、上限値ｙ２＝１０ｄＢ程度とし、雑音が多い場合はｙ１＝５，ｙ２＝８程度とした方が、雑音成分も除去され易い。しきい値修正量Δthは０．１〜０．２ｄＢ程度がよい。
【００２３】
目的音源の位置、方向が変化したり、雑音環境が変化したりする点で設定したしきい値は適当な周期で更新するようにするとよい。また更新するようにすれば初期値として例えば０を設定させて、分離処理と、しきい値更新処理とを並列的に行わせることもできる。
例えば図５に示すように、まずしきい値の初期値としきい値設定部１５に設定する（Ｓ１）。この初期値は前述したようにして求めてもよいし、適当な値を設定してもよい。
その後一定時間経過するのを待つ（Ｓ２）。この一定時間は例えば１０〜３０秒とするが、使用環境や目的に応じ、速く適したしきい値に追従させる必要があれば、それに応じて短時間にする。帯域分割部４において例えば離散的フーリエ変換を行うが、その変換フレーム単位でしきい値を更新してもよい。
【００２４】
一定時間が経過すると、しきい値の更新処理を行う（Ｓ３）。この更新処理は図２に示した処理を行うことになる。この際、ステップＳ２で待っている間においても音源分離処理と併行して、有音判定部１１の出力によりパラメータ値差保持部１３に対する更新格納保持を行うようにすれば、更新処理が始まると、その時のパラメータ値差保持部１３に保持されている帯域別チャネル間パラメータ値差を用いてしきい値を決定し、このしきい値でしきい値設定部１５のしきい値を更新することにより、短時間で更新処理を行うことができる。この更新処理の後、ステップＳ２に戻って一定時間の経過を待つ。このようにして一定時間ごとにしきい値が更新され、品質のよい分離信号を得ることができる。
【００２５】
この更新処理の際に、前回の評価値よりよくなった場合に更新を行い、よくならなければ更新を行わないようにすることが好ましい。例えば図５に示すように、ステップＳ２の後、図２に示した処理によりしきい値を計算し（Ｓ４）、このしきい値を用いて、分離処理を行って、分離評価部２１で評価値（ＳＮＲ又はＳＤＲ）を計算する（Ｓ５）。この評価値と前回の評価値とを更新判定部２５で比較し（Ｓ６）、前回より良く（大きく）なっていれば更新を実行し、つまりステップＳ４で計算したしきい値、従ってステップＳ４で評価値計算に用い、その時、しきい値設定部１５に設定されているしきい値をそのまま分離処理に用いる。
【００２６】
しかし前回の評価値より悪く（小さく）なった場合は、それまでに用いていたしきい値、つまりステップＳ２で待っている時間の間、分離処理に用いていたしきい値を用いる。従って、一定時間経過しても、しきい値の更新は行わないでステップＳ２に戻る。この更新を行わない処理ができるように、ステップＳ５における評価値計算のための分離処理中は、それまでのしきい値、メモリに避難させておくか、評価値計算のための分離処理の際に用いるしきい値はしきい値決定部１４内のしきい値を用い、ステップＳ７の更新実行において、しきい値決定部１４内のしきい値をしきい値設定部１５に設定更新するようにする。ステップＳ４のしきい値計算も、一定時間の経過を待っている間に、有音判定部１１の出力により、パラメータ値差保持部１３に対する更新格納を行わせておくことにより、短時間でしきい値を計算することができる。
【００２７】
図４に示した計算したしきい値の修正処理は、図５中に破線で示すように、ステップＳ１の初期設定が図２に示した処理により行う場合は、そのしきい値に対し行ってもよく、またステップＳ３の更新処理で計算したしきい値に対して行ってもよく、ステップＳ４で計算したしきい値に対して行ってもよい。
【００２８】
上述においては混合信号の帯域別チャネル間レベル差ΔＬ_M[ｆｉ］と雑音信号の帯域別チャネル間レベル差ΔＬ_N[ｆｉ］を用いてしきい値Ｌth［ｆｉ］を決定したが、混合信号のチャネル間レベル差ΔＬ_Mと雑音信号のチャネル間レベル差ΔＬ_Nを用いてしきい値Ｌthを決定し、これを各帯域信号について共通に用いてどの音源からの信号であるかの判定を行うようにしてもよい。このチャネル間レベル差ΔＬ_MとΔＬ_Nは例えばチャネル間差検出部３からの出力を、有音判定部１１の出力により選別して取り出せばよい。更にこの発明は混合信号の帯域別チャネル間時間差Δτ_M[ｆｉ］と雑音信号の帯域別チャネル間時間差Δτ_N[ｆｉ］を用いて、あるいは混合信号のチャネル間時間差Δτ_Mと雑音信号のチャネル間時間差Δτ_Nを用いて、しきい値τth［ｆｉ］又はτthを決定してもよい。要は混合信号と雑音信号について帯域別を含む広義のチャネル間パラメータ値差を用いてしきい値を決定すればよい。
【００２９】
この発明による音源分離装置はコンピュータに機能させてもよい。この場合は上述したこの発明の音源分離方法の各過程をコンピュータに実行させるためのプログラムをＣＤ−ＲＯＭ、磁気ディスク、その他の記録媒体あるいは通信回線を介してコンピュータ内にダウンロードして、このプログラムを実行させればよい。
【００３０】
【発明の効果】
以上述べたようにこの発明によれば、目的音源信号＋雑音信号のチャネル間パラメータ値差と、雑音信号のチャネル間パラメータ値差を元にしきい値を決定することができ、従来、人手で試行錯誤して決定していたしきい値を、あらかじめ測定可能な物理量から決定でき、多くの手間と時間が省け、しかも一定の性能で音源分離装置を動作させることが可能となる。
また、必要に応じてしきい値を逐次更新する構成とすることにより、信号の時間変動にしきい値が追随し、この際、更新前のしきい値と新たに求めたしきい値の評価値を比較し、性能の高いほうのしきい値を常に選べば、この評価値の比較を行うことなく毎回しきい値を更新する場合に比べてより安定した性能で分離処理が可能となる。
【図面の簡単な説明】
【図１】この発明の装置の機能の構成例を示す図。
【図２】この発明によるしきい値を求める手順の例を示す流れ図。
【図３】評価値としてのＳＮＲ，ＳＤＲの算出に必要な信号を示す図。
【図４】しきい値修正処理の手順の例を示す流れ図。
【図５】しきい値更新の処理手順の例を示す流れ図。[0001]
BACKGROUND OF THE INVENTION
The present invention receives acoustic signals from a plurality of sound sources at spatially different positions using two microphones, divides the microphone output signal into a plurality of narrow bands, and from each sound source for each band. The present invention relates to a sound source separation method, an apparatus, and a program for collecting a single sound source signal by determining whether the signal is a signal and synthesizing the determined signal with a signal from the same sound source.
[0002]
[Prior art]
For example, conventional zone separation and sound collection techniques utilize the following characteristics of sound. It is known that sound is expressed as the sum of several frequency components. Therefore, when a plurality of sound sources are sounding simultaneously, they are received by two or more microphones, and the output channel signals of these microphones are banded so that the frequency components of the sound source signals do not overlap on the frequency axis. To determine which sound source each frequency component is based on the acoustic signal parameter value difference between channel signals, that is, the arrival phase difference or arrival level difference of the frequency components, and components from the same sound source. A method of collecting sounds for each sound source individually by collecting and synthesizing them is used (see, for example, Patent Document 1).
[0003]
This prior art will be briefly described with reference to FIG. For example, the output channel signals from the

microphones

1 and 2 provided at intervals of about 20 cm are acoustic signals that reach the

microphones

1 and 2 that change due to the positions of the

microphones

1 and 2 in the inter-channel parameter value difference detection unit 3. Parameter value difference, that is, the level difference or arrival time difference between the acoustic signals reaching the

microphones

1 and 2 is detected as the inter-channel parameter value difference. The output channel signals of the

microphones

1 and 2 are each subjected to, for example, discrete Fourier transform by the band dividing unit 4 and further divided into signals of a plurality of frequency bands. The width of each band is set to, for example, 20 Hz to the extent that only one sound source signal component exists mainly due to the difference in frequency characteristics of each sound source signal. The band-divided channel parameter value difference detection unit 5 detects the difference between the parameter values of the acoustic signal for each of the same band signal as the band-specific channel parameter value difference.
[0004]
The inter-channel parameter value difference and the inter-band parameter value difference between the bands are input to the sound source signal determination unit 6A, and the sound source signal determination unit 6A determines from which sound source the band signal is for each band. This is performed using a threshold value (steps S34 and S35 in FIG. 9 of Patent Document 1). Based on the determination result, those determined as the same sound source signal in the band-divided signal of each channel signal are selected by the sound source signal selection unit 6B, and those determined as the same sound source signal are synthesized for each sound source signal. The signals are synthesized by the units 7A and 7B and output as signals from the

sound sources

8 and 9, respectively.
[0005]
The threshold values used for determination by the sound source signal determination unit 6A are shown in paragraph numbers [0033] and [0034] of Patent Document 1 as follows. When the

sound sources

8 and 9 are located symmetrically with respect to the bisector of the line connecting the

microphones

1 and 2, 0 is set as a threshold value, and the

sound sources

8 and 9 are not in such a relationship, ΔL _{A is} the level difference between the channels for the sound source 8 that reaches the

microphones

1 and 2, Δτ _{A is} the time difference between the channels that reach the sound source 9, and the channel level difference for each band that the signal of the sound source 9 reaches the

microphones

1 and 2. Is ΔL _B , and the time difference between channels reaching each band is Δτ _B.
ΔLth = (ΔL _A + ΔL _B ) / 2
And the threshold value Δτth of the time difference between channels for each band is Δτth = (Δτ _A + Δτ _B ) / 2.
And However, this is an ideal case where only one sound source is sounding. Actually, since the direction and distance of the sound source with respect to the microphone are not known, the thresholds ΔLth and Δτth can be made variable to perform separation well. It is described that ΔLth and Δτth are adjusted as described.
[0006]
[Patent Document 1]
Japanese Patent No. 3355598 [0007]
[Problems to be solved by the invention]
Conventionally, there is no technique for automatically measuring the threshold values ΔLth and Δτth, and conventionally, it has been necessary to determine the threshold values through trial and error.
Even if the threshold value is determined by taking time and effort, if the threshold value is used for a long time, there is a problem that the performance deteriorates due to the fluctuation of the sound source.
[0008]
[Means for Solving the Problems]
According to the method of the present invention, each output channel signal of two microphones arranged apart from each other is divided into a plurality of frequency bands, and each of the divided output channel signals is divided into the plurality of the above-mentioned plurality of frequency channels. The difference in the parameter value of the acoustic signal reaching the microphone, which changes due to the position of the microphone, is detected as the parameter value difference between channels for each band, and based on the parameter value difference between channels for each band, It is determined by using a threshold value which one of the output channel signals divided into the bands is a signal input from which sound source, and based on the determination, from each output channel signal divided in the band Select at least one signal input from the same sound source, and synthesize multiple band signals selected as signals from this same sound source as sound source signals In that sound source separation method,
The difference between the acoustic signal parameter values between the channels of the mixed signal in which the target sound source signal and the noise signal are mixed, and the difference between the acoustic signal parameter values between the channels of the noise signal is detected. The threshold value is obtained based on the value of the parameter value difference between channels of the noise signal.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
An example of the functional configuration of the embodiment of the present invention is shown in FIG. As will be understood from FIG. 1, a new function is added to the conventional sound source separation apparatus.
In the present invention, an acoustic signal parameter value difference between channels of a mixed signal in which a target sound source signal and a noise signal are mixed and an acoustic signal parameter value difference between channels of a noise signal are detected. Therefore, the sound determination unit 11 determines whether each output channel signal of the

microphones

1 and 2 is a mixed signal in which the target sound source signal and the noise signal are mixed or a noise signal. In this example, the output channel signals of the

microphones

1 and 2 are input to the sound determination unit 11, and the sound determination unit 11 determines that both channel signals are mixed signals when the power of both channel signals exceeds a predetermined level for a predetermined time. If the signal level is lower than the predetermined level, it is determined that the mixed signal is not in the state. If the power of both channel signals continues below the predetermined level lower than the predetermined level, both channel signals are noise signals. Judge that there is. Alternatively, when the sound source is, for example, a person who speaks by operating a switch, a signal based on the switch operation is input to the sound determination unit 11, and the sound source is in a sounding state, that is, both channel signals at that time are Both channel signals that are mixed signals and in which the switch operation is stopped may be determined as noise signals.
[0010]
In this example, the channel-specific parameter value difference for each band is detected for the band-divided channel signal. That is, the inter-band parameter value difference between channels detected by the inter-band parameter value difference detection unit 5 is supplied to the switching unit 12 that can be switched for each band. Control is performed based on the output determined to be a mixed signal, and the input inter-band parameter value difference between channels is updated and stored in the signal parameter value difference holding unit 13S. If the output from the sound determination unit 11 is not determined to be a mixed signal, the input inter-band parameter value difference between channels is updated and stored in the noise parameter value difference storage unit 13N. In this case, when the sound determination unit 11 determines that the signal is a noise signal, the parameter value difference between channels for each band is updated and stored in the noise parameter value difference holding unit 13N, and neither the mixed signal nor the noise signal is determined. It is preferable not to perform update storage holding for the noise parameter value difference holding unit 13N.
[0011]
Based on the inter-channel parameter value difference for each band of the mixed signal held in the signal parameter value difference holding unit 13S and the band-by-band parameter value difference of the noise signal held in the noise parameter value difference holding unit 13N. A threshold value is determined by the threshold value determination unit 14.
For example, the sound source 8 is the target sound source, the target sound source signal is s (t), and the noise signal is n (t). t is a discrete time. The output mixed signal of the microphone 1 is (s + n) _1M (t), the output mixed signal of the microphone 2 is (s + n) _2M (t), the output noise signal of the microphone 1 is n _1N (t), and the output noise signal of the microphone 2 is n _2N (t), and a level is used as a channel parameter. For example, each band fi (i = 1, 2) obtained by dividing the band of each of the mixed signals (s + n) _1M (t) and (s + n) _2M (t). ,..., I: where I is the number of band divisions), the band signal levels are L _1M [fi] and L _2M [fi], and each noise signal n _1N (t), n _2N (t) is divided into each band. Let the band signal levels of fi be L _1N [fi] and L _2N [fi]. At this time, the inter-channel level difference ΔL _M [fi] of the mixed signal is given by the following equation.
ΔL _M [fi] = L _2M [fi] −L _1M [fi]
The unit of the level is dB display.
[0012]
The inter-channel level difference ΔL _N [fi] of the noise signal is given by the following equation.
ΔL _N [fi] = L _2N [fi] −L _1N [fi]
These inter-channel level differences ΔL _M [fi] and ΔL _N [fi] are held in the signal parameter value difference holding unit 13S and the noise parameter value difference holding unit 13N, respectively. The threshold value determination unit 14 determines the band-specific threshold value Lth [fi] as an average value, for example, as shown in the following equation, using these band-specific channel level differences ΔL _M [fi] and ΔL _N [fi]. The
Lth [fi] = (ΔL _M [fi] + ΔL _N [fi]) / 2
Although it is the average value as a threshold value Lth [fi] From the viewpoint of improving the separation performance from the initial value between the [Delta] L _N [fi] and [Delta] L _M [fi], in predetermined ratio [Delta] L _M [ A value closer to ΔL _N [fi] than fi] may be used as the threshold value Lth [fi].
[0013]
The band-specific threshold value Lth [fi] determined in this way is set in the threshold value unit 15, and the sound source signal determination unit 6A determines from which sound source the band signal of the band fi is input. Used as a threshold value for determination. As the threshold value, a representative value Lth of the threshold value Lth [fi] for each band is determined by the representative determining unit 16, and a signal input from any sound source using the threshold value Lth common to each band fi. You may determine whether it is. The representative value Lth is determined by, for example, the average value of Lth [fi] or the maximum Lth [fi].
[0014]
When this sound source separation device is used for the first time, as shown in FIG. 2, mixed signals (s + n) _1M (t), (s + n) _2M (t), noise signals n _1N (t), n _2N (t) are used, for example. Stored in a buffer (not shown in FIG. 1) for a predetermined time of 3 to 5 seconds (S1), and obtained a stable signal of these signals, the difference in parameter values between channels of the mixed signal by band Is detected (S2), and a parameter value difference between channels of the noise signal for each band is detected (S3). These detections are performed by the band dividing unit 4 and the inter-channel parameter value difference detecting unit 5 for each band. Even if the buffer is omitted and the parameter value difference holding unit 13 acts as a buffer, the parameter value difference between channels for stable signals by band is updated and held in the parameter value difference holding unit 13. Next, based on the inter-band parameter value difference between the mixed signals and the inter-channel parameter value difference between the noise signals, a per-band threshold value is determined and set in the threshold setting unit 15 (S4). After this initial setting, the sound source separation process is started. As the threshold value, the representative value may be determined from the threshold value for each band (S5) and used for sound source signal determination.
[0015]
When the threshold value Lth [fi] or Lth determined as described above is adopted as it is, the following problems may occur. That is, in order to obtain the threshold value Lth [fi] with the highest accuracy, the inter-band level difference ΔL _M [fi] of the mixed signal does not contain noise n (t) as much as possible, and the target signal s (t) It is desirable to use signals that only sound. However, in an actual environment, it is difficult to obtain a state where all noise is not generated and only the target sound source is sounding. Accordingly, ΔL _M [fi], that is, the inter-band level difference ΔL _A described in the section of the prior art is approximately calculated from a signal in which the target sound source signal and the noise signal are mixed. Therefore, the optimum threshold value Lth [fi] is not always obtained.
[0016]
Therefore, the threshold value Lth [fi] determined as described above is used as an initial value, and the separation result when the initial value is used is evaluated, that is, for example, separation is performed using the signal before separation and the signal after separation. An evaluation value representing performance is calculated by the separation evaluation unit 21 and the threshold value Lth [fi] is corrected by the correction unit 22 so that the evaluation falls within a predetermined range.
[0017]
First, using the initial threshold value Lth [fi], the mixed signal of the target sound source signal and the noise signal stored in advance in the buffer 24 is separated. That is, as shown in FIG. 3A, since the noise source 9 is also sounding when the target sound source 8 is sounding, as described above, the mixed signal (s + n) _1M (t), (S + n) _2M (t) is separated by the sound source separation apparatus 10 according to the present invention, and the target sound source signal (s + n) ′ _1M (t) is separated and output. Alternatively, as shown in FIG. 3B, in a state where the target sound source 8 is not producing sound, the noise signals n _1N (t) and n _2N (t) from the

microphones

1 and 2 are separated by the sound source separation device 10 and the noise signal n ' _1N (t) is output separately.
[0018]
The separation performance is expressed by using some of these signals (s + n) _1M (t), n _1N (t) before separation and signals (s + n) ′ _1M (t), n ′ _1N (t) after separation processing. An evaluation value is calculated by the separation evaluation unit 21 (S1, FIG. 4). As the evaluation value representing the separation performance, for example, any of the following various signal-to-noise ratios (SNR) can be used.
[Expression 1]

The operator · represents a correlation function. 0 ≦ SNR ₄ ≦ 1.
[0019]
Any _{one of the} evaluation values SNR _{1 to} SNR ₄ is calculated, and the threshold value Lth [fi] is corrected so that the evaluation value falls within a predetermined range x1 <SNR <x2 dB. For example, the correction determination unit 23 checks whether the evaluation value SNR exceeds the upper limit value x2 (S2). If it exceeds, the threshold value Lth [fi] in the threshold value setting unit 15 is set to a predetermined value Δth. The value is reduced by the value correction unit 22 (S3).
If the evaluation value SNR does not exceed the upper limit value x2 in step S2, the correction determination unit 23 checks whether it is smaller than the lower limit value x1 (S4), and if smaller, sets the threshold value Lth [fi] in the threshold value setting unit 15. The threshold value correction unit 22 increases the threshold value by the predetermined value Δth (S5). After steps S3 and S5, the process returns to step S1, and separation processing is performed again with the corrected threshold value Lth [fi] to obtain the evaluation value SNR. Similarly, if the evaluation value SNR is not smaller than the lower limit value in step S4, the correction process is terminated.
[0020]
For example, x1 = 15 dB and x2 = 20 dB. However, when the noise is large, a large SNR cannot be obtained, so that the band determined to be a band signal from the same sound source is reduced and the distortion of the separated signal is increased. In such a case, x1 = 10 dB and x2 = 15 dB are preferable. The correction amount Δth is preferably about 0.1 to 0.2 dB, for example.
The evaluation values SNR _{2 to} SNR ₄ have good correspondence with the separation performance in terms of using the target sound source signal component and the noise signal component in the actual environment, but SNR ₂ and SNR ₃ are The point that the amount of calculation is smaller than SNR ₄ is good.
[0021]
A signal-to-distortion ratio SDR may be used as the evaluation value. For example, the SDR is obtained by any of the following.
[Expression 2]

The operator · represents a correlation function.
[0022]
SDR ₁ and SDR ₂ have the same degree of correspondence with the separation performance, but SDR ₁ is advantageous in that it has a smaller amount of calculation than SDR ₂ . Even when this evaluation value SDR is used, the threshold value Lth [fi] is corrected by a procedure shown in parentheses in FIG. 4 so that it falls between the lower limit value y1 and the upper limit value y2. If the SDR is smaller than y1, the band determined to be a band signal from the same sound source decreases and distortion increases, so the threshold value Lth [fi] is reduced, and if the SDR is larger than y2, noise is mixed. Therefore, the threshold value Lth [fi] is increased. For example, when the lower limit value y1 = 8 dB and the upper limit value y2 = 10 dB, and when there is a lot of noise, y1 = 5 and y2 = 8 are more likely to remove noise components. The threshold correction amount Δth is preferably about 0.1 to 0.2 dB.
[0023]
The threshold value set in terms of the change in the position and direction of the target sound source or the change in the noise environment may be updated at an appropriate period. Further, if updating is performed, for example, 0 can be set as an initial value, and the separation process and the threshold value update process can be performed in parallel.
For example, as shown in FIG. 5, first, the initial value of the threshold value and the threshold value setting unit 15 are set (S1). This initial value may be obtained as described above, or an appropriate value may be set.
After that, it waits for a certain time to elapse (S2). The predetermined time is, for example, 10 to 30 seconds. If it is necessary to quickly follow a suitable threshold according to the use environment and purpose, the time is shortened accordingly. The band dividing unit 4 performs, for example, discrete Fourier transform, but the threshold value may be updated for each transform frame.
[0024]
When a certain time has elapsed, threshold value update processing is performed (S3). This update process is the process shown in FIG. At this time, if the update storage is held for the parameter value difference holding unit 13 by the output of the sound determination unit 11 in parallel with the sound source separation process while waiting in step S2, the update process starts. Then, the threshold value is determined using the parameter value difference between the channels for each band held in the parameter value difference holding unit 13 at that time, and the threshold value of the threshold setting unit 15 is updated with this threshold value. Thus, the update process can be performed in a short time. After this update process, the process returns to step S2 to wait for a certain time. In this way, the threshold value is updated at regular intervals, and a high-quality separated signal can be obtained.
[0025]
In this update process, it is preferable that the update is performed when the evaluation value becomes better than the previous evaluation value, and the update is not performed unless it becomes better. For example, as shown in FIG. 5, after step S2, a threshold value is calculated by the process shown in FIG. 2 (S4), separation processing is performed using this threshold value, and evaluation is performed by the separation evaluation unit 21. A value (SNR or SDR) is calculated (S5). This evaluation value and the previous evaluation value are compared by the update determination unit 25 (S6). If the evaluation value is better (larger) than the previous time, the update is executed, that is, the threshold value calculated in step S4, and accordingly in step S4. The threshold value set in the threshold value setting unit 15 is used as it is for the separation process.
[0026]
However, if the evaluation value is worse (smaller) than the previous evaluation value, the threshold value used so far, that is, the threshold value used for the separation process during the time waiting in step S2 is used. Therefore, even if a predetermined time elapses, the threshold value is not updated and the process returns to step S2. During the separation process for calculating the evaluation value in step S5, the threshold value and the memory are evacuated or the separation process for calculating the evaluation value is performed during the separation process for the evaluation value in step S5 so that the update can be performed. As the threshold value used for the threshold value, the threshold value in the threshold value determination unit 14 is used, and the threshold value in the threshold value determination unit 14 is set and updated in the threshold value setting unit 15 in the update execution in step S7. To. The threshold value calculation in step S4 is also performed in a short time by performing update storage for the parameter value difference holding unit 13 based on the output of the sound determination unit 11 while waiting for a certain period of time. Threshold value can be calculated.
[0027]
The correction processing of the calculated threshold value shown in FIG. 4 is performed for the threshold value when the initial setting in step S1 is performed by the processing shown in FIG. Alternatively, it may be performed on the threshold value calculated in the update process of step S3, or may be performed on the threshold value calculated in step S4.
[0028]
In the above description, the threshold Lth [fi] is determined using the inter-channel level difference ΔL _M [fi] for each band of the mixed signal and the inter-channel level difference ΔL _N [fi] for the noise signal. determining the threshold Lth with reference to inter-channel level differences [Delta] L _N of level difference [Delta] L _M and the noise signal between the channels, which to make a determination of whether the signal from which the sound source using commonly to each band signal It may be. The output from the inter-channel level differences [Delta] L _M and [Delta] L _N are for example inter-channel difference detecting unit 3, may be taken out and screened by the output of the sound determination unit 11. Further, the present invention uses the inter-channel time difference Δτ _M [fi] of the mixed signal for each band and the inter-channel time difference Δτ _N [fi] for the noise signal, or the inter-channel time difference Δτ _{M of the} mixed signal and the channel of the noise signal. using the time difference .DELTA..tau _N, it may determine the threshold τth [fi] or Tauth. In short, the threshold value may be determined by using a parameter value difference between channels in a broad sense including band-specific values for mixed signals and noise signals.
[0029]
The sound source separation device according to the present invention may be caused to function in a computer. In this case, a program for causing the computer to execute each process of the sound source separation method of the present invention described above is downloaded into the computer via a CD-ROM, a magnetic disk, other recording media, or a communication line, and this program is downloaded. Just execute.
[0030]
【The invention's effect】
As described above, according to the present invention, the threshold value can be determined based on the inter-channel parameter value difference of the target sound source signal + noise signal and the inter-channel parameter value difference of the noise signal. The threshold value that has been determined through mistakes can be determined from physical quantities that can be measured in advance, saving a lot of time and effort, and allowing the sound source separation device to operate with a certain level of performance.
In addition, the threshold value is sequentially updated as necessary, so that the threshold value follows the time variation of the signal. At this time, the threshold value before the update and the newly obtained threshold evaluation value If the threshold value with higher performance is always selected, the separation process can be performed with more stable performance than when the threshold value is updated each time without comparing the evaluation values.
[Brief description of the drawings]
FIG. 1 is a diagram showing a functional configuration example of an apparatus according to the present invention.
FIG. 2 is a flowchart showing an example of a procedure for obtaining a threshold value according to the present invention.
FIG. 3 is a diagram showing signals necessary for calculating SNR and SDR as evaluation values.
FIG. 4 is a flowchart showing an example of a procedure for threshold correction processing.
FIG. 5 is a flowchart showing an example of a threshold update processing procedure;

Claims

Dividing each output channel signal from two microphones arranged apart from each other into a plurality of frequency bands;
For each same band of each of the divided output channel signals, a process of detecting the acoustic signal parameter value difference between the channels for each band;
Based on the acoustic signal parameter value difference between the band-specific channels in each band, the threshold value is used to determine which of the output channel signals divided in the band is a signal input from which sound source. The process of judging,
Based on the determination, a process of selecting at least one signal input from the same sound source from each of the band-divided output channel signals;
In a sound source separation method comprising a step of synthesizing a plurality of band signals selected as signals from the same sound source as sound source signals,
The process of detecting the acoustic signal parameter value difference between the channels of the mixed signal in which the target sound source signal and the noise signal are mixed,
Detecting a difference in acoustic signal parameter value between channels of the noise signal;
A sound source separation method comprising: obtaining a threshold value based on an acoustic signal parameter value difference between channels of the mixed signal and an acoustic signal parameter value difference between channels of the noise signal.

Periodically, the acoustic signal parameter value difference between the mixed signal channels and the acoustic signal parameter value difference between the noise signal channels are detected, and the threshold value is obtained using these difference values. The sound source separation method according to claim 1, further comprising an update process of updating a threshold value used for determination.

In the update process, before the threshold value used for the determination, sound source separation is performed using the newly obtained threshold value, and the separation performance is expressed using the separated signal and the signal before separation. 3. The sound source separation according to claim 2, wherein an evaluation value is calculated, and if the evaluation value is better than an evaluation value when a previous threshold value is used, the newly obtained threshold value is used for the determination. Method.

Perform sound source separation using the obtained threshold value, calculate an evaluation value representing separation performance using the separated signal and the signal before separation,
The calculated threshold value is corrected so that the evaluation value is within a predetermined range, and the corrected threshold value, or the threshold value before correction if it is within the predetermined range before correction, is described above. 4. The sound source separation method according to claim 1, further comprising a correction process for setting a threshold value used for determination.

A band dividing unit that divides each output channel signal of two microphones arranged apart from each other into a plurality of frequency bands;
For each same band of each output channel signal divided by the band dividing unit, a parameter value difference detecting unit for inter-band channel parameter values for detecting an acoustic signal parameter value difference between channels for each band;
A sound source that uses a threshold value to determine which of the output channel signals obtained by dividing the band of each band is a signal input from which sound source based on the parameter value difference between channels in each band. A signal determination unit;
A sound source signal selection unit that selects at least one signal input from the same sound source from each of the output channel signals divided in band based on the determination of the sound source signal determination unit;
In a sound source separation device including a sound source synthesis unit that synthesizes a plurality of band signals selected as signals from the same sound source by the sound source signal selection unit as a sound source signal,
A sound determination unit for determining whether the output channel signal is a mixed signal in which a target sound source signal and a noise signal are mixed, or a noise signal;
A parameter value difference holding unit for storing and holding an acoustic signal parameter value difference between channels of the mixed signal and an acoustic signal parameter value difference between channels of the noise signal, based on a determination signal of the sound determination unit;
A threshold value determination unit that obtains the threshold value based on the acoustic signal parameter value difference between the retained mixed signal channels and the acoustic signal parameter value difference between the noise signal channels; Sound source separation device.

The program for making a computer perform each process of the sound source separation method in any one of Claims 1-4.