JP4293340B2 - Dialogue understanding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To understand spoken words more accurately by solving a problem that a speech conventional system recognizes an inputted speech signal basically by a method for sequential acoustic recognition in word units, and in this method, a sound source is a naturally generated speech and easily influenced by ambient noise to hinder a conversation as user's spoken words can not correctly be understood. <P>SOLUTION: A conversation understanding device as an embodiment of the present invention performs speech recognition for words and then classifies the obtained words by categories and classes to enable detailing, answering and correction, and an interaction corresponding to a speaking type of reinput, and also takes influence of past recognition records into consideration to select a more likely word while considering the relation with the context. <P>COPYRIGHT: (C)2004,JPO&amp;NCIPI

Description

（数１）式において単語ｗがＮ−ｂｅｓｔ候補の中でｉ番目の候補に含まれている確からしさｐ_ｉは下記の（数２）式から求められる。ここで、Ｌ_ｉはＮ−ｂｅｓｔ候補それぞれに対する対数尤度である。
【００１７】
【数２】

また、クラス単位での信頼度は上記単語単位の場合と同様に、第１位候補に含まれる各単語ｗのクラスＣ_ｗにより、信頼度Conf(Ｃ_ｗ)を以下の（数３）式から求められる。
【００１８】
【数３】

ここで、上記単語単位の場合と同様、ｐ_ｉは下記の（数４）式から求められる。
【００１９】
【数４】

以上のようにして得られた認識データ（認識結果候補文、尤度及び信頼度）は言語理解部１０４に入力される。この言語理解部１０４はクラススコア生成部１０５、カテゴリー理解部１０６、単語スコア生成部１０７および理解内容生成部１０８の各部で構成されており、使用者からの複数回にわたる発話に伴って入力される単語と、その属するクラスの信頼度とから理解結果を生成する機能を有する。ここで、クラススコア生成部１０５は、使用者からの複数回にわたる発話に伴って入力される単語のクラス信頼度からどのクラスが発話されたかを示すスコアを計算するものであり、カテゴリー理解部１０６は使用者からの複数回にわたる発話に伴って入力されるクラススコアからクラスの分類を示すカテゴリーの理解結果、すなわち、どのカテゴリーが発話されたかを出力するものである。また、単語スコア生成部１０７は、使用者からの複数回にわたる発話に伴って入力される単語の信頼度から、どの単語が発話されたかを示すスコアを計算し、理解内容生成部１０８は、上記で得られたカテゴリー理解結果（１０６出力）および単語スコア（１０７出力）から理解内容を生成する機能を有する。
【００２０】
以上のようにして得られた言語理解部１０４の出力情報は応答生成部１０９に入力され、上記言語理解部１０４で得られた理解内容から応答文を生成する。この応答文は音声合成部１１０でデジタル信号として合成され、図示しないが音声合成部１１０内蔵のＤ／Ａコンバータ、出力増幅器を経て音声出力として出力する。一方、この出力応答文はＧＵＩ表示部１１１を経て図示しないが表示装置上に表示する。なお、認識履歴１１２は過去の認識状況を履歴データとして記憶しておく例えばハードディスク記憶装置等の記憶装置である。
【００２１】
次に上記装置構成の作用について説明する。
まず本発明の実施の形態で扱う目的地の表現形式を説明する。インターチェンジ、駅、市区町村名を目的地に設定することができ、各々には県、自動車道、鉄道路線を付加することができる。前記のように図２はこれら表現形式を階層構造的に表示したものである。すなわち、本実施の形態では、目的地を上位、中位、下位３段階の部分発話の組み合わせにより発話することができ、これを本実施の形態ではこの３段階の各々をカテゴリーと呼ぶ。上位カテゴリーＰＲでは、県（都道府県）を発話することができ、中位カテゴリーＨＲでは自動車道、または鉄道路線を発話することができ、下位カテゴリーＬＭではインターチェンジ、市区町村、駅を発話することができる。
【００２２】
本発明の実施の形態においては、対話形式での目的地設定をより柔軟な発話によって行うことを目的としている。すなわち、使用者は例えば、「静岡県の東名自動車道の浜松西インターチェンジ」と言うように、一度ですべてのカテゴリーを発話することもできる。また第一の発話で「静岡県」と発話し、第二の発話で「東名高速の浜松西インターチェンジ」と発話するように複数回に分けて発話することも可能である。
また使用者が複数回の発話を行うとき、過去の発話に対してより詳細な情報を追加していく詳細化発話を可能とするものである。例えば、第一の発話で、「静岡県の」と発話し、第二の発話で「浜松市」と発話することが可能である。また使用者が複数回の発話を行うとき、システムの応答結果を訂正する発話を可能とするものである。例えば、第一の発話「静岡県の浜松市」に対して、第一の応答「静岡県の浜松西インターチェンジですか」と誤った応答がなされたとき、第二の発話で「いいえ浜松市です。」と発話することが可能である。
また使用者が複数回の発話を行うとき、システムからの応答が質問であったときに、それに回答する発話も可能とするものである。例えば、第一の応答が「静岡県の何インターチェンジですか」であったとき、第二の発話で「浜松西インターチェンジです」と発話することが可能である。
また使用者が複数回の発話を行うとき、システムからの応答が再入力を促す発話であったときに、それに回答する発話を可能とするものである。例えば、第一の応答「もう一度発話してください」であったとき、第二の発話で第一の発話と同様の発話を行うことが可能である。
本実施の形態における認識対象語は図３に例示するようなものである。本実施の形態における対話例は図４に示すようなものである。図４中、Ｕは使用者の発話であり、Ｓはシステムからの応答であり、数字は発話順である。
【００２３】
次に、本発明の実施の形態における動作を図５のフローチャートを用いて説明する。
ステップ３０１で処理を開始し、まず、使用者が発話開始を指示するために、図示しないが音声入力スイッチ（発話スイッチ）がオン状態に操作されたこと検出（ステップ３０２）した場合、音声信号の取り込み開始のステップ（ステップ３０３）に移行する。ここで、音声入力スイッチのオン状態への操作が検出されない場合は、この操作が検出されるまでステップ３０２で待ち状態となる。
ステップ３０３では、使用者は認識対象文に含まれる発話を行う（例えば図３に例示した語等）。図１における音声入力部１０１は、マイクロホンからの信号をＡ／Ｄコンバータでデジタル信号に変換し、音声認識部１０２に出力する。音声認識部１０２は発話スイッチの操作がなされるまでは、前記デジタル信号の平均パワーの演算を継続している。前記発話スイッチが操作された後、前記平均パワーにくらべてデジタル信号の瞬時パワーが所定値以上に大きくなった時、使用者が発話したと判断し、音声信号の取り込みが開始される。
取り込まれた音声信号は、図１における音声認識部１０２において、記憶してある認識対象文と入力されたデジタル化された音声信号とを比較し、尤度を演算する（ステップ３０４）ことにより、複数の候補を設定する。なお本ステップ３０４を実行する間も、並列処理により上記の音声信号取り込みは継続されている。
デジタル化された音声信号の瞬時パワーが所定時間以上所定値以下の状態が継続した時、システム側では使用者の発話が終了したと判断し、音声信号の入力処理を終了する（ステップ３０５）。これにより、図１における音声認識部１０２は複数の認識結果候補文を尤度順にならべた上位Ｎ候補を、尤度データとともに出力する。図６にこの出力結果の例を示す。図６において、ＸＸＸと記されている部分は、各単語に対する算出された尤度を示している。
前記のＮ−Ｂｅｓｔ候補と呼ばれる音響的な尤度で順位付けられた複数の候補からなる認識結果をもとに、単語とクラスの２種類の信頼度について音響的な尤度とＮ−Ｂｅｓｔ候補中の出現頻度から、事後確立に基づく尺度として信頼度が演算される（ステップ３０６）。この演算は図１における信頼度生成部１０３において実行されるもので、演算結果の例を図７に示す。図７において、左側の表は図６で示した音声認識部出力であり、右側の表の単語信頼度は、ある単語が発話された可能性を示し、クラス信頼度はあるクラスの単語が発話された可能性を示す。なお、本演算に関しては前記「従来の技術」の項で述べた「非特許文献２」駒谷他、”音声対話システムにおける音声認識結果の信頼度の利用法”、日本音響学会講演論文集、3-5-2、pp73-74、2000に詳述されている。
【００２４】
以上のようにして発話された単語の信頼度を求めて尤らしい単語の推定が行われるが、本発明においては、システムと使用者との間での対話により単語推定の精度をさらに向上させている。このため、図１におけるクラススコア生成部１０５においてクラススコアが演算されるが（ステップ３０７）、このクラススコア演算に先立ち、使用者の発話タイプの判定が行われる。すなわち、第一の発話タイプは、以前の情報に新しい情報を追加する働きがある発話タイプである。例えば、詳細化および回答の処理がこれに相当する。また第二の発話タイプは、以前の情報を訂正する働きがある。例えば、訂正および再入力の処理がこれに相当する。このいずれの発話タイプであるかの判定は図８に示すように、判定材料の欄に記載されている判定材料の状況に対して発話タイプが判定される。また、これ以外の判定方法も存在する。例えば、地名入力でよく用いられる部分的な言い直し発生をＤＰマッチングによるワードスポッティング法を用いて検出する方法があり、これに関しては、角谷、北岡、中川”カーナビの地名入力における誤認識時の訂正発話の分析と検出、情報処理学会研究報告、音声言語情報処理37-11、2001に詳述されている。
発話タイプが判定された後に、クラススコア生成部１０５においてクラススコアが生成される。クラススコアは、対話中すなわち使用者の複数回の発話中におけるクラスの尤もらしさを示す値である。この場合、以前に理解した情報を残しつつ、新しい情報を付加することで、より適切にスコアを生成することができる。このクラススコアの生成は前記の発話タイプ別に異なる生成式を用いて行われる。したがって、図５におけるステップ３０７は図９に示すように２分割された処理が行われることになる。すなわち図８の判定材料の欄に記載の状況によりステップ３１５で詳細化、回答の発話タイプに該当するか否かを判定し、該当する場合はステップ３１６で処理し、該当しないで訂正、再入力の発話タイプの場合はステップ３１７で処理された後いずれの場合も処理はステップ３０８に移行する。
【００２５】
詳細化、回答の発話タイプにおける場合、すなわち図９におけるステップ３１６の場合のクラススコアは（数５）式で求められる。
Score(c) = Score(c)*weight_s + Conf(c) （数５）
但し、Scoreはクラススコアであり、（数５）式の左辺が新たに求められたクラススコアであり、（数５）式の右辺が過去の（認識履歴１１２から読み出した）クラススコアに対する処理である。Confは最新の認識結果から得られたクラス信頼度である。weight_sは0.0〜1.0の値を採る重みである。cはスコアを生成するクラスである。重みweight_sにより一定の割合で更新前のクラススコアを下げているのは、”情報が古くなるごとに信頼性が低下する”という方針を適用しているからである。また、weight_sは、実際の発話データを用いて実験的に求めることができる。更新されたクラススコアは認識履歴１１２に書き込まれる。
【００２６】
詳細化・回答発話タイプのクラス生成の様子を図１０に示す。使用者は、過去の発話（旧クラススコア1.00）で「県」「鉄道路線」の発話を行っており、最新の発話（新クラス信頼度欄が0.81）で「駅」を発話している。この場合のクラススコア生成は（数５）式に基づいて行われる。
【００２７】
訂正・再入力の発話タイプの場合、すなわち図９におけるステップ３１７の場合におけるクラススコアは（数６）式で求められる。
Score(ca)=Score(ca)*weight_t-Conf(cb)+ Conf(ca) （数６）
但し、Scoreはクラススコアであり、（数６）式の左辺が新たに得られたクラススコアであり、（数６）式の右辺が過去の（認識履歴１１２から読み出した）クラススコアである。Confは最新の認識結果から得られたクラス信頼度である。weight_tは0.0〜1.0の値を採る重みである。caはスコアを生成するクラスであり、cbはcaと同じカテゴリーで異なる全てのクラスである。（数５）式と比較し、同カテゴリー、異クラスの信頼度を減算していることである。これによりクラスを間違えた場合にスコアが修正され易くなる。更新されたクラススコアは認識履歴１１２に書き込まれる。
訂正・再入力発話タイプのクラス生成の様子を図１１に示す。使用者は、過去の発話で「県」クラスの発話を行っており、クラススコアの値が不十分でカテゴリーを特定できず、システム応答は「もう一度発話して下さい」を出力している。使用者は次に再度同じ「県」クラスの発話を行い更新後のクラススコアを得ている（例えば、「県」の発話に対しては旧クラススコアと新クラス信頼度の両方の欄にスコアが記載されている）。この場合のクラススコア生成は（数６）式に基づいて行われている。
【００２８】
続いて、カテゴリー理解処理のステップ３０８に移るが、この処理は図１におけるカテゴリー理解部１０６で、過去の（認識履歴から読み出した）クラススコアと最新の認識結果におけるクラス信頼度との両方に対してカテゴリースコアを計算することにより実行される。この処理の様子を図１２に示す。カテゴリースコアは、図１２のａで表示した部分およびＢで表示した部分におけるそれぞれの欄の数字から知れるように、同じカテゴリーに属する全てのクラススコアあるいは信頼度を加算したものである。それぞれのカテゴリースコアは閾値で判定され、ＰＲ（上位）、ＨＲ（中位）、ＬＭ（下位）の３カテゴリーに対して、判定結果の論理和を計算する。そこで得られた結果が、現在までに発話されたカテゴリーの組み合わせを示している。クラススコアが図１２であった場合、それに続くカテゴリー理解の様子を図１３に示す。すなわち、旧および新スコアから各カテゴリーに対して判定を行い、その結果としてカテゴリー理解が得られる。
【００２９】
次に、ステップ３０９の単語スコア生成が行われるが、このステップ３０９は図１における単語スコア生成部１０７で実行され、
１）過去の（認識履歴１１２中に既に存在する）単語、および
２）新たに出現した単語（最新の認識結果中の単語）
の２つに対して、各々別々の方針を用いてスコアを生成する。後者２）の場合の単語は、最新の認識結果のＮ−Ｂｅｓｔ候補に含まれる全単語が対象となる。スコア生成は、図１における言語理解部１０４が最新の認識率を獲得するたびに、１）→２）の順番で実行される。
【００３０】
上記１）の認識履歴中に存在する単語は、単語の新しさ、システムの応答内容とユーザ発話タイプ（詳細化、訂正、回答、再入力）から、既存の単語スコアを上下させて、新しい単語スコアを生成する。これには以下５種類の方針を使用する。
方針１：古い情報は、信頼性が低くなるという仮定のもとに、新しい認識結果が入力されるたびに、認識履歴中に存在する全ての単語のスコアを下げる。
方針２：認識履歴中の単語Ａと認識結果単語Ｂが詳細化の関係にあった場合、単語Ａのスコアを上げる。
方針３：認識履歴中の単語Ａと認識履歴中の単語Ｂが訂正の関係にあった場合、単語Ａのスコアを下げる。
方針４：認識結果に肯定（はい、うん等）が含まれていた場合、応答に含まれていた単語のスコアを上げる。
方針５：認識結果に否定後（いいえ、ちがう等）が含まれていた場合、応答に含まれていた単語のスコアを下げる。
認識履歴中の単語スコアの生成は、下記の（数７）式による。
【００３１】
Score(Wd)=Score(Wd)-p1+p2*Conf(Ws)-p3*Conf(Wt)
+i*(p4*Conf(yes)-p5*Conf(no)-p6*Conf(rej)) （数７）
但し、Scoreは認識履歴中の単語のスコアであり、右辺が更新前、左辺が更新後である。Ｗｄは計算対象となる認識履歴１１２中の単語である。方針１に対応する項としては、ｐ１があり単語のスコアを下げる項である。方針２と方針３に対応する項に関しては、ｐ２、ｐ３は重み付け、Confは最新の認識結果から得られる信頼度であり、Ｗｓは最新の認識結果に含まれ、Ｗｄと詳細化の関係にある全ての単語であり、Ｗｔは最新の認識結果に含まれＷｄとは訂正の関係にある全ての単語である。方針４、方針５に対応する項に関しては、ｉは前回のシステム応答に単語が含まれている場合はｉ＝１となり、含まれていない場合はｉ＝０となる。またｙｅｓは最新の認識結果に含まれる肯定語を示し、ｎｏは今回の認識結果に含まれる否定後を示し、ｒｅｊは今回の認識結果に含まれる文末否定語を示す。
【００３２】
前記２）における最新の認識履歴中の単語であって、認識履歴にまだ登録されていない単語、すなわち新たに出現した単語のスコアの生成は、応答内容とユーザ発話タイプ（詳細化、訂正、回答、再入力）、Ｎ−Ｂｅｓｔの順位、発話長（発話された単語の数）により、音声認識の信頼度を上下させて、単語スコアを生成する。これには以下４種類の方針を使用する。
方針６：認識結果の単語Ａと応答とに含まれる単語Ｂが詳細化の関係にある場合
、単語Ａのスコアを上げる。
方針７：システム応答が質問（例、何インターチェンジですか？）であって、認識結果の内容が回答である場合、認識結果の単語のスコアを上げる。
方針８：認識結果の上位には正解単語が多く含まれているので、上位に含まれる単語のスコアを上げる。
方針９：発話長が長い発話（短い発話）は認識されやすい（認識されにくい）ため、１カテゴリーの結果はその単語のスコアを下げ、２カテゴリー以上の単語はそのスコアを上げる。
【００３３】
最新の認識履歴中の単語であって、認識履歴にまだ登録されていない単語のスコアの生成は、以下の（数８）式による。
【００３４】
Score(Wd)=Conf(Wd)+p6*Score(Ws)+p7*Conf(Wa)
+Conf(Wd)*(p8+p9*len2-p10*len1) （数８）式
但し、Scoreは認識履歴中の単語のスコアであり、Confは最新の認識結果から得られる信頼度である。Ｗｄは計算対象となる認識履歴中の単語である。方針６に対応する項に関しては、ｐ６が重み付けであり、Ｗｓは認識履歴に含まれるＷｄと詳細化の関係を持つ全ての単語である。方針７に対応する項に関しては、ｐ７は重み付け、認識結果が質問に対する回答である場合の認識結果に含まれる単語である。方針８に対応する項としてはｐ８がＮ−Ｂｅｓｔの順位の高さに応じた重み付けである。方針９に対応する項としてはｐ９、ｐ１０が重み付けであり、ｌｅｎ２は認識のカテゴリーが２以上であるときｌｅｎ２＝１になり、ｌｅｎ１は認識のカテゴリーが１であるときｌｅｎ１＝１になる値である。
上記１）で更新された単語のスコア、上記２）で追加された単語、およびそのスコアは統合された認識履歴として、認識履歴１１２に書き込まれる。統合された認識結果の例を実際の県名、鉄道名等を実例として図１４に示す。図中同名が複数存在する場合（厚木、田無等）があるが、これは複数路線に含まれる駅の名称などである。
【００３５】
上記により得られたカテゴリー理解結果、および前記統合された認識履歴とから、妥当な組み合わせとして複数個の候補を生成する。すなわち、上記により得られた情報を基に本装置が理解した内容として、尤らしい候補を複数個生成する（ステップ３１０）。この処理は図１における理解内容生成部１０８において実行される。図１３の結果から、ＰＲ、ＨＲおよびＬＭの３カテゴリーが発話されており、図１４から前記に該当し、実際に存在する組み合わせを抽出し候補とする。各カテゴリーのスコアの和が最大のものを選択する。その結果を図１５に示す。理解結果として、＜ＰＲカテゴリー＝愛知、スコア＝1.47＞、＜ＨＲカテゴリー＝名古屋鉄道、スコア＝1.17＞、＜ＬＭカテゴリー＝豊橋、スコア＝0.62＞が選択されている。
【００３６】
以上、図１における言語理解部１０４の各ステップで処理された結果である理解内容から応答フラグを生成する（ステップ３１１）までの全処理過程を説明した。これにより得られた出力(応答)情報は図１の応答生成部１０９で実行される。この応答フラグの種類を図１６に示す。なお、図１６における各ビット（ａ乃至Ｆの各ビット）が示す内容を図１７に示す。前記理解結果から、カテゴリーに該当する単語が存在する場合、該当するフラグを立てるがこの場合スコアを４段階で評価した値（ビット数）のフラグを立てる。すなわち、スコアが最大から最小までを評価１から評価４とし、フラグは１０００、０１００、００１０、０００１とする。
【００３７】
応答生成部１０９は、上記の応答フラグを利用し、対話における以下の方針に沿った応答を行う。
応答方針１：了承（相槌）
下位カテゴリーがなく、上位カテゴリーまたは中位カテゴリーのスコア評価が評価１の場合、対話をスムーズに進めるための応答を行う。
例 ユーザ発話 …「静岡県」
システム応答…「はい」
応答方針２：復唱
スコア評価が２の場合や、ユーザ発話の文頭に否定後が来た場合は確認の意味も込めて復唱を行う。
例 ユーザ発話 …「静岡県」
システム応答…「静岡県」
応答方針３ 最終確認
下位カテゴリーが発話され、信頼できる（スコア評価が１か２）場合は、最終確認を行う。
例 ユーザ発話 …「浜松インターから乗ります」
システム応答…「浜松インターを設定してよろしいですか」
応答方針４：目的地設定
前応答に下位カテゴリーがあり、肯定発話が信頼できる（スコア評価が１か２）場合は、目的地に設定する。
例 システム応答…「浜松インターを設定してよろしいですか」
ユーザ発話 …「はい」
システム応答…「目的地に設定しました」
応答方針５：分からない情報のみ尋ねる
ユーザに対して分からない情報のみを尋ねる。
例 ユーザ発話…「静岡県の東名自動車道です」 （下線部のスコア評価が低い時）
システム応答…「静岡県の何自動車道ですか？」
応答方針６：自信のない情報は応答しない
上位カテゴリー（ＰＲ）と中位カテゴリー（ＨＲ）の組み合わせで、どちらか一方だけ信頼できない（スコア評価が４）場合、スコアの高いものだけ応答することで対話を進める。
例 ユーザ発話…「静岡県の東名自動車道」（下線部のスコア評価が低いとき）
システム応答…「東名自動車道の」
応答方針７ 別情報の付加情報が少なく、スコア評価が悪いときに、上のカテゴリーも聞くことによって認識率の向上を図る。
例 ユーザ発話…「浜松インターから乗る」（下線部のスコア評価が低いとき）
システム応答…「何県のインターですが」
応答方針８：次の発話を促す
上位カテゴリーにつづいて肯定発話がきて、信頼できる場合（スコア評価が１か２の場合）次の発話を促す。
例 システム応答…「東名自動車道」
ユーザ発話 …「はい」
システム応答…「東名自動車道のどこですか」
応答方針９：別の候補を返す
否定発話が信頼できる場合（スコア評価が１か２の場合）前回の応答に用いていない別候補を返す。
例 システム応答…「浜松インターを設定しますか」
ユーザ発話 …「いいえ」
システム応答…「浜松西インターを設定しますか」
応答方針１０：前応答の繰り返し
肯定発話や否定発話が信頼できない場合（スコア評価が４の場合）
例 システム応答…「浜松インターを設定してよろしいですか」
ユーザ発話 …「はい」 （下線部のスコア評価が低いとき）
システム応答…「浜松インターを設定してよろしいですか」
応答方針１１：聞き返し
全ての情報に対して信頼できない場合（スコア評価が４の場合）
例 ユーザ発話 …「静岡県」 （下線部のスコア評価が低いとき）
システム応答…「もう一度発話してください」
応答生成部１０９は、上記の対話方針を実施するために、前記の応答フラグを、図１６のフラグテーブルと照らし合わせ、フラグが最初に一致した応答パターンで応答を返す。
応答生成部１０９が前記理解結果から生成した応答フラグは
"1 111000 1000 1000 0100 0000 0000 0000 0"
であり、図１６のフラグテーブルとの参照の結果、図示しないが応答パターン
“ＰＲカテゴリー単語”、“ＰＲカテゴリークラス”の
“ＨＲカテゴリー単語”、“ＨＲカテゴリークラス”の
“ＬＭカテゴリー単語”、“ＬＭカテゴリークラス”を設定してよろしいですか。が選択され、その結果、
「愛知県の名古屋鉄道の豊橋駅を設定してよろしいですか」が応答文として生成される。
【００３８】
以上のようにして生成された応答はステップ３１２で実行されるもので、図１における音声合成部１１０を経由して音声信号として出力され、またＧＵＩ表示部１１１を経由してディスプレイ上に表示される。
この段階で、入力処理が全て完了したか否かの確認が行われる（ステップ３１３）。すなわち、下位カテゴリー（ＬＭ）の単語が確定している場合は（ステップ３１３でｙｅｓの場合）、ステップ３１４に移行し全ての入力処理を終了する。もし、下位カテゴリー（ＬＭ）の単語が確定していない場合（ステップ３１３でｎｏの場合）は処理を継続する。本例では、「愛知県の名古屋鉄道の豊橋駅を設定してよろしいですか」が応答されている段階であり、次に使用者が「はい」を発話することで、「目的地に設定しました」の応答を行ったのち処理を終了する。
【図面の簡単な説明】
【図１】本発明による対話理解装置の基本構成ブロック図。
【図２】発話された単語の階層構造的分類法を示す構成図。
【図３】認識対象語と発話タイプとの関係を示す対応図。
【図４】システム／使用者間での対話の例を示す発話・応答図。
【図５】システムの動作を示すフロー図。
【図６】音声認識部の出力としての認識結果候補文と尤度との関係を示す対象図。
【図７】認識結果候補文と尤度との関係から信頼度を求める対象図。
【図８】発話タイプと発話タイプ判定材料との対象図。
【図９】発話タイプによる処理の使い分けを示すフロー図。
【図１０】詳細化・回答発話タイプにおける更新後のクラススコア生成過程を示す旧クラススコアとの対象図。
【図１１】訂正・再入力発話タイプにおける更新後のクラススコア生成過程を示す旧クラススコアとの対象図。
【図１２】クラススコア演算手順を示す新旧スコア比較図。
【図１３】カテゴリー理解処理における新旧スコア演算過程を示す対象図。
【図１４】統合された認識結果の実例を示すスコア対象図。
【図１５】言語理解最終スコアの項目別比較図。
【図１６】応答フラグと応答パターン対象図。
【図１７】応答フラグとその内容対象図。
【符号の説明】
１０１：音声入力部 １０２：音声認識部
１０３：信頼度生成部 １０４：言語理解部
１０５：クラススコア生成部 １０６：カテゴリ理解部
１０７：単語スコア生成部 １０８：理解内容生成部
１０９：応答生成部 １１０：音声合成部
１１１：ＧＵＩ表示部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to device control by a voice dialogue system, and more particularly to a dialogue understanding device (voice information input / output device) that is required to perform control without bothering an operator's “hand” or “eye”. .
[0002]
[Prior art]
[Patent Document 1]
JP-A-8-278793
[Non-Patent Document 1]
Kai, Ishimaru, Ito, Konishi, Ito “Analysis of features of correction utterances in destination setting tasks and its application to detection” The Acoustical Society of Japan Annual Conference 2-1-8, pp.63-64,2001
[Non-Patent Document 2]
Komatani, Kawahara, "How to use the reliability of speech recognition results in spoken dialogue systems" Proceedings of the Acoustical Society of Japan 3-5-2, pp.73-74,2000
In conventional voice dialogue systems, the sound source is a naturally generated voice, and in vehicles, etc., there is an influence of noise during driving, etc., so it is not possible to correctly understand the user's utterance. In some cases, the response was different from the person's intention. As a result, the dialogue between the system and the user does not proceed smoothly, which may cause discomfort to the user. For example, “Non-Patent Document 1” or “Non-Patent Document 2” has been reported as a countermeasure. The former is a research on misrecognition in speech recognition, and the latter uses reliability in the speech recognition result. This is a study on dialogue control. The methods adopted in these studies are based on the recognition of input speech signals acoustically in units of words, and language recognition including contextual information that humans are performing. not going. For this reason, there is a limit to reducing the influence of the background noise and the like on the speaking condition of the speaker and on both the transmitting side and the receiving side.
[0003]
Also, in Patent Document 1, candidates are sequentially determined from the result of parsing, and the likelihood with a standard pattern corresponding to the candidate (the definition of likelihood will be described later) and the candidate calculated from the text database are associated. A method is also disclosed in which the sum of occurrence likelihood / occurrence order likelihood is used as a candidate to increase the matching speed with a small storage capacity. However, in this method, only candidates are selected from the recognition results, and the likelihood of the candidates is obtained, and it is unquestionable whether or not the final response has the information required by the user. .
[0004]
[Problems to be solved by the invention]
As described above, in the conventional method, the input speech signal is focused on sequentially recognizing and improving the recognition accuracy, and it has been examined whether it has reached the contents required by the user. There wasn't. In addition, it is also performed for recognition in sentence form, but this is performed by comparison with a text database prepared in advance, and whether or not the result is in accordance with the user's request. There was no room for recognition. The present invention has a language comprehension ability applicable to a dialog control system that has a recognition ability that can exceed practical performance and that can withstand practical use and that can acquire information required by a user in a short time. It is an object of the present invention to provide a user / system interaction understanding device.
[Means for Solving the Problems]
One way to achieve the above objective is to interactively understand speech information from the context flow in the utterance content of the speaker. According to this method, a better result can be expected than the conventional method that simply focuses on improving the intelligibility or intelligibility of the speech. The present invention discloses a specific algorithm for realizing this method, and provides a specific apparatus based thereon.
[0005]
In the present invention, it is fundamental to perform language understanding and response generation by combining speech information processing using context information with reliability of speech recognition. In other words, not only using the reliability of conventional speech recognition, but also using the results learned using the type of utterance and information of the conversation history (recognition history), more probable language understanding interactively Was made to run.
[0006]
  For this reason, according to claim 1, the utterances included in the dialogue are classified into a plurality of categories and classes formed by subdividing the categories in the order in which the utterances are included, and which class of words is included. Understand the content of the dialogue from the class score that gives the certainty of the utterance or the word included in the utterance, and perform detailed / reply dialogue to add more detailed information to the utterance content, and the refinement / response Generate new class scores for utterance typesWhen reading the past class score calculated based on the user's past utterance and stored as a recognition history,ExcessiveLastAdd a new confidence level based on the latest recognition result to the weighted value to reduce the class score.Updated class score and updatedNew class scoreOf class score sequentially usingA dialog understanding device that updates the recognition history is defined.
[0007]
  According to a second aspect of the present invention, there is provided a voice input means comprising a microphone and a voice amplifier, a voice recognition means for digitizing the output of the voice input means for voice recognition, and the reliability of the result recognized by the voice recognition means. A hierarchical structure comprising a plurality of categories set in advance using the results obtained by the speech recognition means and the reliability generation means, and the classes obtained by subdividing the categories. Class score generator for classifying and determining the likelihood of utterances classified into the class, category understanding unit for determining each category from the results obtained thereby, and word score generation for determining the probability of the recognized word Language understanding means comprising: an understanding content generation unit that generates an understanding content as a result processed by each processing unit, and the language understanding means Storage means for storing past recognition histories used for executing processing, response generation means for creating response information from results obtained from the language understanding means, and output for outputting the response information The interaction understanding device according to claim 1, further comprising: means for generating the class score for the refinement / answer utterance type.When reading the past class score calculated based on the user's past utterance and stored as a recognition history,pastNo kuAdd a new confidence level based on the latest recognition result to the weighted value to reduce the lath score.Updated class score and updatedNew class scoreOf class score sequentially usingIt defines a dialogue understanding device that updates the recognition history.
[0008]
  In Claim 3, in the dialog understanding device according to Claim 1 or Claim 2,RememberedpastNo kuIt defines a dialogue comprehension device that executes the update of the last score by the following equation.
[0009]
  Score (c) = Score (c) * weight_s+ Conf (c)
  However, Score: PastNo kuRuscore
          Conf: Class reliability for the latest recognition result
        weight _s : Weight (0.0 <weight_s<1.0)
            c: Class that generates a score
  In claim 4, the utterances included in the dialogue are classified into a plurality of categories and classes formed by subdividing the categories in the order in which the utterances are included, and which class of words is uttered Understanding the content of the dialogue from the class score that gives the certainty and the words included in the utterance, there is an incorrect response from the system, and processing to correct it, that is, correction / re-entry, and correction / re-entry The class score is newly generated for the input utterance type.When reading the past class score calculated based on the user's past utterance and stored as a recognition history,pastNo kuThis is a score obtained by adding a new confidence level based on the latest recognition result to a value that is weighted so that the lath score is reduced, and subtracting all the confidence levels of the same category and different classes.The class score is updated, and the class score is sequentially updated using the updated new class score.It defines a dialogue understanding device that updates the recognition history.
[0010]
  6. The voice input means comprising a microphone and a voice amplifier, voice recognition means for digitizing the output of the voice input means for voice recognition, and reliability of the result recognized by the voice recognition means A hierarchical structure comprising a plurality of categories set in advance using the results obtained by the speech recognition means and the reliability generation means, and the classes obtained by subdividing the categories. Class score generator for classifying and determining the likelihood of utterances classified into the class, category understanding unit for determining each category from the results obtained thereby, and word score generation for determining the probability of the recognized word Language understanding means comprising: an understanding content generation unit that generates an understanding content as a result processed by each processing unit, and the language understanding means Storage means for storing past recognition histories used for executing processing, response generation means for creating response information from results obtained from the language understanding means, and output for outputting the response information 5. The dialogue understanding device according to claim 1, further comprising: means for generating the class score for the correction / re-input utterance type.When reading past class scores calculated based on the user's past utterances and recorded as recognition history,AbovepastNo kuThis is a score obtained by adding a new confidence level based on the latest recognition result to a value that is weighted so that the lath score is reduced, and subtracting all the confidence levels of the same category and different classes.The class score is updated, and the class score is sequentially updated using the updated new class score.It defines a dialogue understanding device that updates the recognition history.
[0011]
  In claim 6, in the dialog understanding device according to claim 4 or claim 5,RememberedIt defines a dialogue understanding device that updates past recognition history using the following formula.
[0012]
  Score (ca) = Score (ca) * weight_t-Conf (cb) + Conf (ca)
However, Score: Class score of recognition history
            Conf: Class confidence of the latest recognition result
          Weight_t: Weight (0.0 <weight_t<1.0)
          Ca: class that generates scores
          Cb: different class in the same category as ca
  In claim 7In the dialogue understanding device having the configuration according to claim 1 and claim 4, speech capture processing is performed while calculating likelihood and reliability of the input signal, and the speech type of the captured speech is “detailed”・ "Response" or "Correction / Re-input" is discriminated, and based on the discrimination result, it is determined whether to use the arithmetic expression according to claim 3 or the arithmetic expression according to claim 6, and to generate a class score The past recognition history stored in the storage medium is updated based on the generated class score.It stipulates a story understanding device.
[0013]
【The invention's effect】
  According to the present invention, as described above, not only speech recognition of words is performed, but the recognized words are further classified into categories and classes, and the relation between the words and the context is considered. By adopting the selection method, the recognition accuracy can be further improved efficiently. For example, this is particularly effective when used in a noisy environment such as voice input in a vehicle navigation system.
DETAILED DESCRIPTION OF THE INVENTION
  Embodiments of the present invention will be described below with reference to the drawings.
  FIG. 1 shows a basic configuration of a dialogue understanding device according to the present invention. An input analog voice input signal is converted into a digital signal by a voice input unit 101. Here, the voice input unit 101 includes a microphone, an input amplifier, and an A / D converter. The digitized voice signal is input to the voice recognition unit 102, and a matching process is performed between the voice signal input from the user and the recognition target sentence stored in the voice signal recognition unit 102, and a plurality of recognitions are performed. The result candidate sentences and their likelihoods (details will be described later) are output. The output information is output from the plurality of recognition result candidate sentences input with a single utterance from the user to the words included in the recognition result candidate sentences and the classification of these words in the reliability generation unit 103. Outputs the reliability indicating the likelihood of the indicated class.
  Here, as shown in FIG. 2, a class classifies an expression format indicating a destination hierarchically. Here, categories are included in order from wide to narrow.ArrayA class is a classification of words contained in each category by content. In the example of FIG. 2, for example, each word is classified into three categories of upper (PR), middle (HR), and lower (LM), and further classified into a plurality of classes in each category. For example, in the case of FIG. 2, the upper category has only one class of “prefecture”, but the lower category has three classes of “interchange”, “city”, and “station”.
[0014]
The reliability in units of words is obtained as follows. That is, first, a word string (hereinafter referred to as N-best) arranged in descending order of likelihood from the first place to the Nth place of a candidate word string (for example, a sentence formed of a plurality of words) obtained from a word recognition result. Logarithmic likelihood for each word is obtained. Here, the likelihood is a value defined by the posterior probability that the speech signal sequence uttered by the user is W when the speech signal sequence obtained from the recognition result is Y. On the other hand, it is the maximum probability among the ratios of the product of the “prior probability that the speech signal sequence Y is observed” and the “probability that the speech signal sequence W is uttered” and the probability that the speech signal sequence Y is observed.
[0015]
As a result, the reliability Conf (w) of the word w included in the first candidate is obtained from the following equation (1).
[0016]
[Expression 1]

In formula (1), the probability that the word w is included in the i-th candidate among the N-best candidates p_iIs obtained from the following (Equation 2). Where L_iIs the log likelihood for each N-best candidate.
[0017]
[Expression 2]

Also, the reliability in class units is the class C of each word w included in the first candidate as in the case of the above word units._wThe reliability Conf (C_w) Is obtained from the following (Equation 3).
[0018]
[Equation 3]

Here, as in the case of the above word unit, p_iIs obtained from the following equation (4).
[0019]
[Expression 4]

The recognition data (recognition result candidate sentence, likelihood and reliability) obtained as described above is input to the language understanding unit 104. The language understanding unit 104 includes a class score generation unit 105, a category understanding unit 106, a word score generation unit 107, and an understanding content generation unit 108. The language understanding unit 104 is input with a plurality of utterances from the user. It has a function of generating an understanding result from a word and the reliability of the class to which the word belongs. Here, the class score generation unit 105 calculates a score indicating which class has been uttered from the class reliability of the word input in association with a plurality of utterances from the user, and the category understanding unit 106. Is a category understanding result indicating a classification of a class from a class score inputted with a plurality of utterances from a user, that is, which category is uttered. Further, the word score generation unit 107 calculates a score indicating which word is uttered from the reliability of the word input with a plurality of utterances from the user, and the understanding content generation unit 108 It has a function of generating an understanding content from the category understanding result (106 output) and the word score (107 output) obtained in the above.
[0020]
The output information of the language understanding unit 104 obtained as described above is input to the response generation unit 109, and a response sentence is generated from the understanding content obtained by the language understanding unit 104. This response sentence is synthesized as a digital signal by the voice synthesizer 110 and is output as a voice output through a D / A converter and an output amplifier built in the voice synthesizer 110 (not shown). On the other hand, the output response text is displayed on a display device (not shown) via the GUI display unit 111. The recognition history 112 is a storage device such as a hard disk storage device that stores past recognition status as history data.
[0021]
Next, the operation of the above apparatus configuration will be described.
First, an expression format of a destination handled in the embodiment of the present invention will be described. Interchanges, stations, city names can be set as destinations, and prefectures, expressways, and railway lines can be added to each destination. As described above, FIG. 2 shows these representation formats in a hierarchical structure. That is, in the present embodiment, a destination can be uttered by a combination of partial utterances of upper, middle, and lower three stages, and in the present embodiment, each of these three stages is called a category. In the upper category PR, you can utter prefectures (prefectures), in the middle category HR, you can utter expressways or railway lines, and in the lower category LM, you can speak interchanges, municipalities, and stations. Can do.
[0022]
In the embodiment of the present invention, an object is to perform destination setting in an interactive manner by more flexible speech. That is, the user can speak all categories at once, for example, “Hamamatsu Nishi Interchange on Tomei Expressway in Shizuoka Prefecture”. It is also possible to speak in multiple sessions, such as “Shizuoka Prefecture” in the first utterance and “Hamamatsu Nishi Interchange at Tomei Expressway” in the second utterance.
Further, when the user utters a plurality of times, a detailed utterance in which more detailed information is added to the past utterance is enabled. For example, it is possible to speak “Shizuoka Prefecture” in the first utterance and “Hamamatsu City” in the second utterance. Further, when the user utters a plurality of times, the utterance for correcting the response result of the system is enabled. For example, when the first response “Hamamatsu City in Shizuoka Prefecture” is answered incorrectly as the first response “Is Hamamatsu Nishi Interchange in Shizuoka Prefecture”, the second utterance is “No Hamamatsu City” Can be spoken. "
In addition, when the user utters a plurality of times, when the response from the system is a question, it is also possible to utter the answer. For example, when the first response is “How many interchanges in Shizuoka Prefecture”, it is possible to say “This is Hamamatsu Nishi Interchange” in the second utterance.
In addition, when the user utters a plurality of times, when the response from the system is an utterance that prompts re-input, the utterance can be answered. For example, when the first response is “Please speak again”, it is possible to perform the same utterance as the first utterance in the second utterance.
The recognition target words in the present embodiment are as illustrated in FIG. An example of dialogue in the present embodiment is as shown in FIG. In FIG. 4, U is the user's utterance, S is the response from the system, and the numbers are the utterance order.
[0023]
Next, the operation in the embodiment of the present invention will be described with reference to the flowchart of FIG.
In step 301, the processing is started. First, when it is detected that the voice input switch (speech switch) is operated in the on state (not shown) in order to instruct the user to start the utterance (step 302), The process proceeds to the capture start step (step 303). Here, if an operation to turn on the voice input switch is not detected, the process waits at step 302 until this operation is detected.
In step 303, the user utters an utterance included in the recognition target sentence (for example, the word illustrated in FIG. 3). The voice input unit 101 in FIG. 1 converts a signal from the microphone into a digital signal by an A / D converter and outputs the digital signal to the voice recognition unit 102. The voice recognition unit 102 continues to calculate the average power of the digital signal until the speech switch is operated. After the utterance switch is operated, when the instantaneous power of the digital signal becomes larger than a predetermined value as compared with the average power, it is determined that the user has uttered, and voice signal capturing is started.
The captured speech signal is compared in the speech recognition unit 102 in FIG. 1 with the stored recognition target sentence and the input digitized speech signal, and the likelihood is calculated (step 304). Set multiple candidates. Note that while the step 304 is executed, the above audio signal capturing is continued by parallel processing.
When the instantaneous power of the digitized audio signal continues for a predetermined time or more and a predetermined value or less, the system determines that the user's speech has ended, and ends the audio signal input processing (step 305). Thereby, the speech recognition unit 102 in FIG. 1 outputs the top N candidates obtained by arranging a plurality of recognition result candidate sentences in order of likelihood together with the likelihood data. FIG. 6 shows an example of the output result. In FIG. 6, the part marked XXX indicates the calculated likelihood for each word.
Based on the recognition result consisting of a plurality of candidates ranked by the acoustic likelihood referred to as the N-Best candidate, the acoustic likelihood and the N-Best candidate for two types of reliability of the word and the class From the appearance frequency, the reliability is calculated as a measure based on the subsequent establishment (step 306). This calculation is performed in the reliability generation unit 103 in FIG. 1, and an example of the calculation result is shown in FIG. In FIG. 7, the table on the left is the output of the speech recognition unit shown in FIG. 6, the word reliability in the table on the right indicates the possibility that a certain word has been uttered, and the class reliability has a word in a certain class The possibility that was done. Regarding this calculation, “Non-Patent Document 2” Komagaya et al., “How to use reliability of speech recognition results in a spoken dialogue system” described in the section of “Prior Art”, Proc. -5-2, pp73-74, 2000.
[0024]
As described above, the reliability of the spoken word is obtained to estimate the likelihood of the word. In the present invention, the accuracy of the word estimation is further improved by the dialogue between the system and the user. Yes. Therefore, the class score is calculated by the class score generation unit 105 in FIG. 1 (step 307), but the user's speech type is determined prior to the class score calculation. That is, the first utterance type is an utterance type that serves to add new information to the previous information. For example, details and response processing correspond to this. The second utterance type has the function of correcting previous information. For example, correction and re-input processing correspond to this. As shown in FIG. 8, the utterance type is determined based on the status of the determination material described in the determination material column. There are also other determination methods. For example, there is a method of detecting the occurrence of partial rephrasing often used in place name input using the word spotting method by DP matching. Regarding this, correction at the time of misrecognition in the place name input of Kakutani, Kitaoka, Nakagawa Details of speech analysis and detection, Information Processing Society of Japan Research Report, Spoken Language Information Processing 37-11, 2001.
After the utterance type is determined, the class score generation unit 105 generates a class score. The class score is a value indicating the likelihood of the class during the dialogue, that is, during the user's multiple utterances. In this case, a score can be generated more appropriately by adding new information while leaving previously understood information. The class score is generated using a different generation formula for each utterance type. Therefore, in step 307 in FIG. 5, the process divided into two is performed as shown in FIG. That is, it is determined in step 315 whether or not it corresponds to the utterance type of the answer according to the situation described in the judgment material column of FIG. 8, and if so, it is processed in step 316. In the case of the utterance type, the process proceeds to step 308 in any case after the process in step 317.
[0025]
In the case of refinement and utterance type of answer, that is, the class score in the case of step 316 in FIG.
Score (c) = Score (c) * weight_s + Conf (c) (Formula 5)
However, Score is a class score, the left side of the formula (5) is a newly obtained class score, and the right side of the formula (5) is a process for a past class score (read from the recognition history 112). is there. Conf is the class reliability obtained from the latest recognition result. weight_sIs a weight taking a value of 0.0 to 1.0. c is a class that generates a score. Weight_sThe reason why the class score before the update is lowered by a certain ratio is that the policy that “the reliability decreases as the information becomes older” is applied. Weight_sCan be obtained experimentally using actual speech data. The updated class score is written in the recognition history 112.
[0026]
FIG. 10 shows how the refinement / answer utterance type class is generated. The user has uttered “prefecture” and “railway” in the past utterance (old class score 1.00), and uttered “station” in the latest utterance (new class reliability column is 0.81). In this case, the class score is generated based on the equation (5).
[0027]
In the case of the utterance type of correction / re-input, that is, in the case of step 317 in FIG. 9, the class score is obtained by Expression (6).
Score (ca) = Score (ca) * weight_t-Conf (cb) + Conf (ca) (Equation 6)
However, Score is a class score, the left side of the formula (6) is a newly obtained class score, and the right side of the formula (6) is a past class score (read from the recognition history 112). Conf is the class reliability obtained from the latest recognition result. weight_tIs a weight taking a value of 0.0 to 1.0. ca is a class that generates scores, and cb is all classes that are different in the same category as ca. Compared with equation (5), the reliability of the same category and different class is subtracted. This makes it easier to correct the score if you make a mistake in the class. The updated class score is written in the recognition history 112.
FIG. 11 shows how the correction / re-input utterance type class is generated. The user has uttered the “prefecture” class in the past utterance, the class score value is insufficient and the category cannot be specified, and the system response outputs “Please speak again”. Next, the user utters the same “prefecture” class again and obtains an updated class score (for example, for the utterance of “prefecture”, the score is in both the old class score and new class reliability fields). Is listed). In this case, class score generation is performed based on the equation (6).
[0028]
Subsequently, the process proceeds to step 308 of the category understanding process. This process is performed by the category understanding unit 106 in FIG. 1 for both the past class score (read from the recognition history) and the class reliability in the latest recognition result. This is done by calculating the category score. The state of this processing is shown in FIG. The category score is obtained by adding all class scores or reliability belonging to the same category, as can be seen from the numbers in the respective columns in the portion indicated by a and the portion indicated by B in FIG. Each category score is determined by a threshold value, and the logical sum of the determination results is calculated for three categories of PR (upper rank), HR (middle rank), and LM (lower rank). The results obtained there show a combination of categories spoken to date. When the class score is FIG. 12, the subsequent category understanding is shown in FIG. That is, each category is determined from the old and new scores, and as a result, category understanding is obtained.
[0029]
Next, word score generation in step 309 is performed. This step 309 is executed by the word score generation unit 107 in FIG.
1) past words (already present in recognition history 112), and
2) A new word (word in the latest recognition result)
For each of the two, a score is generated using a separate policy. In the case of the latter 2), all words included in the N-Best candidate of the latest recognition result are targeted. The score generation is executed in the order of 1) → 2) every time the language understanding unit 104 in FIG. 1 acquires the latest recognition rate.
[0030]
Words existing in the recognition history of 1) above are obtained by changing the existing word score from the newness of the word, the response contents of the system and the user utterance type (detailed, corrected, answered, re-input). Generate a score. The following five policies are used for this.
Policy 1: Under the assumption that the old information becomes less reliable, every time a new recognition result is input, the scores of all words existing in the recognition history are lowered.
Policy 2: When the word A in the recognition history and the recognition result word B are in a detailed relationship, the score of the word A is increased.
Policy 3: When the word A in the recognition history and the word B in the recognition history are in a correction relationship, the score of the word A is lowered.
Policy 4: If the recognition result includes affirmation (yes, yes, etc.), increase the score of the word included in the response.
Policy 5: If the recognition result includes a negative result (No, wrong, etc.), the score of the word included in the response is lowered.
The generation of the word score in the recognition history is based on the following equation (7).
[0031]
Score (Wd) = Score (Wd) -p1 + p2 * Conf (Ws) -p3 * Conf (Wt)
+ i * (p4 * Conf (yes) -p5 * Conf (no) -p6 * Conf (rej)) (Equation 7)
However, Score is a score of a word in the recognition history, and the right side is before update and the left side is after update. Wd is a word in the recognition history 112 to be calculated. The term corresponding to the policy 1 is p1 and is a term for lowering the word score. For terms corresponding to policy 2 and policy 3, p2 and p3 are weights, Conf is the reliability obtained from the latest recognition result, Ws is included in the latest recognition result, and there is a relationship between Wd and refinement. Wt is all words, Wt is included in the latest recognition result, and Wd is all words that are in a correction relationship. For terms corresponding to policies 4 and 5, i is i = 1 if a word is included in the previous system response, and i = 0 if it is not included. “Yes” indicates an affirmative word included in the latest recognition result, “no” indicates after negation included in the current recognition result, and “rej” indicates a sentence end negative word included in the current recognition result.
[0032]
The generation of the score of the word in the latest recognition history in 2) that has not yet been registered in the recognition history, that is, the newly appearing word, is the response content and user utterance type (detailed, corrected, answered) , Re-input), the rank of N-Best, and the speech length (the number of spoken words), the reliability of speech recognition is raised and lowered to generate a word score. For this, the following four types of policies are used.
Policy 6: When the recognition result word A and the word B included in the response are in a refinement relationship
, Raise the score of word A.
Policy 7: If the system response is a question (eg, how many interchanges?) And the content of the recognition result is an answer, the score of the word of the recognition result is increased.
Policy 8: Since many correct words are included at the top of the recognition result, the score of the words included at the top is increased.
Policy 9: Since an utterance with a long utterance length (short utterance) is easy to be recognized (not easily recognized), the result of one category lowers the score of the word, and the score of two or more categories raises the score.
[0033]
The generation of the score of a word in the latest recognition history that is not yet registered in the recognition history is based on the following equation (8).
[0034]
Score (Wd) = Conf (Wd) + p6 * Score (Ws) + p7 * Conf (Wa)
+ Conf (Wd) * (p8 + p9 * len2-p10 * len1) (Expression 8)
However, Score is a score of a word in the recognition history, and Conf is a reliability obtained from the latest recognition result. Wd is a word in the recognition history to be calculated. As for the term corresponding to the policy 6, p6 is a weight, and Ws is all words having a detailed relationship with Wd included in the recognition history. For the term corresponding to the policy 7, p7 is a word included in the recognition result when the weighting and recognition result is an answer to the question. As a term corresponding to the policy 8, p8 is a weight according to the height of the N-Best ranking. As terms corresponding to the policy 9, p9 and p10 are weights, len2 is len2 = 1 when the recognition category is 2 or more, and len1 is len1 = 1 when the recognition category is 1. is there.
The score of the word updated in 1), the word added in 2), and the score are written in the recognition history 112 as an integrated recognition history. An example of the integrated recognition result is shown in FIG. 14 using actual prefecture names, railway names, and the like as actual examples. There are cases where there are a plurality of the same names in the figure (Atsugi, Tanashi, etc.), which are the names of stations included in the plurality of routes.
[0035]
A plurality of candidates are generated as a reasonable combination from the category understanding result obtained as described above and the integrated recognition history. That is, a plurality of likely candidates are generated as contents understood by the present apparatus based on the information obtained as described above (step 310). This processing is executed in the understanding content generation unit 108 in FIG. From the result of FIG. 13, three categories of PR, HR, and LM are uttered, and combinations corresponding to the above from FIG. 14 and actually existing are extracted as candidates. Select the one with the largest sum of scores for each category. The result is shown in FIG. As an understanding result, <PR category = Aichi, score = 1.47>, <HR category = Nagoya Railway, score = 1.17>, <LM category = Toyohashi, score = 0.62> are selected.
[0036]
Heretofore, the entire processing process from generating the response flag to the understanding contents (step 311) as a result of processing in each step of the language understanding unit 104 in FIG. 1 has been described. The output (response) information thus obtained is executed by the response generation unit 109 in FIG. The types of response flags are shown in FIG. FIG. 17 shows the contents indicated by the bits (bits a to F) in FIG. Based on the result of understanding, when a word corresponding to a category exists, a corresponding flag is set. In this case, a flag of a value (number of bits) obtained by evaluating the score in four stages is set. That is, the score from the maximum to the minimum is set from evaluation 1 to evaluation 4, and the flags are 1000, 0100, 0010, 0001.
[0037]
The response generation unit 109 makes a response according to the following policy in the dialog using the response flag.
Response policy 1: Acknowledgment (consideration)
When there is no lower category and the score evaluation of the upper category or the middle category is evaluation 1, a response is made to smoothly advance the dialogue.
Example User utterance “Shizuoka”
System response ... "Yes"
Response policy 2: Repeat
If the score evaluation is 2, or if a negative word comes after the beginning of the user's utterance, it is repeated with the meaning of confirmation.
Example User utterance “Shizuoka”
System response… “Shizuoka”
Response policy 3 Final confirmation
If the lower category is spoken and reliable (score rating is 1 or 2), final confirmation is performed.
Example User utterance “Ride from Hamamatsu Inter”
System response: “Are you sure you want to set up Hamamatsu Inter”
Response policy 4: Destination setting
When there is a lower category in the previous response and the positive utterance is reliable (score evaluation is 1 or 2), the destination is set as the destination.
Example System response: “Are you sure you want to set Hamamatsu Inter”
User utterance “Yes”
System response… “Destination set”
Response policy 5: Ask only information you do not know
Ask users only for information they don't know.
Example User utterance… “Shizuoka PrefectureTomeiIt ’s a motorway. ”(When the underlined score is low)
System response ... “How many expressways in Shizuoka?”
Response policy 6: Do not respond to unconfident information
If only one of the combinations of the higher category (PR) and middle category (HR) is unreliable (score rating is 4), the dialogue is advanced by responding only to the higher score.
Example User utterance ...ShizuokaTomei Expressway in the prefecture "(when the score evaluation of the underline is low)
System response ... "Tomei Expressway"
Response policy 7 When the additional information of other information is small and the score evaluation is bad, the recognition rate is improved by listening to the above categories.
Example User utterance ...HamamatsuRide from the interchange "(when the underlined score is low)
System response… “How many interns are you?”
Response policy 8: Encourage next utterance
If an affirmative utterance comes after the upper category and it is reliable (score evaluation is 1 or 2), the next utterance is prompted.
Example System response… “Tomei Expressway”
User utterance “Yes”
System response… “Where is Tomei Expressway”
Response policy 9: Return another candidate
When the negative utterance is reliable (when the score evaluation is 1 or 2), another candidate not used for the previous response is returned.
Example System response “Do you want to set Hamamatsu Inter”
User utterance “No”
System response… “Do you want to set up Hamamatsu Nishi Inter”
Response policy 10: Repeat previous response
When positive or negative utterances are unreliable (score rating of 4)
Example System response: “Are you sure you want to set Hamamatsu Inter”
User utterance “Yes(When the underlined score is low)
System response: “Are you sure you want to set up Hamamatsu Inter”
Response policy 11: Listen back
When all information is unreliable (score rating is 4)
Example User utterance “Shizuoka Prefecture(When the underlined score is low)
System response… "Please speak again"
In order to implement the above dialogue policy, the response generation unit 109 compares the response flag with the flag table of FIG. 16 and returns a response with a response pattern in which the flag first matches.
The response flag generated from the understanding result by the response generation unit 109 is:
"1 111000 1000 1000 0100 0000 0000 0000 0"
As a result of referring to the flag table of FIG.
"PR category word", "PR category class"
"HR category word", "HR category class"
Are you sure you want to set “LM category word” and “LM category class”? Is selected and, as a result,
“Are you sure you want to set up Toyohashi Station on the Nagoya Railway in Aichi Prefecture?” Is generated as a response sentence.
[0038]
The response generated as described above is executed in step 312 and is output as a voice signal via the voice synthesizer 110 in FIG. 1 and displayed on the display via the GUI display unit 111. The
At this stage, it is confirmed whether or not all the input processes have been completed (step 313). That is, when the word of the lower category (LM) is fixed (in the case of yes in step 313), the process proceeds to step 314 and all input processes are terminated. If the word of the lower category (LM) is not fixed (in the case of no in step 313), the processing is continued. In this example, “Are you sure you want to set up Toyohashi Station on the Nagoya Railroad in Aichi Prefecture” is being answered, and then the user will say “Yes” to “Set as the destination. The process is terminated after the response “Yes” is made.
[Brief description of the drawings]
FIG. 1 is a basic configuration block diagram of a dialogue understanding device according to the present invention.
FIG. 2 is a block diagram showing a hierarchical classification method for spoken words.
FIG. 3 is a correspondence diagram showing the relationship between recognition target words and utterance types.
FIG. 4 is an utterance / response diagram showing an example of a dialogue between a system and a user.
FIG. 5 is a flowchart showing the operation of the system.
FIG. 6 is an object diagram showing a relationship between a recognition result candidate sentence as an output of a speech recognition unit and likelihood.
FIG. 7 is an object diagram for obtaining the reliability from the relationship between the recognition result candidate sentence and the likelihood.
FIG. 8 is an object diagram of an utterance type and an utterance type determination material.
FIG. 9 is a flowchart showing the proper use of processing by utterance type.
FIG. 10 is an object diagram with an old class score showing a class score generation process after updating in the detail / answer utterance type.
FIG. 11 is an object diagram with an old class score showing an updated class score generation process in the correction / re-input utterance type.
FIG. 12 is an old and new score comparison diagram showing a class score calculation procedure.
FIG. 13 is an object diagram showing an old and new score calculation process in category understanding processing.
FIG. 14 is a score target diagram showing an example of an integrated recognition result.
FIG. 15 is a comparative view of each language understanding final score.
FIG. 16 is a response flag and response pattern object diagram;
FIG. 17 is a response flag and its content target diagram.
[Explanation of symbols]
101: Voice input unit 102: Voice recognition unit
103: Reliability generation unit 104: Language understanding unit
105: Class score generation unit 106: Category understanding unit
107: Word score generation unit 108: Understanding content generation unit
109: Response generator 110: Speech synthesizer
111: GUI display section

Claims (7)

Class that classifies the utterances included in the dialogue into a class composed of multiple categories and subdivided categories in the order of the breadth of the utterances, and gives a certainty of which class of words were uttered Understand the conversation from the score and the words in the utterance,
When a refinement / answer dialogue is performed to add more detailed information to the utterance content, and the class score is newly generated for the utterance type of the refinement / answer , the past utterance of the user is recorded. reading past class score stored as computed recognized history based, the a value obtained by adding the weighted so past click Rasusukoa decreases, class score by adding the new reliability with the latest recognition result And a class score recognition history is sequentially updated using the updated new class score. A voice input means comprising a microphone and a voice amplifier;
Voice recognition means for digitizing the output of the voice input means for voice recognition; reliability generation means for calculating the reliability of the result recognized by the voice recognition means;
Using the results obtained by the voice recognition unit and the reliability generation unit, the plurality of categories set in advance and a hierarchical structure composed of the classes obtained by subdividing the categories are classified into the classes. Processed by a class score generation unit for determining the probability of utterance, a category understanding unit for determining each category from the results obtained thereby, a word score generation unit for determining the probability of a recognized word, and the above processing units A language understanding means comprising an understanding content generation unit for generating an understanding content as a result of
Storage means for storing past recognition history used for executing the processing in the language understanding means;
Response generation means for creating response information from the result obtained from the language understanding means;
The dialogue understanding device according to claim 1, further comprising an output unit for outputting the response information.
When the class score is newly generated for the refinement / answer utterance type, a past class score calculated based on the user's past utterance and stored as a recognition history is read, and the past of a value obtained by adding the weighted so click Rasusukoa decreases, to update the class scores by adding a new reliability using the latest recognition result, the recognition history sequential class score using a new class scores and the updated A dialogue understanding device characterized by updating. In dialogue understanding device according to claim 1 or claim 2, dialogue understanding device which updates the historical click Rasusukoa being the storage and executes the following equation.
Score (c) = Score (c) * weight _s + Conf (c)
However, Score: past click Rasusukoa
Conf: Class reliability for the latest recognition result
weight _s : Weight (0.0 <weight _s <1.0)
c: Class that generates a score Class that classifies the utterances included in the dialogue into a class composed of multiple categories and subdivided categories in the order of the breadth of the utterances, and gives a certainty of which class of words were uttered Understand the conversation from the score and the words in the utterance,
When there is an erroneous response from the system and corrects it, that is, when correction / re-entry is performed and the class score is newly generated for the utterance type of the correction / re-entry , the user's is calculated based on past speech reading past class score stored as a recognition history, the a value obtained by adding the weighted so past click Rasusukoa decreases, adding a new confidence with the latest recognition result The class score is updated with a score obtained by subtracting all the reliability of the same category and different classes, and the recognition history of the class score is sequentially updated using the updated new class score. Dialogue understanding device. A voice input means comprising a microphone and a voice amplifier;
Voice recognition means for digitizing the output of the voice input means for voice recognition; reliability generation means for calculating the reliability of the result recognized by the voice recognition means;
Using the results obtained by the voice recognition unit and the reliability generation unit, the plurality of categories set in advance and a hierarchical structure composed of the classes obtained by subdividing the categories are classified into the classes. Processed by a class score generation unit for determining the probability of utterance, a category understanding unit for determining each category from the results obtained thereby, a word score generation unit for determining the probability of a recognized word, and the above processing units A language understanding means comprising an understanding content generation unit for generating an understanding content as a result of
Storage means for storing past recognition history used for executing the processing in the language understanding means;
Response generation means for creating response information from the result obtained from the language understanding means;
The dialogue understanding device according to claim 1, further comprising: an output unit configured to output the response information.
Against utterance type of the correction and re-input, when the newly generated the class score reads past class score recorded as calculated based on past speech user recognition history, the past of a value obtained by adding the weighted so click Rasusukoa decreases, adds a new confidence with the latest recognition result, the class scores in the same category, and score obtained by subtracting all the reliability of a different class And a class score recognition history is sequentially updated using the updated new class score . 6. The dialogue understanding device according to claim 4, wherein the stored past recognition history is updated by the following equation.
Score (ca) = Score (ca) * weight _t -Conf (cb) + Conf (ca)
However, Score: Class score of recognition history
Conf: Class confidence of the latest recognition result
Weight _t : Weight (0.0 <weight _t <1.0)
Ca: class that generates scores
Cb: different class in the same category as ca In the dialogue understanding device having the configuration according to claim 1 and claim 4,
Perform voice capture processing while calculating likelihood and reliability of input signal,
Determine whether the utterance type of the captured voice is “detail / answer” or “correct / re-enter”
A class score is generated by determining whether to use the arithmetic expression according to claim 3 or the arithmetic expression according to claim 6 based on the discrimination result, and the class score is stored in the storage medium by the generated class score. and is past recognition history update-to-talk understanding device you and performing of.

Images