JP3166708B2 - Speech recognition device and method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech recognition apparatus, and more particularly, to a speech recognition apparatus and method having improved speaker adaptation functions.

[0002]

2. Description of the Related Art As a conventional speaker adaptation system of a speech recognition apparatus, for example, the Institute of Electronics, Information and Communication Engineers magazine DII (J78-D-
A method described in a paper entitled "Speaker Adaptation Using Tree-structured Speaker Clustering" in Vol. II, No. 1, pp. 1-9, January 1995) is known.

In this conventional method, a large number of speakers are clustered in a tree structure in advance (this is called a "speaker tree"). The root (root) node of the speaker tree corresponds to a set of all speakers (this set of speakers is referred to as a “speaker cluster”), and the terminal (leaf) nodes correspond to individual speakers. . The node on the way between the root and the leaf is
It corresponds to a set of acoustically similar speakers (speaker cluster).

[0004] Each node is associated with a standard pattern (Hidden Markov Model; abbreviated as "HMM") created (learned) using speech data of a corresponding set of speakers. The standard pattern is a set of all HMMs of all syllables when syllables such as “A”, “I”,.

[0005] A node in the speaker tree is selected using a small number of utterances of a new speaker, and subsequent speech recognition is performed using a standard pattern connected to the node.

[0006]

However, the above-mentioned conventional speaker adaptation method has the following problems.

That is, each node of the speaker tree corresponds to the entire HMM of all recognition units for the corresponding speaker cluster. Thus, for example, the voice of a new speaker may have a portion (eg, the sound of “a”) similar to one speaker cluster, but another portion (eg, the sound of “ku”) may have another speaker cluster. It cannot handle cases that are similar to

In the above conventional method, instead, a speaker cluster (node) in which a small amount of speech of a new speaker is the most similar on average is selected, and a standard pattern of the cluster (HMM of all recognition units) is selected. ) Must be used for all recognition units.

[0009] Accordingly, the present invention has been made in view of the above problems, and has as its object the purpose of speaker adaptation to a new speaker, for each recognition unit, or more finely, for each HMM state. It is an object of the present invention to provide a speech recognition device that enables selection of an optimum speaker cluster and an HMM or a probability distribution representing the cluster.

[0010]

According to the present invention, there is provided a speech recognition apparatus based on a Hidden Markov Model, wherein a plurality of speaker trees each having a probability distribution for a speaker cluster as a node are stored. Node tree storage unit, known node selecting means for selecting an optimal node in the speaker tree using a new speaker's utterance, and frequency information at a terminal node of a partial tree having the selected node as a parent node Speaker frequency calculating means for counting the number of unknown nodes, an unknown node selecting means for selecting an optimal node in the speaker tree using the frequency information of the terminal node, and a selection by the known node selecting means and the unknown node selecting means. A standard pattern updating unit that adapts the standard pattern using the probability distribution of the selected node.

[0011]

Embodiments of the present invention will be described below. In a preferred embodiment of the speech recognition apparatus of the present invention, referring to FIG. 1, a speaker tree storage unit (7), a known node selection unit (4), , A speaker frequency calculating unit (5), an unknown node selecting unit (6), and a standard pattern updating unit (8).

The speaker tree storage unit (7) stores the configured speaker tree for each recognition unit or for each state of the HMM.

The known node selecting section (4) uses the utterance of the new speaker and, for the recognition unit (or its HMM state) appearing in the utterance, stores the corresponding speaker tree in the speaker tree storage section (7). Select the optimal node from.

The speaker frequency calculating section (5) is a speaker tree stored in the speaker tree storage section (7). The terminal node of the partial tree having the selected optimum node as a parent node corresponds to the speaker node corresponding to the selected optimum node. The cumulative usage frequency of (cluster) is counted.

The unknown node selection unit (6) uses the usage frequency information for each speaker (cluster) counted by the speaker frequency calculation unit (5) to recognize that it did not appear in the utterance of the new speaker. From the speaker tree for a unit (or its HMM state):
Select the optimal node (speaker cluster).

The standard pattern updating section (8) uses the known node selecting section (4) and the HMM (or its state) corresponding to the node selected by the unknown node selecting section (6) to notify the new speaker. A corresponding standard pattern is created in the standard pattern storage unit (3).

As described above, according to the embodiment of the present invention, as speaker adaptation, each recognition unit or its H
An optimum speaker cluster can be selected for each MM state, and a more accurate standard pattern for a new speaker can be prepared.

[0018]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an embodiment of the present invention;

FIG. 1 is a diagram showing the configuration of an embodiment of the speech recognition apparatus of the present invention. Referring to FIG.
, A recognition / matching unit 2, a standard pattern storage unit 3, a known node selection unit 4, a speaker frequency calculation unit 5, an unknown node selection unit 6, a speaker tree storage unit 7, and a standard pattern update unit 8.
And is provided.

The analysis unit 1 performs feature extraction (mel cepstrum analysis, etc.) from the input speech waveform, and outputs a time series of feature vectors.

The standard pattern storage unit 3 stores hidden Markov models (HMMs) of all recognition units.

The recognition / collation unit 2 reads out the HMM stored in the standard pattern storage unit 3, compares the HMM with the time-series data of the feature vector sent from the analysis unit 1, and outputs a recognition result. In one embodiment of the present invention, the analysis unit 1, the recognition / collation unit 2, and the standard pattern storage unit 3
For any of the above, well-known techniques well known to those skilled in the art can be applied as they are, and the details thereof are not directly related to the subject of the present invention, and therefore, detailed description of the configuration is omitted.

The speaker tree storage unit 7 stores a speaker tree prepared for each recognition unit (or for each HMM state, as described later).

Hereinafter, a case where a speaker tree is prepared for each recognition unit will be described. Assume that syllables (“a”, “i”,..., “Ka”,...) Are used as recognition units.

It is assumed that the total number of syllable types is N. When a speaker tree is prepared for each syllable, a total of N speaker trees are stored in the speaker tree storage unit 7.

The case where the tree structure shown in FIG. 2 is a speaker tree of the syllable "A" will be described. In this case, the top root node is all speakers (mixed gender)
The HMM of the recognition unit "a" which is created or learned by collecting the voice data of "a" corresponds to the HMM.

The next node (one node below the root) is a node corresponding to the HMM created from the voice data of “a” of all male speakers, and a node of “a” of all female speakers. The HMM created from the voice data branches to the corresponding node.

The lowest-level (leaf) node of the speaker tree is composed of the individual speakers (speaker A, speaker B, C,...), Each of which is created from the voice data of "a". The HMM of the person "a" corresponds.

Referring to FIG. 1 again, the known node selecting section 4 uses the speaker adaptation utterance of the new speaker to store the recognition unit appearing in the utterance or the HMM state thereof in the speaker tree storage. An optimum node is selected from the corresponding speaker tree of the unit 7.

For a more specific description, also refer to the flowchart of FIG. 4 for a procedure for selecting an optimum node from the speaker tree of the syllable "A" using the voice data of the syllable "A" of the new speaker. This will be described below. FIG. 4 is a flowchart showing a main part of the processing of the speaker frequency calculation unit according to one embodiment of the present invention.

Step A1: The variable “current node” is
Let it be the highest root node of the speaker tree of syllable "a".

Step A2: If the "current node" is the terminal node (leaf) of the speaker tree, step A
Move to 7.

Step A3: HMM of "current node"
, The likelihood P0 of the voice data of the syllable “a” of the new speaker
Is calculated.

Step A4: Using the HMMs of all nodes immediately below the "current node", the likelihoods Q1 to QM of the voice data of the syllable "a" of the new speaker are calculated. Here, M is the number of nodes immediately below.

Step A5: The maximum value of the likelihoods Q1 to QM is set to Q
Set to 0. Also, the node giving the maximum value is stored as a variable “next node”.

Step A6: The maximum likelihood Q0 is P0 +
If it is larger than θ (θ is a predetermined threshold), “next node” is substituted for “current node”, and the process returns to step A2.

Step A7: Select the "current node" as the optimal node.

The speaker frequency calculation unit 5 counts the cumulative use frequency of the speaker (cluster) corresponding to the terminal node of the partial tree having the selected optimum node as a parent node.

As a specific example, a case where a new speaker utters “Ao” for speaker adaptation will be described.

The known node selection unit 4 uses the utterances to select the optimum nodes in the speaker tree of the syllable “A” and the speaker tree of the syllable “O”.

In FIG. 3, a node indicated by a double circle “◎” is a selected node.

The speaker frequency calculating unit 5 increases the cumulative use frequency of the speaker corresponding to the terminal node of the partial tree having the optimum node as the parent node by "1".

In the case of FIG. 3, from the speaker tree of the syllable "a", the cumulative use frequencies of the speakers C and D are each "1".
Only increased.

From the speaker tree of the syllable "O", the cumulative use frequencies of the speakers C and G are each increased by "1".

As a result, the cumulative use frequency of the speaker C is “2”, the cumulative use frequency of the speakers D and G is “1”, and the cumulative use frequency of the other speakers is “0”. Becomes

The unknown node selecting section 6 includes a speaker frequency calculating section 5
Using the cumulative usage frequency of each speaker counted in the above, an optimum node (speaker cluster) is selected from a recognition unit that did not appear in the utterance of the new speaker or the speaker tree for the HMM state.

The example shown in FIG. 3 will be specifically described. As mentioned above, new speakers use "Ao" for speaker adaptation.
And there is no utterance data for the syllable "i". For this reason, the known node selection unit 4 cannot determine the optimal node in the speaker tree of the syllable “i”.

Therefore, in one embodiment of the present invention, the optimum node of the speaker tree of the syllable "i" is selected by the following procedure. Hereinafter, a description will be given with reference to FIG. 5 which shows a main part of the processing of the unknown node / calculation unit according to one embodiment of the present invention.

Step B1: The cumulative usage frequency of each speaker calculated by the speaker frequency calculation unit 5 is given to the terminal node (corresponding to each speaker) of the speaker tree of the syllable “I”.

Step B2: The cumulative use frequency of each node is given by the sum of the cumulative use frequencies of the immediately lower (lower) nodes directly connected to the node. Thereby, the cumulative use frequency of the upper node is calculated in order from the terminal node.

Step B3: The variable “current node” is
Let it be the highest root node of the speaker tree.

Step B4: The maximum value of the cumulative use frequency of all the nodes immediately below the "current node" is set to R times. Also, the node giving the maximum value is stored as a variable “next node”. However, if there are a plurality of nodes giving the maximum value, the process proceeds to step B7.

Step B5: If the cumulative use frequency of the "next node" is equal to or less than the predetermined minimum use frequency (for example, twice), the process proceeds to step B7.

Step B6: Substitute "next node" for "current node" and return to step B4.

Step B7: Select the "current node" as the optimal node.

In this way, the optimum node can be selected for the speaker tree of the recognition unit that is not included in the adaptation utterance of the new speaker.

The standard pattern updating unit 8 collects the HMMs (or their states) corresponding to the nodes selected by the known node selecting unit 4 and the unknown node selecting unit 6, and generates a standard pattern corresponding to a new speaker. It is stored in the standard pattern storage unit 8.

As described above, an optimum node (speaker cluster) and an HMM representing the node (cluster) can be selected for each recognition unit. In particular, it is possible to select an optimum node not only for the recognition unit included in the adaptation utterance of the new speaker, but also for the recognition unit not included.

Incidentally, for each state of the HMM of each recognition unit,
When preparing separate and independent speaker trees, instead of preparing a speaker tree for each recognition unit, H
If a speaker tree is prepared for each MM state, an optimum node can be selected in exactly the same procedure as that described in the above embodiment.

[0060]

As described above, according to the present invention,
For each recognition unit or more finely for each state of the HMM, the optimal speaker cluster and the HM representing the cluster
There is an effect that M or a probability distribution can be selected.

The reason is that, in the present invention, a separate independent speaker tree is prepared for each recognition unit or, more specifically, for each state of the HMM, and a small amount of speaker adaptation utterances of a new speaker are prepared. This is because the optimum nodes can be selected for all speaker trees, and the corresponding HMMs can be used as standard patterns adapted for new speakers.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention.

FIG. 2 is a diagram for explaining one embodiment of the present invention;
It is a figure for explaining a speaker tree.

FIG. 3 is a diagram for explaining operations of a speaker frequency calculation unit and an unknown node calculation unit according to one embodiment of the present invention.

FIG. 4 is a flowchart showing a processing flow of a speaker frequency calculation unit according to one embodiment of the present invention.

FIG. 5 is a flowchart showing a processing flow of an unknown node calculating unit according to one embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Analysis part 2 Recognition collation part 3 Standard pattern storage part 4 Known node selection part 5 Speaker frequency calculation part 6 Unknown node selection part 7 Speaker tree storage part 8 Standard pattern update part

Continuation of the front page (56) References The Acoustical Society of Japan Fall Meeting, 1997, I-14, 1-1-14, Toshio Abe, "Speaker Adaptation Based on Speaker Class for Each Phoneme," p. . 27-28 (published September 17, 1997) IEICE Transactions, Vol. J 82-D- ▲ II No. 6, June 1999, Motoyuki Suzuki, "Tree Structure for Each Phoneme, Speaker Adaptation by Speaker Clustering," p. 981-989, (issued June 25, 1999) Transactions of the Institute of Electronics, Information and Communication Engineers, Vol. J 78-D- ▲ II ▼ No. 1, January 1995, Tetsuo Kosaka et al. "Speaker adaptation using tree-structured speaker clustering", p. 1-9, (issued January 25, 1995) Proceedings of Fourth International Conference on Spoken Language Processing, ICSLP 96, Vol. Ishii et al, "Speaker Adaptation Using Tree Structured Shared-State HMMs", p. 1149-1152, Octo bar 3-6, 1996, Philadelphia, U.S.A. S. A. Proceedings of 1995 IEEE International Conference on Acoustics, Speech and Signal Processing, Vol. 1, T. Watanabe et al, "High Speed Speech Recognition Using Tree-Structured Probability Density Function" p. 556-559 Proceedings of 1994 IEEE International Conference on Acoustics, Speech and Signal Processing, Vol. 1, T. Kosaka e t al, "Tree- Structu red Speaker Cluste ring for Fast Spea ker Adaptation", p. I-245~I-248 (58) investigated the field (Int.Cl. ^7, DB name) G10L 15/14 INSPEC (DIALOG) JICST file (JOIS) WPI (DIALOG) IEEE / IEEE Electronic Library Online

Claims (9)

(57) [Claims] 1. A speech recognition apparatus based on a hidden Markov model, a speaker tree storage unit for storing a plurality of speaker trees each having a probability distribution for a speaker cluster as a node, for each hidden Markov model of a recognition unit , Node selecting means for selecting an optimum node in the speaker tree using the utterance of a speaker, and speaker frequency calculating means for counting frequency information at a terminal node of a partial tree having the selected node as a parent node An unknown node selecting unit that selects an optimum node in the speaker tree using the frequency information of the terminal node; and a probability distribution of the nodes selected by the known node selecting unit and the unknown node selecting unit. And a standard pattern updating means for adapting the standard pattern. 2. The speech recognition apparatus according to claim 1, wherein a speaker tree is stored in the speaker tree storage unit for each state of a hidden Markov model of a recognition unit. 3. An apparatus for performing speech recognition based on a Hidden Markov Model, comprising: analysis means for extracting a feature from an input speech waveform and outputting a time series of feature vectors; and a Hidden Markov Model (HMM) of a recognition unit. Or a standard pattern storage unit that stores the state of the HMM, and reads out the state of the HMM or the HMM stored in the standard pattern storage unit and recognizes it by comparing it with the time-series data of the feature vector sent from the analysis unit. A recognition / matching unit that outputs a result; a speaker tree storage unit that stores a plurality of speaker trees for each of the HMMs or HMM states of the recognition unit; A known node selecting means for selecting an optimal node; and counting frequency information at a terminal node of a partial tree having the selected node as a parent node Speaker frequency calculating means, and calculating the frequency information of the upper node using the frequency information of the terminal node of the speaker tree, whereby the speech of the recognition unit not included in the adaptation utterance of the new speaker is obtained. An unknown node selecting means for selecting an optimum node in the speaker tree, an HMM corresponding to the node selected by the known node selecting means and the node selected by the unknown node selecting means, and a state of the HMM corresponding to the selected node. A standard pattern updating unit that stores the standard pattern corresponding to the speaker in the standard pattern storage unit. 4. A speaker adaptation method for an apparatus for performing speech recognition based on a Hidden Markov Model, comprising a speaker tree storage unit for storing a plurality of speaker trees each having a probability distribution for a speaker cluster as a node for each recognition unit. (A) known node selection processing for selecting a known node in the speaker tree using the utterance of a new speaker, and (b) end of a partial tree having the selected node as a parent node (C) using the frequency information of the terminal node, the speaker tree of the recognition unit not included in the adaptation utterance of the new speaker; An unknown node selection process for selecting an optimal node among the nodes, (d) corresponding to the new speaker using the known node selection process and the probability distribution of the nodes selected in the unknown node selection process. A standard pattern updating process for adapting to a standard pattern, the speaker adapting method for a speech recognition device. 5. The method according to claim 4, wherein a speaker tree is stored in the speaker tree storage unit for each hidden Markov model of a recognition unit. 6. A speaker tree is stored in the speaker tree storage unit for each state of a hidden Markov model of a recognition unit.
The speaker adaptation method for a speech recognition device according to claim 4, characterized in that: 7. The known node selection processing includes the steps of: (a) setting a root node of a syllable speaker tree as a recognition unit of the utterance of the new speaker as a current node; Determining whether the speaker tree is a leaf, and if it is a leaf, moving to step (f); (c) using the HMM of the current node to determine the likelihood P0 of the speech data of the syllable of the new speaker (D) calculating the likelihood of the speech data of the syllable of the new speaker using the HMM of each node immediately below the current node, and determining the node giving the maximum value Q0 of the likelihood as follows: (E) when the maximum value Q0 is larger than the P0 by a predetermined value, setting the next node as a current node and moving to the step (b); (F) the present 5. The method according to claim 4, further comprising the step of: selecting the node as an optimal node. 8. The unknown node selection processing includes the steps of: (a) using the cumulative use frequency of each speaker calculated by the speaker frequency calculation means as a recognition unit which did not appear in the utterance of the new speaker; , Given to the terminal node of the syllable speaker tree,
Calculating the cumulative use frequency of the upper nodes in order from the terminal node by giving the cumulative use frequency of each node of the speaker tree as the sum of the cumulative use frequencies of the lower nodes directly connected to the node; (B) making the current node the topmost root node of the speaker tree; and (c) finding the maximum value of the cumulative use frequency of all the nodes immediately below the current node, and assigning the node giving the maximum value. Storing as the next node, and when there are a plurality of nodes giving the maximum value, moving to step (f); (d) when the cumulative use frequency of the next node is equal to or less than a predetermined value Goes to step (f), (e) takes the next node as the current node, returns to step (c), and (f) selects the current node as the optimal node That step, the speech recognition apparatus according to claim 4, wherein the containing. 9. A speech recognition process based on a hidden Markov model.
An apparatus for executing includes a speaker tree storage unit for storing a plurality of speakers tree to node a probability distribution for the speaker cluster by the recognition unit, the speaker tree in using speech (a) new speaker (B) a speaker frequency calculation process for counting frequency information at a terminal node of a partial tree having the selected node as a parent node; (c) the terminal node By calculating the frequency information of the upper node using the frequency information of the node, the optimum node in the speaker tree is also selected for the speaker tree of the recognition unit not included in the utterance for adaptation of the new speaker. Unknown node selection processing, and (d) a standard pattern for adapting a standard pattern using the known node selection processing and the probability distribution of the nodes selected in the unknown node selection processing. Updating process, the (a) ~ a recording medium each processing recording a program for executing on a computer that constitutes the device (d).