JP6095588B2 - Speech recognition WFST creation device, speech recognition device, speech recognition WFST creation method, speech recognition method, and program - Google Patents

Description

  The present invention relates to a technique for creating a weighted finite state transducer for use in speech recognition and performing speech recognition using the weighted finite state transducer.

  In speech recognition using weighted finite state transducers (hereinafter referred to as WFST), information necessary for speech recognition such as acoustic models, dictionaries, and language models is first converted into WFST. WFST for speech recognition (hereinafter referred to as WFST for speech recognition) is created by synthesizing and optimizing the WFST. The created speech recognition WFST is decoded as a search space, and the input speech signal is converted into a character string as a speech recognition result. Here, the optimization is a general term for WFST optimization operations such as determinization and minimization. In addition, the acoustic score, which is a matching score between the input speech and the acoustic model, and the language score based on the language model are accumulated as weights, and finally the output symbol string having the highest weight is the speech recognition result.

  In recent years, many voice recognition services have been constructed by server / client type voice recognition systems. In a server / client type speech recognition system, when a speech signal uttered by a user is transmitted from a client terminal to a server device, the speech signal received by the server device is converted into a speech recognition result of text, and the server device to the client terminal. Voice recognition result is returned. At this time, in order to realize a quick response, the server device always develops a huge speech recognition WFST on the memory. The speech recognition WFST includes information on an acoustic model, a language model, and a word dictionary, and forms a huge network.

  In such a speech recognition service, it is possible to construct an optimum speech recognition WFST for each user by performing an adaptation such as updating information of a model or dictionary using speech or text accumulated for each user. Thus, the voice recognition accuracy can be improved. In particular, the word dictionary is updated by adding a new word specific to the user (for example, the name of a family member or friend) so that the word can be newly recognized, which is highly effective in improving voice recognition accuracy.

  However, in order to construct an optimum speech recognition WFST for each user, it is necessary to prepare huge general-purpose WFSTs for the number of users. Moreover, the general-purpose WFST must always be developed on the memory of the server device in order to ensure a quick response. On the other hand, the computing resources provided in the server device are limited. Therefore, this method is very difficult in practice.

  For an acoustic model and a language model, a framework has been proposed in which model adaptation is performed without rewriting the general-purpose WFST. For example, Patent Document 1 discloses that each phoneme HMM (Hidden Markov Model) of an acoustic model is learned separately from the general-purpose WFST, and the corresponding HMM state in the network of the general-purpose WFST synthesized beforehand. Describes a technique for performing speech recognition processing by dynamically applying to the above. If the technique of Patent Document 1 is used, the adaptation of the acoustic model is only the process of updating the parameters of the HMM. In other words, the acoustic model optimized for each user and the general-purpose WFST are held separately, and adaptation is realized by dynamically applying the acoustic model for each user to the general-purpose WFST that is always developed in the memory. become able to. Thus, if the technique of patent document 1 is used, it will become unnecessary to hold | maintain huge general purpose WFST for every user.

  Non-Patent Document 1 prepares a plurality of small models specialized for a specific topic from a large corpus in advance as language models, and mixes the weights of the language models according to the input speech application. Describes a method for constructing a WFST that can dynamically change. As a result, if there is only a mixture weight set for a specific user, a model specialized for that user can be generated dynamically without changing the structure of the general-purpose WFST.

JP 2012-113087 A

Brandon Ballinger, “On-Demand Language Model Interpolation for Mobile Speech Input”, INTERSPEECH 2010, pp.1812-1815, 2010.

  In the prior art, recombination of a huge general-purpose WFST as a base cannot be avoided in order to be able to recognize a new word. Therefore, it is necessary to create a speech recognition WFST that individually covers an appropriate vocabulary for each user.

  Therefore, if only the difference information of the acoustic model, the language model, and the word dictionary for each user with respect to a huge general-purpose WFST is compactly stored, and the difference information can be instantly loaded when voice recognition is executed, it can be applied to the general-purpose WFST. Model adaptation for each user can be realized without deploying a large amount of general-purpose WFST on a memory.

  An object of the present invention is to create a speech recognition WFST that can add a new word without recombining a large general-purpose WFST, and perform speech recognition using the speech recognition WFST.

  In order to solve the above-described problem, a speech recognition WFST creation apparatus according to an aspect of the present invention receives a phoneme string and inputs a phoneme string representing a subword based on the input phoneme string and a predetermined general-purpose word. A dictionary WFST creation unit that creates a dictionary WFST that outputs a generic word that matches a phoneme string that is input, and a generic word that is output from the dictionary WFST is given a weight of the generic word, and is assigned to a subword that is output from the dictionary WFST. Is a language model WFST creation unit that creates a language model WFST that gives a weight smaller than any of the weights of general-purpose words, an acoustic model WFST that converts an input speech signal into an acoustic environment, and a phoneme that converts the acoustic environment into a phoneme A general-purpose WFST creation unit that creates a general-purpose WFST using the WFST, the dictionary WFST, and the language model WFST; Is matched with one of the phoneme strings representing the reading of the new word, the weight of the matched new word is assigned and the new word is output, and the output of the general-purpose WFST matches with any of the general-purpose words Includes a word addition WFST creation unit that creates a word addition WFST that maintains the weight of the matched new word and outputs the general-purpose word.

  A speech recognition apparatus according to an aspect of the present invention includes a general-purpose WFST storage unit that stores a general-purpose WFST created by a speech recognition WFST creation apparatus, and a word that stores a word addition WFST created by a speech recognition WFST creation apparatus An additional WFST storage unit, and a speech recognition unit that converts an input speech signal into a speech recognition result using a speech recognition WFST in which a word addition WFST is dynamically synthesized with respect to a general-purpose WFST.

  According to the present invention, it is possible to obtain a speech recognition WFST that can add a new word without recombining a large general-purpose WFST. Since the difference information can be instantly adapted at the time of executing speech recognition, model adaptation for each user can be realized without re-deploying a large amount of huge WFST on the memory.

FIG. 1 is a diagram illustrating a functional configuration of the speech recognition WFST creation apparatus according to the first embodiment. FIG. 2 is a diagram illustrating a processing flow of the speech recognition WFST creation method according to the first embodiment. FIG. 3 is a diagram illustrating a configuration example of the dictionary WFST. FIG. 4 is a diagram illustrating a configuration example of the language model WFST. FIG. 5 is a diagram illustrating a configuration example of the word addition WFST according to the first embodiment. FIG. 6 is a diagram illustrating a functional configuration of the speech recognition apparatus according to the first embodiment. FIG. 7 is a diagram illustrating a processing flow of the speech recognition method according to the first embodiment. FIG. 8 is a diagram illustrating a configuration example of the word addition WFST according to the second embodiment. FIG. 9 is a diagram illustrating a functional configuration of the speech recognition WFST creation apparatus according to the third embodiment. FIG. 10 is a diagram illustrating a processing flow of the speech recognition WFST creation method according to the third embodiment. FIG. 11 is a diagram illustrating a functional configuration of the speech recognition apparatus according to the third embodiment. FIG. 12 is a diagram illustrating a configuration example of the word addition WFST according to the fourth embodiment. FIG. 13 is a diagram illustrating a configuration example of the word addition WFST according to the fifth embodiment. FIG. 14 is a diagram for explaining the pushing process. FIG. 15 is a diagram illustrating a configuration example of the word addition WFST of the sixth embodiment.

  In speech recognition processing by a general speech recognition WFST, an acoustic model WFST that converts an input speech signal into a phoneme environment, a phoneme WFST that converts a phoneme environment into a phoneme, and a dictionary WFST that converts a phoneme string into a word A general-purpose WFST in which a language model WFST for giving a language score to a word string and outputting the word string is synthesized in advance. When performing speech recognition, a general-purpose WFST created in advance is read, speech recognition processing of the input speech signal is performed, and a character string as a speech recognition result obtained by converting the speech signal is output.

Although the details will be described later, the speech recognition WFST creation apparatus and speech recognition apparatus of the present invention differ from the general speech recognition WFST creation apparatus and speech recognition apparatus in the following points.
1. The configurations of the dictionary WFST and the language model WFST are different. Accordingly, the process for creating each WFST is different.
2. A word addition WFST is created in addition to the general-purpose WFST.
3. In addition to the general-purpose WFST, the word addition WFST is read to perform speech recognition processing.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the component which has the same function in drawing, and duplication description is abbreviate | omitted.

[First embodiment]
<WFST creation device for voice recognition>
With reference to FIG. 1, an example of a functional configuration of the speech recognition WFST creation apparatus according to the first embodiment will be described. The speech recognition WFST creation device 1 according to the first embodiment is a device that creates the speech recognition WFST of the present invention.

  The speech recognition WFST creation apparatus 1 includes a general-purpose word pronunciation dictionary storage unit 10, a phoneme list storage unit 12, a general-purpose language model storage unit 14, a dictionary WFST creation unit 16, a language model WFST creation unit 18, a phoneme WFST storage unit 20, an acoustic Model WFST storage unit 22, dictionary WFST storage unit 24, language model WFST storage unit 26, general-purpose WFST creation unit 28, general-purpose WFST storage unit 30, general-purpose word list storage unit 32, new word reading list storage unit 34, WFST for adding words A creation unit 36 and a word addition WFST storage unit 38 are included.

  The speech recognition WFST creation apparatus 1 is configured, for example, by loading a special program into a known or dedicated computer having a central processing unit (CPU), a main memory (Random Access Memory, RAM), and the like. Special equipment. For example, the speech recognition WFST creation apparatus 1 executes each process under the control of the central processing unit. The data input to the speech recognition WFST creation apparatus 1 and the data obtained in each process are stored in, for example, the main storage device, and the data stored in the main storage device is read out as necessary to perform other processing. Used for Each storage unit included in the speech recognition WFST creation apparatus 1 includes, for example, a main storage device such as a RAM (Random Access Memory), an auxiliary storage configured by a semiconductor memory element such as a hard disk, an optical disk, or a flash memory. It can be configured by a device, or middleware such as a relational database or key-value store. Each storage unit included in the speech recognition WFST creation apparatus 1 may be logically divided, and may be stored in one physical storage device.

  The general-purpose word pronunciation dictionary storage unit 10 stores a general-purpose word pronunciation dictionary prepared in advance. The general-purpose word pronunciation dictionary is a list of words included in a general-purpose language model described later and their reading information. For example, information such as “red / aka” and “autumn / aki” in which one-to-one correspondence between general-purpose words and phoneme strings representing their readings is defined as one entry.

  The phoneme list storage unit 12 stores a phoneme list prepared in advance. The phoneme list is a list of all phonemes that appear in the reading information of the general word pronunciation dictionary.

  The general-purpose language model storage unit 14 stores a general-purpose language model prepared in advance. The general language model is a list of N-gram word strings, their N-gram probabilities, and their backoff coefficients. For example, N word strings, their probability values and their backoff coefficients, such as the trigram (3-gram) word string “red flowers”, trigram probability value “0.01” and backoff coefficient “0.1” Is one entry. Here, N is an integer value, and a value such as 3, 4 is used. The general language model may include an entry having no backoff coefficient.

  The general-purpose word list storage unit 32 stores a general-purpose word list prepared in advance. The general word list is a list of all words included in the general language model.

  The new word reading list storage unit 34 stores a new word reading list prepared in advance. The new word reading list is a list of new words different from general-purpose words and their reading information. For example, information such as “Agawa / akou” and “Akio / akio” in which a new word and a phoneme string representing the reading are associated one-to-one is defined as one entry. However, “Agawa” and “Akio” are not included in the general-purpose word model.

  With reference to FIG. 2, an example of the processing flow of the speech recognition WFST creation method executed by the speech recognition WFST creation device 1 will be described in the order of procedures actually performed.

  In step S16, the dictionary WFST creation unit 16 reads the general word pronunciation dictionary stored in the general word pronunciation dictionary storage unit 10 and the phoneme list stored in the phoneme list storage unit 12, and inputs the phoneme string as an input. A dictionary WFST is generated that outputs a general word that matches the input phoneme string among the subwords based on the phoneme string and the general word included in the general word pronunciation dictionary. A subword is a speech unit shorter than a word, such as a phoneme, a phoneme, or a syllable.

  An example of the configuration of the dictionary WFST of the present invention will be described with reference to FIG. Symbols associated with arcs represent input and output when passing through the arc. Among the symbols associated with the arc, the symbol string in front of “: (colon)” is an input symbol, and the symbol string behind is an output symbol. ε represents an empty symbol string. For example, when the input symbol “a, k, a” is input to the WFST shown in FIG. 3, the “a: ε” arc, the “k: ε” arc, and the “a: red” arc are sequentially passed. , The word “red” is output. When the input symbol “a, k, i” is input, the word “autumn” passes through the arc “a: ε”, the arc “k: ε”, and the arc “i: autumn”. Is output.

  In the conventional dictionary WFST, phonetic symbols such as an arc chain that outputs “red” with “a, k, a” as input and an arc chain that outputs “autumn” with “a, k, i” as input. There are only arcs that convert columns to words. In the dictionary WFST of the present invention, there are arcs that input phonemes and output phonemes, such as three arcs accompanied by “a: a”, “k: k”, and “i: i”. Also, as shown in the top arc chain, for example, a phoneme string such as “k, i” may be input and an arc chain that outputs a subword unit such as “ki” may be used.

  The procedure in which the dictionary WFST creation unit 16 creates the dictionary WFST will be described in more detail. First, the dictionary WFST creation unit 16 reads the phoneme list stored in the phoneme list storage unit 12, and creates an arc that inputs the phoneme for each phoneme and outputs a phoneme or subword. Next, the dictionary WFST creation unit 16 reads the general-purpose word pronunciation dictionary stored in the general-purpose word pronunciation dictionary storage unit 10, and creates an arc for inputting a phoneme string and outputting a word for each word. Then, the dictionary WFST creation unit 16 stores the created dictionary WFST in the dictionary WFST storage unit 24.

  In step S18, the language model WFST creation unit 18 reads the general language model stored in the general language model storage unit 14 and the phoneme list stored in the phoneme list storage unit 12, and sets the general language word output from the dictionary WFST. Gives a weight of the output general word, and creates a language model WFST that gives a weight smaller than any of the weights of the general word to the phoneme or subword output from the dictionary WFST.

An example of the configuration of the language model WFST of the present invention will be described with reference to FIG. As for the symbol associated with the arc, the symbol string in front of “: (colon)” is an input symbol and the symbol string behind is an output symbol, as in the dictionary WFST. “Φ” represents a failure transition when there is no arc corresponding to the input symbol. Further, the symbol string shown at the rear and delimited by “/ (slash)” indicates the weight when passing through the arc. For example, “p (w ₃ | w ₁ w ₂ )” represents the trigram probability of the word string w ₁ w ₂ w ₃ . “Α (w ₁ w ₂ )” represents a back-off coefficient for the bigram w ₁ w ₂ .

The conventional language model WFST has only arcs that input and output words. In addition to this, the language model WFST of the present invention has arcs that input and output phonemes or subword units as unigrams. For example, the arc accompanying “a: a / δ” shown in FIG. 4 is an arc that inputs and outputs the phoneme “a” as a unigram. Here, a very small probability δ is given to the weight. As a result, the unigram of the phoneme “a” is output with a value obtained by multiplying the probability δ by the backoff coefficients α (w ₁ w ₂ ) and α (w ₂ ).

The value of the probability δ is set to a sufficiently small value as compared with the probability value included in the general language model so that the subword unit unigram is pruned by the beam search. In the example of FIG. 4, it is desirable that the trigram probability p (w ₃ | w ₁ w ₂ ) of the word string w ₁ w ₂ w ₃ is 1/100 or less. Actually, these probability values are converted to a logarithmic scale, and the weight of the logarithmic scale associated with the arc is added every time the arc passes during decoding.

  The procedure in which the language model WFST creation unit 18 creates the language model WFST will be described in more detail. First, the language model WFST creation unit 18 reads a general language model stored in the general language model storage unit 14. Subsequently, the language model WFST creation unit 18 reads the phoneme list stored in the phoneme list storage unit 12, assigns a probability δ to each phoneme, and adds it as a unigram entry to the general language model. Based on the general language model to which phoneme entries are added in this way, the language model WFST is constructed so as to conform to the configuration of the language model WFST described above. Then, the language model WFST creation unit 18 stores the created language model WFST in the language model WFST storage unit 26.

  In step S28, the general-purpose WFST creation unit 28 stores the phoneme WFST stored in the phoneme WFST storage unit 20, the acoustic model WFST stored in the acoustic model WFST storage unit 22, and the dictionary WFST storage unit 24. The dictionary WFST and the language model WFST stored in the language model WFST storage unit 26 are read. The general-purpose WFST creation unit 28 creates a general-purpose WFST by synthesizing and optimizing the read phoneme WFST, acoustic model WFST, dictionary WFST, and language model WFST. Then, the general purpose WFST creation unit 28 stores the created general purpose WFST in the general purpose WFST storage unit 30.

  Since the general-purpose WFST of the present invention is constructed using the dictionary WFST and the language model WFST configured as described above, it becomes possible to accept an output in units of phonemes or subwords. In addition, when decoding is performed using general-purpose WFST alone, arcs in units of phonemes or subwords are accompanied by a very small probability δ, so that they are pruned by beam search and are not output as recognition results. .

  In step S36, the word addition WFST creation unit 36 reads the general word list stored in the general word list storage unit 32 and the new word reading list stored in the new word reading list storage unit 34, and stores the general word WFST. When the phoneme string to be output matches any of the phoneme strings representing the reading of the new word, the weight of the matched new word is given and the new word is output, and the output of the general WFST is one of the general words If they match, a word addition WFST that outputs the general-purpose word while maintaining the weight assigned to the general-purpose WFST is created.

With reference to FIG. 5, an example of the configuration of the word addition WFST introduced in the present invention will be described. The word addition WFST includes an arc for inputting / outputting general-purpose words and an arc for inputting a sequence of phonemes or subwords and outputting a new word. Here, the general-purpose words are “red” and “autumn”, and the new words are “Akou” and “akio”. Arcs for inputting / outputting general-purpose words are the two arcs from the top accompanied by “red: red / 1” and “autumn: autumn / 1”. Here, in the arc for inputting / outputting general-purpose words, a weight having a probability of 1 (0 on a logarithmic scale) is given in order not to change the weight. An arc for inputting a phoneme or subword unit sequence and outputting a new word is a chain of two arcs from the bottom. That is, an arc chain that inputs “a, k, o, u” and outputs “Agawa” and an arc chain that outputs “A, k, i, o” and outputs “Akio”. Here, the weight “1 / δ ⁴ ” is given only to the first arc in the arc chain that outputs a new word. This is because if the new word is composed of four phonemes, the fourth power of the very small probability δ is applied as a weight according to the language model WFST described above, so if a value that cancels out this small probability δ is not given, This is because they are pruned and are not output in the recognition result.

The procedure in which the word addition WFST creation unit 36 creates the word addition WFST will be described in more detail. First, the word addition WFST creation unit 36 reads the general-purpose word list stored in the general-purpose word list storage unit 32 and generates an arc that uses the word as an input / output for each word. Next, the word addition WFST creation unit 36 reads the new word reading list stored in the new word reading list storage unit 34, and adds an arc that outputs each new phoneme string as an input. At this time, a weight of probability “1 / δ ^M ” is given to the first arc of the arc chain corresponding to the new word. Here, M represents the number of phonemes of the new word. Then, the word addition WFST creation unit 36 stores the created word addition WFST in the word addition WFST storage unit 38.

<Voice recognition device>
With reference to FIG. 6, an example of a functional configuration of the speech recognition apparatus according to the first embodiment will be described. The speech recognition device 2 of the first embodiment is a device that executes speech recognition processing using the speech recognition WFST created by the speech recognition WFST creation device 1.

  The speech recognition apparatus 2 includes a general-purpose WFST storage unit 30, a word addition WFST storage unit 38, an input unit 50, a speech recognition unit 52, and an output unit 54. The speech recognition device 2 may be configured as the same device as the speech recognition WFST creation device 1 and share the general-purpose WFST storage unit 30 and the word addition WFST storage unit 38.

  The speech recognition device 2 is a special configuration in which a special program is read by a known or dedicated computer having a central processing unit (CPU), a main storage device (Random Access Memory, RAM), and the like. Device. For example, the voice recognition device 2 executes each process under the control of the central processing unit. The data input to the voice recognition device 2 and the data obtained in each process are stored in, for example, the main storage device, and the data stored in the main storage device is read out as necessary and used for other processing. The Each storage unit included in the speech recognition device 2 includes, for example, a main storage device such as a RAM (Random Access Memory), an auxiliary storage device configured by a semiconductor memory element such as a hard disk, an optical disk, or a flash memory, or It can be configured with middleware such as a relational database or key-value store. Each storage unit included in the speech recognition device 2 may be logically divided, and may be stored in one physical storage device.

  The general-purpose WFST storage unit 30 stores in advance the general-purpose WFST created by the speech recognition WFST creation apparatus 1. When the speech recognition device 2 is configured as the same device as the speech recognition WFST creation device 1, the general-purpose WFST created by the speech recognition WFST creation device 1 may be used as it is. When the speech recognition device 2 is configured as a device different from the speech recognition WFST creation device 1, the generic WFST stored in the generic WFST storage unit 30 included in the speech recognition WFST creation device 1 is replicated online or offline, The data is stored in the general-purpose WFST storage unit 30 included in the voice recognition device 2.

  The word addition WFST storage unit 38 stores in advance the word addition WFST created by the speech recognition WFST creation apparatus 1. When the speech recognition device 2 is configured as the same device as the speech recognition WFST creation device 1, the word addition WFST created by the speech recognition WFST creation device 1 may be used as it is. When the speech recognition device 2 is configured as a device different from the speech recognition WFST creation device 1, the word addition WFST stored in the word addition WFST storage unit 38 included in the speech recognition WFST creation device 1 is online or offline. It duplicates and memorize | stores in the WFST memory | storage part 38 for word addition with which the speech recognition apparatus 2 is provided.

  With reference to FIG. 7, an example of the processing flow of the speech recognition method executed by the speech recognition apparatus 2 will be described according to the order of procedures actually performed.

  In step S50, audio data is input to the input unit 50. The input voice data is input to the voice recognition unit 52.

  In step S52, the speech recognition unit 52 reads the general WFST stored in the general WFST storage unit 30 and the word addition WFST stored in the word addition WFST storage unit 38, and adds the word WFST to the general WFST. Using the speech recognition WFST that dynamically synthesizes the WFST, the input speech data is converted into a speech recognition result.

  The voice recognition unit 52 enables decoding by on-the-fly synthesis. On-the-fly synthesis is a method of synthesizing WFST at the time of executing speech recognition processing. By dynamically synthesizing only a necessary part of a search network, it is possible to reduce the amount of memory consumed during recognition. On-the-fly synthesis is particularly effective for language models. When a language model is converted to WFST, a network for two or three large vocabularies is constructed, and therefore the size becomes huge if synthesized in advance. Therefore, memory consumption can be greatly reduced by performing on-the-fly synthesis only for language models. Details on on-the-fly synthesis are described in “Tsubasa Onishi et al.,“ T3 speech recognition decoder based on WFST ”, Journal of Information Processing Society of Japan, Vol. 51, No. 11, 2010”.

  The speech recognition unit 52 sequentially synthesizes the general-purpose WFST and the word addition WFST at the time of decoding using on-the-fly synthesis. Sequential synthesis is performed based on the following equation.

Here, H represents an acoustic model WFST, C represents a phoneme WFST, L represents a dictionary WFST, G represents a language model WFST, and L _p represents a word addition WFST. The ◇ operator represents on-the-fly synthesis, and the ○ operator represents general synthesis processing that is not sequential with optimization processing.

  The language model WFST in the general-purpose WFST can be configured such that only the trigram portion is synthesized on-the-fly. In this case, the sequential synthesis is performed based on the following formula.

Here, G _uni represents a language model WFST composed only of unigrams, and G _{tri / uni} represents a language model WFST to which trigram probabilities are subtracted from unigram probabilities.

  In the on-the-fly synthesis, since it is not necessary to synthesize the general-purpose WFST and the word addition WFST in advance, each can be stored separately. At the time of decoding, only the general-purpose WFST is expanded on the memory, and when a user's voice is input, the word addition WFST is instantaneously loaded onto the memory and synthesized with the general-purpose WFST to execute voice recognition.

  In step S54, the recognition result obtained by the voice recognition unit 52 is output from the output unit 54 to the outside.

  Since a general WFST assumes a large vocabulary model, words unique to the user are limited to a small number of words such as names of family members and friends. Thereby, the size of the word addition WFST specific to the user can be kept small. Therefore, the speech recognition apparatus 2 can instantly load the word addition WFST into the memory.

[Second Embodiment]
In the WFST for speech recognition according to the first embodiment, the probability finally given to a new word is the back-off coefficient value itself. However, a new word is usually given an appearance probability of an unknown word in a general WFST language model in addition to a back-off coefficient. Therefore, the weight is changed so that the unknown word appearance probability is given to the new word in the word addition WFST.

  In the speech recognition WFST creation device and speech recognition device according to the second embodiment, the configuration of the word addition WFST and the operation of the word addition WFST creation unit are the speech recognition WFST creation device and speech recognition device according to the first embodiment. And different.

  An example of the configuration of the word addition WFST according to the second embodiment will be described with reference to FIG. In the word addition WFST according to the second embodiment, the first arc of each new word is multiplied by the appearance probability p (unk) of the unknown word. With this configuration, all new words can be output with an appearance probability of p (unk).

Hereinafter, the process of creating the word addition WFST will be described. First, the word addition WFST creation unit 36 reads the general-purpose word list stored in the general-purpose word list storage unit 32 as in the first embodiment, and for each word, an arc that uses the word as an input / output is read. Generate. Next, the word addition WFST creation unit 36 reads the new word reading list stored in the new word reading list storage unit 34, and adds an arc that outputs each new phoneme string as an input. At this time, a weight of probability “1 / δ ^M * p (unk)” is given to the first arc of the arc chain corresponding to the new word.

  With this configuration, in the word addition WFST of the second embodiment, the unknown word appearance probability p (unk) can be assigned to all new words.

[Third embodiment]
In the first embodiment and the second embodiment, the appearance probabilities assigned to the new words by the word addition WFST are all uniform values. However, since the frequency of utterance differs for each word, the recognition accuracy can be further improved by adjusting the appearance probability according to the frequency. Furthermore, not only new words but also general-purpose words can be improved in recognition accuracy by adjusting the appearance probability for each user. The speech recognition WFST creation device and speech recognition device according to the third embodiment of the present invention are configured so that the appearance probabilities of general-purpose words and new words can be adjusted for each user.

<WFST creation device for voice recognition>
With reference to FIG. 9, an example of a functional configuration of the speech recognition WFST creation apparatus according to the third embodiment will be described. Similar to the speech recognition WFST creation apparatus 1 according to the first embodiment, the speech recognition WFST creation apparatus 3 according to the third embodiment is a general word pronunciation dictionary storage unit 10, a phoneme list storage unit 12, and a general language model storage. Unit 14, dictionary WFST creation unit 16, language model WFST creation unit 18, phoneme WFST storage unit 20, acoustic model WFST storage unit 22, dictionary WFST storage unit 24, language model WFST storage unit 26, general purpose WFST creation unit 28, general purpose WFST A storage unit 30, a general word list storage unit 32, a new word reading list storage unit 34, a word addition WFST creation unit 36 and a word addition WFST storage unit 38; and a personal language model storage unit 40, a personal language model WFST A creation unit 42 and a personal language model WFST storage unit 44 are included.

  The personal language model storage unit 40 stores a personal language model prepared in advance. The configuration of the personal language model is the same as that of the general-purpose language model, and a combination of an N-gram word string, its N-gram probability, and its back-off coefficient is one entry. The personal language model can be created, for example, by calculating a trigram probability using personal transcription data or the like. The word string included in the personal language model is a word included in a general-purpose word or a new word.

  With reference to FIG. 10, an example of the processing flow of the speech recognition WFST creation method executed by the speech recognition WFST creation device 3 will be described in the order of procedures actually performed. Since the process from step S16 to step S36 is the same as that of the speech recognition WFST creation apparatus 1 according to the first embodiment, detailed description thereof is omitted.

  In step S42, the personal language model WFST creation unit 42 reads the personal language model stored in the personal language model storage unit 40 and outputs the personal language model WFST. The method of constructing the personal language model WFST from the personal language model is the same as the process of creating the language model WFST from the general language model. Then, the personal language model WFST creation unit 42 stores the created personal language model WFST in the personal language model WFST storage unit 44.

<Voice recognition device>
With reference to FIG. 11, an example of a functional configuration of the speech recognition apparatus according to the third embodiment will be described. Similar to the speech recognition device 2 according to the first embodiment, the speech recognition device 4 according to the third embodiment includes a general-purpose WFST storage unit 30, a word addition WFST storage unit 38, an input unit 50, a speech recognition unit 52, and an output. A personal language model WFST storage unit 44. Similar to the first embodiment, the speech recognition device 4 is configured as the same device as the speech recognition WFST creation device 3, and includes a general-purpose WFST storage unit 30, a word addition WFST storage unit 38, and a personal language model WFST storage unit 44. It is also possible to adopt a configuration in which

  The speech recognition unit 52 of the third embodiment includes a general-purpose WFST stored in the general-purpose WFST storage unit 30, a word addition WFST stored in the word addition WFST storage unit 38, and a personal language model WFST storage unit 44. The personal language model WFST stored in the memory, and using the speech recognition WFST in which the word addition WFST and the personal language model WFST are dynamically synthesized with respect to the general-purpose WFST, the input speech data is converted into a speech recognition result. Convert to That is, in the speech recognition unit 52 of the first embodiment, the personal language model WFST is synthesized on-the-fly after the speech recognition WFST. Specifically, on-the-fly synthesis is performed based on the following equation.

Here, G _p represents the personal language model WFST.

  As in the first embodiment, the language model WFST can be configured to perform on-the-fly synthesis only on the trigram portion. In this case, on-the-fly synthesis is performed based on the following equation.

  With this configuration, the speech recognition apparatus 4 according to the third embodiment can perform decoding reflecting the word appearance probability for each individual, and can improve speech recognition accuracy.

[Fourth embodiment]
In speech recognition, uttered phonemes are matched with words in the speech recognition WFST in chronological order, and only a word set with a high probability of being matched is left as a recognition hypothesis. Even if it matches the uttered phoneme, it falls out of the recognition hypothesis if the probability is low. In the configuration of the first embodiment, a phenomenon occurs in which a large number of new words remain in the recognition hypothesis during speech recognition, and general-purpose words are lost. Then, many new words appear as recognition errors in the recognition result.

This problem will be specifically described by taking as an example a case where there is a word “red” as a general word and a word “Aga / aka” as a new word. When the speaker utters the phoneme string “a, k, a”, when the first phoneme “a” is input, the general word “red” has not reached the end of the word, so the general word “ Language probabilities for “red” are not given. On the other hand, the new word “Aga / aka” is given a very small probability δ with respect to “a” by the language model WFST, and the word addition WFST cancels the probability corresponding to the number of phonemes for “a: Aga”. Since the weight “1 / δ ³ ” is given, the new word “Aga / aka” is given a weight of δ * 1 / δ ³ = 1 / δ ² . The probability δ can take a value of 0 or more and 1 or less. For example, if the probability δ is small such as less than 0.1, 1 / δ ² is a large value. As a result, when the first phoneme “a” is input, the general word “red” is not given a weight, and the new word “Aga / aka” is given a large weight. Actually, there are many general-purpose words and new words, so there is a possibility that “red” may be removed from the recognition hypothesis. Note that such a problem is unlikely to occur when the probability δ is given a relatively large probability, such as 0.8 or 0.9.

  In order to solve this problem, in the speech recognition WFST creation device and speech recognition device according to the fourth embodiment, the configuration of the word addition WFST and the operation of the word addition WFST creation unit according to the first embodiment The WFST creation apparatus and the speech recognition apparatus are configured differently.

  An example of the configuration of the word addition WFST according to the fourth embodiment will be described with reference to FIG. In the word addition WFST according to the fourth embodiment, the reciprocal number “1 / δ” of the probability δ given to the subword by the language model WFST is given as a weight to each arc forming the arc chain corresponding to each new word. For example, in the new word “Aga / aka”, it is understood that the weight “1 / δ” is assigned to each of the phonemes “a: Aga”, “k: ε”, and “a: ε”. With this configuration, it is possible to avoid giving a large weight to a new word when the first phoneme is input.

  Hereinafter, a process of creating the word addition WFST according to the fourth embodiment will be described. First, the word addition WFST creation unit 36 reads the general-purpose word list stored in the general-purpose word list storage unit 32 as in the first embodiment, and for each word, an arc that uses the word as an input / output is read. Generate. Next, the word addition WFST creation unit 36 reads the new word reading list stored in the new word reading list storage unit 34, and adds an arc that outputs each new phoneme string as an input. At this time, a weight of probability “1 / δ” is given to each arc forming the arc chain corresponding to the new word.

  With this configuration, in the word addition WFST of the fourth embodiment, a large weight is not given to the new word when the first phoneme is input, and the new word appears in the recognition result. Can be avoided.

[Fifth embodiment]
In the second embodiment, an unknown word appearance probability p (unk) is given to a new word when the first phoneme is uttered. However, the general word is given a word probability when all phoneme strings are uttered in the dictionary WFST and the language model WFST. Since the new word is given the unknown word appearance probability p (unk) first, the probability is low and the phenomenon of deviating from the recognition hypothesis occurs.

  In order to solve this problem, in the speech recognition WFST creation device and speech recognition device according to the fifth embodiment, the configuration of the word addition WFST and the operation of the word addition WFST creation unit according to the second embodiment The WFST creation apparatus and the speech recognition apparatus are configured differently.

  An example of the configuration of the word addition WFST according to the fifth embodiment will be described with reference to FIG. In the word addition WFST according to the fifth embodiment, the weight of the unknown word appearance probability “p (unk)” is given to the last arc of the arc chain corresponding to each new word. For example, in the new word “Akawa / akou”, it can be seen that the weight “p (unk)” is given to the last phoneme “u: ε”. With this configuration, the word probabilities are given simultaneously for the new word and the general-purpose word.

  Hereinafter, a process of creating the word addition WFST according to the fifth embodiment will be described. First, the word addition WFST creation unit 36 reads the general-purpose word list stored in the general-purpose word list storage unit 32 in the same manner as in the second embodiment, and for each word, an arc that uses the word as an input / output is read. Generate. Next, the word addition WFST creation unit 36 reads the new word reading list stored in the new word reading list storage unit 34, and adds an arc that outputs each new phoneme string as an input. At this time, the weight of the unknown word appearance probability “p (unk)” is given to the last arc of the arc chain corresponding to the new word.

  With this configuration, in the word addition WFST of the fifth embodiment, the timing at which the word probabilities are given to the new word and the general-purpose word is the same, and the phenomenon that the new word deviates from the recognition hypothesis can be prevented. .

[Sixth embodiment]
The weighted finite state transducer may be subjected to determinizing / minimizing processing and weight pushing processing for the purpose of improving processing efficiency. FIG. 14 illustrates an example of the decision / minimization process and the pushing process. In FIG. 14, the arc chain located in the upper stage is an unprocessed WFST, the arc chain located in the middle stage is a result of determinizing / minimizing processing, and the arc chain located in the lower stage is further subjected to pushing processing. It is a result. The pushing process is a process of shifting the weight assigned to the subsequent arc to the previous arc when the common arcs between words are collected by the determinizing / minimizing process. It is assumed that there are two words “red / aka” and “autumn / aki”, “red / aka” has a weight of 0.8, and “autumn / aki” has a weight of 0.6. Since “red / aka” and “autumn / aki” have the same first two letters “ak”, common parts can be combined into one arc by determinizing and minimizing processing. A common value for the weight assigned to each word is equivalent to the first arc, even if it is assigned to the first arc, so 0.6 is assigned to the first phoneme “a: ε”. The weight of “/ aka” can be 0.2, and the weight of “autumn / aki” can be 0. The pushing process is a process for pushing out a common weight before determinizing and minimizing a plurality of words.

  In the second embodiment or the fifth embodiment, the unknown word appearance probability p (unk) is assigned to the first or last arc of a new word. However, when the dictionary WFST and the language model WFST are subjected to the determinizing / minimizing process and the weight pushing process, the word probability for the general word is given to the arc of each phoneme of the general word. For the dictionary WFST and the language model WFST, the search is given in a distributed manner. In the fifth embodiment, the probabilities assigned to new words are concentrated in the last arc, but the probabilities of general-purpose words are actually distributed for each phoneme. As a result, the probability is lowered, and as a result, new words are generated.

  In order to solve this problem, in the speech recognition WFST creation device and speech recognition device according to the sixth embodiment, the configuration of the word addition WFST and the operation of the word addition WFST creation unit are the speech recognition according to the fifth embodiment. The WFST creation apparatus and the speech recognition apparatus are configured differently.

An example of the configuration of the word addition WFST according to the sixth embodiment will be described with reference to FIG. In the word addition WFST according to the sixth embodiment, a value obtained by dividing the appearance probability p (unk) of an unknown word by a phoneme number is assigned to each arc forming an arc chain corresponding to each new word as a weight. For example, in the new word “Akawa / akou”, the first phoneme “a: Agawa” is given the weight “1 / δ ⁴ × p (unk) / 4”, and the subsequent phonemes “k: ε” “o: ε” It can be seen that the weight “p (unk) / 4” is given to “u: ε”. With this configuration, the timing at which word probabilities are given does not differ greatly between general-purpose words and new words.

  Hereinafter, the process of creating the word addition WFST according to the sixth embodiment will be described. First, the word addition WFST creation unit 36 reads the general-purpose word list stored in the general-purpose word list storage unit 32 in the same manner as in the second embodiment, and for each word, an arc that uses the word as an input / output is read. Generate. Next, the word addition WFST creation unit 36 reads the new word reading list stored in the new word reading list storage unit 34, and adds an arc that outputs each new phoneme string as an input. At this time, a value “p (unk) / M” obtained by dividing the appearance probability p (unk) of the unknown word by the number of phonemes is assigned as a weight to each arc forming the arc chain corresponding to the new word. Here, M represents the number of phonemes of the new word.

  With this configuration, in the word addition WFST of the sixth embodiment, the timing at which word probabilities are given does not differ greatly between general-purpose words and new words, and the occurrence of new words can be prevented.

[Combination of Embodiments]
The configuration of the fourth embodiment can be combined with the configurations of the second embodiment, the fifth embodiment, and the sixth embodiment.

  When the configuration of the fourth embodiment is combined with the second embodiment, the reciprocal “1 / δ” of the probability δ is assigned as a weight to each arc forming the arc chain corresponding to each new word, and further unknown to the first arc. What is necessary is just to give the weight of the word appearance probability “p (unk)”. For example, in the new word “Akawa / akou”, a weight “p (unk) / δ” is assigned to the first phoneme “a: Agawa”, and the remaining phonemes “k: ε”, “o: ε”, “u: ε” Is assigned a weight “1 / δ”.

  When the configuration of the fourth embodiment is combined with the fifth embodiment, the reciprocal “1 / δ” of the probability δ is assigned as a weight to each arc forming the arc chain corresponding to each new word, and the unknown is further unknown to the last arc. What is necessary is just to give the weight of the word appearance probability “p (unk)”. For example, in the new word “Akawa / akou”, the first triphone “a: Agawa” “k: ε” “o: ε” is given a weight “1 / δ” and the last phoneme “u: ε” A weight “p (unk) / δ” is assigned.

  When combining the configuration of the fourth embodiment with the sixth embodiment, each arc forming the arc chain corresponding to each new word is multiplied by the reciprocal of probability δ and the occurrence probability p (unk) of the unknown word and divided by the number of phonemes. What is necessary is just to give a value as a weight. For example, in the new word “Akawa / akou”, a weight “p (unk) / 4δ” is assigned to all phonemes “a: Akawa”, “k: ε”, “o: ε”, and “u: ε”.

[Program, recording medium]
The present invention is not limited to the above-described embodiment, and it goes without saying that modifications can be made as appropriate without departing from the spirit of the present invention. The various processes described in the above-described embodiments are not only executed in time series according to the order described, but may be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes.

  When various processing functions in each device described in the above embodiment are realized by a computer, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

DESCRIPTION OF SYMBOLS 1,3 Speech recognition WFST creation device 2, 4 Speech recognition device 10 General-purpose word pronunciation dictionary storage unit 12 Phoneme list storage unit 14 General-purpose language model storage unit 16 Dictionary WFST creation unit 18 Language model WFST creation unit 20 Phoneme WFST storage unit 22 Acoustic model WFST storage unit 24 Dictionary WFST storage unit 26 Language model WFST storage unit 28 General-purpose WFST creation unit 30 General-purpose WFST storage unit 32 General-purpose word list storage unit 34 New word reading list storage unit 36 Word addition WFST creation unit 38 Word addition WFST storage unit 40 Personal language model storage unit 42 Personal language model WFST creation unit 44 Personal language model WFST storage unit 50 Input unit 52 Speech recognition unit 54 Output unit

Claims (11)

A dictionary WFST creation unit for creating a dictionary WFST that takes a phoneme string as an input and outputs a subword based on the phoneme string and a phoneme string representing a reading out of predetermined generic words that matches the phoneme string;
A language for creating a language model WFST in which a weight of the general word is given to a general word output from the dictionary WFST, and a weight δ smaller than any of the weights of the general word is given to a subword output from the dictionary WFST A model WFST creation unit;
An acoustic model WFST that converts an input audio signal into an acoustic environment, a phoneme WFST that converts the acoustic environment into a phoneme, a generic WFST creation unit that creates a generic WFST using the dictionary WFST and the language model WFST;
When the output of the general WFST matches any phoneme string representing the reading of the new word, the new word is output with the weight of the new word matched, and the output of the general WFST is the output of the general word. A word addition WFST creation unit that creates a word addition WFST that outputs the generic word while maintaining the weight of the matched generic word if it matches any of the above;
WFST creation device for speech recognition. The speech recognition WFST creation apparatus according to claim 1,
The word addition WFST creation unit assigns a weight obtained by multiplying the weight of the matched new word by the appearance probability of an unknown word in the language model WFST when the output of the general-purpose WFST matches the new word. A speech recognition WFST creation device for creating an additional WFST. The speech recognition WFST creation apparatus according to claim 2,
The word addition WFST creation unit creates a word addition WFST that gives the appearance probability of the unknown word as a weight to the last phoneme of the phoneme string representing the reading of the new word. apparatus. The speech recognition WFST creation apparatus according to claim 2,
The word addition WFST creation unit assigns, as a weight, a value obtained by dividing the appearance probability of the unknown word by the number of phonemes in the phoneme string to each phoneme forming a phoneme string representing the reading of the new word. WFST creation device for voice recognition. The speech recognition WFST creation apparatus according to any one of claims 1 to 4,
The word addition WFST creation unit creates a word addition WFST that assigns the reciprocal of the weight δ as a weight to each phoneme forming a phoneme string representing the reading of the new word. . A speech recognition WFST creation apparatus according to any one of claims 1 to 5,
A personal language model WFST creating unit for creating a personal language model WFST that gives a weight different from the weight output by the word adding WFST to at least one word included in the general word or the new word;
A WFST creation device for speech recognition further comprising: A general-purpose WFST storage unit that stores the general-purpose WFST created by the speech recognition WFST creation device according to claim 1;
A word addition WFST storage unit for storing the word addition WFST created by the speech recognition WFST creation device according to claim 1;
A speech recognition unit that converts an input speech signal into a speech recognition result using a speech recognition WFST in which the word addition WFST is dynamically synthesized with respect to the general-purpose WFST;
A speech recognition device. The speech recognition device according to claim 7,
A personal language model storage unit for storing a personal language model WFST created by the speech recognition WFST creation device according to claim 6;
The voice recognition unit converts the voice signal into the voice recognition result using a voice recognition WFST obtained by dynamically synthesizing the word addition WFST and the personal language model WFST with respect to the general-purpose WFST. Voice recognition device. A dictionary WFST is a dictionary WFST that takes a phoneme string as an input and creates a dictionary WFST that outputs a subword based on the phoneme string and a generic word in which a phoneme string representing a reading out of predetermined generic words matches the phoneme string Creation steps,
The language model WFST creation unit assigns a weight of the general word to the general word output from the dictionary WFST, and assigns a weight δ smaller than any of the weights of the general word to the subword output from the dictionary WFST. A language model WFST creation step for creating a language model WFST;
A general-purpose WFST creation unit creates a general-purpose WFST using an acoustic model WFST that converts an input audio signal into an acoustic environment, a phoneme WFST that converts the acoustic environment into a phoneme, the dictionary WFST, and the language model WFST. A WFST creation step;
The word addition WFST creation unit assigns a weight of the matched new word and outputs the new word when the output of the general-purpose WFST matches any of the phoneme strings representing the reading of the new word. A word addition WFST creation step for creating a word addition WFST that outputs the generic word while maintaining the weight of the matched generic word when the output of the WFST matches any of the generic words;
WFST creation method for speech recognition including The general-purpose WFST created by the speech recognition WFST creation method according to claim 9 is stored in the general-purpose WFST storage unit,
The word addition WFST created by the speech recognition WFST creation method according to claim 9 is stored in the word addition WFST storage unit,
A speech recognition step in which a speech recognition unit converts an input speech signal into a speech recognition result using a speech recognition WFST in which the word addition WFST is dynamically synthesized with respect to the general-purpose WFST;
A speech recognition method including:   A program for causing a computer to function as the speech recognition WFST creation device according to any one of claims 1 to 6 or the speech recognition device according to claim 7 or 8.

Images