JP4595415B2 - Voice search system, method and program - Google Patents

Description

  The present invention relates to a voice search system, method, and program, and more particularly, to a system, method, and program for searching for words and phrases in voice data that has been voice-recognized.

  Conventionally, this type of voice search system is used to search for keywords from voice data such as news and lectures and to access desired contents. In this case, although speech recognition is used, unknown words that are not registered in the dictionary cannot be recognized, and erroneous recognition is unavoidable even for known words. Non-patent document 1 describes an example of a speech search system for searching for words and phrases from speech recognition results including such misrecognition.

  FIG. 12 is a block diagram showing the configuration of the voice search system described in Non-Patent Document 1. In FIG. 12, the speech search system includes a speech data storage unit 101, a continuous phoneme recognition unit 102, a recognition result phoneme sequence storage unit 103, a search character string input unit 104, a phoneme conversion unit 105, and a matching unit 106. , And search result output means 107. The conventional voice search system having such a configuration operates as follows.

  The voice data storage unit 101 stores voice data to be searched. The continuous phoneme recognition unit 102 performs continuous phoneme recognition on the speech data, and stores the recognition result phoneme string in the recognition result phoneme string storage unit 103. On the other hand, the search character string input from the search character string input means 104 is converted into a phoneme string by the phoneme conversion means 105. The matching means 106 collates the phoneme converted search character string with the entire recognition result phoneme string by DP (Dynamic Programming) matching in phonemes, and the search character string exists in a section where the cumulative distance is smaller than a predetermined threshold. Judge that. The search result output means 107 outputs a section of voice data determined to have a search character string. By performing DP matching on a phoneme basis, it is possible to search to some extent even if a character string to be searched is erroneously recognized.

  Note that a frame-synchronized beam search used as one of continuous word speech recognition algorithms is described in Patent Document 1. Non-patent document 2 describes various search techniques in speech recognition.

Japanese Patent No. 3346285 Ryuichi Oka et al., "Speech and text retrieval using phoneme sequence representation", IEICE technical report, 2001, SP2001-29, 29-35 Lawrence Rabiner et al., “Translation by Sadahiro Furui, Basics of Speech Recognition (below)”, NTT Advanced Technology Corporation, 1995, pp. 194-229

  The problem with the prior art is that the amount of calculation required for the search is large. As a result, when the scale of the speech data to be searched increases, it will take a long time to enter the search string until the search result is actually obtained, which reduces the practicality of the voice search system. End up. The reason why the amount of calculation required in the conventional technique is large is that distance calculation of a plurality of paths is executed in units of phonemes in DP matching between a phoneme converted search character string and a recognition result phoneme string.

  An object of the present invention is to provide a voice search system, method, and program that can cope with a case where a character string to be searched is erroneously recognized or is an unknown word and can be searched at high speed.

  The principle of the present invention that achieves the above-mentioned object is to search after an input search character string is expanded into similar words or similar word strings that can appear in the speech recognition result of the speech data to be searched.

A speech search system according to one aspect of the present invention includes a recognition result word string storage unit that stores a word-unit speech recognition result of speech data to be searched as a recognition result word string, and words that can appear in the speech recognition result. A candidate word string comprising a candidate word or a combination of candidate words stored in the word candidate storage means , and converting the input search character string into a phoneme string. A search character string expanding means for converting the search character string into candidate words or candidate word strings based on the degree of matching between the phonemes included in the phoneme strings, and a candidate word expanded by the search character string expanding means Search means for searching for candidate word strings from the recognition result word string storage means.

  In the voice search system of the first development form, it is preferable that the word candidate storage unit is configured to store a recognition vocabulary used when obtaining a recognition result word string from the voice data.

  In the voice search system of the second development form, the speech search system includes a word extraction unit that extracts a list of words appearing in the recognition result stored in the recognition result word string storage unit, and the word candidate storage unit stores the list. It is preferable.

  In the voice search system according to the third development mode, when the word extracting unit extracts a list, the words that appear before and after each word in the recognition result are checked, and the connection between words that allows connection only to the words that appear before and after A table is created, the word candidate storage means stores the connection table together with the list, and the search character string expansion means refers to the list stored in the word candidate storage means and the connection table, and candidates that can connect the search character strings It is preferable to include a function that expands only to words or candidate word strings.

In the speech search system of the fourth expansion form, the search character string expansion means includes a function of expanding a word by a continuous word speech recognition algorithm using a phoneme string of the search character string as an input feature vector series and a candidate word as a recognition vocabulary. Preferably, it is configured.

In the fifth search form speech search system, when the search character string expansion means expands the search character string using the candidate words stored in the word candidate storage means , the search is performed based on the degree of matching between phonemes. It is preferable to include a function of obtaining a distance between the character string and the candidate word or the candidate word string and developing the distance so that the distance is within a predetermined threshold .

The speech expansion system according to the sixth expansion mode includes confusion matrix storage means for storing a confusion matrix representing a tendency of recognition errors of phonemes , and the search character string expansion means determines the degree of matching between phonemes based on the confusion matrix. It is preferable to be configured including the desired function.

In the speech search system of the seventh development form, the search character string development means obtains the degree of matching between phonemes based on the inter-model distance in the acoustic model used when obtaining the recognition result word string from the speech data It is preferable that it is comprised including.

In the voice search system according to the eighth development form, words or words that are difficult to appear linguistically by referring to the language model used when the search character string development means obtains the recognition result word string from the voice data in addition to the distance. It is preferable to include a distance adding function for further increasing the distance from the column.

The speech search method according to the second aspect of the present invention stores, as candidate words , input means, expansion means, search means, and words that can appear in speech recognition results in word units of speech data to be searched. speech retrieval system comprising a storage means is a method for searching a word string. In this method, the input means inputs the search character string, and the expansion means converts the candidate word string consisting of candidate words or combinations of candidate words in the storage means into a phoneme string, and converts the search character string into a phoneme string. Converting and expanding the search character string into candidate words or candidate word strings based on the degree of matching between the phonemes included in each phoneme string, and the search means representing the speech recognition result as a recognition result word string and expanded Searching the candidate word or candidate word string from the recognition result word string.

In the audio retrieval method according to the first deployed configuration, the speech retrieval system further comprises a voice recognition unit, the speech recognition means, prior to the step of inputting a search string, performs speech recognition on the speech data, a recognition result As a candidate word.

In the voice search method of the second development mode, the voice search system further includes voice recognition means, and the voice recognition means performs voice recognition on the voice data prior to the step of inputting the search character string, and the recognition result. The step of developing a search character string may be expanded with reference to the generated list.

Voice search method according to a third aspect of the present invention, the search and development unit, a search unit, a voice recognition unit, a recognition result a word sequence storage unit, the speech retrieval system and a word candidate storage unit the word sequence It is a method to do. In this method, the speech recognition means stores the speech recognition result in units of words of the speech data to be searched as a recognition result word string in the recognition result word string storage means, and words that can appear in the recognition result as candidate words word candidate storage means stores the phoneme deployment means converts the entered search string phoneme string, the candidate word sequence consisting candidate word or candidate combination of words stored in the word candidate storage unit And converting the search character string into candidate words or candidate word strings based on the degree of matching between the phonemes included in the phoneme strings, and the search means Retrieving from the recognition result word string storage means.

In the third speech retrieval method deployed configuration of the recognition vocabulary in obtaining recognition result word strings from the speech data word candidate storage means may store.

In the fourth search form voice search method, the voice search system further includes a word extraction unit, and the word extraction unit extracts a list of words appearing in the recognition result stored in the recognition result word string storage unit; further comprising a step of storing a list of words that word extraction means is extracted in the word candidate storing means, the step of deploying the candidate word sequence may be deployed with reference to stored list.

A program according to a fourth aspect of the present invention recognizes a speech recognition result in units of words of speech data to be searched by a computer constituting a speech search system including a recognition result word string storage unit and a word candidate storage unit. A process for storing the result word string in the recognition result word string storage unit, a process for storing a word that can appear in the recognition result in the word candidate storage unit as a candidate word, and converting the input search character string into a phoneme string, A candidate word string composed of a candidate word or a combination of candidate words stored in the word candidate storage means is converted into a phoneme string, and the search character string is converted into a candidate word or a search string based on the degree of matching between the phonemes included in each phoneme string a search string expansion process of expanding the candidate word sequence, the recognition result word string stores the candidate words or candidate word string deployed by the search string expansion means A search process of searching from the stage, to the execution.

  In the program of the first development form, a process of storing the recognition vocabulary for obtaining the recognition result word string from the speech data in the word candidate storage unit may be further executed.

  In the program of the second development form, a word extraction process for extracting a list of words appearing in the recognition result stored in the recognition result word string storage unit, and a word list extracted by the word extraction process in the word candidate storage unit A search character string expansion process may be performed such that the stored character string is expanded with reference to the stored list.

A program according to a fifth aspect of the present invention appears in a speech recognition result in units of words of speech data to be searched on a computer constituting a speech search system including a recognition result word string storage unit and a word candidate storage unit. A candidate word string consisting of a candidate word or a combination of candidate words in a word candidate storage means in which possible words are stored in advance is converted into a phoneme string, and an input search character string is converted into a phoneme string, A search character string expansion process for expanding a search character string into a candidate word or a candidate word string based on the degree of matching of phonemes included in the word, and a recognition result word string in which a speech recognition result is stored in advance as a recognition result word string And a search process for searching for candidate words or candidate word strings developed by the search character string development process from the storage means.

  In the program of the third development form, voice recognition may be performed on voice data, and a process of obtaining candidate words as a recognition result may be further executed.

  According to the present invention, it is possible to cope with a case where a character string to be searched is misrecognized or an unknown word, and search can be performed at high speed. The reason is that it is only necessary to search the recognition result of the word unit which is a larger unit, not the recognition result of the phoneme unit. As a result, the search space is reduced, and the search speed is improved. That is, after the input search character string is expanded into a word or a word string, a recognition result in units of words is searched.

  Another reason is that a search can be made to some extent when a character string to be searched is erroneously recognized without calculating the distance between the search character string and the recognition result by DP matching or the like. That is, the search is performed after previously expanding into a word or a word string having a high possibility as a recognition result of the input search character string.

  Yet another reason is that no word or word string that has no possibility of appearing in the recognition result is searched. That is, even when an unknown word is input as a search character string, the search is not performed wastefully, and the input search character string is expanded into words or word strings that can appear in the recognition result.

  Next, the best mode for carrying out the present invention will be described in detail with reference to the drawings. The voice search system according to the embodiment of the present invention can appear in a recognition result word string storage unit (13 in FIG. 1) that stores a voice recognition result in units of words of voice data to be searched, and a recognition result word string. Word candidate storage means (18 in FIG. 1) for storing word candidates and words or words whose acoustic distance is close to the search character string using the words stored in the word candidate storage means A search character string expanding means (15 in FIG. 1) that expands into a column and a search means (16 in FIG. 1) that searches the expanded word or word string from the recognition result word string.

  The voice search system configured as described above causes a character string to be searched to be erroneously recognized by searching the recognition result word string for a word or word string that is likely to be a recognition result of the input search character string. It is possible to cope with a case where the word is unknown or unknown, and the search can be performed at high speed.

  Hereinafter, the voice search system will be described in more detail in accordance with an embodiment.

  FIG. 1 is a block diagram showing the configuration of the voice search system according to the first embodiment of the present invention. In FIG. 1, the speech search system includes speech data storage means 11 for storing speech data to be searched, continuous word recognition means 12 for recognizing speech data as continuous words, and recognition result words for storing results of continuous word recognition. A word storage means 13, a word candidate storage means 18 for storing word candidates that can appear in the recognition result word string, a search character string input means 14 for inputting a character string to be searched, and the input search character string as a word Alternatively, a search character string expanding means 15 that expands into a word string, a search means 16 that searches the expanded word or word string from the recognition result word string, and a search result output means 17 that outputs the search result are configured. Has been.

  The word candidate storage unit 18 stores a recognition dictionary that defines the recognition vocabulary of the continuous word recognition unit 12. By doing in this way, the word candidate memorize | stored in the word candidate memory | storage means 18 turns into a word which can appear in a recognition result. The word candidate storage means 18 does not necessarily need to store the recognition dictionary itself. For example, the word candidate storage means includes a word list obtained by removing short words such as particles “ha” and “ga” from the recognition dictionary. 18 may be stored. Excluding short words can prevent the number of expanded word strings from becoming enormous.

  Next, the operation of the voice search system according to the first embodiment of the present invention will be described in detail with reference to FIG. 1 and FIG. FIG. 2 is a flowchart showing the operation of the voice search system according to the first embodiment of the present invention.

  First, the continuous word recognition unit 12 reads out the speech data to be searched from the speech data storage unit 11 to perform continuous word recognition, and stores the recognition result in the recognition result word string storage unit 13 in advance. Continuous word recognition is realized by a search using an acoustic model based on HMM (Hidden Markov Model) in units of phonemes, which is a general speech recognition technique, and a language model based on n-gram based on statistical probability between n words. . It is also possible to use syllables and subwords equivalent to them as acoustic model units, and context-free grammar as language models. In this embodiment, the case where speech data stored in the speech data storage means 11 is recognized in advance and a recognition result word string to be searched is generated will be described. This is the state of the speech data in the present invention. It is not intended to limit. For example, it is also possible to perform a voice search in real time by generating recognition data for voice input from a microphone or the like by the successive word recognition means 12 sequentially.

  In this embodiment, the continuous word recognizing unit 12 outputs the recognition result itself of the continuous word recognition to the recognition result word string storage unit 13, but the continuous word recognizing unit 12 does not necessarily recognize the recognition result of continuous word recognition. Need not be output as is. For example, each word of a word string obtained by continuous word recognition may be further divided into short units, or words may be combined into short units, or words may be combined into long units. For example, the long unit “non-scientific” is changed to a short unit word such as “non”, “science” and “target”. At this time, the word candidates stored in the word candidate storage unit 18 are similarly converted into words having a short unit and a long unit, and the types of words appearing in the recognition result word string storage unit 13 are set. It is desirable to match. In addition, the present embodiment includes a result of morphological analysis performed on the result of continuous word recognition as a recognition result word string. In this case, the word candidates stored in the word candidate storage means 18 are preferably words included in a dictionary that defines the vocabulary of the morphological analyzer.

  When a user searches for a word or a word string from voice data, first, a character string to be searched is input from the search character string input means 14 such as a keyboard (step A1). Note that the input of the search character string is not limited to the character string input from a keyboard or the like, and the voice input from a microphone or the like may be recognized. For example, the result of continuous phoneme recognition of speech input by a microphone or the result of isolated word recognition is used as a search character string.

  Next, the search character string expansion unit 15 expands the search character string into a word or a word string using the word candidates stored in the word candidate storage unit 18 (step A2). This expansion is performed so that the acoustic distance between the search character string and the word or word string to be expanded is close. For example, consider a case where “Harry Potter” is input as a search character string. Further, word candidates such as “Harry”, “Poster”, “Ha”, “Reporter”, and the like are stored in the word candidate storage unit 18. At this time, the search character string expansion means 15 arranges the word candidates in the word candidate storage means 18 so that “Harry” + “poster”, “ha” + “has acoustically close to the search character string“ Harry Potter ”. Expands to a word string such as "Reporter". Of course, if “Harry Potter” exists in the word candidate storage means 18, the word “Harry Potter” is also expanded. Since this expansion is performed so that the acoustic distance is close to the search character string, a word string that is highly likely to be a recognition result of the search character string is obtained including erroneous recognition. That is, when “Harry Potter” is recognized, a word string such as “Harry” + “poster” or “ha” + “reporter” is likely to be obtained as a recognition result. At this time, since the word candidate storage means 18 is the recognition vocabulary of the continuous word recognition means 12, it is not expanded into an irrelevant word string that cannot appear in the recognition result. Below, how the search character string expansion | deployment means 15 expand | deploys the word or word string with an acoustic distance close | similar to the input search character string is demonstrated.

  First, the search character string expansion means 15 converts the input search character string into a phoneme string. In order to convert a search character string into a phoneme string, reading information of the search character string is required, but this may be automatically read from a search character string input mixed with kana-kanji, or the search character string The user may input the reading of the search character string by the column input means 14. However, when the search character string is obtained by voice recognition from a microphone or the like, the phoneme string of the search character string can also be obtained by voice recognition. Therefore, the search character string expansion means 15 is described here. There is no need to perform phoneme conversion. On the other hand, the word candidate storage means 18 also stores information on the phoneme string of each word candidate.

  The search character string expansion unit 15 obtains an acoustic distance between the search character string and various arrangements of word candidates stored in the word candidate storage unit 18 by DP matching in units of phonemes, and the distance is determined in advance. Expands to a word string that falls within the specified threshold. In this embodiment, the distance is calculated in units of phonemes, but the distance may be calculated in units of syllables or subwords equivalent thereto. In that case, it is necessary to convert the search character string or word candidate into a syllable string or subword string instead of the phoneme string.

  FIG. 3 shows a distance measure between phonemes, a distance between the same phonemes is 0, a distance between different phonemes is 1, and a phoneme string “ This is the state when the distance between “haripoQtaa (Harry Potter)” (Q represents a prompting sound) and the phoneme string “harii” + “bokusaa (boxer)” of the expanded word string is obtained by DP matching. In this example, the accumulated distance between “haripoQtaa (Harry Potter)” and “harii (Harry)” + “bokusaa (boxer)” is 4. On the other hand, FIG. 4 shows a case where the distance between the phoneme string “haripoQtaa (Harry Potter)” of the search character string and the phoneme string “harii” + “postua” (poster) of the expanded word string is obtained by DP matching. In this case, the cumulative distance between the two is 2. If the distance threshold for determining whether or not to expand is now 3, the search character string “Harry Potter” is expanded into “Harry” + “Poster” by the search character string expanding means 15, but “Harry”. + "Boxer" will not be expanded. It should be noted that the distance threshold for determining whether or not to expand is not always a constant value, and for example, the threshold may be normalized according to the length of the search character string.

  Thus, efficiently obtaining a word string that is close to the search character string can be realized at high speed by a search algorithm used in continuous word speech recognition. Continuous word speech recognition is a problem of obtaining a word string close to a time series of input feature vectors. By comparing the input feature vector with various combinations of words in the word dictionary by the continuous word speech recognition algorithm, a word string that becomes a recognition result can be obtained. Here, the time series of feature vectors in continuous word speech recognition is obtained by analyzing an input speech waveform for each time unit called a frame.

  On the other hand, search character string expansion in the present invention is a problem of obtaining a word string close to the input search character string. This can be realized by inputting a phoneme string of a search character string as an input feature vector in the continuous word speech recognition algorithm. The distance between phonemes may be used to calculate the score when collating the input search character string with various combinations of words. In this embodiment, a phoneme string is used as an input, but a syllable string or a subword string corresponding to the syllable string may be used as an input.

  Hereinafter, a frame-synchronized beam search as described in Patent Document 1 will be described as one of continuous word speech recognition algorithms that are often used. The frame-synchronized beam search develops a word string as a recognition result candidate for each frame as a hypothesis, while deleting hypotheses whose score is equal to or less than a threshold value, so that the time series of input feature vectors and the word string hypothesis Is an algorithm that efficiently performs the collation. Specifically, the following steps 1 to 3 are repeated.

  Step 1: The hypothesis of the I-th frame is expanded to the I + 1-th frame. That is, if the hypothesis of the I-th frame is in the word end state, the word in the dictionary is connected and the hypothesis is developed. The hypothesis for the I-th frame is erased and only the hypothesis for the I + 1-th frame is stored.

  Step 2: Among hypotheses developed in the (I + 1) th frame, only hypotheses whose scores are better than a certain threshold value are stored, and other hypotheses are deleted. This is called beam pruning.

  Step 3: Add 1 to the frame number I to be processed.

  By replacing the input feature vector of the frame-synchronized beam search with the input search phoneme string, the search character string can be expanded in the present invention. The processing in units of frames may be processing in units of phonemes, and the score may be the cumulative interphoneme distance between the input search phoneme sequence and the expanded word sequence hypothesis. Moreover, the branch hunting process of step 2 should just erase | eliminate the hypothesis in which the accumulated distance became more than a fixed value, for example. Alternatively, a hypothesis having a cumulative distance equal to or greater than a certain threshold than that distance may be deleted on the basis of the hypothesis having the smallest cumulative distance among the developed hypotheses. In this way, it is possible to efficiently obtain word strings that are close to the input search character string. It is also possible to control the number of word strings obtained by adjusting the threshold for branch picking.

  The search character string expansion algorithm need not be limited to the frame-synchronized beam search, and a technique generally used as a continuous word speech recognition algorithm can also be applied. For example, it is possible to perform development by a two-stage DP matching, a level building method, a one-stage method, or the like. Details of these algorithms are described in Non-Patent Document 2.

  In this embodiment, the distance between the same phonemes is 0 and the distance between different phonemes is 1 as the distance between phonemes, but another distance scale may be used. For example, the distance between phonemes may be calculated based on the interphoneme confusion matrix. The inter-phoneme confusion matrix is obtained by obtaining in advance a recognition experiment or the like what kind of phoneme each phoneme is likely to be recognized in speech recognition, and expressing the elements of the matrix by probability. An example of this interphoneme confusion matrix is shown in FIG. FIG. 5 is a matrix showing the probabilities that input phonemes k, g, s, z, a,... Are recognized as k, g, s, z, a,. For example, the probability that the phoneme k is recognized as k is 0.6, the probability that it is recognized as g is 0.3, and the probability that it is recognized as z is 0.1. At this time, for example, the reciprocal of the probability in the phoneme confusion matrix can be defined as the interphoneme distance. By defining the distance in this way, it is possible to calculate the distance in consideration of the error tendency in speech recognition. In this case, the distance between phonemes having a probability of 0 is set to a sufficiently large value.

なお、f (x|u, Σ)は、平均ベクトルu、分散共分散行列ΣのK次元ガウス分布であって、（２）式で表される。

Moreover, you may use the distance between models of the acoustic model which the continuous word recognition means 12 used at the time of recognition as another distance scale. For example, the distance between phonemes can be defined by a KL (Kullback-Leibler) distance between probability distributions of an acoustic model representing each phoneme. When the acoustic model of each phoneme is modeled with one state and a single Gaussian distribution, the KL distance between the two models is expressed by equation (1).

Note that f (x | u, Σ) is a K-dimensional Gaussian distribution of an average vector u and a variance-covariance matrix Σ, and is expressed by Equation (2).

  Furthermore, when the acoustic model is represented by a plurality of states, or when the state is represented by a mixed Gaussian distribution, for example, the distance between the Gaussian distributions with the closest distance may be set as the interphoneme distance.

  Further, when the search character string expanding means 15 expands the search character string into words or word strings, it is possible to add not only acoustic proximity but also linguistic restrictions. For example, when calculating the distance between the search character string and the word candidate to be developed, a word model having a low unigram probability may be penalized by referring to the language model used by the continuous word recognition unit 12 at the time of recognition. Also, a penalty may be given when expanding to a word chain with a low bigram probability.

  For example, the penalty between words as shown in FIG. 6 is obtained by multiplying the inverse of the bigram probability of the language model by a constant. At this time, the distance between the phoneme string “haripoQtaa (Harry Potter)” of the search character string and the phoneme string “harii” + “postua (poster)” of the expanded word string obtained in FIG. It is corrected to 4 by adding penalty 2 to distance 2 of. On the other hand, as shown in FIG. 7, the distance between phoneme strings between “haripoQtaa (Harry Potter)” and the phoneme string “wa (ha)” + “ripotaa (reporter)” of the expanded word string is 3, but the penalty is 0. .5 so the final distance is corrected to 3.5. As a result, it is determined that “ha” + “reporter”, which is more likely to be a linguistic recognition result, is closer in distance.

  The same applies to higher-order n-grams. By adding such a penalty when calculating the distance, it is possible to expand the search character string only to words or word strings that tend to appear in the recognition result.

  As described above, the input search character string is expanded into a word or a word string stored in the word candidate storage unit 18 by the search character string expansion unit 15. At this time, the expanded word or word string is a word or word string that is highly likely to be obtained as a result of recognizing the search character string.

  Finally, the search means 16 checks whether the word or word string developed by the search character string development means 15 exists in the recognition result word string storage means 13 (steps A3 and A4). If the expanded word or word string exists in the recognition result word string, it is determined that the search is successful, and the search result output means 17 outputs the section corresponding to the recognition result word string as the search result (step) A5). If the expanded word or word string does not exist in the recognition result word string, it is determined that the search has failed, and the search result output means 17 outputs a message indicating that the search is impossible (step A6). Consider the case where the search character string “Harry Potter” is expanded to “Harry” + “Poster” as in the example described above, if “Harry” + “Poster” exists in the recognition result, It is output as a search result of “Harry Potter”.

  Further, an index for search is created for the recognition result word string stored in the recognition result word string storage unit 13 in advance, and a word or a word string developed by referring to the index by the search unit 16 is used. Searching is also included in this embodiment. FIG. 8 is a diagram illustrating an example of an index for the recognition result of the audio data to be searched. The appearance position information of each word is stored as a combination of the document number where the word appears and the appearance position in the document. For example, a case where a word string “Harry” + “poster” in which “Harry Potter” is expanded is searched. By referring to the index of FIG. 8, “Harry” appears at the 10th word in document 1 and the 20th word in document 1, and “poster” appears in the 11th word in document 1 and in document 2. It can be seen immediately that it appears at the 10th word. Thereafter, by checking whether “Harry” and “Poster” are continuous, it can be found that “Harry” + “Poster” is in the 10th to 11th words in the document 1. By using such an index as shown in FIG. 8 for searching for the expanded word string, it is not necessary to search the entire document, so that the search speed can be greatly improved.

  Next, the effect of the present embodiment will be described. In this embodiment, since the recognition result in units of words is searched for in units of words, the space for performing the search is smaller than in the case of searching for the recognition results of phoneme strings in units of phonemes.

  In addition, since the search character string is expanded into words or word strings in consideration of the possibility of erroneous recognition in advance, it is not necessary to calculate the distance between the search character string and the recognition result by DP matching during the search. In this embodiment, when the search character string is expanded into a word string, it is necessary to calculate the distance between the search character string and the word candidate to be expanded. This is a conventional technique for DP matching between the entire search target and the search character string. Compared to this method, it is not a large amount of calculation. In addition, when searching for a developed word or word string from the recognition result, a search method using an index can be used.

  Furthermore, in this embodiment, since the recognition vocabulary used when obtaining the recognition result word string is used to expand the word or word string, a word or word string that has no possibility of appearing in the recognition result is searched. There is no.

  As a result, according to the present embodiment, the search speed for the voice data is greatly improved.

  Next, a second embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 9 is a block diagram showing the configuration of the voice search system according to the second embodiment of the present invention. The voice search system shown in FIG. 9 further includes word extraction means 29 for extracting a list of words appearing in the recognition result stored in the recognition result word string storage means 23 with respect to the voice search system shown in FIG. The word candidate storage means 28 is different in that it stores a list of words extracted by the word extraction means 29. In FIG. 9, the voice data storage means 21, the continuous word recognition means 22, the search character string input means 24, the search character string expansion means 25, the search means 26, and the search result output means 27 are the voice data storage in FIG. It corresponds to means 11, continuous word recognition means 12, search character string input means 14, search character string expansion means 15, search means 16, and search result output means 17.

  In the first embodiment, the word candidate storage means 18 stores the recognition vocabulary of the continuous word recognition means 12 as word candidates developed by the search character string expansion means. In this embodiment, the word candidate storage unit 28 stores a list of words extracted by the word extraction unit 29. The word extraction unit 29 examines the recognition result stored in the recognition result word string storage unit 23 and extracts a list of words that appear in the recognition result. At this time, it is not always necessary to extract all the words appearing in the recognition result. For example, the frequency with which words appear in the recognition result may be checked so that words with extremely low frequency are not extracted. Moreover, you may make it not extract the word with short length. In this way, a list of words extracted from the recognition result word string is stored in the word candidate storage means 28 and becomes a word candidate developed by the search character string expanding means 25.

  Further, when the word extracting means 29 examines words appearing in the recognition result, it also examines words appearing before and after each word, and creates a connection table between words that allows connection only to words appearing before and after each word. May be. In this case, the word candidate storage unit 28 stores both the list of words and the connection table created by the word extraction unit 29. The search character string expansion means 25 performs substantially the same operation as the search character string expansion means 15 of the first embodiment, but also refers to the connection table stored in the word candidate storage means 28 and connection is impossible. The difference is that it does not expand to the word string.

  Next, the connection table will be described. FIG. 10 is a diagram showing an example of a connection table created from the recognition result, and shows whether the subsequent word can be connected to the preceding word (represented by “◯”) or not connectable (represented by “x”). ing. Referring to the connection table of FIG. 10, it can be seen that the sequence of “Harry” + “Poster” exists in the recognition result, but the sequence of “Harry” + “Boxer” does not exist. No expansion to "Harry" + "Boxer". As a result, the development speed is improved and the search speed is also improved because unnecessary search is not performed.

  Next, the effect of the present embodiment will be described. In this embodiment, the word extraction unit 29 extracts a list of words appearing in the recognition result and stores it in the word candidate storage unit 28. Therefore, the search character string expansion unit 25 uses only words that always appear in the recognition result. Search strings can be expanded into words or word strings. For this reason, the efficiency of both word expansion and search is improved, and the search speed is further improved.

  Next, implementation of the voice search system and the voice search program according to the first and second embodiments described above will be described with reference to the drawings.

  FIG. 11 is a block diagram showing the configuration of the voice search system according to the embodiment of the present invention. In FIG. 11, the voice search system includes an input / output unit 51, a data processing unit 52, and a storage unit 53. The storage unit 53 includes a program storage unit 54, a word candidate storage unit 55, a voice data storage unit 56, and a recognition result word string storage unit 57.

  The input / output unit 51 includes a keyboard, a voice input device, a display device, and the like, and controls input / output of various data in the voice search system. This corresponds to the search character string input means 14 of FIG. 1 or the search character string input means 24 of FIG. The input / output unit 51 also corresponds to the search result output unit 17 in FIG. 1 or the search character string output unit 27 in FIG.

  The storage unit 53 stores the voice search program in the program storage unit 54. Also, the speech data storage means 11 or 21, the recognition result word string storage means 13 or 23, and the word candidate storage means 18 or 28 shown in FIG. 1 or FIG. 9, respectively, are the speech data storage section 56 in the storage section 53, It corresponds to the recognition result word string storage unit 57 and the word candidate storage unit 55, and is read and written by the data processing unit 52.

  The data processing unit 52 performs processing in the continuous word recognition unit 12, the search character string input unit 14, the search character string expansion unit 15, the search unit 16, and the search result output unit 17 shown in FIG. Execute. Alternatively, the data processing unit 52 controls the continuous word recognition unit 22, the search character string input unit 24, the search character string expansion unit 25, the search unit 26, the search result output unit 27, and the like shown in FIG. Processing in the word extraction means 29 is executed. Further, the voice search program searches the input search character string from the voice data to be searched by referring to the voice data storage unit 56, the recognition result word string storage unit 57, and the word candidate storage unit 55. Operate.

  The present invention can be applied to a purpose of retrieving desired content from an audio database such as broadcast audio and lecture audio.

It is a block diagram which shows the structure of the voice search system which concerns on 1st Example of this invention. It is a flowchart figure showing operation | movement of the voice search system which concerns on 1st Example of this invention. It is explanatory drawing when calculating | requiring the distance of a search character string and a certain expansion | deployment word string. It is explanatory drawing when calculating | requiring the distance of a search character string and another expansion | deployment word string. It is a figure showing the example of the confusion matrix between phonemes. It is a figure showing the example of the penalty between words. It is explanatory drawing when calculating | requiring the distance of a search character string and another expansion | deployment word string. It is a figure which shows the example of the index with respect to the recognition result of the audio | voice data used as search object. It is a block diagram which shows the structure of the voice search system which concerns on 2nd Example of this invention. It is a figure which shows the example of the connection table produced from the recognition result. It is a block diagram which shows the structure of the voice search system based on the Example of this invention. It is a block diagram which shows the structure of the conventional voice search system.

Explanation of symbols

11, 21 Speech data storage means 12, 22 Continuous word recognition means 13, 23 Recognition result word string storage means 14, 24 Search character string input means 15, 25 Search character string expansion means 16, 26 Search means 17, 27 Search result output Means 18, 28 Word candidate storage means 29 Word extraction means 51 Input / output unit 52 Data processing unit 53 Storage unit 54 Program storage unit 55 Word candidate storage unit 56 Speech data storage unit 57 Recognition result word string storage unit

Claims (20)

A recognition result word string storage means for storing a speech recognition result in units of words of the voice data to be searched as a recognition result word string;
Word candidate storage means for storing words that can appear in the speech recognition result as candidate words;
An input search character string is converted into a phoneme string, a candidate word string composed of the candidate word or a combination of the candidate words stored in the word candidate storage unit is converted into a phoneme string, and each is included in the phoneme string Search character string expansion means for expanding the search character string into the candidate word or the candidate word string based on the degree of matching between phonemes to be generated;
Search means for searching the candidate word or the candidate word string developed by the search character string development means from the recognition result word string storage means;
A voice search system comprising:   The speech search system according to claim 1, wherein the word candidate storage unit stores a recognition vocabulary for obtaining the recognition result word string from the speech data.   2. The voice according to claim 1, further comprising a word extracting unit that extracts a list of words appearing in a recognition result stored in the recognition result word string storage unit, wherein the word candidate storage unit stores the list. Search system.   When the word extraction means extracts the list, it examines words appearing before and after each word in the recognition result, creates a connection table between words that allows connection only to words appearing before and after, and the word candidates The storage unit stores the connection table together with the list, and the search character string expansion unit refers to the list stored in the word candidate storage unit and the connection table, and the search character string can be connected. The voice search system according to claim 3, further comprising a function that expands only to a candidate word or the candidate word string. The search character string expansion means includes a function of expanding a word by a continuous word speech recognition algorithm using a phoneme string of the search character string as an input feature vector series and the candidate word as a recognition vocabulary. The voice search system described. When the search character string expansion means expands the search character string using the candidate words stored in the word candidate storage means, the search character string and the candidate are based on the degree of matching of the phonemes. The voice search system according to claim 1, further comprising a function of obtaining a distance between a word or the candidate word string and developing the distance so that the distance is within a predetermined threshold . It comprises a confusion matrix storage means for storing a confusion matrix representing the recognition error tendency of the phonemes, and the search character string expansion means includes a function for obtaining a degree of coincidence between the phonemes based on the confusion matrix. The voice search system according to claim 6. The search character string expansion means includes a function for obtaining a degree of coincidence of the phonemes based on a distance between models in an acoustic model used when obtaining the recognition result word string from the speech data. The voice search system according to claim 6. In addition to the distance , the search character string expansion means refers to the language model used when obtaining the recognition result word string from the speech data, and sets the distance for words or word strings that are difficult to appear linguistically. The voice search system according to claim 6, further comprising a distance addition function for further distance . Search input means, a developing means, a search unit, a voice search system word string and a storage means for storing a word that may appear in the speech recognition result of a word unit of audio data to be retrieved as a candidate word A method,
The input means inputting a search character string;
The expansion means converts the candidate word string composed of the candidate word or the combination of candidate words in the storage means into a phoneme string, converts the search character string into a phoneme string, and each phoneme included in the phoneme string a step of developing the search string in the candidate word or the candidate word sequence based on the degree of coincidence between,
The search means representing the speech recognition result as a recognition result word string, and searching the expanded candidate word or the candidate word string from the recognition result word string;
A voice search method comprising: The voice search system further includes voice recognition means,
The speech recognition unit includes a step of performing speech recognition on the speech data and obtaining the candidate word as a recognition result prior to the step of inputting the search character string. Voice search method. The voice search system further includes voice recognition means,
The speech recognition means includes a step of performing speech recognition on the speech data and creating a list of words extracted from a recognition result prior to the step of inputting the search character string, and expanding the search character string The voice search method according to claim 10, wherein the step of expanding is performed with reference to the created list. And expanding means, a search unit, a method for searching a voice recognition unit, a recognition result a word sequence storage unit, the speech retrieval system and a word candidate storage means word strings,
A step of the recognition result stored in the word sequence storage means speech recognition result of a word unit of audio data to which the speech recognition means is searched as a recognition result word string,
The leave the word candidate storage means stores a word that may appear in the recognition result as the candidate words, the expansion means, converts the entered search string phoneme string, stored in the word candidate storage unit It said candidate word or convert the candidate word sequence composed of a combination of the candidate words into a phoneme string, the candidate word or the candidate word the search string based on the degree of coincidence of phonemes each other included the each phoneme string are Steps to expand into columns ,
The search means searching for the expanded candidate word or the candidate word string from the recognition result word string storage means;
A voice search method comprising: 13. The method of voice search, wherein the recognition vocabulary in obtaining the recognition result word string from said audio data said word candidate storage means stores. The voice search system further includes word extraction means,
The word extracting unit extracting a list of words appearing in the recognition result stored in the recognition result word string storage unit; and the word extracting unit storing the extracted word list in the word candidate storage unit. 14. The voice search method according to claim 13, further comprising the step of expanding the candidate word string with reference to the stored list. A computer constituting a speech search system comprising a recognition result word string storage means and a word candidate storage means ,
A process of storing the recognition result word sequence storage means the search speech recognition result of a word unit of audio data made as a recognition result word string,
A process of storing in the word candidate storage means for a word may appear in the recognition result as a candidate word,
An input search character string is converted into a phoneme string, a candidate word string composed of the candidate words or a combination of the candidate words stored in the word candidate storage unit is converted into a phoneme string, and each of the phoneme strings is included in the phoneme string A search character string expansion process for expanding the search character string into the candidate word or the candidate word string based on the degree of matching between phonemes to be generated ;
A search process for searching the candidate word or the candidate word string expanded by the search character string expansion process from the recognition result word string storage unit;
A program that executes   The program according to claim 16, further causing the word candidate storage unit to store a recognition vocabulary for obtaining the recognition result word string from the speech data.   A word extraction process for extracting a list of words appearing in the recognition result stored in the recognition result word string storage means, and a process for storing the word list extracted by the word extraction process in the word candidate storage means. The program according to claim 16, further causing execution of the search character string expansion process so as to expand with reference to the stored list. A computer constituting a speech search system comprising a recognition result word string storage means and a word candidate storage means ,
The candidate word sequence consisting of the candidate words or combinations of said candidate word in said word candidate storage means to the search subject to audio data units of words of the speech recognition result is previously stored words may appear into a phoneme sequence A search character string expansion that converts the input search character string into a phoneme string and expands the search character string into the candidate word or the candidate word string based on the degree of matching between the phonemes included in the respective phoneme strings Processing,
From the recognition result word sequence storage means are previously stored as the speech recognition result recognition results word string, a search process of searching the candidate word or the candidate word sequence which is expanded by the search string expansion process,
A program that executes   The program according to claim 19, further performing a process of performing speech recognition on the speech data and obtaining the candidate word as a recognition result.

Images