JP4949310B2 - Sound processing apparatus and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an acoustic model learning device which is capable of producing an acoustic model which corresponds to a variety of uttering styles by automatically generating the learning data of an acoustic model by a high recognition ratio even when an off-line subtitle in which the matching rate of the subtitle and the uttering content is low is used. <P>SOLUTION: The acoustic processor carries out the recognition processing of voice. It is provided with a recognition processing part which outputs recognition result data including a recognition hypothesis and reliability data accompanying the recognition hypothesis; and a selection processing part which, based on text data corresponding to the voice and recognition result data which are output by the recognition processing part, selects the matching section of both these data, prunes the recognition hypothesis which includes the time of the corresponding section and which does not match the text data, selects the reliability section based on the recognition hypothesis after pruning and accompanying reliability data, and outputs data of the matching section and the reliability section. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to speech processing based on statistical processing. In particular, the present invention relates to an acoustic processing apparatus and a program for processing an acoustic feature quantity statistic of each phoneme as an acoustic model.

  In speech recognition using a statistical model, a large amount of learning data is required to obtain the statistic of each phoneme. This learning data is a pair of a large amount of voice data and a transcription for the voice, and has been created manually and with time and cost.

  Currently, some broadcast programs are given subtitle text corresponding to program audio so that the contents of the program can be understood even in environments where information from the voice is not sufficiently available, such as watching a program for elderly or hearing-impaired people. There is something that is. There are two types of subtitle texts: subtitle text given online and subtitle text given offline. The former is almost identical to the utterance content in the program. However, the latter removes the redundancy of spoken words and can provide subtitles that are simple and easy to read, so the rate of coincidence between the utterance contents and the subtitles is low.

  When online subtitles with a high matching rate are available, audio model learning data can be easily and automatically created by extracting audio segments and subtitle segments corresponding to the extracted audio. However, it has been reported that the accuracy of recognition is improved in the case of a news program in which the reading speech is mainly used and relatively high speech recognition accuracy is obtained (Non-Patent Document 1). In this method, the recognition result of the program sound obtained by using a language model adapted by subtitle text is used for subtitle section extraction, and the coincidence section between the subtitle text and the recognition result is used as learning data for the acoustic model. This program audio recognition is performed mainly for the purpose of extracting a voice section and detecting a mismatched portion between subtitles and utterance content, and is characterized in that the weight of subtitle text used for language model adaptation is relatively small.

FIG. 11 is a block diagram showing a functional configuration of an acoustic model learning device according to the prior art. The acoustic model learning apparatus 200 shown in this figure divides subtitle text into morpheme units, and then assigns a weight of W _{bias to} the subtitle text to obtain a biased language model (biased language model 86). A program sound is recognized using this language model (recognition processing unit 90). Since the main purpose of this apparatus is to detect a mismatch section between subtitles and utterance contents and to cut out audio and subtitles, the bias weight W _bias (about 4) and the language weight W _lm (about 10) use relatively small values. . Next, DP matching between the recognition hypothesis and the caption text is performed for the entire program (alignment processing unit 92), and a section in which the caption text and the recognition result match three or more words continuously is selected, and the voice section in the program sound is selected. Learning data (learning data storage unit 94), which is a corresponding word string, is obtained.
Long Nguyen, 1 other, “Light Supervision in Acoustic Model Training”, IEEE International Conference on Acoustics, Speech, and Signal Processing, 2004. Proceedings. (ICASSP apos; 04), 2004, Volume 1, p. 17-21

  Currently, the recognition accuracy of a part spoken in a relatively free style such as a dialogue part of a broadcast program is lower than the recognition accuracy of a reading speech, and in order to improve the recognition accuracy of such a part, various styles are used. Learning data for the spoken acoustic model is needed.

  In a relatively free style such as a dialogue part of a broadcast program, subtitle text created offline is used. However, as mentioned above, priority is given to simplicity when creating this subtitle text. The matching rate with the utterance content is low. The acoustic model learning device according to the prior art is based on the premise that the matching rate between the subtitle text and the actual utterance content is high, and the method according to the prior art cannot obtain high recognition accuracy from the data with the low matching rate. As a problem, it is impossible to efficiently generate a large amount of learning data for an acoustic model.

  The present invention has been made in view of the above circumstances, and even when offline subtitles with a low match rate between subtitles and utterance contents are used, acoustic model learning data is automatically generated with a high recognition rate, and various The purpose is to create an acoustic model that can handle various utterance styles. Therefore, the present invention provides an acoustic processing device and a program that can extract, for acoustic model automatic generation, sections that do not match when using offline subtitles with a low match rate between subtitles and utterance content. For the purpose.

[1] In order to solve the above-described problem, an acoustic processing device according to an aspect of the present invention performs speech recognition processing and outputs recognition result data including a recognition hypothesis and reliability data associated with the recognition hypothesis. Based on the recognition processing unit, the text data corresponding to the speech, and the recognition result data output by the recognition processing unit, a matching section between these two data is selected, and includes the time of the matching section and the text data A selection processing unit that prunes the recognition hypotheses that do not match, selects a confidence interval based on the recognition hypothesis after pruning and accompanying reliability data, and outputs data of the matching interval and the confidence interval; It is characterized by that.
As a result, it is possible to extract data not only from a section where the recognition hypothesis and text data match but also from a confidence section including a recognition hypothesis that can be trusted at a predetermined level or higher from a section where both do not match. With such a configuration, more learning data can be obtained.

[2] Further, according to one aspect of the present invention, in the above-described acoustic processing device, an acoustic model storage unit that stores an acoustic model in which a phoneme is associated with an acoustic feature amount corresponding to the phoneme, and selection by the selection processing unit And an acoustic model adapting unit that updates the acoustic model stored in the acoustic model storage unit according to the result, and the recognition processing unit reads the acoustic model read from the acoustic model storage unit And the selection processing unit uses, as learning data, a set of the selected phoneme included in the selected matching section and the selected confidence section and an acoustic feature amount corresponding to the phoneme. The acoustic model adaptation unit updates the acoustic model stored in the acoustic model storage unit using the learning data output from the selection processing unit. Characterized in that it is a shall.
According to this configuration, the learning data is obtained from the coincidence section between the speech recognition result data (recognition hypothesis) performed based on the acoustic model and the text data (caption text, etc.) corresponding to the speech, and this matching is obtained. The recognition hypothesis is pruned based on the interval, and learning data is also obtained from the confidence interval obtained as a result. The acoustic model is updated using learning data obtained from the coincidence interval and the confidence interval. That is, the acoustic model can be adapted. Then, more learning data can be obtained by performing recognition processing again using the adapted acoustic model. Such adaptation and learning data acquisition can be repeated, and a large amount of learning data can be obtained efficiently.

  [3] Further, according to one aspect of the present invention, a recognition process in which speech recognition processing is performed and recognition result data including a recognition hypothesis and reliability data accompanying the recognition hypothesis is output, and the speech is supported. Based on the text data and the recognition result data output by the recognition processing step, select a matching section of both data, and pruning the recognition hypothesis that includes the time of the matching section and does not match the text data, This is a program for selecting a confidence interval based on a recognition hypothesis after pruning and accompanying reliability data, and causing the computer to execute processing with a selection processing step of outputting data of the matching interval and the confidence interval.

  Furthermore, the above-described problems may be solved as the following aspects A2 to A6.

[A2] According to one aspect of the present invention, in the acoustic processing apparatus, the acoustic model storage unit stores the acoustic model for each program, and the recognition processing unit Voice recognition processing is performed using the acoustic model corresponding to the program, the selection processing unit outputs the learning data for each program, and the acoustic model adaptation unit selects the acoustic model for the program. It is characterized by updating.
According to this configuration, the sound to be recognized is the sound of a program such as a broadcast, and each program has an acoustic model, and the acoustic model can be adapted using the recognition result for each program. Therefore, when there is a specific acoustic feature for each program, adaptation according to the unique feature can be performed, recognition accuracy is improved, and a larger amount of learning models can be acquired.

[A3] According to one aspect of the present invention, in the above sound processing device, the audio processing apparatus further includes a language model storage unit that stores a language model, and the recognition processing unit includes the language model read from the language model storage unit and the language model The acoustic model read from the acoustic model storage unit is weighted with a predetermined weight ratio value and used for recognition processing, and the recognition processing unit is updated by the acoustic model adaptation unit. The recognition process is performed again using an acoustic model, and the weight ratio value used at this time is a value used by the language model with a smaller weight than in the previous recognition process. .
According to this configuration, when the recognition process and the adaptation of the acoustic model using the recognition process result are repeatedly performed, the weight for using the language model gradually decreases. Therefore, even if the language model weight is increased in order to increase the initial recognition rate, the weight of the language model can be reduced in accordance with the progress of adaptation of the acoustic model, and the recognition accuracy is improved. Acquire a large amount of learning models.

[A4] According to one aspect of the present invention, in the above-described acoustic processing device, an utterance model storage unit that stores an utterance model representing a feature amount of an utterance part in the speech, and the utterance read out from the utterance model storage unit Update the utterance model stored in the utterance model storage unit using the speech segment cutout unit that cuts out the speech segment of the utterance part in the speech using the model and the learning data output from the selection processing unit An utterance model adaptation unit that performs the recognition processing on the voice in the voice segment cut out by the voice segment cut-out unit.
According to this configuration, the speech section of the speech portion in the speech section is cut out using the speech model. Then, the utterance model is adapted using the learning data obtained as a result of the recognition process. Therefore, it is possible to repeat the extraction of the speech section of the utterance portion, the recognition processing of the extracted speech section, and the adaptation process of the utterance model using the learning data based on the recognition result. Therefore, the accuracy of segmentation of speech sections is improved, the recognition accuracy is increased, and a larger amount of learning models can be acquired.

[A5] According to one aspect of the present invention, in the above sound processing device, the speaker attribute-specific utterance model storage unit that stores a speaker attribute-specific utterance model representing a statistic for each speaker or each speaker attribute; Speaker attribute estimation means for estimating a speaker or speaker attributes in a predetermined voice section in the speech using the speaker attribute-specific utterance model read from the speaker attribute-specific utterance model storage means, The acoustic model storage unit stores the acoustic model separately for each speaker or for each speaker attribute, and the recognition processing unit corresponds to the estimation result by the speaker attribute estimation unit. The acoustic model for the speaker or the speaker attribute is read from the acoustic model storage unit, and the speech recognition process is performed using the read acoustic model. For each speaker or each speaker attribute corresponding to the estimation result stored in the speaker attribute-specific utterance model storage unit, using the learning data corresponding to the estimation result by the speaker attribute estimation unit The apparatus includes a speaker attribute-specific utterance model adaptation means for updating the speaker attribute-specific utterance model.
According to this configuration, the speaker or speaker attribute is estimated using the speaker attribute-specific utterance model. In addition, an acoustic model is provided for each speaker or speaker attribute, and recognition processing using an acoustic model corresponding to the estimation result is performed. Using the learning data obtained as a result of this recognition processing, the speech model for each speaker attribute is adapted. By repeating this process, the quality of the utterance model for each speaker attribute is improved, and the accuracy of estimating the speaker or the speaker attribute is improved, that is, the speech section suitable for the acoustic model for each speaker or speaker attribute. The accuracy of cutting out is improved. Therefore, recognition accuracy is improved and a larger amount of learning models can be acquired.

[A6] According to one aspect of the present invention, in the acoustic processing device, the acoustic model adaptation unit corresponds to the estimation result using the learning data corresponding to the estimation result by the speaker attribute estimation unit. The acoustic model is updated for each speaker or each speaker attribute.
According to this configuration, the learning model corresponding to the estimation result is used to adapt the acoustic model corresponding to the estimation result. Therefore, the quality of the acoustic model is further improved, the recognition accuracy is improved, and a larger amount of learning models can be acquired.

  According to the present invention, it is possible to efficiently obtain a large amount of learning data of an acoustic model from various program sounds to which subtitles are assigned. In addition, it is possible to obtain learning data of an acoustic model more efficiently from program audio having a low matching rate between subtitles and actual audio. In particular, learning data can be obtained from a part of a section in which subtitles and audio do not match. In addition, it becomes possible to recognize programs and speakers that could not be recognized before, and to improve the recognition accuracy of voice recognition.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a block diagram illustrating a functional configuration of the acoustic model learning device according to the first embodiment. As shown in the figure, the acoustic model learning device 100 includes a caption text storage unit 1, a morpheme analysis processing unit 2, a weighting processing unit 3, a baseline corpus storage unit 4, a synthesis processing unit 5, and a biased language model. A storage unit 6 (language model storage unit), a weighting processing unit 7, a program audio storage unit 8, an acoustic model storage unit 9, a recognition processing unit 10, a recognition hypothesis storage unit 11, an alignment processing unit 12, A selection processing unit 13, a learning data storage unit 14, and an acoustic model adaptation unit 15 are included.

  The caption text storage unit 1 stores caption text data of a broadcast program. The morpheme analysis processing unit 2 performs a morpheme analysis process on the subtitle text read from the subtitle text storage unit 1 using an existing technique, and outputs the result as a column divided into morphemes.

The weighting processing unit 3 performs weighting for synthesizing the caption text subjected to morphological analysis with the baseline corpus. In other words, the weighting processing unit 3 applies a bias that multiplies the appearance frequency of the caption text by the weight value _Wbias . The baseline corpus storage unit 4 stores a baseline corpus. The baseline corpus is a database composed of a large amount of electronic natural language sentences. For example, the accumulated past news manuscript text is used as the baseline corpus. Note that statistical processing may be performed on the baseline corpus in advance and stored in a format suitable for a language model described later.

  The synthesis processing unit 5 synthesizes the caption text data weighted by the weighting processing unit 3 and the baseline corpus read from the baseline corpus storage unit 4, and calculates statistical numerical values regarding the appearing words. Create a biased language model expressed with The biased language model storage unit 6 stores the biased language model created by the synthesis processing unit 5. Details of the language model will be described later with reference to the drawings.

The weighting processing unit 7 weights the language model stored in the biased language model storage unit 6 with the weight value W _lm for the recognition processing by the recognition processing unit 10. As a result, the recognition processing unit 10 uses the language model and the acoustic model at a weight ratio of W _lm : 1.

The program audio storage unit 8 stores program audio. This program audio corresponds to subtitle text data stored in the subtitle text storage unit 1.
The acoustic model storage unit 9 stores acoustic model data in which a phoneme is associated with an acoustic feature amount corresponding to the phoneme. The acoustic model data will be described later.

The recognition processing unit 10 performs speech recognition processing read from the program audio storage unit 8 using the acoustic model read from the acoustic model storage unit 9 and the language model weighted by the weight value W _lm , and the recognition hypothesis ( (Recognition result data) is output. The recognition hypothesis storage unit 11 stores this recognition hypothesis.

  The alignment processing unit 12 performs a process of finding a matching portion between the caption text data that has been subjected to the morphological analysis processing by the morphological analysis processing unit 2 and the recognition hypothesis read from the recognition hypothesis storage unit 11. Specifically, the alignment processing unit 12 searches both of these data, and performs a process of finding a place where a predetermined number of consecutive words match or a predetermined number of consecutive characters match.

  The selection processing unit 13 selects a matching section between the two data aligned by the alignment processing unit 12, and outputs a set of phonemes included in the selected matching section and an acoustic feature amount corresponding to the phoneme as learning data. Further, the selection processing unit 13 prunes recognition hypotheses that do not match the caption text data in the recognition hypotheses including the selected matching sections, and the recognition hypotheses after pruning and the associated reliability data Based on the above, a section in which the reliability data is equal to or greater than a predetermined threshold is selected as a confidence section, and a set of phonemes included in the selected confidence section and an acoustic feature value corresponding to the phoneme is output as learning data.

  The learning data storage unit 14 stores the learning data output by the selection processing unit 13. The acoustic model adaptation unit 15 performs processing to adapt (update) the acoustic model stored in the acoustic model storage unit 9 using the learning data read from the learning data storage unit 14.

  The acoustic model learning device 100 is characterized in that the acoustic model adaptation unit 15 exists, and the learning data output by the selection processing unit 13 is used to perform maximum a posteriori estimation method (MAP) or maximum likelihood linear regression analysis method (MLLR). ) Etc., the acoustic model stored in the acoustic model storage unit 9 is adapted (updated), and the recognition processing is repeatedly performed using the adapted acoustic model.

In addition, unlike the prior art, it is also characteristic that recognition processing is performed using relatively large weight values W _bias (about 1000) and weight values W _lm (about 16). In a program with a low matching rate between subtitles and utterances and a high ratio of free utterances, using the weight values as described above leads to obtaining more learning data while improving recognition accuracy.
This is an appropriate weight value obtained from experimental results. Specifically, in an experiment conducted by the inventors of the present invention using NHK (Japan Broadcasting Corporation) actual program audio and subtitle text, word misrecognition due to changes in W _{lm in} the region of W _bias <5000. The difference in rate (WER) is small. Also, in the region of W _lm <18, the difference in word recognition rate due to the change in W _bias is small. When W _bias <5000 and W _lm <18, the word error recognition rate does not increase so much, and the amount of learning data to be obtained increases. When W _bias is about 1000 and W _lm is about 16, particularly, the word error recognition rate does not increase so much and the amount of learning data to be obtained increases.

Next, the language model will be described.
FIG. 2 is a schematic diagram showing the structure of language model data stored in the biased language model storage unit 6 and used in the recognition processing by the recognition processing unit 10.
FIG. 2A shows a table representing the appearance probability of a morpheme alone. This table is tabular data, and has items (columns) of morphemes and appearance probabilities. A table row exists for each morpheme type. Each morpheme and the appearance probability of the morpheme are associated and held in this table.
FIG. 2B is a table showing connection probabilities between morphemes and morphemes. This table is tabular data, and has morpheme 1, morpheme 2, and connection probability items (columns). A table row exists for each combination of morpheme 1 and morpheme 2. Each row represents connection probability information that is a probability that a morpheme 2 appears (connects) immediately following the morpheme 1 when the morpheme 1 appears.

Next, an acoustic model will be described.
FIG. 3 is a schematic diagram showing a configuration of acoustic model data stored in the acoustic model storage unit 9 and used in the recognition processing by the recognition processing unit 10. As shown in the figure, the acoustic model is data that holds a phoneme and an acoustic feature amount when the phoneme is spoken in association with each other. As the acoustic feature quantity, for example, MFCC (Mel-Frequency Cepstrum Coefficient) or PLP (Mel-Frequency Cepstrum Coefficient) based on the frequency power distribution in a 25-millisecond frame starting at intervals of 10 msec. Perceptual Linear Predictive) is used.

Next, a recognition hypothesis that is a recognition result by the recognition processing unit 10 will be described.
FIG. 4 is a schematic diagram illustrating a data configuration and a data example of a recognition hypothesis output by the recognition processing unit 10 and stored in the recognition hypothesis storage unit 11. The recognition hypothesis is logically represented by a graph of a lattice structure (partial order structure) with linguistic units (morphemes) as arcs.

  FIG. 4A shows a data structure of recognition hypotheses expressed in a tabular format. As shown in the figure, in the recognition hypothesis expressed in a tabular form, each row corresponds to each arc on the lattice, and has items (columns) of a start point node, an end point node, a linguistic unit, and a reliability. The start point node and the end point node have node identification information given to the nodes on the lattice for convenience. A linguistic unit is a morpheme corresponding to the arc. The reliability is represented by a numerical value representing the reliability of the arc in the recognition result (reliability of the recognition result obtained by statistical processing based on the language model and the acoustic model), and the range of the numerical value is 0. 1 or less). In the illustrated data example, for example, the arc start point node corresponding to the data in the first row is “1”, the end point node is “2”, and the linguistic unit is “F”. Further, for example, the arc start point node corresponding to the data in the second row is “1”, the end point node is “2”, and the linguistic unit is “H”. The same applies to data in the third and subsequent rows. In the figure, the description of the reliability value is omitted.

  FIG. 4B is a schematic diagram pictorially representing the lattice represented by the tabular data in FIG. Circles in the figure correspond to nodes, respectively, and arcs are represented by lines with arrows connecting the nodes. In addition, linguistic elements corresponding to each arc are added. For example, there are three arcs starting from the leftmost node (node "1"), each of which has a linguistic element of "F" and an end point of node "2" (second node from the left), language The target element is “H”, the end point is node “2”, the linguistic element is “M”, and the end point is node “3” (the third node from the left).

Next, the operation and processing procedure of the acoustic model learning device 100 will be described.
FIG. 5 is a flowchart showing a processing procedure of the entire acoustic model learning apparatus 100.
As shown in the figure, in step S01, first, the morpheme analysis processing unit 2 performs a morpheme analysis process on the subtitle text read from the subtitle text storage unit 1. The morphological analysis process can be performed by an existing technique.

Next, in step S02, the weight value _Wbias is used to weight the morpheme analysis process result, and in step S03, the weighted morpheme analysis process result and the baseline corpus are combined. Specifically, the synthesis processing unit 5 statistically processes the processing result of the morpheme analysis processing unit 2, calculates the appearance probability of each morpheme, calculates the connection probability between two consecutive morphemes, and FIG. Appearance probability data in the format shown in a) and connection probability data in the format shown in FIG. 2B are obtained. Further, the synthesis processing unit 5 performs similar statistical processing as necessary based on the baseline corpus stored in the baseline corpus storage unit 4, and appearance probability data in the format shown in FIG. And connection probability data in the format shown in FIG. The process results subtitle text 1 by morphological analysis section 2 of these performs weighting processing unit 3 weight value W _bias, the synthesis processing unit 5 performs probability calculation for synthesizing them, biased as a result A language model is calculated and written into the biased language model storage unit 6.

Next, in step S04, the weight model W _lm is used to weight the language model read from the biased language model storage unit 6, and in step 05, the recognition processing unit 10 stores the weighted biased language model and the acoustic model. Using the acoustic model read from the unit 9, the program audio stored in the program audio storage unit 8 is recognized, and the recognition hypothesis generated as a result is written in the recognition hypothesis storage unit. The generated recognition hypothesis is character string data, and may be accompanied by information for dividing the character string into morphemes. The recognition process itself used here is an existing technology.

  Next, in step S06, the alignment processing unit 12 performs an alignment process between the caption text subjected to the morphological analysis process and the recognition hypothesis obtained as a result of the recognition process. This alignment process is performed by searching for a portion in which both data matches in units of words, or by searching for a section in which a plurality of words in both data match continuously. As a result of the alignment process, the section in the caption text and the section in the program audio are associated in terms of data.

Next, in step S07, the selection processing unit 13 performs a process of selecting a matching section and a confidence section. Specifically, the selection processing unit 13 first uses the alignment result by the alignment processing unit 12 to select a matching section in which the original subtitle text and the recognition hypothesis match continuously for three words or more, and the matching Learning data in which the voice of the section (the voice section in the original program voice) and the corresponding word string are associated is written in the learning data storage unit 14.
Next, the selection processing unit 13 prunes the recognition hypothesis of the lattice structure described above based on the result of selecting the matching section, and uses the reliability data associated with the lattice after pruning, An interval that does not have a sufficient recognition result is selected as a confidence interval. Then, learning data in which the speech of the selected confidence interval and the corresponding word string are associated is written in the learning data storage unit 14.
The recognition hypothesis pruning process by the selection processing unit 13 will be described in detail later with reference to another drawing.

Next, in step S08, it is determined whether or not to end the repetition process. This determination is made based on, for example, whether the accuracy of the recognition result is sufficient. More specifically, this determination includes (a) the number of times the loop of steps S05 to S09 is repeated (for example, this number may be 3 to 4 times), and (b) a selection process among the entire program audio. The ratio of the section selected by the section 13 or (c) the increment ratio from the previous time of the section selected by the selection processing section 13 of the entire program audio is performed.
As a result of the determination, if the repetition process is not terminated (step S08: NO), the process proceeds to the next step S09.
As a result of the determination, when the repetition is finished (step S08: YES), the processing of the entire flowchart is finished. At this time, the data cumulatively written in the learning data storage unit 14 is an acoustic model obtained as a result of the learning process by the present apparatus.

  In step S09, the acoustic model adaptation unit 15 performs a process of adapting (updating) the acoustic model stored in the acoustic model storage unit 9, using the obtained learning data.

  As described above, the acoustic model learning device 100 uses the learning data output by the selection processing unit 13, the acoustic model adaptation unit 15 adapts the acoustic model stored in the acoustic model storage unit 9, Furthermore, the recognition process is repeatedly performed using the adapted acoustic model.

Next, the recognition hypothesis pruning process by the selection processing unit 13 described above will be described in more detail.
FIG. 6 is a schematic diagram illustrating an example of recognition hypothesis data after pruning by the selection processing unit 13. The data shown in FIG. 4 is the recognition hypothesis data before being pruned by the selection processing unit 13.
As described above, the selection processing unit 13 selects a matching section between the recognition hypothesis data and the caption text data shown in FIG. Here, it is assumed that the selection processing unit 13 selects a section in which one or more morphemes are continuously matched as a matching section. As a result, morpheme B (start node is “2” and end node is “3”) and morpheme C (start node is “3” and end point is “4”) and morpheme E (start point node is “5” and end point) Node “6”) is selected as the matching section.

After that, the selection processing unit 13 prunes a recognition hypothesis that includes the time of the matching section selected above and does not match the caption text data. Specifically, morphemes I, M, and N that are recognition hypotheses that include the time of the morpheme B section shown in FIG. 4 and that do not match morpheme B (in other words, recognition hypotheses that oppose morpheme B) are selected. This is a target of pruning by the processing unit 13. Similarly, morphemes J, N, and O, which are recognition hypotheses that oppose the morpheme C selected as the coincidence section, are subject to pruning by the selection processing unit 13. Similarly, morphemes L and P, which are recognition hypotheses that oppose the morpheme E selected as the coincidence section, are subject to pruning by the selection processing unit 13.
That is, in the lattice shown in FIG. 4, the arcs of morphemes I, J, L, M, N, O, and P are pruned by the selection processing unit 13. As a result of the pruning, the arcs of morphemes B, C, E, F, G, H, and K are left, and the data corresponding to these arcs is the data shown in FIG.

FIG. 6A shows tabular data representing such a lattice structure graph. This table shows data of rows corresponding to the morphemes B, C, E, F, G, H, and K. Have. In this table, since morphemes B, C, and E are included in the coincidence sections, as a result, the selection processing unit 13 has the highest reliability corresponding to these morphemes. The data of the recognition hypothesis is updated so that For other morpheme F, H, K, and G arcs, the selection processing unit 13 calculates posterior probabilities after pruning, and updates the recognition hypothesis data using the calculated results as respective reliability data. To do. In the illustrated example, the reliability of the morpheme F is 0.1, the reliability of the morpheme H is 0.9, the reliability of the morpheme K is 0.2, and the reliability of the morpheme G is 0.8.
FIG. 6B is a schematic diagram pictorially showing the lattice represented by the tabular data in FIG.

  The method by which the selection processing unit 13 calculates the posterior probability is as follows. The selection processing unit 13 uses the Forward-Backward algorithm, which is a kind of dynamic programming, to calculate the posterior probability based on the score L (m) of the morpheme m obtained from the output of the decoder (recognition processing unit 10). calculate. Here, the likelihood of the acoustic model, the likelihood of the language model, or the sum thereof is used as L (m). First, the forward probability α (m) and the backward probability β (m) of the morpheme m are defined as the following equations (1) to (5).

  Here, in order to calculate the posterior probability, arcs corresponding to the start word and the end word are added to both ends of the lattice shown in FIG. The starting word is added to the left end of the lattice in the figure and is represented as “b”. The end word is added to the right end of the lattice in the figure and is represented as “e”. Here, the forward direction is the direction from the start to the end, and in the lattice in the figure, the direction is from left to right. The backward direction is the direction from the end to the start, and in the lattice in the figure, the direction is from right to left. LEFT (m) is a set of morphemes connected to the morpheme m from the left (that is, from the start end side). RIGHT (m) is a set of morphemes connected to the morpheme m from the right (that is, from the terminal side).

As expressed in Equation (1), the forward probability of the starting word (b) is 1.0. Further, as expressed in Expression (2), the backward probability of the terminal word (e) is 1.0.
As expressed in Equation (3), the forward probability of the morpheme m is a value obtained by multiplying the forward probability α (m ′) of each morpheme m ′ connected to the morpheme m from the left by the score L (m) of the morpheme m. , The sum of all morphemes connected from the left. Thus, the forward probability is recursively defined. However, the forward probability of all the morphemes in the lattice can be calculated by sequentially calculating forward from the starting word.
As shown in Equation (4), the backward probability of the morpheme m is a value obtained by multiplying the backward probability β (m ′) of each morpheme m ′ connected to the morpheme m from the right by the score L (m) of the morpheme m. , The sum of all morphemes connected from the right. As described above, the backward probability is recursively defined, but the backward probability of all morphemes in the lattice can be calculated by sequentially calculating backward from the terminal word.
As shown in Equation (5), let Z be the forward probability of the end word and the backward probability of the start word.

  Then, the posterior probability γ (m) of the morpheme m can be calculated by Expression (6) using α, β, and Z.

The meaning of the selection processing unit 13 selecting the confidence interval as well as the matching interval in this way is as follows.
Originally, subtitle text is not necessarily created only for the purpose of matching the sound faithfully. For example, the subtitle text is easy to read, easy to understand the meaning, and for a predetermined time (the speaker speaks in the program sound). It is edited considering the length of text that can be read by a person within a certain period of time. Therefore, there are cases where the audio and the subtitle text do not inherently match. Even if the voice and the subtitle text do not match, the recognition result hypothesis may be a correct recognition result. Therefore, if there is a section in which reliability data of a predetermined number or more of morphemes continuously exceed the threshold, the recognition result hypothesis in that section is a correct recognition result even if it does not match the caption text. Can be estimated. Therefore, selecting such a section as a confidence section and acquiring learning data from the recognition result of this section leads to an increase in the obtained learning data.

  Further, for example, when the reliability of the morpheme H is relatively low due to the relatively high likelihood of the path of the morpheme M shown in FIG. 4B, the morpheme B is selected as the matching section. As a result, M, which is one of the morphemes that opposes morpheme B, is pruned, and as a result, the reliability data of morpheme H calculated may have a relatively high value. And when the reliability of the morpheme H exceeds a threshold value by this, the area containing this morpheme H may be selected as a reliability area. That is, by pruning the recognition hypothesis by the selection processing unit 13 in this way, another recognition hypothesis that shares the time interval with the pruned recognition hypothesis emerges, and the interval including the recognition hypothesis is trusted. It may be selected as a section. This leads to a result that more learning data with high reliability can be obtained.

  FIG. 7 is a diagram for supplementarily explaining an outline of learning of an acoustic model by the acoustic model learning device 100. FIG. 7A shows the acquisition of learning data based on the coincidence interval, and FIG. 7B shows the acquisition of learning data based on the confidence interval.

  In FIG. 7A, the morphological analysis result is data obtained by morphological analysis of the caption text, and this data is a one-dimensional column of morphemes (words) (each square in the figure is a morpheme). Equivalent). The recognition result is data corresponding to the maximum likelihood path selected from the recognition hypothesis lattice obtained as a result of the speech recognition processing by the recognition processing unit 10, and this data is also a one-dimensional sequence of morphemes ( Each square in the figure corresponds to a morpheme). A section that matches three or more words continuously is a matching section that is selected by the selection processing unit 13, and this section is indicated by hatching in the figure. Moreover, the learning data to be obtained is obtained by associating the text of the selected section with the voice data of the section and the acoustic feature data corresponding thereto.

  In FIG. 7B, the recognition result is included in the lattice after the above-described pruning process is performed by the selection processing unit 13 from the recognition hypothesis lattice obtained as a result of the speech recognition processing by the recognition processing unit 10. Data corresponding to one of the paths. That is, this data is a row of morphemes, and each square in the figure corresponds to a morpheme, and each has reliability data. A section where the reliability data is continuously greater than or equal to a predetermined threshold value for three or more morphemes is a confidence section selected by the selection processing unit 13, and this section is represented by hatching in the figure. Further, learning data obtained by associating the text of the selected confidence interval with the speech data of the interval and the acoustic feature data corresponding thereto is obtained.

  In the present embodiment, the acoustic model is adapted using the learning data that is the result of the recognition process. And it recognizes again using the adapted acoustic model, and produces | generates learning data. By repeating the adaptation of the acoustic model and the recognition process using the adapted acoustic model, the accuracy of the learning data is increased, and a larger amount of learning data can be efficiently obtained.

[Second Embodiment]
Next, a second embodiment of the present invention will be described.
FIG. 8 is a block diagram showing a functional configuration of the acoustic model learning apparatus 101 according to the embodiment. As shown in the figure, an acoustic model learning device 101 includes an acoustic model learning device (for program A) 101A, an acoustic model learning device (for program B) 101B, an acoustic model learning device (for program C) 101C, and an acoustic model. And a learning device (for program D) 101D. Each of these acoustic model learning apparatuses 101A to 101D is for a single program.

Here, the program is typically a broadcast program of television or radio, but is not limited to a “broadcast” program that is a transmission of wireless communication intended to be directly received by the public, but wired Television and radio broadcast programs via communication, broadcast and audio broadcast programs (including audio) distributed as packet data via the Internet, programs sent to many recipients from communication satellites, viewing Content including audio, such as a program distributed on-demand in response to a request from a listener or listener.
Note that the number of acoustic model learning devices for individual programs included in the acoustic model learning device 101 is not limited to four, and may be configured as an arbitrary natural number.

As illustrated, each of the acoustic model learning devices 101A to 101D has the same configuration and function as the acoustic model learning device 100 described in the first embodiment, and performs the same operation.
Also in the present embodiment, the selection processing unit 13 selects a matching section between the two data aligned by the alignment processing unit 12, and sets a set of phonemes included in the selected matching section and an acoustic feature amount corresponding to the phoneme. Output as learning data. Further, the selection processing unit 13 prunes recognition hypotheses that do not match the caption text data in the recognition hypotheses including the selected matching sections, and the recognition hypotheses after pruning and the associated reliability data Based on the above, a section in which the reliability data is equal to or greater than a predetermined threshold is selected as a confidence section, and a set of phonemes included in the selected confidence section and an acoustic feature value corresponding to the phoneme is output as learning data.

The point of this embodiment is that each of the acoustic model learning devices 101 </ b> A to 101 </ b> D includes a dedicated program audio storage unit 8 and an acoustic model storage unit 9, and each dedicated acoustic model is stored in each acoustic model storage unit 9. The recognition processing unit 10 performs a recognition process using each dedicated acoustic model, and the learning data obtained as a result is written in the dedicated learning data storage unit 14, and the learning data is accumulated and output. The acoustic model adapting unit 15 adapts the acoustic model storage unit 9 for the program using the obtained learning data, and the learning data for each program is obtained using the acoustic model for each program. It is a point to repeat the process to obtain.
In other words, the acoustic model storage unit 9 stores an acoustic model for each program, and the recognition processing unit 10 performs speech recognition processing using the acoustic model corresponding to the program. The selection processing unit 13 outputs learning data for each program, and the acoustic model adaptation unit 15 updates the acoustic model for the program.

  Note that the acoustic model learning devices 101A to 101D may be provided with all of the components dedicated to each other, or some components may be shared. For example, the processing functions of the recognition processing unit 10, the alignment processing unit 12, the selection processing unit 13, and the acoustic model adaptation unit 15 use the same hardware, and the acoustic data stored in the acoustic model storage unit 9 and the recognition hypothesis. The apparatus may be configured such that the recognition hypothesis stored in the storage unit 11 and the learning data stored in the learning data storage unit 14 are managed separately in areas dedicated to each program.

  The advantages of this embodiment are as follows. For example, when learning data is obtained by recognizing the voice of a conversational TV broadcast program, the program format is regular speakers (speakers who appear every time) and guest speakers (seen only a specific time) When a series of programs by two people (speakers) is targeted (in this case, each time of broadcasting corresponds to programs A to D), if learning data of different guest speakers is important, It can be said that the effect of speaker adaptation can be obtained by repeatedly performing the acoustic model adaptation process rather than performing the common acoustic model adaptation process multiple times. In this case, the learning data obtained from each time is collected to obtain the final learning data of the acoustic model.

  Furthermore, not only the above-mentioned programs in the form of conversation, but also in the broadcast programs, there are programs that have only a limited number of speakers, programs that a specific speaker continues to appear regularly, etc. is there. In this embodiment, since the acoustic model is adapted for each program, the learning data of other programs are not mixed, and as a result, high recognition accuracy is obtained.

[Third Embodiment]
Next, a third embodiment of the present invention will be described. In the above embodiment, a fixed value is used as the weight value W _lm that affects the weights of the language model and the acoustic model in the recognition process by the recognition processing unit 10. In the present embodiment, the weight value W _lm is changed according to the degree of adaptation of the acoustic model.

  Also in the present embodiment, the selection processing unit 13 selects a matching section of both data aligned by the alignment processing unit 12, and the phoneme included in the selected matching section and the acoustic feature amount corresponding to the phoneme are selected. The set is output as learning data. Further, the selection processing unit 13 prunes recognition hypotheses that do not match the caption text data in the recognition hypotheses including the selected matching sections, and the recognition hypotheses after pruning and the associated reliability data Based on the above, a section in which the reliability data is equal to or greater than a predetermined threshold is selected as a confidence section, and a set of phonemes included in the selected confidence section and an acoustic feature value corresponding to the phoneme is output as learning data.

FIG. 9 is a flowchart showing a processing procedure of the acoustic model learning apparatus according to the present embodiment. Steps S31 to S39 in this flowchart correspond to S01 to S09 shown in FIG. 5 and have the same processing procedure, and therefore the description thereof is omitted here. Then, following the process of step S39 of this flowchart, in step S40, the weighting coefficient of the biased language model is updated. As an example, a weight value in the n-th iteration (n = 1, 2, 3,...) Of the iterative processing from steps S34 to S40 (this weight value is a function of n, and is expressed as W _lm (n). )
W _lm (n) = 13−0.5n (7)
And That is, according to the above equation (7), the weight value W _lm (1) used for the first recognition process is 12.5, and the weight value W _lm (2) used for the second recognition process is 12. The weight value W _lm (3) used for the third recognition process is 11.5, and the smaller weight value W _lm (n) is used in the same manner as in the previous recognition process. This means that in the recognition process, the influence of restrictions by the language model is made smaller than in the previous recognition process.
When the process of step S40 is completed, the process returns to the process of step S34 to perform weighting of the language model by the new weighting coefficient _Wlm , and the process from step S35 onward is continued.

Here, the recognition processing is performed using the weight value W _lm for the language model, and this means that the ratio of the weight of the language model and the acoustic model is W _lm : 1. This is relatively the same even when the weight value of the language model is fixed at 1 and the weight value W _am of the acoustic model is used (that is, the ratio of the weight of the language model and the acoustic model is 1: W _am ). In this case, in this embodiment, even if the weight value W _am of the acoustic model is gradually increased every time the recognition process is repeated, the language model is similarly used as compared with the previous recognition process. This means that the influence of constraints is reduced. The same applies to the weight ratio between the language model and the acoustic model using the weight values W _lm and W _am, and the ratio of W _lm : W _am is the same. The effect of restrictions by the language model is made smaller.

Further, as a modification of the present embodiment, the weight value W _bias for the subtitle text corresponding to the program sound is gradually decreased as the recognition process is repeated while the weight value W _lm is fixed. However, in the current recognition process, a similar effect is obtained in that the language model is used with a relatively smaller weight than the acoustic model than in the previous recognition process.
Furthermore, as a modification of the present embodiment, even if both the weight value W _lm and the weight value W _bias are gradually reduced as the recognition process is repeated, the current recognition process is the same as the previous recognition process. A similar effect is obtained in that the language model is used with a relatively smaller weight than the acoustic model.

  In other words, in the present embodiment or its modification, the recognition processing unit 10 uses a predetermined weight ratio value between the language model read from the biased language model storage unit 6 and the acoustic model read from the acoustic model storage unit 9. The recognition process is carried out using the weights. The recognition processing unit 10 performs the recognition process again using the acoustic model updated by the acoustic model adaptation unit 15, and the weight ratio value used at this time is higher than that in the previous recognition process. The language model is a value used with a smaller weight.

Thereby, linguistic restrictions are reduced according to the degree of adaptation of the acoustic model, and this further improves the accuracy of learning of the acoustic model.
More specifically, in the first embodiment, in order to increase the recognition accuracy, recognition processing using linguistic restrictions stronger than the conventional method is performed. That is, the degree of weighting of the language model is made relatively large. However, by repeating the process of adapting the acoustic model, the recognition accuracy can be ensured with relatively small linguistic restrictions. Therefore, in this embodiment, linguistic restrictions are reduced according to the improvement of the accuracy of the acoustic model by iterative processing. By adjusting this linguistic restriction, it is possible to improve the detection system for the inconsistent section between subtitles and speech.

[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described. The present embodiment is characterized in that a speaker change or speaker attribute change is detected, and recognition processing is performed using an acoustic model dependent on the speaker or an acoustic model dependent on the speaker attribute. .

  FIG. 10 is a block diagram illustrating a functional configuration of the acoustic model learning device according to the present embodiment. In the illustrated configuration, the subtitle text storage unit 1, the morphological analysis processing unit 2, the weighting processing unit 3, the baseline corpus processing unit 4, the synthesis processing unit 5, and the biased language model storage unit 6 included in the acoustic model learning device 102 are weighted. The processing unit 7 and the program audio storage unit 8 are the same as those in the above-described embodiment. Furthermore, the acoustic model learning device 102 of the present embodiment has acoustic model storage units 9F and 9M, a recognition processing unit 10MF, recognition hypothesis storage units 11F and 11M, and alignment processing units 12F and 12M as its unique configuration. The selection processing units 13F and 13M, the learning data storage units 14F and 14M, the acoustic model adaptation units 15F and 15M, and the female utterance model storage unit 16F (utterance model storage unit, utterance model storage unit by speaker attribute), , A male voice utterance model storage unit 16M (speech model storage unit, utterance model storage unit by speaker attribute), utterance model adaptation units 17F and 17M, and a voice segment cutout unit 18 (speaker attribute estimation unit). Composed.

The female voice utterance model storage unit 16F stores a female voice utterance model including statistical acoustic features of female voices. The male voice utterance model storage unit 16M stores a male voice utterance model including statistical acoustic features of male voices. That is, the female utterance model storage unit 16F and the male voice utterance model storage unit 16M serve as speaker attribute-specific utterance model storage means for storing a speaker attribute-specific utterance model that represents a statistic for each speaker or each speaker attribute. Also works. Note that MFCC is used as the statistical acoustic feature quantity of female voice and male voice. As the statistical acoustic feature amount, PLP, HMM (Hidden Markov Model), formant frequency distribution, and the like can be used.
The voice segment cut-out unit 18 uses a female voice utterance model read from the female voice utterance model storage unit 16F and a male voice utterance model read from the male voice utterance model storage unit 16M, and is a portion of the program voice uttered by a female voice. And a portion uttered by male voice and a portion other than the utterance (for example, a silent portion, a background noise only portion, a music portion, a sound effect portion, etc. in the program). That is, the voice segment cut-out unit 18 detects whether or not the input voice is an utterance by a human, and if the voice is an utterance, detects the speaker attribute (here, the gender of the speaker). That is, the speech segment cutout unit 18 also functions as a speaker attribute estimation unit that estimates a speaker or a speaker attribute in a predetermined speech segment in the speech using the speaker attribute-specific speech model. Then, the voice segment cutout unit 18 passes the female voice segment and the male voice segment to the recognition processing unit 10MF in a cutout form.

  In addition, the acoustic model learning device 102 manages and stores a female voice and a male voice acoustic model separately. Specifically, the acoustic model storage unit 9F stores an acoustic model for female voices, and the acoustic model storage unit 9M stores an acoustic model for male voices. That is, the acoustic model storage units 9F and 9M store the acoustic model separately for each speaker or speaker attribute.

  Then, the recognition processing unit 10MF uses the female acoustic model read from the acoustic model storage unit 9F for the female voice segment passed from the voice segment cutout unit 18, and the male voice passed from the voice segment cutout unit 18. For the sections, recognition processing is performed using the male voice model read from the acoustic model storage unit 9M. That is, the recognition processing unit 10MF performs a recognition process on the voice in the voice section cut out by the voice section cutout unit 18. In other words, the recognition processing unit 10MF performs the speech recognition process using the corresponding speaker or speaker attribute acoustic model according to the estimation result by the speaker attribute estimation means. Then, the recognition processing unit 10MF writes the recognition result hypothesis obtained from the female voice segment to the recognition hypothesis storage unit 11F, and writes the recognition result hypothesis obtained from the male voice segment to the recognition hypothesis storage unit 11M.

  To detect voice segments and male / female speaker changes using gender-specific speech statistics (Male speech model, Female speech model), and perform recognition using gender-dependent acoustic models, Toru IMAI, Shoei SATO, Shinichi HOMMA, Kazuo ONOE, Akio KOBAYASHI "Online Speech Detection and Dual-Gender Speech Recognition for Captioning Broadcast News" (IEICE Transactions on Information and Systems 2007 E90-D (8): 1286-1291) Method is available.

The alignment processing unit 12F performs alignment processing similar to that in the above-described embodiment using only the recognition hypothesis of the female voice segment read from the recognition hypothesis storage unit 11F. Based on the processing result of the alignment processing unit 12F, the selection processing unit 13F selects a portion where the subtitle text and the recognition hypothesis are continuously matched as three or more words as a matching section, and stores learning data obtained from the matching section as learning data Write to part 14F. In addition, the selection processing unit 13F prunes the recognition hypotheses that do not match the caption text data in the recognition hypotheses including the matching sections selected from the female voice sections, and the recognition hypotheses after the pruning and the accompanying hypotheses On the basis of the reliability data to be selected, a section in which the reliability data is equal to or greater than a predetermined threshold is selected as a confidence section in the female voice section, and learning data obtained from the confidence section is written in the learning data storage unit 14F.
Similarly, the alignment processing unit 12M performs the same alignment processing as in the above-described embodiment using only the recognition hypothesis of the male voice segment read from the recognition hypothesis storage unit 11M. The selection processing unit 13M selects a location where the subtitle text and the recognition hypothesis match three or more words continuously based on the processing result by the alignment processing unit 12M, and writes the selection result in the learning data storage unit 14M. Further, the selection processing unit 13M prunes the recognition hypothesis that does not match the caption text data in the recognition hypothesis including the matching section selected from the male voice section, and the recognition hypothesis after the pruning and the accompanying hypothesis On the basis of the reliability data to be selected, a section in which the reliability data is equal to or greater than a predetermined threshold is selected as a confidence section in the male voice section, and learning data obtained from the confidence section is written in the learning data storage unit 14M.
In this way, dedicated learning data for women and men is obtained based on the female voice segment and male voice segment cut out by the voice segment cutout unit 18, respectively.

  The acoustic model adaptation unit 15F adapts (updates) the female acoustic model stored in the acoustic model storage unit 9F using only the female voice learning data read from the learning data storage unit 14F. Also, the acoustic model adaptation unit 15M adapts (updates) the male voice acoustic model stored in the acoustic model storage unit 9M using only the male voice learning data read from the learning data storage unit 14M. That is, the acoustic model adaptation units 15F and 15M update the acoustic model for each speaker or each speaker attribute corresponding to the estimation result using the learning data corresponding to the estimation result by the speaker attribute estimation unit. It is.

  The utterance model adaptation unit 17F adapts (updates) the female voice utterance model stored in the female voice utterance model storage unit 16F using only the female voice learning data read from the learning data storage unit 14F. The utterance model adaptation unit 17M adapts (updates) the male voice utterance model stored in the male voice utterance model storage unit 16F using only the male voice learning data read from the learning data storage unit 14M. That is, the utterance model adaptation units 17F and 17M use the learning data corresponding to the estimation result by the speaker attribute estimation unit, and the speaker corresponding to the estimation result stored in the speaker attribute-specific utterance model storage unit. Or a speaker attribute-specific utterance model for each speaker attribute.

  The acoustic model learning device 102 repeatedly performs the above-described series of recognition processing, alignment processing, selection processing, acoustic model adaptation processing, and utterance model adaptation processing.

With the configuration and operation as described above, the learning obtained in the case of using in combination with a method of segmenting a high-accuracy speech segment using statistical acoustic features (female utterance model and male utterance model) of input speech It is possible to repeat adapting the above-mentioned statistics used for extraction using data. Thereby, for example, a voice segment cut-off error that causes a noise segment to be recognized as speech speech is reduced, and the quality of the learning data can be further improved.
Also, by performing recognition processing using an acoustic model that depends on speaker attributes (gender), it is possible to obtain higher recognition accuracy than when using an acoustic model that does not depend on speaker attributes. Learning data can be created.
In addition, when a speech algorithm including a speaker or speaker attribute estimation means (speech segment extraction unit 18) is used in combination with the statistic of the input speech, the obtained learning data is used for speaker estimation. Statistic (female voice utterance model and male voice utterance model), and an acoustic model is provided for each speaker (or speaker attribute) (acoustic model storage units 9F and 9M), respectively. Therefore, the recognition accuracy can be further improved.

  As a further modification of the present embodiment, a configuration for managing the acoustic model for each program described in the second embodiment (at this time, the utterance model may be managed for each program as appropriate) A configuration in which the weighting of the language model is gradually changed as the processing described in the third embodiment is repeated may be used in combination.

  Also, instead of providing an utterance model for voice segment extraction and an acoustic model for recognition processing for each gender, or in addition to gender, an utterance model or acoustic model for each other speaker attribute or individual speaker May be provided to perform voice segment extraction processing or recognition processing for each speaker attribute or for each speaker individual. The “other speaker attribute” is, for example, an age group. At this time, the speaker's age group is, for example, boyhood (5 to 14 years), adolescence (15 to 24 years), middle age (25 to 44 years), middle age (45 to 64 years) ), And older age (65 years and older).

  Further, an utterance model or an acoustic model may be provided for each utterance environment. Here, “for each utterance environment” includes, for example, a case where the speaker is reading a manuscript, a conversational or negotiated form, and a chat.

  In this embodiment, the speech section is cut out using an utterance model, and an acoustic model is provided for each speaker attribute, and the voice section is cut out for each speaker attribute and recognized using a dedicated acoustic model. However, it is also possible to include only one of these configurations.

  In this embodiment, the recognition hypothesis is logically expressed by a lattice-structured graph in which linguistic units (morphemes) are arcs, but instead, phonemes are arced as linguistic units. It is also possible to perform the same processing by expressing it as a lattice structure graph.

<Embodiment using a computer system>
In addition, you may make it implement | achieve the function of the acoustic model learning apparatus in each embodiment mentioned above with a computer. In that case, the program for realizing the acoustic model learning function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer system and executed. good. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” dynamically holds a program for a short time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It is also possible to include those that hold a program for a certain time, such as a volatile memory inside a computer system serving as a server or client in that case. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.
For example, in the description of the first embodiment, it has been described that the selection processing unit 13 selects a section in which the caption text and the recognition hypothesis are continuously matched by three words or more. However, the selection processing unit 13 is not limited to “3”. You may make it select the area continuously located more than a predetermined number of words. Further, instead of using the number of words as a reference, a section that continuously matches a predetermined number of characters (for example, 5 characters) may be selected.

<Results of recognition experiment>
Here, a recognition experiment actually performed by the inventors will be described.
The learning data of the acoustic model was acquired by using 94 voices and subtitles of NHK “Kyoto Health” broadcasted from 2004 to 2007, and a recognition experiment was conducted by adapting the acoustic model with MLLR and MAP. For the evaluation voice, three broadcasts broadcast on June 25, 27 and 28, 2007 were used. The language model used in the recognition experiment is the addition of 94 subtitle texts of broadcasts other than the evaluation program weighted 100 times to the transcript of the news program (2 million sentences) and the life information program (670,000 sentences). I learned.
In this experiment, the language model for each program is not adapted, but a language model with higher accuracy can be used by adapting using the program related information.
The recognition processing unit (decoder) used in the recognition experiment detects a speech section using a monophone HMM for each gender and performs recognition while performing automatic gender determination (speaker attribute estimation). Baseline acoustic models are 340h (male voice) and 250h (female voice) news program voices, monophone HMM (3-state 32 mixture distribution) (speech model) for speech detection, and speech recognition A triphone HMM (4000 states, 3 states and 16 mixed distributions) was created for each gender. These monophones and triphone HMMs for each gender were adapted by learning data selected from the recognition results in the adaptive acoustic model for each program.

Then, learning data was created by repeating the adaptation and decoding (recognition processing) of the acoustic model at most 5 times with W _bias = 1000 and W _lm = 16 as weight values, and evaluation was performed for each number of repetitions. As a result, as the number of repetitions increased, the amount of learning data obtained increased and the rate of increase gradually decreased. The word misrecognition rate was the smallest when the number of repetitions was 3 to 4. From this experimental result, the number of repetitions of 3 or 4 is preferable.

  The evaluation program “Kyoto's Health” consists of a dialogue between a female announcer presenter (free speech), a female commentator (reading a manuscript), and a male guest (free speech). For the moderator and the commentator, since the same speaker's voice is included in the adaptation data, the effect of speaker adaptation is also expected. On the other hand, the guest with the largest number of speakers is a different speaker almost every time, so the effect of speaker adaptation cannot be expected, and only improvement by speech style adaptation can be obtained. As a result of performing recognition processing and measuring the correct word rate for each speaker, the recognition rate of the presenter and the commentator was greatly improved after adaptation compared to before adaptation of the acoustic model. The recognition rate of guest speakers (free utterances) was also improved.

  In addition, a recognition experiment was performed to confirm the usefulness of the selection processing unit which is one element of the configuration of the present invention. When all recognition results (recognition hypotheses) are collected and used for acoustic model adaptation without selection by the selection processing unit, the amount of learning data obtained is improved even if the recognition processing and adaptation processing are repeated. In addition, the word recognition rate was not improved. That is, due to this fact, the acoustic model learning device of the present invention includes the selection processing unit, and the adaptation of the acoustic model is performed using the learning data selected by the selection processing unit. It was also confirmed that

It is the block diagram which showed the function structure of the acoustic model learning apparatus by the 1st Embodiment of this invention. It is the schematic which showed the data structure of the language model used in the embodiment. It is the schematic which showed the data structure of the acoustic model used in the embodiment. It is the schematic which showed the data structure and data example of the recognition hypothesis memory | storage part used in the embodiment, and the lattice structure which the data represents. It is the flowchart which showed the processing procedure of the acoustic model learning by the same embodiment. It is the schematic which showed the data example after the pruning process of the recognition hypothesis memory | storage part used in the same embodiment, and the lattice structure which the data represents. It is the schematic which showed the outline of the alignment process in the process of the acoustic model learning by the same embodiment, and a selection process. It is the block diagram which showed the function structure of the acoustic model learning apparatus by the 2nd Embodiment of this invention. It is the flowchart which showed the process sequence of the acoustic model learning apparatus by the 3rd Embodiment of this invention. It is the block diagram which showed the function structure of the acoustic model learning apparatus by the 4th Embodiment of this invention. It is the block diagram which showed the function structure of the acoustic model learning apparatus by a prior art.

Explanation of symbols

1 Caption text storage unit 2 Morphological analysis processing unit 3 Weighting processing unit 4 Baseline corpus storage unit 5 Composition processing unit 6 Biased language model (Biased LM) storage unit (language model) Storage unit)
7 Weighting processing unit 8 Program audio storage unit 9, 9F, 9M Acoustic model (AM) storage unit 10, 10MF Recognition processing unit 11, 11F, 11M Recognition hypothesis storage unit 12, 12F, 12M Alignment ( Alignment) processing unit 13, 13F, 13M Selection processing unit 14, 14F, 14M Learning data (Transcripts) storage unit 15, 15F, 15M Acoustic model adaptation unit (Adaptation)
16F Female speech model storage unit (speech model-specific speech model storage means)
16M Male speech model storage unit (speech model storage means by speaker attributes)
17F, 17M Utterance model adaptation unit (Adaptation)
18 Voice segment cutout part (speaker attribute estimation means)
100, 101, 101A to 101D, 102 Acoustic model learning apparatus

Claims (3)

A recognition processing unit that performs speech recognition processing and outputs recognition result data including a recognition hypothesis and reliability data associated with the recognition hypothesis;
Based on the text data corresponding to the speech and the recognition result data output by the recognition processing unit, the matching hypothesis of both the data is selected, and the recognition hypothesis that includes the time of the matching section and does not match the text data Pruning, selecting a confidence interval based on the recognition hypothesis after pruning and accompanying reliability data, and a selection processing unit that outputs the data of the coincidence interval and the confidence interval;
A sound processing apparatus comprising: The sound processing apparatus according to claim 1,
An acoustic model storage unit that stores an acoustic model in which a phoneme and an acoustic feature corresponding to the phoneme are associated;
An acoustic model adaptation unit that updates the acoustic model stored in the acoustic model storage unit according to a result of selection by the selection processing unit;
And further comprising
The recognition processing unit performs speech recognition processing using the acoustic model read from the acoustic model storage unit,
The selection processing unit outputs a set of the phoneme included in the selected matching section and the selected confidence section and the acoustic feature amount corresponding to the phoneme as learning data,
The acoustic model adaptation unit updates the acoustic model stored in the acoustic model storage unit using the learning data output from the selection processing unit.
A sound processing apparatus. A recognition process for performing speech recognition processing and outputting recognition result data including a recognition hypothesis and reliability data associated with the recognition hypothesis;
Based on the text data corresponding to the speech and the recognition result data output by the recognition processing step, a matching section between the two data is selected, and the recognition hypothesis that includes the time of the matching section and does not match the text data Pruning, selecting a confidence interval based on the recognition hypothesis after pruning and the associated reliability data, and outputting the data of the matching interval and the confidence interval; and
A program that causes a computer to execute this process.

Images