JP7222828B2 - Speech recognition device, speech recognition method and storage medium - Google Patents

Description

The present invention relates to a speech recognition apparatus and a speech recognition method for recognizing individual speeches in speech data in which speeches of a plurality of speakers are mixed and simultaneously identifying the speaker of each speech.

When performing speech recognition on speech data in which speeches of a plurality of speakers are mixed, it is conceivable to separate the speech (sound source) for each speaker. Non-Patent Documents 1 and 2 are known as techniques for separating sound sources. These are called "source separations".

Speech recognition processing can be performed on the separated speech. A wide range of means are known for speech recognition. Non-Patent Document 3 is known as a typical non-patent document.

Furthermore, there is known a technique for identifying which speaker made an utterance by extracting a feature quantity representing the speaker's character of the separated voices and clustering them according to the speaker's character of each sound. This is called "speaker identification". Techniques for speaker identification are widely known. Non-Patent Document 4 is known as a typical non-patent document for speaker identification.

On the other hand, Patent Document 1 is known as a technique of obtaining a sample of a speaker in advance and removing a voice close to the sample of the speaker. In addition, Non-Patent Document 5 and Non-Patent Document 6 are known as techniques for recognizing only speech that is close to the voice quality of a speaker's sample. These are called "target speaker speech recognition".

JP 2016-042152 A

Chao Weng et al., “Deep Neural Networks for Single-Channel Multi-Talker Speech Recognition”, IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 23, NO. 10, OCTOBER 2015 John R. Hershey, Zhuo Chen, Jonathan Le Roux, Shinji Watanabe, “Deep clustering: Discriminative embeddings for segmentation and separation” Rabiner, Lawrence R., Biing-Hwang Juang, and Janet C. Rutledge. Fundamentals of speech recognition. Vol. 14. Englewood Cliffs: PTR Prentice Hall, 1993. Anguera, Xavier, et al. "Speaker diarization: A review of recent research." IEEE Transactions on Audio, Speech, and Language Processing 20.2 (2012): 356-370. Zmolikova, Katerina, et al. "Speaker-Aware Neural Network Based Beamformer for Speaker Extraction in Speech Mixtures." Interspeech. 2017. Delcroix, Marc, et al. "Single channel target speaker extraction and recognition with speaker beam." 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018.

When performing speech recognition and speaker identification on speech data in which the speech of multiple speakers is mixed, first, after applying sound source separation technology, speech recognition processing and speaker identification processing are performed on the separated individual speech. There is a way to do it. However, in this technology, the sound source separation, speech recognition, and speaker identification processes are individually optimized, and there is a problem that the speech recognition accuracy and the speaker identification accuracy are poor when each process is combined.

On the other hand, if the above-described target speaker speech recognition is used, speech recognition can be performed accurately while identifying the target speaker, but there is a problem that a sample of the target speaker is required in advance.

Thus, there is no method that can accurately perform speech recognition and speaker identification on speech data in which speeches of multiple speakers are mixed without requiring a target speaker sample in advance. rice field.

The present invention is a speech recognition apparatus having a processor and a memory, which receives speech data in which speeches of a plurality of speakers are mixed, and extracts an initial speaker sample from the speech data as an initial value of the speaker sample. A speaker sample initial value calculator, a target speaker speech recognition unit that performs target speaker speech recognition based on the initial speaker sample extracted by the speaker sample initial value calculator, and the target speaker speech recognition. and a speaker sample update unit that extracts speaker samples again based on the execution result of and updates the initial speaker sample, wherein the target speaker speech recognition unit is updated by the speaker sample update unit. The target speaker speech recognition is performed again based on the obtained speaker samples.

Therefore, according to the present invention, speech recognition and speaker identification can be accurately performed from speech data of a plurality of speakers without requiring speaker samples in advance.

The details of at least one implementation of the subject matter disclosed in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the disclosed subject matter will become apparent from the following disclosure, drawings, and claims.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows Example 1 of this invention and shows an example of a speech recognition apparatus. 1 is a diagram showing Embodiment 1 of the present invention and showing an example of speech recognition and speaker identification performed by a speech recognition device; FIG. FIG. 2 shows Embodiment 1 of the present invention and shows an example of a target speaker speech recognition unit; 4 is a flow chart showing Example 1 of the present invention and showing an example of processing of a target speaker speech recognition unit; 4 is a flow chart showing Example 1 of the present invention and showing an example of processing of a speaker sample initial value calculator. 4 is a flow chart showing Example 1 of the present invention and showing an example of processing of a speaker sample updating unit; It is a figure which shows Example 2 of this invention and shows an example of the speech recognition and speaker identification which are performed by a speech recognition apparatus. It is a figure which shows Example 2 of this invention and shows an example of the target speaker speech recognition part which has an auxiliary|assistant output. FIG. 12 shows Embodiment 3 of the present invention and shows an example of speech recognition and speaker identification performed by a speech recognition device. FIG. 4 shows the first embodiment of the present invention, and shows an example of a GUI for receiving parameters for speech recognition and speaker identification performed by a speaker sample initial value calculator;

An embodiment of the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is a block diagram showing a first embodiment of the present invention and showing an example of a speech recognition device 100. As shown in FIG. A speech recognition apparatus 100 is composed of a computer including a processor 1 , a memory 2 , a storage device 3 , an input device 4 , an output device 5 and a network interface 6 .

A speaker sample initial value calculation unit 40 , a speaker sample update unit 41 , a target speaker speech recognition unit 42 and a convergence determination unit 43 are loaded into the memory 2 as programs and executed by the processor 1 . The storage device 3 stores data and the like used by each program. In the first embodiment, speech recognition data 32, an acoustic model 33, a language model 34, and a speaker feature extraction model 35 are stored. be done.

The input device 4 is composed of a mouse, a keyboard, a touch panel, or the like. The output device 5 is configured by a display or the like. The network interface 6 is connected to a network (not shown).

The processor 1 operates as a functional unit that provides a predetermined function by processing according to the program of each functional unit. For example, the processor 1 functions as the target speaker speech recognition unit 42 by processing according to the target speaker speech recognition program. The same is true for other programs. Furthermore, the processor 1 also operates as a functional unit that provides functions of multiple processes executed by each program. Computers and computer systems are devices and systems that include these functional units.

Information such as programs and tables that realize each function of the speech recognition apparatus 100 is stored in storage devices such as the storage device 3, nonvolatile semiconductor memory, hard disk drive, SSD (Solid State Drive), IC card, SD card, and DVD. It can be stored in a computer-readable non-transitory data storage medium such as.

FIG. 2 is a diagram showing an example of processing of the speech recognition apparatus 100. As shown in FIG. In this embodiment, an example in which recognition speech data 32 is stored in the storage device 3 in advance is shown. Here, the speech data for recognition 32 is speech data in which speeches of a plurality of speakers are mixed.

First, the speaker sample initial value calculator 40 receives the recognition speech data 32, and initial values of speaker samples (speech samples) expected to include the speaker included in the speech (hereinafter referred to as initial speaker sample 203) is generated (step 202). The speaker sample initial value calculator 40 generates initial speaker samples 203 corresponding to the number of speakers included in the speech.

The number of speakers included in the recognition speech data 32 may be input by the user of the speech recognition apparatus 100, or may be automatically estimated. Details of the processing of the speaker sample initial value calculator 40 will be described later with reference to FIG.

Subsequently, the target speaker speech recognition unit 42 performs speech recognition on the recognition speech data 32 so that only the speech of the same speaker as the initial speaker sample 203 is recognized (step 204). As a result, a speech recognition result 206 corresponding to the initial speaker sample 203 is obtained.

Here, the speech recognition result 206 output by the target speaker's speech recognition unit 42 includes information obtained by transcribing the utterance content of the speech into text, and the start (start) time and end (end) time of each speech. Details of the configuration and processing of the target speaker speech recognition unit 42 will be described later with reference to FIGS.

Subsequently, the convergence determination unit 43 determines whether or not to end the processing based on a predetermined convergence condition (step 207). This processing is always determined as No at the first time.

In the second and subsequent determinations, the convergence determination unit 43, for example, if the degree of matching between the speech recognition result 206 in the previous iteration and the character string of the latest speech recognition result 206 is equal to or greater than a predetermined threshold, the result of the convergence determination is can be determined to be Yes. Alternatively, processing may be introduced to determine Yes when the convergence determining unit 43 is called a predetermined number of times.

If the determination result of the convergence determination unit 43 is No, the speaker sample update unit 41 is executed to generate updated speaker samples 209 (step 208). Details of processing in the speaker sample update unit 41 will be described later with reference to FIG.

Each updated speaker sample 209 is input to the target speaker speech recognition unit 42, and target speaker speech recognition processing is executed again (204). After that, the above processing is repeated until the convergence determination unit 43 determines Yes, and when the determination is Yes, the speech recognition result 211 for each speaker is output.

Although the convergence determination unit 43 has been described above as an independent processing unit, it is not limited to this and may be implemented as a part of the target speaker speech recognition unit 42 .

FIG. 3 shows an example of components of the target speaker speech recognition unit 42. As shown in FIG. The target speaker's speech recognition unit 42 is composed of the speech feature amount calculation unit 10, the speaker feature amount calculation unit 20, and the decoding unit 316, and the acoustic model 33 and the language model 34 are used for calculation.

The acoustic model 33 is a mathematical model including the output probability distribution of phonemes calculated by the neural network. The acoustic model 33 is configured by, for example, a neural network, and has an input layer 304 that receives the speech feature amount 303 and an input layer 309 that receives the speaker feature amount 315 .

Also, from the output layer 308 of the acoustic model 33, an output probability distribution of each state of HMM (Hidden Markov Model) is calculated. In the case of target speaker speech recognition, the network parameters are trained such that the phoneme probabilities of the same speaker as the speaker sample 209 (or the initial speaker sample 203) are output. As for this learning method, the method disclosed in Non-Patent Document 6 or the like can be used.

The language model 34 is composed of symbol sequences such as phonemes and words. The decoding unit 316 uses the phoneme probability output from the acoustic model 33 and the language score calculated from the language model 34 to output the word string with the highest probability as the text of the recognition result.

As a network used in the acoustic model 33, for example, RNN (Recurrent Neural Network) or LSTM (Long short-term memory) can be used. Alternatively, it may be a DNN (Deep Neural Network).

Although blocks 305, 306, 307, 310, and 311 are set as hidden layers in FIG. 3, the configuration is not limited to this, and a network with a desired configuration can be used. .

It is also possible to integrate the acoustic model 33 and the language model 34 and use one neural network for predicting character strings from voice data. In this case, the output of the neural network is a string of characters rather than phoneme probabilities. The input layer 304 that receives the speech feature 303 and the input layer 309 that receives the speaker feature 315 are the same.

FIG. 4 shows an example of processing performed by the target speaker speech recognition unit 42 . This process is the process performed in step 202 in FIG.

First, the target speaker's speech recognition unit 42 receives the recognition speech data 32 (step 401), and the speech feature quantity calculation unit 10 calculates the speech feature quantity 303 of the recognition speech data 32 (step 402).

The speech feature quantity 303 output by the speech feature quantity calculation unit 10, for example, processes the input recognition speech data 32 by FFT (Fast Fourier Transform), and performs MFCC (Mel Frequency Cepstral Coefficient: Calculate the mel-frequency cepstrum coefficients). Then, the speech feature amount calculation unit 10 can calculate the MFCC difference in units (frames) of a predetermined time interval (for example, 10 msec) and obtain the speech feature amount 303 . Note that the speech feature amount 303 of the recognition speech data 32 is not limited to the above, and a well-known or known technique may be applied.

In parallel with step 401, the target speaker speech recognition unit 42 receives the speaker sample 209 (or the initial speaker sample 203) of the target speaker (step 403), and the speaker feature amount calculation unit 20 calculates the A speaker feature quantity 315 is calculated from the speaker sample 209 (or the initial speaker sample 203) (step 404).

The speaker sample received by the target speaker speech recognition unit 42 is the initial speaker sample 203 calculated by the speaker sample initial value calculation unit 40 for a new speaker, and the initial speaker sample 203 for an existing speaker. receives the speaker sample 209 updated by the speaker sample update unit 41 .

The speaker feature quantity 315 output by the speaker feature quantity calculation unit 20, for example, processes the input speaker sample 209 by FFT, and calculates MFCC (Mel Frequency Cepstral Coefficient) based on the FFT processing result. coefficient), and the difference is calculated in units (frames) of a predetermined time interval (for example, 10 msec). Then, the speaker feature amount calculation unit 20 can further calculate the speaker feature amount 315 by calculating an i-vector using the MFCC and the pre-calculated Gaussian Mixture Model.

As a method for calculating i-vector, a known or well-known method may be applied. For example, "Front-end factor analysis for speaker verification." Processing 19.4 (2010): 788-798.) and is widely known. Note that the speaker feature amount 315 is not limited to the above, and other well-known or known methods may be applied.

Finally, the decoding unit 316 uses the probability of the phoneme output from the acoustic model 33 based on the speech feature amount 303 and the speaker feature amount 315 and the language score calculated from the language model 34 to determine the highest probability. The word string is output as the text of the recognition result (step 405).

As a result, only the voice of the same speaker as the speaker sample 209 (or the initial speaker sample 203) among the multiple voices included in the speech recognition data 32 is transcribed into text. The transcript includes start and end times for each utterance.

In the above example, steps 401 and 402 and steps 403 and 404 are executed in parallel, but the present invention is not limited to this and may be executed sequentially.

FIG. 5 is a flow chart showing an example of processing performed by the speaker sample initial value calculator 40. As shown in FIG. This process is performed at step 202 in FIG.

First, the speaker sample initial value calculator 40 receives recognition speech data 32 (step 501), and then divides the recognition speech data 32 into a plurality of speech segments (step 502).

In the present embodiment, it is possible to use a method of automatically dividing into specified audio lengths (for example, 2 seconds), a method of assuming silence when the volume is low, and dividing when the volume is low.

Subsequently, the speaker sample initial value calculation unit 40 calculates the speaker feature amount 315 from each divided speech data (step 503). The speaker feature quantity 315 of this embodiment may be the same as the speaker feature quantity calculation unit 20 (see FIG. 3) used in the target speaker speech recognition unit 42, or may be different. You can use it.

Finally, the speaker sample initial value calculator 40 clusters the speaker feature quantities 315 calculated for each divided speech section (step 504). The speaker sample initial value calculator 40 outputs all or a predetermined part of the speech corresponding to the speaker feature quantity 315 determined to be of the same class as the initial speaker sample 203 . For the clustering processing used in this embodiment, for example, known or well-known methods such as the K-means method and the agglomerative clustering method can be used.

In addition, the speaker sample initial value calculator 40 can use only a part of the speech data for recognition 32 instead of using all of it. For example, it is conceivable to use the first 10% of the recognition speech data 32, or to use only the first minute. By doing so, the processing can be made lighter than using the entire speech.

Furthermore, it is also possible to allow the user to specify settings for audio data to be used using a GUI such as that shown in FIG. FIG. 10 is a diagram showing an example of a GUI for receiving parameters for speech recognition and speaker identification performed by the speaker sample initial value calculator 40. In FIG.

In the GUI shown in FIG. 10, the data input section 1001 allows the user of the speech recognition apparatus 100 to specify what percentage of the recognition speech data 32 to use for the initial speaker detection.

Through the above processing, the speaker sample initial value calculation unit 40 outputs the new speaker's speech data as the initial speaker sample 203 and inputs it to the target speaker speech recognition unit 42 .

FIG. 6 is a flow chart showing an example of processing performed by the speaker sample updating unit 41. As shown in FIG. This process is performed at step 208 in FIG.

The speaker sample update unit 41 receives recognition speech data 32 (step 601). At the same time, the recognition result of the target speaker's speech recognition unit 42 is received (step 602), and the section of the recognition speech data 32 that does not overlap with other speakers is calculated for each speaker (step 603).

Step 603 can be easily calculated by using the information on the start time and end time of each utterance included in the speech recognition result. Finally, all or part of the segmented recognition speech data 32 based on the segment information obtained in step 603 is output as updated speaker samples 209 (604). The above process is repeatedly executed up to the number of speakers. In the above, an example of executing the processing of step 601 and steps 602 and 603 in parallel was shown, but the processing is not limited to this, and may be executed sequentially.

The speaker sample updating unit 41 can use the reliability included in the output result of the target speaker speech recognizing unit 42 instead of calculating the non-overlapping utterance section from the recognition speech data 32 .

In this case, the speaker sample updating unit 41 sets x _tgt to be the reliability calculated by the target speaker speech recognition unit 42 targeting the target speaker for generating the speaker sample 209 for a certain speech segment, and Assuming that the reliability calculated by the target speaker speech recognition unit 42 for speakers other than the target speaker is xi for speaker _i , the weighting factor of the section is calculated by the following equation.

When the reliability is between 0 and 1, the weighting factor is set to the smaller value of the value obtained by subtracting the reliability x _i of speaker i from 1 and the reliability x _tgt .

The speaker sample update unit 41 can also generate speaker samples 209 to which weighting factors calculated from the above equation are added. In this case, the target speaker's speech recognition unit 42 can perform the target speaker's speech recognition according to the weighting factor by calculating the speaker feature quantity 315 by weighted average according to the weighting factor. can.

Note that the reliability x _tgt calculated by the speaker sample update unit 41 is a value indicating the likelihood of the target speaker. (L. Mangu, E. Brill, A. Stolcke, Computer Speech & Language, Volume 14, Issue 4, October 2000, Pages 373-400) or the like can be applied.

In the above equation, 1-max(x _i ) from the reliability x _i calculated by the target speaker speech recognition unit 42 targeting speakers other than the target speaker is replaced by a value indicating the unity of the speaker (single It can be calculated and used as one speaker degree). This single-speaker degree may be added to the speaker sample 209 as the weighting factor.

Through the series of processes described above, the speech recognition apparatus 100 extracts the initial speaker sample 203 from the speech included in the recognition speech data 32, executes the target speaker speech recognition unit 42, and further performs the target speaker speech recognition. speaker samples 209 for updating the initial speaker samples 203 are generated based on the results of (2), and the target speaker speech recognition unit 42 is executed with the updated speaker samples 209 .

As a result, speech recognition and speaker identification can be realized with high accuracy from the speech of a plurality of speakers without requiring the initial speaker sample 203 in advance.

FIG. 7 shows a second embodiment of the present invention, showing an example of processing using the target speaker speech recognition unit 42 with auxiliary output. FIG. 8 shows the configuration of the target speaker speech recognizer 42 with auxiliary output.

First, the target speaker speech recognition unit 42 having auxiliary output will be described with reference to FIG. The configuration of the target speaker speech recognition unit 42 of the second embodiment is almost the same as that of the target speaker speech recognition unit 42 shown in FIG. The difference is that a hidden layer 801 and an output layer 802 are added.

From the output layer 802, phoneme probabilities for speakers other than the speaker sample (speech sample) of the target speaker are calculated. By outputting the word string with the highest combination of the phoneme probability and the language score obtained from the language model 34, it is possible to obtain the speech recognition result of the speaker other than the speaker sample (block 803). Note that the network configuration related to the auxiliary output can have a desired configuration other than that shown in the figure.

Next, the flow of processing will be described according to FIG. First, the speaker sample initial value calculator 40 receives the recognition speech data 32 and generates an initial speaker sample 702 expected to include the speaker included in the speech.

The processing flow of the speaker sample initial value calculator 40 is the same as that of the first embodiment, but in this embodiment, only the initial speaker sample 702 of one speaker needs to be generated. For example, among the initial speaker samples 702 calculated by the speaker sample initial value calculator 40, it is possible to select only the speaker with the longest speech length.

Subsequently, the target speaker speech recognition unit 42 with auxiliary output recognizes the speech of the same speaker as the initial speaker sample 702 (speaker A in the figure) and the speech of other speakers (auxiliary output in the figure). output the speech recognition result 705 for each. Here, the recognition result includes information on start time and end time for each utterance.

Subsequently, the convergence determination unit 43 determines whether or not a predetermined convergence condition is satisfied for the end of processing (step 706). This process is always determined as No at the first time. For the second and subsequent times, for example, if the degree of matching between the speech recognition result 705 in the previous iteration and the character string of the latest speech recognition result 705 is equal to or greater than a predetermined threshold, the result of the convergence determination is determined to be Yes. can be done. Alternatively, a process may be introduced that determines Yes when the convergence determination process is called a predetermined number of times.

If the determination result of the convergence determination unit 43 is No, the speaker sample update unit 41 is executed to generate updated speaker samples 708 . The processing in the speaker sample updating unit 41 is the same as that in FIG. 6, but differs in that the section where the speakers do not overlap is calculated using the start time and end information obtained from the auxiliary output. In other words, the section is a section in which a single speaker speaks.

Based on each updated speaker sample 708, target speaker speech recognition 703 is performed again. Thereafter, the process is repeated until the convergence determination process (706) determines Yes, and when the determination is Yes, the speech recognition result for each speaker is output.

With the above processing, the speech recognition apparatus 100 can accurately recognize the speech of the same speaker as the speaker sample and the speech of a speaker other than the speaker sample from the speech included in the recognition speech data 32. can. This makes it possible to achieve highly accurate speech recognition and speaker identification from the speech of multiple speakers without requiring speaker samples in advance.

FIG. 9 shows the third embodiment and shows an example of speech recognition and speaker identification performed by the speech recognition apparatus 100. In FIG.

The speech recognition apparatus 100 of the third embodiment uses the target speaker's speech recognition unit 42 having an auxiliary output as in the second embodiment. An example of sequentially outputting recognition results by processing the speech data for recognition 32 in units will be shown.

First, the speaker sample initial value calculator 40 receives the recognition speech data 32 (901) and generates an initial speaker sample 902 expected to include the speaker included in the speech. This processing is basically the same as that of the second embodiment, but differs in that only a predetermined section from the beginning is used instead of using all of the recognition speech data 32 .

In this embodiment, the predetermined interval is, for example, after the voice power (for example, dB) exceeds a predetermined threshold (assumed that the voice is uttered) from the start time of the recognition voice data 32, and after the predetermined A section (assumed to be silent) can be used until the time when the threshold was below the threshold reaches a certain value or above. Alternatively, a section of a preset length (time) from the beginning of the speech data for recognition 32 may be used.

Subsequently, the target speaker speech recognition unit 42 having auxiliary output recognizes the speech of the same speaker as the initial speaker sample 902 (speaker A in the figure) and the speech of other speakers (auxiliary output in the figure). , a recognition result 904 is output only for a predetermined section from the beginning (903). Here, the recognition result includes information on start time and end time for each utterance.

Subsequently, the speaker sample update unit 41 is executed (905) to generate updated speaker samples 906. FIG. This processing is the same as that of the second embodiment, but differs in that it uses only a predetermined section from the beginning as described above, instead of using all of the recognition speech data 32 .

Based on the updated speaker samples 906, the target speaker speech recognition unit 42 is activated again (907). Here, speech recognition is executed only for a predetermined section from the time processed in the previous step in the recognition speech data 32 (908). That is, the target speaker's speech recognition unit 42 executes the target speaker's speech recognition for a predetermined section from the beginning of the unrecognized section in the target section for speech recognition.

Thereafter, updating of the speaker sample 906 and target speaker speech recognition are repeated until the end time of the recognition speech data 32 is reached.

Through the series of processes described above, speech recognition and speaker identification can be performed sequentially from the beginning of the recognition speech data 32 . This makes it possible to achieve highly accurate speech recognition and speaker identification from the speech of multiple speakers without requiring speaker samples in advance.

In the second and third embodiments, it is also possible to allow the user to designate a predetermined section from the beginning of the speech data for recognition 32 using a GUI as shown in FIG. 10 of the first embodiment. In this embodiment, the method specifying unit 1002 makes it possible to select either the second embodiment in which the entire recognition speech data 32 is used or the third embodiment in which the recognition speech data 32 is sequentially recognized.

<Conclusion>
As described above, the speech recognition apparatuses of the first to third embodiments can be configured as follows.

(1) A speech recognition device (100) having a processor (1) and a memory (2), which receives speech data (32) in which speeches of a plurality of speakers are mixed, and from said speech data (32) A speaker sample initial value calculation unit (40) for extracting an initial value of a speaker sample as an initial speaker sample (203), and an initial speaker sample (203) extracted by the speaker sample initial value calculation unit (40) a target speaker speech recognition unit (42) for performing target speaker speech recognition based on the target speaker speech recognition, and a speaker sample (209) is extracted again based on the execution result of the target speaker speech recognition to obtain the initial speaker sample (203), and the target speaker speech recognition unit (42) updates the speaker sample (209) updated by the speaker sample update unit (41). re-perform the target speaker speech recognition based on .

With the above configuration, the speech recognition apparatus 100 extracts the initial speaker sample 203 from the speech included in the recognition speech data 32, executes the target speaker speech recognition unit 42, Based on this, speaker samples 209 for updating the initial speaker samples 203 are generated, and the target speaker speech recognition unit 42 is executed with the updated speaker samples 209 .

As a result, the speech recognition apparatus 100 does not require the initial speaker sample 203 in advance, and can achieve highly accurate speech recognition and speaker identification from voices of a plurality of speakers.

(2) In the speech recognition apparatus (100) according to (1) above, the target speaker speech recognition unit (42) and the speaker sample update unit (41) repeat until a predetermined convergence condition is satisfied. executed.

With the above configuration, the speaker sample update unit 41 repeatedly updates the speaker samples 209 until the convergence determination unit 43 satisfies a predetermined condition, and executes target speaker speech recognition with the updated speaker samples 209. This makes it possible to achieve highly accurate speech recognition and speaker identification.

(3) In the speech recognition apparatus (100) according to (1) above, the speaker sample update unit (41) updates the target speaker's speech recognition execution result by a single speaker. The speaker sample (209) is updated by extracting the interval where the speaker is present.

With the above configuration, it is possible to generate a highly accurate speaker sample 209 by extracting the target speaker sample from the interval where there is no utterance by another speaker.

(4) In the speech recognition apparatus (100) according to (1) above, the speaker sample updating unit (41) calculates the reliability of target speaker speech recognition, and weights a weighting factor based on the reliability. is calculated and the weighting factor is added to the speaker sample (209).

With the above configuration, the target speaker's speech recognition unit 42 executes target speaker's speech recognition according to the weighting factor by calculating the speaker feature quantity 315 by weighted average according to the weighting factor. can be done.

In addition, the present invention is not limited to the above-described embodiments, and includes various modifications. For example, the above embodiments are described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the described configurations. In addition, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, addition, deletion, or replacement of other configurations for a part of the configuration of each embodiment can be applied singly or in combination.

Further, each of the above configurations, functions, processing units, processing means, and the like may be realized by hardware, for example, by designing them in an integrated circuit. Further, each of the above configurations, functions, etc. may be realized by software by a processor interpreting and executing a program for realizing each function. Information such as programs, tables, and files that implement each function can be stored in recording devices such as memories, hard disks, SSDs (Solid State Drives), or recording media such as IC cards, SD cards, and DVDs.

Further, the control lines and information lines indicate those considered necessary for explanation, and not all control lines and information lines are necessarily indicated on the product. In practice, it may be considered that almost all configurations are interconnected.

1 Processor 2 Memory 3 Storage Device 4 Input Device 5 Output Device 10 Speech Feature Amount Calculator 20 Speaker Feature Amount Extractor 32 Recognition Speech Data 33 Acoustic Model 34 Language Model 35 Speaker Feature Amount Extraction Model 40 Initial Speaker Sample Value calculation unit 41 Speaker sample update unit 42 Target speaker speech recognition unit

Claims (9)

A speech recognition device having a processor and a memory,
a speaker sample initial value calculation unit that receives voice data in which voices of a plurality of speakers are mixed, and extracts an initial speaker sample from the voice data as an initial value of the speaker sample;
a target speaker speech recognition unit that performs target speaker speech recognition based on the initial speaker sample extracted by the speaker sample initial value calculation unit;
a speaker sample updating unit that re-extracts speaker samples based on execution results of the target speaker speech recognition and updates the initial speaker samples;
The target speaker speech recognition unit,
A speech recognition apparatus, wherein the target speaker's speech recognition is performed again based on the speaker sample updated by the speaker sample updating unit. The speech recognition device according to claim 1,
A speech recognition apparatus, wherein the target speaker speech recognition unit and the speaker sample update unit are repeatedly executed until a predetermined convergence condition is satisfied. The speech recognition device according to claim 1,
The speaker sample update unit includes:
A speech recognition apparatus, wherein a section in which a single speaker speaks is extracted from the execution result of the speech recognition of the target speaker, and the speaker sample is updated. The speech recognition device according to claim 1,
The speaker sample update unit includes:
1. A speech recognition apparatus, comprising: calculating a reliability of speech recognition of a target speaker; calculating a weighting factor based on the reliability; and adding the weighting factor to the speaker sample. A speech recognition method for a speech recognition device having a processor and a memory, comprising:
a speaker sample initial value calculation step in which the speech recognition device receives speech data in which speeches of a plurality of speakers are mixed, and extracts an initial speaker sample initial value from the speech data as an initial speaker sample;
a target speaker speech recognition step in which the speech recognizer performs target speaker speech recognition based on the extracted initial speaker sample;
a speaker sample update step in which the speech recognition device re-extracts speaker samples based on execution results of the target speaker speech recognition and updates the initial speaker samples;
The target speaker speech recognition step includes:
A speech recognition method, wherein the target speaker speech recognition is performed again based on the speaker samples updated in the speaker sample update step. A speech recognition method according to claim 5,
The speech recognition method, wherein the target speaker speech recognition step and the speaker sample update step are repeatedly executed until a predetermined convergence condition is satisfied. A speech recognition method according to claim 5,
The speaker sample update step includes:
A speech recognition method comprising: extracting a section in which a single speaker is speaking from the result of execution of the target speaker's speech recognition, and updating the speaker sample. A speech recognition method according to claim 5,
The speaker sample update step includes:
A speech recognition method, comprising calculating a reliability of the speech recognition of the target speaker, calculating a weighting factor based on the reliability, and adding the weighting factor to the speaker sample. A computer having a processor and a memory, and a storage medium storing a program for executing speech recognition,
a speaker sample initial value calculation step of receiving voice data in which voices of a plurality of speakers are mixed, and extracting initial values of speaker samples from the voice data as initial speaker samples;
a target speaker speech recognition step of performing target speaker speech recognition based on the extracted initial speaker sample;
a speaker sample update step of re-extracting speaker samples based on the target speaker speech recognition execution result and updating the initial speaker samples;
The target speaker speech recognition step includes:
A non-temporary computer-readable storage medium storing a program for causing the computer to re-execute the target speaker speech recognition based on the speaker samples updated in the speaker sample updating step.

Images