JP7650132B2 - Learning method, speaker identification method, and program - Google Patents

Description

This disclosure relates to technology for identifying speakers.

Technology for identifying speakers using a speaker identification model is known (see, for example, Non-Patent Document 1).

David Snyder, Daniel Garcia-Romero, Gregory Sell, Daniel Povey, Sanjeev Khudanpur, “X-VECTORS: ROBUST DNN EMBEDDINGS FOR SPEAKER RECOGNITION” ICASSP 2018:5329-5333.

I want to identify speakers with high accuracy.

A learning method according to one aspect of the present disclosure is a method for learning a speaker identification model that, when voice data is input, outputs speaker identification information that identifies the speaker of an utterance contained in the voice data, and performs a voice conversion process on first voice data of a first speaker to generate second voice data of a second speaker, and performs a learning process of the speaker identification model using the first voice data and the second voice data as learning data.

A speaker identification method according to one aspect of the present disclosure inputs voice data to the speaker identification model that has been trained in advance using the training method described above, and causes the speaker identification model to output the speaker identification information.

A program according to one aspect of the present disclosure is a program for causing a computer to execute a process of training a speaker identification model that, when voice data is input, outputs speaker identification information that identifies the speaker of an utterance contained in the voice data, the process including a first step of generating second voice data of a second speaker by performing a voice quality conversion process on first voice data of a first speaker, and a second step of performing a training process of the speaker identification model using the first voice data and the second voice data as training data.

These general or specific aspects may be realized as a system, method, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, method, integrated circuit, computer program, and recording medium.

The learning method and other methods disclosed herein enable accurate speaker identification.

FIG. 1 is a block diagram showing an example of the configuration of a speaker identification device according to an embodiment. FIG. 2 is a schematic diagram showing an example of how the voice data storage unit according to the embodiment stores voice data and speaker identification information in association with each other. FIG. 3 is a schematic diagram showing how the voice conversion unit according to the embodiment converts voice data of one speaker into voice data of a plurality of other speakers and outputs the converted voice data. FIG. 4 is a block diagram showing an example of the configuration of a voice quality conversion unit according to an embodiment. FIG. 5 is a flowchart of a speaker identification model training process according to the embodiment. FIG. 6 is a flowchart of a voice conversion model training process according to the embodiment. FIG. 7 is a flowchart of a speaker identification process according to the embodiment.

(How one aspect of the present disclosure was achieved)
2. Description of the Related Art There is known a speaker identification technology that identifies a speaker by using a speaker identification model that has been previously trained using speech data linked to identification information that identifies a speaker as training data.

Conventionally, in order to increase the amount of training data (hereinafter, "increasing the amount of training data" is also referred to as "expanding the training data"), noise, reverberation, etc. are added to the original training speech data. However, the above-mentioned conventional expansion of training data by adding noise, reverberation, etc. does not allow for an increase in the speech content or language (Japanese, English, etc.) of a single speaker. For this reason, it may not be possible to sufficiently reduce the influence of speech content and language in the training process of a speaker identification model.

The inventors therefore conducted extensive research and experiments to identify speakers with high accuracy when using a speaker identification model. As a result, the inventors came up with the following learning method, etc.

A learning method according to one aspect of the present disclosure is a method for learning a speaker identification model that, when voice data is input, outputs speaker identification information that identifies the speaker of an utterance contained in the voice data, and performs a voice conversion process on first voice data of a first speaker to generate second voice data of a second speaker, and performs a learning process of the speaker identification model using the first voice data and the second voice data as learning data.

According to the above learning method, when expanding the learning data in the learning process of the speaker identification model, the amount of voice data of the second speaker can be increased without being limited by the speech content or language. This makes it possible to improve the accuracy of speaker identification by the speaker identification model.

Therefore, the above learning method allows for accurate speaker identification.

The voice conversion process may be based on the voice data of the first speaker and the voice data of the second speaker.

The voice conversion process may also include a process of inputting the first voice data into a voice conversion model that has been trained in advance to output the voice data of the second speaker when the voice data of the first speaker is input, thereby outputting the second voice data from the voice conversion model.

The voice conversion model may also include a deep neural network that receives audio data in WAV format as input and outputs audio data in WAV format.

The voice conversion process may be based on the voice data of the first speaker and the voice data of the third speaker.

The speaker identification model may also include a deep neural network that receives speech features indicating characteristics of speech contained in the audio data and outputs speaker characteristics features indicating characteristics of the speaker.

A speaker identification method according to one aspect of the present disclosure inputs voice data to the speaker identification model that has been trained in advance using the training method described above, and causes the speaker identification model to output the speaker identification information.

According to the above speaker identification method, when expanding the training data in the training process of the speaker identification model, the amount of speech data of the second speaker can be increased without being limited by the speech content or language. This makes it possible to improve the accuracy of speaker identification by the speaker identification model.

Therefore, the above speaker identification method allows for accurate speaker identification.

A program according to one aspect of the present disclosure is a program for causing a computer to execute a process of training a speaker identification model that, when voice data is input, outputs speaker identification information that identifies the speaker of an utterance contained in the voice data, the process including a first step of generating second voice data of a second speaker by performing a voice quality conversion process on first voice data of a first speaker, and a second step of performing a training process of the speaker identification model using the first voice data and the second voice data as training data.

According to the above program, when expanding the learning data in the learning process of the speaker identification model, the amount of voice data of the second speaker can be increased without being restricted by the speech content or language. This makes it possible to improve the accuracy of speaker identification by the speaker identification model.

Therefore, the above program can accurately identify speakers.

These comprehensive or specific aspects may be realized as a system, method, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, method, integrated circuit, computer program, and recording medium.

Embodiments of the present disclosure will be described below with reference to the drawings. Each embodiment described below shows a specific example of the present disclosure. The numerical values, shapes, components, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, the contents of each of the embodiments can be combined.

(Embodiment)
A speaker identification device according to an embodiment of the present invention will be described below. The speaker identification device acquires voice data and outputs identification information for identifying the speaker of an utterance contained in the voice data.

<Configuration>
FIG. 1 is a block diagram showing an example of the configuration of a speaker identification device 1 according to an embodiment.

As shown in FIG. 1, the speaker identification device 1 includes a voice data expansion unit 10, a speaker identification model 20, a learning unit 30, and a recognition target voice data acquisition unit 40.

The voice data expansion unit 10 expands the training data for performing the training process of the speaker identification model 20 (i.e., increases the number of training data). The voice data expansion unit 10 may be realized, for example, by a computer equipped with a microprocessor, memory, a communication interface, etc. In this case, the various functions of the voice data expansion unit 10 are realized by the microprocessor executing a program stored in the memory. The voice data expansion unit 10 may also be realized, for example, by distributed computing or cloud computing using multiple computers that communicate with each other.

As shown in FIG. 1, the voice data expansion unit 10 includes a voice data storage unit 11, a first voice data acquisition unit 12, a voice quality conversion unit 13, a noise reverberation adding unit 14, a first feature calculation unit 15, a comparison unit 16, a voice data storage unit 17, and an extended voice data storage unit 18.

The learning unit 30 performs a learning process for the speaker identification model 20 using the learning data expanded by the voice data expansion unit 10. The learning unit 30 may be realized, for example, by a computer equipped with a microprocessor, memory, a communication interface, etc. In this case, the various functions of the learning unit 30 are realized by the microprocessor executing a program stored in the memory. The learning unit 30 may also be realized, for example, by distributed computing or cloud computing using multiple computers that communicate with each other.

As shown in FIG. 1, the learning unit 30 has a second voice data acquisition unit 31, a second feature calculation unit 32, and a first learning unit 33.

When voice data is input, the speaker identification model 20 outputs speaker identification information that identifies the speaker of the utterance contained in the voice data. The speaker identification model 20 may be realized, for example, by a computer equipped with a microprocessor, memory, a communication interface, etc. In this case, the various functions of the speaker identification model 20 are realized by the microprocessor executing a program stored in the memory. The speaker identification model 20 may also be realized, for example, by distributed computing or cloud computing using multiple computers that communicate with each other.

As shown in FIG. 1, the speaker identification model 20 has a third feature calculation unit 21, a deep neural network (DNN) 22, and a determination unit 23.

The recognition target voice data acquisition unit 40 acquires voice data to be recognized in the speaker recognition performed by the speaker recognition model 20. The recognition target voice data acquisition unit 40 may have, for example, a communication interface for communicating with an external device, and may acquire voice data from the external device via the communication interface. The recognition target voice data acquisition unit 40 may also have, for example, an input/output port (for example, a USB port), and may acquire voice data from an external storage device (for example, a USB memory) connected to the input/output port. The recognition target voice data acquisition unit 40 may also have, for example, a microphone, and may acquire voice data by converting voice input to the microphone into an electrical signal.

The components that make up the audio data expansion unit 10 are explained below.

The voice data storage unit 11 stores voice data and speaker identification information associated with the voice data, which identifies the speaker of the utterance contained in the voice data, in association with each other.

Figure 2 is a schematic diagram showing an example of how the voice data storage unit 11 stores voice data and speaker identification information in association with each other.

As shown in FIG. 2, the voice data storage unit 11 stores multiple pieces of voice data linked to multiple different speaker identification information. The voice data and speaker identification information stored in the voice data storage unit 11 are used as learning data for performing the learning process of the speaker identification model 20.

Returning to Figure 1, we will continue explaining the speaker identification device 1.

The voice data storage unit 11 may, for example, have a communication interface for communicating with an external device, and store voice data acquired from the external device via the communication interface and speaker identification information linked to the voice data. The voice data storage unit 11 may also, for example, have an input/output port (for example, a USB port), and store voice data acquired from an external storage device (for example, a USB memory) connected to the input/output port, and speaker identification information linked to the voice data.

Here, the audio data will be described as being in WAV format. However, the audio data does not necessarily have to be limited to WAV format, and may be in, for example, AIFF format, AAC format, etc.

The first voice data acquisition unit 12 acquires voice data and speaker identification information linked to the voice data from the voice data storage unit 11.

The voice conversion unit 13 converts the voice data acquired by the first voice data acquisition unit 12 into voice data spoken by a speaker (hereinafter also referred to as "other speaker") other than the speaker identified by the speaker identification information linked to the voice data, and outputs the voice data. More specifically, the voice conversion unit 13 generates and outputs voice data spoken by another speaker by changing the frequency components of the speech included in the voice data.

The voice conversion unit 13 converts the voice data of one speaker into voice data of multiple other speakers and outputs the converted voice data, thereby making it possible to output multiple voice data that are different speakers but have the same speech content. Furthermore, when the voice data of one speaker is voice data that includes speech in Japanese, the voice conversion unit 13 can convert it into voice data that includes speech in Japanese of other speakers who do not necessarily speak Japanese. In other words, the voice conversion unit 13 can convert the voice data of one speaker into voice data of multiple other speakers and output the converted voice data, without being limited by the speech content or language of the voice data before conversion.

Figure 3 is a schematic diagram showing how the voice conversion unit 13 converts voice data of one speaker into voice data of multiple other speakers and outputs it.

As shown in FIG. 3, the voice quality conversion unit 13 can increase the amount of voice data used as training data for performing the training process of the speaker identification model 20, without being limited by the speech content or language.

Returning to Figure 1, we will continue explaining the speaker identification device 1.

The voice conversion unit 13 may be realized, for example, by a conventional voice conversion device that is widely available. The voice conversion unit 13 may also be realized, for example, by using a voice conversion model that has been trained in advance so that when voice data of a first speaker is input, voice data of a second speaker is output. Here, the voice conversion unit 13 is described as being realized by using a voice conversion model that has been trained in advance so that when voice data of a first speaker is input, voice data of a second speaker is output.

Figure 4 is a block diagram showing an example configuration of the voice quality conversion unit 13.

As shown in FIG. 4, the voice conversion unit 13 has a voice conversion training data storage unit 131, a second learning unit 132, and a voice conversion model 133.

The voice conversion model 133 is a deep neural network (DNN) that has been trained in advance to output voice data of a second speaker, the other speaker of the speaker pair, when voice data of a first speaker, one of the speakers of the speaker pair, is input, and to output voice data of the first speaker when voice data of the second speaker is input. Here, as an example, the voice conversion model 133 is described as a cycleVAE that has been trained in advance to output voice data of a second speaker in WAV format when voice data of a first speaker is input, and to output voice data of a first speaker in WAV format when voice data of a second speaker is input. However, the voice conversion model 133 does not necessarily have to be limited to the above cycleVAE, as long as it is a DNN that has been trained in advance to output voice data of a second speaker when voice data of a first speaker is input for each of a plurality of speaker pairs, and to output voice data of a first speaker when voice data of a second speaker is input.

The voice conversion training data storage unit 131 stores training data for performing training processing for the voice conversion model 133. More specifically, the voice conversion training data storage unit 131 stores voice data (here, voice data in WAV format) for each of multiple speakers targeted by the voice conversion model 133.

The second learning unit 132 performs a learning process of the voice conversion model 133 using the learning data stored in the voice conversion learning data storage unit 131, so that for each of a plurality of speaker pairs, when voice data of a first speaker, one of the speakers in the speaker pair, is input, voice data of a second speaker, the other speaker in the speaker pair, is output, and when voice data of the second speaker is input, voice data of the first speaker is output.

Returning to Figure 1, we will continue explaining the speaker identification device 1.

The noise reverberation unit 14 adds noise (e.g., four types) and reverberation (e.g., one type) to each piece of voice data output from the voice quality conversion unit 13, and outputs the voice data after noise addition and the voice data after noise addition. This allows the noise reverberation unit 14 to further increase the number of pieces of voice data.

The first feature calculation unit 15 calculates speech features indicating the characteristics of speech contained in the speech data from each of the speech data output from the voice quality conversion unit 13 and the noise reverberation adding unit 14. Here, as an example, the first feature calculation unit 15 calculates MFCC (Mel-Freuyency Cepstrum Coefficients) indicating the vocal tract characteristics of a speaker as the speech feature. However, as long as the first feature calculation unit 15 can calculate speech features indicating the characteristics of a speaker, it is not necessarily limited to the example of calculating MFCC. For example, the first feature calculation unit 15 may calculate the speech feature by applying a Mel filter bank to the speech audio signal, or may calculate the spectrogram of the speech audio signal as the speech feature.

The comparison unit 16 compares, for each speaker feature (hereinafter also referred to as "first speaker feature") output from the first feature calculation unit 15, the first speaker feature with the speaker feature (hereinafter also referred to as "second speaker feature") of the speaker whose utterance is included in the voice data from which the first speaker feature is calculated.

If, as a result of the comparison, (1) the similarity between the first speaker feature and the second speaker feature is within a predetermined range, the comparison unit 16 links the voice data from which the first speaker feature is calculated to speaker identification information that identifies the speaker of the utterance contained in the voice data. In this way, the comparison unit 16 can increase the number of voice data linked to one speaker identification information. The comparison unit 16 then outputs the voice data and the speaker identification information linked to the voice data.

If, as a result of the comparison, (2) the similarity between the first speaker feature and the second speaker feature is not within a predetermined range, the comparison unit 16 links the voice data from which the first speaker feature is calculated with identification information that identifies a third party different from the speaker of the utterance contained in the voice data. This allows the comparison unit 16 to increase the number of speaker identification information linked to the voice data. That is, the comparison unit 16 can increase the number of speakers in the training data for performing the training process of the speaker identification model 20. By increasing the number of speakers, it is possible to suppress over-training in the training process of the speaker identification model 20 described below. This allows the generalization performance of the speaker identification model 20 to be improved. Then, the comparison unit 16 outputs the voice data and the speaker identification information linked to the voice data.

Like the voice data storage unit 11, the extended voice data storage unit 18 stores voice data and speaker identification information associated with the voice data that identifies the speaker of the utterance contained in the voice data, in association with each other.

The voice data storage unit 17 stores in the extended voice data holding unit 18, in correspondence with each other, the voice data and the speaker identification information linked to the voice data, for each of the voice data and the speaker identification information linked to the voice data output from the comparison unit 16. The voice data storage unit 17 also stores in the extended voice data holding unit 18, in correspondence with each of the voice data and the speaker identification information linked to the voice data, for each of the voice data and the speaker identification information linked to the voice data acquired by the first voice data acquisition unit 12. As a result, in addition to the voice data stored by the voice data storage unit 11 as learning data for performing the learning process of the speaker identification model 20, the extended voice data holding unit 18 also stores the voice data output from the comparison unit 16 as learning data for performing the learning process of the speaker identification model.

The components that make up the speaker identification model 20 are explained below.

The third feature calculation unit 21 calculates speech features indicating characteristics of speech contained in the speech data acquired by the recognition target speech data acquisition unit 40 from the speech data. Here, as an example, the third feature calculation unit 21 is described as calculating an MFCC indicating the vocal tract characteristics of a speaker as the speech feature. However, as long as the third feature calculation unit 21 can calculate speech features indicating characteristics of a speaker, it is not necessarily limited to the example of calculating an MFCC. The third feature calculation unit 21 may, for example, calculate a speech feature by applying a Mel filter bank to the speech audio signal, or may, for example, calculate a spectrogram of the speech audio signal as the speech feature.

The deep neural network 22 is a deep neural network (DNN) that has been trained in advance so that, when the speech feature calculated by the third feature calculation unit 21 is input, it outputs a speaker feature indicating the characteristics of the speaker of the utterance included in the voice data from which the speech feature is calculated. Here, as an example, the deep neural network 22 is described as Kaldi that has been trained in advance so that, when the MFCC indicating the vocal tract characteristics of the speaker is input, it outputs an x-Vector, which is an acoustic feature of the utterance in which a variable-length utterance is mapped to a fixed-dimensional embedding, as a speaker feature. However, the deep neural network 22 does not necessarily need to be limited to the above-mentioned Kaldi, as long as it is a DNN that has been trained in advance so that, when the speech feature calculated by the third feature calculation unit 21 is input, it outputs a speaker feature indicating the characteristics of the speaker. Note that details of the calculation method of the x-Vector are disclosed in Non-Patent Document 1, and therefore will not be described in detail here.

The determination unit 23 determines the speaker of the utterance included in the voice data acquired by the recognition target voice data acquisition unit 40 based on the speaker feature values output from the deep neural network 22. More specifically, the determination unit 23 stores the x-Vector of multiple speakers, identifies the x-Vector that is most similar to the x-Vector output from the deep neural network 22 from among the multiple stored x-Vector, and determines the speaker of the identified x-Vector as the speaker of the utterance included in the voice data acquired by the recognition target voice data acquisition unit 40. The determination unit 23 then outputs speaker identification information that identifies the determined speaker.

The components that make up the learning unit 30 are explained below.

The second voice data acquisition unit 31 acquires voice data and speaker identification information linked to the voice data from the extended voice data storage unit 18.

The second feature calculation unit 32 calculates speech features indicating characteristics of speech contained in the speech data acquired by the second speech data acquisition unit 31 from the speech data. Here, as an example, the second feature calculation unit 32 is described as calculating an MFCC indicating the vocal tract characteristics of a speaker as the speech feature. However, as long as the second feature calculation unit 32 can calculate speech features indicating the characteristics of a speaker, it is not necessarily limited to the example of calculating an MFCC. For example, the second feature calculation unit 32 may calculate a speech feature by applying a Mel filter bank to the speech audio signal, or may calculate a spectrogram of the speech audio signal as the speech feature.

The first learning unit 33 performs a learning process for the speaker identification model 20 using the speech features calculated by the second feature calculation unit 32 and speaker identification information that identifies the speaker of the utterance included in the voice data from which the speech features are calculated as learning data, and outputs speaker identification information that identifies the speaker of the utterance included in the voice data when voice data is input.

More specifically, the first learning unit 33 uses the MFCC calculated by the second feature calculation unit 32 and the speaker identification information corresponding to the MFCC as learning data, and when the MFCC is input, performs a learning process on the deep neural network 22 so as to output an x-Vector indicating the characteristics of the speaker of the utterance contained in the voice data from which the MFCC is calculated.

<Operation>
The speaker identification device 1 having the above configuration performs a speaker identification model training process, a voice quality conversion model training process, and a speaker identification process.

These processes are explained below in order with reference to the drawings.

Figure 5 is a flowchart of the speaker identification model training process.

The speaker identification model training process is a process that performs training processing of the speaker identification model 20.

The speaker identification model training process is started, for example, when a user of the speaker identification device 1 performs an operation on the speaker identification device 1 to start the speaker identification model training process.

When the speaker identification model learning process is started, the first voice data acquisition unit 12 acquires one voice data and one speaker identification information linked to the one voice data from the voice data storage unit 11 (step S100).

When a piece of voice data and a piece of speaker identification information are acquired, the voice data storage unit 17 associates the piece of voice data with the piece of speaker identification information and stores them in the extended voice data holding unit 18 (step S110).

On the other hand, the voice conversion unit 13 selects one speaker from among the other speakers who are speakers other than the speaker identified by the one speaker identification information (step S120).Then, the voice conversion unit 13 converts the one voice data into voice data spoken by the one speaker (step S130) and outputs it.

When the voice data is output from the voice conversion unit 13, the noise reverberation adding unit 14 adds noise and reverberation to the voice data output from the voice conversion unit 13 (step S140) and outputs one or more pieces of voice data.

When one or more pieces of voice data are output from the noise reverberation unit 14, the first feature calculation unit 15 calculates speech features from each of the voice data output from the voice conversion unit 13 and the one or more pieces of voice data output from the noise reverberation unit 14 (step S150).

Once the speech features are calculated, the comparison unit 16 compares each of the calculated speech features with the speech features of the selected speaker and determines whether the similarity between the calculated speech features and the speech features of the selected speaker is within a predetermined range (step S160).

If the comparison unit 16 judges the result of the process of step S160 to be positive (step S160: Yes), the comparison unit 16 links speaker identification information that identifies the selected speaker to the voice data from which the positively judged speech feature is calculated (step S170). Then, the comparison unit 16 outputs the voice data and the speaker identification information linked to the voice data.

If the comparison unit 16 judges the result of the process of step S160 to be negative (step S160: No), the comparison unit 16 links the voice data from which the speech feature quantity judged to be negative was calculated with identification information that identifies a third party different from the selected speaker (step S180). Then, the comparison unit 16 outputs the voice data and the speaker identification information linked to the voice data.

When the comparison unit 16 performs the process of step S170 or the process of step S180 for all the speech features that were compared in the process of step S160, the voice data storage unit 17 stores the voice data and the speaker identification information linked to the voice data output from the comparison unit 16 in the extended voice data holding unit 18 in association with each other (step S190).

When the processing of step S190 is completed, the voice quality conversion unit 13 determines whether or not there is a speaker among the other speakers who has not been selected in the processing of step S120 (hereinafter also referred to as an "unselected speaker") (step S200).

If it is determined in the processing of step S200 that there is an unselected speaker (step S200: Yes), the voice conversion unit 13 selects one speaker from the unselected speakers (step S210) and proceeds to the processing of step S130.

If it is determined in the processing of step S200 that there is no unselected speaker (step S200: No), the first voice data acquisition unit 12 determines whether there is any unacquired voice data among the voice data stored in the voice data storage unit 11 (step S220).

If it is determined in the processing of step S220 that there is unacquired voice data (step S220: Yes), the first voice data acquisition unit 12 acquires one piece of voice data from the unacquired voice data (step S230) and proceeds to the processing of step S110.

If it is determined in the processing of step S220 that there is no unacquired voice data (step S220: No), the second voice data acquisition unit 31 acquires voice data and speaker identification information linked to the voice data from the extended voice data storage unit 18 for all voice data stored in the extended voice data storage unit 18 (step S240).

Once the voice data and the speaker identification information linked to the voice data have been acquired for all the voice data, the second feature calculation unit 32 calculates speech features indicating the characteristics of the speech contained in the voice data from the voice data for all the voice data (step S250).

Once the speech features have been calculated for all the speech data, the first learning unit 33 performs a learning process for the speaker identification model 20 using the speech features and the speaker identification information identifying the speaker of the utterance included in the speech data from which the speech features were calculated as learning data for all the speech features, so that when speech data is input, the speaker identification information identifying the speaker of the utterance included in the speech data is output (step S260).

When the processing of step S260 is completed, the speaker identification device 1 ends the speaker identification model training process.

Figure 6 is a flowchart of the voice conversion model training process.

The voice conversion model training process is a process that performs training processing of the voice conversion model 133.

The voice conversion model training process is started, for example, when a user of the speaker identification device 1 performs an operation on the speaker identification device 1 to start the voice conversion model training process.

When the voice conversion model training process is started, the second training unit 132 selects one speaker pair from among the multiple speakers targeted by the voice conversion model 133 (step S300). Then, the second training unit 132 performs training process of the voice conversion model 133 using training data for each of the two speakers constituting the selected speaker pair from among the training data stored in the voice conversion training data storage unit 131, so that when voice data of a first speaker, one of the speakers in the speaker pair, is input, voice data of a second speaker, the other speaker in the speaker pair, is output, and when voice data of the second speaker is input, voice data of the first speaker is output (step S310).

When the second learning unit 132 has performed the learning process of the voice conversion model 133 for one speaker pair, it determines whether there are any unselected speaker pairs among the multiple speakers targeted by the voice conversion model 133 (step S320).

If it is determined in the processing of step S320 that there is an unobtained speaker pair (step S320: Yes), the second learning unit 132 selects one speaker pair from the unselected speaker pairs (step S330) and proceeds to the processing of step S310.

If it is determined in the processing of step S320 that there are no unacquired speaker pairs (step S320: No), the speaker identification device 1 ends the voice conversion model training processing.

Figure 7 is a flowchart of the speaker identification process.

The speaker identification process is a process for identifying the speaker of an utterance contained in voice data. More specifically, the speaker identification process is a process for inputting voice data to a speaker identification model 20 that has undergone a learning process in advance, and causing the speaker identification model 20 to output speaker identification information.

The speaker identification process is started, for example, when a user of the speaker identification device 1 performs an operation on the speaker identification device 1 to start the speaker identification process.

When the speaker identification process is started, the identification target voice data acquisition unit 40 acquires the voice data to be identified (step S400).

When the voice data is acquired, the third feature calculation unit 21 calculates speech features indicating characteristics of the speech contained in the acquired voice data from the voice data (step S410), and inputs the calculated speech features to the deep neural network 22. The deep neural network 22 then outputs speaker characteristics indicating characteristics of the speaker of the utterance contained in the voice data from which the input speech features were calculated (step S420).

When the speaker characteristic features are output, the determination unit 23 determines the speaker of the utterance contained in the voice data acquired by the recognition target voice data acquisition unit 40 based on the output speaker characteristic features (step S430). Then, the determination unit 23 outputs speaker identification information that identifies the determined speaker (step S440).

When the processing of step S440 is completed, the speaker identification device 1 ends the speaker identification processing.

<Considerations>
As described above, the speaker identification device 1 expands the training data stored in the voice data storage unit 11 for training the speaker identification model 20 without being restricted by speech content or language. Then, the expanded training data is used to perform training processing for the speaker identification model 20. Therefore, the speaker identification device 1 can improve the accuracy of speaker identification performed using the speaker identification model 20. Therefore, the speaker identification device 1 can accurately identify speakers.

(supplement)
Although the speaker identification device according to the embodiment has been described above, the present disclosure is not limited to this embodiment.

For example, each processing unit included in the speaker identification device according to the above embodiment is typically realized as an LSI, which is an integrated circuit. These may be implemented individually as single chips, or may be implemented as a single chip that includes some or all of the processing units.

In addition, the integrated circuit is not limited to LSI, but may be realized by a dedicated circuit or a general-purpose processor. It is also possible to use an FPGA (Field Programmable Gate Array) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells inside the LSI.

The present disclosure may also be realized as a method for learning a speaker identification model, or as a speaker identification method, executed by a speaker identification device according to an embodiment.

In the above embodiment, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

The division of functional blocks in the block diagram is just one example, and multiple functional blocks may be realized as one functional block, one functional block may be divided into multiple blocks, or some functions may be transferred to other functional blocks. In addition, the functions of multiple functional blocks with similar functions may be processed in parallel or in a time-shared manner by a single piece of hardware or software.

The order in which each step in the flowchart is performed is merely an example to specifically explain the present disclosure, and orders other than those described above may also be used. Some of the steps may also be performed simultaneously (in parallel) with other steps.

The above describes a speaker recognition device according to one or more aspects based on an embodiment, but the present disclosure is not limited to this embodiment. As long as it does not deviate from the spirit of the present disclosure, various modifications conceivable by a person skilled in the art to this embodiment, and forms constructed by combining components in various modifications, etc. may also be included within the scope of one or more aspects.

This disclosure can be widely used in devices for identifying speakers, etc.

REFERENCE SIGNS LIST 1 Speaker identification device 10 Voice data expansion unit 11 Voice data storage unit 12 First voice data acquisition unit 13 Voice quality conversion unit 14 Noise reverberation adding unit 15 First feature amount calculation unit 16 Comparison unit 17 Voice data storage unit 18 Extended voice data storage unit 20 Speaker identification model 21 Third feature amount calculation unit 22 Deep neural network 23 Judgment unit 30 Learning unit 31 Second voice data acquisition unit 32 Second feature amount calculation unit 33 First learning unit 40 Identification target voice data acquisition unit 131 Voice quality conversion training data storage unit 132 Second learning unit 133 Voice conversion model

Claims (8)

A method for training a speaker identification model, which receives voice data and outputs speaker identification information for identifying a speaker of an utterance included in the voice data, comprising the steps of:
performing a voice conversion process on the first voice data of the first speaker to generate second voice data of a second speaker;
performing a learning process of the speaker identification model using the first voice data and the second voice data as learning data;
further comparing a first speaker feature which is a speaker feature of the second voice data generated by performing the voice conversion process with a second speaker feature which is a speaker feature of the first speaker who is a speaker of an utterance included in the first voice data from which the second voice data is generated by performing the voice conversion process, and determining whether or not a similarity between the first speaker feature and the second speaker feature is within a predetermined range;
If it is determined in the judgment that the similarity is not within the predetermined range, the learning process performs the learning process by treating the first voice data, which is the source of the second voice data generated by the voice conversion process, as voice data of a third party other than the first speaker, who is the speaker of the utterance contained in the first voice data, which is the source of the second voice data generated by the voice conversion process. The voice conversion process is a process based on the voice data of the first speaker and the voice data of the second speaker.
The learning method according to claim 1 . the voice conversion process includes a process of inputting the first voice data into a voice conversion model that has been trained in advance so as to output the voice data of the second speaker when the voice data of the first speaker is input, and outputting the second voice data from the voice conversion model;
The learning method according to claim 2 . The voice conversion model includes a deep neural network that receives voice data in a WAV format as input and outputs voice data in a WAV format.
The learning method according to claim 3. The voice conversion process is a process based on the voice data of the first speaker and the voice data of a third speaker.
The learning method according to claim 1 . The speaker identification model includes a deep neural network that receives an utterance feature indicating a feature of an utterance included in the voice data and outputs a speaker characteristic feature indicating a feature of a speaker.
The learning method according to claim 1 . 2. A method for learning a speaker identification model comprising: inputting speech data to the speaker identification model that has been trained in advance by the training method according to claim 1; and causing the speaker identification model to output the speaker identification information.
Speaker identification methods. A program for causing a computer to execute a process of learning a speaker identification model that outputs speaker identification information for identifying a speaker of an utterance included in voice data when voice data is input, the program comprising:
The process comprises:
A first step of generating second voice data of a second speaker by performing a voice conversion process on first voice data of a first speaker;
a second step of performing a learning process of the speaker identification model using the first voice data and the second voice data as learning data;
and a third step of comparing a first speaker feature which is a speaker feature of the second voice data generated by performing the voice conversion process with a second speaker feature which is a speaker feature of the first speaker who is a speaker of an utterance included in the first voice data from which the second voice data is generated by performing the voice conversion process, and determining whether or not a similarity between the first speaker feature and the second speaker feature is within a predetermined range,
When it is determined in the third step that the similarity is not within the predetermined range, in the second step, the first voice data, which is the source of generating the second voice data by performing the voice conversion process, is treated as voice data of a third party different from the first speaker, who is the speaker of the utterance included in the first voice data, which is the source of generating the second voice data by performing the voice conversion process, and the learning process is performed.
program.

Images