JP6695830B2 - Speech recognition accuracy deterioration factor estimation device, speech recognition accuracy deterioration factor estimation method, program - Google Patents

Description

  The present invention relates to a speech recognition accuracy deterioration factor estimation device, a speech recognition accuracy deterioration factor estimation method, and a program for estimating a deterioration factor of speech recognition accuracy from a speech feature.

  A method of detecting and correcting a cause of a voice recognition error such as an unknown word or a homonym has been proposed (Non-Patent Documents 1 and 2).

Taichi Asami, Ryo Masumura, Yushi Aono, Koichi Shinoda, "Recurrent Out-of-Vocabulary Word Detection Using Distribution of Features", [online], September 2016, ISCA, [May 18, 2017 search], Internet <URL: http://t2r2.star.titech.ac.jp/rrws/file/CTT100716848/ATD100000413/> Satoshi Kaki, Eiichiro Sumida, Hitoshi Iida, “Speech Recognition Error Correction Method Using Statistical Features of Character Chain”, [online], March 1998, Linguistic Processing Society, [May 18, 2017 search] , Internet <URL: http://www.anlp.jp/proceedings/annual_meeting/1998/pdf_dir/Q1-5.pdf ＞ Srikanth R Madikeri, Hema A Murthy, "Mel filter bank energy-based slope feature and its application to speaker recognition", [online], January 2011, ISCA, [May 18, 2017 search], Internet < URL: http://ieeexplore.ieee.org/abstract/document/5734713/〉

  In Non-Patent Document 1, phoneme recognition and word recognition are executed in parallel, and an unknown word is detected using a feature amount for each transition of a confusion network generated in word recognition, but it is necessary to always perform speech recognition. And only unknown words can be detected. In Non-Patent Document 2, the speech recognition error pattern is typified and the part corresponding to the pattern is corrected, but only the linguistic error pattern can be dealt with.

  Therefore, it is an object of the present invention to provide a speech recognition accuracy deterioration factor estimation device capable of estimating a deterioration factor of speech recognition accuracy from the characteristics of speech.

  The speech recognition accuracy deterioration factor estimation device of the present invention includes a speech feature amount extraction unit, a posterior probability calculation unit, a filtering unit, and a deterioration factor output unit.

  The voice feature amount extraction unit extracts a voice feature amount for each frame from the input voice. The posterior probability calculation unit calculates the posterior probability of each audio event of the audio feature amount of each frame, based on a plurality of audio events that are pre-classified into one of the deterioration factor class and the non-deterioration factor class. The filtering unit performs filtering by a time-series filter, which is a filter in which weighting coefficients are expanded in the time direction to the posterior probability for each acoustic event, and obtains the corrected posterior probability. The deterioration factor output unit, if the sound event having the maximum corrected posterior probability in each frame among the sound events classified into the deterioration factor class is continuously the same within a predetermined time period, the corresponding sound Output the event as a deterioration factor.

  According to the speech recognition accuracy deterioration factor estimation device of the present invention, the speech recognition accuracy deterioration factor can be estimated from the characteristics of the speech.

3 is a block diagram showing the configuration of a speech recognition accuracy deterioration factor estimation device according to the first embodiment. FIG. 3 is a flowchart showing the operation of the speech recognition accuracy deterioration factor estimation device according to the first exemplary embodiment.

  Hereinafter, embodiments of the present invention will be described in detail. It should be noted that components having the same function are denoted by the same reference numeral, and redundant description will be omitted.

  Hereinafter, the configuration of the speech recognition accuracy deterioration factor estimation device according to the first exemplary embodiment will be described with reference to FIG. As shown in the figure, the speech recognition accuracy deterioration factor estimation apparatus 1 of the present embodiment has a speech feature amount extraction unit 11, a posterior probability calculation unit 12, a filtering unit 13, a deterioration factor output unit 14, and a model storage unit. Including 15. Hereinafter, the operation of each component will be described in detail with reference to FIG.

<Voice feature amount extraction unit 11>
Input: Voice output: Voice feature amount processing of each frame: The voice feature amount extraction unit 11 extracts a voice feature amount for each frame from the input voice (S11). The audio feature amount extraction unit 11 extracts the audio feature amount of each frame by a known technique described in Non-Patent Document 3, for example.

<< Definition of term: frame >>
When acoustically analyzing an input voice, it is often assumed that the waveform to be analyzed has a certain property, that is, the property is the same at the beginning and the end of the analysis target. Therefore, the voice waveform to be analyzed is cut out in a very short time width so that the voice can be said to have the same characteristics in the very short time width. As the time width here, a value of 20 to 30 milliseconds is often adopted.

  When cutting out a voice in a short time width as described above, by avoiding a sudden change in the acoustic analysis result and by continuously changing the acoustic analysis result, by shifting the cutout position by a width shorter than the time width, The cut out voice waveforms are partially overlapped so that a common voice waveform is included in the overlapped portions. At this time, the time width of shifting is often called a frame shift. A value such as 10 to 15 milliseconds is often adopted as the frame shift. Non-Patent Document 3 FIG. See 1.

<Posterior probability calculator 12>
Input: Voice feature amount of each frame of the entire utterance in the input voice Output: posterior probability processing of each acoustic event of each frame: The posterior probability calculation unit 12 selects either the deterioration factor class or the non-deterioration factor class. The posterior probability of each audio event of the audio feature amount of each frame is calculated based on a plurality of audio events that have been classified in advance (S12). The posterior probability calculation unit 12 is a model that has been learned in advance, and inputs the audio feature amount of each frame to the acoustic event estimation model stored in the model storage unit 15, and calculates the posterior probability of each acoustic event for each frame. Calculate and output.

  The following table is an example of an acoustic event.

  The acoustic event is one of a deterioration factor class, which is a class considered to be a factor that adversely affects the speech recognition (recognition accuracy is deteriorated), and a non-degradation factor class, which is a class considered not to adversely affect the speech recognition. It is supposed to be classified into classes beforehand.

  In the example of the acoustic event shown in the above table, each acoustic event is classified as follows.

  As the acoustic event estimation model, for example, a deep neural network (DNN: Deep Neural Netword), a convolutional neural network (CNN: Convolutional Neural Network), or a Gaussian Mixture Model (Gaussian Mixture Model) may be used.

<Filtering unit 13>
Input: posterior probability for each acoustic event of each frame Output: corrected posterior probability processing for each acoustic event of each frame: The filtering unit 13 is a filter in which a weighting coefficient is expanded in the temporal direction to the posterior probability for each acoustic event. Then, the corrected posterior probability is output by performing the filtering with the time-series filter (S13). The filtering unit 13 performs a time-series filtering process for each event with respect to the posterior probability for each acoustic event calculated by the posterior probability calculating unit 12.

  The following method can be given as a specific procedure of the filtering unit 13. First, the filtering unit 13 sets a filter window for the entire utterance section (entire input voice). The filter window sets a time width shorter than the entire utterance section (for example, 10 times the frame shift length). Next, the filtering unit 13 acquires the posterior probability for each frame included in the filter window. If applied to the above example, the posterior probability for 10 frames = 10 posterior probabilities. The filtering unit 13 applies a time-series filter to the posterior probabilities obtained for each acoustic event. Here, the time series filter is a filter in which weighting coefficients are expanded in the time direction, and may be, for example, a moving average filter or a median filter.

In the above method, there are cases in which frames that do not satisfy the filter window length occur at the beginning and the end of the utterance period. In that case, the filtering unit 13 executes the following processing to avoid a state in which filtering cannot be performed.
First utterance: Copy the posterior probability of the first frame, and use the copied value if the filter window length is less than that.
End of utterance: Copy the posterior probability of the last frame, and use the copied value for the part that is less than the filter window length.

<Deterioration factor output unit 14>
Input: Corrected posterior probability for each acoustic event in each frame Output: Degradation factor, Degradation factor index (dominant / non-dominant)
Process: The deterioration factor output unit 14 determines, among the sound events classified into the deterioration factor class, the sound event having the maximum corrected posterior probability in each frame is continuously the same within a predetermined time period, The corresponding acoustic event is output as the deterioration factor (S14-1). The predetermined time period is an arbitrary time width shorter than the time length of the entire utterance period. For example, the predetermined time period is 10 frame shift length. In this case, when a certain acoustic event belonging to the deterioration factor class has the highest posterior probability for 10 consecutive frames, the deterioration factor output unit 14 outputs the sound event as a deterioration factor.

  In addition, the deterioration factor output unit 14 sets the sum of the corrected posterior probabilities in each frame of the acoustic events classified in the deterioration factor class as the intra-frame total value, and the intra-frame total value as the utterance between frames. When the summed value is the interframe total value and the interframe total value is normalized based on the number of frames as the normalized probability value over the entire section, and the normalized probability value exceeds the predetermined threshold value. Then, a deterioration factor index indicating that the deterioration factor is dominant is output (S14-2). On the other hand, when the normalized probability value does not exceed the predetermined threshold, the deterioration factor output unit 14 outputs a deterioration factor index indicating that the deterioration factor is not dominant (S14-2).

<Effect>
Since the voice recognition accuracy deterioration factor estimation device 1 of the present embodiment estimates the factors that deteriorate the voice recognition accuracy from the characteristics of the voice, the following effects can be expected. 1) Since the deterioration factor can be presented to the user, the user tries to adjust the recording environment. Therefore, it contributes to the improvement of the voice recognition accuracy when the voice is recorded again. 2) It contributes to the improvement of the voice recognition accuracy by excluding the voice recognition result in which the deterioration factor is predominant.

<Additional notes>
The device of the present invention is, for example, as a single hardware entity, an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating with the outside of the hardware entity. Connectable communication unit, CPU (Central Processing Unit, cache memory and registers may be provided), memory RAM and ROM, hard disk external storage device and their input unit, output unit, communication unit , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged among external storage devices. If necessary, the hardware entity may be provided with a device (drive) capable of reading and writing a recording medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the hardware entity stores a program necessary to realize the above-mentioned functions and data necessary for the processing of this program (not limited to the external storage device, for example, the program is read). It may be stored in a ROM that is a dedicated storage device). Further, data and the like obtained by the processing of these programs are appropriately stored in the RAM, the external storage device, or the like.

  In the hardware entity, each program stored in an external storage device (or ROM, etc.) and the data necessary for the processing of each program are read into the memory as needed, and interpreted and executed / processed by the CPU as appropriate. .. As a result, the CPU realizes a predetermined function (each constituent element represented by the above, ... Unit, ... Means, etc.).

  The present invention is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit of the present invention. Further, the processes described in the above embodiments may be executed not only in time series according to the order described, but also in parallel or individually according to the processing capability of the device that executes the processes or the need. .

  As described above, when the processing functions of the hardware entity (the apparatus of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on the computer, the processing functions of the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded in a computer-readable recording medium. The computer-readable recording medium may be, for example, a magnetic recording device, an optical disc, a magneto-optical recording medium, a semiconductor memory, or the like. Specifically, for example, a hard disk device, a flexible disk, a magnetic tape or the like is used as a magnetic recording device, and a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), or a CD-ROM (Compact Disc Read Only) is used as an optical disc. Memory), CD-R (Recordable) / RW (ReWritable), etc., as magneto-optical recording medium, MO (Magneto-Optical disc), etc., as semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The distribution of this program is performed by, for example, selling, transferring, or lending a portable recording medium such as a DVD or a CD-ROM in which the program is recorded. Further, the program may be stored in a storage device of a server computer and transferred from the server computer to another computer via a network to distribute the program.

  A computer that executes such a program first stores, for example, the program recorded on a portable recording medium or the program transferred from the server computer in its own storage device. Then, when executing the process, this computer reads the program stored in its own recording medium and executes the process according to the read program. As another execution form of this program, a computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to this computer. Each time, the processing according to the received program may be sequentially executed. In addition, a configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by the execution instruction and result acquisition without transferring the program from the server computer to this computer May be Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (such as data that is not a direct command to a computer but has the property of defining computer processing).

  Further, in this embodiment, the hardware entity is configured by executing a predetermined program on the computer, but at least a part of the processing content may be implemented by hardware.

Claims (5)

A voice feature amount extraction unit that extracts a voice feature amount for each frame from the input voice,
A posterior probability calculation unit that calculates a posterior probability for each acoustic event of the audio feature amount for each frame, based on a plurality of acoustic events that are pre-classified in one of the deterioration factor class and the non-degradation factor class,
A filtering unit that performs a correction posterior probability by performing filtering with a time-series filter that is a filter in which a weighting coefficient is expanded in the time direction in the posterior probability for each acoustic event,
Of the acoustic events classified into the deterioration factor class, if the acoustic event having the maximum corrected posterior probability in each frame is continuously the same within a predetermined time period, the corresponding acoustic event A speech recognition accuracy deterioration factor estimation device including a deterioration factor output unit for outputting as a deterioration factor. The speech recognition accuracy deterioration factor estimation device according to claim 1,
The deterioration factor output unit,
A value obtained by summing the corrected posterior probabilities in each frame of the acoustic events classified in the deterioration factor class within a frame is defined as an intra-frame total value, and a value obtained by summing the intra-frame total values between frames is summed between frames. In the case where a value obtained by normalizing the interframe total value based on the number of frames is set as a value and the normalized probability value exceeds a predetermined threshold value, the deterioration factor is predominant. A speech recognition accuracy deterioration factor estimation device that outputs a deterioration factor index indicating A speech recognition accuracy deterioration factor estimation method executed by a speech recognition accuracy deterioration factor estimation device, comprising:
Extracting a voice feature amount for each frame from the input voice,
Calculating a posterior probability for each acoustic event of the audio feature amount for each frame, based on a plurality of acoustic events that are pre-classified in one of the deterioration factor class and the non-degradation factor class,
A step of performing a correction posterior probability by performing filtering with a time series filter that is a filter in which a weighting coefficient is expanded in the time direction in the posterior probability for each of the acoustic events,
Of the acoustic events classified into the deterioration factor class, if the acoustic event having the maximum corrected posterior probability in each frame is continuously the same within a predetermined time period, the corresponding acoustic event A method for estimating a factor for deteriorating speech recognition accuracy, which includes the step of outputting as a deterioration factor. The speech recognition accuracy deterioration factor estimation method according to claim 3,
A value obtained by summing the corrected posterior probabilities in each frame of the acoustic events classified in the deterioration factor class within a frame is defined as an intra-frame total value, and a value obtained by summing the intra-frame total values between frames is summed between frames. In the case where a value obtained by normalizing the interframe total value based on the number of frames is set as a value and the normalized probability value exceeds a predetermined threshold value, the deterioration factor is predominant. A method of estimating a deterioration factor of speech recognition accuracy, the method including the step of outputting a deterioration factor index indicating.   A program that causes a computer to function as the speech recognition accuracy deterioration factor estimation device according to claim 1.

Images