JP6944329B2 - Speech recognizer and speech recognition program - Google Patents

Description

The present invention relates to a voice recognition device and a voice recognition program for performing voice recognition.

Conventionally, the "cocktail party effect" has been known as an effect of listening to a topic of interest in an environment in which a plurality of conversations are held by a plurality of speakers. For example, even when many people are talking on different topics, the cocktail party effect allows you to naturally hear conversations of people who are interested or conversations of content that you are interested in.

There is a growing demand for engineering realization of human abilities related to the "cocktail party effect". However, it is not easy to realize the "cocktail party effect" engineeringly. As a study for engineeringly realizing the "cocktail party effect", a study on separation of mixed voices is known (see, for example, Patent Document 1, Non-Patent Document 1 to Non-Patent Document 6). The mixed voice corresponds to a voice composed of a plurality of conversations performed by a plurality of speakers as described above.

In a general study on the separation of mixed voices, more than the number of microphones of the speakers are installed, and the voices of each speaker are separated based on the waveforms of the multiple voices picked up by each microphone. The technical field for such mixed audio separation is called "blind source separation". As a main method for separating mixed speech, a technique called "Independent component analysis" (hereinafter referred to as "ICA") is used.

JP-A-2000-125397

Edited by Pierre Comon, Christian Jutten, Handbook of Blind Source Separation: Independent Component Analysis and Applications, Academic Press, Chapter 8,2010, pp.281-324 Emmanuel Vincent, Nancy Bertin, Remi Gribonval, Frederic Bimot, From Blind to Guided Audio Source Separation: How models and side information can improve the separation of sound, IEEE Signal Processing Magazine, Volume: 31, Issue 3, May 2014, pp.107 -115 Sebastian Ewert, Bryan Pardo, Meinard Mueller, Mark D. Plumbley, Score-Informed Source Separation for Musical Audio Recordings: An overview, IEEE Signal Processing Magazine, Volume: 31, Issue 3, May 2014, pp.116-124 Simon Arberet, Pierre Vandergheynst, R. Carrillo, J. Thiran, Y. Wiaux, "Sparse reverberant audio source separation via reweighted analysis", IEEE Transactions on Speech, Audio and Language Processing, Vol. 21, No. 7, Jul. 2013 , pp. 1391-1402 Simon Arberet, Pierre Vandergheynst, "Reverberant Audio Source Separation via Sparse and Low-Rank Modeling", IEEE Signal Processing Letters, Vol. 21, Issue 4, January 28, 2014, pp.404-408 Antoine Liutkus, Derry Fitzgerald, Zafar Rafii, Bryan Pardo, Laurent Daudet, "Kernel Additive Models for Source Separation", IEEE Transactions on Signal Processing, Vol. 62, Issue 16, pp.4298-4310, August 15, 2014

In many cases, the number of speakers is about 2 to 3 when ICA is actually used. In a situation where there are five or more speakers, it is difficult to achieve mixed voice separation by ICA. Also, when the number of speakers increases or decreases in the middle, it becomes difficult to separate mixed voices well. Furthermore, in ICA, it is necessary to install more microphones than speakers. For this reason, in the ICA, the burden of preparing equipment and the burden of installation work for installing a large number of sound collecting devices are incurred.

Further, in order to technically realize the "cocktail party effect", it is necessary to separately perform voice recognition after separating the waveform of the mixed voice by ICA. Therefore, in order to engineeringly realize the "cocktail party effect", multiple microphones are required, the mixed voice is a voice that can be separated by ICA, and the voice recognition process is performed after the voice separation. It was necessary to solve each of multiple issues, such as the need to do.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a voice recognition device and a voice recognition program capable of determining whether or not a reference voice is included in an input voice.

In order to solve the above problems, the voice recognition device according to the present invention converts the input voice into data in the frequency region at first predetermined time intervals, so that the frequency x (1 ≦ x ≦ X) and the time t can be set. By converting the input pattern composed of the two-dimensional data and the reference voice into the data in the frequency region at the second predetermined time, the frequency y (1 ≦ y ≦ X) and the time τ (1 ≦ τ ≦ T) can be obtained. From the value of the coordinate f (t, x) of the input pattern to the value of the coordinate f (t, x) from x = 1 to x = X, using the reference pattern composed of the two-dimensional data of By dividing the sum, the normalized value of the coordinate f (t, x) is calculated, and from the value of the coordinate z (τ, y) of the reference pattern, the above from y = 1 to y = X. The value of the coordinate z (τ, y) normalized by dividing the sum of the values of the coordinate z (τ, y) is calculated, and the coordinate f of the input pattern is calculated by a two-dimensional continuous dynamic programming method. The coordinates normalized from the values of the coordinates f (t, x) normalized by performing a non-linear matching process between (t, x) and the coordinates z (τ, y) of the reference pattern. The local distance d (t, x, τ, y) indicating the absolute value of the value obtained by subtracting the value of z (τ, y) is calculated, and the calculated local distance d (t, x, τ, y) is calculated. A cumulative local distance calculating means for cumulatively calculating the cumulative local distance D (t, X, T, X) and the cumulative local distance D (t, X, T, X) calculated by the cumulative local distance calculating means. The value of D (t, X, T, X) / (TX) divided by the product of T and X is calculated, and the value of D (t, X, T, X) / (TX) is calculated. a dip detecting means for detecting a dip indicating the trough bottom from changing state at time t value, the occurrence time of the dip detected by the dip detecting means and ^{t *,} from the time ^t * -2T to time ^{t *} From the maximum value of the D (t, X, T, X) / (XT) value in the time interval, the value ^{of D (t * , X, T, X) / (XT) in the time t *} When the difference value calculation means for calculating the difference value h by subtracting the value and the difference value h calculated by the difference value calculation means are larger than the preset first threshold value, the cumulative local distance D ( ^{t *, X, T, X} ) of the ^(t *, X, T, by performing the back-tracing from X), a normalized value of the coordinate f (t, x), the coordinates f (t , X) Optimal matching to the normalized value of the coordinates z (τ, y) and the value of the local distance d (t, x, τ, y) Is calculated, and the set of the coordinates z (τ, y) of the reference pattern in which the calculated value of the local distance d (t, x, τ, y) is equal to or less than the preset second threshold value is targeted. It is characterized by having a reference pattern area detecting means for detecting as a region and a voice determining means for determining that the input voice includes the reference voice when the target area is detected by the reference pattern area detecting means. And.

Further, the voice recognition program according to the present invention is a voice recognition program of a voice recognition device for determining whether or not the reference voice is included in the input voice, and the control means of the voice recognition device is the first. By converting the input voice into data in the frequency domain at predetermined time intervals, an input pattern composed of two-dimensional data of frequency x (1 ≤ x ≤ X) and time t, and the second predetermined time interval are described. By converting the reference voice into data in the frequency domain, the input is performed using a reference pattern composed of two-dimensional data of frequency y (1 ≦ y ≦ X) and time τ (1 ≦ τ ≦ T). The normalized coordinates f (t, x) are normalized by dividing the sum of the values of the coordinates f (t, x) from x = 1 to x = X from the values of the pattern coordinates f (t, x). The value of x) is calculated, and the sum of the values of the coordinates z (τ, y) from y = 1 to y = X is divided from the value of the coordinates z (τ, y) of the reference pattern. The normalized value of the coordinate z (τ, y) is calculated, and the coordinate f (t, x) of the input pattern and the coordinate z (τ, y) of the reference pattern are calculated by a two-dimensional continuous dynamic programming method. The absolute value of the value obtained by subtracting the normalized value of the coordinate z (τ, y) from the normalized value of the coordinate f (t, x) by performing a non-linear matching process with y). The local distance d (t, x, τ, y) indicating is calculated, and the calculated local distance d (t, x, τ, y) is accumulated to accumulate the local distance D (t, X, T, X). ), And D (t) obtained by dividing the value of the cumulative local distance D (t, X, T, X) calculated by the cumulative local distance calculation function by the product of T and X. , X, T, X) / (TX) is calculated, and the dip indicating the valley bottom is detected from the change state of the value of D (t, X, T, X) / (TX) at time t. a dip detection function causes the in the time interval of the time of occurrence of the dip detected by the dip detection and ^{t *,} from the time ^t * -2T to time ^{t * D (t, X,} T, X) / Difference value calculation in which the value obtained by subtracting the value of D (t ^{* , X, T, X) / (XT) at time t *} from the maximum value of the value of (XT) is calculated as the difference value h. When the function and the difference value h calculated by the difference value calculation function are larger than the preset first threshold value, the cumulative local distance D (t ^* , X, T, X) (t ^* , X) , T, X) to perform backtrace processing, so that the coordinates f (t, x) ) And the normalized value of the coordinate z (τ, y) that optimally matches the coordinate f (t, x), so that the local distance d (t, x, τ, y) The set of the coordinates z (τ, y) of the reference pattern in which the calculated value of the local distance d (t, x, τ, y) is equal to or less than the preset second threshold value is calculated. , A reference pattern area detection function for detecting as a target area and a voice determination function for determining that the input sound includes the reference sound when the target area is detected by the reference pattern area detection function are realized. It is characterized by that.

Further, the voice recognition device and the voice recognition program described above detect the minimum time τs and the maximum time τe of the coordinates z (τ, y) constituting the target area when the target area is detected. Then, among the reference voices, the voice included in the time range from the time τs to the time τe may be determined to be included in the input voice.

Further, in the above-mentioned voice recognition device, the frequency conversion means for converting the input voice into the data in the frequency region at the first predetermined time and the data in the frequency region converted by the frequency conversion means are normalized. , The input pattern recording means is recorded in the converted time order, and the input pattern recording means is the most recorded in the input pattern recording means every time new data is generated by the frequency conversion means. The old data was erased, the new data was recorded next to the data generated immediately before the new data, and the cumulative local distance calculation means recorded all the data recorded in the input pattern recording means. The cumulative local distance D (t, X, T, X) may be calculated by reading the data while maintaining the order and using it as the input pattern.

In the voice recognition device and the voice recognition program according to the present invention, the input voice is used as a two-dimensional input pattern, the reference voice is used as a two-dimensional reference pattern, and a non-linear matching process is performed by a two-dimensional continuous dynamic programming method. , Determine if the reference voice is included in the input voice. Therefore, even if the input voice is a sound uttered by a plurality of speakers at the same time, the voice can be recognized as one united sound (single voice) without performing waveform separation. Therefore, it is not necessary to perform the voice separation process or the voice recognition process individually after the voice separation, and it is possible to speed up the process and reduce the processing load.

In addition, since voice recognition is performed by uttering multiple speakers as one united sound (single voice), even if the number of speakers increases or decreases in the middle, the increase or decrease of speakers is taken into consideration. Voice recognition can be performed without any need.

Further, in the voice recognition process, the input voice is used as a two-dimensional input pattern, the reference voice is used as a two-dimensional reference pattern, and voice recognition is performed by pattern matching between the two-dimensional data and the two-dimensional data. Speech recognition processing can be performed in consideration of the elements. Therefore, it is possible to perform voice recognition with higher accuracy than in the case where the input voice or the reference voice is regarded as vector data and the voice recognition is performed as in the conventional voice recognition process.

Further, in the voice recognition device and the voice recognition program according to the present invention, the difference value h is calculated based on the change state of the value of D (t, X, T, X) / (TX) at time t, and the difference value h is calculated. Is it possible to detect the target area of the reference pattern in which the value of the local distance d (t, x, τ, y) is equal to or less than the preset second threshold value when the value h is larger than the preset first threshold value? By examining, it is possible to determine whether or not the input voice includes the reference voice.

In particular, in the voice recognition device and the voice recognition program according to the present invention, when the target area is detected, the minimum time τs and the maximum time τe of the coordinates z (τ, y) constituting the target area are detected. By doing so, it becomes possible to determine that the voice included in the time range from the time τs to the time τe among the reference voices is included in the input voice.

Further, in the input pattern recording means, each time new data is generated by the frequency conversion means, the oldest data among the data recorded in the converted time order is deleted, and new data is generated immediately before the new data. The data is recorded next to the data, and the cumulative local distance calculation means reads out all the data recorded in the input pattern recording means while maintaining the recorded order and uses it as an input pattern, and the cumulative local distance D (t). , X, T, X). Therefore, it is possible to perform voice recognition processing of the reference voice for the input voice that is continuously input. Further, the input voice to be processed by the voice recognition process is not limited by the start and end of the voice. Therefore, in the voice recognition device and the voice recognition program according to the present invention, it is possible to use the input voice following the endless as the target of the voice recognition process.

It is a block diagram which showed the schematic structure of the voice recognition apparatus which concerns on embodiment. It is a block diagram which showed the function which the CPU which concerns on embodiment execute based on a program. It is a figure which showed the time change of the value of D (t, X, T, X) / (XT) obtained by a matching process, and (a) is a figure which showed D (t, X, T, in dip The case where the value of X) / (XT) becomes zero is shown, and the case where the value of D (t, X, T, X) / (XT) in the dip is not zero is shown. It is a figure. It is a figure for demonstrating the case where the reference data is the voice data of "it is sunny tomorrow, is it rain", and the input data contains the voice corresponding to "sunny". It is a flowchart which showed the processing content of the matching processing part which concerns on embodiment. It is a figure for demonstrating the conventional speech recognition processing method based on the vector-based matching processing. It is a figure for demonstrating that the reference data and input data are regarded as area-based data, and matching processing is performed.

Hereinafter, an example of the voice recognition device according to the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a voice recognition device. The voice recognition device 100 includes a microphone 1, a CPU (Central Processing Unit: control means, cumulative local distance calculation means, dip detection means, difference value calculation means, reference pattern area detection means) 2, and a ROM (Read Only Memory) 3. It also has a RAM (Random Access Memory: input pattern recording means) 4, a recording unit (input pattern recording means) 5, and a display unit 6. The voice recognition device 100 is installed, for example, in a space in which a plurality of speakers exist, in which conversations having different contents are carried out in parallel by the plurality of speakers.

As described above, the microphone 1 picks up the sound of the space in which conversations having different contents proceed in parallel. In the voice recognition device 100 according to the present embodiment, a case where only one microphone 1 is installed is shown as an example. However, the number of microphones 1 installed is not necessarily limited to one. The number of microphones 1 installed is not limited as long as the sound in the space where different contents of conversation are performed by a plurality of speakers can be picked up as a single voice.

The recording unit 5 is composed of a general hard disk or the like. The configuration of the recording unit 5 is not limited to the hard disk, and may be a flash memory, an SSD (Solid State Drive / Solid State Disk), or the like. The recording unit 5 records voice data for performing voice recognition as reference data. The reference data is generated based on a voice (reference voice) containing words such as sentences and words (hereinafter, these are referred to as "sentence symbols").

The CPU 2 performs voice recognition processing for whether or not the input sound (input sound, input data) picked up by the microphone 1 includes a reference sound (reference data). Specifically, the CPU 2 performs voice recognition processing by reading and executing a program (a program based on the flowchart of FIG. 5 described later) recorded in the ROM 3 described below.

A program or the like showing the processing contents in the CPU 2 is recorded in the ROM 3. The RAM 4 is used as a work area used for processing by the CPU 2. The RAM 4 is used as the buffer memory unit 12 and the working memory 14a described below. For example, when the RAM 4 is used as the buffer memory unit 12, the voice data picked up by the microphone 1 is recorded in the RAM 4 as input data. When the RAM 4 is used as the working memory 14a, the RAM 4 is used as a working area for matching processing and backtrace processing using the two-dimensional continuous dynamic programming method.

The display unit 6 is used to display a determination result or the like of whether or not the input voice (input data) includes the reference voice (reference data). A general display device such as a liquid crystal display or a CRT display is used for the display unit 6.

Next, a method in which the CPU 2 performs voice recognition processing based on the input data and the reference data will be described. FIG. 2 is a block diagram showing a function that the CPU 2 executes based on the program recorded in the ROM 3. As shown in FIG. 2, the voice recognition device 100 refers to an A / D conversion unit 10, an FFT unit (frequency conversion means, cumulative local distance calculation means) 11, and a buffer memory unit (input pattern recording means) 12. A data memory unit 13, a matching processing unit (control means, cumulative local distance calculation means, dip detection means, difference value calculation means, reference pattern area detection means) 14, and a detection determination unit (control means, voice judgment means) 15. have.

A microphone 1 is connected to the A / D conversion unit 10. The A / D conversion unit 10 performs a process of converting the input voice picked up by the microphone 1 from analog data to digital data. The input voice picked up by the microphone 1 is an endless voice waveform. For example, a voice waveform in which voices uttered by a plurality of speakers at the same time are overlapped to form a single voice is applicable. The A / D conversion unit 10 converts the input sound, which is analog data picked up by the microphone 1, into digital data at a sampling rate of, for example, 12 kHz or 16 kHz. The digital data converted by the A / D conversion unit 10 is output to the FFT unit 11.

The FFT unit 11 performs a process of converting the input sound converted into digital data by the A / D conversion unit 10 from the data in the time domain to the data in the frequency domain. Specifically, the FFT unit 11 performs a fast Fourier transform process after filtering the input voice converted into digital data by using a window function. The data subjected to the fast Fourier transform by the FFT unit 11 is obtained, for example, as a spectrum vector having 20 bands. In this embodiment, this spectrum vector is referred to as a frame vector. This frame vector is data used in general speech analysis. The number of bands of the frame vector (spectrum vector) is not necessarily limited to 20, and may be the number of bands such as 12 and 16.

The frame vector is sequentially created by the FFT unit 11. Further, as described above, the input voice is endlessly picked up by the microphone 1. Therefore, by sequentially performing the fast Fourier transform process in the FFT unit 11, it is possible to create a frame vector corresponding to the input voice that is endlessly picked up. In the present embodiment, as an example, it is assumed that a frame vector is created every 5 msec (every first predetermined time). The frame vector creation interval is not limited to 5 msec, and may be 8 msec, 12 msec, or any other creation interval.

Further, the FFT unit 11 performs a process of normalizing the magnitude of the voice waveform for each time with respect to the frame vector created by the fast Fourier transform process. Let t be the time when the frame vector is created based on the input voice. This time t also corresponds to the time when the input voice is input to the FFT unit 11 via the microphone 1. The frame vector (spectral vector) created at time t is indicated by f (t, x). However, 1 ≦ x ≦ X, t = 1, 2, ... x indicates the number of bands of the frame vector, and in the present embodiment, X = 20 as described above. The FFT unit 11 obtains the sum of the values of f (t, x) from x = 1 to x = X, and divides the value of f (t, x) by the value of the obtained sum to be normal. Calculate the value of the converted frame vector.

・・・式１
により求められる。そして、ＦＦＴ部１１は、正規化されたフレームベクトルｆ'（ｔ，ｘ）の値を、改めてｆ（ｔ，ｘ）に設定する。 Assuming that the normalized frame vector is f'(t, x), the value of f'(t, x) is

・ ・ ・ Equation 1
Demanded by. Then, the FFT unit 11 sets the value of the normalized frame vector f'(t, x) to f (t, x) again.

That is, the value of the frame vector in which the normalization processing is performed on the frame vector of the input voice obtained by the fast Fourier transform processing becomes the value of f (t, x).

By this normalization processing, the value of f (t, x) becomes 0 ≦ f (t, x) ≦ 1. In the input voice according to the present embodiment, the time t is determined as a substantially endless time interval, and the details thereof will be described later. The frame vector created in this way is output to the buffer memory unit 12.

The frame vector normalized by the FFT unit 11 (hereinafter, the normalized frame vector is simply referred to as a frame vector) is recorded in the buffer memory unit 12 in time series. That is, the frame vector is recorded in the buffer memory unit 12 in the order of time created (converted) by the FFT unit 11.

The buffer memory unit 12 secures a memory capacity capable of recording frame vectors for _TM pieces ( _{TM frames).} For example, _{T M} pieces of data of the frame vector is created for each 5 msec, i.e., it is possible to record the data of the time length of the input speech 5 msec × _{T M.} When the new frame vector of T _M +1 th is created by FFT unit 11, a buffer memory 12, the FIFO process (First-In First-Out process), the oldest frame is recorded Clear the vector, create other frame vector recorded by the recording side by side in the order in which they are created by the FFT unit 11, immediately before the T _M-th frame vectors created (newest frame vector (conversion ), Then the newest frame vector ( _TM + 1st frame vector) is recorded.

The data group T _M pieces worth of frame vectors recorded in the buffer memory unit 12, referred to as input data. As described above, since the frame vector is data obtained by converting the input voice into the frequency domain, it is vector data in which the signal level for each frequency is recorded. Further, the buffer memory unit 12, since the frame vector is sequentially generated for each 5msec is T _M pieces (T _M frame) recording the input data, the vertical axis represents the frequency, and the horizontal axis represents time 2 It is dimensional data and corresponds to the data in which the signal level is recorded at each coordinate.

Reference data in which a predetermined voice (voice of a predetermined sentence symbol) is recorded is recorded in the reference data memory unit 13. The reference data is a digital data of the reference audio. When creating reference data, first create symbolic expressions (sentence symbols) such as words and sentences. After that, the reference voice is created by converting these symbolic expressions (sentence symbols) into synthetic sounds and creating a waveform. In the present embodiment, the reference voice is a finite voice. That is, the waveform of the reference voice is composed of a finite time interval.

A frame vector for reference data is generated by performing a fast Fourier transform on the created reference voice, for example, every 5 msec (every second predetermined time). The frame vector for the reference data is also processed to normalize the magnitude of the voice waveform for each time. The time of the audio waveform of the reference data is indicated by τ. The range of time τ is 1 ≦ τ ≦ T. The time T will be described later. The frame vector (spectral vector) for the reference data generated by the fast Fourier transform process at time τ is indicated by z (τ, y). However, 1 ≦ y ≦ X. x indicates the number of bands in the frame vector for reference data, and in this embodiment, X = 20 as described above. For the frame vector z (τ, y) for reference data, the sum of the values of z (τ, y) from y = 1 to y = X is calculated, and z (τ, y) is calculated from the calculated sum value. ) Is divided to calculate the value of the normalized frame vector (spectral vector).

・・・式２
により求められる。そして、正規化されたフレームベクトルｚ'（τ，ｙ）の値を、改めてｚ（τ，ｙ）に設定する。 Assuming that the normalized frame vector is z'(τ, y), the value of z'(τ, y) is

・ ・ ・ Equation 2
Demanded by. Then, the value of the normalized frame vector z'(τ, y) is set to z (τ, y) again.

That is, z (τ, y) is the one obtained by normalizing the frame vector for the reference data obtained by the fast Fourier transform process. By this normalization processing, the value of z (τ, y) becomes 0 ≦ f (τ, y) ≦ 1.

As described above, the reference data is configured by collecting _{T frame vectors (T frames) (where TM} > T) every 5 msec (every second predetermined time). The reference data has the same data format as the input data, and is composed of a spectrum vector having 20 bands. Therefore, as an example, the reference data in the present embodiment can be determined to be data in which T (T frames) of frame vectors having 20 bands (20 elements) exist. That is, one reference data can be represented by a matrix having a length of 20 in the vertical direction and a length of T in the horizontal direction.

As the finite voice data (reference voice) used for the reference data, in the present embodiment, the synthetic voice is used as described above. For example, artificial voice data (waveform data by synthetic sound) using voice synthesis software or the like is generated based on the sentence symbol "Tomorrow is sunny or rainy". The reference data recorded in the reference data memory unit 13 is in the same data format as the input data described above, that is, two-dimensional data in which the vertical axis is frequency and the horizontal axis is time, and the signal level is set at each coordinate. It will be the recorded data.

The interval at which the frame vector is created in the reference data (every second predetermined time) is not limited to 5 msec. The frame vector creation interval may be 8 msec, 12 msec, or any other creation interval.

The matching processing unit 14 includes input data recorded in the buffer memory unit 12 ( _{a data group in which the frame vector of the input voice is normalized and TM} pieces are collected) and reference data recorded in the reference data memory unit 13 (reference data). The frame vector for the data is normalized and T data groups) are read out, and data matching processing and back tracing processing are performed using the two-dimensional continuous dynamic planning method. The matching processing unit 14 is provided with a working memory 14a used for the matching processing and the backtrace processing. In the working memory 14a, the amount of memory required for the calculation of the matching process and the backtrace process is secured. The amount of memory required for the matching process will be described later.

The matching processing unit 14 performs matching processing and backtrace processing using a two-dimensional continuous dynamic programming method. As described above, the input data to be the target of the matching processing and backtrace processing is an endless input voice. It is composed of a plurality of frame vectors based on the above, and has a feature that the content of the input data is updated every unit time. Therefore, the matching processing unit 14 performs a two-dimensional continuous dynamic programming method using input data that is updated endlessly. In the present embodiment, a two-dimensional continuous dynamic programming method performed using input data that is updated endlessly is referred to as an "incremental two-dimensional continuous dynamic programming method".

Further, a two-dimensional data pattern of input data having a vertical axis as a frequency and a horizontal axis as a time is referred to as an input pattern. In the input pattern, as described above, the horizontal axis indicating time is indicated by t, the vertical axis indicating frequency is indicated by x, and 1 ≦ x ≦ X. Further, the time t of the input data indicates the creation (conversion) time of the newest frame vector in the buffer memory unit 12. The input data recorded in the buffer memory unit 12 can be determined to be a substantially endless time interval because the oldest frame vector is erased when the newest frame vector is recorded.

Further, a two-dimensional data pattern of reference data having a vertical axis as frequency and a horizontal axis as time is referred to as a reference pattern. In the reference pattern, as described above, the horizontal axis indicating time is indicated by τ and the vertical axis indicating frequency is indicated by y, and 1 ≦ τ ≦ T and 1 ≦ y ≦ X.

The matching processing unit 14 performs matching processing and backtrace processing of the reference pattern (τ, y) and the input pattern (t, x) by using a two-dimensional continuous dynamic programming method. The processing content of the two-dimensional continuous dynamic programming method is a known technical content. For example, Japanese Patent Application Laid-Open No. 2010-165104 describes a description of matching processing between a reference image and an input image (target image). Assuming that the input image is the image f (i, j) and the reference image is the image g (m, n), the image f (i, j) and the image g (m, m, by using the two-dimensional continuous dynamic programming method are used. It is possible to find the optimum pixel correspondence with n).

In the two-dimensional continuous dynamic programming method, a four-dimensional space consisting of each variable i, j, m, n is constructed, and the local distance d (i, j, m, n) in the four dimensions = | f (i, j). -G (m, n) | is calculated. Further, the variable obtained by accumulating the local distance d (i, j, m, n) is defined as the cumulative local distance D (i, j, m, n). i is fixed, and for the fixed i, 1 ≦ j ≦ J, 1 ≦ m ≦ M, 1 ≦ n ≦ N, and by moving (changing) the values of j, m, and n within the above-mentioned ranges, respectively. The cumulative local distance D (i, j, m, n) is calculated. In this calculation, the cumulative local distance D (t, X, T, X) in the present embodiment can be obtained by associating i, j, m, n with the above-mentioned t, x, τ, y. can. The ranges of x, τ, and y corresponding to j, m, and n are 1 ≦ x ≦ X, 1 ≦ τ ≦ T, and 1 ≦ y ≦ X, as described above.

The cumulative local distance D (t, X, T, X) is a value obtained by optimally accumulating the local distance d at time t. Therefore, it is judged that the smaller the value of the cumulative local distance D (t, X, T, X) is, the more the optimum coordinate correspondence is obtained. That is, it can be considered that the smaller the value of the cumulative local distance D (t, X, T, X) is, the more the reference pattern and the input pattern match.

When the cumulative local distance D (t, X, T, X) is obtained, the value of t indicating the time of the input pattern is incremented by one. In this way, using the conventionally known two-dimensional continuous dynamic programming method, the time t is increased one by one (the time t is advanced), and the time t is accumulated according to the increase (progress). The method of calculating the local distance D (t, X, T, X) and the like (advancing the calculation) is a feature of the above-mentioned "incremental two-dimensional continuous dynamic programming method". The matching processing unit 14 calculates the cumulative local distance D (t, X, T, X) based on the incremental two-dimensional continuous dynamic programming method.

Further, the matching processing unit 14 performs backtrace processing in the two-dimensional continuous dynamic programming method based on the calculated cumulative local distance D (t, X, T, X). By performing the backtrace processing, it is possible to obtain the optimum coordinate correspondence between the coordinates (t, x) of the input pattern and the coordinates (τ, y) of the reference pattern. The backtrace process is a known process generally used in dynamic programming, and is described in detail in the above-mentioned JP-A-2010-165104 and the like.

The cumulative local distance D (t, X, T, X) is a two-dimensional reference pattern consisting of time-frequency and a two-dimensional input pattern in which the input time (time) is time-frequency up to time t. The two-dimensional matching result is shown. Since the cumulative local distance D (t, X, T, X) that is the matching result corresponds to the scalar value of time, the calculation result of the cumulative local distance D (t, X, T, X) is output side by side in the time direction. By doing so, the changing state of the value of the cumulative local distance D (t, X, T, X) for each time can be indicated by a series that strikes one wave.

The matching processing unit 14 calculates the cumulative local distance D (t, X, T, X) and then multiplies the value X and the value T (XT) to obtain the cumulative local distance D (t, X). , T, X) is divided to obtain D (t, X, T, X) / (XT). Since the cumulative local distance D (t, X, T, X) is a value that depends on T and X, the value of the cumulative local distance D (t, X, T, X) is divided by (XT). As a result, the value of D (t, X, T, X) / (XT) always falls between 0 and 1. D (t, X, T, X) / (XT) corresponds to the normalized cumulative local distance D (t, X, T, X).

3A and 3B are diagrams showing the time change (change state) of the value of D (t, X, T, X) / (XT) obtained by the matching process. As shown in FIGS. 3 (a) and 3 (b), the trajectory of the wave showing the output of D (t, X, T, X) / (XT) has valleys and peaks. .. A local valley portion (valley bottom portion) as shown in FIGS. 3A and 3B is referred to as a dip (local depression).

FIG. 3 (a) shows the case where the value of D (t, X, T, X) / (XT) in the dip becomes zero, and FIG. 3 (b) shows the case where the value of D (t, t, in the dip) becomes zero. The case where the value of X, T, X) / (XT) is not zero is shown. In the wave trajectory of shown in FIG. 3 (a) (b), if a valid dip is present, and time t ^* of the reference patterns dip occurs, and the time t ^* previous input patterns, partially or All match. The input pattern is continuously input without a break before ^{the time t *.} Therefore, it is a local value of the cumulative local distance D (t, X, T, X) which time (time) starts (temporal start point) and the reference pattern and the input pattern partially or wholly match. Two-dimensional continuous dynamic from the ^{coordinates (t * , X, T, X) indicating the cumulative local distance D (t *} , X, T, X) at the time t ^* (time when a valid dip occurs). It can be determined by performing back tracing processing by the planning method and obtaining the time t ^{* 2 to be (t * 2,1,1,1) after the fact.}

As shown in FIGS. 3A and 3B, the time (time) at which the dip occurs is defined as the time t ^*, and D in the section [t ^* -2T, t ^* ] (t ^* -2T to t ^*). Find the maximum value of (t, X, T, X) / (XT). ^{The value obtained by subtracting the value of D (t *} , X, T, X) / (XT) from the maximum value is defined as the difference value h. Since the value of D (t, X, T, X) / (XT) is a value included between 0 and 1, the difference value h is also 0 ≦ h ≦ 1. In the voice recognition device 100 according to the present embodiment, when the difference value h exceeds a preset first threshold value, it is determined that a dip effective for voice recognition exists.

Here, the first threshold value is a value preset by an experiment or the like. For example, by experimentally using the input voice and the reference voice that can be determined to include the reference voice and obtaining the difference value h, a threshold value that serves as a boundary as to whether or not the input voice includes the reference voice is used. (First threshold value) can be set in advance.

The interval of [t ^* -2T, t ^* ] is determined in consideration of the limit that can be the start end, that is, the limit of the matchable range when the ^{time t * is the end point. The maximum value of D (t *} , X, T, X) / (XT) in the interval of [t ^* -2T, t ^* ] is among the elements of the input pattern ending at ^{time t *.} , Means that there is an element that is least matched to the reference pattern. Further, the above-mentioned difference value h is a value indicating a decrease in the cumulative value due to partial matching from the stage of the least matching. The reason for this is that, as described above, the cumulative local distance D (t, X, T, X) is the cumulative value of the local distance d.

When D (t ^* , X, T, X) / (XT) is zero (that is, when the cumulative local distance D (t ^* , X, T, X) is zero) (FIG. 3 (a)) In the case), the values of the elements of the reference pattern composed of the two-dimensional data pattern are completely the values of all the elements corresponding to the optimum in the input pattern also composed of the two-dimensional data pattern. There is a matching section. Such perfect matching of the values of all the corresponding elements is called perfect matching.

In the case of perfect matching, the values of the elements of the reference pattern will exactly match the values of all the elements that correspond optimally in the input pattern, but there are situations where the cocktail party effect can be judged to be effective. , Not necessarily when the value of the element of the reference pattern exactly matches the value of the element of all the corresponding input patterns.

For example, even if the reference voice corresponding to the reference pattern is the voice of a woman with a predetermined sentence symbol and the input voice corresponding to the input pattern is the voice of a man with the same sentence symbol. If the sentence symbol of each voice is the same, it is determined that the reference voice is included in the input voice, and the cocktail party effect can be judged to be effective. In this case, the values of the elements of the reference pattern do not completely match (perfect match) with the values of all the elements that correspond optimally in the input pattern, but partially match (partially match). become. Exact matching (perfect matching) is an exception, and usually partial matching (partial matching) is common.

Even in the case of partial matching, if it can be recognized that the predetermined sentence symbol included in the reference voice is included in the input voice, the voice recognition corresponding to the cocktail party effect can be performed. That is, even in the case of partial matching, it may be determined that the element corresponding to the sentence symbol of the reference pattern is included in the element corresponding to the optimum in the input pattern. Therefore, in the voice recognition device 100 according to the present embodiment, it is based on partial matching, which corresponds to the case where the value of the element of the reference pattern partially matches the value of the element corresponding to the optimum in the input pattern. To perform voice recognition.

^{When performing voice recognition, backtrace processing is performed from (t *} , X, T, X) of the cumulative local distance D (t ^* , X, T, X). The backtrace processing of the cumulative local distance D (t ^* , X, T, X) corresponds to the processing of searching for the grid points (t, x, τ, y) in the four-dimensional space. The search point (t, x, τ, y) by the back trace of this four-dimensional space is considered as a pair of (t, x) and (τ, y). The search points (t, x, τ, y) obtained by the backtrace processing are the two-dimensional data patterns (reference patterns) of the reference data, and the coordinates (τ, y) of the time-frequency pattern are 2 of the input data. The coordinates (t, x) of the time-frequency, which is a two-dimensional data pattern (input pattern), indicate the coordinates of the matching portion. The local distance of the matching coordinates is d (t, x, τ, y). A light-dark pattern is created by projecting (arranging) the value of the local distance d (t, x, τ, y) onto the (τ, y) point.

Then, the region of the grid point where the value of the grid point (τ, y) is equal to or less than the preset second threshold value (this threshold value is a value between 0 and 1) is obtained. This second threshold value is a value set in advance by an experiment or the like. It can be determined that the reference pattern (τ, y) and the input pattern (t, x) are optimally matched in the coordinates of the grid points where the value of the grid points is equal to or less than the second threshold value or in the region thereof. .. That is, when the value of the local distance d (t, x, τ, y) is equal to or less than the preset second threshold value, the reference pattern (τ, y) and the input pattern (t, x) are optimal. It can be judged that it matches with.

Specifically, using the coordinates z (τ, y) of the reference pattern (reference data) and the coordinates f (t, x) of the input pattern (input data), the local distance d (t, x, τ, y),
Local distance d (t, x, τ, y) = | f (t, x) -z (τ, y) |
To calculate the cumulative local distance D (t, X, T, X). Furthermore, by performing backtrace processing from the cumulative local distance D (t, X, T, X), the coordinates z (τ, y) of the reference pattern and the coordinates f (t, x) of the input pattern are optimally matched. Find the local distance d (t, x, τ, y) of the coordinates to be used.

By finding the region of the grid points (τ, y) where the local distance d (t, x, τ, y) thus obtained is equal to or less than the second threshold value, the coordinates f (t, x) of the input pattern are obtained. ) And the coordinates z (τ, y) of the reference pattern, and the corresponding coordinates (time-frequency) can be obtained.

Next, by finding the maximum region from the region composed of the coordinate z (τ, y) points that are equal to or less than the second threshold value, and finding the minimum time and the maximum time of that region. , Find the start and end of the area.

Specifically, the maximum region is obtained from the region of the reference pattern (reference data) in which the value of the local distance d (t, x, τ, y) is equal to or less than the second threshold value. Next, the minimum time and the maximum time are obtained in the range of the maximum area of the obtained reference pattern (reference data). That is, among the corresponding coordinates (τ, y) of the reference pattern in which the value of the local distance d (t, x, τ, y) is equal to or less than the second threshold value, the time at which the time τ becomes the minimum and the time at which the time τ becomes the maximum And ask.

FIG. 4 shows the maximum region where the value of the local distance d (t, x, τ, y) is equal to or less than the second threshold value. Based on this region, the minimum time τs and the maximum time τe are obtained. In FIG. 4, the reference pattern (reference data) is the voice data of the sentence symbol "tomorrow is sunny, rainy", and the sentence symbol corresponding to "sunny" is included in the input pattern (input data). The case where the voice data of is included, that is, the case where partial matching is established is shown as an example. The entire voice of the reference pattern (reference data) (synthetic voice of sentence symbols) is "Tomorrow is sunny or rainy", but from time τs to time τe in the region of the reference pattern (reference data) in FIG. The sentence symbol of the voice (synthetic voice) corresponding to is "sunny". The portion of the reference data from the time τs to the time τe corresponds to the portion recognized from the input data (for example, the voice data of simultaneous utterances by a plurality of speakers) input endlessly.

FIG. 5 is a flowchart showing the processing contents of the matching processing unit 14 described above. As described above, the matching processing unit 14 has a reference pattern (τ, y) composed of a time-frequency two-dimensional data pattern and an input pattern (t, x) composed of a time-frequency two-dimensional data pattern. Incremental two-dimensional continuous dynamic programming is applied to and. Then, the matching processing unit 14 obtains the cumulative local distance D (t, X, T, X) obtained by accumulating the local distances d (t, x, τ, y) (S.1, cumulative local distance calculation function). ..

The matching processing unit 14 divides the cumulative local distance D (t, X, T, X) by (TX) to change the value of D (t, X, T, X) / (TX) over time. Based on this, the dip (valley bottom portion) of the wave trajectory is detected (S.2, dip detection function). Next, the difference value h in the section of [t ^* -2T, t ^* ] is obtained with the time corresponding to the dip as the time t ^* (S.3, difference value calculation function). After that, the matching processing unit 14 determines whether or not the difference value h is equal to or greater than a preset first threshold value (S.4, reference pattern area detection function).

When the difference value h is smaller than the first threshold value (No in S.4), the matching processing unit 14 determines that all or part of the audio data of the reference pattern is not included in the input pattern. do. Then, the matching processing unit 14 outputs information to the effect that the reference pattern is not included in the input pattern to the detection determination unit 15 (S.5).

On the other hand, when the difference value h is equal to or greater than the first threshold value (Yes in S.4), the matching processing unit 14 has a cumulative local distance D (t ^* , X, T, X) (t ^* , X, X,). By performing the backtrace processing from T, X), the local distance d (t, t,) of the coordinates at which the optimum matching between the coordinates z (τ, y) of the reference pattern and the coordinates f (t, x) of the input pattern is established x, τ, y) is calculated (S.6, reference pattern area detection function).

Then, it is determined whether or not there is a reference pattern region in which the obtained local distance d (t, x, τ, y) is equal to or less than the second threshold value (S.7, reference pattern region detection function). That is, it is determined whether or not there are coordinates (τ, y) of the reference pattern in which the local distance d (t, x, τ, y) is equal to or less than the second threshold value. When there is no reference pattern region in which the local distance d (t, x, τ, y) is equal to or less than the second threshold value (No in S.7), the matching processing unit 14 includes the reference pattern in the input pattern. The information indicating that the information has not been obtained is output to the detection / determination unit 15 (S.5).

On the other hand, when there is a reference pattern region (coordinates) in which the local distance d (t, x, τ, y) is equal to or less than the second threshold value (yes in S.7), the matching processing unit 14 obtains it. The minimum time τs and the maximum time τe in the range of the specified region are obtained (S.8, reference pattern region detection function). Then, the information of the corresponding time range indicating the time range from the minimum time to the maximum time is output to the detection determination unit 15 (S.9).

After outputting the information that the reference pattern is not included in the input pattern (S.5) or outputting the information of the corresponding time range (S.9), the matching processing unit 14 is transferred by the FFT unit 11. A new frame vector for the input pattern (input data) is generated, the oldest frame vector recorded in the buffer memory unit 12 is erased, and it is determined whether or not a new frame vector has been recorded. That is, the matching processing unit 14 determines whether or not the input data (input pattern) recorded in the buffer memory unit 12 has been updated (S.10).

If the buffer memory unit 12 has not been updated (No in S.10), the matching processing unit 14 will perform S.M. The process of 10 is repeated. When the buffer memory unit 12 is updated (yes in S.10), the matching processing unit 14 performs processing in S.M. Move to 1. That is, the matching processing unit 14 applies the two-dimensional continuous dynamic programming method to the reference pattern (τ, y) and the new input pattern (t, x), so that the local distance d (t, t, The process (S.1) for obtaining the cumulative local distance D (t, X, T, X) by accumulating x, τ, y) is executed, and then the above-mentioned process (S.2 to S.10) is performed. Run.

The detection determination unit 15 acquires information indicating that the reference pattern is not included in the input pattern and information on the corresponding time range from the matching processing unit 14. Then, the detection determination unit 15 determines whether or not the voice of the reference pattern (speech of the sentence symbol) is included in the input pattern based on the acquired information (voice judgment function).

When information indicating that the reference pattern is not included in the input pattern is acquired from the matching processing unit 14 (in the case of S.5), the detection determination unit 15 inputs the voice of the reference pattern (voice of the sentence symbol). Judge that it was not included in the pattern (it did not exist) (voice judgment function). Then, the detection determination unit 15 outputs information to the effect that the reference voice is not included in the input voice at all to the display unit 6. The display unit 6 displays the information received from the detection determination unit 15 and notifies the user that the matching has not been established.

Further, when the information of the corresponding time range indicating the time range from the minimum time τs to the maximum time τe is acquired from the matching processing unit 14 (in the case of S.9), the detection determination unit 15 uses the entire reference pattern. Alternatively, it is determined that some voices (voices of sentence symbols) are included (existed) in the input pattern (speech judgment function). Then, the detection determination unit 15 outputs information to the effect that the reference voice is included in the input voice to the display unit 6. Further, the detection determination unit 15 outputs to the display unit 6 information that the voice (voice of the sentence symbol) in the corresponding time range (time τs to time τe) in the reference voice is included in the input voice. The display unit 6 displays the information received from the detection determination unit 15 and notifies the user that the partial matching has been established (voice determination function).

In this way, the matching processing unit 14 executes the processing shown in FIG. 5, and the detection determination unit 15 causes the display unit 6 to display the presence or absence of matching based on the processing result of the matching processing unit 14. On the other hand, the input pattern contains all or part of the reference pattern (in the case of S.9), or the input pattern does not contain the reference pattern at all (in the case of S.5). I can inform you. Based on the result displayed on the display unit 6, the user can determine whether or not the mixed voice (input voice) by a plurality of speakers includes a predetermined voice (reference voice, voice of a sentence symbol). It becomes.

As described above, in the voice recognition device 100 according to the embodiment, it is determined that the input voice includes a sentence symbol in a time range from the time τs of the reference voice to the maximum time τe. In particular, when the value of D (t, X, T, X) / (XT) is close to zero (0), the start time τs may be 1 and the end time τe may be T. be. In this case, it is determined that all the sentence symbols of the reference voice are included in the input voice. In normal partial matching, the start time τs is a value larger than 1, and the end time τe is a value smaller than T. As a result, a part of the sentence symbol of the reference voice becomes a recognition target. Further, the deeper the dip, the longer the length of the reference voice portion (the sentence symbol corresponding to that portion) included in the input voice.

Next, the difference between the voice recognition method shown in the present embodiment and the conventionally known voice recognition method will be described. The region determined by the reference pattern matrix (time-frequency pattern, which is a two-dimensional representation) shown in the present embodiment is a partial region of the two-dimensional plane. Therefore, in the matrix of the reference pattern, the time axis and the frequency axis can be considered, and there is a region that is a part on the time axis and a part on the frequency axis. Moreover, the shape of the region is various. Therefore, the reference data in the present embodiment can be regarded as a regional collection of two-dimensional points consisting of time and frequency, and it may be handled as a unit called a vector for voice analysis created for each time. No.

On the other hand, in the conventional speech recognition research, the reference data is judged in vector units in most cases. For example, as conventional speech recognition research, vector unit matching processing by temporal transition, vector unit quantization processing of the code used in the Hidden Markov Model, which is one of the typical methods of speech recognition, or Studies such as creating a continuous distribution of vectors have been conducted. As described above, the processing method of the voice recognition device 100 according to the present embodiment is also different from the conventional voice recognition processing method in terms of how to capture the reference data and how to think about it.

As an exception, a formant tracking method for a time-frequency pattern is known. This tracking method is characterized in that a point having a large value (referred to as a formant) is tracked in a time-frequency pattern, and vowels are mainly recognized based on the tracking result. However, this tracking result is only the result of the locus of the point sequence, and "the partial region is detected by optimally matching the two-dimensional patterns indicating the regions with each other using the two-dimensional continuous dynamic programming method. The voice recognition method according to the present embodiment is not performed.

・・・式３
ここで、ｆ_０（ｔ，ｘ）は、入力パターンにおけるベクトルのｘ要素を示し、ｚ_０（τ，ｘ）は参照パターンにおけるベクトルのｘ要素を示す。式３に示すように、それぞれの要素の差分（ｆ_０（ｔ，ｘ）−ｚ_０（τ，ｘ））の絶対値の和を求めることによって、ベクトルを基準とする入力パターンのｆ_０（ｔ，ｘ）と、ベクトルを基準とする参照パターンのｚ_０（τ，ｘ）以外の要素ｚ_０（τ，ｙ）（ｙ≠ｘ）との関係が、検出対象から外される。そして、話者の話すスピードの違いにより、ベクトルを基準とする参照データと、ベクトルを基準とする入力データとの対応点が異なるという問題を、動的計画法におけるマッチング（整合）を行い、時間軸tと時間軸τとの対応を取ることにより解決して、音声認識処理を行う。このように、ベクトルを基準として音声認識処理を行う方法は、ベクトルベースのマッチング処理と呼ばれており、従来の音声認識処理において、広く用いられる手法である。 In the conventional method, as described above, since the reference data is determined in vector units, a method of matching the reference pattern by time change is generally performed by using a dynamic programming method. When finding the similarity of time change in vector units, when the time change of the reference pattern and the time change of the input pattern are different due to the difference in the speaking speed of the speaker, the change state of the pattern between the reference pattern and the input pattern is changed. It does not match. Therefore, by using the dynamic programming method, non-linear matching due to time change is performed. Here, when matching is obtained by the conventional method, since the data is determined in vector units, the matching amount is obtained by the distance d (t, τ) between the following vectors.

... Equation 3
Here, f ₀ (t, x) indicates the x element of the vector in the input pattern, and z ₀ (τ, x) indicates the x element of the vector in the reference pattern. As shown in Equation 3, the difference of each element _{(f 0 (t, x)} -z 0 (τ, x)) by summing the absolute value of the input pattern relative to the vector _f 0 ( _{The relationship between t, x) and the element z 0} (τ, y) (y ≠ x) _{other than z 0} (τ, x) of the reference pattern based on the vector is excluded from the detection target. Then, matching (matching) in the dynamic programming method is performed to solve the problem that the correspondence point between the reference data based on the vector and the input data based on the vector is different due to the difference in the speaking speed of the speaker. The problem is solved by taking the correspondence between the axis t and the time axis τ, and the voice recognition process is performed. As described above, the method of performing the speech recognition process based on the vector is called the vector-based matching process, and is a method widely used in the conventional speech recognition process.

FIG. 6 is a diagram for explaining a conventional speech recognition processing method based on vector-based matching processing. In FIG. 6, since the vector is used as a unit, the similarity of the elements in the set (vector group) of a plurality of vectors having different times is not the target of voice recognition. To the last, there is a feature that only the similarity between vectors is treated as the object of recognition.

However, the speech recognition method of the speech recognition device 100 according to the present embodiment is characterized in that the reference data and the input data are not captured as vector-based data, but are captured as area-based data and the matching process is performed. It is supposed to be.

FIG. 7 is a diagram for explaining that the reference data and the input data are regarded as area-based data and the matching process is performed. In the incremental two-dimensional continuous dynamic planning method used in the present embodiment, the two-dimensional reference pattern consisting of time-frequency and the pattern showing the analysis result of the input waveform are two-dimensional consisting of the time-frequency pattern. A non-linear correspondence in the region is required with the input pattern.

Then, the matching processing unit 14 of the voice recognition device 100 obtains the above-mentioned value of D (t, X, T, X) / (XT) as a value proportional to the area of the matching region. The value of D (t, X, T, X) / (XT) decreases, and a dip (local depression) occurs at a position showing a local minimum value. ^{Let t *} be the time when dip (local depression) occurs, and as already explained, the maximum value of D (t, X, T, X) / ( ^{XT) in the [t *} -2T, t ^{*] interval.} from the time that indicates the time that dip occurs ^{t *} to D (t, X, T, X) the value of / (X · T) is reduced. This decrease in the value of D (t, X, T, X) / (XT) is caused by the matching region. Matching in such a region does not occur in the vector-based (frame unit) matching process as described with reference to FIG. If all (time-frequency) coordinates in the reference pattern (time-frequency) and all (time-frequency) coordinates in the input pattern (time-frequency) do not have a non-linear correspondence, matching in the region will be performed. Does not occur. In addition, when finding a dip (local depression) from the locus of the D (t, X, T, X) / (XT) value that is the output, it is possible to use incremental two-dimensional continuous dynamic programming. Optimal.

The working memory 14a provided in the matching processing unit 14 is used for matching processing and the like in the above-mentioned incremental two-dimensional continuous dynamic programming method. In the incremental two-dimensional continuous dynamic programming method, the input voice is input endlessly, but the memory amount of the working memory 14a cannot be an endless capacity. Therefore, the amount of memory is limited to the range actually used for calculation, and the contents of the working memory 14a are updated every frame, that is, every time t. By this data update, even when the input voice is input endlessly, the value of the cumulative local distance D (t, X, T, X) can be calculated with a certain amount of memory.

At time t, the frequency x of the input pattern is 1 ≦ x ≦ X, the frequency of the reference pattern is 1 ≦ y ≦ X, and the time τ is 1 ≦ τ ≦ T. , X) values are calculated for one input spectrum. The range of time t that affects the calculation of the value of the cumulative local distance D (t, X, T, X) is the range of [t-2T, t] and is constant. Therefore, the required amount of memory is 2T × X × X × T by using the maximum value 2T in the range of time t, the maximum value T of time τ, and the maximum value X of the frequency of the reference pattern and the input pattern. Become. This amount of memory (2T × X × X × T) is constant. When the time t becomes t + 1, the entire cumulative local distance D (t, X, T, X) is shifted by the time +1, that is, the data corresponding to the time t is shifted to the time t-1. The calculation is performed using the newly input input pattern.

As described above, the voice recognition device and the voice recognition program according to the present invention have been described in detail by showing the voice recognition device 100 as an example. However, the voice recognition device and the voice recognition program according to the present invention are not limited to the configuration of the voice recognition device 100 in the embodiment.

For example, in the voice recognition device 100 according to the embodiment, a case where only one microphone 1 is provided has been described. However, the number of microphones installed is not limited to one. In the voice recognition device and the voice recognition program according to the present invention, sounds uttered by a plurality of speakers at the same time are regarded as one sound (waveform, single voice), and one sound (single voice) corresponds to reference data. It is characterized by recognizing the sound to be played. Therefore, unlike the ICA, which is a conventionally known technique, it is not necessary to separate the waveform of the mixed voice. Therefore, it is sufficient if the sounds emitted by a plurality of speakers can be picked up as one sound (waveform), and if the sound can be picked up as one sound (waveform) (single voice), one microphone is required. However, even if there are a plurality of units, the number is not particularly limited.

Further, in the voice recognition device 100 according to the embodiment, a case where the input voice is picked up by the microphone 1 and continuously converted into the data in the frequency domain by the FFT unit 11 and the input voice is input endlessly will be described. bottom. However, it is not always limited to the configuration in which the input voice is input endlessly. In the speech recognition device and the speech recognition program according to the present invention, the reference data is used as a two-dimensional reference pattern consisting of time-frequency, and the input data is used as a two-dimensional input pattern consisting of time-frequency for two-dimensional continuous dynamic planning. It is characterized in that speech recognition processing is performed by performing non-linear matching processing between an input pattern and a reference pattern using a method.

Therefore, if the input data can be used as a two-dimensional input pattern consisting of time and frequency, the input voice is not necessarily limited to endless voice, and even if it is based on a finite input voice, It can be used in a voice recognition device or the like according to the present invention. Therefore, when performing voice recognition processing based on the input voices of a plurality of speakers, it is of course possible for the input voices to be endless sounds (waveforms) that do not assume the start and end. However, it is also possible to perform voice recognition processing for sounds (waveforms) that assume either the start end or the end end or both.

Further, when artificial voice data (waveform data by synthetic sound) generated by using voice synthesis software or the like is used as the reference data recorded in the reference data memory unit 13 of the voice recognition device 100 according to the embodiment. Was explained. However, the reference data used in the voice recognition device and the voice recognition program according to the present invention is not necessarily limited to artificial voice data. In the voice recognition device and the voice recognition program according to the present invention, the voice recognition process is performed by the non-linear matching process of the input pattern and the reference pattern. Therefore, the data format (type) of the reference data is not particularly limited as long as the reference data is voice data including sentence symbols.

In the speech recognition device and the speech recognition program according to the present invention, speech recognition is performed by non-linear matching of a two-dimensional data pattern consisting of time and frequency. Therefore, the reference data is a waveform of a sentence symbol that is the basis of sound. Anything is fine. Therefore, the reference data may be generated by directly recording human voice, regardless of the type of synthetic voice (male sound or female sound). As described above, since the voice recognition process can be performed by the non-linear matching process of the reference data and the input data, various voice data can be used as the reference data. In addition, since voice recognition processing can be performed by non-linear matching processing between reference data and input data, in a situation where multiple unspecified speakers speak at the same time, the number of speakers increases or decreases in the middle, and the input data Even when the number of speakers in the above changes, it becomes possible to perform voice recognition.

Further, in the voice recognition device and the voice recognition program according to the present invention, the input voice is regarded as a single voice and the voice recognition process is performed without performing waveform separation in the input data. Therefore, the voice recognition process can be performed by one processing method as shown in FIG. 5, for example, without requiring a plurality of processes such as voice separation. Therefore, the processing can be simplified, and it is possible to suppress an increase in processing load and occurrence of processing delay.

1 ... Microphone 2 ... CPU (control means, cumulative local distance calculation means, dip detection means, difference value calculation means, reference pattern area detection means)
3 ... ROM
4 ... RAM (input pattern recording means)
5 ... Recording unit (input pattern recording means)
6 ... Display unit 10 ... A / D conversion unit 11 ... FFT unit (frequency conversion means, cumulative local distance calculation means)
12 ... Buffer memory unit (input pattern recording means)
13 ... Reference data memory unit 14 ... Matching processing unit (control means, cumulative local distance calculation means, dip detection means, difference value calculation means, reference pattern area detection means)
14a ... Working memory 15 ... Detection and determination unit (control means, voice judgment means)
100 ... Speech recognition device

Claims (5)

By converting the input voice into data in the frequency region every first predetermined time, an input pattern composed of two-dimensional data of frequency x (1 ≦ x ≦ X) and time t, and every second predetermined time By converting the reference voice into data in the frequency region, the input is performed using a reference pattern composed of two-dimensional data of frequency y (1 ≦ y ≦ X) and time τ (1 ≦ τ ≦ T). The coordinate f (t, t, normalized by dividing the sum of the values of the coordinate f (t, x) from x = 1 to x = X from the value of the coordinate f (t, x) of the pattern. By calculating the value of x) and dividing the value of the coordinate z (τ, y) of the reference pattern by the sum of the values of the coordinate z (τ, y) from y = 1 to y = X, The normalized value of the coordinate z (τ, y) is calculated, and the normalized coordinate f (t, x) of the input pattern and the reference pattern are normalized by a two-dimensional continuous dynamic programming method. by performing been the coordinate z (τ, y) a non-linear matching process with the value of normalized the coordinate f (t, x) the normalized from the value of the coordinate z (τ, y) The local distance d (t, x, τ, y) indicating the absolute value of the value obtained by subtracting is calculated, and the calculated local distance d (t, x, τ, y) is accumulated to accumulate the cumulative local distance D ( Cumulative local distance calculation means for calculating t, X, T, X) and
The value of the cumulative local distance D (t, X, T, X) calculated by the cumulative local distance calculating means is divided by the product of T and X, and D (t, X, T, X) / (T. A dip detecting means for calculating X) and detecting a dip indicating a valley bottom portion from a change state of the value of D (t, X, T, X) / (TX) at time t.
^{Let t * be} the generation time of the dip detected by the dip detecting means, and the D (t, X, T, X) / (XT) in the time interval from the time t ^* -2T to the time t ^*. A difference value calculation means for calculating the difference value h by subtracting the value of D (t ^{* , X, T, X) / (XT) at the time t *} from the maximum value of the value.
When the difference value h calculated by the difference value calculation means is larger than the preset first threshold value, the cumulative local distance D (t ^* , X, T, X) (t ^* , X, T, By performing backtrace processing from X), the normalized value of the coordinate f (t, x) and the normality of the coordinate z (τ, y) that optimally matches the coordinate f (t, x). The value of the local distance d (t, x, τ, y) is calculated from the converted value, and the calculated value of the local distance d (t, x, τ, y) is set in advance as a second threshold value. A reference pattern area detecting means for detecting the set of the coordinates z (τ, y) of the reference pattern below as a target area, and
A voice recognition device comprising: a voice recognition means for determining that the input voice includes the reference voice when the target area is detected by the reference pattern area detecting means.
When the target area is detected, the reference pattern area detecting means detects the minimum time τs and the maximum time τe of the coordinates z (τ, y) constituting the target area.
The voice recognition according to claim 1, wherein the voice recognition means determines that the voice included in the time range from the time τs to the time τe among the reference voices is included in the input voice. Device. A frequency conversion means for converting the input voice into data in the frequency domain at the first predetermined time, and
It has an input pattern recording means in which the data in the frequency domain converted by the frequency conversion means is normalized and recorded in the converted time order.
The input pattern recording means erases the oldest data recorded in the input pattern recording means each time new data is generated by the frequency conversion means, and generates the new data immediately before the new data. Record next to the data
The cumulative local distance calculating means reads out all the data recorded in the input pattern recording means while maintaining the recording order and uses it as the input pattern, whereby the cumulative local distance D (t, X, T, The voice recognition device according to claim 1 or 2, wherein X) is calculated. It is a voice recognition program of a voice recognition device for determining whether or not the reference voice is included in the input voice.
As a control means of the voice recognition device,
By converting the input voice into data in the frequency region every first predetermined time, an input pattern composed of two-dimensional data of frequency x (1 ≦ x ≦ X) and time t, and every second predetermined time. By converting the reference voice into data in the frequency region, a reference pattern composed of two-dimensional data of frequency y (1 ≦ y ≦ X) and time τ (1 ≦ τ ≦ T) is used. The coordinate f (normalized) by dividing the sum of the values of the coordinates f (t, x) from x = 1 to x = X from the value of the coordinates f (t, x) of the input pattern. To calculate the value of t, x) and divide the sum of the values of the coordinates z (τ, y) from y = 1 to y = X from the value of the coordinates z (τ, y) of the reference pattern. To calculate the value of the coordinate z (τ, y) normalized by the two-dimensional continuous dynamic programming method, the coordinate f (t, x) normalized to the input pattern and the reference pattern. The coordinate z (τ, y) normalized from the value of the coordinate f (t, x) normalized by performing a non-linear matching process with the coordinate z (τ, y) normalized. The local distance d (t, x, τ, y) indicating the absolute value of the value obtained by subtracting the value of) is calculated, and the calculated local distance d (t, x, τ, y) is accumulated and accumulated locally. Cumulative local distance calculation function that calculates the distance D (t, X, T, X),
The value of the cumulative local distance D (t, X, T, X) calculated by the cumulative local distance calculation function is divided by the product of T and X, and D (t, X, T, X) / (T. A dip detection function that calculates X) and detects the dip indicating the valley bottom from the change state of the value of D (t, X, T, X) / (TX) at time t.
^{Let t * be} the generation time of the dip detected by the dip detection function, and the D (t, X, T, X) / (XT) in the time interval from the time t ^* -2T to the time t ^*. A difference value calculation function that calculates the difference value h by subtracting the value of D (t ^{* , X, T, X) / (XT) at time t *} from the maximum value of the value.
When the difference value h calculated by the difference value calculation function is larger than the preset first threshold value, the cumulative local distance D (t ^* , X, T, X) (t ^* , X, T, By performing backtrace processing from X), the normalized value of the coordinate f (t, x) and the coordinate z (τ, y) that optimally matches the coordinate f (t, x). The value of the local distance d (t, x, τ, y) is calculated by the normalized value, and the calculated value of the local distance d (t, x, τ, y) is set in advance. A reference pattern area detection function that detects a set of coordinates z (τ, y) of the reference pattern that is equal to or less than a threshold value as a target area.
A voice recognition program characterized by realizing a voice determination function of determining that the input voice includes the reference voice when the target area is detected by the reference pattern area detection function.

To the control means
When the target area is detected in the reference pattern area detection function, the minimum time τs and the maximum time τe of the coordinates z (τ, y) constituting the target area are detected.
The voice recognition according to claim 4, wherein in the voice determination function, it is determined that the voice included in the time range from the time τs to the time τe among the reference voices is included in the input voice. program.

Images