JP5385876B2 - Speech segment detection method, speech recognition method, speech segment detection device, speech recognition device, program thereof, and recording medium - Google Patents

Description

  The present invention relates to a speech segment detection method and speech segment detection device for determining a speech segment in a sound signal input from a microphone or the like, a speech recognition method and speech recognition device for a speech segment identified by the speech segment detection method, and the like The present invention relates to a program and a recording medium.

  There is a technique for digitizing and transmitting a sound signal recorded by a microphone or the like, and a voice recognition technique for converting the sound signal into a sequence of words or characters. Here, generally, the input sound signal includes a voice section in which the target person is speaking and a non-speech section in which the target person is not speaking. In voice communication and voice recognition technology, a non-voice section may be unnecessary. Here, the target person refers to a speaker who speaks into a microphone, a person who speaks a voice to be subjected to voice recognition, and the like. Therefore, the non-speech section may include not only silence and noise but also speech other than the target person (such as surrounding speech).

  If the voice section can be identified, it is only necessary to transmit the voice section in voice communication, so the amount of information to be transmitted can be reduced. Moreover, if it is used as preprocessing for speech recognition, recognition errors caused by recognizing non-speech intervals can be avoided. In other words, since the non-speech section may contain noise, the voice of a person other than the target person, etc., when the non-speech section is recognized as a voice, the noise or the voice of the other person is mistaken for the target person's voice. It may recognize and get an incorrect recognition result. Note that identifying a voice section from a sound signal is called voice section detection (VAD: Voice Activity Detection).

  FIG. 1 shows a configuration of a conventional general speech segment detection device 10. The voice section detection device 10 has a voice input unit 11 that acquires a sound signal x (k) from a microphone or the like, and a voice / non-voice that determines whether the voice is voice or non-voice at certain time intervals (hereinafter referred to as frames). The voice determination unit 13 and the result output unit 15 that outputs the result y of the voice / non-speech segment determination.

  FIG. 2 shows a configuration example of a conventional speech recognition apparatus 20 that uses speech segment detection. The voice recognition unit 24 receives only the sound signal of the voice section from the voice / non-voice judgment unit 13 and performs voice recognition processing. The result output unit 25 outputs the recognition result y ′.

  As a method for performing speech / non-speech determination in speech section detection, for example, there are the following methods. Non-Patent Document 1 discloses a speech signal section estimation method that defines signal state transitions. In this method, an input sound signal is regarded as a signal that transitions between a voice state and a non-voice state as time passes. The state transition between the voice state and the non-voice state is determined based on the probability that the input sound signal belongs to the voice state and the probability that it belongs to the non-voice state. And only the sound signal which belongs to an audio | voice state is output. The configuration of the voice state and the non-voice state is performed adaptively.

In the method of Non-Patent Document 1, since it is determined for each input frame whether a speech period or a non-speech period for each frame, an extremely short speech period or non-speech period may be estimated. is there. Therefore, error correction processing may be performed on the voice / non-voice determination result as necessary. The error correction process is, for example, the following process. If a frame regarded as voice by voice / non-voice judgment continues for S _duraiton frames or more, it is determined as a voice section. Also, if a frame regarded as non-speech continues for N _duraiton frames or more, it is determined as a non-speech interval. Further, when a short non-speech segment existing in the speech segment is detected, if the duration of the non-speech segment is equal to or shorter than the Pause frame, the corresponding segment is regarded as a speech segment.

  On the other hand, a technology for detecting a voice section for each of a plurality of target persons from sound signals obtained from a conversation spoken by a plurality of persons (hereinafter referred to as “Diarization”), and a voice section of each of a plurality of target persons There is a technology that recognizes who spoke what by voice recognition.

  For example, as described in Non-Patent Document 2, there is a method of dividing an input sound signal into sections spoken by each speaker, as a diarization when there is one microphone (single channel). This is done by first dividing the input sound signal into blocks of appropriate length (for example, 1 second), and using the speech feature value (for example, MFCC) for each frame of each block, the model of each block (mixed normal distribution GMM). ) Model parameters. Next, for each block, whether or not adjacent blocks are to be combined is determined using an information criterion such as ΔBIC. When ΔBIC is used for the information criterion, it is assumed that ΔBIC is combined when it is negative, and is not combined when it is positive. This procedure is repeated many times for all blocks. Finally, each block obtained when ΔBIC becomes positive in all the block combinations corresponds to the voice section of each speaker.

  As for Diarization when there are a plurality of microphones (multiple channels), there is a method using arrival direction information of an audio signal (see Non-Patent Document 3). This is because a phase difference between channels is calculated in each frame, and a frame having the same phase difference is considered to be a frame in which sound comes from the same direction. Then, a frame having the same phase difference is determined as a voice section of a single speaker.

In addition, when detecting a speech section of a conversation spoken by a plurality of people, as shown in FIG. 3, a sound source separation unit 32 is provided that separates the speech of each speaker of the conversation into separated sound signals by sound source separation technology. There is also a method for detecting the speech section of the separated sound signal of each speaker. Then, the voice recognition unit 23 performs voice recognition on the voice section of each separated sound signal. With such a configuration, even when a plurality of people speak at the same time, the separated sound signals are recognized, so that a relatively high accuracy speech recognition result y ′ ⁿ can be obtained. Here, y ′ ⁿ represents the speech recognition result of speaker n. Normally, if a voice signal in which a plurality of human voices are mixed is recognized as it is without separation, the recognition accuracy is extremely low. As a sound source separation method, for example, a method by beam forming described in Patent Document 1 can be used.

JP 2008-203474 A

Masayoshi Fujimoto, Kentaro Ishizuka, Hiroko Kato, "Detection of noisy speech segments with both state transition processes of speech and noise", Information Processing Society of Japan, 2006, 2006, 136 (SLP-64), pp.13 -18 C. Wooters and M. Huijbregts, “The ICSI RT07s Speaker Diarization System”, proceedings of CLEAR2007 and RT2007, 2007, pp.509-519, S. Araki, M. Fujimoto, K. Ishizuka, H. Sawada, and S. Makino, "Speaker indexing and speech enhancement in real meetings / conversations", ICASSP2008, 2008, pp.93-96

  In any of the methods described in Non-Patent Documents 1 to 3, it is difficult to accurately detect the speech section of the target person from noise signals including noise and sounds other than the target person. For this reason, the determination value of the speech section / non-speech section in each frame of such a sound signal tends to include many errors. Therefore, there is a problem that an erroneous speech section is detected even if the error correction process described above is performed. In particular, since the beginning and end of speech may be determined as a non-speech segment even though it is a speech segment, a fatal recognition error will occur when speech with speech breaks before and after the speech segment is recognized. There was a problem of being connected to.

  Moreover, even if the voice of the target person is emphasized by sound source separation, it is difficult to completely erase the voice other than the target person. Therefore, there is a problem that it is often difficult to correctly detect a voice section from a sound signal obtained from a conversation spoken by a plurality of people. Furthermore, there is a problem that a large number of insertion errors occur when a portion including the unerased part is erroneously determined as a speech section and speech recognition is performed.

  The present invention provides a speech section detection method and apparatus for accurately detecting a speech section of a target person from noise signals including noise and voices other than the target person, and further detects the speech sections. It is an object of the present invention to provide a speech recognition method and speech recognition apparatus with a low word error rate using the method and apparatus. Moreover, it aims at providing those programs and a recording medium.

  In order to solve the above problems, the speech section detection technology according to the present invention extracts a sound signal for each frame of a predetermined length, analyzes the sound signal of the frame, and targets the sound signal of the frame. It is determined whether or not the voice of the speaker is included, the determination result is determined as a voice / non-voice determination value, the sequence of recognition units included in the sound signal, and the utterance time information of each recognition unit, Receives speech / non-speech judgment value for each frame obtained in speech / non-speech judgment step, recognition unit sequence and speech time information of each recognition unit obtained in speech recognition step, and corresponds to speech time of recognition unit It is determined whether or not an utterance has been made by the target speaker for each recognition unit based on the sum of the voice / non-voice determination values of the frames to be recognized.

  In addition, when detecting a speech section from a sound signal including a plurality of target speakers, a sound signal including a plurality of target speakers is extracted for each frame of a predetermined length, and the frame The sound signal of the frame is analyzed, it is determined whether the sound signal of the target speaker is included in the sound signal of the frame, the determination result is obtained as a sound / non-speech determination result, and a plurality of target stories A plurality of separated sound signals in which each speaker's voice is emphasized using a sound signal including a person's voice, a sequence of recognition units included in the separated sound signal, speech time information of each recognition unit, and And a speech / non-speech determination value for each frame of each speaker obtained in the speech / non-speech judgment step, a sequence of recognition units of each speaker and speech time information of each recognition unit obtained in the speech recognition step, Receive each speaker's recognition unit Based on the magnitude of the aggregate value of the audio / non-speech decision value of the frame corresponding to the speech time, determines whether or not spoken by the speaker for each recognition unit.

  Since the present invention determines whether or not it is a speech section based on the total value of the speech / non-speech determination value in the utterance time for each recognition unit of the speech recognition result, there are some errors in the speech / non-speech determination value. Even if it exists, there exists an effect that the influence of these errors is reduced and a speech section can be detected more accurately.

The block diagram of the audio | voice area detection apparatus 10. FIG. The block diagram of the speech recognition apparatus 20. FIG. The block diagram of the speech recognition apparatus 30. FIG. The figure which shows the structural example of the audio | voice area detection apparatus 100 and the speech recognition apparatus 100 '. The figure which shows the processing flow of the audio | voice area detection apparatus 100 and the speech recognition apparatus 100 '. The figure which shows the structural example of the audio | voice / non-audio | voice determination part 130. FIG. The figure which shows the structural example of the speech recognition part 140. FIG. The figure for demonstrating a recognition graph. The figure which shows the structural example of the recognition unit utterance determination part 150. FIG. The figure which shows the structural example of the audio | voice area detection apparatus 200 and the audio | voice recognition apparatus 200 '. The figure which shows the processing flow of the audio | voice area detection apparatus 200 and the speech recognition apparatus 200 '. The figure which shows the structural example of the audio | voice area detection apparatus 300. FIG. The figure which shows the structural example of the audio | voice area detection apparatus 400. The figure which shows the structural example of the audio | voice area detection apparatus 500. FIG. The figure for demonstrating the experimental condition of the speech recognition apparatus 200 '.

  Hereinafter, embodiments of the present invention will be described in detail.

<Audio section detection device 100>
The speech segment detection apparatus 100 according to the first embodiment will be described with reference to FIGS. 4 and 5.

  The speech section detection apparatus 100 includes a speech input unit 110, a speech / non-speech determination unit 130, a speech recognition unit 140, a recognition unit utterance determination unit 150, and a result output unit 160. Hereinafter, an outline of processing of the speech section detection apparatus 100 will be described.

  The voice input unit 110 converts the analog sound signal x (k) recorded by a microphone or the like into a digital sound signal x (u) and sends it to the voice / non-voice judgment unit 130 and the voice recognition unit 140. However, k represents continuous time and u represents discrete time.

  The voice / non-voice judgment unit 130 extracts, for each frame, the amplitude value series x (t) corresponding to the time length of the frame from the digital sound signal x (u) received from the voice input unit 110. However, t represents a frame number and the time corresponding to the frame. Furthermore, the voice / non-voice determination unit 130 outputs, for example, the voice / non-voice determination value d (t) indicating whether or not the voice of the person who is talking for each frame by the method of Non-Patent Document 1.

In the speech recognition unit 140, a sequence v = {v ₁ , v ₂ ,..., V of a recognition unit v _r (word, character, phoneme, etc.) that best matches the digital sound signal x (u) received from the speech input unit 110. _R } is requested. Here, r = 1, 2,..., R. Speech time information obtained at this time each recognition unit v _r was (e.g., start time given frame number as a unit b (v _r) and the end time e (v _r)) is also determined at the same time. Note that the sequence of utterance time information is be = {(b (v ₁ ), e (v ₁ )), (b (v ₂ ), e (v ₂ )), ..., (b (v _R ), e ( v _R ))}.

In the recognition unit utterance determination unit 150, a speech / non-speech determination value corresponding to each recognition unit v _r received from the speech recognition unit 140 and its utterance time information be _r = (b (v _r ), e (v _r )). Based on the magnitude of the total value s (v _r ) (average value, etc.) of d (t), it is determined whether or not an utterance is made by a target person for each recognition unit v _r . The total value s (v _r ) of the voice / non-voice judgment value is

として計算する。この集計値がある閾値TX₁より大きい場合、認識単位v_rが発話されたと判定する。w(・)は任意の窓関数である。例えば、w(・)=1のような関数にすれば、式(1)は以下のように表され、

Calculate as If greater than the threshold TX ₁ with this aggregate value, it determines that the recognition unit v _r is spoken. w (•) is an arbitrary window function. For example, if a function such as w (·) = 1 is used, formula (1) is expressed as follows:

s(v_r)は、時刻b(v_r),e(v_r)間のd(t)の平均値になる。

s (v _r ) is an average value of d (t) between times b (v _r ) and e (v _r ).

It determined that when the recognition unit v _r of the speech decision value s (v _r) is a threshold TX ₁ or with, between the v _r start time b (v _r) and the end time e (v _r) is a speech segment However, if it is less than TX ₁ , it is determined that it is a non-speech interval between times b (v _r ) and e (v _r ). The recognition unit utterance determination unit 150 outputs a determination result g (v _r ). For example, when it is determined that v _r is a voice interval, g (v _r ) = 1 is set. When v _r is determined to be a non-voice interval, g (v _r ) = 0 is set and output. To do.

When the determination result g (v _r ) represents a speech section, the result output unit 160 utters time information (b (v _r ), e (v _r )) corresponding to the recognition unit v _r or the correspondence. Output sound signal x (b (v _r ), e (v _r )).

  Hereinafter, the processing content of each part is demonstrated.

<Voice input unit 110>
The audio input unit 110 is an input interface for inputting, for example, a microphone output signal, a signal read from an external storage device, a signal received from a network, or the like. The voice input unit 110 converts the input sound signal x (k) into a digital sound signal x (u) by an A / D converter and outputs the digital sound signal x (u) (s110). For example, the audio input unit 110 samples an analog sound signal x (k) obtained by a microphone or the like at regular time intervals, and generates a series of amplitude values x (u) of sound signals (hereinafter referred to as “digital sound signals”). Convert. For example, when sampling (sampling) at 16 kHz and quantizing at 16 bits, the audio input unit 110 extracts the amplitude value of the sound signal every 1/16000 seconds, and for each amplitude value, an integer of 16 bits, that is, Assign a value between -32768 and +32767.

<Voice / non-voice determination unit 130>
The voice / non-voice determination unit 130 extracts the digital sound signal x (u) for each frame having a predetermined length, analyzes the sound signal x (t) of the frame, and converts it into the sound signal x (t) of the frame. It is determined whether or not the speech of the target speaker i is included, and the determination result is obtained as a speech / non-speech determination value d (t) (s130). Here, t represents a frame number used in the voice / non-voice judgment unit 130.
For example, as shown in FIG. 6, the speech / non-speech determination unit 130 uses a feature extraction unit 131 that extracts an acoustic feature amount for identifying speech / non-speech and a corresponding frame using the extracted feature amount. And an identification unit 133 for identifying whether the sound signal x (t) is voice or non-voice.

(Feature extraction unit 131)
The feature extraction unit 131 extracts, for each frame, a sample series x (t) of amplitude values corresponding to the time length of the frame from the digital sound signal x (u) output from the audio input unit 110. For example, the digital sound signal x (u) having a fixed time length (frame length) is cut out as a frame while moving the start point of the input digital sound signal x (u) with a fixed time width (shift width) in the time axis direction. A digital sound signal in units of frames is represented as x (t). For example, a sound signal of 1024 sample points (the time length is equivalent to 64 ms when the sampling frequency is 16,000 Hz) is cut out while moving the start point by 512 sample points (also the time length is equivalent to 32 ms).

  Next, the feature extraction unit 131 performs fast Fourier transform on the extracted digital sound signal x (t) to calculate a frequency-domain sound signal x (f, t).

Further, the feature extraction unit 131 applies a P-dimensional mel filter bank analysis (for example, P = 12) using the frequency-domain sound signal x (f, t), and calculates the P-dimensional log mel spectrum of the frame t. The acoustic feature vector O _t = {o _{t, 0} , o _{t, 1} ,..., O _{t, P−1} } of the element is calculated and used as the output of the feature extraction unit 131. At this time, the sample point of the frame and the dimension P of the acoustic feature vector may be appropriately changed according to the feature of the input sound signal.

(Identification part 133)
The identification unit 133 first calculates the likelihood b _j (O _t ) between the speech feature vector O _t of the input speech and a separately prepared speech and non-speech probability model, for example, a mixed normal distribution model (GMM: Gaussian Mixture Model). Calculate as follows:

但し、jはGMMの種別を表し、j=0のとき非音声モデルとし、j=1のとき音声モデルとする。kはGMMに含まれる正規分布の番号であり、k=1,2,…,Kとし、w_j,kはK番目の正規分布の混合重みを、μ_j,k,lは平均値を、σ² _j,k,lは分散値を表す。なお、確率モデル（GMM）は事前になんらかの学習データを用いて学習しておいても良く、オンラインで逐次的に推定、更新を行っても良い。図示しないＧＭＭ記憶部に、確率モデル(GMM)を記憶しておき、識別部１３３は、ＧＭＭ記憶部を参照して、上述の尤度b_j(O_t)を計算する。

However, j represents the type of GMM, and when j = 0, it is a non-voice model, and when j = 1, it is a voice model. k is the number of the normal distribution included in the GMM, k = 1,2, ..., K, w _{j, k} is the mixing weight of the Kth normal distribution, μ _{j, k, l} is the average value, σ ² _{j, k, l} represents a variance value. Note that the probability model (GMM) may be learned in advance using some learning data, or may be estimated and updated sequentially online. A probability model (GMM) is stored in a GMM storage unit (not shown), and the identification unit 133 refers to the GMM storage unit and calculates the above-described likelihood b _j (O _t ).

  Next, the identification unit 133 calculates the likelihood ratio L (t) of each model by the following equation.

α_t,j＝（α_t-1,0a_0,j+α_t-1,1a_1,j）b_j(O_t) (5)
但し、α_t,jは前向き確率であり、a_i,jは非音声状態H₀及び音声状態H₁への状態遷移確率であり、iは遷移前の状態番号、jは遷移後の状態番号であり、a_i,0+a_i,1=1を満たす（状態番号0は非音声状態を、状態番号1は音声状態をそれぞれ示す）。なお、t=0のとき初期値として前向き確率α_t,jを以下のように設定する。

α _{t, j} = (α _t-1,0 a _{0, j} + α _t-1,1 a _{1, j} ) b _j (O _t ) (5)
Where α _{t, j} is the forward probability, a _{i, j} is the state transition probability to the non-voice state H ₀ and the voice state H ₁ , i is the state number before the transition, and j is the state number after the transition. And satisfies a _{i, 0} + a _{i, 1} = 1 (state number 0 indicates a non-speech state and state number 1 indicates a speech state). Note that the forward probability α _{t, j} is set as follows as an initial value when t = 0.

α _{t, 0} = 1 (6)
α _{t, 1} = 0 (7)
Next, the identification unit 133 determines whether the frame t belongs to a voice state or a non-voice state using the likelihood ratio L (t). The identification unit 133 may output the value of the likelihood ratio L (t) as the speech / non-speech determination value d (t). In addition, the identification unit 133 determines that the frame t belongs to the audio state when the threshold TH _{2 is} set separately or higher, and outputs the audio / non-audio determination value d (t) = 1, which is less than the threshold TH ₂ If so, it may be determined that frame t belongs to the non-speech state and output as d (t) = 0. The value of the threshold TH ₂ may be determined in advance or may be determined adaptively according to the characteristics of the input sound signal.

<Voice recognition unit 140>
Speech recognition unit 140, the sequence v of recognition units included in the sound signal x (u), speech time information b of each recognition unit v _i (v _i), obtaining an e (v _i) (s140) .

  For example, as illustrated in FIG. 7, the speech recognition unit 140 includes a feature extraction unit 141, a recognition unit 143, an acoustic model storage unit 145, and a recognition graph storage unit 147.

(Feature extraction unit 141)
The feature extraction unit 141 extracts, for each frame, a sample series x (t ′) of amplitude values corresponding to the time length of the frame from the digital sound signal x (u) output from the audio input unit 110. However, t ′ represents a frame number used in the speech recognition unit 140.

  Next, the feature extraction unit 141 performs fast Fourier transform on the extracted digital sound signal x (t ′), and calculates the sound signal x (f, t ′) in the frequency domain.

Further, the feature extraction unit 141 uses the sound signal x (f, t ′) in the frequency domain to generate a P′-dimensional speech feature vector O ′ _{t ′} = {o ′ _{t ′, 0} , o ′ _{t ′, 1} ,..., O ′ _{t ′, P′−1} } are calculated and output as the output of the feature extraction unit 141. At this time, the frame sample points and the speech feature vectors are not necessarily the same as the feature extraction unit 131 in the speech / non-speech determination unit 130, and may be different. For example, speech feature vectors include mel cepstrum (called mel-frequency cepstral coefficients (MFCC)), delta MFCC, LPC cepstrum, logarithmic power obtained by analyzing a sound signal every short time interval (eg, 10 milliseconds). Etc. Next, the feature extraction unit 141 inputs the speech feature vector O ′ _{t ′} obtained for each frame to the recognition unit 143 in order along the time order.

(Recognition unit 143)
The recognition unit 143 refers to an acoustic model storage unit 145 that stores an acoustic model that represents a model of a phonetic feature such as a phoneme, and a recognition graph storage unit 147 that stores a recognition graph that represents constraints such as pronunciation and grammar. A sequence v of recognition units v _r (words, characters, phonemes, etc.) that best fits the sequence of input speech feature vectors O ′ _{t ′} is obtained. The speech time information of each recognition unit v _r obtained at this time, that is, the start time b (v _r ) and the end time e (v _r ) are also obtained at the same time.

For example, the recognition unit 143 stores, as a frame number, how many speech feature vectors O ′ _{t ′} have been read since the start of processing, and each recognition unit v _r obtained as a result of speech recognition is a speech feature. Information indicating which frame number in the sequence of the vector O ′ _{t ′} starts and which frame number ends is obtained. _Let the start frame number and end frame number of each recognition unit v _{r be} the start time b (v _r ) and end time e (v _r ) of each recognition unit.

(Acoustic model storage unit 145)
The acoustic model storage unit 145 stores, as an acoustic model, for example, a standard speech feature vector of a speech fixed unit (for example, phoneme) or its distribution.

The recognizing unit 143 refers to the acoustic model stored in the acoustic model storage unit 145, and determines the similarity between the speech fixed unit and the input speech feature vector O ′ _{t ′} , for example, a standard belonging to the speech fixed unit. It is obtained as a logarithmic value of the probability density of the speech feature vector O ′ _{t ′} calculated by the distribution of various speech feature vectors. As a method using an acoustic model, for example, there is a Hidden Markov Model method (hereinafter referred to as HMM) that models a set of speech feature vector time series in a fixed speech unit based on probability / statistical theory. (See Reference 1).
[Reference 1] Seiichi Nakagawa, “Speech Recognition by Stochastic Model”, The Institute of Electronics, Information and Communication Engineers, 1988

(Recognition graph storage unit 147)
In the recognition graph storage unit 147, for example, a recognition graph configured using a weighted finite state transducer is stored. In addition, a method of describing a word pronunciation dictionary and a language model with a weighted finite state transducer (English translation Weighted Finite-State Transducer: WFST) and composing them to form one weighted finite state transducer is, for example, a reference. 2.
[Reference 2] M.Mohri, F.Pereira, M.Riley, “Weighted finite-state transducers in speech recognition”, Computer Speech and Language, 2002, Vol.16, No.1, pp.69-88

  Hereinafter, the recognition graph will be described. In general, a label indicating one state of the HMM is assigned to each arc (also referred to as an edge, a link, a branch, or the like) of a recognition graph for speech recognition. The state of the HMM is a speech fixed unit corresponding to the approximate position (for example, the first half, the middle, and the second half) of a certain phoneme in each phoneme, and each state is a probability density distribution (for example, multidimensional mixed normal) of a speech feature vector. Distribution).

The recognition unit 143 calculates the probability density using the speech feature vector O ′ _{t ′} added to the hypothetical cumulative weight and the probability density distribution of the state corresponding to the arc label, and the logarithmic value of the obtained probability density is used as the speech feature. It is calculated as the similarity between the vector O ′ _{t ′} and the arc label. The probability density distribution of the state corresponding to each arc label is stored in the acoustic model storage unit 145.

  The recognition graph for speech recognition is a directed graph representing rules such as phoneme and word arrangement. The phoneme and word chain patterns (logarithm values) are given to the arc as weights, and the phoneme and word acoustic patterns. And a label representing the distribution function (for example, a distribution function representing a phoneme / a / speech feature vector) is assigned to each corresponding arc. In addition, a start point node and an end point node are also set according to the rules of phoneme and word arrangement.

  For example, the recognition graph is configured as shown in FIG. The start node is indicated by a black circle, the end node is indicated by a double circle, and the other nodes are indicated by circles. A label is given to the arc connecting the nodes, and the phoneme name and the position in the phoneme are indicated. For example, a2 indicates the probability density distribution of the second state of the HMM of phoneme a.

The recognizing unit 143 refers to the recognition graph, and for each input speech feature vector O ′ _{t ′} series, the cumulative value of the similarity between the speech feature vectors O ′ _{t ′} and the labels associated with the arcs. Find the path in the recognition graph that has the highest. The similarity between the input speech feature vector O ′ _{t ′} and the label is calculated as a value when the speech feature vector O ′ _{t ′} is input to the distribution function corresponding to the label.

In addition, in FIG. 8, when two labels are written with “:” like “a1: red”, the similarity between the label and the speech feature vector O ′ _{t ′} is based on a1 before “:”. The “red” after the “:” is a label (recognition unit) that is a recognition result when a route passing through this arc is selected. In the example of FIG. 8, there are a route that passes through and a route that passes below. When a route that passes above is selected, “red” is the recognition result, and when a route that passes below is selected, “ “Blue” is the recognition result.

(Processing content of the recognition unit 143)
First, as an initial setting, the recognition unit 143 generates, as an initial hypothesis of a route to be searched, one hypothesis that has reached the start node on a zero-length route, gives 0 to the hypothesis score, to add. Next, each time one speech feature vector O ′ _{t ′} is read, the recognizing unit 143 applies each arc to each arc that can transition from the last node reached by each hypothesis included in the current hypothesis list. A new hypothesis is generated by extending the path by adding a node to the hypothesis and the transition destination node, and the similarity between the arc weight and the voice feature vector O ′ _{t ′} read immediately before and the label of the arc Is added to the new hypothesis list as a score of the new hypothesis. Next, the recognition unit 143 empties the current hypothesis list, moves all hypotheses included in the new hypothesis list to the current hypothesis list, reads the next speech feature vector O ′ _{t ′} , and performs similar processing. repeat. In order to reduce the calculation amount, a hypothesis having a relatively small score in the current hypothesis list may be deleted from the current hypothesis list.

Then, when the recognition unit 143 has read the sequence of the speech feature vector O ′ _{t ′} to the end, the recognition unit sequence v = (v ₁₎ on the path of the hypothesis having the highest score in the current hypothesis list. , v ₂ , ..., v _R ). Here, R represents the number of recognition units included in the series of recognition units.

The recognizing unit 143 obtains utterance time information b (v _r ) and e (v _r ) of each recognition unit v _r by the following method, and a sequence of speech time information corresponding to the recognition unit sequence v = ((b (v ₁ ), e (v ₁ )), (b (v ₂ ), e (v ₂ )),... (b (v _R ), e (v _R ))) are output. The recognizing unit 143 checks the number of transitions from the start node on the path to which the recognition unit v _r included in the recognition unit sequence v on the hypothetical path with the highest score is assigned, and The start time of the recognition unit v _r is acquired as the frame number b (v _r ). If the start time of each recognition unit v _r on the path is known, the frame number immediately before each start time becomes the frame number representing the end time of the recognition unit v _r−1 immediately before the start time. Therefore, the recognizing unit 143 acquires the start time as the frame number immediately before the frame number b (v _r ), and the end time of the immediately previous recognition unit v _r−1 as the frame number b (v _r−1 ). Note that the end time e (v _R ) of the last recognition unit v _R of the sequence v is acquired as the last processed frame number.

<Recognition unit utterance determination unit 150>
The recognition unit utterance determination unit 150 includes a speech / non-speech determination value d (t) for each frame obtained by the speech / non-speech determination unit 130, a recognition unit sequence v obtained by the speech recognition unit 140, and each recognition unit. A sequence be of utterance time information is received, and the sum of voice / non-voice determination values d (t) of frames corresponding to the utterance time information b (v _r ) and e (v _r ) of each recognition unit v _r ( Average value or the like) Based on the magnitude of s (v _r ), it is determined for each recognition unit whether or not the target speaker has spoken (s150).

FIG. 9 shows a processing procedure in the recognition unit utterance determination unit 150 when R recognition units v _r are obtained from the speech recognition unit 140. First, initial setting is performed by setting r = 1, S = 0, t = b (v _r ) (s150a, s150b). Next, w (tb (v _r )) d (t) is obtained for each frame included from the start time b (v _r ) to the end time e (v _r ), and S is obtained by adding all (s150c, s150d, s150e). S is represented by the following equation.

さらに、得られた値Sを、対応するフレーム数e(v_r)-b(v_r)+1で除算することにより、集計値s(v_r)を求める（ｓ１５０ｆ）。つまり、r番目の認識単位v_rについて音声／非音声判定値の集計値s(v_r)を式(1)に従って計算する。その集計値s(v_r)が閾値TX₁以上であれば（ｓ１５０ｇ）、v_rが音声区間であるとの判定結果g(v_r)（例えば、g(v_r)=1）を出力する（ｓ１５０ｈ）。その集計値s(v_r)が閾値TX₁未満であれば（ｓ１５０ｇ）、v_rが非音声区間であるとの判定結果g(v_r)（例えば、g(v_r)=0）を出力する（ｓ１５０ｉ）。ｓ１５０ｂ〜ｓ１５０ｉを全ての認識単位に対して繰り返し行う（ｓ１５０ｊ、ｓ１５０ｋ）。

Further, the obtained value S is divided by the corresponding number of frames e (v _r ) −b (v _r ) +1 to obtain a total value s (v _r ) (s150f). That is, the total value s (v _r ) of the voice / non-voice judgment value is calculated according to the equation (1) for the r-th recognition unit v _r . If the total value s (v _r ) is greater than or equal to the threshold value TX ₁ (s150g), a determination result g (v _r ) (for example, g (v _r ) = 1) is output that v _r is a voice interval. (S150h). If the total value s (v _r ) is less than the threshold value TX ₁ (s 150 g), a determination result g (v _r ) (for example, g (v _r ) = 0) is output that v _r is a non-speech segment. (S150i). s150b to s150i are repeated for all recognition units (s150j, s150k).

The total value of the voice / non-voice judgment value is calculated by the equation (1) as described above. As described above, by setting the window function w (•) = 1, the speech determination value s (v _r ) of the recognition unit v _r is the voice / non-voice determination value between the times b (v _r ) and e (v _r ). This is the average value of d (t) (see equation (1 ′)). On the other hand, the window function w (•) may be a function that takes a large value toward the center of the speech time of the recognition unit v _r (that is, (b (v _r ) + e (v _r )) / 2). . For example, a Gaussian distribution with an average value of (b (v _r ) + e (v _r )) / 2 may be used as the window function w (•).

Also, by setting b (v _r ) smaller than the actual start time and e (v _r ) larger than the actual end time, the voice / non-voice judgment value d (t around the recognition unit v _r is determined. ) May be considered. Conversely, by making b (v _r ) larger than the actual start time and e (v _r ) smaller than the actual end time, the speech / non-speech determination value d at the front or rear of the recognition unit (t) may be ignored.

  However, when the time length of the frame used in the voice / non-voice determination unit 130 and the time length of the frame used in the voice recognition unit 140 are different, the different time is indicated even if the frame number is the same. Therefore, correction is necessary. Hereinafter, the correction method will be described. For example, the frame number t ′ of the speech recognition unit 140 is converted into the frame number t used by the speech / non-speech determination unit 130, and t converted in Equation (1) is used. The conversion formula is shown below.

ここで、Dは音声／非音声判定部１３０におけるフレーム時間長、Hは音声認識部１４０におけるフレーム時間長を表す。但し、式(8)で求めたtが整数にならない場合は、tの近傍の整数に対応するフレーム番号の音声／非音声判定値を用いて補間しても良い。例えば、tの近傍の２点のフレーム番号の音声／非音声判定値を用いて、以下のように線形補間する。

Here, D represents the frame time length in the speech / non-speech determination unit 130, and H represents the frame time length in the speech recognition unit 140. However, when t obtained by Expression (8) does not become an integer, interpolation may be performed using the speech / non-speech determination value of the frame number corresponding to the integer in the vicinity of t. For example, linear interpolation is performed as follows using speech / non-speech determination values of two frame numbers in the vicinity of t.

d ~ (t) = d (τ (t)) + (t-τ (t)) × (d (τ (t) +1) -d (τ (t))) (9)
Here, τ (t) is an integer obtained by rounding down the decimal part of t. The values d to (t) obtained in this way may be used instead of d (t) in equation (1).

During the recognition unit utterance determination unit 150 recognition unit v _r of the speech decision value s (v _r) in the case of the threshold TX ₁ or more there is, v start time of _r b (v _r) and the end time e (v _r) Determines that it is a voice interval and outputs g (v _r ) = 1.If it is less than TX ₁ , it determines that it is a non-voice interval between times b (v _r ) and e (v _r ). Output g (v _r ) = 0.

<Result output unit 160>
The result output unit 160 outputs the detection result of the speech section according to the determination result g (v _r ) (s160). As a detection result, a sound signal x (b (v _r ), e (v _r )) of v _r corresponding to g (v _r ) = 1 may be output, or speech time information (start time) of v _r b (v _r ) and end time e (v _r )) may be output. Note that x (b (v _r ), e (v _r )) represents a digital sound signal that starts from the start time b (v _r ) and ends at the end time e (v _r ).

The result output unit 160 is an output interface for outputting a sound signal to a speaker, for example. For example, when outputting the sound signal detected by the speech section detection device 100, the result output unit 160 outputs the digital sound signal x (b (v _r )) of the recognition result v _r corresponding to g (v _r ) = 1. , e (v _r )) is converted into an analog sound signal by a D / A converter and output to a speaker.

Digital sound signal x (u), frame sound signal x (t), voice / non-voice judgment value d (t), voice recognition result v _r , speech time information (b (v _r ), e (v _r )), The determination result g (v _r ) may be stored in association with a storage unit (not shown) in each processing step.

<Effect>
With such a configuration, since it is determined based on the total value of the speech / non-speech determination value at the utterance time for each recognition unit of the speech recognition result, it is determined whether the speech / non-speech determination value. Even if there are some errors, it is possible to reduce the influence of these errors and to detect the speech section more accurately.

<Modification 1>
A case where the speech segment detection method of the present invention is used for speech recognition will be described. The speech recognition apparatus 100 ′ will be described with reference to FIGS. The speech recognition apparatus 100 ′ has the same configuration as that of the first embodiment. However, the processing contents of the result output unit 160 ′ are different.

The result output unit 160 ′ outputs the recognition unit v _r determined to be uttered by the target speaker in the recognition unit utterance unit 150 as a speech recognition result of the target speaker (s160 ′). That is, instead of outputting the sound signal x (u) and the utterance time information (b (v _r ), e (v _r )), it functions as a voice recognition device by outputting the recognition result v _r .

The result output unit 160 is an output interface for outputting text data to a display, for example. For example, when outputting the recognition result recognized by the speech recognition apparatus 100 ′, the result output unit 160 outputs the recognition result v _r corresponding to g (v _r ) = 1.

With such a configuration, it is possible to select and output only the recognition unit v _r determined as a speech section by the recognition unit utterance determination unit 150 from the output of the speech recognition unit 140 as a recognition result. Since the speech recognition unit 140 performs speech recognition without detecting a speech section, the speech recognition unit 140 is not affected by a detection error that determines that the beginning or end of speech is a non-speech segment. Therefore, there is an effect that it is possible to avoid a speech recognition error caused by a conventional speech segment detection error.

<Other variations>
When a digital sound signal is input to the sound section detection device 100 (for example, when A / D conversion is performed in a sound collection device such as a microphone or a digital sound signal is received via an external storage device or a network) The input unit 110 may not perform A / D conversion.

  The result output unit 160 may be an output interface for outputting a signal to be written to an external storage device, a signal to be transmitted to a network, or the like.

<Audio section detection device 200>
A speech segment detection apparatus 200 according to the second embodiment will be described with reference to FIGS. 10 and 11. The speech section detection apparatus 200 includes a speech input unit 210, a sound source separation unit 220, a speech / non-speech determination unit 230, a speech recognition unit 240, a recognition unit utterance determination unit 250, and a result output unit 260. The voice section detection apparatus 200 detects a voice section of each person when a plurality of people speak using a sound source separation technique together. In this embodiment, a sound source separation technique using arrival direction information of a voice signal is used. Hereinafter, the processing content of each part is demonstrated.

<Voice input unit 210>
The sound input unit 210 converts the input sound signals x ₁ (k), x ₂ (k),..., X _M (k) into digital sound signals x ₁ (u) by an A / D converter, respectively. , x ₂ (u),..., x _M (u) and output (s210). Here, M represents the number of microphones used when recording sound signals.

<Sound source separation unit 220>
The sound source separation unit 220 uses the digital sound signals x ₁ (u), x ₂ (u),..., X _M (u) including the voices of a plurality of target speakers, for example, beam forming in Patent Document 1. Accordingly, a plurality of separating sound signal emphasizing a sound of each speaker _{z ~ 1, z ~ 2,} ..., and generates a z ~ _N (s220), and outputs to the voice recognition unit 240. N represents the number of target speakers. For example, the sound source separation unit 220 includes an arrival direction estimation unit 221 and a separated sound signal generation unit 223.

(Arrival direction estimation unit 221)
The arrival direction estimation unit 221 extracts a plurality of digital sound signals x ₁ (u), x ₂ (u),..., X _M (u) for each frame of a predetermined length, and the sound signal x ₁ ( t), x ₂ (t), ..., x _M (t) is subjected to fast Fourier transform, and the frequency domain sound signal x ₁ (f, t), x ₂ (f, t), ..., x _M Calculate (f, t).

The arrival direction estimation unit 221 applies the GCC-PHAT method to all microphone pairs mm ′ (see Reference 3), and estimates the arrival time difference (TDOA) of speech by the following equation.
[Reference 3] CHKnapp and GCCarter, “The generalized correlation method for estimation of time delay”, IEEE Trans. Acoust. Speech and Signal Processing, 1976, vol.24, no. 4, pp. 320-327

但し、m=1,2,…,M、m'=1,2,…,M、m≠m'とする。全てのマイクペアmm'におけるTDOAの推定値τ_mm'(t)を並べたベクトルをτ(t)とする。

Here, m = 1, 2,..., M, m ′ = 1, 2,..., M, m ≠ m ′. Let τ (t) be a vector in which the estimated values τ _{mm ′} (t) of TDOA in all microphone pairs mm ′ are arranged.

Next, the arrival direction estimation unit 221 estimates the arrival direction q (t) of the speech signal from the vector τ (t) in which the estimated arrival time difference values are arranged and the matrix D representing the microphone coordinates (s211) (reference) Reference 4).
[Reference 4] S. Araki, H. Sawada, R. Mukai and S. Makino, "DOA estimation for multiple sparse sources with normalized observation vector clustering", ICASSP2006, 2006, Vol. 5, pp.33-36

ここで、cは音速、⁻は一般化逆行列、Dは各マイクの座標ベクトル（この実施例では2次元とする）を並べた行列である。式(11)を以下の式(11')によって正規化する。

Here, c is the speed of sound, ⁻ is a generalized inverse matrix, and D is a matrix in which coordinate vectors of each microphone (in this embodiment, two-dimensional) are arranged. Expression (11) is normalized by the following expression (11 ′).

なお、到来方向q(t)は、時刻tに到来している音声の方位角をθ(t)とすると、q(t)=[cosθ(t) sinθ(t)]である。

The arrival direction q (t) is q (t) = [cos θ (t) sin θ (t)], where θ (t) is the azimuth angle of the voice arriving at time t.

Next, the arrival direction estimation unit 221 clusters the direction information θ (t) obtained from q (t) = [cos θ (t) sin θ (t)], and at the time t when the voice reaches from a certain range direction. Get the set T _n . For example, n = 1, 2,..., N and Δ are values indicating a certain range, and a set of time t satisfying θ _n −Δ ≦ θ (t) <θ _n + Δ is defined as T _n . Note that θ _n represents the direction in which the speaker n is present. The arrival direction estimation unit 221 outputs T _n to the separated sound signal generation unit 223.

(Separated sound signal generator 223)
The separated sound signal generation unit 223 emphasizes the sound from the direction θ _n using T _n . Note that the emphasized voice is considered as the voice of speaker n. A signal x _m (f, t) obtained by converting the sound signal recorded by the microphone m into the frequency domain

と表す。ここで、nは、時刻tで発話している話者の番号であり、z_n(f,t)は話者nの音声信号の時間周波数表現であり、h_mn(f)は話者nからマイクmまでの周波数応答を表す。また全てのマイクの入力を並べたベクトルX(f,t)=[x₁(f,t), x₂(f,t),…, x_M(f,t)]^Tを
X(f,t)=H(f)^TZ(f,t)
Z(f,t)=[z₁(f,t), z₂(f,t),…, z_N(f,t)]^T
とする。なお、Tは転置を表し、H(f)はh_mn(f)を(n,m)要素として持つ行列である。この行列H(f)をここでは混合行列と呼ぶ。分離音信号生成部２２３は、混合行列H(f)を推定し、その逆行列H~^-1(f)を用いて分離音信号z~_n(f,t)を求める（ｓ２２３）。

It expresses. Where n is the number of the speaker speaking at time t, z _n (f, t) is the time frequency representation of the speech signal of speaker n, and h _mn (f) is the speaker n Represents the frequency response from to m. Also, the vector X (f, t) = [x ₁ (f, t), x ₂ (f, t), ..., x _M (f, t)] ^T , where all microphone inputs are arranged
X (f, t) = H (f) ^T Z (f, t)
Z (f, t) = [z ₁ (f, t), z ₂ (f, t),…, z _N (f, t)] ^T
And T represents transposition, and H (f) is a matrix having h _mn (f) as (n, m) elements. This matrix H (f) is called a mixing matrix here. Separated sound signal generation unit 223 estimates the mixing matrix H (f), determining the separated sound signals z ~ _n (f, t) using the inverse matrix ^{H ~ -1 (f) (s223} ).

ここで混合行列の逆行列の推定値H~^-1(f)は以下のように推定する。

Here, the estimated value H ~ ^-1 (f) of the inverse matrix of the mixing matrix is estimated as follows.

但し、E[]は、方向θ_nから音声が到来する時刻の集合Ｔ_ｎに関して、平均値をとる操作を意味する。分離音信号生成部２２３は、求めた複数の分離音信号z~_n(f,t)を音声認識部２４０へ出力する。

However, E [] means an operation for taking an average value with respect to a set T _n of times when voice comes from the direction θ _n . The separated sound signal generation unit 223 outputs the obtained plurality of separated sound signals z to _n (f, t) to the speech recognition unit 240.

<Voice / non-voice determination unit 230>
The speech / non-speech determination unit 230 generates digital sound signals x ₁ (u), x ₂ (u),..., X _M (u) including speech of a plurality of target speakers for each frame of a predetermined length. , And the sound signal x ₁ (t), x ₂ (t), ..., x _M (t) of the frame is analyzed, and the sound signal x ₁ (t), x ₂ (t), ..., of the frame is analyzed. It is determined whether or not the speech of each target speaker n is included in x _M (t), the determination result is obtained as a speech / non-speech determination result d _n (t), and the recognition unit utterance determination unit 250 determines Output (s230). The voice / non-voice determination unit 230 includes, for example, an arrival direction estimation unit 231 and a determination unit 233.

(Arrival direction estimation unit 231)
The arrival direction estimation unit 231 has the same configuration as the arrival direction estimation unit 231 in the sound source separation unit 220. However, the frame time length and the like may be set as appropriate. In this embodiment, it is assumed that the frame time lengths used by the voice / non-voice judgment unit 230, the sound source separation unit 220, and the voice recognition unit 240 are the same.

The arrival direction estimation unit 231 extracts a plurality of digital sound signals x ₁ (u), x ₂ (u),..., X _M (u) for each frame of a predetermined length, and the sound signal x ₁ ( t), x ₂ (t), ..., x _M (t) is subjected to fast Fourier transform, and the frequency domain sound signal x ₁ (f, t), x ₂ (f, t), ..., x _M Calculate (f, t).

Then, the arrival direction estimation unit 231 estimates the arrival direction q (t) = [cosθ (t) sinθ (t)] of the speech signal using Expressions (10), (11), and (12), and The direction information θ (t) obtained from q (t) is clustered to obtain a set T _{n of} time t arriving from a certain direction of voice (s231).

(Determination unit 233)
The determination unit 233 uses the set T _n obtained by the arrival direction estimation unit 231 to calculate the speech / non-speech determination value d _n (t).

として求め（ｓ２３３）、認識単位発話判定部１５０に出力する。つまり、時刻tが方向θ_nから音声が到来する時刻の集合Ｔ_ｎに含まれる場合には、音声であると判定し、含まれない場合には、非音声であると判定する。

(S233) and output to the recognition unit utterance determination unit 150. That is, when the time t is included in the set T _n of times when sound arrives from the direction θ _n , it is determined that the sound is sound, and when it is not included, it is determined that the sound is non-sound.

<Voice recognition unit 240>
The voice recognition unit 240 recognizes each of the plurality of separated sound signals z to _n (f, t) sent from the sound source separation unit 220 (s240), and responds to each separated sound signal z to _n (f, t). The speech recognition result is obtained as a sequence v ⁿ = {v ⁿ ₁ , v ⁿ ₂ ,..., V ⁿ _R } of the recognition unit v ⁿ _r and is output to the recognition unit utterance determination unit 150. In addition, the speech recognition unit 240 obtains utterance time information b (v ⁿ _r ) and e (v ⁿ _r ) of each recognition unit v ⁿ _r and sends the sequence be ⁿ of the utterance time information to the recognition unit utterance determination unit 250. Output. The voice recognition method is the same as that in the first embodiment. However, instead of the sound signal x (u), the input uses the separated sound signals z to _n (f, t) for each frame, and the speech recognition is performed on the separated sound signals z to _n (f, t). The difference is that processing is performed to obtain a sequence v ⁿ of recognition units v ⁿ _r for each speaker and a sequence be ⁿ of corresponding speech time information.

<Recognition unit utterance determination unit 250>
The recognition unit utterance determination unit 250 recognizes each speaker's speech / non-speech determination value d _n (t) obtained by the speech / non-speech determination unit 230 and each speaker's recognition obtained by the speech recognition unit 240. receiving a series bE ⁿ sequence v ⁿ and speech time information unit, using the processing procedure of FIG. 9, the speech time of the recognition units v ⁿ _r of each speaker _{^{n b (v n r),}} e (v ⁿ _r) to seek an aggregate value s of the audio / non-speech decision value of the frame corresponding (v ⁿ _r), based on the magnitude of the aggregate value s (v ⁿ _r), it is spoken by the speaker for each recognition unit It is determined whether or not (s250). When the recognition unit utterance determination unit 250 of the threshold TX ₁ or there is a recognition unit of each speaker v ⁿ _r utterance determination value s (v ⁿ _r), and end v ⁿ _r start time b (v ⁿ _r) During time e (v ⁿ _r ), it is determined that it is a voice interval and g (v ⁿ _r ) = 1 is output, and when it is less than TX ₁ , time b (v ⁿ _r ), e (v ⁿ _r ) Is determined as a non-speech interval, and g (v ⁿ _r ) = 0 is output. The processing contents are the same as in the first embodiment. However, the difference is that the input is a sequence v ⁿ of recognition units for each speaker and a sequence be ⁿ of speech time information, and the determination result is a value g (v ⁿ _r ) for each speaker. Also, the frame time length used in the voice / non-voice judgment unit 230, the sound source separation unit 220, and the voice recognition unit 240 may be different as in the first embodiment. Number conversion processing and interpolation processing may be performed.

<Result output unit 260>
The result output unit 260 outputs the detection result of the speech section according to the determination result g (v ⁿ _r ) (s260). As a detection result, a digital sound signal x (b (v ⁿ _r ), e (v ⁿ _r )) of v ⁿ _r corresponding to g (v ⁿ _r ) = 1 may be output, or v ⁿ _r The utterance time information (start time b (v ⁿ _r ) and end time e (v ⁿ _r )) may be output.

<Effect>
With such a configuration, in addition to the effect of the first embodiment, there is an effect that a voice section can be detected from a digital sound signal including voices of a plurality of target speakers.

<Modification 1>
A case where the speech segment detection method of the present invention is used for speech recognition will be described. 10 and 11, the speech recognition apparatus 200 ′ has the same configuration as that in the second embodiment. However, the processing content of the result output unit 260 ′ is different.

The result output unit 260 ′ outputs the recognition unit v ⁿ _r determined to be uttered by the target speaker in the recognition unit utterance unit 250 as the speech recognition result of the target speaker (s260 ′). In other words, it does not output the sound signal x (u) or utterance time information (b (v ⁿ _r ), e (v ⁿ _r )) but functions as a voice recognition device by outputting the recognition result v ⁿ _r To do.

  With such a configuration, the same effects as those of the first modification of the first embodiment can be obtained, and voice recognition can be performed from digital sound signals including a plurality of target speaker voices.

  Similarly, in the third, fourth, and fifth embodiments described below, the voice section detection devices 300, 400, and 500 can function as a voice recognition device.

  Further, as a sound source separation technique in the sound source separation unit 220, another conventional technique may be used.

<Audio section detection device 300>
A speech segment detection apparatus 300 according to the third embodiment will be described with reference to FIG. The speech segment detection apparatus 300 includes a speech input unit 210, a sound source separation unit 320, a speech / non-speech determination unit 330, a speech recognition unit 240, a recognition unit utterance determination unit 250, and a result output unit 260. The processing contents of the sound source separation unit 320 and the voice / non-voice judgment unit 330 are different from those in the second embodiment. Only the different parts will be described.

<Sound source separation unit 320>
The sound source separation unit 320 emphasizes the speech of each speaker using the digital sound signals x ₁ (u), x ₂ (u),..., X _M (u) including the speech of a plurality of target speakers. a plurality of separated sound signals z ~ _1, z ~ ₂ was, ..., and generates a z ~ _N, and outputs the voice / non-voice determining unit 330 and a voice recognition unit 240. As a sound source separation means, a method similar to the method of the second embodiment may be used.

<Voice / non-voice determination unit 330>
The speech / non-speech determination unit 330 analyzes the separated sound signals z ~ ₁ (f, t), z ~ ₂ (f, t), ..., z ~ _N (f, t) and converts them into the sound signal of the frame. It is determined whether or not the speech of each target speaker is included, and the determination result is obtained as a sound / non-voice determination result. The determination method is the same as that of the voice / non-voice determination unit 130 of the first embodiment. However, instead of the digital sound signal x (u), each separated sound signal z ~ ₁ (f, t), z ~ ₂ (f, t), ..., z ~ _N (f, t) is used, respectively. The speech / non-speech determination value d _n (t) is obtained from this signal and output to the recognition unit utterance determination unit 250. Therefore, the voice / non-voice judgment unit 330 does not have to perform a process of taking out an input signal for each frame and converting it to the frequency domain.

  By adopting such a configuration, the same effect as in the second embodiment can be obtained.

<Audio section detection device 400>
A speech segment detection apparatus 400 according to the fourth embodiment will be described with reference to FIG. The speech section detection apparatus 400 includes a speech input unit 210, a sound source separation unit 420, a speech / non-speech determination unit 430, a speech recognition unit 240, a recognition unit utterance determination unit 250, and a result output unit 260. The processing contents of the sound source separation unit 420 and the voice / non-voice judgment unit 430 are different from those in the second embodiment. Only the different parts will be described.

<Sound source separation unit 420>
The arrival direction estimation unit 421 in the sound source separation unit 420 outputs T _n to the separated sound signal generation unit 223 and the speech / non-speech determination unit 430. Other processes are the same as those in the second embodiment.

<Voice / non-voice determination unit 430>
The speech / non-speech determination unit 430 receives a set T _n of times when speech arrives from the direction θ _n , and uses this to set a speech / non-speech determination value d _n (t).

として求め、認識単位発話判定部１５０に出力する。つまり、時刻ｔが方向θ_nから音声が到来する時刻の集合Ｔ_ｎに含まれる場合には、音声であると判定し、含まれない場合には、非音声であると判定する。

And output to the recognition unit utterance determination unit 150. That is, when the time t is included in the set T _n of times when sound arrives from the direction θ _n , it is determined to be sound, and when it is not included, it is determined to be non-speech.

By adopting such a configuration, the same effect as in the second embodiment can be obtained. Furthermore, by using the output value T _n of the arrival direction estimation unit 420 in the sound source separation unit 420 in the separated sound signal generation unit 223 and the voice / non-voice determination unit 430, the amount of calculation can be reduced.

  A speech segment detection apparatus 500 according to the fifth embodiment will be described with reference to FIG. The speech section detection apparatus 500 includes a speech input unit 210, a sound source separation unit 220, a speech / non-speech determination unit 530, a speech recognition unit 240, a recognition unit utterance determination unit 250, and a result output unit 260. The processing contents of the voice / non-voice judgment unit 530 are different from those in the second embodiment. Only the different parts will be described.

<Voice / non-voice determination unit 530>
The speech / non-speech determination unit 530 outputs digital sound signals x ₁ (u), x ₂ (u),..., X _M (u) including the speech of a plurality of target speakers for each frame of a predetermined length. , And the sound signal x ₁ (t), x ₂ (t), ..., x _M (t) of the frame is analyzed, and the sound signal x ₁ (t), x ₂ (t), ..., of the frame is analyzed. It is determined whether or not the speech of each target speaker n is included in x _M (t), the determination result is obtained as a speech / non-speech determination result d _n (t), and the recognition unit utterance determination unit 250 determines Output.

The voice / non-voice determination unit 530 includes, for example, a feature extraction unit 531 and an identification unit 533. The feature extraction unit 531 has the same configuration as the feature extraction unit 131 in the voice / non-voice determination unit 130 of the first embodiment. However, since the digital sound signals x ₁ (u), x ₂ (u),..., X _M (u) including the voices of a plurality of target speakers are input, one digital sound signal x ( The difference is that u) is selected and processed.

(Feature extraction unit 531)
The feature extraction unit 531 selects one digital sound signal x (u) from the digital sound signals x ₁ (u), x ₂ (u),..., X _M (u) output from the sound input unit 210. Then, the digital sound signal x (u) is divided for each frame, and a sample series x (t) of amplitude values corresponding to the time length of the frame is taken out. Next, the feature extraction unit 531 performs fast Fourier transform on the extracted digital sound signal x (t) to calculate a frequency-domain sound signal x (f, t).

Further, the feature extraction unit 531 uses the sound signal x (f, t) in the frequency domain, and an acoustic feature vector O _t = {o _{t, 0} , o _t having a P-dimensional log mel spectrum of the frame t as an element. _{, 1} ,..., O _{t, P−1} } are calculated and used as the output of the feature extraction unit 531.

(Identification part 533)
In the identification unit 533, first, the likelihood of the speech feature vector O _t of the input speech and a separately prepared speech probability model or non-speech probability model of each speaker, for example, a mixed normal distribution model (GMM: Gaussian Mixture Model) b _j (O _t ) is calculated as follows:

但し、n'はGMMの種別を表し、n'=0,1,…,Nであり、n'=0のとき非音声モデルとし、n'=1,2,…，Nのとき、それぞれ各話者の音声モデルとする。kはGMMに含まれる正規分布の番号であり、k=1,2,…,Kとし、w_n',kはK番目の正規分布の混合重みを、μ_n',k,lは平均値を、σ² _n',k,lは分散値を表す。なお、非音声及び各話者の混合正規分布モデル（GMM）は、事前に非音声と各話者が発声した音声信号の音声特徴ベクトルを用いて学習し、用意しておく。図示しないＧＭＭ記憶部に、確率モデル(GMM)を記憶しておき、識別部５３３は、ＧＭＭ記憶部を参照して、上述の尤度b_n'(O_t)を計算する。

However, n ′ represents the type of GMM, and n ′ = 0, 1,..., N. When n ′ = 0, a non-speech model is used, and when n ′ = 1, 2,. The speaker's voice model. k is the normal distribution number included in the GMM, k = 1,2, ..., K, w _{n ', k} is the Kth normal distribution mixture weight, and μ _{n', k, l} is the mean value Σ ² _{n ′, k, l} represents a variance value. Note that a mixed normal distribution model (GMM) of non-speech and each speaker is prepared by learning in advance using speech feature vectors of non-speech and speech signals uttered by each speaker. A probability model (GMM) is stored in a GMM storage unit (not shown), and the identification unit 533 refers to the GMM storage unit to calculate the above-described likelihood b _{n ′} (O _t ).

Next, the identification unit 533 obtains a voice / non-voice judgment value d _n (t) for each frame by the following formula.

つまり、話者nの音声モデルが、全ての音声モデル及び非音声モデルの中で、最大の尤度を与える場合には、その話者の音声であると判定し、d_n(t)=1とし、それ以外の場合は、d_n(t)=0とする。

That is, when the speech model of speaker n gives the maximum likelihood among all speech models and non-speech models, it is determined that the speech is that speaker's speech, and d _n (t) = 1 Otherwise, d _n (t) = 0.

  By adopting such a configuration, the same effect as in the second embodiment can be obtained.

<Modification>
In this embodiment, the digital sound signals x _1, including a voice of a plurality of speakers of interest _{(u), x 2 (u} ), ..., 1 single digital sound signal x of x _M (u) (u) , And processing is performed, but the voice / non-voice judgment value d _mn (t) is obtained from all the digital sound signals x ₁ (u), x ₂ (u), ..., x _M (u). The average value may be used as the final output value d _n (t). By adopting such a configuration, it is possible to prevent a bias caused by selecting one microphone.

The digital sound signals x _1, including a voice of a plurality of speakers of interest _{(u), x 2 (u} ), ..., selects one of the digital sound signal x (u) of the x _M (u) Using the technique described in Non-Patent Document 2, the speech / non-speech determination value d _n (t) may be obtained by determining the speech section of each speaker using the information criterion. Furthermore, a speech / non-speech determination value may be obtained from all digital sound signals, and the average value may be used as the final output value d _n (t).

  When the voice / non-voice determination unit 530 of the present embodiment is used, only one digital sound signal x (u) is used, and a plurality of speaker's voices are included in the digital sound signal x (u). Diarization can be performed even if it is included. However, at this time, the voice recognition unit also performs voice recognition using only x (u) including the voices of a plurality of speakers.

<Program and recording medium>
Also, the speech segment detection method or speech recognition method according to the present invention is implemented by a computer program to constitute a speech segment detection device or speech recognition device comprising a computer. In this case, each process of a program for causing a computer to function as a target device (a device having the functional configuration shown in the drawings in various embodiments) or a processing procedure (shown in each embodiment) is processed by the computer. A program to be executed by the computer may be downloaded from a recording medium such as a CD-ROM, a magnetic disk, or a semiconductor storage device or via a communication line into the computer, and the program may be executed. The sound signal recorded by the microphone is used as an input, the computer program of the speech section detection method or speech recognition method according to the present invention is executed, and only the resulting speech section is reproduced from the speaker via the result output unit, or the sound You may display a recognition result on a display via a result output part.

<Experimental result>
A voice recognition experiment was performed in the environment shown in FIG. 15 using the voice recognition device 200 ′ of the first modification of the second embodiment. In the experiment, the speech recognition unit 200 ′ having the configuration shown in FIG. There were 4 speakers. The conversation held here is an announcement about one subject among the speakers, and a question and answer session where other speakers ask questions at any time.

  The sound source separation unit 220 uses eight microphones, and the diarization in the voice / non-voice judgment unit 230 uses the three microphones in the center of FIG. The frame length was 64 ms and the frame shift was 32 ms. The speech recognition unit 240 used a speech recognition system having the form shown in FIG. The acoustic model has HMMs for 43 types of phonemes, and there are three states for each phoneme. Each state has its phoneme context (what is the phoneme in front and what is the phoneme that follows) Accordingly, one of 3064 probability density distributions is assigned. As a recognition graph, we constructed a recognition graph with the pronunciation of 100,000 words and the logarithmic value of the 3-word chain probability of the words contained in the 100,000 words as the weight of the arc. The following table shows the word error rate of the speech recognition result when the speech speech of four participants and 10 participants is input using the speech recognition apparatus 200 '. Here, the word error rate is a ratio of errors per word when the correct sentence is compared with the recognition result, and is usually calculated as follows.

The number of replacement errors is the number of times the correct sentence word is recognized as another word in the recognition result, the number of insertion errors is the number of times a word that does not exist in the correct sentence is inserted into the recognition result, and the number of deletion errors is the correct sentence. This represents the number of times that the included word was not included in the recognition result. A smaller word error rate means higher recognition accuracy.

  For comparison, the word error rate when each speech segment obtained by conventional speech segment detection is recognized is also shown. In the speech section detection of the conventional method, the acoustic features are extracted from the input speech with a frame length of 64 ms and a shift length of 32 ms, and a 12-dimensional log mel spectrum vector (P ′ = 12) is extracted. The identification unit in the speech / non-speech determination unit 230 uses a GMM as a probability model for speech and non-speech, and each GMM has a normal distribution of K = 32. Each GMM sequentially generates and updates using the method disclosed in Non-Patent Document 1 described above. Furthermore, the above-described error correction processing is also applied.

単語誤り率を大幅に削減する効果があることが示された。また、単語誤り率が大幅に削減されたことから、音声区間検出の精度も改善されていることが分かる。

It was shown that the word error rate is greatly reduced. Also, since the word error rate has been greatly reduced, it can be seen that the accuracy of speech segment detection is also improved.

100, 200, 300, 400, 500 Voice section detection device 100 ′, 200 ′ Voice recognition device 110, 210 Voice input unit 130, 230, 330, 430, 530 Voice / non-voice judgment unit 140, 240 Voice recognition unit 150, 250 recognition unit utterance determination unit 160, 160 ′, 260, 260 ′ result output unit 220, 320, 420 sound source separation unit

Claims (10)

A sound signal is extracted for each frame of a predetermined length, the sound signal of that frame is analyzed, it is determined whether or not the sound signal of the target speaker is included in the sound signal of that frame, and the determination result is determined as sound / A voice / non-voice determination step to be obtained as a non-voice determination value;
A speech recognition step for obtaining a sequence of recognition units included in the sound signal and speech time information of each recognition unit; a speech / non-speech determination value for each frame obtained in the speech / non-speech determination step; Receiving the series of recognition units and the speech time information of each recognition unit obtained in the speech recognition step, and based on the total value of the speech / non-speech determination value of the frame corresponding to the speech time of the recognition unit, A recognition unit utterance determination step for determining whether or not an utterance is made by a target speaker for each recognition unit;
A speech section detection method having A sound signal including the speech of a plurality of target speakers is extracted for each frame of a predetermined length, the sound signal of that frame is analyzed, and the sound signals of the target speakers are included in the sound signal of that frame. A voice / non-speech determination step for determining whether the determination result is a speech / non-speech determination result;
A sound source separation step for generating a plurality of separated sound signals that emphasize the voices of each speaker, using a sound signal including the voices of the plurality of speakers as targets;
Speech recognition step for obtaining a sequence of recognition units included in the separated sound signal and utterance time information of each recognition unit; speech / non-speech for each speaker frame obtained in the speech / non-speech determination step The determination value, the sequence of recognition units of each speaker obtained in the speech recognition step, and the speech time information of each recognition unit are received, and the speech / frame of the frame corresponding to the speech time of the recognition unit of each speaker is received. A recognition unit utterance determination step for determining whether or not an utterance is made by the speaker for each recognition unit, based on the magnitude of the total value of the non-voice determination values;
A speech section detection method having A speech recognition method using the speech section detection method according to claim 1 or 2,
In the recognition unit utterance determination step, a result output step of outputting only the recognition unit determined to be uttered by the target speaker as a speech recognition result of the target speaker is provided.
A speech recognition method characterized by the above. A sound signal is extracted for each frame of a predetermined length, the sound signal of that frame is analyzed, it is determined whether or not the sound signal of the target speaker is included in the sound signal of that frame, and the determination result is determined as sound / A voice / non-voice determination unit to be calculated as a non-voice determination value;
A sequence of recognition units included in the sound signal, a speech recognition unit for obtaining speech time information of each recognition unit, a speech / non-speech determination value for each frame obtained in the speech / non-speech determination unit, Receiving a series of recognition units and speech time information of each recognition unit obtained in the speech recognition unit, and based on the total value of the speech / non-speech determination value of the frame corresponding to the speech time of the recognition unit, A recognition unit utterance determination unit that determines whether or not an utterance is made by a target speaker for each recognition unit;
A speech section detecting device having A sound signal including the speech of a plurality of target speakers is extracted for each frame of a predetermined length, the sound signal of that frame is analyzed, and the sound signals of the target speakers are included in the sound signal of that frame. A voice / non-voice determination unit that determines whether or not the determination result is obtained as a voice / non-voice determination result;
A sound source separation unit that generates a plurality of separated sound signals in which the voices of the speakers are emphasized using a sound signal including the voices of the plurality of speakers as the target;
A speech recognition unit for obtaining a sequence of recognition units included in the separated sound signal and speech time information of each recognition unit, and speech / non-speech for each speaker frame obtained in the speech / non-speech determination unit The determination value, the sequence of the recognition units of each speaker obtained in the speech recognition unit, and the speech time information of each recognition unit are received, and the voice / frame of the frame corresponding to the speech time of the recognition unit of each speaker is received. A recognition unit utterance determination unit that determines whether or not an utterance has been made by the speaker for each recognition unit, based on the magnitude of the aggregate value of the non-voice determination values;
A speech section detecting device having A speech recognition device using the speech section detection device according to claim 4 or 5,
The recognition unit utterance determination unit includes a result output unit that outputs only the recognition unit determined to be uttered by the target speaker as a speech recognition result of the target speaker.
A speech recognition apparatus characterized by that. Program for causing a computer to function as the voice interval detection equipment according to claim 4 or claim 5.   A computer-readable recording medium on which the program according to claim 7 is recorded.   A program for causing a computer to function as the speech recognition apparatus according to claim 6.   A computer-readable recording medium on which the program according to claim 9 is recorded.

Images