JP7404657B2 - Speech recognition device, speech recognition program, and speech recognition method - Google Patents

Description

The present invention relates to a speech recognition device, a speech recognition program, and a speech recognition method, such as a system that emphasizes sound in a specific area and suppresses sound in other areas, such as a sound collection system used in a noisy environment. can be applied to

When using a voice communication system or a voice recognition application system in a noisy environment, ambient noise that mixes in with the necessary target voice is a nuisance that hinders good communication and causes a decrease in voice recognition rate. Conventionally, in environments where multiple sound sources exist, beamformers using microphone arrays have been used as a technology to separate and collect only sounds in specific directions, thereby avoiding the mixing of unnecessary sounds and obtaining the necessary target sounds. (Beam Former; hereinafter referred to as "BF").

BF is a technology that forms directivity using the time difference between signals reaching each microphone (see Non-Patent Document 1). BF can be roughly divided into two types: additive type and subtractive type. In particular, the subtractive BF has the advantage that directivity can be formed with a smaller number of microphones than the additive BF.

FIG. 4 is a block diagram showing the configuration of the subtraction type BF 400 when the number of microphones is two (M1, M2).

The subtractive BF 400 includes a delay device 410 and a subtracter 420.

The subtractive BF 400 calculates the time difference between signals in which a sound existing in a target direction (hereinafter referred to as "target sound") arrives at each microphone, and adds a delay using a delay device 410 to adjust the phase of the target sound. match. The time difference at this time is calculated by the following equation (1). Here, d is the distance between the microphones, c is the speed of sound, τ _i is the amount of delay, and θ _L is the angle from the perpendicular direction to the target direction with respect to the straight line connecting each microphone.

Further, here, if the blind spot exists in the direction of the microphone mc1 with respect to the centers of the microphones mc1 and mc2, the delay device 410 performs delay processing on the input signal x ₁ (t) of the microphone mc1. Thereafter, the subtracter 420 performs subtraction processing according to equation (2).

The subtracter 420 can perform subtraction processing in the frequency domain as well, in which case equation (2) is changed to equation (3) below.

Here, when θ _L = ±π/2, the formed directivity becomes a cardioid type unidirectivity, as shown in FIG. 5(a), and when θ _L =0,π, This results in a figure-eight bidirectional pattern as shown in (b). Here, a filter that forms a unidirectional pattern from an input signal will be referred to as a "unidirectional filter," and a filter that forms a bidirectional pattern will be referred to as a "bidirectional filter."

In the subtracter 420, by using a spectral subtraction method (hereinafter also referred to as "SS"), it is also possible to form strong directivity in the blind spot of the bidirectional pattern. The directivity due to the SS is formed at all frequencies or at a specified frequency band according to equation (4). In equation (4), the input signal X ₁ of the microphone mc1 is used, but the same effect can be obtained with the input signal X ₂ of the microphone mc2. Here, β is a coefficient for adjusting the strength of SS. When performing SS processing according to equation (4), if the value becomes negative during subtraction, flooring processing is performed to replace it with 0 or a value smaller than the original value. This method uses a bidirectional filter to extract sounds that exist in directions other than the target direction (hereinafter referred to as "non-target sounds"), and then subtracts the amplitude spectrum of the extracted non-target sounds from the amplitude spectrum of the input signal. , the direction of the target sound can be emphasized.

If you want to collect only the sounds that exist in a certain area (hereinafter referred to as "target area sounds"), it is not possible to collect only the sound sources that exist around that area (hereinafter referred to as "non-target area sounds") by simply using a subtractive BF. There is also a possibility that sound may be picked up. Therefore, the technology described in Patent Document 1 uses a method (area sound collection).

FIG. 6 is an explanatory diagram showing a process of collecting target area sound from a target area sound source using two microphone arrays MA1 and MA2.

FIG. 6A is an explanatory diagram showing a configuration example of each microphone array. FIGS. 6(b) and 6(c) are graphs (image diagrams) showing the BF outputs of the microphone arrays MA1 and MA2 shown in FIG. 6(a) in the frequency domain, respectively. In area sound collection, as shown in FIG. 6A, sound is collected by crossing the directivity of microphone arrays MA1 and MA2 from different directions at an area where sound is desired to be collected (target area). In the state shown in Fig. 6(a), the directivity of each microphone array MA1, MA2 includes not only the sound existing within the target area (target area sound) but also the noise in the direction of the target area (non-target area sound). There is. However, as shown in FIGS. 6(b) and 6(c), when comparing the directivity of microphone arrays MA1 and MA2 in the frequency domain, the target area sound components are included in both outputs, but the non-target area The sound components will be different for each microphone array. In conventional area sound collection technology, by utilizing such characteristics, only the target area sound can be extracted by suppressing components other than those commonly included in the BF outputs of the two microphone arrays MA1 and MA2.

When performing conventional area sound collection processing in the environment shown in Figure 6(a), as an example, first estimate the ratio of the amplitude spectrum of the target area sound included in the BF output of each microphone array, and use that as the correction coefficient. Two microphone arrays will be used. At this time, the correction coefficient for the target area sound amplitude spectrum is calculated by "Equations (5) and (6)" or "Equations (7) and (8)." Here, Y _1k (n), Y _2k (n) are the amplitude spectra of the BF outputs of microphone arrays MA1 and MA2, N is the total number of frequency pins, k is the frequency, α ₁ (n), α ₂ (n) is an amplitude spectrum correction coefficient for each BF output. Moreover, here, mode represents the mode, and median represents the median value.

Thereafter, by correcting each BF output using a correction coefficient and performing SS, it is possible to extract non-target area sounds that exist in the direction of the target area. Furthermore, the target area sound can be extracted by performing SS on the extracted non-target area sound from the output of each BF. For example, in order to extract the non-target area sound N ₁ (n) existing in the direction of the target area as seen from microphone array MA1, as shown in equation (9), from the BF output Y ₁ (n) of microphone array MA1, SS is obtained by multiplying the BF output Y ₂ (n) of array MA2 by the amplitude spectrum correction coefficient. Similarly, according to equation (10), non-target area sound N ₂ (n) existing in the target area direction as seen from microphone array MA2 is extracted.

Thereafter, according to equations (11) and (12), the non-target area sound can be SSed from each BF output to extract the target area sound. Equation (11) indicates that the target area sound is extracted using microphone array MA1 as a reference, and equation (12) indicates that target area sound is extracted using microphone array MA2 as a reference. Note that in equations (11) and (12), γ ₁ (n) and γ ₂ (n) are coefficients for changing the strength during SS.

By the way, when the volume level of the background noise or non-target area sound is high, the SS performed when extracting the target area sound may distort the target area sound or generate a harsh abnormal sound called musical noise.

Therefore, in the method of Patent Document 2, the volume levels of the microphone input signal and the estimated noise are respectively adjusted according to the loudness of the background noise and the non-target area sound, and are mixed with the extracted target area sound. The musical noise generated by the process of extracting target area sound becomes stronger as the volume level of background noise and non-target area sound increases, so the volume level of the sum of the input signal to be mixed and the estimated noise also increases Increase the volume in proportion to the volume level of the area sound. The volume level of the background noise can be calculated from the estimated noise obtained in the process of suppressing the background noise. In addition, the volume level of the non-target area sound is calculated from the sum of the non-target area sound that exists in the direction of the target area extracted in the process of emphasizing the target area sound, and the non-target area sound that exists in the direction other than the target area. can do.

In the method of Patent Document 2, the ratio of the input signal to be mixed and the estimated noise is determined from the volume level of the estimated noise and the non-target area sound. However, if non-target area sound exists near the target area, if the volume level of the input signal to be mixed is too high, the non-target area sound will be mixed with the target area sound, making it difficult to tell which is the target area sound. . Therefore, in the method of Patent Document 2, when the non-target area sound is loud, the volume level of the input signal to be mixed is lowered, and the volume level of the estimated noise is increased and mixed. In other words, in the method of Patent Document 2, if the non-target area sound does not exist or the volume level is low, the proportion of the input signal is increased, and conversely, if the volume level of the non-target area sound is high, the proportion of the estimated noise is increased. Mix.

In this way, by using the technique of Patent Document 2, by mixing the input signal and the estimated noise with the target area sound, musical noise can be masked and can be heard as normal background noise without any discomfort. Further, with the method of Patent Document 2, distortion of the target area sound can be corrected using the target area sound component included in the microphone input signal, and the sound quality can be improved.

Japanese Patent Application Publication No. 2014-072708 Japanese Patent Application Publication No. 2017-183902

Futoshi Asano, “Acoustic Technology Series 16 Sound Array Signal Processing - Localization, Tracking and Separation of Sound Sources”, Edited by the Acoustical Society of Japan, Corona Publishing, February 25, 2011.

By the way, with the advancement of speech recognition technology over the years, it has become possible to achieve fairly high-precision recognition even in a quiet environment. However, in a real environment, the target speaker's voice is mixed with various noises and interfering voices from the surroundings, which causes a significant decrease in the recognition rate. Therefore, how to remove background noise from the speech input to the speech recognition engine has become an important issue in realizing a speech interface. Therefore, there is a need for preprocessing that is highly effective in suppressing noise, especially in suppressing interfering speech, even if the target speech has some deformation or distortion.

However, in recent years, the introduction of machine learning, particularly deep learning, has brought about revolutionary advances in speech recognition, making it far more robust against background noise than ever before. For such a speech recognition engine, it is important not to give top priority to noise suppression performance, but to balance the noise suppression performance and the speech quality after noise suppression.

Although methods such as area sound collection and BF as in Patent Documents 1 and 2 suppress ambient noise and improve speech intelligibility, they do not necessarily improve the speech recognition rate. The optimal level of noise suppression depends on various factors, such as the environment of use, the type of noise, and the characteristics of the speech recognition engine, and is difficult to determine unconditionally.

Therefore, there is a need for a speech recognition device, a speech recognition program, and a speech recognition method that perform speech recognition processing based on sound collection processing that contributes to improving the recognition accuracy of speech recognition processing.

A first aspect of the present invention provides: (1) Based on input signals input from a plurality of microphone arrays capable of directing directivity toward a target area, beamformer outputs of each of the microphone arrays are acquired; (2) area sound extraction means for extracting a target area sound by performing area sound collection processing of the target area using the beamformer output; a signal mixing means that performs a mixing process of mixing signals at a plurality of mixing amounts to generate a mixed sound for each mixing amount, and (3) obtaining the results of performing voice recognition processing on each of the mixed sounds. (4) a voice recognition means that performs a reliability calculation process to calculate the reliability of the voice recognition process results for each of the mixed sounds; (5) the signal mixing means selects and outputs a high speech recognition processing result; (6) applying the changed mixing amount to the mixing process to generate mixed sounds for the number of stages; The method further includes a mixture content determining means for determining the quantity, the range of change, and the number of stages.

The speech recognition program according to the second aspect of the present invention causes a computer to: (1) select a beamformer for each of the microphone arrays based on input signals input from a plurality of microphone arrays capable of directing directionality toward a target area; (2) an area sound extraction means for acquiring the output and performing area sound collection processing of the target area using the acquired beamformer output to extract the target area sound; and (2) a purpose extracted by the area sound extraction means. (3) signal mixing means for performing a mixing process of mixing a mixing signal in a plurality of mixing amounts with respect to the area sound to generate a mixed sound for each mixing amount; and (3) speech recognition for each of the mixed sounds. (4) a voice recognition unit that performs a reliability calculation process that acquires the results of the processing and calculates the reliability of the voice recognition process result for each of the mixed sounds; and (4) a voice recognition process result performed by the voice recognition unit. (5) The signal mixing means selects and outputs the most reliable speech recognition processing result from among the speech recognition processing results; (6) applying the mixture amount changed by the number of stages set in the mixing process to generate mixed sounds for the number of stages; (6) controlling the computer according to the selection result of the recognition result selection means; It is characterized in that it also functions as a mixture content determining means for determining the central mixing amount, the variation width, and the number of stages to be applied to the signal mixing means.

A third aspect of the present invention provides a speech recognition method, which includes (1) an area sound extraction means, a signal mixing means, a speech recognition means, a recognition result selection means, and a mixed content determination means, and (2) the area sound extraction means. acquires the beamformer output of each of the microphone arrays based on the input signals input from the plurality of microphone arrays that can direct the directivity toward the target area, and uses the acquired beamformer output to (3) The signal mixing means applies a mixing signal to the target area sound extracted by the area sound extracting means, and mixes a plurality of mixing amounts. (4) The voice recognition means obtains the results of voice recognition processing for each of the mixed sounds, and generates mixed sounds for each mixing amount. (5) the recognition result selection means selects the voice with the highest reliability among the voice recognition processing results performed by the voice recognition means; Selecting and outputting the recognition processing result, (6) the signal mixing means applies to the mixing process a mixture amount that has been changed by a set number of steps in a set change width around the set center mixing amount; (7) the mixture content determining means selects the central mixing amount to be applied to the signal mixing means, the mixing amount to be applied to the signal mixing means, and The method is characterized in that the variation width and the number of steps are determined.

According to the present invention, it is possible to perform speech recognition processing based on sound collection processing that contributes to improving the recognition accuracy of speech recognition processing.

FIG. 1 is a block diagram showing the functional configuration of a speech recognition device according to a first embodiment. 1 is a block diagram illustrating an example of a hardware configuration of a speech recognition device according to a first embodiment; FIG. FIG. 2 is a block diagram showing the functional configuration of a speech recognition device according to a second embodiment. FIG. 2 is a block diagram showing the configuration of a conventional subtractive BF (when the number of microphones is two). FIG. 3 is an explanatory diagram showing an example of a directional filter formed by a conventional subtractive BF (when the number of microphones is two). FIG. 2 is an explanatory diagram illustrating a configuration example in which the beam formers (BF) of two microphone arrays direct the directivity toward the target area from different directions in a conventional sound collection device.

(A) First Embodiment Hereinafter, a first embodiment of a speech recognition device, a speech recognition program, and a speech recognition method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound collection system of the present invention is applied to a sound collection system will be described.

(A-1) Configuration of First Embodiment FIG. 1 is a block diagram showing the overall configuration of a speech recognition device 200 according to the first embodiment.

Based on the input signal supplied from the microphone array unit 100, the speech recognition device 200 distinguishes and collects target area sounds (voices of speakers present in the target area) whose sound source is the target area. This is a device that performs speech recognition processing on target area sounds (for example, processing that converts collected speech into text) and outputs the speech recognition processing results.

The microphone array unit 100 is a means for capturing sound in an area including a sound collection area using a plurality of microphone arrays.

In this embodiment, it is assumed that the microphone array section 100 includes two microphone arrays MA1 and MA2.

Microphone arrays MA1 and MA2 are each placed at an arbitrary location in the space where the target area exists. The positions of the microphone arrays MA1 and MA2 with respect to the target area may be anywhere as long as the directivity of each microphone array overlaps only in the target area. For example, the microphone arrays MA1 and MA2 may be placed opposite to each other with the target area in between. The number of microphone arrays constituting the microphone array section 100 is not limited to two, and if there are multiple target areas, the number of microphone arrays that can cover all the areas is arranged.

The microphone arrays MA1 and MA2 can each be configured using two or more microphones. In this embodiment, each of the microphone arrays MA1 and MA2 will be described as having two microphones. Here, it is assumed that microphone array MA1 includes mc1 and mc2, and microphone array MA2 includes mc3 and mc4. Further, in the example of this embodiment, it is assumed that the distance between two microphones in microphone arrays MA1 and MA2 is 3 cm.

Next, the internal configuration of the speech recognition device 200 will be explained.

As shown in FIG. 1, the speech recognition device 200 includes a signal input section 201, a time/frequency conversion section 202, a directivity formation section 203, an area sound extraction section 204, a signal mixing section 205, a frequency/time conversion section 206, an amplitude It has a spectrum ratio calculation section 207, a speech section detection section 208, a speech recognition section 209, and a recognition result selection section 210. Details of each element constituting the speech recognition device 200 will be described later.

The speech recognition device 200 may, for example, cause a computer equipped with a processor, a memory, etc. to execute a program (including the speech recognition program according to the embodiment), but even in that case, functionally, It can be shown as shown in FIG. Details of the processing of each component of the speech recognition device 200 will be described later.

FIG. 2 is a block diagram showing an example of the hardware configuration of the speech recognition device 200. Note that the symbols in parentheses in FIG. 2 are the symbols used in the second embodiment described later.

FIG. 2 shows a configuration in which the speech recognition device 200 is implemented using software (computer).

The speech recognition device 200 shown in FIG. 2 includes, as hardware components, at least a signal input section 201 and a computer 500 in which a program (a program including the speech recognition program of the embodiment) is installed.

The signal input section 201 can be configured using, for example, an A/D converter. Note that if the computer 500 itself is equipped with an A/D converter, there is no need to separately provide the signal input section 201.

The computer 500 performs area sound collection processing on the audio signal (digital audio signal) supplied from the signal input unit 201 and outputs the resultant sound signal. In this embodiment, it is assumed that a program (software) including the sound collection program of this embodiment is installed in the computer 500.

Note that the computer 500 may be a computer dedicated to the sound collection program, or may be configured to be shared with programs for other functions (for example, the recording device 300).

The computer 500 shown in FIG. 2 includes a processor 501, a primary storage section 502, and a secondary storage section 503. The primary storage unit 502 is a storage unit that functions as a working memory (work memory) of the processor 501, and for example, a memory that operates at high speed such as a DRAM (Dynamic Random Access Memory) is applied. The secondary storage unit 503 is a storage unit that records various data such as an OS (Operating System) and program data (including data of the sound collection program according to the embodiment), and is, for example, a nonvolatile storage such as a FLASH memory or an HDD. sexual memory is applied. In the computer 500 of this embodiment, when the processor 501 starts up, it reads the OS and programs (including the sound collection program according to the embodiment) recorded in the secondary storage unit 503 and expands them onto the primary storage unit 502. Execute.

Note that the specific configuration of the computer 500 is not limited to the configuration shown in FIG. 2, and various configurations can be applied. For example, if the primary storage unit 502 is a nonvolatile memory (for example, a FLASH memory, etc.), the configuration may be such that the secondary memory is excluded.

(A-2) Operation of First Embodiment Next, the operation of the speech recognition device 200 of the first embodiment having the above configuration will be explained.

The signal input section 201 converts the acoustic signals picked up by the microphones mc1 to mc4 of the microphone arrays MA1 and MA2 from analog signals to digital signals, and supplies the digital signals to the time/frequency conversion section 202. In the following, digital audio signals (hereinafter also referred to as "input signals") picked up by the microphones mc1 to mc4 are expressed as x1 to x4, respectively.

The time/frequency converter 202 converts the supplied microphone signal from the time domain to the frequency domain. Below, signals obtained by converting the input signals x1 to x4 into the frequency domain are expressed as X1 to X4, respectively.

The directivity forming unit 203 uses the input signals (X1 to X4) of each microphone that have been time-frequency converted by the time/frequency conversion unit 202 to determine the directivity by BF according to the above equations (3) and (4). Form. In the following, the BF output by the microphone array MA1 is referred to as Y1, and the BF output by the microphone array MA2 is referred to as Y2.

The area sound extraction unit 204 performs SS according to equation (9) or (10) using the BF outputs Y1 and Y2 generated by the directivity forming unit 203, and extracts non-target area sounds existing in the rear direction of the target construction. do. Further, the area sound extracting unit 204 extracts the target area sound Z by performing SS on the extracted noise from the output of each BF according to equation (11) or (12).

The area sound collection processing up to this point extracts only the sounds that occur in the area where the target sound exists, and even if there is noise unnecessary for speech recognition in the surroundings, the target sound (of the speaker in the target area) is extracted. Only the spoken voice can be extracted. Generally, in noise suppression processing, there is a trade-off relationship between the amount of noise suppression and sound quality. If the amount of suppression is increased, the distortion will increase. Area sound collection is an excellent method that can emphasize only the sounds occurring in the target area, but as with general noise suppression, the higher the emphasis effect, the more the distortion will increase. Therefore, even if the target sound signal extracted by area sound collection is directly supplied to the recording device 300 (speech recognition unit 301), a high recognition rate may not be obtained.

As described above, by mixing the input signal component with the area sound output Z as a mixing signal, it is possible to reduce the distortion of the target sound caused by the area sound collection process. Therefore, in the speech recognition device 200 of this embodiment, a certain amount of the input signal as a mixing signal is mixed with the area sound output Z to reduce distortion of the target sound component in the area sound output Z.

If the amount of input signal mixing (level of the input signal to be mixed) with respect to the area sound output Z is increased, the distortion of the target sound component will be reduced, but in exchange, the amount of noise suppression will be reduced. The ratio of the input signal to the area sound output depends on various factors, such as the volume of the target sound, the amount of noise, the type of noise, the environment in which the device is used, and the characteristics of the speech recognition engine used. , it is difficult to uniformly determine. Therefore, it is assumed that the signal mixing unit 205 of this embodiment generates a plurality of mixed sounds M having different mixing amounts of input signals (levels of input signals to be mixed).

Specifically, the signal mixing unit 205 generates Q (Q is an integer of 2 or more) mixed sounds M (M1 to MQ) is generated. The mixed sounds M1 to MQ are obtained by mixing input signals having different attenuation amounts At (At1 to AtQ) with respect to the original input signal into the area sound output Z, respectively. In the following, Q will be referred to as the "number of stages."

In this embodiment, the number of stages Q will be explained as seven. It is assumed that the attenuation amounts At1 to At7 are set in the range of -5 db to 35 db in increments of -5 dB (set in 7 steps with a width of -5 db). That is, the attenuation amounts At1 to At7 are set to -5 dB, -10 dB, -15 dB, -20 dB, -25 dB, -30 dB, and -35 dB [small mixing amount], respectively. In this case, the attenuation amount of At1 (=-5 db) is the smallest (the mixing amount is the largest), and the attenuation amount of At7 (=-35 db) is the largest (the mixing amount is the smallest).

Then, the signal mixing unit 205 generates mixed sounds M1 to M7 by attenuating the input signal as the mixing signal by the respective attenuation amounts of At1 to At7.

The frequency/time conversion unit 206 converts the mixed sounds M1 to M7, which have reduced distortion by mixing the input signal (input sound) to the area sound output Z, into time domain signals (mixed sounds) m1 to m7, The signal is supplied to the speech recognition unit 209.

In the speech recognition device 200, when inputting the mixed sounds m1 to m7 to the speech recognition section 209, the speech section detection section 208 detects the speech section.

The method by which the audio section detection unit 208 determines the presence or absence of target area sound within the sound collection area is not limited, and various methods can be applied. For example, the sound section detection unit 208 uses a method of using the amplitude spectrum ratio of the area sound collection output and the input sound as a method for determining the presence or absence of target area sound within the sound collection area (see Reference 1 below). Alternatively, a method using coherence between BF outputs when performing rear sound pickup can be applied. In the example of this embodiment, the voice section detection unit 208 will be described as determining the presence or absence of target area sound within the sound collection area by a method using an amplitude spectrum ratio.
Reference 1: Special Seki No. 2016-127457

The amplitude spectrum ratio calculation unit 207 obtains the input signal from the time/frequency conversion unit 202 and the area sound output Z from the area sound extraction unit 204, and calculates the amplitude spectrum ratio R.

For example, the amplitude spectrum ratio calculation unit 207 calculates the amplitude spectrum ratio (R 1 or R ₂ ) of the area sound output (Z ₁ or Z ₂ ) and the input signal using the following equation ( ₁₃ ) or (14). Calculate for each frequency. In equations (13) and (14), Win ₁ is the amplitude spectrum of the input signal to microphone array MA1, and Win ₂ is the amplitude spectrum of the input signal to microphone array MA2. Note that the microphones used for calculating Win ₁ and Win ₂ may be any of the microphones forming the microphone arrays MA1 and MA2. Here, it is assumed that Win ₁ is calculated based on the input signal X1 of the microphone mc1, and Win ₂ is calculated based on the input signal X3 of the microphone mc3. In addition, here, Z ₁ is the amplitude spectrum of the area sound output when area sound collection processing is performed using the microphone array MA1 as the main (when the above equation (11) is used), and Z ₂ is the amplitude spectrum of the area sound output when the microphone array MA1 is used as the main one. This is an amplitude spectrum of area sound output when area sound collection processing is performed using array MA2 as the main (when formula (12) described above is used).

Then, the amplitude spectrum ratio calculation unit 207 adds the amplitude spectrum ratios of all frequencies using the following equation (15) or (16) to obtain an amplitude spectrum ratio addition value (U ₁ or U ₂ ).

Here, U ₁ obtained in the process performed using equation (15) is the sum of the amplitude spectrum ratio R _1i of each frequency in the band from the lower frequency limit j to the upper frequency limit k, and is obtained by equation (16). ) is obtained by adding up the amplitude spectrum ratio R _2i of _each frequency in the band from the lower frequency limit j to the upper frequency limit k. Here, the frequency band to be calculated in the amplitude spectrum ratio calculating section 207 may be limited. For example, the above calculation may be performed by limiting the calculation target to 100 Hz to 6 kHz, where audio information is sufficiently included.

Then, the amplitude spectrum ratio calculating section 207 supplies the result of calculating U ₁ or U ₂ as U to the voice section detecting section 208 .

Note that the amplitude spectrum ratio calculation unit 207 calculates the target area sound Z when the area sound extraction unit 204 performs area sound collection processing mainly using the BF output Y ₁ (n) of the microphone array MA1 (as described above). (11)), it is preferable to output _U1 calculated using equation (15) as the amplitude spectrum ratio addition value U. In addition, when the area sound extraction unit 204 calculates the target area sound Z, the amplitude spectrum ratio calculation unit 207 performs area sound collection processing using the BF output Y ₂ (n) of the microphone array MA2 as the main (as described above). When using equation (12)), it is preferable to output _U2 calculated using equation (16) as the amplitude spectral ratio addition value U.

The voice section detection unit 208 compares the amplitude spectrum ratio addition value U supplied from the amplitude spectrum ratio calculation unit 207 with a preset threshold, and determines whether or not the target area sound (voice) exists within the target area. do. In order to distinguish between sudden noise and voice, the voice section detection unit 208 identifies a voice section (a period during which a speaker in the target area is speaking) when an area sound exists for a certain period of time or more. Alternatively, a general method may be used to determine a speech interval, such as regarding a certain period of time after detection of silence as a speech interval in order to distinguish between the end of an utterance and a temporary silent interval such as a plosive or a breather. The speech section detection section 208 detects a speech section through these processes, and supplies the speech section detection result S to the speech recognition section 209.

Here, when the voice section detection unit 208 detects that the target area sound exists within the target area, it outputs "true" as the voice zone detection result S, and indicates that the target area sound does not exist within the target area. If detected, "false" is output as the voice section detection result S.

The speech recognition unit 10 attempts speech recognition processing individually using the mixed sounds m1 to m7 whose mixing levels are changed in stages, and obtains the results of the speech recognition processing for each (hereinafter referred to as "recognition results") A1 to A process to obtain values Re1 to Re7 (hereinafter referred to as "recognition reliability") that quantify the degree of reliability of each of A7 and recognition results A1 to A7 (hereinafter referred to as "recognition reliability calculation process") I do.

When the speech recognition unit 10 performs speech recognition processing (for example, processing for converting speech into text; so-called "Speech to Text" processing) on each of the mixed sounds m1 to m7 to generate recognition results A1 to A7. The specific method is not limited, and various methods can be applied.

Further, the method of recognition reliability calculation processing in which the speech recognition unit 10 analyzes each of the recognition results A1 to A7 to calculate the recognition reliability Re1 to Re7 is not limited, and various methods may be applied. I can do it. For example, the speech recognition unit 10 may use the methods described in References 2 and 3 below for recognition reliability calculation processing. The interval at which the speech recognition unit 10 calculates the recognition reliability (hereinafter referred to as "reliability calculation interval") is not limited. The speech recognition unit 10 may calculate the recognition reliability at regular intervals, for example.
Reference document 2: Japanese Patent Application Publication No. 2005-148342 Reference document 3: Japanese Patent Application Publication No. 2010-175807

The recognition result selection unit 210 selects the recognition result with the highest reliability for the voice section (the section where S=true) and outputs it as the final recognition result As. For example, the recognition result selection unit 210 may perform a process of switching the recognition results (A1 to A7) to be output to the recognition result with the highest recognition reliability at each reliability calculation interval.

(A-3) Effects of the first embodiment According to the first embodiment, the following effects can be achieved.

The speech recognition device 200 of the first embodiment generates a plurality of mixed sounds with different mixing amounts (attenuation amounts; mixing levels), calculates the reliability of the speech recognition processing result of each mixed sound, and calculates the reliability of the speech recognition processing result of each mixed sound. The speech recognition processing result with a high value is output as the final recognition result. As a result, the speech recognition device 200 of the first embodiment can use the optimum mixing amount for speech recognition processing in various usage environments. As a result, the speech recognition device 200 improves the accuracy of speech recognition.

(B) Second Embodiment Hereinafter, a second embodiment of a speech recognition device, a speech recognition program, and a speech recognition method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound collection system of the present invention is applied to a sound collection system will be described.

(B-1) Configuration of Second Embodiment FIG. 3 is a block diagram showing the overall configuration of a speech recognition device 200A according to the second embodiment.

In FIG. 3, the same or corresponding parts as in FIG. 1 described above are given the same or corresponding symbols. The second embodiment will be described below, focusing on the differences from the first embodiment.

In the speech recognition device 200A of the second embodiment, the signal mixing unit 205 and the recognition result selection unit 210 are replaced with the signal mixing unit 205A and the recognition result selection unit 210A, and a mixing level determination unit 211 is further added. This is different from the first embodiment.

In the first embodiment, a plurality of mixing amounts (attenuation amounts) of input signals in the signal mixing section 205 are fixed. In contrast, the second embodiment includes a mixing level determining unit 211 that adaptively determines the mixing amount (attenuation amount) to be applied by the signal mixing unit 205A based on the recognition result by the speech recognition unit 209. However, this embodiment is different from the first embodiment. The specific processing contents (differences from the first embodiment) of the signal mixing section 205, the recognition result selection section 210, and the mixing level determination section 211 will be described later.

(B-2) Operation of Second Embodiment Next, the operation of the speech recognition device 200A of the second embodiment having the above configuration will be described.

The operations of the microphone array section 100, signal input section 201, time/frequency conversion section 202, directivity formation section 203, and area sound extraction section 204 are the same as those in the first embodiment, and therefore the description thereof will be omitted.

In the first embodiment, seven different mixing levels were set. If the number of stages is large, finer stages can be set and a more suitable mixing amount can be selected, but the amount of recognition processing for one speech section will increase and the scale of the device will increase. , problems such as processing delays may occur. On the other hand, if the number of stages is reduced, the process becomes simpler, but since the amount of mixture that can be set is limited, the accuracy of extracting the appropriate amount decreases. Therefore, in this embodiment, the mixing amount of the input signal with respect to the area sound output is adaptively determined based on the recognition result of the speech recognition process.

The signal mixing unit 205A mixes the input signals with respect to the area sound output Z as in the first embodiment, but the mixing level (attenuation amount of the input signal to be mixed) is determined by the mixing level determining unit 211. shall be carried out.

Here, as an example, in the mixing level determining section 211, as in the first embodiment, the mixing level is set to 7 stages (number of stages Q = 7), and the attenuation amount for the input signal at that time is set to At1 to 7. As At7, as in the first embodiment, there are 7 steps (-5 dB, -10 dB, -15 dB, -20 dB, -25 dB, -30 dB, -35 dB) in -5 dB increments between -5 db and -35 db. It is assumed that this has been set. Note that, hereinafter, the width of the attenuation amount for one step when setting the attenuation amounts At to AQ will be referred to as a "change width." For example, as described above, when the attenuation amounts At1 to At7 are set in steps of -5 dB in the range of -5 db to 35 db, the "change width" is -5. Here, in order to simplify the explanation, the variation width is assumed to be constant, but the variation width does not necessarily have to be constant.

Then, as in the first embodiment, the mixed sounds m1 to m7 time-converted by the frequency/time conversion section 206 are input to the speech recognition section 209.

Similarly to the first embodiment, the speech recognition unit 209 individually performs speech recognition on the mixed sounds m1 to m7 whose mixing levels are changed in stages, and generates recognition results A1 to A7 and recognition reliability levels Re1 to Re7 for each of the mixed sounds m1 to m7. get.

Similar to the first embodiment, the recognition result selection unit 210A selects the recognition result with the highest reliability for the voice section (the section where S=true) and outputs it as the final recognition result As.

Next, a method of adapting the mixing amount in the mixing level determining section 211 will be explained.

As a method for adapting the mixing amount in the mixing level determination unit 211 (hereinafter referred to as a "mixing amount adaptation method"), for example, the following two methods can be considered.

As a first mixture amount adaptation method, there is a method in which the range is limited without changing the number of stages Q, and the setting of the mixture amount is made more dense (reducing the range of change). Further, as a second mixture amount adaptation method, there is a method of reducing the number of steps Q to lighten the processing while not changing the variation width of one step of the set mixing amount.

[About the first mixture amount adaptation method]
First, the details of applying the first mixture amount adaptation method (mixture amount densification) to the mixture level determination unit 211 will be described.

Here, it is assumed that the speech recognition unit 209 performs speech recognition processing on the mixed sounds m1 to m7 from an initial state, and as a result, the reliability R4 of the mixed sound m4 is the highest. This result is supplied from the recognition result selection section 210 to the mixture level determination section 211. In response to this result, the mixing level determining unit 211 reduces the range of change in the mixing amount for the subsequent area sound output Z, centering on the attenuation amount (-20 dB) that had the highest reliability last time. do. In this case, the mixing level determining unit 211 resets the attenuation amounts At1 to At7 with −20 db as the center (At4=−20 db), and the change width in steps of −2.5 dB, which is 1/2. In this case, the mixing level determining section 211 sets the attenuation amounts At1 to At7 to -12.5 dB, -15 dB, -17.5 dB, -20 dB, -22.5 dB, -25 dB, and -27.5 dB, respectively. Thereafter, the mixture level determining unit 211 converges the mixture level to the optimum value by further refining the change range using the same method based on the next recognition result.

The mixing level determining unit 211 may adapt the mixing amount (refining the range of variation) until a predetermined condition is met. The mixing level determining unit 211, for example, continues until the change range reaches a predetermined value (hereinafter referred to as “minimum change range”) or until the recognition reliability recognized by the speech recognition unit 209 reaches a predetermined value or higher. The mixing amount may be adapted (the variation range may be refined).

As described above, the mixture level determination unit 211 performs the process of the first mixture amount adaptation method.

[About the second mixture amount adaptation method]
Next, details of applying the second mixture amount adaptation method to the mixture level determining section 211 will be described.

Here, it is assumed that the speech recognition unit 209 performs speech recognition processing on the mixed sounds m1 to m7 from an initial state, and as a result, the reliability R4 of the mixed sound m4 is the highest. At this time, in response to this result, the mixing level determination unit 211 determines the mixing amount for the subsequent area sound output Z by a number of stages centered on the attenuation amount (-20 dB) of the mixed sound m4 that had the highest reliability last time. Q may be reduced from 7 to 5. Since it is easier to set the central attenuation amount when the number of stages Q is an odd number, it is preferable that the mixing level determining unit 211 varies the number of stages Q by two. Then, based on the next recognition result, the mixture level determining unit 211 may further reduce the number of stages Q to reduce the amount of processing using the same method.

The mixture level determination unit 211 may adapt the mixture amount (reduce the number of stages Q) until a predetermined condition is met. In this case, the mixture level determining unit 211 may adapt the mixing amount (reduce the number of stages Q) when the number of stages Q reaches a predetermined number of stages (hereinafter referred to as the "minimum number of stages"). It may be configured to end.

As described above, the mixture level determination unit 211 performs the process of the second mixture amount adaptation method.

In the above, the first and second mixture amount adaptation methods have shown a method of adapting the variation width or the number of stages Q as the parameter to be adapted, but the mixture level determination unit 211 is limited to only one of them. You may adapt both without doing so.

In the above example, only the direction of reducing the adaptation target parameter (change width, number of stages Q) was explained, but if only one direction (reducing direction) is used, the recognition reliability falls to a local value and the value does not change. Therefore, for adaptation, it is necessary to also provide a direction for increasing parameters (change width, number of stages Q). As an evaluation index to be increased, for example, the reliability R of the recognition result can be used. If the reliability value selected by the recognition result selection unit 210A as the final recognition result with the highest reliability among the recognition results does not reach a certain level, the mixture level determination unit 211 determines whether the mixture amount has changed. Adjustments may be made in the direction of increasing the width or the number of steps. At this time, the mixing level determining unit 211 may change the change width by 2 at a time, or may change the change width by 2 times.

Note that the mixing level determining unit 211 may set the amount by which the parameter to be adapted (variation width, number of stages Q) is varied at one time (hereinafter referred to as "adaptation speed") as constant, or You may also do so. For example, the mixture level determining unit 211 may use the recognition reliability R to adjust the adaptation speed. That is, the mixture level determining unit 211 may change the adaptation speed depending on the height of the recognition reliability R (for example, the highest value of the recognition reliability R). For example, if the recognition reliability R (for example, the value of the highest recognition reliability R) is very high (low), the mixture level determining unit 211 greatly reduces (increases) the range of change or the number of stages, and if the recognition reliability R (for example, the value of the highest recognition reliability R) is very high (low), If the amount is low (low), it may be possible to reduce the amount of increase or decrease.

(B-3) Effects of the second embodiment According to the second embodiment, the following effects can be achieved.

In the speech recognition device 200A of the second embodiment, the mixture amount is adaptively adjusted and determined to the optimum value based on the recognition result, so it is possible to determine the mixture amount with very high accuracy or with a small amount of processing. It becomes possible to determine the amount of mixture.

(C) Other Embodiments The present invention is not limited to the above embodiments, and may include modified embodiments as exemplified below.

(C-1) In each of the above embodiments, the speech recognition devices 200 and 200A themselves have the speech recognition section 209, and obtain the results of speech recognition processing using the speech recognition section 209 of the own device. However, the speech recognition devices 200 and 200A may not themselves include the speech recognition unit 209 and may use an external speech recognition means to obtain the results of speech recognition processing.

DESCRIPTION OF SYMBOLS 10... Voice recognition part, 100... Microphone array part, 200, 200A... Voice recognition device, 201... Signal input part, 202... Frequency conversion part, 203... Directivity formation part, 204... Area sound extraction part, 205, 205A... Signal mixing section, 206... Time conversion section, 207... Amplitude spectrum ratio calculation section, 208... Speech section detection section, 209... Speech recognition section, 210, 210A... Recognition result selection section, 211... Mixing level determination section, 300... Recording Apparatus, 301... Speech recognition unit, 410... Delay unit, 420... Subtractor, 500... Computer, 501... Processor, 502... Primary storage unit, 503... Secondary storage unit, MA1, MA2... Microphone array, mc1 to mc4... Microphone.

Claims (5)

Based on the input signals input from a plurality of microphone arrays capable of directing directivity toward the target area, the beamformer output of each of the microphone arrays is acquired, and the acquired beamformer output is used to direct the direction toward the target area. area sound extraction means for extracting target area sound by performing area sound collection processing;
signal mixing means for performing a mixing process of mixing a mixing signal in a plurality of mixing amounts with respect to the target area sound extracted by the area sound extracting means to generate a mixed sound for each mixing amount;
a voice recognition unit that performs a reliability calculation process of acquiring the results of voice recognition processing for each of the mixed sounds and calculating the reliability of the voice recognition process results of each of the mixed sounds;
recognition result selection means for selecting and outputting the most reliable voice recognition processing result from among the voice recognition processing results performed by the voice recognition means;
The signal mixing means applies to the mixing process a mixture amount that is changed by a set number of steps in a set change width around a set center mixing amount, and generates mixed sounds for the number of steps. death,
The speech recognition device further comprises a mixture content determining unit that determines the central mixing amount, the change width, and the number of stages to be applied to the signal mixing unit according to the selection result of the recognition result selecting unit. . further comprising a speech section detection means for detecting a speech section in which the voice uttered by the speaker is occurring in the target area,
The speech recognition device according to claim 1, wherein the speech recognition means performs the speech recognition process and the reliability calculation process only while the speech interval is detected by the speech interval detection means. 2. The mixture content determining means applies the mixture amount selected by the recognition result selection means as a new central mixture amount, and increases or decreases the range of change and/or the number of stages. Speech recognition device. computer,
Based on the input signals input from a plurality of microphone arrays capable of directing directivity toward the target area, the beamformer output of each of the microphone arrays is acquired, and the acquired beamformer output is used to direct the direction toward the target area. area sound extraction means for extracting target area sound by performing area sound collection processing;
signal mixing means for performing a mixing process of mixing a mixing signal in a plurality of mixing amounts with respect to the target area sound extracted by the area sound extracting means to generate a mixed sound for each mixing amount;
a voice recognition unit that performs a reliability calculation process of acquiring the results of voice recognition processing for each of the mixed sounds and calculating the reliability of the voice recognition process results of each of the mixed sounds;
functioning as recognition result selection means for selecting and outputting the most reliable voice recognition processing result among the voice recognition processing results performed by the voice recognition means;
The signal mixing means applies to the mixing process a mixture amount that is changed by a set number of steps in a set change width around a set center mixing amount, and generates mixed sounds for the number of steps. death,
The computer is also made to function as a mixture content determining unit that determines the central mixing amount, the change width, and the number of stages to be applied to the signal mixing unit in accordance with the selection result of the recognition result selecting unit. speech recognition program. In the speech recognition method,
It has an area sound extraction means, a signal mixing means, a voice recognition means, a recognition result selection means, and a mixed content determination means,
The area sound extraction means acquires beamformer outputs of each of the microphone arrays based on input signals input from a plurality of microphone arrays capable of directing directivity toward a target area, and extracts beamformer outputs from the acquired beamformer outputs. Extract the target area sound by performing area sound collection processing of the target area using
The signal mixing means performs a mixing process of mixing a mixing signal at a plurality of mixing amounts with respect to the target area sound extracted by the area sound extracting means, and generates a mixed sound for each mixing amount,
The voice recognition means obtains the results of voice recognition processing for each of the mixed sounds, and performs a reliability calculation process of calculating the reliability of the voice recognition processing results for each of the mixed sounds,
The recognition result selection means selects and outputs the most reliable speech recognition processing result from among the speech recognition processing results performed by the speech recognition means. The signal mixing means selects and outputs the most reliable speech recognition processing result from among the speech recognition processing results performed by the speech recognition means. applying a mixture amount changed by a set number of steps with a set change width to the mixing process, and generating mixed sounds for the number of steps;
The speech recognition method characterized in that the mixture content determining means determines the central mixing amount, the change width, and the number of stages to be applied to the signal mixing means in accordance with the selection result of the recognition result selection means. .

Images