JP7569973B2 - Mask Voice Improvement Device - Google Patents

Description

The present invention relates to a sound quality improving device for improving the sound quality of unclear speech, and more particularly to a masked speech improving device for making speech clearer when speaking while wearing a mask.

In recent years, the World Health Organization (WHO) has issued a global pandemic declaration due to the spread of infectious diseases, and wearing masks to prevent viral infections has become the norm. Many masks are made of nonwoven fabrics and other materials, and have filters of about 5 μm in size to prevent the spread or penetration of viruses, bacteria, and fungi, which are pathogens of infectious diseases. In addition, N95, DS2, KN95, and other masks used as surgical masks and industrial masks are designed to achieve a filtration efficiency of 95% or more for particles of 3 μm or less.

For this reason, when wearing a mask and talking, the voice becomes unclear, causing inconvenience. In addition, wearing a mask is necessary in noisy places such as surgery, airplanes, construction sites, construction sites, waste disposal, factories, etc., and conversation and communication in these places requires accuracy. In addition, voice input is widespread in daily life and social life as an input method for smartphones and various electronic devices, and it is necessary to pronounce clearly even when wearing a mask.

Various techniques have been disclosed in the past to improve the voice that becomes unclear when wearing such a mask. In order to clarify the voice, many ideas have been made, such as making the mesh of the filter of the mouth part of the mask coarse or making holes (Patent Document 1 and Patent Document 2, etc.). However, in such techniques, the filter effect of the mask is sacrificed in exchange for clarifying the voice. In addition, Patent Document 3 and the like are methods of providing a microphone inside the mask to amplify the microphone input voice. In Patent Document 4, a microphone is provided in the pharynx near the outside of the mask to amplify the voice. In such a method, the installation and configuration of the microphone are complicated because the vibration sound of the pharynx is amplified. Furthermore, there are methods such as the spectral subtraction (SS) method and the Werner filter method (comb filter method) as methods of removing noise superimposed on the voice components to make the voice clear, but even if the noise components superimposed on the voice are removed, it is difficult to improve the voice clarity through the mask.

JP 2017-2454 A JP 2019-26999 A JP 2017-50594 A JP 2002-271881 A

In the conventional technology, the mask's fiber structure is roughened only at the mouth area, or holes are drilled to make the mask's voice clearer, but this reduces the antibacterial, dustproof, and filtering effects of the mask, or simply collects and amplifies the sound with a microphone, and does not make the sound clearer when the mask is worn. Therefore, when important matters are to be communicated during surgery or at a noisy construction site, or when voice input is required on a smartphone, the mask must be removed to communicate. Therefore, there is a demand for a method to make conversation clearer while wearing the mask, without sacrificing the mask's filtering effect.

The present invention has been made in consideration of the above-mentioned circumstances, and has an object to provide a voice clarifying device and a mask to which such a device is applied, as described below.
(1) A voice quality improvement device that improves voice and sound quality, even when talking while wearing a mask, and extracts clear conversational voices.
(2) A mask and mask voice improvement aid incorporating a device for clarifying conversational voice.

In order to solve the above-mentioned problems, the masked voice improving device according to the present invention comprises a frequency characteristic correction means for raising a band of approximately 2 KHz or more including a third formant or more of a voice input signal that is received through a mask having a mesh diameter of approximately 5 μm or less, a first differential signal generation means for subtracting a retained voice input signal X(b) without a mask from a retained voice input signal X(a) through a mask to generate a first differential signal G(c), and a second differential signal generation means for subtracting the first differential signal G(c) from a voice input signal X(a') through a mask when actually wearing the mask to generate a second differential signal α, and the second differential signal α is used as a voice output signal when actually wearing the mask .

In the masked voice improving device according to the present invention, the first differential signal G(c) can be learned and stored in advance for each user of the masked voice improving device.

In addition, in the masked voice improving device according to the present invention, the first difference signal G(c) can be modeled for each pattern according to the characteristics of the user (male/female, age) or the material of the mask, and can be used as a reference signal for each user .

The present invention may also be utilized as a mask or an attachment for a mask incorporating the masked voice improving device of the present invention.

By using the masked voice improving device according to the present invention and a mask incorporating such a device, it is possible to improve the voice and sound quality and extract clear conversational voice even when talking with a mask on. This makes it easy to talk and communicate with a mask on even in noisy environments such as during surgery, at construction sites, and in noisy locations, and makes it easy to distribute and input voice to smartphones and the like via a network.

FIG. 1 is a schematic block diagram illustrating a first embodiment to which the present invention is applied. FIG. 2 is an explanatory diagram of formant frequency characteristics. FIG. 11 is a schematic block diagram illustrating a second embodiment to which the present invention is applied. 11A and 11B are diagrams illustrating an example of frequency characteristic correction in the embodiment. 1A and 1B are diagrams showing an example of mounting a dual microphone that can be applied to the present invention.

Hereinafter, the embodiments of the present invention will be described in detail with reference to the drawings. Note that all drawings described in the following examples are drawn as schematic or schematic diagrams for the purpose of explaining the present invention, and the actual dimensions and shapes are not particularly limited. Furthermore, the circuit configuration, dimensions, materials, shapes, relative positions, etc. of the components are not intended to limit the technical scope of the invention to only those unless otherwise specified.

As methods for voice clarity and voice noise reduction, a filter method that uses a filter corresponding to the human voice band to emphasize only the voice part, a spectral subtraction method that estimates the average value of the noise spectrum and subtracts it from the original signal containing noise to reduce noise, and a Wiener filtering method that reduces noise using a comb filter or a linear filter have been proposed. Such noise reduction methods are applied to constant stationary noise such as motor noise and train noise, and non-stationary noise such as constantly changing wind noise, impact noise, and construction site noise, and are effective in reducing noise sounds, but noise reduction alone is insufficient for clarifying the mask voice itself. When recognizing a voice when wearing a mask (hereinafter referred to as "mask sound or mask voice"), it is necessary to consider not only the reduction of stationary and non-stationary noise but also the clarity of the mask voice. The conversation sound quality and sound pressure level of the mask vary greatly depending on the material of the mask, and the conversation frequency band also differs depending on the user (old and young, male and female, age, etc.).

Most household masks are generally made of nonwoven fabric, gauze, or cotton, but recently, products made of general-purpose resins such as polypropylene and polyurethane, as well as functional masks that have been modified to have functions such as sweat absorption, quick drying, antibacterial properties, and a soft texture, are available on the market, and the clarity of conversation voices varies depending on the material. In addition, because surgical masks and industrial masks place importance on antibacterial and dust resistance, it is extremely difficult to talk through the mask. Therefore, conversations while wearing surgical or industrial masks must be amplified through a microphone.

The most common household masks made of nonwoven fabric have a mesh diameter of about 5 μm as mentioned above, and when the sound blocking characteristics of commercially available general-purpose nonwoven masks were measured, it was found that the frequency band below about 2 KHz was not absorbed, and the frequency band above 2 KHz was absorbed (blocked). In addition, in industrial masks such as medical and construction masks, the mesh may be finer, being 3 μm or less. It was found that such masks tend to block sound levels across almost the entire frequency range as a whole compared to nonwoven masks, although this depends on the material. For this reason, when wearing a mask, the sound tends to be heard as a "muffled sound" with high-pitched sounds attenuated, resulting in a sound that lacks clarity. This "muffled sound" varies depending on the material of the mask, the size of the mesh, the characteristics of the user's voice, etc.

The source of human voice is the vibration of the vocal cords. The voice generated by the vocal cord vibration passes through the vocal tract (larynopsis larynx) and is emitted from the mouth by resonance with the oral cavity, nasal cavity, and paranasal sinuses. The voice generated from the mouth contains many overtones. Factors that indicate the clarity of a person's voice in a conversation must take into account objective indicators (sound quality, voice pitch, sound pressure, noise, etc.) and subjective indicators (language, accent, speaking speed, pauses in speech, etc.), but in the case of clarity of masked sounds, attention is mainly focused on improving objective indicators. In particular, in conversations with masked sounds, it is also important to be able to recognize who is speaking without significantly changing the speaker's sound quality. For this reason, a different approach is taken from the voice clarity means using voice synthesis.

The key points for improving the muffled sound caused by masked voice are the sound quality, voice pitch (frequency characteristics of the generated sound), sound pressure (voice strength level), and noise removal. The sound quality improvement here is for improving the recognition rate of the listener, and in particular, the extent to which formant discrimination, which is considered to be important for discriminating vowels (a, i, u, e, o), should be included. The voice band in a conventional analog telephone line is from 300 Hz to 3.4 KHz, based on the grounds that a band that sufficiently covers up to the second formant is secured since the first and second formants of vowels are greatly affected, but it attenuates from around 2.5 KHz, and does not sufficiently cover the third formant and above. In the muffled sound of masked voice, it is necessary to improve the sound quality of this vowel recognition.

FIG. 2 shows the spectral decomposition of some vowels as examples. If the vowel "i" uttered by a female singer is expressed in terms of formant frequencies uttered with a fundamental tone (X _f1 ) of 440 Hz, a peak appears at the third formant (X _f3 ) of around 3 kHz. This third formant (X _f3 ) is at a level equal to or higher than the second formant. However, there are almost no notable peaks at frequencies higher than this level. This is because the peak appears at the third formant due mainly to the resonance of the fundamental tone by the oral cavity and nasal cavity. In other words, it can be seen that at least the third formant contributes greatly to the sound quality in relation to the fundamental tone of the voice.

The central band of the voice band generally referred to is 100 to 200 Hz for men and 200 to 400 Hz for women. From this, it is preferable to secure the band up to the frequency of the third formant, while raising the level of 2 KHz or more where the vocalization sound is attenuated in the nonwoven mask sound, and maintaining it up to about 4.5 KHz from 3 KHz or more including the third formant. At frequencies higher than this, not only does the voice recognition rate not improve, but it is also recognized as a crackling or noise-like harsh sound, so it does not matter if it is attenuated. In addition, mask sounds with finer filter meshes such as those for medical and construction use generally have a lower sound pressure level and a slightly lower frequency of the formant fundamental tone, although this depends on the mask material, so it is preferable to raise the overall sound pressure level and increase the peaking in the high range more than nonwoven masks.

In addition to the correction of the high-frequency range, noise suppression is also a major factor in improving the clarity of mask sound. Since the sound pressure level of mask sound from surgical masks for medical and construction use is low, simply increasing the sound pressure level amplifies the environmental noise, and the clarity of the mask sound is significantly reduced. Environmental noise in operating rooms and construction sites includes air conditioning motor sounds, warning sounds from treatment equipment and vital sign measuring instruments, construction destruction sounds, wind noise, trains, cars, and other noises that appear as stationary noise or impact noise. Various methods have already been proposed for removing these environmental noises. A representative method is the active noise control (ANC) method, which captures external sounds using an additional microphone and extracts the difference between the captured external sounds and the output sound. In addition, if a lot of stationary noise is included, there is also a method of removing specific frequencies called the passive noise cancellation (CVC: Clear Voice Capture) method.

These noise cancellation methods depend on the environment in which the mask is used: when there is a lot of noise or the environmental noise is complex, the ANC method is used, and when suppressing a specific stationary noise, the CVC method may be used. In addition, when clarifying masking sounds, since the purpose is to extract only the voice, a method of mainly extracting only the voice using a dual microphone, as described later, is also conceivable.

Based on the above findings, a specific embodiment of the present invention will be described. FIG. 1 is a schematic block diagram showing an embodiment 1 to which the present invention is applied. The mask sound improvement device is configured to include a mask sound correction circuit 1 and a differential processing circuit 2. When using this device, the mask-specific "muffled sound" component characteristics are extracted in advance by obtaining sample sounds of a sound (masked sound) when wearing a mask and a direct sound without a mask. For this purpose, the user utters appropriate conversation sounds (for example, a, i, u, e, o, etc.) with and without a mask to obtain a voice sample. The same conversation sounds are obtained by the microphone 10 as the mask sound (a) and the direct sound without a mask (b), but this voice sample is mainly for obtaining the component characteristics of the sound blocked by the mask, and normal conversation sounds may be used, and phase alignment of both is not necessary.

The masked sound (a) and the unmasked direct sound (b) are switched by a switch 11. The input sound picked up by the microphone 10 collects a conversation voice sample, but also includes ambient noise. In the input processing means 12, the input signal is amplified and smoothed, and the masked sound (a) and the unmasked direct sound (b) are roughly matched in level. The input processing means 12 also includes an ALC (Automatic Level Control) circuit, which matches the levels so that the respective audio signals are not distorted. The output is supplied to the noise suppression means 13. Noise suppression at this stage removes buzz sounds and impulsive noise, and a passive noise cancellation method such as CVC (Clear Voice Capture) can be applied.

The output of the noise suppression means 13 is supplied to the frequency characteristic correction means 14. In the frequency characteristic correction means 14, as described above, in the case of a household nonwoven mask, the high frequency components (approximately 2KHz or more) that are attenuated according to the size of the mesh are raised, and the third formant component and higher formants are cut, as well as the above. As an example, in the case of a nonwoven mask with a mesh of about 5 μm, a frequency peaking correction characteristic of approximately 2KHz to 4.5KHz is obtained, which has a preferable result. In addition, in the case of a mask for medical use, construction work, etc., in which the mesh diameter is finer than that of nonwoven fabric and is smaller than 3 μm, as described above, the overall sound pressure level is raised while the peaking of the high frequency band is made higher. Since each characteristic differs depending on the mask material and the size of the mesh diameter, several frequency correction characteristics are applied at the sampling stage to select an appropriate one. FIG. 4 shows examples of frequency characteristic correction applied to several mask samples (the levels are relative to the reference, not absolute amounts). The audio signal corrected by the mask sound correction circuit 1 is supplied to the input side A of the differential processing circuit 2 at the next stage.

The masked sound X(a) and the unmasked direct sound X(b) that have been frequency-corrected by the frequency characteristic correction means 14 are stored and held in the masked sound storage circuit 16 and the unmasked direct sound storage circuit 17 by the switch 15. Here, the switch 11 and the switch 15 are synchronized at the time of input. The held masked sound X(a) and the unmasked direct sound X(b) are used by the subtraction circuit 18 and the difference signal generation means 19 to extract a difference signal G(c). This difference signal G(c) differs depending on the mask material and the user, so it is used as a unique reference for each user. In addition, since this reference is a rough difference "muffled sound" component, it is desirable to find a difference value that is adapted to each user by learning several times. Therefore, the memory learning means 20 acquires the reference each time it is used, and forms a reference signal while updating it each time.

The difference signal G(c) thus obtained is used to create a standard pattern in the modeling circuit 21 according to the characteristics of the user (male/female, age, etc.) and the characteristics of the mask (mesh diameter, material, etc.). In other words, the characteristics of each difference signal can be prepared by creating appropriate model signals (a, b, c, d, ...) according to the characteristics of the user, the characteristics of the mask, and the usage environment. The difference between the masked sound X(a) and the unmasked sound X(b) thus created in advance is used as a reference for the difference signal for the normal masked sound X(a). Here, the reference signal learning function by the memory learning means 20 and the modeling circuit 21 is not necessarily essential as long as the characteristics of the user and the mask characteristics are properly grasped in advance.

The generated differential signal reference G(c) contains the characteristics of the user, the characteristics of the mask, and the sound and noise factors of the usage environment. Therefore, by extracting the differential signal (α) between the mask sound X(a') when actually wearing a mask and the differential signal reference G(c) using the subtraction circuit 22 and the differential signal generating means 23, the obtained differential signal (α) can obtain a sound close to the direct sound with the excess noise components added to the mask sound removed to a certain extent. The differential signal (α) is supplied as an output signal to the output means 27 via the switch 26.

However, the mask sound X(a) in the reference differential signal G(c) and the mask sound X(a') when actually wearing a mask have different conversation voice contents and use times, so the differential residual signal [X(a)-X(a')] is not cancelled out. However, the memory learning means 20 is set to evaluate the differential residual signal and store the minimized differential residual signal of the sample reference input each time. Therefore, although the differential signal (α) is different from the unmasked direct sound X(b), a sample signal is acquired each time the differential signal is used, and the memory learning means 20 repeatedly performs record learning to evaluate the differential residual signal [X(a)-X(a')] and constantly updates it to minimize it.

The differential output signal thus obtained is output as an output signal (α) 24. There are also cases where the differential output signal (α) is not generated and an audio output (β) 25 is output. This audio output (β) 25 is used when the "muffled sound" caused by the mask is sufficiently corrected by the noise suppression means 13 and the frequency correction means 14 alone and does not interfere with conversation. Depending on the situation and the usage environment, the differential output signal (α) 24 and the direct audio output (β) 25 are selectively output by a switch 26 and supplied to an output means 27.

The output means 27 amplifies the signal and outputs it as conversational voice from a speaker 28. This output signal can also be set to communicate with other devices via a network 30 by a communication means 29, or to wirelessly communicate with the speaker's own headset or speaker. This communication means 29 can be achieved by a communication means such as WiFi or Bluetooth.

By combining various noise removal means 31 in cooperation with the output signal of the output means 27, a clearer voice can be obtained. The noise removal means 31 removes noise sounds that are not sufficiently removed by the noise suppression means 13 and the differential signal generation means 23 applied to the above-mentioned mask sound. Environmental noise differs depending on the environment in which the mask is used, and when the type of noise can be specified, it is desirable to combine an appropriate noise removal means with the specified noise. For example, power supply-derived noise (buzzing noise) and motor noise are removed by a notch filter to remove the power supply specific frequency, and for sir noise and white noise, only the voice is extracted by an adaptive filter. In addition, pop noise and impact noise such as at a construction site are removed by a limiter. Active noise control (ANC) is desirable as a noise removal method that can be applied to many types of noise. In ANC, environmental noise is acquired by an environmental sound acquisition microphone 32, and the environmental noise component is applied to the output signal in the opposite phase to reduce the environmental noise, and any commercially available general-purpose ANC can be applied.

FIG. 3 is a schematic block diagram showing an embodiment 2 to which the present invention is applied. In contrast to the single microphone in FIG. 1, the embodiment 2 is a specific example of a dual microphone system using two microphones. As shown in FIG. 5, the two microphones 41 (M-R) and 42 (M-L) may be attached to the left and right headsets, masks, or face shields, respectively, or may be attached around the neck in a shoulder-hanging manner. In either case, the mouth from which the voice is emitted is set to be located in the center of the two microphones. The voices collected by the microphones 41 and 42 are input to input processing means 43 and 44, respectively, where the input signals are amplified and smoothed, and the levels are adjusted by an ALC (Automatic Level Control) circuit so that the respective voice signals are not distorted. The output of the input processing means is supplied to a normalization means 45. This normalization means (normalizer) is a means for adjusting waveforms that have been out of phase with digital processing or the like, and for flattening the sound pressure, but is not necessarily necessary if there is no level matching or phase shift in the input processing.

The left and right microphone sounds M-R and M-L flattened by the normalization means 45 are used in a subtraction circuit 46 to extract a difference signal between the left and right microphone sound signals. As a result, the in-phase audio signal is cancelled and a signal C mainly containing a large amount of environmental noise components is extracted. The left and right microphone sounds are added in an addition circuit 47. In the addition circuit 47, the in-phase audio signal is enhanced. The output signal of the addition circuit 47 is reduced in level by a level reduction circuit 48 and supplied to a subtraction circuit 49. In the subtraction circuit 49, the output signal of the subtraction circuit 46 is subtracted from the signal obtained by reducing the level of the output signal of the addition circuit 47 in the level reduction circuit 48. As a result, the output signal of the subtraction circuit 49 has further reduced environmental noise components and a signal D containing mainly audio components is extracted.

The signal C containing a large amount of noise components and the signal D containing a large amount of voice components extracted by the above circuit configuration are supplied to an adaptive filter means 50. Signal C is converted into digital form by an A/D converter 51 and supplied to an adaptive filter circuit 52, the output of which is provided to a subtraction circuit 54. Meanwhile, signal D containing a large amount of voice components is converted into digital form by an A/D converter 53 and supplied to the subtraction circuit 54. The subtraction circuit 54 works in cooperation with the adaptive filter 52 to adaptively filter so as to pass only sections containing a large amount of voice signal D. By inserting this adaptive filter means 50, noise in non-voice sections is reduced.

The output of the adaptive filter 50 is an extracted voice signal, and this output signal is sent to a masking sound frequency characteristic correction means 55. The correction characteristics of the frequency characteristic correction means 55 perform frequency correction similar to that shown in Fig. 4, and correction characteristics are selected taking into consideration the type of mask and the individual characteristics of the speaker. The masking sound correction circuit output E obtained by the dual microphone in this manner is supplied to the input A of the differential processing circuit 2 in Fig. 1. The subsequent differential processing circuit and subsequent steps are the same as those in Fig. 1, and a description thereof will be omitted.

The method of acquiring the reference signals of the masked sound (a) and the unmasked direct sound (b) in the second embodiment is the same as that in the first embodiment. The user utters appropriate conversational sounds (e.g., a, i, u, e, o, etc.) in advance in the state of the masked sound when wearing a mask and the direct sound without a mask to acquire and use the voice samples. In this case, if the mask characteristics, the usage environment, and the characteristics of the user (male/female, age, etc.) are known, it is also possible to select and use the modeled sample reference signal. By using two microphones as in the second embodiment, voice extraction can be easily and effectively configured with a simple circuit, and the "muffled sound" associated with the masked sound can be removed from the clear voice with the noise section removed. Therefore, it is suitable for use in places with a large amount of environmental noise.

In the embodiment, the microphone and the processing device are arranged outside the mask near the mask, but in the first and second embodiments, or in other embodiments, only the microphone and the transmitting means such as Bluetooth may be arranged inside or near the mask, and the mask voice improving device may be provided outside. In addition, it is not limited to being inside the mask, but can also be incorporated into a face shield, mouth shield, mask cover, eyeglass-shaped attachment, neck pendant, hat, etc. that covers the mask and used as a mask voice improving accessory device. In addition, the circuit configuration may be analog or may be digitalized by digital signal processing of the voice.

According to this invention, it is possible to eliminate the muffled sound that occurs when wearing a mask, and to pick up clear voice for conversation. This invention can be applied to various industrial masks such as nonwoven masks for home use, medical masks, construction masks, and disaster prevention masks, and since it can eliminate inconveniences when using masks, it can be used in a wide variety of situations where masks are used.

REFERENCE SIGNS LIST 1 Masking sound correction circuit 2 Difference processing circuit 10, 41, 42 Microphone 11, 15 Masking/non-masking sound changeover switch 12, 43, 44 Input processing means 13 Noise suppression means 14, 55 Frequency characteristic correction means 18, 22 Subtraction circuit 19, 23 Difference signal generation means 20 Memory learning means 21 Modeling means 27 Output means 31 Active noise canceller 50 Adaptive filter

Claims (4)

a frequency characteristic correction means for increasing a band of about 2 KHz or more including a third formant or more of the voice input signal, the band being equal to or higher than about 2 KHz , for a voice input signal passing through a mask having a mesh diameter of about 5 μm or less ;
a first differential signal generating means for subtracting a retained voice input signal X(b) without a mask from a retained voice input signal X(a) through a mask to generate a first differential signal G(c); and a second differential signal generating means for subtracting the first differential signal G(c) from a voice input signal X(a') through a mask when actually wearing the mask to generate a second differential signal α,
The masked voice improving device is characterized in that the second differential signal α is used as a voice output signal when the mask is actually worn . 2. The masked voice improving device according to claim 1 , wherein the first difference signal G(c) is learned and stored in advance for each user of the masked voice improving device. The masked voice improving device according to claim 2, characterized in that the first difference signal G(c) is modeled for each pattern according to the characteristics of the user (male/female, age ) or the material of the mask, and is used as a reference signal for each user. A mask and a mask accessory incorporating a mask voice improving device according to any one of claims 1 to 3 .

Images