JP7653311B2 - Wireless communication device and wireless communication system - Google Patents

Description

The present invention relates to a wireless communication device for, for example, a specific low-power wireless communication station, and a wireless communication system including multiple wireless communication devices.

For example, Patent Document 1 discloses in Figures 1 and 4 that a noise cancellation circuit is used to perform noise cancellation processing on an audio signal from a microphone of a wireless communication device.

JP 2004-165962 A (FIGS. 1 and 4)

The noise cancellation circuit disclosed in Patent Document 1 only discloses that it "removes noise from the audio signal input from the microphone," but in the prior art, so-called phase-inversion type noise cancellation circuits are widely used, which invert the noise signal outside the microphone and add it to the audio signal input to the microphone. In this phase-inversion type noise cancellation circuit, there is a problem in that when the noise is loud, even when a person is speaking, the level of the audio signal is suppressed, making it impossible to perform noise cancellation so as to effectively output the human speech.

The object of the present invention is to solve the above problems and provide a wireless communication device for wireless communication that can perform noise cancellation to effectively output human speech compared to conventional techniques, and a wireless communication system including multiple such wireless communication devices.

A wireless communication device according to one aspect of the present invention comprises:
A wireless communication device including a modulation transmission unit that modulates a wireless carrier wave in accordance with an input audio signal and transmits a wireless signal,
a first noise canceling unit that performs audio signal processing so as to cancel noise from the input audio signal and outputs the processed audio signal to the modulation transmitting unit;
The first noise cancellation unit performs noise cancellation processing using a deep learning model unit that is trained using characteristic parameters of human voice and determines whether or not the demodulated voice signal contains noise in a non-voice period.

Therefore, according to the wireless communication device of the present invention, by providing a noise cancellation circuit using a deep learning model unit instead of a phase inversion type noise cancellation circuit, it is possible to perform noise cancellation so as to output human speech more effectively than with conventional technology in a wireless communication device that performs wireless calls.

1 is a block diagram showing an example of the configuration of a wireless device 1 according to an embodiment; 2 is a block diagram showing an example of the configuration of a noise canceling unit 13 in FIG. 1 . A block diagram showing an example configuration of the deep learning model unit 35 of Figure 2.

Embodiments and modifications of the present invention will be described below with reference to the drawings. Note that identical or similar components are given the same reference numerals.

Fig. 1 is a block diagram showing an example of the configuration of a radio 1 according to an embodiment. In Fig. 1, the radio 1 is an example of a wireless communication device, and is configured with a receiving antenna 11, a receiving demodulation unit 12, a noise cancellation unit 13, an audio signal amplifier 14, a speaker 15, a control unit 20, an operation unit 21 including a PTT (Push To Talk) key 21A, a microphone 22, an audio signal amplifier 23, a noise cancellation unit 24, a modulation transmission unit 25, and a transmission antenna 26.

Here, the radio 1 according to the embodiment is an example of a radio communication device, for example, a specific low-power radio station for a specific low-power radio communication system. In this embodiment, the radio 1 is characterized in that, in its transmitting section, instead of a phase-inversion type noise cancellation circuit, it is provided with a noise cancellation section 24 that is particularly effective in reducing noise, for example, in FM (frequency modulation) demodulation or PM (phase modulation) demodulation. Also, in this embodiment, the radio 1 further includes, in its receiving section, a noise cancellation section 13 that is particularly effective in reducing noise. Also, a wireless communication system is configured with a plurality of radios 1.

In FIG. 1, a radio signal received by a receiving antenna 11 is input to a receiving demodulation unit 12. The receiving demodulation unit 12 performs low-noise amplification, low-frequency conversion, intermediate frequency amplification, etc. on the received radio signal, and then demodulates it into an audio signal using a predetermined demodulation method, such as FM (frequency modulation) demodulation or PM (phase modulation) demodulation, and outputs the signal to a noise cancellation unit 13. The noise cancellation unit 13 uses a deep learning model unit 35 (FIG. 3) that has been deep-learned using human voice to process the demodulated audio signal so as to cancel noise by passing the audio signal only during the audio signal period, and then outputs the processed audio signal to a speaker 15 via an audio signal amplifier 14.

The microphone 22 converts the input voice into a voice signal and outputs it to the modulation transmission unit 25 via the voice signal amplifier 23 and the noise cancellation unit 24, which has a configuration similar to that of the noise cancellation unit 13. When the PTT key 21A is turned on, the control unit 20 operates the modulation transmission unit 25, which modulates the radio carrier wave according to the input voice signal using a predetermined modulation method, and then transmits the radio signal, which is the modulated radio carrier wave, from the transmission antenna 26 after high-frequency conversion and power amplification.

In this embodiment, the radio 1 has been described as operating in a simultaneous call system in which the transmission frequency and the reception frequency are different, but the present invention is not limited to this. If the radio 1 uses the same transmission frequency and reception frequency, the control unit 20 stops the operation of the reception demodulation unit 12 when the PTT key 21A is turned on. In other words, in a radio 1 that does not simultaneously transmit and receive, during reception, the noise cancellation unit 24 may be operated instead of the noise cancellation unit 13 to perform noise cancellation processing during reception.

Next, referring to FIG. 2, the configuration and operation of the noise cancellation units 13 and 24 in FIG. 1 using the deep learning model unit 35 will be described below.

Figure 2 is a block diagram showing an example configuration of the noise cancellation units 13 and 24 in Figure 1.

Here, the term "phoneme" refers to a unit of sound that distinguishes one word from another in a particular language, the term "vibration rate" refers to the number of shifts between 0 and 1 in the digitized vibration data in each second, and the term "vibration count (VC)" refers to the sum of the values of the digitized vibration data in each frame. Also, the term "vibration pattern" refers to the data distribution of the sum of the vibration counts calculated for each predetermined number of frames along the time axis. The deep learning model unit 35 performs noise cancellation processing by taking into account the difference in the data distribution of different vibration patterns, i.e., the sum of different vibration counts (VS values), and the vibration rate is similar to the vibration count, but the higher the vibration rate, the higher the vibration count.

Both the amplitude and vibration rate of the audio signal are observable. A feature of the noise cancellation units 13 and 24 is to detect audio events according to the amplitude and vibration rate of the audio signal. Another feature is to distinguish between voice and non-voice/silence by measuring the sum of vibration count values of the digitized vibration data for a predefined number of frames. Another feature is to classify the input audio signal data stream into different phonemes according to their vibration patterns. Another feature is to correctly distinguish the first activation phoneme from the input audio signal data stream so as to trigger a downstream processing unit, thereby saving computational costs such as power consumption of the computing system including the processing unit.

In FIG. 2, the noise cancellation units 13 and 24 perform noise cancellation processing using audio event detection, and are configured with an audio signal pre-processing unit 38, an AD converter 39, and an audio signal processing unit 30. Here, the audio signal pre-processing unit 38 performs audio signal pre-processing including high-pass filtering, low-pass filtering, amplification, or a combination thereof, on the analog audio signal, and outputs the processed analog audio signal to the AD converter 39. That is, the audio signal pre-processing unit 38 passes only a predetermined level range of human audio signals and a predetermined bandwidth for the audio signal from the microphone 22. Next, the AD converter 39 AD converts the analog audio signal into a digital audio signal according to a predetermined reference voltage Vref and an allowable voltage Vadm (<Vref), and outputs the signal to the input interface 36 of the audio signal processing unit 30.

In this embodiment, in the AD converter 39, the allowable voltage Vadm, which is smaller than the reference voltage Vref, is used in combination with the reference voltage Vref to form a first threshold voltage Vth1 (= Vref + Vadm) and a second threshold voltage Vth2 (= Vref - Vadm). The AD converter 39 does not perform AD conversion on noise equal to or greater than the first threshold voltage Vth1 or equal to or less than the second threshold voltage Vth2 based on the first threshold voltage Vth1 and the second threshold voltage Vth2, but performs AD conversion on the audio signal in between, thereby removing noise and interference from the input analog audio signal. Here, for example, if Vref = 1.0 V and Vadm = 0.01 V, it can be understood that the number of vibrations in the vibration data is small in a quiet environment and the number of vibrations in the vibration data is large in a voice environment. In this embodiment, the "frame size" means the number of sampling points corresponding to the digitized vibration data in each frame, and the "phoneme window Tw" means the time for collecting the audio features of each phoneme. In a preferred embodiment, the duration Tf of each frame is, for example, 0.1 to 1 millisecond (ms), and the phoneme window Tw is, for example, about 0.3 seconds. In a further preferred embodiment, the number of sampling points corresponding to the digitized vibration data in each frame ranges, for example, from 1 to 16.

When analyzing audio signals, short-term analysis methods are usually adopted since most audio signals are stable over a short period of time. For example, assuming that the sampling frequency fs used by the AD converter 39 is 16000 and the duration Tf of each frame is 1 ms, the frame size is fs x 1/1000 = 16 sample points.

In FIG. 2, the audio signal processing unit 30 is, for example, a computer device.
(1) a CPU (Central Processing Unit) 31 that executes predetermined audio signal processing such as noise cancellation;
(2) a ROM (Read Only Memory) 32 for storing an operating system for executing the basic processing of the CPU 31, a program for the audio signal processing, and data necessary for executing the program;
(3) a RAM (Read Access Memory) 33 for storing data being processed when the CPU 31 executes an operating system for executing basic processing and the audio signal processing program;
(4) a non-volatile EEPROM (Electrically Erasable Programmable Memory) 34 for storing setting data, etc., which are necessary for executing the audio signal processing, as described later;
(5) A deep learning model unit 35 that is composed of, for example, a neural network, removes noise from voice signal data that is input through deep learning based on human voice signal data, and extracts and outputs substantially only the voice signal;
(6) an input interface 36 that performs a predetermined signal conversion process for converting the audio signal data input from the AD converter 39 into a signal specification value for a subsequent stage and outputs the result to the CPU 31;
(7) an output interface 37 that performs a predetermined signal conversion process for converting the voice signal data from which noise has been removed by the deep learning model unit 35 into a signal specification value in a subsequent stage, and outputs the data to the wireless device 1 via the terminal T12, the audio line L2, etc.;
The present invention is configured to include the following components:

Here, the EEPROM 34 stores, for example, a series of vibration count values VC, a sum of vibration count values VS, a sum of vibration count values VSf, a sum of vibration count values VSp (described later), and audio feature values of all feature vectors. Note that the EEPROM 34 may be a storage device such as an external memory. The audio event detection method applied to the audio signal processing unit 30 is executed by the CPU 31 during runtime to capture audio events. Audio event detection is performed assuming fs=16000, Tf=1 ms, and Tw=0.3 s.

Specifically, the CPU 31 calculates the sum of the vibration data values of the current frame to be processed (i.e., within 1 ms) to obtain the vibration count value VC, and then stores the VC value of the current frame at time Tj in the EEPROM 34. Here, the vibration count values VC of x frames are added together to obtain the sum VS of the vibration count values of the current frame at time Tj. The x frames include the current frame. In one embodiment, the CPU 31 adds the vibration count value VC of the current frame at time Tj to the sum VSp of the vibration count values of the (x-1) frames immediately preceding it to obtain the sum VS (=VC+VSp) of the vibration count values of the x frames at time Tj.

In a modified example, the CPU 31 adds the vibration count value VC of the current frame at time Tj, the sum of the vibration count values of the immediately succeeding y frames VSf, and the sum of the vibration count values of the immediately preceding (x-y-1) frames VSp to obtain the sum of the vibration count values of the x frames at time Tj (=VC+VSf+VSp), where y is equal to or greater than zero. The CPU 31 stores the values of VS, VSf, and VSp in the EEPROM 34. In a preferred embodiment, the duration (x×Tf) of x frames (phoneme window Tw) is about 0.3 seconds. In a further preferred embodiment, the number of sampling points corresponding to the digitized vibration data of x frames is in the range of x to 16x.

In general, for voice signal data, the vibration patterns of the vibration count value VC are similar for the same phoneme, but the vibration patterns of the VS values for different phonemes are completely different. Therefore, the vibration patterns of the vibration count value VC can be used to distinguish between phonemes. In particular, it is known that the frequency distribution of the vibration count value VC is completely different between, for example, the cry of a chicken or a cat and human voice, and most of the vibration count values VC of human voice are distributed below 40.

In the training phase, the CPU 31 of the audio signal processing unit 30 first executes a predetermined audio signal data collection method multiple times to collect multiple feature vectors for multiple phonemes, and then adds labels corresponding to the multiple feature vectors to form multiple labeled training examples. The multiple labeled training examples for different phonemes including the activation phoneme are then applied to training of the deep learning model unit 35. Finally, a trained deep learning model unit 35 (which constitutes a predictive model of the audio signal data) is created to classify whether the input audio signal data stream includes the activation phoneme. When a predetermined phoneme is specified as the activation phoneme of the audio signal processing unit 30, the deep learning model unit 35 is trained with multiple labeled training examples for different phonemes including at least the specified phoneme.

That is, in the training phase, the set of labeled training examples is used to train the deep learning model unit 35 so that the deep learning model unit 35 recognizes a given activation phoneme, for j=0-299, based on three speech features (e.g., (VSj, TDj, TGj)) of each frame of the labeled training examples. At the end of the training phase, the trained deep learning model unit 35 provides a trained score corresponding to the activation phoneme, which is then used as a criterion for classifying an input stream of speech signal data at run-time. Note that VSj, TDj, and TGj are defined as follows:
(1) VSj: the sum of vibration counts in frame j (VS value);
(2) TDj: the time period between non-zero sums of vibration counts (VS values) in frame j; and (3) TGj: the time gap between non-zero sums of vibration counts (VS values) in frame j.

Various machine learning techniques related to supervised learning can be used to train the deep learning model unit 35, such as support vector machine (SVM) methods, random forest methods, convolutional neural network methods, etc. In supervised learning, a function calculation unit (i.e., the deep learning model unit 35) is created using a number of labeled training examples, each of which consists of an input feature vector and a labeled output. When trained, the deep learning model unit 35 can be applied to new unlabeled examples to generate a corresponding score or prediction value.

Figure 3 is a block diagram showing an example of a detailed configuration of the deep learning model unit 35 in Figure 2.

The deep learning model unit 35 is implemented using a neural network, for example, as shown in FIG. 3. Here, the neural network includes an input layer 41, at least one and preferably multiple intermediate layers 42, and an output layer 43. The input layer 41 includes three input neurons 51, 52, 53, each of which corresponds to three audio feature values (i.e., VSj, TDj, TGj) of each frame of the feature vector. The intermediate layer 42 includes neurons 61-74, each of which has a weighting factor associated with each input neuron 51, 52, 53 and a biasing factor for each neuron. By changing the weighting factor and biasing factor for each neuron 61-74 of the intermediate layer 42 through cycles of a learning phase, the neural network can be trained to report a prediction value for a given type of input. The output layer 43 includes an output neuron 81 that provides a prediction value corresponding to a phoneme (specifically, indicating whether it is a speech period or a noisy non-speech period).

As described above, in the noise cancellation units 13 and 24, the deep learning model unit 35 learns using the characteristic parameters of human voice and determines whether or not the input voice signal is a non-voice period containing noise. Then, based on the determination by the deep learning model unit 35, the CPU 31 of the voice signal processing unit 30 performs noise cancellation processing so as not to pass non-voice periods containing noise from the input voice signal, and outputs the voice signal after the noise cancellation processing. Here, the deep learning model unit 35 is configured by the neural network of FIG. 3, which receives the characteristic parameters of human voice as input and outputs the determination result of whether or not the input voice signal is a non-voice period containing noise.

As described above, in this embodiment, the radio 1 has a noise cancellation unit 24 in its transmission section, which is particularly effective in reducing noise, for example, in FM (frequency modulation) demodulation or PM (phase modulation) demodulation, instead of a phase inversion type noise cancellation circuit. This allows a wireless communication device that performs wireless calls to perform noise cancellation so as to more effectively output human speech than conventional technology. By providing the noise cancellation unit 24 on the transmission side, noise can be effectively removed from audio signals in circuits and devices (e.g., wireless relay devices, etc.) subsequent to the transmission side.

In addition, in this embodiment, the radio 1 further includes a noise cancellation unit 13 in the receiving section, which is particularly effective in reducing noise. As a result, in a wireless communication device that performs wireless communication, noise cancellation can be performed to effectively output human speech compared to conventional technology. When FM or PM is used as the modulation method, the demodulated audio signal is passed through the noise cancellation unit 13 before being output to the speaker 15, eliminating white noise when approaching no signal, and the received audio continues to maintain a clear voice even when approaching the communication limit distance. In addition, by using the noise cancellation unit 13 that uses the deep learning model unit 35 to cancel noise instead of the conventional noise squelch circuit, the output of noise in a no-signal state is stopped (or reduced), and the output of the audio signal that can be received is not stopped. Therefore, the audio signal received up to the limit of the communication distance can be output, making it possible to extend the communication distance compared to the noise squelch circuit.

(Modification)
In the above embodiment, the wireless device 1 includes the noise canceling unit 24. However, the present invention is not limited to this, and the wireless device 1 does not necessarily need to include the noise canceling unit 24.

As described above in detail, according to the wireless communication device of the present invention, instead of a phase inversion type noise cancellation circuit, noise cancellation is performed using a noise cancellation unit 13 that utilizes a deep learning model unit 35 on the transmitting side of the radio 1, or even on the receiving side, thereby making it possible to perform noise cancellation so as to effectively output human speech sounds in a wireless communication device that performs wireless calls, compared to conventional technology.

REFERENCE SIGNS LIST 1 Radio 11 Receiving antenna 12 Receiving and demodulating section 13 Noise canceling section 14 Audio signal amplifier 15 Speaker 20 Control section 21 Operation section 21A PTT key 22 Microphone 23 Audio signal amplifier 24 Noise canceling section 25 Modulation transmitting section 26 Transmitting antenna 30 Audio signal processing section 31 CPU
32 ROM
33 RAM
34 EEPROM
35 Deep learning model unit 36 Input interface 37 Output interface 38 Audio signal pre-processing unit 39 AD converter 41 Input layer 42 Intermediate layer 43 Output layer 51 to 81 Neurons

Claims (13)

A wireless communication device including a modulation transmission unit that modulates a wireless carrier wave in accordance with an input audio signal and transmits a wireless signal,
a first noise canceling unit that performs audio signal processing so as to cancel noise from the input audio signal and outputs the processed audio signal to the modulation transmitting unit;
The first noise cancellation unit is a wireless communication device that performs noise cancellation processing using a deep learning model unit that is trained using characteristic parameters of human voice, which are vibration frequencies calculated for each predetermined number of frames along the time axis in the input audio signal, and determines whether the input audio signal is a non-voice period containing noise based on a vibration pattern that represents a data distribution of the sum of the vibration frequencies. The wireless communication device according to claim 1, wherein the modulation transmission unit modulates a wireless carrier wave according to an input audio signal using a frequency modulation method or a phase modulation method. The first noise cancellation unit includes an audio signal processing unit that performs a noise cancellation process based on the determination of the deep learning model unit so as not to pass a non-voice period including noise from the input audio signal, and outputs the audio signal after the noise cancellation process.
3. A wireless communication device according to claim 1 or 2. The first noise cancellation unit includes:
4. The wireless communication device according to claim 3, further comprising an audio signal pre-processing unit provided in front of the audio signal processing unit, which passes only the input audio signal within a predetermined level range of human audio signals and a predetermined bandwidth. The wireless communication device according to any one of claims 1 to 4, wherein the deep learning model unit is configured with a predetermined neural network that receives human voice characteristic parameters as input and outputs a determination result of whether the input voice signal is a non-voice period that includes noise. The wireless communication device includes:
a receiving and demodulating unit that demodulates a received wireless signal into an audio signal ;
a second noise canceling unit that performs audio signal processing so as to cancel noise from the demodulated audio signal and outputs the audio signal after the audio signal processing ,
The wireless communication device according to claim 1 or 2, wherein the second noise cancellation unit performs noise cancellation processing using a deep learning model unit that is trained using characteristic parameters of human voice and determines whether the demodulated voice signal is a non-voice period containing noise. The wireless communication device according to claim 6, wherein the receiving and demodulating unit demodulates the received wireless signal using a frequency modulation method or a phase modulation method. The second noise cancellation unit includes an audio signal processing unit that performs a noise cancellation process based on the determination of the deep learning model unit so as to pass the audio signal only during an audio signal period from the demodulated audio signal, and outputs the audio signal after the noise cancellation process.
8. A wireless communication device according to claim 6 or 7. The second noise cancellation unit includes:
The wireless communication device according to claim 8, further comprising an audio signal pre-processing unit provided in front of the audio signal processing unit, which passes only the input audio signal within a predetermined level range of human audio signals and a predetermined bandwidth. The wireless communication device according to any one of claims 6 to 9, wherein the deep learning model unit is configured with a predetermined neural network that receives human voice characteristic parameters as input and outputs a determination result of whether the input voice signal is a non-voice period that includes noise. A wireless communication device that does not simultaneously perform transmission and reception operations, and that operates the first noise cancellation unit instead of the second noise cancellation unit during reception, according to any one of claims 6 to 10. The wireless communication device according to any one of claims 1 to 11, wherein the wireless communication device is a specific low-power radio station that is a wireless communication device for a specific low-power wireless communication system. A wireless communication system including a plurality of wireless communication devices according to any one of claims 1 to 12.

Images