JP6612274B2 - Voice aggregation device, voice division device, method and program thereof - Google Patents

Description

  The present invention relates to a method for easily transmitting / receiving multi-channel audio and accompanying information within a limited frequency band.

  For example, consider a case where voices of individual speakers are collected in a conference room and voice recognition is performed while holding speaker information. As a sound collection method, for example, it is conceivable to collect sound by using a headset, a gooseneck microphone, or the like for each speaker, or by using a multi-channel microphone to collect sound. By combining this with voice recognition, it is possible to process the voices of multiple speakers at the same time without greatly changing the operation of voice applications that have been used in a single channel, and it is possible to improve convenience. . Currently, the following three possible voice acquisition means can be considered to realize the above.

Example 1: Multichannel audio interface By using a commercially available audio interface such as a USB connection, a general personal computer (hereinafter also referred to as a PC) can acquire “multichannel audio” of two or more channels. . However, at present, it is often necessary to install a “device driver” when using such a device. The user needs to follow the procedure to obtain and install this device driver. This device driver needs to be prepared for each operation system (hereinafter also referred to as OS), and there is a demerit that the device development side must deal with each time the OS specification or version changes.

Example 2: General-purpose audio interface Generally, a PC or the like is equipped with an analog microphone terminal or line input terminal and a USB terminal. By simply connecting to these terminals, the device can operate as a microphone and use the voice input function. can do. However, the number of channels of any terminal is limited to one or two channels, and it is not possible to acquire audio of more channels unless specially supported on the PC side.

Example 3: General-purpose data interface
A general-purpose interface such as a USB terminal can be used as a data communication (data interface) in addition to the “audio interface” as in Example 2. In this case, the number of channels in Example 2 is not limited, and multi-channel data can be transmitted without preparing a device driver as in Example 1. However, since an input / output that is not an audio interface (not real time) is used, the input cannot be recognized from the application side on the premise of voice input, and the application must be significantly modified.

  In addition to the emphasized audio signal, the multi-channel microphone has information related to the audio obtained in the analysis process, but such information cannot be used in a device having only an audio input channel.

  Here, from the viewpoint of user convenience, it is assumed that audio of two or more channels is output under the condition of using the general audio interface of Example 2.

  A method of superimposing a plurality of audio signals on one band is conceivable. As an example, there is a method called frequency division multiplexing (see Non-Patent Document 1).

R. Chang, R. Gibby, "A Theoretical Study of Performance of an Orthogonal Multiplexing Data Transmission Scheme", IEEE Transactions on Communication Technology, 1968.

  However, according to the method of Non-Patent Document 1, for example, when the bandwidth that can be processed by the voice input function on the PC side is 48 kHz, when trying to superimpose a 16 kHz sampling audio signal, a maximum of 3 speakers per channel However, it is necessary to fix the relationship between the bandwidth used and the speaker. Therefore, it cannot be used for conversations of four or more speakers. In addition, when a filter for filtering each audio signal from the multiplexed signal cannot be sharply formed, when an audio signal in a certain band is extracted from the multiplexed signal, an audio signal in a band adjacent to the band is extracted. There is a problem that the ingredients remain. For example, when an audio signal in the lower band (0 to 8 kHz band) is extracted from the multiplexed signal in the 48 kHz band by filtering, the audio signal component in the band adjacent to that band (8 to 16 kHz band) is the boundary. In the vicinity (8kHz) can not be eliminated and remains.

  The present invention uses a 1-channel or 2-channel audio input available in a general OS to transmit a 2-channel or 4-channel audio signal and information accompanying the audio signal, thereby enabling more than four people It is an object of the present invention to provide a voice aggregating apparatus that can also be used for a speaker's conversation, a voice dividing apparatus that extracts a two-channel or four-channel voice signal from signals multiplexed by the voice aggregating apparatus, a method, and a program thereof.

In order to solve the above-described problem, according to one aspect of the present invention, the voice aggregating apparatus is configured such that M and N are each an integer of 2 or more, i = 1, 2,. A speech analysis unit that obtains M speech signals y _i (n) and speech information q _i (n), which is information related to each speech signal y _i (n), using signals collected by the microphones of audio information q _i (n), the output audio signal y utterance state of _i (n) v _i (n), the speech signal y _i number of microphones for outputting (n) or speech signal y _i (n) Information θ _i (n) corresponding to the beam to be transmitted and the speaker number p _i (n) of the audio signal y _i (n), g ≠ h, and included in the audio information q _i (n) Utterance order v _i (n) is determined, voice information q ′ _g (n) corresponding to the g-th utterance s _g , and voice information q ′ _h (n corresponding to the h-th utterance s _h ) and the audio information control unit for obtaining a speech signal z _g corresponding to the utterance s _g (n), to respond to the utterance s _h A voice signal controller for obtaining an audio signal z _h (n), the speech signal z _g (n) and the audio signal z _h sampling frequency (n) and UHz, audio signal z _g (n) and the audio signal z _h A sampling frequency conversion unit that performs upsampling 3 times with respect to (n) and obtains a sampling frequency 3 UHz of 3 times the sampling frequency u _g (n), u _h (n), and an audio signal u _g (n) And a modulation unit that multiplies the component of the sign inversion and modulates the audio signal t _h (n), information on the microphone number or direction included in the audio information q ′ _g (n), θ ′ _g (n), and the speech. bit string indicating 'and _g (n), the audio information q' number p of users and _h number or direction information of the microphone included in the _{(n) θ 'h (n} ) and the speaker number p' _h (n) And a signal conversion unit for obtaining a signal D (n) obtained by converting the signal into a speech waveform, and the speech waveform is generated so as to have a component corresponding to a bit string in a band from UHz to 2 UHz, and the speech signal u _g (n) And audio signal a signal adder that adds t _h (n) and the signal D (n) to obtain the output signal z (n).

In order to solve the above-described problem, according to another aspect of the present invention, the audio dividing device sets g ≠ h, and the input signal z (n) includes the audio signal u _g (n) and the audio signal t _h ( n) and the signal D (n) are added to each other, and the audio signal u _g (n) is a signal obtained by upsampling three times the audio signal z _g (n). The signal t _h (n) is a signal obtained by multiplying u _h (n) obtained by upsampling the audio signal z _h (n) three times by the sign inversion component, and the signal D (n) Is the microphone number or direction information θ ′ _g (n) and the speaker number p ′ _g (n) included in the audio information q ′ _g (n) corresponding to the audio signal z _g (n), and the audio signal A bit string B indicating the microphone number or direction information θ ′ _h (n) and the speaker number p ′ _h (n) included in the voice information q ′ _h (n) corresponding to z _h (n) is spoken. This is a signal converted into a waveform, and the audio waveform is a band from UHz to 2UHz. A low-pass filter that filters the band components below UHz of the input signal z (n) and the band components above 2 UHz of the input signal z (n). A high-pass filter unit, a signal obtained as a result of filtering by the high-pass filter unit, a modulation unit that multiplies the component of the sign inversion and modulates the audio signal u ′ _h (n), and an input signal z (n) An information decoding unit that decodes a band component of UHz to 2 UHz to obtain a bit string corresponding to bit string B.

  According to the present invention, it is possible to collect voice signals of conversations of four or more speakers.

The functional block diagram of the audio | voice aggregation apparatus which concerns on 1st embodiment. The figure which shows the example of the processing flow of the audio | voice aggregation apparatus which concerns on 1st embodiment. The figure for demonstrating the process of a modulation | alteration part. The figure for demonstrating the process of a code conversion part. The figure for demonstrating the process of a code conversion part. The functional block diagram of the audio | voice division | segmentation apparatus which concerns on 2nd embodiment. The figure which shows the example of the processing flow of the audio | voice division | segmentation apparatus which concerns on 2nd embodiment. The figure for demonstrating the process of an information decoding part. The functional block diagram of the audio | voice aggregation apparatus which concerns on 3rd embodiment. The functional block diagram of the audio | voice aggregation apparatus which concerns on 4th embodiment. The figure which shows the example of the processing flow of the audio | voice aggregation apparatus which concerns on 4th embodiment. The functional block diagram of the audio | voice division | segmentation apparatus which concerns on 4th embodiment. The figure which shows the example of the processing flow of the audio | voice division | segmentation apparatus which concerns on 4th embodiment.

  Hereinafter, embodiments of the present invention will be described. In the drawings used for the following description, constituent parts having the same function and steps for performing the same process are denoted by the same reference numerals, and redundant description is omitted. In the following description, it is assumed that processing performed for each element of a vector or matrix is applied to all elements of the vector or matrix unless otherwise specified.

<Points of first embodiment>
In this embodiment, one band is divided into three, the low band and the high band are audio signals, and the middle band is information accompanying the audio signal (hereinafter also referred to as `` audio information ''). Used to send. For example, in the case of 48 kHz sampling, the 0 to 8 kHz band is used for transmitting the first speaker voice, the 16 to 24 kHz band for the second speaker voice, and the 8 to 16 kHz band for transmitting voice information. As a result, voice signals and voice information (speaker number, utterance position, etc.) can be transmitted in pairs using only signals in the voice band, so up to two people (4 people or less when using 2ch) at the same moment With simultaneous speech, information on the combination of “speech” and “speaker” can be transmitted without loss, no matter how many people are participating in the conversation. In addition, by arranging the audio signal bands apart and arranging audio information with relatively small amplitude in the central band, the influence on the audio signal is reduced even if the audio signal is separated by a non-steep filter. . Furthermore, even if a terminal that does not have the function of restoring multiplexing according to the present embodiment is connected, it is possible to transmit 0 to 8 kHz audio information, and thus compatibility with low-frequency audio.

  With such a configuration, a plurality of audio information can be transmitted to a device having a limited interface.

<First embodiment>
FIG. 1 is a functional block diagram of the voice aggregation device 100 according to the first embodiment, and FIG. 2 shows an example of the processing flow.

  The voice aggregating apparatus 100 includes a CPU, a RAM, and a computer including a ROM that stores a program for executing the following processing, and is functionally configured as follows. The voice aggregating apparatus 100 includes a voice analysis unit 102, a voice signal control unit 103, a voice information control unit 104, sampling period conversion units 105 and 106, a modulation unit 107, a code conversion unit 108, and a signal addition unit 109. Including.

The voice aggregating apparatus 100 has N voice signals x _i (n) picked up by N microphones 101-i as inputs, and two voice signals included in the N voice signals x _i (n) and A signal z (n) on which the audio information is superimposed is output. Here, i = 1, 2,..., N and N are any integers of 2 or more, and n is an index of sampled time.

The microphone 101-i collects sound (S101), converts it into an audio signal x _i (n), and outputs it. This embodiment includes an AD conversion function for converting an analog signal into a digital signal. For the sampling frequency at the time of sound collection, for example, 16 kHz is selected as a band in which the audio signal is generally retained. An AD conversion function may be provided on the voice aggregation device 100 side.

<Speech analysis unit 102>
The speech analysis unit 102 receives N speech signals x _i (n) as input, and uses these signals to provide information on M speech signals y _i (n) and each speech signal y _i (n). The voice information q _i (n) is obtained (S102) and output. Incidentally, the audio information q _i (n) is a speech signal y utterance state of _i (n) v _i (n), the speech signal y _i number of microphones for outputting (n) or speech signal y _i (n) of direction information corresponding to the output beams (hereinafter, the "output corresponding to a direction of the information beam voice signals y _i (n) number or audio signal of the microphone for outputting y _i (n)", "direction information" Θ _i (n) and the speaker number of the audio signal y _i (n) (hereinafter “speaker number” is also referred to as “speaker information”) p _i (n). Note that M is an integer of 2 or more.

(i) When the microphone 101-i is a headset prepared for each speaker, N = M, x _i (n) = y _i (n), θ _i (n) = i, p _i (n ) = i. In this case, the direction information θ _i (n) and the speaker information p _i (n) match the microphone number.

(ii) When N microphones 101-i are microphone arrays and the beam forming technique (see Reference 1) is used, it is assumed that there is at most one speaker in each beam direction.
(Reference 1) Tadashi Asano, “Sound Array Signal Processing”, Corona, 2011.

The speech analysis unit 102 uses the N speech signals x _i (n) to obtain M speech signals y _i (n) respectively corresponding to the M beams by a beamforming technique. However, in the subsequent processes, i = 1, 2,. M and N may be different values or the same value. Information on the direction corresponding to the beam from which the audio signal y _i (n) is output is θ _i (n) and the speaker information p _i (n) of the audio signal y _i (n). For example, the speaker number p _i (n) may be specified based on the direction information θ _i (n). For example, the direction information θ _i (n) takes the value of angle information 0 to 360 set in advance for each beam, and when the value of θ _i (n) is 0 to 30, p _i (n) ← 1 and when the value of θ _i (n) is 30 to 60, set p _i (n) ← 2, and set as follows. Based on the direction information θ _i (n), the speaker number p _i (n ) May be specified.

Furthermore, the speech analysis unit 102, using the M speech signal y _i (n), obtains the utterance state v _i (n) of the speech signal y _i (n). For example, when the signal power of the audio signal y _i (n) is equal to or greater than a predetermined threshold, it is determined that the speech state is present, and a value indicating the speech state is substituted for v _i (n) (for example, v _i (n) ← 1). In other cases (when the signal power of the audio signal y _i (n) is less than a predetermined threshold value), it is determined that the voice state is not in the utterance state, and a value indicating that it is not in the utterance state is substituted into v _i (n) ( For example, v _i (n) ← 0). Note that any existing method may be used as the method for calculating the utterance state v _i (n), and an optimal method may be selected as appropriate in accordance with the usage environment.

The voice analysis unit 102 outputs M pieces of voice information q _i (n) = (v _i (n), θ _i (n), p _i (n)) to the voice information control unit 104 and the voice signal control unit 103. Then, M audio signals y _i (n) are output to the audio signal control unit 103.

<Audio information control unit 104>
The voice information control unit 104 receives M pieces of voice information q _i (n) as input, obtains the utterance order based on the utterance state v _i (n) included in the voice information q _i (n), and determines the first utterance ( Speech uttered at the nearest time from time n) Voice information q ' ₁ (n) corresponding to s ₁ and the second utterance (utterance uttered at the second nearest time from time n) The speech information q ′ ₂ (n) corresponding to s ₂ is obtained (S104), and the utterances s ₁ and s ₂ are output to the speech signal control unit 103, and the speech information q ′ ₁ (n) and q ′ ₂ (n ) Is output to the code converter 108. Note that s ₁ and s ₂ are not utterances themselves but indexes indicating utterances.

For example, utterances s ₁ and s ₂ are obtained as follows.

First, the value of the utterance order array s _j (n) is initialized. For example, s _j (n) ← 0. However, the total number of elements of the utterance order array s _j (n) is J, J is an integer of 2 to M, and j = 1, 2,.

When there is i where v _i (n) = 1 and v _i (n−1) = 0, it is considered that there is an utterance at time n with the i-th audio signal y _i (n). At this time
s ₁ (n) ← θ _i (n)
s _j (n) ← s _j-1 (n-1) (j = 2,3, ..., J)
The direction information θ _i (n) of the most recently uttered voice is reflected in s ₁ (n), and j = 2, 3, ... J are overwritten with different values one by one, and the oldest information ( s _J-1 (n-1)) is deleted. At the same time, when the two utterance states v _i (n) satisfy the conditions, the processes may be performed in a predetermined order. For example, processing is performed in order from the smallest i.

The voice information control unit 104 converts θ _i (n) and v ′ ₁ (n), v ′ ₂ (n), p ′ ₁ (n), and p ′ ₂ (n) corresponding to the utterances s ₁ and s _2. Output. When s ₁ (n) = θ _i (n), v ′ ₁ (n) = v _i (n) and p ′ ₁ (n) = p _i (n). In other words, s ₁ ← θ _i (n) and s ₂ (n) ← s ₁ (n-1) when there is i where v _i (n) = 1, v _i (n-1) = 0, Output v _i (n) and p _i (n) corresponding to s ₁ and s ₂ .

<Audio signal control unit 103>
The voice signal control unit 103 receives M pieces of voice information q _i (n), M pieces of voice signals y _i (n), and two utterances (indexes) s ₁ and s ₂ , and receives M pieces of voice information. From the signals y _i (n), speech signals z ₁ (n) and z ₂ (n) corresponding to the utterances s ₁ and s ₂ are obtained (S103) and output. For example, the audio signal y _i (n) and the direction information θ _i (n) are associated with each other, and the audio signal y _i (n) corresponding to the direction information θ _i (n) matching the utterances s ₁ and s ₂ is associated. ) As audio signals z ₁ (n) and z ₂ (n).

<Sampling period converters 105 and 106>
Sampling period converters 105 and 106 receive audio signals z ₁ (n) and z ₂ (n), respectively, and perform upsampling three times with respect to audio signals z ₁ (n) and z ₂ (n), respectively. The audio signals u ₁ (n) and u ₂ (n) having a sampling frequency of 3 times 48 kHz are obtained (S105, S106) and output. For example, 2 samples are zero-padded for each signal sample to generate a signal whose bandwidth is tripled, and a band of 8 kHz or higher is cut by a low-pass filter. In this example, since the sampling frequency of the original audio signal x _i (n) is 16 kHz and the band to be sampled is 8 kHz, the band above 8 kHz is cut by a low-pass filter to remove the distortion component. As a result, audio signals u ₁ (n) and u ₂ (n) with a sampling frequency of 48 kHz (24 kHz band) whose band is limited to 8 kHz are output.

<Modulation unit 107>
The modulation unit 107 receives the audio signal u ₂ (n) as input, multiplies the audio signal u ₂ (n) by a sign inversion component as shown in the following equation, and modulates the audio signal t ₂ (n) ( S107) and output.
t ₂ (n) = u ₂ (n) * (-1) ⁿ
Thereby, the audio signal u ₂ (n) is converted into an audio signal t ₂ (n) having a component in a band of 16 to 24 kHz (see FIG. 3).

<Code conversion unit 108>
The code conversion unit 108 includes direction information θ ′ ₁ (n) and speaker information p ′ ₁ (n) included in the voice information q ′ ₁ (n), and directions included in the voice information q ′ ₂ (n). A signal D (n) obtained by converting the bit string B indicating the information θ ′ ₂ (n) and the speaker information p ′ ₂ (n) into a speech waveform is obtained (S108) and output. Note that the speech waveform is generated so as to have a component corresponding to the bit string B in a band of 8 kHz or more and 16 kHz or less.

The purpose of the code converter 108 is to place the information θ ′ ₁ (n), p ′ ₁ (n), θ ′ ₂ (n), p ′ ₂ (n) received from the audio information controller 104 on the audio signal. Is to transmit. Consider that information is superimposed on 8 to 16 kHz where the audio signals u ₁ (n) and t ₂ (n) do not exist. At this time, since the voice component that cannot be blocked by the low-pass filter appears near the band boundary, the voice information avoids the voice band boundary and has a margin of f _{ml on} the low frequency side and f _mh [Hz] on the high frequency side. The provided area is used (see FIG. 4). For example, 8400 to 15600 Hz is used with f _ml = f _mh = 400 Hz. Note that since the voice speaker, position information, and the like do not change rapidly with time, it is sufficient to transmit at a low bit rate. Therefore, priority is given to less error and less influence on the voice.

FIG. 5 shows an example of the operation flow of the code conversion unit 108. For example, among the information received from the voice information control unit 104, 6 bits for transmitting direction information θ ′ ₁ (n) and θ ′ ₂ (n), speaker information p ′ ₁ (n), p ′ ₂ ( Each 4 bits shall be used to transmit n). Furthermore, 10 bits of this information and the least significant bit of the sum of 10 bits as a checksum (parity) are made into a 1-bit 11-bit bit string. Since there are two voices, A1 and A2 are used, and a bit string B = (A1, A2). Bit string B is divided and transmitted at several times. When audio is frame-processed, information is sent over several frames. For example, as shown in FIG. 4, consider a band division of W [Hz] intervals by a sine wave. This W is set to an interval such that main lobes of adjacent sine wave components can be sufficiently distinguished, for example, W = 800 [Hz]. For two speakers, the above-mentioned use band (8400 to 15600 Hz) is further divided in half and used for transmission of the bit string B = (A1, A2). The presence or absence of information is represented by the presence or absence of this sine wave component. For example, when the bit value is 1, there is a sine wave component, and when the bit value is 0, there is no sine wave component. Further, the transmission waveform is windowed in order to suppress the occurrence of abnormal noise as a time waveform. That is

When two sine waves are represented, the signal waveform transmitted every L samples is

It can be expressed. Here, L is the frame length of the frame processing, l = 1, 2,..., L, A ₁ (k) and A ₂ (k) are the k-th bit values (0 or 1) from the beginning of the bit strings A1 and A2, respectively. ), C is a sinusoidal amplitude, and w (n) is a window function of length L (for example, a Hamming window). In this example, K = 4.

The code conversion unit 108 outputs D (n) calculated as D (n) = D ₁ (n) + D ₂ (n) (S108-7). Further, a bit shift of A1 (k) = A1 (k + K) is performed (S108-9). Set to 0 if the end of the bit string is reached and A1 (k + K) is not defined.

  However, when the receiving side receives this encoded sine wave, it cannot determine where the information of the frame unit starts. Therefore, at the initial timing of transmission of bit string B, a 12 kHz band (11.6 to 12.4 Khz) that is not used for transmission of bit string B is used.

The reference signal is output. In addition, there are other methods for expressing the start position of the information, such as adding a special arrangement pattern to the bit string.

  A processing example of the code conversion unit 108 will be described with reference to FIG.

When processing starts, first, the counter is initialized (S108-1, count ← 0).
Next, when the value of the counter is 0 (S108-2), bit sequences A1 and A2 are created from the received audio information q ′ ₁ (n), q ′ ₂ (n) (S108-3). Before transmitting the speech waveform corresponding to the bit strings A1 and A2, the speech waveform corresponding to the reference signal is generated and transmitted by the above-described method (S108-4).

  The counter is incremented (S108-5, count ← count + 1).

  When the value of the counter is other than 0 (S108-2), K bits are extracted from the heads of the bit strings A1 and A2, and a speech waveform corresponding to the bit strings A1 and A2 is generated and transmitted by the above method (S108-7).

  If all of the bit strings A1 and A2 have not been extracted (S108-8), the heads of the bit strings A1 and A2 are shifted by K bits (S108-9), and the counter is incremented (S108-5, count ← count + 1) .

  When all of the bit strings A1 and A2 are extracted (S108-8), the counter value is returned to the initial value (S108-10).

  The processes S108-2 to S108-5 and S108-7 to S108-10 are repeated until all the audio information is processed (S108-6).

<Signal adder 109>
The signal adding unit 109 receives the audio signal u ₁ (n), the audio signal t ₂ (n), and the signal D (n), and adds these values to obtain an output signal z (n) (S109). Output.
z (n) = u ₁ (n) + t ₂ (n) + D (n)

<Effect>
With such a configuration, two channels of audio signals can be sent simultaneously, and voice information (direction information, speaker information, etc.) of the audio signals of two channels can be sent together to participate in the conversation. You can tell who is the voice of more than 4 speakers. At this time, a conversation of four or more speakers can be transmitted using a one-channel voice input that can be used in a general OS. In an actual conversation, even if the number of conversation participants is 4 or more, the number of people who speak at the same time is not so large. The device can be used. Of course, this embodiment can also be applied to conversations of three or less people.

<Modification>
When the speech signal v ₁ (n) is v ′ ₁ (n) = 0 when the speech signal controller 103 of the first embodiment outputs the speech signal z ₁ (n), the speech signal z ₁ (n ) Instead of 0, and when the speech signal z ₂ (n) is output, if the speech state v ′ ₂ (n) is v ′ ₂ (n) = 0, it is replaced with the speech signal z ₂ (n). Output 0. That is, the sound from the direction is output only when the utterance is being made.

In this embodiment, the audio signal u ₁ (n) corresponding to the _first utterance s ₁ and the audio signal t ₂ (n) corresponding to the _second utterance s ₂ are superimposed, but not necessarily the first and second It is not necessary to superimpose audio signals corresponding to the utterances s ₁ and s ₂ . Audio signals corresponding to other utterances may be superimposed. However, the voice signal u ₁ (n) corresponding to the _first utterance s ₁ and the voice signal t ₂ (n) corresponding to the _second utterance s ₂ should be superimposed to transmit the voice of the current conversation. It is easy to understand the whole conversation.

  In this embodiment, the sampling frequency at the time of sound collection is 16 kHz, and the bandwidth that can be processed by the voice input function on the PC side is 48 kHz, but may be appropriately changed according to the microphone and the voice input function on the PC side. .

<Second embodiment>
In the present embodiment, an audio dividing device 500 that receives an output signal z (n) will be described. In the present embodiment, the output signal z (n) is referred to as the input signal z (n).

  FIG. 6 is a functional block diagram of the audio dividing device 500 according to the second embodiment, and FIG. 7 shows an example of the processing flow.

  The audio dividing device 500 is configured by a computer including a CPU, a RAM, and a ROM that stores a program for executing the following processing, and is functionally configured as follows. Audio splitting apparatus 500 includes a receiving unit 508, a low-pass filter unit 504, a sampling cycle conversion unit 501, a high-pass filter unit 505, a modulation unit 503, a sampling cycle conversion unit 502, and an information decoding unit 506.

The audio dividing device 500 receives an input signal z (n) as an input, two audio signals z ′ ₁ (n), z ′ ₂ (n) and respective audio information q ′ _{D, 1} (n), q ′. _{D, 2} (n) is output. However, the voice information q ′ _{D, r} (n) is information θ ′ _D corresponding to the direction information θ ′ _r (n) and the speaker information p ′ _r (n) included in the voice information q ′ _r (n). _{, r} (n), p ′ _{D, r} (n). However, r = 1,2. For example, a voice recognition device may be provided at the subsequent stage of the voice division device 500. In that case, the speech recognition apparatus may perform speech recognition processing on the _two speech signals z ′ ₁ (n) and z ′ ₂ (n) and output the speech recognition result and the speech information together. Further, for example, a playback device may be provided in the subsequent stage, and the audio signal may be played back from the direction indicated by the direction information included in the audio information. In short, the audio signals z ' ₁ (n), z' ₂ (n) and the audio information q ' _{D, 1} (n), q' _{D, 2} (n) are used as appropriate according to the processing content of the subsequent stage. That's fine.

<Receiving unit 508>
The receiving unit 508 is a general-purpose audio interface, and includes an analog microphone terminal, a line input terminal, a USB terminal, and the like. One-channel or two-channel audio signals can be received.

  The receiving unit 508 receives the input signal z (n) as an input and outputs the input signal z (n) to the low-pass filter unit 504, the high-pass filter unit 505, and the information decoding unit 506.

<Low-pass filter unit 504>
The low-pass filter unit 504 receives the input signal z (n) as an input, filters a band component of 8 kHz or less of the input signal z (n) (S504), and outputs an audio signal u ′ ₁ (n) obtained as a result of the filtering. To do. Audio signal u _'1 (n) is a signal corresponding to the audio signal u ₁ (n) of the first embodiment.

<High-pass filter unit 505>
The high-pass filter unit 505 receives the input signal z (n) as an input, filters a band component of 16 kHz or more of the input signal z (n) (S505), and outputs an audio signal t ′ ₂ (n) obtained as a result of the filtering. To do. Audio signals t _'2 (n) is a signal corresponding to the audio signal t ₂ (n) of the first embodiment.

<Modulation unit 503>
The modulation unit 503 receives the audio signal t ′ ₂ (n) as input, multiplies the audio signal t ′ ₂ (n) by a sign-inverted component as shown in the following equation, and modulates the audio signal u ′ ₂ (n). (S503) and output.
u ' ₂ (n) = t' ₂ (n) * (-1) ⁿ
Incidentally, the audio signal u _'2 (n) is a signal corresponding to the audio signal u ₂ (n) of the first embodiment.

<Sampling cycle conversion unit 501 and sampling cycle conversion unit 502>
The sampling cycle conversion unit 501 and the sampling cycle conversion unit 502 receive the audio signal u ′ ₁ (n) and the audio signal u ′ ₂ (n), respectively, and the audio signal u ′ ₁ (n) and the audio signal u ′ _{2, respectively.} Downsampling is performed 1/3 times (n), and audio signals z ′ ₁ (n) and z ′ ₂ (n) having a sampling frequency 16 kHz of 1/3 are obtained (S501, S502) and output. . The audio signals z ′ ₁ (n) and z ′ ₂ (n) are signals corresponding to the audio signals z ₁ (n) and z ₂ (n) of the first embodiment, respectively.

<Information decoding unit 506>
The information decoding unit 506 receives the input signal z (n) as an input, and a band component of 8 kHz to 16 kHz of the input signal z (n) (hereinafter, this band component is also referred to as “signal D ′ (n)”. '(n) is a signal corresponding to the signal D (n) of the first embodiment) to obtain a bit string B' corresponding to the bit string B (S506) and output it. FIG. 8 shows an example of the processing flow of the information decoding unit 506.

  First, the signal extraction timing at which the frequency component of the reference signal is equal to or greater than the threshold is detected from the signal D ′ (n) (S506-1).

  When the threshold value is exceeded, the signal D ′ (n) for one frame is acquired (S506-2).

  When the frame is the frame of the reference signal (S506-3), the signal D ′ (n) for one frame is further acquired (S506-2).

  If the frame is not a reference signal frame (S506-3), the presence / absence of frequency components of each information is checked (S506-4).

  If bit strings A1 and A2 remain (S506-5), steps S506-2 to S506-4 are repeated.

When all the bit strings A1 and A2 are extracted (S506-5), the bit strings A1 and A2 are determined (S506-6), and the bit string B ′ = (q ′ _{D, 1} (n), q ′ _{D, 2} (n)) is obtained and output.

  The processes S506-1 to S506-7 are repeated until the process is completed for all the signals D ′ (n).

<Effect>
With such a configuration, it is possible to simultaneously receive voice signals for two channels using one-channel voice input, specify the direction of the voice signal and the speaker, and receive the conversation of four or more speakers. In this embodiment, the down-sampled audio signals z ′ ₁ (n) and z ′ ₂ (n) are output, but the audio signals u ′ ₁ (n) and the audio signals u ′ ₂ (n) are output. You may output as it is. In that case, the audio dividing device 500 may not include the sampling period conversion unit 501 and the sampling period conversion unit 502.

<Third embodiment>
A description will be given centering on differences from the first embodiment.

  In the present embodiment, a case where the output signal of the audio aggregating apparatus 600 is 2 channels (stereo channels) will be described.

  FIG. 9 shows a functional block diagram of the voice aggregation device 600 according to the third embodiment.

  The voice aggregating apparatus 600 includes, in addition to the configuration of the voice aggregating apparatus 100, sampling period converters 105-B and 106-B, a modulator 107-B, a code converter 108-B, and a signal adder 109-B. including.

<Audio information control unit 104>
The voice information control unit 104 receives M pieces of voice information q _i (n), obtains the utterance order based on the utterance state v _i (n) included in the voice information q _i (n), and is first to fourth Utterances (utterances uttered at the time closest to the first to fourth times from time n) s _{1 to} s ₄ corresponding to speech information q ′ ₁ (n) to q ′ ₄ (n) outputs ₁ ~s ₄ to the audio signal controller 103, the audio information _{q '1 (n), q} ' code conversion unit 108 ₃ (n), audio information _{q '2 (n), q} ' 4 (n ) Is output to the code conversion unit 108-B.

<Audio signal control unit 103>
The audio signal control unit 103 receives M audio information q _i (n), M audio signals y _i (n), and 4 utterances (indexes) s _{1 to} s ₄ , and receives M audio information Speech signals z ₁ (n) to z ₄ (n) corresponding to the utterances s _{1 to} s ₄ are obtained from the signals y _i (n), and the speech signals z ₁ (n) and z ₃ (n) are obtained. Output to one output channel (eg, L channel), and output audio signals z ₂ (n), z ₄ (n) to the other output channel (eg, R channel).

On one output channel (for example, L channel) side, that is, the sampling period conversion units 105 and 106, the modulation unit 107, the code conversion unit 108, and the signal addition unit 109, the audio signal z ₂ (n), the audio information q Except for using audio signal z ₃ (n) and audio information q ' ₃ (n) instead of' ₂ (n), the same processing as in the first embodiment is performed and output signal z _1ch (n) is output To do.

On the other output channel (for example, R channel) side, that is, the sampling period conversion units 105-B and 106-B, the modulation unit 107-B, the code conversion unit 108-B, and the signal addition unit 109-B, Signals z ₁ (n), z ₂ (n), audio information q ′ ₁ (n), q ′ ₂ (n) instead of audio signals z ₂ (n), z ₄ (n), audio Except for using the information q ′ ₂ (n) and q ′ ₄ (n), the same processing as in the first embodiment is performed to output the output signal z _2ch (n).

<Effect>
With such a configuration, voice signals of four speakers can be transmitted together. In addition, since the latest speaker and the second speaker use the low frequency side of the two output channels, even if there is no demodulation processing, a two-speaker dialogue can be heard as speech.

  Similar modifications can be made with respect to the second embodiment. By making the input signal 2 channels (stereo channel) and preparing two configurations of the audio dividing apparatus 500 of the second embodiment, the audio signal for 4 channels can be received simultaneously using the 2-channel audio input, You can identify the direction and the speaker, and receive conversations of four or more speakers.

<Fourth embodiment>
A description will be given centering on differences from the first embodiment.

  In the first embodiment, two audio signals are superimposed on one output signal, but in this embodiment, E audio signals are superimposed. E is any integer of 1 or more.

  FIG. 10 is a functional block diagram of the voice aggregating apparatus 700 according to the present embodiment, and FIG. 11 shows an example of its processing flow.

  The voice aggregation device 700 includes a voice analysis unit 102, a voice signal control unit 703, a voice information control unit 704, a sampling period conversion unit 705, a modulation unit 707, a code conversion unit 708, and a signal addition unit 709. .

The voice aggregating apparatus 700 receives N voice signals x _i (n) picked up by the N microphones 101-i as input, and E voice signals included in the N voice signals x _i (n). And a signal z (n) in which the audio information is superimposed.

<Audio information control unit 704>
The voice information control unit 704 receives M pieces of voice information q _i (n) as input, obtains the utterance order based on the utterance state v _i (n) included in the voice information q _i (n), and obtains the e-th utterance s E-number of determined speech information q _'e (n) corresponding to _e (S704), and outputs. However, e = 1, 2,..., E. The audio information control unit 704 outputs the E-number of utterances s _e to the audio signal controller 703.

<Audio signal control unit 703>
The voice signal control unit 703 receives M pieces of voice information q _i (n), M pieces of voice signals y _i (n), and E utterances s _e as input, and E pieces corresponding to E utterances s _e. Audio signal z _e (n) is obtained (S703) and output.

<Sampling period conversion unit 705>
The sampling period conversion unit 705 receives E audio signals z _e (n) as input, and performs up-sampling at least (3E-3) times on each of the E audio signals z _e (n) to obtain (3E -3) E audio signals u _e (n) having a double sampling frequency are obtained (S705) and output.

<Modulation unit 707>
Modulator 707 receives E audio signals u _e (n) as input, and converts E audio signals u _e (n) into non-overlapping bands of (3E-3) bands each having a bandwidth of 8 kHz. The voice signal t _e (n) is modulated (S707) and output. At this time, the band in which the audio signal t _e (n) has a component and the band in which the audio signal t _{e ′} (n) has a component are set so as not to be adjacent to each other. However, e ′ is any one of 1, 2,..., E, and e ≠ e ′. It should be noted that any existing method may be used as the modulation method, and an optimum method may be selected as appropriate in accordance with the use environment.

<Code conversion unit 708>
The code conversion unit 708 receives E pieces of speech information q ′ _e (n) as input, and includes E pieces of direction information θ ′ _e (n) and E pieces of information included in the E pieces of speech information q ′ _e (n). A signal D (n) obtained by converting a bit string indicating the speaker information p ′ _e (n) into a speech waveform is obtained (S708) and output. Note that the speech waveform is generated so as to have a component corresponding to the bit string in any band where there is no component of the E speech signals t _e (n). It should be noted that the band may be any band that does not have components of E audio signals t _e (n), and may be generated so as to have components in a plurality of bands. For example, when (3E-3) times upsampling is performed, out of (3E-3) bands with a bandwidth of 8 kHz, (2E-2) pieces output the audio signal t _e (n). Used. Therefore, (E-1) bands can be used to output the signal D (n), and the method may be set as appropriate.

<Signal Adder 709>
The signal adder 709 receives the E audio signals t _e (n) and the signal D (n), adds these values, obtains an output signal z (n) (S709), and outputs it.

<Effect>
With such a configuration, E channel audio signals can be sent at the same time, and voice information (direction information, speaker information, etc.) of E channel audio signals can be sent together to participate in the conversation. You can tell who the voice is from among the four or more speakers, and you can send the conversation of four or more speakers using a one-channel voice input that can be used with a general OS. Note that the present embodiment and the third embodiment may be combined, the number of output signal channels may be two, and audio signals for E × 2 channels may be sent.

  In addition, since it becomes 1st embodiment when E = 2, it can be said that 1st embodiment is an example of this embodiment.

<Fifth embodiment>
A description will be given centering on differences from the second embodiment.

  In the present embodiment, a voice dividing device 800 that receives the output signal z (n) of the voice aggregation device 700 of the fourth embodiment will be described. In the present embodiment, the output signal z (n) is referred to as the input signal z (n).

  FIG. 12 is a functional block diagram of the audio dividing device 800 according to the present embodiment, and FIG. 13 shows an example of the processing flow.

  Audio splitting apparatus 800 includes a reception unit 508, a bandpass filter unit 804, a sampling period conversion unit 801, a modulation unit 803, and an information decoding unit 806.

The audio dividing device 800 receives the input signal z (n) and outputs E audio signals z ′ _e and respective audio information q ′ _{D, e} (n).

<Band pass filter unit 804>
The bandpass filter unit 804 receives the input signal z (n) as input, filters a band component including the component of the audio signal t _e (n) of the input signal z (n) (S804), and obtains the result of filtering. An audio signal t ' _e (n) is output. E band components are respectively filtered, and E audio signals t ′ _e (n) are output.

<Modulation unit 803>
The modulation unit 803 receives E audio signals t ′ _e (n) as input, and modulates the audio signal t ′ _e (n) into an audio signal u ′ _e (n) having a component of 0 to 8 kHz ( S803) and output. As a modulation method, a method corresponding to the modulation unit 707 may be used.

<Sampling period converter 801>
The sampling period conversion unit 801 receives E audio signals u ′ _e (n) as input, performs 1 / E times downsampling on each audio signal u ′ _e (n), and 1 / E times sampling. An audio signal y ′ _e (n) having a frequency of 16 kHz is obtained (S801) and output.

<Information decoding unit 806>
The information decoding unit 806 has the input signal z (n) as an input, and any band that does not have the components of the E audio signals t _e (n) of the input signal z (n) (this band is the same as that of the voice aggregation device 700). The band component (predetermined between the two) is decoded to obtain a bit string B ′ corresponding to the bit string B (S806) and output. However, the bit string B ′ = (q ′ _{D, 1} (n), q ′ _{D, 2} (n),..., Q ′ _{D, E} (n)).

<Effect>
With such a configuration, it is possible to simultaneously receive voice signals for E channels using one-channel voice input, specify the direction and speaker of the voice signal, and receive conversations of four or more speakers.

<Other variations>
The present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

<Program and recording medium>
In addition, various processing functions in each device described in the above embodiments and modifications may be realized by a computer. In that case, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its storage unit. When executing the process, this computer reads the program stored in its own storage unit and executes the process according to the read program. As another embodiment of this program, a computer may read a program directly from a portable recording medium and execute processing according to the program. Further, each time a program is transferred from the server computer to the computer, processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program includes information provided for processing by an electronic computer and equivalent to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In addition, although each device is configured by executing a predetermined program on a computer, at least a part of these processing contents may be realized by hardware.

Claims (5)

M and N are each an integer greater than or equal to 2, i = 1, 2,..., M, and M audio signals y _i (n) A voice analysis unit that obtains voice information q _i (n) that is information about each voice signal y _i (n), and the voice information q _i (n) is an utterance state v _i of the voice signal y _i (n). (n), and the audio signal y _i number or audio signal of the microphone for outputting (n) y _i direction information corresponding to the beam for outputting an (n) θ _i (n), and the audio signal y _i (n) And the speaker number p _i (n),
Assume that g ≠ h, determine the utterance order based on the utterance state v _i (n) included in the voice information q _i (n), and obtain voice information q ′ _g (n) corresponding to the g-th utterance s _g and h A voice information control unit that obtains voice information q ′ _h (n) corresponding to the first utterance s _h ;
An audio signal control unit for obtaining an audio signal z _g (n) corresponding to the utterance s _g and an audio signal z _h (n) corresponding to the utterance s _h ;
Sampling frequency of the audio signal z _g (n) and the audio signal z _h (n) is set to UHz, and the audio signal z _g (n) and the audio signal z _h (n) are up-sampled three times. The audio signal u _g (n) composed of components from 0 to (U / 2) Hz out of the audio signal with 3 times sampling frequency 3 UHz, and UHz (3 U / out of the audio signal with 3 times sampling frequency 3 UHz 2) a sampling frequency converter for obtaining an audio signal u _h (n) composed of Hz components ;
The audio signal u _h a _{(n) t h (n)} = u h (n) * (- 1) by ^n, and a modulator for modulating the voice signal t _h (n),
The 'and _g (n), the speech information q' audio information q _'g number or direction information of the microphone included in the _{(n) θ' g (n} ) and the speaker number p contained in _h (n) A code conversion unit for obtaining a signal D (n) obtained by converting a bit string indicating microphone number or direction information θ ′ _h (n) and a speaker number p ′ _h (n) into a speech waveform, and The waveform is generated to have a component corresponding to the bit string in the band of (U / 2) Hz or more and UHz or less ,
A signal adder that adds the audio signal u _g (n), the audio signal t _h (n), and the signal D (n) to obtain an output signal z (n),
Voice aggregation device. g ≠ h, and the input signal z (n) is a signal obtained by adding the audio signal u _g (n), the audio signal t _h (n), and the signal D (n), and the audio signal u _g ( n) is a signal obtained by upsampling the audio signal z _g (n) three times, and the audio signal t _h (n) is obtained by upsampling the audio signal z _h (n) three times. Is obtained by modulating u _h (n) by t _h (n) = u _h (n) * (-1) ⁿ , and the signal D (n) corresponds to the audio signal z _g (n) Microphone number or direction information θ ′ _g (n) and speaker number p ′ _g (n) included in audio information q ′ _g (n), and audio information q corresponding to audio signal z _h (n) a signal converted into a speech waveform bit string B showing the _'h number or direction information of the microphone θ included in the (n)' _h (n) and the speaker ID p _'h (n), the speech waveform Is generated so as to have a component corresponding to a bit string in a band from UHz to 2 UHz,
A low-pass filter unit that filters a band component of UHz or less of the input signal z (n);
A high-pass filter unit that filters a band component of 2 UHz or more of the input signal z (n);
The high-pass filter portion of the filtering of the resulting signal t _'h a _{(n) u' h (n} ) = t 'h (n) * (- 1) by ^n, the audio signal u' modulates the _h (n) A modulation unit;
An information decoding unit that obtains a bit string corresponding to the bit string B by decoding a band component from UHz to 2 UHz of the input signal z (n),
Voice splitting device. M and N are each an integer of 2 or more, i = 1, 2,..., M, and the speech analysis unit uses the signals collected by the N microphones to generate M speech signals y. a voice signal analysis step for obtaining voice information q _i (n) that is information on _i (n) and each voice signal y _i (n), wherein the voice information q _i (n) is a voice signal y _i (n the utterance state v _i (n)) of the speech signal y _i (number or audio signal of the microphone for outputting n) y _i (direction information corresponding to the beam for outputting n) theta _i (n), and voice Including the speaker number p _i (n) of the signal y _i (n),
and g ≠ h, audio information control unit, the audio information q _i seek utterance order based on the utterance state contained in _{(n) v i (n)} , the speech information q corresponding to g-th utterance s _{g 'g} a voice information control step for obtaining (n) and voice information q ′ _h (n) corresponding to the h-th utterance s _h ;
An audio signal control unit for obtaining an audio signal z _g (n) corresponding to the utterance s _g and an audio signal z _h (n) corresponding to the utterance s _h ;
Sampling frequency of the audio signal z _g (n) and the audio signal z _h (n) is set to UHz, and the sampling frequency conversion unit performs the audio signal z _g (n) and the audio signal z _h (n). 3 times upsampling of the audio signal u _g (n) consisting of components from 0 to (U / 2) Hz out of the audio signal with 3 times sampling frequency 3 UHz and the audio signal with 3 times sampling frequency 3 UHz A sampling frequency conversion step for obtaining an audio signal u _h (n) consisting of a component from UHz to (3U / 2) Hz ,
Modulation unit, the audio signal u _h a _{(n) t h (n)} = u h (n) * (- 1) by ^n, a modulation step of modulating the audio signal t _h (n),
Code conversion unit, the audio information q _'g number or direction information of the microphone included in the _{(n) θ' g (n} ) and the speaker ID p _'g (n), and the audio information q' _h ( a code conversion step for obtaining a signal D (n) obtained by converting a bit string indicating the microphone number or direction information θ ′ _h (n) and the speaker number p ′ _h (n) included in n) into a speech waveform; The speech waveform is generated so as to have a component corresponding to a bit string in a band of (U / 2) Hz or more and UHz or less ,
A signal addition unit includes a signal addition step of adding the audio signal u _g (n), the audio signal t _h (n), and the signal D (n) to obtain an output signal z (n),
Voice aggregation method. g ≠ h, and the input signal z (n) is a signal obtained by adding the audio signal u _g (n), the audio signal t _h (n), and the signal D (n), and the audio signal u _g ( n) is a signal obtained by upsampling the audio signal z _g (n) three times, and the audio signal t _h (n) is obtained by upsampling the audio signal z _h (n) three times. Is obtained by modulating u _h (n) by t _h (n) = u _h (n) * (-1) ⁿ , and the signal D (n) corresponds to the audio signal z _g (n) Microphone number or direction information θ ′ _g (n) and speaker number p ′ _g (n) included in audio information q ′ _g (n), and audio information q corresponding to audio signal z _h (n) a signal converted into a speech waveform bit string B showing the _'h number or direction information of the microphone θ included in the (n)' _h (n) and the speaker ID p _'h (n), the speech waveform Is generated so as to have a component corresponding to a bit string in a band from UHz to 2 UHz,
A low-pass filter unit that filters a band component of UHz or less of the input signal z (n); and
A high-pass filter unit that filters a band component of 2 UHz or higher of the input signal z (n); and
Modulation unit, the high-pass filter step of filtering the resulting signal t _'h (n) to _{u' h (n) = t} 'h (n) * (- 1) by ^n, the audio signal u' _h (n Modulation step to modulate to
An information decoding unit including an information decoding step of decoding a band component of UHz to 2 UHz of the input signal z (n) to obtain a bit string corresponding to the bit string B,
Audio segmentation method.   A program for causing a computer to function as the voice aggregation device according to claim 1 or the voice division device according to claim 2.

Images