JP7532791B2 - SOUND PROCESSING DEVICE, SOUND PROCESSING PROGRAM, AND SOUND PROCESSING METHOD - Google Patents

Description

The present invention relates to an audio processing device, an audio processing program, and an audio processing method, and can be applied, for example, to sound masking processing used as a method for preventing the contents of a conversation from being leaked to third parties around a speaker.

In recent years, there has been a problem with the content of conversations being leaked to nearby third parties when a speaker and the other person are talking at reception counters, windows, meeting spaces, etc. in facilities where a large number of people are present (e.g. hospitals, pharmacies, banks, etc.).

Preventing conversation content from being leaked to third parties is called speech privacy, and the masking effect of sound is used to achieve speech privacy.

The masking effect is a phenomenon in which a certain sound (hereafter referred to as the target sound) becomes difficult to hear (is masked) when another sound with acoustic characteristics (e.g. frequency characteristics, pitch, formants, etc.) similar to the target sound is present. The masking sound is generally called the masker, and the masked target sound is called the maskee.

Patent Document 1 proposes a technology that uses this masking effect to prevent the contents of a conversation from being leaked to a third party (protecting speech privacy).

The voice processing method described in Patent Document 1 extracts the spectral envelope and spectral fine structure of the input voice signal from a microphone, and transforms the extracted spectral envelope to generate a transformed spectral envelope. The transformed spectral envelope and the extracted spectral fine structure are then synthesized to generate a transformed spectrum, and the signal generated based on the transformed spectrum is output as a "masker signal," thereby preventing the contents of the conversation from being heard by third parties.

JP 2006-243178 A

However, in the voice processing method described in Patent Document 1, the spectral envelope of the input voice signal from the microphone is extracted, the spectral envelope is transformed to generate a transformed spectral envelope, and the transformed spectral envelope is combined with the spectral fine structure to use as a signal to generate a masker signal. As a result, the masker signal generated by transforming the speaker's voice signal ends up sounding artificial, and there is a possibility that the masker signal will sound unpleasant.

Furthermore, because the spectral envelope of speech changes over time, extracting the spectral envelope requires a short processing cycle (approximately 80 to 160 milliseconds). As a result, a signal with a deformed spectral envelope in a short processing cycle continues to be output as the masker signal, which can make the sound even more artificial and unpleasant.

In view of the above problems, there is a need for an audio processing device, an audio processing program, and an audio processing method that can achieve a high masking effect without analyzing the acoustic features of the speaker who is speaking the voice (hereinafter referred to as the "target speaker").

The first sound processing device of the present invention is characterized by having: (1) an input signal storage means for storing a microphone input signal supplied from a microphone that picks up the voice of a target speaker while he is speaking, in storage frame units of a predetermined length that can be recognized as a voice signal; (2) a signal selection means for selecting a masker segment signal to be used for generating a masker signal that makes the voice spoken by the target speaker difficult to hear from past storage frames stored in the input signal storage means while the target speaker is speaking, and outputting the selection result; and (3) a masker signal generation means for generating and outputting the masker signal using the masker segment signal selected by the signal selection means while the target speaker is speaking .

The second sound processing device of the present invention is characterized in that it causes a computer to function as: (1) an input signal storage means for storing a microphone input signal supplied from a microphone that picks up the voice of a target speaker while he is speaking , in storage frame units of a predetermined length that can be recognized as a voice signal; (2) a signal selection means for selecting a masker segment signal to be used for generating a masker signal that makes the voice spoken by the target speaker difficult to hear from past storage frames stored in the input signal storage means while the target speaker is speaking , and outputting the selection result; and (3) a masker signal generation means for generating and outputting the masker signal using the masker segment signal selected by the signal selection means while the target speaker is speaking.

The third invention is an acoustic processing method performed by an acoustic processing device, which comprises (1) an input signal storage means, a signal selection means, and a masker signal generation means, (2) the input signal storage means stores a microphone input signal supplied from a microphone that picks up the voice of a target speaker while he is speaking in units of storage frames of a predetermined length that can be recognized as a voice signal, (3) the signal selection means selects a masker segment signal to be used for generating a masker signal that makes the voice spoken by the target speaker difficult to hear from past storage frames stored in the input signal storage means while the target speaker is speaking, and outputs the selection result, and (4) the masker signal generation means generates and outputs the masker signal using the masker segment signal selected by the signal selection means while the target speaker is speaking.

According to the present invention, by increasing the length of the signal used to generate the masker signal, the generated masker signal does not sound artificial, and therefore it is possible to provide an audio processing device, an audio processing program, and an audio processing method that can reduce the discomfort felt by the masker signal.

1 is a block diagram showing a functional configuration of a sound masking device according to a first embodiment. 1 is a block diagram showing an example of a hardware configuration of a sound masking device according to a first embodiment. FIG. 1 is a conceptual diagram (part 1) showing an output of a masker signal generated by the sound masking device according to the first embodiment. FIG. 2 is a conceptual diagram (part 2) showing an output of a masker signal generated by the sound masking device according to the first embodiment. FIG. 11 is a block diagram showing a functional configuration of a sound masking device according to a second embodiment. FIG. 11 is a block diagram showing a functional configuration of a sound masking device according to a third embodiment. FIG. 13 is a block diagram showing a functional configuration of a sound masking device according to a fourth embodiment. FIG. 13 is a block diagram showing a functional configuration of a sound masking device according to a fifth embodiment.

(A) First embodiment Hereinafter, a first embodiment of a sound processing device, a sound processing program, and a sound processing method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound processing device, the sound processing program, and the sound processing method according to the present invention are applied to a sound masking device will be described.

(A-1) Configuration of the First Embodiment FIG. 1 is a block diagram showing the functional configuration of a sound masking device 100 according to this embodiment.

The sound masking device 100 has a microphone 101, a microphone amplifier 102, an AD converter 103, a DA converter 104, a speaker amplifier 105, a speaker 106, and a sound masking processing unit 200.

The microphone 101 converts air vibrations such as human voice and sounds into electrical signals.

The microphone amplifier 102 amplifies the electrical signal received (picked up) by the microphone 101.

The AD converter 103 converts the electrical signal (analog signal) amplified by the microphone amplifier 102 into a digital signal. Hereinafter, the digital signal output from the AD converter 103 will be referred to as the "microphone input signal."

The sound masking processing unit 200 generates and outputs a signal masker signal that masks the microphone input signal from the input microphone input signal.

The DA converter 104 converts the output signal (digital signal) output from the sound masking processing unit 200 into an electrical signal (analog signal).

The speaker amplifier 105 amplifies the electrical signal output from the DA converter 104.

The speaker 106 converts electrical signals into vibrations in the air and outputs them as sound.

Next, we will explain the detailed configuration of the sound masking processing unit 200.

The sound masking processing unit 200 has a frame division unit 201, a long-term frame signal creation unit 202, a DB (database) writing unit 203, an input signal DB 204, a frame signal DB 205, a frame signal selection unit 206, a delay time calculation unit 207, a masker signal generation unit 208, a sound input terminal IN, and a sound output terminal OUT.

The sound input terminal IN is an interface (audio interface) that inputs a microphone input signal to the sound masking processing unit 200.

The frame division unit 201 divides the microphone input signal input to the sound masking processing unit 200 into frames (hereinafter referred to as "divided frames") of a predetermined length (hereinafter referred to as "frame length L1") and outputs the frames. It is desirable to apply a length suitable for analyzing sound as the frame length L1. For example, in the frame division unit 201, the frame length L1 may be 100 to 200 msec. The frame division unit 201 then outputs the divided frame signals (hereinafter referred to as "divided frame signals").

The long-term frame signal creation unit 202 combines the divided frame signals into a frame (hereinafter referred to as a "long-term frame") of a predetermined length (hereinafter referred to as "frame length L2") and outputs the combined signal. It is desirable to apply a frame length L2 (length of combining divided frames; number of combined divided frames) that is long enough to recognize a word or sentence of a speech signal (long enough to recognize as a part of a word or sentence spoken by a human when heard by the human ear). For example, in the long-term frame signal creation unit 202, the frame length L2 may be a length obtained by combining 3 to 5 frames of divided frame signals (e.g., L2 = L2 x 3 to L2 = L2 x 5), may be combined to be the length of a sound segmentation unit (e.g., 1 mora to 2 moras), or may be a time unit (e.g., any length in the range of 300 ms to 1000 msec). The long-term frame signal creation unit 202 then outputs the combined long-term frame signal (hereinafter referred to as a "long-term frame signal").

The DB writing unit 203 writes the long-term frame signal to the frame signal DB 205 of the input signal DB 204. The input signal DB 204 is a storage means for storing (holding) each past long-term frame signal for each long-term frame. Although the data format in the input signal DB 204 is not limited, it is assumed here that the input signal DB 204 is composed of a frame signal DB 205 that stores past long-term frame signals. The frame signal selection unit 206 selects multiple frames of the past long-term frame signals stored in the frame signal DB 205 of the input signal DB 204 as signals to be used to generate a masker signal (hereinafter referred to as "masker segment signals"), and outputs the selection result.

The delay time calculation unit 207 reads out multiple frames of the selected past long-time frame signal as a masker segment signal from the frame signal DB 205 of the input signal DB 204 based on the selection result of the frame signal selection unit 206, calculates the delay time of the masker segment signal that maximizes the masking effect based on the read masker segment signals of the multiple frames and the current divided frame signal, and outputs the calculated delay time.

The masker signal generation unit 208 reads out the selected masker segment signal from the frame signal DB 205 of the input signal DB 204 for multiple frames based on the selection result of the frame signal selection unit 206 and the delay time of the masker segment signal calculated by the delay time calculation unit 207, and generates and outputs a masker signal using a signal obtained by delaying the read masker segment signal for the multiple frames by the delay time calculated by the delay time calculation unit 207.

The sound output terminal OUT is an interface (audio interface) that outputs the masker signal generated by the masker signal generation unit 210 to the DA converter 104.

The sound masking processing unit 200 may be configured entirely as hardware (for example, constructed using a dedicated board or DSP (Digital Signal Processor)), or may be configured entirely as software using a computer. The sound masking processing unit 200 may be configured, for example, by installing a program (including the acoustic processing program according to the embodiment) on a computer having a memory and a processor.

In this embodiment, the AD converter 103 and the DA converter 104 are disposed outside the sound masking processing unit 200, but the AD converter 103 and the DA converter 104 may be mounted on the sound masking processing unit 200.

Figure 2 shows the configuration for implementing the sound masking processing unit 200 in software (computer).

The sound masking processing unit 200 shown in FIG. 2 is configured in software using a computer 300. Programs (including the sound processing program of the embodiment) are installed in the computer 300. Note that the computer 300 may be a computer dedicated to the sound processing program, or may be configured to be shared with programs of other functions.

The computer 300 shown in FIG. 2 has a processor 301, a primary memory unit 302, a secondary memory unit 303, a sound input terminal IN, and a sound output terminal OUT. The sound input terminal IN and the sound output terminal OUT are the same as the elements shown in FIG. 1.

The primary memory unit 302 is a storage means that functions as a working memory (work memory) for the processor 301, and is, for example, a high-speed memory such as a DRAM (Dynamic Random Access Memory).

The secondary storage unit 303 is a storage means for recording various data such as the OS (Operating System) and program data (including data of the sound processing program according to the embodiment), and may be, for example, a non-volatile memory such as a flash memory, HDD (Hard Disk Drive), or SSD (Solid State Drive).

In the computer 300 of this embodiment, when the processor 301 starts up, it reads the OS and programs (including the acoustic processing program according to the embodiment) recorded in the secondary storage unit 303, and deploys them on the primary storage unit 302 for execution. Note that the specific configuration of the computer 300 is not limited to the configuration in FIG. 2, and various configurations can be applied. For example, if the primary storage unit 302 is a non-volatile memory (e.g., a flash memory, etc.), the secondary storage unit 303 may be excluded from the configuration.

(A-2) Operation of the First Embodiment Next, the operation of the sound masking device 100 of the first embodiment having the above-mentioned configuration (the sound processing method of the embodiment) will be described.

When the operation of the sound masking device 100 starts and the target speaker U1 of the sound masking device 100 speaks into the microphone 101, an audio signal is input to the microphone 101.

The analog audio signal input to the microphone 101 is converted to an electrical signal (analog signal), amplified by the microphone amplifier 102, converted from the analog signal to a digital signal by the AD converter 103, and input to the sound input terminal IN of the sound masking processing unit 200 as the microphone input signal x(n). In the microphone input signal x(n), n is a parameter indicating a discrete time series of the input signal.

When the microphone input signal x(n) begins to be input to the sound input terminal IN of the sound masking processing unit 200, it is input to the frame division unit 201.

The frame division unit 201 divides the microphone input signal x(n) into divided frame signals having a frame length L1. The frame division unit 201 divides the microphone input signal x(n) into divided frames according to, for example, equation (1).

In equation (1), x_frame(l;m) is the divided frame signal, l is the frame number, and m is the time within the divided frame (m = 0, 1, 2, ..., L1-1).

The frame splitting unit 201 outputs the split frame signal x_frame(l;m) to the long-time frame signal creation unit 202 and the delay time calculation unit 207.

The long-term frame signal creation unit 202 combines the divided frame signals into a long-term frame signal with a frame length L2. The specific method by which the long-term frame signal creation unit 202 creates the long-term frame is not limited, and various methods can be applied. For example, when combining the divided frame signals with a divided frame, the long-term frame signal creation unit 202 may create the long-term frame signal x_frame_long(s) according to formula (2).

In equation (2), i is the index (i = 0, 1, 2, ..., I-1), and I is the number of divided frames used in the long-term frame signal (hereafter referred to as "number of frames used") (I = L2/L1).

The long-term frame signal x_frame_long(s) may be generated, for example, according to equations (3) and (4).

In equation (3), s is the time in the long-time frame (s = 0, 1, 2, ..., L2-L1-1). Equation (3) shifts the long-time frame signal x_frame_long(s) forward by the divided frame length L1, and equation (4) stores the divided frame signal x_frame(l;m) after the long-time frame signal x_frame_long(s). Also, the divided frame signal x_frame(l;m) may be combined in time units.

The long-time frame signal creation unit 202 outputs the long-time frame signal x_frame_long(s) to the DB writing unit 203.

The DB writing unit 203 writes the long time frame signal x_frame_long(t) to the frame signal DB 205 of the input signal DB 204. The DB writing unit 203 writes the long time frame signal x_frame_long(t) to the frame signal DB 205 DB_singal(j;t) of the input signal DB 204 for each long time frame according to, for example, equation (5).

In equation (5), t is the time in the long-term frame (t = 0, 1, 2, ..., L2-1), j is the index (j = 0, 1, 2, ..., DB_LEN-1; primary key; long-term frame identifier) that is incremented when a long-term frame signal is written to the frame signal DB205 of the input signal DB204, and DB_LEN is the database length. As shown in equations (5) and (6), the long-term frame signal x_frame_long(s) is stored for each long-term frame in the frame signal DB205DB_singal(j;t).

The input signal DB204 is a storage means that accumulates (holds) each past long-term frame signal.

As described above, in this embodiment, the input signal DB204 includes the frame signal DB205. Here, each long-time frame signal x_frame_long(t) is stored in the frame signal DB205.

The frame signal selection unit 206 selects a plurality of frames as masker segment signals from past long-time frame signals stored in the frame signal DB 205 of the input signal DB 204. The specific method by which the frame signal selection unit 206 selects the masker segment signal is not limited, and various methods can be applied. For example, the frame signal selection unit 206 selects a frame selection result T(p) as shown in equation (7).

In equation (7), p (p = 0, 1, ..., SEL_NUM-1) is the index of the selected frame selection result T(p), SEL_NUM (SEL_NUM <= DB_LEN-1) is the number of selected masker segment signals, and j is the index of the database in equation (6). Equation (7) selects long-term frame signals stored in frame signal DB205 of input signal DB204 in chronological order, and assigns the index number of the database in which the selected long-term frame signal is stored to T(p).

Moreover, the method of selecting the masker segment signal may be such that the masker segment signal is selected according to, for example, equation (8).

In equation (8), rand(p) is a function that generates a random number for a natural number p. Equation (8) uses a MOD function to return the remainder when the random number generated by rand(p) is divided by DB_LEN-1, so that the random number is randomly selected from the long-term frame signal stored in the frame signal DB205 of the input signal DB204.

As described above, the frame signal selection unit 206 selects multiple frames from the long-time frame signals stored in the frame signal DB 205, and outputs the selection result T(p) to the delay time calculation unit 207 and the masker signal generation unit 208.

The delay time calculation unit 207 reads out the masker segment signal from the frame signal DB 205 of the input signal DB 204 based on the selection result T(p) of the frame signal selection unit 206, calculates the delay time of the masker segment signal based on the masker segment signal and the divided frame signal x_frame(l;m) output from the frame division unit 201 so as to maximize the masking effect, and outputs the calculated delay time. The delay time calculation unit 207 may calculate the delay amount delay(l;p) according to, for example, equation (9).

In equation (9), τ1 (τ1 = 0, 1, ..., L1-1) is the delay of the cross-correlation function, and cross_corr(l; τ1) is the cross-correlation function of the masker segment signal. Equations (9) and (10) are used to find the cross-correlation function cross_corr(l; k) between the masker segment signal and the divided frame signal x_frame(l; m), and to find the index τ1 at which the cross-correlation function cross_corr(l; k) is maximized, and substitute this for the delay amount delay(l; p).

The delay time calculation unit 207 outputs the calculated delay amount delay(l;p) to the masker signal generation unit 208.

The masker signal generating unit 208 reads out a plurality of frames of the masker segment signal from the frame signal DB 205 of the input signal DB 204 based on the selection result T(p) of the frame signal selecting unit 206, and generates a masker signal by delaying the masker segment signal by the delay amount delay(l;p) calculated by the delay time calculating unit 207. The specific method by which the masker signal generating unit 208 generates the masker signal is not limited, and various methods can be applied. The masker signal generating unit 210 generates the masker signal h(l;t), for example, according to equation (11).

Equation (11) is an equation for reading out multiple masker segment signals selected by the frame signal selection unit 206 from the frame signal DB 205 of the input signal DB 204, delaying the read out masker segment signals by a delay amount delay(l;p) and then superimposing them to generate a masker signal h(l;t).

Then, the masker signal generating unit 208 performs overlap-add processing on the masker signal h(l;t) in accordance with equation (12) and outputs the result as an output signal y(n) to the sound output terminal OUT.

The masker signal y(n) output from the sound output terminal OUT of the sound masking processing unit 200 is converted from a digital signal to an analog signal by the DA converter 104, amplified by the speaker amplifier 105, and then output from the speaker 106.

Figures 3 and 4 show examples of the relative positions of the microphone 101, a target speaker U1 speaking into the microphone 101, a person other than the target speaker U1 standing behind the target speaker U1 (a person for whom the speech of the target speaker U1 is made difficult to hear by a masker signal: hereafter referred to as the "masking subject") U2, and the speaker 106 (the positioning configuration of the speaker 106).

3 and 4, the directivity of the direct sound DS (Direct Sound) output from the speaker 106 is shown by a dotted line. Also, in FIG. 3, the directivity of the reflected sound RS (Reflected Sound) generated by the direct sound being reflected by the floor FR (FLOOR) is shown by a dashed line.

In FIG. 3, the speaker 106 is installed in front of the target speaker U1 at knee height, with the vibration surface of the speaker 106 facing downward and at an angle to the surface of the floor FR, so that the direct sound DS is reflected by the floor FR and the reflected sound RS is directed to be transmitted to the masking subject U2 behind the target speaker U1. The masker signal emitted from the speaker 106 is output toward the surface of the floor FR and is reflected when it reaches the floor FR. As a result, the masker signal reflected by the floor FR is transmitted to the masking subject U2 behind the target speaker U1. At this time, the direct sound of the voice spoken by the target speaker U1 is also transmitted to the masking subject U2, but is masked by the masker signal.

As described above, various installation methods can be widely applied to the speaker 106 as long as the speaker 106 is installed so that the masker signal is not audible to the target speaker U1 and the masker signal is audible to the masking subject U2. For example, as shown in FIG. 4(a), if there is space to install a speaker 106 behind the target speaker U1, the speaker may be installed behind the target speaker U1 and the vibration surface of the speaker 106 may be directly directed toward the masking subject U2. Alternatively, as shown in FIG. 4(b), if there is space to embed the speaker 106 in the floor FR near the masking subject U2, the speaker 106 may be embedded in the floor FR and the vibration surface of the speaker 106 may be directly directed toward the masking subject U2 to output a masker signal. Alternatively, as shown in FIG. 4(c), if there is space to install the speaker 106 on the ceiling CE (CEILING) near the masking subject U2, the speaker 106 may be installed on the ceiling CE and the vibration surface of the speaker 106 may be directly directed toward the masking subject U2 to output a masker signal.

(A-3) Effect of the First Embodiment As described above, according to the first embodiment, in order to prevent the generated masker signal from becoming an artificial sound (a mechanical sound), the sound masking device 100 stores the generated masker signal in the input signal DB 204 with a long-term frame length L2 that is long enough to recognize the target speaker's voice as a voice signal (long enough to recognize as a word or part of a sentence spoken by a human when heard by the human ear), and uses the past long-term frame signal stored in the input signal DB 2054 as a masker segment signal without modifying it.

Furthermore, by calculating the delay amount at which the correlation function between the masker segment signal and the divided frame signal is maximized and then generating the masker signal after delaying it, the correlation between the target speaker's voice and the generated masker signal is increased, resulting in a higher masking effect.

(B) Second embodiment A second embodiment of the sound processing device, sound processing program, and sound processing method of the present invention will be described below in detail with reference to the drawings. In this embodiment, an example in which the sound processing device, sound processing program, and sound processing method of the present invention are applied to a sound masking device will be described.

(B-1) Configuration of the Second Embodiment Fig. 5 is a block diagram showing the functional configuration of a sound masking device 100A according to the second embodiment. In Fig. 5, the same or corresponding parts as those in Fig. 1 are denoted by the same or corresponding reference numerals.

The following describes the second embodiment, focusing on the differences from the first embodiment, and omitting explanations of parts that overlap with the first embodiment.

The sound masking device 100A of the second embodiment differs from the first embodiment in that the sound masking processing unit 200 is replaced with a sound masking processing unit 200A.

The sound masking processing unit 200A differs from the first embodiment in that a voice section determination unit 209, a DB accumulation determination unit 210, and a masker signal generation determination unit 211 are added, and further, the DB writing unit 203, the frame signal selection unit 206, and the masker signal generation unit 208 are replaced with a DB writing unit 203A, a frame signal selection unit 206A, and a masker signal generation unit 208A.

The sound masking device 100A of the second embodiment differs from the sound masking device 100 of the first embodiment in that a speech section determination unit 209 and a DB accumulation determination unit 210 are added and replaced with a DB writing unit 203A, resulting in a different accumulation method for long-term frame signals accumulated in a frame signal DB 205 of an input signal DB 204; a masker signal generation determination unit 211 is added and replaced with a frame signal selection unit 206A, resulting in a different masker segment signal selection method; and a masker signal generation unit 208A, resulting in a different masker signal generation method.

The detailed configuration of the sound masking processing unit 200A is explained below.

The sound masking processing unit 200A has a frame division unit 201, a long-time frame signal creation unit 202, a voice section determination unit 209, a DB accumulation determination unit 210, a DB writing unit 203A, an input signal DB 204, a frame signal DB 205, a masker signal generation determination unit 211, a frame signal selection unit 206A, a delay time calculation unit 207, a masker signal generation unit 208A, a sound input terminal IN, and a sound output terminal OUT.

The voice section determination unit 209 determines whether the divided frame signal is a voice section or a non-voice section (a section other than a voice section) and outputs the determination result.

The DB accumulation determination unit 210 determines whether to write and accumulate the long-term frame signal in the DB based on the determination result of the voice section determination unit 209, and outputs the determination result.

The DB writing unit 203A writes a long-term frame signal to the frame signal DB 205 of the input signal DB 204 based on the result of the DB accumulation determination unit 210's determination as to whether or not to write and accumulate the divided frame signal in the DB.

The masker signal generation determination unit 211 determines whether or not to generate a masker signal based on the determination result of the voice section determination unit 209, and outputs the determination result.

Based on the judgment result of the masker signal generation judgment unit 211, the frame signal selection unit 206A selects a multi-frame masker segment signal from past long-term frame signals stored in the frame signal DB 205 of the input signal DB 204, and outputs the selection result.

In the second embodiment, the masker signal generation determination unit 211 may be excluded.

(B-2) Operation of the Second Embodiment Next, the operation of the sound masking device 100A in the second embodiment having the above-mentioned configuration (the sound processing method according to the embodiment) will be described in detail.

The basic operation of the sound masking process in the sound masking device 100A according to the second embodiment is similar to the sound masking process described in the first embodiment.

The following describes in detail the differences from the first embodiment, focusing on the operations of the speech segment determination unit 209, DB accumulation determination unit 210, DB writing unit 203A, masker signal generation determination unit 211, frame signal selection unit 206A, and masker signal generation determination unit 2011.

The frame division unit 201 divides the microphone input signal x(n) into divided frame signals of frame length L1, and outputs the divided frame signals x_frame(l;m) to the long-time frame signal creation unit 202, the voice section determination unit 209, and the delay time calculation unit 207.

The voice section determination unit 209 uses the divided frame signal x_frame(l;m) to determine whether it is a voice section or a non-voice section. There is no limitation on the specific method by which the voice section determination unit 209 determines whether it is a voice section or a non-voice section, and various methods can be applied. For example, the voice section determination unit 209 determines whether it is a voice section or a non-voice section according to equations (13) and (14).

In equations (13) and (14), x_frame_amp(l) is the average amplitude value of the divided frame signal, VAD(l) is the voice section determination result, and TH is the threshold value used to determine the voice section. Equation (13) is an equation for finding the average amplitude value x_frame_amp(l) of the divided frame signal x_frame(l;m), and equation (14) is an equation for finding that if the average amplitude value x_frame_amp(l) of the divided frame signal x_frame(l;m) found by equation (13) is greater than the threshold value TH, it is determined to be a voice section and 1 is substituted for the voice section determination result VAD(l), and if it is less than the threshold value TH, it is determined to be a non-voice section and 0 is substituted for the voice section determination result VAD(l).

The threshold value TH in equation (14) may be any method capable of determining the presence or absence of speech, and various methods may be widely applied. For example, as shown in equation (15), a fixed threshold value TH may be used in which a frame (hereinafter referred to as the "initial frame") of a predetermined length (hereinafter referred to as the "frame length L3") from when the sound masking device 100A begins to operate is treated as a silent section, and the average amplitude value of that initial frame is used as the threshold value TH; or, as shown in equation (16), a threshold value TH(l) that varies for each divided frame using a time constant filter for x_frame_amp(l) may be used.

In equation (16), a is the coefficient of the time constant filter, and is a value between 0 and 1. In equation (16), if you want to slow down the updating of the threshold, a value close to 1 is desirable (for example, a = 0.9, etc.), and if you want to speed up the updating of the threshold, a value close to 0 is desirable (for example, a = 0.1, etc.).

The means for determining whether it is a voice section or a non-voice section can be a wide variety of methods, for example, the method may be to calculate the autocorrelation of the divided frame signal x_frame(l;m) to determine whether it is a voice section or a non-voice section.

The speech segment determination unit 209 outputs the speech segment determination result VAD(l) to the DB accumulation determination unit 210 and the masker signal generation determination unit 211.

The DB accumulation determination unit 210 determines whether or not to accumulate the long-term frame signal x_frame_long(s) in the frame signal DB 204 of the input signal DB 203 based on the voice activity determination result VAD(l) of the voice activity determination unit 209. The determination means performs the determination according to, for example, equation (17).

In equation (17), DB_flag(l) is the result of the decision as to whether to store or not. In equation (17), when the speech segment decision result VAD(l) is 1, it is decided to store in the DB, and 1 is substituted for the decision result DB_flag(l). On the other hand, when the speech segment decision result VAD(l) is 0, it is decided not to store in the DB, and 0 is substituted for the decision result DB_flag(l).

The DB storage determination unit 210 outputs the determination result DB_flag(l) of whether to store in the DB to the DB writing unit 203A.

The DB writing unit 203A writes the long-time frame signal x_frame_long(t) to the frame signal DB205 of the input signal DB204 according to, for example, equations (5) and (6) only when the judgment result DB_flag(l) output from the DB accumulation judgment unit 210 is 1. On the other hand, when the judgment result DB_flag(l) of the DB accumulation judgment unit 210 is 0, the DB writing unit 203A does not write the long-time frame signal x_frame_long(s) to the frame signal DB205 of the input signal DB204.

The masker signal generation determination unit 211 determines whether or not to generate a masker signal based on the voice activity determination result VAD(l) of the voice activity determination unit 209. The determination means performs the determination according to, for example, equation (18).

In equation (18), mask_flag(l) is the result of the decision as to whether or not to generate a masker signal. When the speech segment decision result VAD(l) is 1, equation (18) decides that a masker signal is to be generated, and assigns 1 to the decision result mask_flag(l). On the other hand, when the speech segment decision result VAD(l) is 0, it decides that a masker signal is not to be generated, and assigns 0 to the decision result mask_flag(l).

The masker signal generation determination unit 211 outputs the determination result mask_flag(l) of whether to generate a masker signal to the frame signal selection unit 206A and the masker signal generation unit 208A.

The frame signal selection unit 206A selects the frame selection result T(p) according to, for example, equation (7) or (8) only when the judgment result mask_flag(l) of whether to generate a masker signal output from the masker signal generation judgment unit 211 is 1. On the other hand, when the judgment result mask_flag(l) of whether to generate a masker signal is 0, multiple frames are not selected.

As described above, the frame signal selection unit 206 selects multiple frames from the long-time frame signals stored in the frame signal DB 205 only when the determination result mask_flag(l) of whether or not to generate a masker signal is 1, and outputs the selection result T(p) to the delay time calculation unit 207 and the masker signal generation unit 208A.

The masker signal generating unit 208A reads out a past long-time frame signal from the frame signal DB 205 of the input signal DB 204 as a masker segment signal based on the decision result mask_flag(l) of whether or not to generate a masker signal output from the masker signal generation deciding unit 211 and the selection result T(p) of the frame signal selecting unit 206, and generates and outputs a masker signal delayed by the delay amount delay(l;p) calculated by the delay time calculating unit 207. The masker signal generating unit 208A generates a masker signal ha(l;t) according to, for example, equation (19).

Equation (19) generates a masker signal h(l;t) and assigns it to ha(l;t) only when the judgment result of whether or not to generate a masker signal output from the masker signal generation judgment unit 211, mask_flag(l), is 1. On the other hand, when the judgment result of whether or not to generate a masker signal, mask_flag(l), is 0, 0 (silence) is assigned to ha(l;t).

As described above, the masker signal generating unit 208A generates a masker signal only when the determination result of whether or not to generate a masker signal, mask_flag(l), is 1.

Then, the masker signal generating unit 208A performs overlap-add processing on the masker signal ha(l;t) in accordance with equation (20) and outputs the result as an output signal y(n) to the sound output terminal OUT.

(B-3) Effect of the Second Embodiment As described above, according to the second embodiment, in the sound masking device 100A of the second embodiment, in order to prevent the generated masker signal from being an artificial sound (mechanical sound), the signal is stored in the input signal DB 204 at a length that allows it to be recognized as a voice signal, and past long-term frame signals stored in the input signal DB 204 are used as masker segment signals without modification, and the masker segment signals are delayed before generating the masker signal, thereby achieving the effect of increasing the correlation between the voice of the target speaker and the generated masker signal and improving the masking effect.

In addition, by storing the voice of the target speaker U1 in the frame signal DB205 of the input signal DB204 only when it is determined to be a voice section, only the voice section is stored in the input signal DB204, so that a masker signal can be generated from only the voice, and a high masking effect can be maintained. Furthermore, since the masker signal is generated only when it is determined to be a voice section, the masker signal can be generated and output only when the voice of the target speaker U1 is input, so that the masker signal can be configured to be output only when voice is input.

(C) Third embodiment Hereinafter, a third embodiment of the sound processing device, sound processing program, and sound processing method of the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound processing device, sound processing program, and sound processing method of the present invention are applied to a sound masking device will be described.

(C-1) Configuration of the Third Embodiment Fig. 6 is a block diagram showing the functional configuration of a sound masking device 100B according to a third embodiment. In Fig. 6, the same or corresponding parts as those in Figs. 1 and 5 are denoted by the same or corresponding reference numerals.

The following describes the third embodiment, focusing on the differences between the first and second embodiments, and omitting explanations of the parts that overlap with the first and second embodiments.

The sound masking device 100B of the third embodiment differs from the second embodiment in that the sound masking processing unit 200A is replaced with a sound masking processing unit 200B.

The sound masking processing unit 200B differs from the second embodiment in that a pitch estimation unit 212 is added, and the DB accumulation determination unit 210 and the masker signal generation determination unit 211 are replaced with a DB accumulation determination unit 210B and a masker signal generation determination unit 211B, respectively.

The sound masking device 100B of the third embodiment differs from the sound masking device 100 of the second embodiment in that the pitch of the divided frame signal is estimated by adding a pitch estimation unit 212, the method of accumulating the long-time frame signal accumulated in the frame signal DB205 of the input signal DB204 is different by replacing it with a DB accumulation determination unit 210B, and the method of generating the masker signal is different by replacing it with a masker signal generation determination unit 211B.

The detailed configuration of the sound masking processing unit 200B is explained below.

The sound masking processing unit 200B has a frame division unit 201, a long-time frame signal creation unit 202, a voice section determination unit 209, a pitch estimation unit 212, a DB accumulation determination unit 210B, a DB writing unit 203A, an input signal DB 204, a frame signal DB 205, a masker signal generation determination unit 211B, a frame signal selection unit 206A, a delay time calculation unit 207, a masker signal generation unit 208A, a sound input terminal IN, and a sound output terminal OUT.

The pitch estimation unit 212 estimates the pitch (voice pitch) of the divided frame signal only when a voice section is determined based on the voice section determination result output from the voice section determination unit 209, and outputs the pitch estimation result (hereinafter referred to as the "pitch estimate").

The DB accumulation determination unit 210B determines whether or not to accumulate the divided frame signal in the frame signal DB205 of the input signal DB204 based on the pitch estimate value of the pitch estimation unit 212, and outputs the determination result.

The masker signal generation determination unit 211B determines whether or not to generate a masker signal based on the pitch estimate from the pitch estimation unit 212, and outputs the determination result.

In the third embodiment, the masker signal generation determination unit 211B may be omitted, as in the first embodiment.

(C-2) Operation of the Third Embodiment Next, the operation of the sound masking device 100B in the third embodiment having the above-mentioned configuration (the sound processing method according to the embodiment) will be described in detail.

The basic operation of the sound masking process in the sound masking processing unit 200B according to the third embodiment is similar to the sound masking process described in the first and second embodiments.

The following provides a detailed explanation of the processing operations of the pitch estimation unit 212, DB accumulation determination unit 210B, and masker signal generation determination unit 211B, which are differences from the second embodiment.

The speech section determination unit 209 determines whether the divided frame signal x_frame(l;m) is a speech section or a non-speech section, for example, according to equations (13) to (15), and outputs the speech section determination result VAD(l) to the pitch estimation unit 212.

The pitch estimation unit 212 estimates the pitch of the divided frame signal x_frame(l;m) only for divided frames determined to be voice segments by the voice segment determination unit 209 (divided frames with VAD(l)=1). The specific method by which the pitch estimation unit 212 estimates the pitch is not limited, and various methods can be applied. The pitch estimation unit 212 makes the determination according to, for example, equations (21) to (23).

In equation (21), τ2 (τ2 = 0, 1, ..., L1-1) is the autocorrelation delay, in equation (22), fs is the sampling frequency, tmp_pitch(l) is a variable that temporarily holds the estimated pitch, and in equation (23), pitch(l) is the estimated pitch. Equation (21) calculates the autocorrelation function x_frame_corr(l; τ2) of the divided frame signal x_frame(l; m), and in equation (22), the delay τ2 at which the autocorrelation function x_frame_corr(l; τ2) is maximized is calculated and divided by the sampling frequency fs to estimate the pitch, which is temporarily substituted for tmp_pitch(l). In equation (23), when the speech segment determination result VAD(l) is 1 (when there is a speech segment), the pitch estimate pitch(l) is substituted with the estimated pitch value, and when the speech segment determination result VAD(l) is 0 (when there is a non-speech segment), the pitch estimate pitch(l) is substituted with 0.

The pitch can be estimated using a wide variety of methods. For example, the divided frame signal x_frame(l;m) can be subjected to a discrete Fourier transform or a fast Fourier transform, and then a cepstrum analysis can be performed to calculate the pitch.

The pitch estimation unit 212 outputs the pitch estimate pitch(l) to the DB accumulation determination unit 210B and the masker signal generation determination unit 211B.

The DB accumulation determination unit 210B determines whether or not to accumulate the long-term frame signal x_frame_long(s) in the frame signal DB 205 of the input signal DB 204 based on the pitch estimate pitch(l) of the pitch estimation unit 212. The determination means makes the determination according to, for example, equation (24).

In equation (24), DB_flag(l) is the result of the decision as to whether to store, and TH_PITCH is the threshold value used to decide whether to store in the DB. In equation (24), if the pitch estimate picth(l) is greater than the threshold value TH_PITCH, it is decided to store in the DB, and 1 is substituted for the decision result DB_flag(l). On the other hand, if the value is smaller than the threshold value TH_PITCH, it is decided not to store in the DB, and 0 is substituted for the decision result DB_flag(l).

The threshold value TH_PITCH can be set to determine whether or not to store in the DB, and various methods can be widely applied. For example, when the pitch estimation unit 212 determines that picth(l) is other than 0, TH_PITCH=0 may be set to store in the DB, or TH_PITCH=100 may be set to be equal to or greater than the lower limit of the fundamental frequency of human speech (e.g., 100 Hz).

The DB storage determination unit 210B outputs the determination result DB_flag(l) of whether to store in the DB to the DB writing unit 203A.

The masker signal generation determination section 211B determines whether or not to generate a masker signal based on the pitch estimate pitch(l) of the pitch estimation section 212. The determination means performs the determination according to, for example, equation (25).

In equation (25), mask_flag(l) is the result of the decision as to whether or not to generate a masker signal, and TH2_PITCH is the threshold value used to decide whether or not to store. In equation (25), if the pitch estimate picth(l) is greater than the threshold value TH2_PITCH, it is decided to store in DB, and 1 is substituted for the decision result DB_flag(l). On the other hand, if the value is smaller than the threshold value TH2_PITCH, it is decided not to generate a masker signal, and 0 is substituted for the decision result mask_flag(l).

The threshold value TH2_PITCH only needs to be able to determine whether or not to store in the DB, and various methods can be widely applied. For example, when the pitch estimation unit 212 determines that picth(l) is other than 0, TH2_PITCH=0 is set to store in the DB, or TH2_PITCH=100 is set to be equal to or greater than the lower limit of the fundamental frequency of human voice (e.g., 100 Hz). Also, TH2_PITCH=TH_PITCHDB may be set to be the same as the threshold value TH_PITCH in equation (24) used in the storage determination unit 209A.

The masker signal generation determination unit 211 outputs the determination result mask_flag(l) of whether to generate a masker signal to the frame signal selection unit 206A.

(C-3) Effect of the Third Embodiment As described above, according to the third embodiment, in the sound masking device 100B of the third embodiment, in order to prevent the generated masker signal from being an artificial sound (mechanical sound), the signal is stored in the input signal DB 204 at a length that allows it to be recognized as a voice signal, and past long-term frame signals stored in the input signal DB 204 are used as masker segment signals without modification, and the masker segment signals are delayed before generating the masker signal, thereby achieving the effect of increasing the correlation between the voice of the target speaker and the generated masker signal and improving the masking effect.

In addition, by estimating the pitch of the voice of target speaker U1 and using the estimated pitch value in DB accumulation determination unit 210B and masker signal generation determination unit 211B, only voiced sounds are accumulated in the input signal DB 204 during the speech section, making it possible to generate a masker signal from speech alone and maintaining a high masking effect. Furthermore, since the masker signal is generated only when a speech section is determined to be voiced, the masker signal is generated and output only when the voice of target speaker U1 is input. This makes it possible to configure the masker signal to be output only when speech is input.

(D) Fourth embodiment Hereinafter, a fourth embodiment of the sound processing device, sound processing program, and sound processing method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound processing device, sound processing program, and sound processing method according to the present invention are applied to a sound masking device will be described.

(D-1) Configuration of the Fourth Embodiment Fig. 7 is a block diagram showing the functional configuration of a sound masking device 100C according to a fourth embodiment. In Fig. 7, the same or corresponding parts as those in Figs. 1, 5, and 6 are denoted by the same or corresponding reference numerals.

The following describes the fourth embodiment, focusing on the differences between the first, second, and third embodiments, and omitting descriptions of the parts that overlap with the first, second, and third embodiments.

The fourth embodiment of the sound masking device 100C differs from the second embodiment in that the sound masking processing unit 200B is replaced with a sound masking processing unit 200C.

The sound masking processing unit 200C differs from the third embodiment in that a long-term pitch estimation information creation unit 213 and a pitch information DB 214 are added, and further in that the DB writing unit 203A, the input signal DB 204, and the frame signal selection unit 206A are replaced with a DB writing unit 203C, an input signal DB 204C, and a frame signal selection unit 206C.

The sound masking device 100C of the fourth embodiment differs from the sound masking processing unit 200B of the third embodiment in that a long-term pitch estimation information creation unit 213 and a pitch information DB 214 have been added, and have been replaced with an input signal DB 204C, so that pitch is stored in the input signal DB, the DB writing unit 203C has been replaced with a different method of storing the input signal DB 204C, and the frame signal selection unit 206C has been replaced with a different method of selecting a masker segment signal.

The detailed configuration of the sound masking processing unit 200C is explained below.

The sound masking processing unit 200C has a frame division unit 201, a long-term frame signal creation unit 202, a voice section determination unit 209, a pitch estimation unit 212, a long-term pitch estimation information creation unit 213, a DB accumulation determination unit 210B, a DB writing unit 203C, an input signal DB 204C, a frame signal DB 205, a pitch estimation information DB 214, a masker signal generation determination unit 211B, a frame signal selection unit 206C, a delay time calculation unit 207, a masker signal generation unit 208A, a sound input terminal IN, and a sound output terminal OUT.

The long-term pitch estimation information creation unit 213 creates pitch estimation information for a long-term frame (hereinafter referred to as "long-term pitch estimation information") based on the pitch estimates for each divided frame estimated by the pitch estimation unit 212, and outputs the long-term pitch estimation information.

Based on the result of the determination of whether or not to store in the DB of the DB storage determination unit 210B, the DB writing unit 203C writes the long-term frame signal to the frame signal DB 205 of the input signal DB 204C and the long-term pitch estimation information to the pitch estimation information DB 214 of the input signal DB 204C.

The input signal DB204C is a storage means that accumulates (holds) past long-term frame signals and each piece of past long-term pitch estimation information in association with each long-term frame. The data format in the input signal DB204C is not limited, but here, the input signal DB204C is assumed to be composed of a frame signal DB205 that accumulates past long-term frame signals, and a pitch information DB214 that accumulates past long-term pitch estimation information.

The frame signal selection unit 206C selects a masker segment signal from the frame signal DB205 of the input signal DB204C based on the judgment result of the masker signal generation judgment unit 211B and the comparison result between the pitch estimate of the pitch estimation unit 212 and the past long-term pitch estimation information stored in the pitch estimation information DB214 of the input signal DB204C, and outputs the selection result.

In the fourth embodiment, the masker signal generation determination unit 211B may be omitted, as in the first embodiment.

(D-2) Operation of the Fourth Embodiment Next, the operation of the sound masking device 100C in the fourth embodiment having the above-mentioned configuration (the sound processing method according to the embodiment) will be described in detail.

The basic operation of the sound masking process in the sound masking processing unit 200C according to the fourth embodiment is similar to the sound masking process described in the first, third, and second embodiments.

The following provides a detailed explanation of the differences from the third embodiment, focusing on the processing operations of the long-term pitch estimation information creation unit 213, DB writing unit 203C, input signal DB 204C, and frame signal selection unit 206C.

The pitch estimation unit 212 estimates the pitch of the divided frame signal x_frame(l;m) and outputs the pitch estimate pitch(l) to the long-term pitch estimation information creation unit 213, the DB accumulation determination unit 210B, the masker signal generation determination unit 211B, and the frame signal selection unit 206C.

The long-term pitch estimation information creation unit 213 combines the pitch estimates pitch(l) of the divided frame signals estimated by the pitch estimation unit 212 to create long-term pitch estimation information for the long-term frame signal x_frame_long(s). The long-term pitch estimation information creation unit 204 may create the long-term pitch estimation information pitch_long(i) by combining the pitch estimates according to, for example, equation (26).

Equation (26) shows that a process is performed to generate long-term pitch estimation information pitch_long(i) by combining the pitch estimates pitch(l) of the divided frame signals used in generating the long-term frame signal.

Then, long-term pitch estimation information creation section 213 outputs the created long-term pitch estimation information pitch_long(i) to DB writing section 203C. DB writing section 203C writes long-term frame x_frame_long(t) and long-term pitch estimation information pitch_long(i) into frame signal DB 205 of input signal DB 204 and pitch estimation information DB 214 in association with each other, based on determination result DB_flag(l) output from DB accumulation determination section 210B. Only when the determination result DB_flag(l) output from the DB accumulation determination unit 210B is 1, the DB writing unit 203C writes the long-term frame signal x_frame_long(t) to DB_singal(j;t) of the frame signal DB 205 and simultaneously writes the long-term pitch estimation information pitch_long(i) to the pitch estimation information DB 214 DB_pitch(i) according to, for example, equations (5), (27), and (6).On the other hand, when the determination result DB_flag(l) of the DB accumulation determination unit 210B is 0, the long-term frame signal x_frame_long(s) and the long-term pitch estimation information pitch_long(i) are not written to the frame signal DB 205 and the pitch estimation information DB 214 of the input signal DB 204C.

The input signal DB204C stores (holds) the long-term frame x_frame_long(t) and the long-term pitch estimation information pitch_long(i) in association with the frame signal DB205 and the pitch estimation information DB214 of the input signal DB204.

As described above, the input signal DB204C in this embodiment includes a frame signal DB205 and a pitch information DB214. Here, each long-term frame signal x_frame_long(t) is recorded in the frame signal DB205, and long-term pitch estimation information pitch_long(i) is recorded in the pitch information DB214.

The frame signal selection unit 206C compares the pitch estimate (pitch based on the current divided frame) of the pitch estimation unit 212 with past long-term pitch estimation information stored in the pitch information DB 214 of the input signal DB 204C based on the decision result mask_flag(l) of whether to generate a masker signal output from the masker signal generation decision unit 211, and selects a long-term frame signal having pitch information (close pitch value) close to the pitch estimate of the pitch estimation unit 212 as a masker segment signal. The frame signal selection unit 206C selects the frame selection result Tc(p) according to, for example, equation (28) or (29) only when the decision result mask_flag(l) of whether to generate a masker signal output from the masker signal generation decision unit 211 is 1. On the other hand, when the decision result mask_flag(l) of whether to generate a masker signal is 0, the frame selection result Tc(p) is not selected.

In equation (28), DB_pitch_ave(i) is the average value of past long-term pitch estimation information, in equation (29), Sub_pitch(j) is the absolute value of the difference between the pitch estimate pitch(l), the average value of past long-term pitch estimation information, and DB_pitch_ave(i), and in equation (30), Tc(p) is the selected frame number, and p (p = 0, 1, ..., SEL_NUM-1) is the number of selected frames. Also, small(x(k), p) in equation (30) is a function that outputs the index k_p of the p-th smallest x(k_p) in array x(k). Equations (26) and (27) calculate the average value DB_pitch_ave(i) of the past long-term pitch estimation information stored in the pitch information DB 214 for each index j, and select multiple frames of the index j (long-term frame) for which the absolute value of the difference between the pitch estimation value pitch(l) of the pitch estimation unit 212 and the average value DB_pitch_ave(i) of the past pitch estimation information is the smallest, as a masker segment signal.

The frame signal selection unit 206C may calculate a value indicating the pitch of the entire long-term frame based on the long-term pitch estimation information for each past long-term frame stored in the pitch information DB 214 (hereinafter referred to as "long-term frame pitch"), and select a long-term frame to be selected as a masker segment signal based on the comparison result between the pitch estimate of the pitch estimation unit 212 and the long-term frame pitch. For example, the frame signal selection unit 206C may select a long-term frame having a long-term frame pitch close to the pitch estimate of the pitch estimation unit 212 as a masker segment signal.

As described above, the frame signal selection unit 206C selects a long-term frame signal (index) having pitch information (close pitch value) close to the pitch estimate of the pitch estimation unit 212 as a masker segment signal, and outputs the frame selection result Tc(p).

(D-3) Effect of the Fourth Embodiment As described above, according to the fourth embodiment, in the sound masking device 100C of the fourth embodiment, in order to prevent the generated masker signal from being an artificial sound (a mechanical sound), the signal is stored in the input signal DB 204C at a length that allows it to be recognized as a voice signal, and past long-term frame signals stored in the input signal DB 204C are used as masker segment signals without modification, and the masker segment signals are delayed before generating the masker signal, thereby achieving the effect of increasing the correlation between the voice of the target speaker and the generated masker signal and improving the masking effect.

In addition, by estimating the pitch of the voice of the target speaker U1, creating long-term pitch information from the estimated pitch value and storing it in the input signal DB204C, and using it in the frame signal selection section, a long-term frame signal with a similar pitch can be selected from the frame signal DB205, and the frequency characteristics of the scar signal become close to the pitch of the voice of the target speaker U1, making it possible to generate a masker signal with a high masking effect.

(E-1) Configuration of the Fifth Embodiment Hereinafter, a fifth embodiment of the sound processing device, sound processing program, and sound processing method of the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound processing device, sound processing program, and sound processing method of the present invention are applied to a sound masking device will be described.

(E-1) Configuration of the Fifth Embodiment Fig. 8 is a block diagram showing the functional configuration of a sound masking device 100D according to a fifth embodiment. In Fig. 8, the same or corresponding parts as those in Figs. 1, 5, 6, and 7 are denoted by the same or corresponding reference numerals.

The following describes the fifth embodiment, focusing on the differences between the first, second, third, and fourth embodiments, and omits a description of the parts that overlap with the first, second, third, and fourth embodiments.

The fifth embodiment of the sound masking device 100D differs from the second embodiment in that the sound masking processing unit 200C is replaced with a sound masking processing unit 200D.

The sound masking processing unit 200D differs from the fourth embodiment in that the delay time calculation unit 207 is replaced with a delay time calculation unit 207D.

The sound masking device 100D of the fifth embodiment differs from the sound masking processing unit 200C of the fourth embodiment in that the delay time calculation method is different due to the replacement with a delay time calculation unit 207D.

The detailed configuration of the sound masking processing unit 200D is explained below.

The sound masking processing unit 200D has a frame division unit 201, a long-term frame signal creation unit 202, a voice section determination unit 209, a pitch estimation unit 212, a long-term pitch estimation information creation unit 213, a DB accumulation determination unit 210B, a DB writing unit 203C, an input signal DB 204C, a frame signal DB 205, a pitch estimation information DB 214, a masker signal generation determination unit 211B, a frame signal selection unit 206C, a delay time calculation unit 207D, a masker signal generation unit 208A, a sound input terminal IN, and a sound output terminal OUT.

The delay time calculation unit 207D reads out the long frame selected by the frame signal selection unit 206C from the signal DB 206 of the input signal DB 205 as a masker segment signal, calculates the delay time of the masker segment signal based on the read out masker segment signal and the current divided frame signal so as to maximize the masking effect, and outputs the calculated delay time.

In the fifth embodiment, the masker signal generation determination unit 211B may be omitted, as in the first embodiment.

(E-2) Operation of Fifth Embodiment Next, the operation of the sound masking device 100D in the fifth embodiment having the above-mentioned configuration (the sound processing method according to the embodiment) will be described in detail.

The basic operation of the sound masking process in the sound masking processing unit 200D according to the fifth embodiment is similar to the sound masking process described in the first, second, third, and fourth embodiments.

The following provides a detailed explanation of the processing operations in the delay time calculation unit 207D, which is different from the fourth embodiment.

The pitch estimation unit 212 estimates the pitch of the divided frame signal x_frame(l;m) and outputs the pitch estimate to the long-term pitch estimation information creation unit 213, the DB accumulation determination unit 210B, the masker signal generation determination unit 211B, the frame signal selection unit 206C, and the delay time calculation unit 207D.

The delay time calculation unit 207D reads out the long-time frame signal selected as the masker segment signal by the frame signal selection unit 206C from the frame signal DB 205 of the input signal DB 204C, calculates the delay time of the masker segment signal so as to maximize the masking effect based on the pitch estimation value of the pitch estimation unit 212 and the pitch estimation information DB 214DB_pitch(j;i) of the input signal DB 204C, and outputs the calculated delay time. The delay time calculation unit 207D may calculate the delay amount delay_D(l;p) according to, for example, equations (29) and (32).

In equation (31), Sub_pitch2(p;l) is the absolute value of the difference between the pitch estimate pitch(l) and the past long-term pitch estimation information DB_pitch(j;i), and delay_D(l;p) is the calculated delay amount. Equations (31) and (32) define the index i with the smallest absolute value Sub_pitch2(p;l) of the difference between the pitch estimate pitch(l) and the past long-term pitch estimation information DB_pitch(j;i) as the delay amount, and assign it to the delay amount delay_D(l;p).

The delay time calculation unit 207D supplies the calculated delay amount delay_D(l;p) to the masker signal generation unit 210.

(E-3) Effect of the First Embodiment As described above, according to the fifth embodiment, in the sound masking device 100D of the fifth embodiment, in order to prevent the generated masker signal from being an artificial sound (mechanical sound), the signal is stored in the input signal DB 204C at a length that allows it to be recognized as a voice signal, and past long-term frame signals stored in the input signal DB 204C are used as masker segment signals without modification, and the masker segment signals are delayed before generating the masker signal, thereby achieving the effect of increasing the correlation between the voice of the target speaker and the generated masker signal and improving the masking effect.

In addition, by estimating the pitch of the voice of the target speaker U1, creating long-term pitch information from the estimated pitch value and storing it in the input signal DB 204C, and using it in the delay time calculation unit 207D, it becomes possible to delay the masker segment signal so that the pitch becomes closer, and a masker signal with a high masking effect can be generated.

(F) Other Embodiments The present invention is not limited to the above-described embodiments, and modified embodiments such as those exemplified below can also be mentioned.

(F-1) For example, the sound masking device 100 of the above embodiment may be installed in a device that prevents the contents of a conversation from being leaked to people other than the target person in a conference call. In this case, in the sound masking device, the target speaker U1 is the person speaking in the conference call.

(F-2) In the above embodiment, the sound masking processing unit 200 of the sound masking device 100 may be configured to be processed by a processing device (e.g., a server) on a network.

(F-3) In the above embodiment, the sound masking device 100 has been described as including audio devices (microphone, microphone amplifier, AD converter, speaker, speaker amplifier, and DA converter), but the sound masking device 100 may be manufactured without the audio devices, and the audio devices may be connected separately at the site where it is actually used. In other words, the sound masking device 100 may be configured to include at least the sound masking processing unit 200.

(F-4) In the above embodiment of the sound masking device 100, a long-term frame is constructed using multiple divided frames, but a similar long-term frame may be obtained directly from the microphone input signal and stored in the input signal DB 204 (frame signal DB 205). In this case, the long-term pitch estimation information creation unit 213 may treat the long-term frame as one frame, perform a process to estimate the pitch using the same process as the divided frames, and store the estimation result as long-term pitch estimation information in the input signal DB 204 (pitch estimation information DB 214).

100, 100A, 100B, 100C, 100D...sound masking device, 101...microphone, 102...microphone amplifier, 103...AD converter, 104...DA converter, 106...speaker, 105...speaker amplifier, 200, 200A, 200B, 200C, 200D...sound masking processing unit, 201...frame division unit, 202...long-term frame signal creation unit 203, 203A, 203C...DB writing unit, 204, 204C...input signal DB, 205...frame signal DB, 2 06, 206A, 206C...frame signal selection unit, 207, 207D...delay time calculation unit, 208, 208A...masker signal generation unit, 209...voice section determination unit, 210, 210B...DB accumulation determination unit, 211, 211B...masker signal generation determination unit, 212...pitch estimation unit, 213...long-term pitch estimation information creation unit, 214...pitch estimation information DB, IN...sound input terminal, OUT...sound output terminal, 300...computer, 301...processor, 302...primary memory unit, 303...secondary memory unit.

Claims (8)

an input signal storage means for storing a microphone input signal supplied from a microphone that collects the voice of a target speaker in storage frame units of a predetermined length that can be recognized as a voice signal;
a signal selection means for selecting a masker segment signal to be used for generating a masker signal that makes the speech of the target speaker difficult to hear from the past storage frames stored in the input signal storage means while the target speaker is speaking, and outputting the selection result;
a masker signal generating means for generating and outputting the masker signal by using the masker segment signal selected by the signal selecting means while the target speaker is speaking . a pitch estimation means for calculating a pitch estimate of the microphone input signal,
the input signal storage means stores the respective storage frames together with pitch estimation information based on the pitch estimation value calculated by the pitch estimation means,
The sound processing device according to claim 1, characterized in that the signal selection means acquires an accumulation frame pitch, which is a value indicating the pitch of the accumulation frame created based on the pitch estimation information of each of the accumulation frames, and selects one of the accumulation frames as the masker segment signal based on a comparison result between the pitch estimation value of the current microphone input signal and the accumulation frame pitch of each of the accumulation frames. The sound processing device according to claim 2, characterized in that the signal selection means selects the accumulation frame having the accumulation frame pitch closer to the pitch estimate value of the current microphone input signal as the masker segment signal. a delay time calculation means for calculating a delay time for delaying the masker segment signal,
The sound processing device according to any one of claims 1 to 3, characterized in that the masker signal generating means generates the masker signal using a delayed signal obtained by delaying the masker segment signal selected by the signal selecting means by the delay time calculated by the delay time calculating means. The sound processing device according to claim 4, characterized in that the delay time calculation means calculates the delay time based on the current microphone input signal and the masker segment signal selected by the signal selection means. The sound processing device according to any one of claims 1 to 5, further comprising a speaker that emits the masker signal output by the masker signal generating means toward a masking subject other than the target speaker. Computer,
an input signal storage means for storing a microphone input signal supplied from a microphone that collects the voice of a target speaker in storage frame units of a predetermined length that can be recognized as a voice signal;
a signal selection means for selecting a masker segment signal to be used for generating a masker signal that makes the speech of the target speaker difficult to hear from the past storage frames stored in the input signal storage means while the target speaker is speaking, and outputting the selection result;
a masker signal generating means for generating and outputting the masker signal by using the masker segment signal selected by the signal selecting means while the target speaker is speaking . In a sound processing method performed by a sound processing device,
The input signal storage means includes a signal selection means and a masker signal generation means.
the input signal storage means stores a microphone input signal supplied from a microphone that picks up the voice of a target speaker while the target speaker is speaking in storage frame units of a predetermined length that can be recognized as a voice signal;
the signal selection means selects a masker segment signal to be used for generating a masker signal that makes the speech of the target speaker difficult to hear from the past storage frames stored in the input signal storage means while the target speaker is speaking, and outputs the selection result;
the masker signal generating means generates and outputs the masker signal by using the masker segment signal selected by the signal selecting means while the target speaker is speaking .

Images