JP4702043B2 - Digital watermark encoding apparatus, digital watermark decoding apparatus, digital watermark encoding method, digital watermark decoding method, and program - Google Patents

Description

  The present invention relates to a digital watermark encoding apparatus, a digital watermark decoding apparatus, a digital watermark encoding method, a digital watermark decoding method, and a program.

As a technique for synthesizing speech, there is a technique called a recording editing system. The recording / editing system is used in a station voice guidance system, an in-vehicle navigation system, and the like.
The recording and editing method associates a word with voice data representing a voice that reads out the word, divides a sentence to be synthesized into words, and acquires voice data associated with these words. This is a technique of joining them together (for example, see Patent Document 1).
JP 10-49193 A

  As a technique for making it possible to change the speaker of the synthesized speech obtained by the recording and editing method, or to diversify the synthesized speech obtained, record the speech data on removable media (portable recording media). As one to use, a plurality of removable media in which different audio data are recorded may be replaced as necessary. However, there is a problem that the audio data recorded on the removable medium is likely to be illegally copied, falsified or otherwise illegally used.

  The present invention has been made in view of the above circumstances, and is an electronic watermark encoding apparatus, electronic watermark decoding apparatus, electronic watermark encoding for effectively protecting audio data while freely supplying data representing sound. It is an object to provide a method, a digital watermark decoding method, and a program.

In order to achieve the above object, a digital watermark encoding apparatus according to the first aspect of the present invention provides:
An audio signal acquisition means for acquiring an audio signal representing the audio;
Audio signal processing for processing the acquired audio signal so that the pattern on the time axis of the frequency shift of the pitch component of the audio represented by the audio signal represents the value of data to be embedded as a digital watermark Means,
It is characterized by that.

The audio signal processing means is, for example,
Pitch component extraction means for extracting a pitch component signal representing the pitch component of the voice represented by the voice signal from the acquired voice signal;
A non-pitch component extracting means for extracting a non-pitch component signal representing a component other than the pitch component of the sound represented by the sound signal from the acquired sound signal;
Pitch component frequency shift means for acquiring the pitch component signal and shifting the frequency of the pitch component signal so as to have a pattern on the time axis that represents the value of data to be embedded as a digital watermark;
What is necessary is just to have an adding means for generating a digital watermarked audio signal by obtaining the pitch component signal subjected to the frequency shift and the non-pitch component signal and adding them together. .

The pitch component extraction means is, for example,
Pitch frequency specifying means for specifying the frequency of the pitch component of the voice represented by the voice signal;
What is necessary is just to include a bandpass filter that extracts a component in the vicinity of the identified frequency as the pitch component of the audio signal,
The non-pitch component extraction unit is configured by a band rejection filter that extracts a residual component as the non-pitch component by removing a component near the specified frequency from the audio signal, for example. I just need it.

A digital watermark decoding apparatus according to the second aspect of the present invention provides:
An audio signal acquisition means for acquiring an audio signal to which a digital watermark is applied, which represents audio;
Pitch component extraction means for extracting a pitch component signal representing the pitch component of the voice represented by the voice signal from the acquired voice signal;
Means for specifying a frequency shift pattern on the time axis of the extracted pitch component signal, and specifying data embedded as a digital watermark in the audio signal based on the specified pattern. ,
It is characterized by that.

A digital watermark encoding method according to a third aspect of the present invention is a digital watermark encoding method executed in an apparatus having an audio signal acquisition unit and an audio signal processing unit,
An audio signal acquisition step in which the audio signal acquisition unit acquires an audio signal representing the audio;
The audio signal processing unit, the acquired audio signal, and that the pattern on the time axis of the shift of the frequency of the pitch component of the speech to which the sound signal is represented, represents the value of the data to be embedded as an electronic watermark and the audio signal processing step of processing so that,
And wherein a call with a.

A digital watermark decoding method according to a fourth aspect of the present invention is a digital watermark decoding method executed in an apparatus having an audio signal acquisition unit, a pitch component extraction unit, and a specification unit,
The audio signal acquisition unit, representing sound, an audio signal acquiring step of acquiring speech signal subjected to digital watermark,
The pitch component extraction unit, wherein the acquired voice signal, the pitch component extraction step of extracting a pitch component signal representing a pitch component of the speech to which the sound signal is represented,
The specific portion, wherein a is extracted pitch component signal to identify the pattern of the frequency shift on the time axis, based on the specified pattern has been embedded as a digital watermark into the audio signal data Specific steps to identify ,
And wherein a call with a.

A program according to the fifth aspect of the present invention is
Computer
An audio signal acquisition means for acquiring an audio signal representing the audio;
Audio signal processing for processing the acquired audio signal so that the pattern on the time axis of the frequency shift of the pitch component of the audio represented by the audio signal represents the value of data to be embedded as a digital watermark Means,
It is for making it function.

A program according to the sixth aspect of the present invention is
Computer
An audio signal acquisition means for acquiring an audio signal to which a digital watermark is applied, which represents audio;
Pitch component extraction means for extracting a pitch component signal representing the pitch component of the voice represented by the voice signal from the acquired voice signal;
Means for specifying a frequency shift pattern on the time axis of the extracted pitch component signal, and specifying data embedded as a digital watermark in the audio signal based on the specified pattern;
It is for making it function.

  According to the present invention, there are provided a digital watermark encoding device, a digital watermark decoding device, a digital watermark encoding method, a digital watermark decoding method, and a program for effectively protecting audio data while freely providing data representing audio. Realized.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings, taking a digital watermark encoding apparatus and a digital watermark decoding apparatus as examples.

(Digital watermark encoding device)
FIG. 1 is a diagram showing a configuration of a digital watermark encoding apparatus E according to an embodiment of the present invention. As shown in the figure, the digital watermark encoding device E includes an audio data input unit E1, a pitch specifying unit E2, a BRF (band rejection filter) E3, a BPF (bandpass filter) E4, a pitch shift unit E5, And an adder E6.

The audio data input unit E1, pitch identification unit E2, BRFE3, BPFE4, pitch shift unit E5, and addition unit E6 are, for example, a processor such as a CPU (Central Processing Unit) or a memory that stores a program to be executed by the processor. Etc.
A single processor may perform a part or all of the functions of the audio data input unit E1, the pitch identification unit E2, BRFE3, BPFE4, the pitch shift unit E5, and the addition unit E6.

  The voice data input unit E1 obtains input voice data in a digital format (for example, PCM (pulse code modulation) format) representing a voice waveform from the outside, and supplies the input voice data to the pitch specifying unit E2, BRFE3, and BPFE4.

  Note that the voice data input unit E1 may acquire any input voice data. For example, the voice data input unit E1 may acquire the voice data from an external device (for example, a hard disk device) or a network via an interface circuit (not shown). You may read from the recording medium (for example, a flexible disk, CD-ROM, etc.) set to the recording medium drive device which does not carry out via this recording medium drive device.

  The pitch specifying unit E2 specifies the time change of the frequency of the pitch component of the voice represented by the input voice data supplied from the voice data input unit E1, and represents the specified time change (hereinafter referred to as pitch frequency data). Are supplied continuously to BRFE3 and BPFE4 simultaneously.

  Specifically, the pitch specifying unit E2 may specify the time change of the frequency of the pitch component, for example, by performing cepstrum analysis on the input voice data. That is, for example, the waveform represented by the input voice data is divided into a number of small parts on the time axis, and the intensity of each obtained small part is substantially equal to the logarithm of the original value (the base of the logarithm is arbitrary). Convert this value into a value, and then convert this small portion of the spectrum (ie, the cepstrum) to a Fast Fourier Transform technique (or any other technique that produces data representing the result of Fourier transforming a discrete variable) ) Then, the minimum value among the frequencies giving the maximum value of the cepstrum is specified as the frequency of the pitch component in this small portion.

  The time change of the frequency of the pitch component is specified based on the pitch waveform data after converting the input voice data into the pitch waveform data according to the method disclosed in Japanese Patent Laid-Open No. 2003-108172, for example. A good result can be expected. Specifically, the input voice data is filtered to extract a pitch signal, and based on the extracted pitch signal, the waveform represented by the input voice data is divided into sections of unit pitch length. The input speech data may be converted into a pitch waveform signal by identifying the phase shift based on the correlation between the two and aligning the phases of the sections. Then, the obtained pitch waveform signal is handled as input voice data, and a cepstrum analysis is performed, for example, so that the time change of the frequency of the pitch component may be specified.

BRFE3 performs filtering so as to substantially cut off the center frequency of the signal input to itself and the frequency components in the vicinity thereof and allow other components to pass therethrough.
Specifically, the self center frequency is set to be substantially equal to the value indicated by the pitch frequency data supplied from the pitch specifying unit E2 (that is, the current value of the frequency of the pitch component of the input audio data). . Then, the input voice data supplied from the voice data input unit E1 is filtered, and the filtered input voice data (hereinafter referred to as a non-pitch component) is supplied to the adding unit E6.

The BPFE 4 performs filtering so as to pass components of its own center frequency and frequencies in the vicinity of the signal inputted to itself and substantially block other components.
Specifically, the self center frequency is set to be substantially equal to the value indicated by the pitch frequency data supplied from the pitch specifying unit E2. Then, the input voice data supplied from the voice data input unit E1 is filtered, and the filtered input voice data (hereinafter referred to as pitch component) is supplied to the pitch shift unit E5.

It is assumed that BRFE3 and BPFE4 perform filtering so that the non-pitch component and the pitch component have the relationship that when the two are added together, substantially the same audio data as the input audio data is obtained.
The non-pitch component and the pitch component are both composed of the same type of data. For example, a digital format having a sampling interval substantially the same as the sampling interval of the input audio data supplied to BRFE3 and BPFE4. It shall consist of data.

  The pitch shift unit E5 obtains the pitch component from the BPFE 4, while the binary bit string to be embedded from the outside in the form of a digital watermark (for example, data representing the creator name of the input audio data as a binary bit string) To get. Then, by shifting the frequency of the acquired pitch component according to each bit value in the bit string, the pitch component is encoded so as to include information represented by the bit string. The encoded pitch component is supplied to the adding unit E6.

  Specifically, for example, as shown in FIG. 2, the pitch shift unit E5 determines the value of each bit in order from the top of the bit string, and if the value of the bit is “1”, the pitch shift unit E5 determines a certain time (FIG. 2). The frequency of the pitch component is shifted by a certain width (the frequency indicated as “Δf” in FIG. 2) by the time indicated as “ΔT” in FIG. 2, and if the value is “0”, the frequency of the pitch component is increased during the time ΔT. The operation of not shifting may be continued until the last bit of the bit string is reached.

  The frequency shift of the pitch component is, for example, by generating a local oscillation signal having a frequency equal to the above-described value Δf, mixing the local oscillation signal and the pitch component, and filtering the obtained signal, This may be done by extracting a component having a frequency corresponding to the sum or difference between the frequency of the local oscillation signal and the frequency of the pitch signal.

  Further, the above-described value Δf is always a fixed amount (described later) at the boundary between the section where the frequency shift is performed and the section where the frequency shift is not performed among the pitch components as a result of the frequency shift of the pitch component by the pitch shift unit E5. It is assumed that the value is sufficiently large so as to cause a frequency difference equal to or greater than the amount shown as “ΔF” in FIG. The frequency Δf that satisfies this condition may be determined empirically based on, for example, experiments.

  The technique by which the pitch shift unit E5 acquires the bit string is arbitrary, and for example, the bit string may be acquired by a technique that is substantially the same as the technique by which the audio data input unit E1 acquires the input audio data. In addition, a processor that performs processing for realizing the function of the pitch shift unit E5 performs processing for realizing the function of the pitch shift unit E5 using a bit string generated by other processing executed by itself as a bit string embedded in the form of a digital watermark. You may make it hand over.

  The adder E6 acquires the non-pitch component supplied from BRFE3, and also acquires this pitch component while the pitch component is supplied from BPFE4. The output audio data having a value corresponding to the sum of the values of the non-pitch component and the pitch component acquired at the same time (provided that the value of the pitch component is 0 while the pitch component is not supplied). Is generated and output. The output audio data is assumed to be composed of data in the same format as the input audio data acquired by the audio data input unit E1.

  The output audio data is equivalent to the input audio data acquired by the audio data input unit E1 and digital watermark is applied, and the information represented by the bit string acquired by the pitch shift unit E5 is embedded. It can be said that there is.

  Note that the adding unit E6 can output the output audio data in any manner. For example, the adding unit E6 may store the output audio data in a storage area of an external device (for example, a hard disk device) via an interface circuit (not shown) or to a network. May be sent. Further, the recording medium may be written to a recording medium set in a recording medium drive device (not shown) via the recording medium drive device.

(Digital watermark decoding device)
FIG. 3 is a diagram showing the configuration of the digital watermark decoding apparatus D according to the embodiment of the present invention. As shown in the figure, the digital watermark decoding apparatus D is composed of an audio data input unit D1, a pitch specifying unit D2, a BPFD 3, and a bit string extraction unit D4.

  The audio data input unit D1, the pitch identification unit D2, the BPFD 3, and the bit string extraction unit D4 are configured by, for example, a processor such as a CPU and a memory that stores a program to be executed by the processor. Note that a single processor may perform a part or all of the functions of the audio data input unit D1, the pitch identification unit D2, the BPFD 3, and the bit string extraction unit D4.

  The audio data input unit D1 obtains, from the outside, digital audio data (hereinafter referred to as decoding target audio data) to which digital watermarking is applied, corresponding to the output audio data described above, and the pitch specifying unit D2 and Supply to BPFD3. The method by which the audio data input unit D1 acquires the decoding target audio data is arbitrary. For example, the audio data input unit E1 of the digital watermark encoding device E described above is substantially the same as the method by which the input audio data is acquired. Get it.

  The pitch specifying unit D2 is substantially the same as the technique in which the pitch specifying unit E2 of the digital watermark encoding apparatus E performs the time change of the frequency of the pitch component of the voice represented by the decoding target audio data supplied from the audio data input unit D1. Specify by the same method. And the pitch frequency data showing the specified said time change is continuously supplied to BPFD3.

  Note that the pitch specifying unit D2 interpolates a portion corresponding to a section of the audio data to be decoded that does not include a pitch component in the pitch frequency data by a method such as Lagrange interpolation or linear interpolation, for example. do it.

The BPFD 3 performs substantially the same function as the BPFE 4 of the digital watermark encoding apparatus E, for example, and passes the center frequency of the signal input to itself and components in the vicinity thereof, Filtering is performed so as to substantially block the components.
The filtered decoding target audio data (hereinafter referred to as a decoding pitch component) is supplied to the bit string extraction unit D4.

  The bit string extraction unit D4 acquires the decoding pitch component supplied from the BPFD 3, and extracts the bit string embedded in the decoding target audio data based on the frequency shift pattern of the decoding pitch component. Then, the extracted bit string is output to the outside.

  Specifically, for example, when the audio data to be decoded has been subjected to the digital watermark according to the above-described aspect with respect to the pitch component with reference to FIG. 2, the bit string extraction unit D4, for example, as shown in FIG. Whether the frequency of the pitch component immediately after the time point is higher by ΔF or lower than the frequency of the pitch component immediately before or immediately after the time point that has arrived from the beginning of the decoding pitch component every time ΔT (The example shown in FIG. 4 represents a case where ΔF is a positive value). When it is determined that it is higher than ΔF, it is determined that the section for the time ΔT after the time point represents the bit having the value “1”, and the bit having the value “1” is generated. When it is determined that it is lower than ΔF, it is determined that a section of time ΔT after that time point represents a bit having a value “0”, and a process of generating a bit having a value “0” is sequentially performed. By doing so, the bit string may be extracted.

  Note that the bit string extraction unit D4 can output the bit string in any manner. For example, the bit string extraction unit D4 may store the bit string in a storage area of an external device via an interface circuit (not shown) or send the bit string to a network. Further, the recording medium may be written to a recording medium set in a recording medium drive device (not shown) via the recording medium drive device.

  The digital watermark encoding apparatus E described above separates audio data into a pitch component and a non-pitch component, shifts the frequency of the pitch component according to the value of the bit string to be embedded, and then adds the pitch component and the non-pitch component. As a result, the information represented by the bit string is embedded in the audio data (that is, the audio data is digitally watermarked). Then, the digital watermark decoding device D extracts the bit string embedded in the audio data by the digital watermark encoding device E from the audio data.

  In general, the frequency of the pitch component of the voice uttered by human beings is around 200 [Hz], while the human audible range is in the range of about 20 to 20000 [Hz], but the frequency around 200 [Hz]. The sensitivity of the component is low, and even if a change is caused in such a component, it is difficult to distinguish this change. Therefore, the difference between the audio reproduced using the audio data that has been digitally watermarked by the digital watermark encoding apparatus E and the audio reproduced using the audio data before the digital watermarking is Is difficult to identify. In other words, the influence of the digital watermark by the digital watermark encoding apparatus E on the sound is suppressed to a negligible level.

The digital watermark encoding device E applies digital watermarking by partially shifting the frequency of the pitch component of the audio data. Therefore, a person who does not know the width of the shift cannot completely restore the audio data before being subjected to the digital watermark from the audio data subjected to the digital watermark by the digital watermark encoding apparatus E.
Also, it is possible to extract an embedded bit string from data obtained by analog copying audio data that has been digitally watermarked by the digital watermark encoding device E.

The configurations of the digital watermark encoding device E and the digital watermark decoding device D are not limited to those described above.
For example, the audio data input unit E1 may include a microphone, an amplifier, a sampling circuit, an A / D (Analog-to-Digital) converter, a PCM encoder, and the like. In this case, the audio data input unit E1 amplifies the audio signal representing the audio collected by its own microphone, performs sampling and A / D conversion, and then performs PCM modulation on the sampled audio signal, thereby generating audio. Data may be created.

Further, the rule by which the pitch shift unit E5 shifts the frequency of the pitch component is not limited to that shown in FIG. Therefore, for example, as shown in FIG. 5, the value of each bit is determined in order from the beginning of the bit string to be embedded. Shift in the direction opposite to the previous shift direction (ie, if the previous shift is Δf, shift by (−Δf), if the previous shift is (−Δf), shift by Δf), and the value is “ If it is “0”, the operation of continuing the above-mentioned shift of the constant width in the same direction as the previous shift during the time ΔT may be continued until the last bit of the bit string is reached.
In this case, for example, as shown in FIG. 6, the bit string extraction unit D4 increases the frequency of the pitch component before and after the time point by more than ± ΔF every time when the time point arrives from the beginning of the decoding pitch component every time ΔT. What is necessary is just to discriminate | determine whether there exists a difference. When it is determined that there is a difference of ± ΔF or more, it is determined that the change in the frequency of the pitch component at that time point represents the bit having the value “1”, and the bit having the value “1” is generated If it is determined that there is no difference of ± ΔF or more, it is determined that the change in the frequency of the pitch component at that time point represents the bit having the value “0”, and the bit having the value “0”. The bit string may be extracted by sequentially performing the process of generating.

  Although the embodiment of the present invention has been described above, the digital watermark encoding apparatus and the digital watermark decoding apparatus according to the present invention can be realized using a normal computer system, not a dedicated system.

  For example, a recording medium (CD-ROM, flexible disk) storing a program for causing a personal computer to execute the operations of the audio data input unit E1, pitch identification unit E2, BRFE3, BPFE4, pitch shift unit E5, and addition unit E6. Etc.), the digital watermark encoding apparatus E that executes the above-described processing can be configured.

  Further, the program is stored from a recording medium (CD-ROM, flexible disk, etc.) that stores a program for causing the personal computer to execute the operations of the audio data input unit D1, pitch identification unit D2, BPFD3, and bit string extraction unit D4. By installing, it is possible to configure the digital watermark decoding apparatus D that executes the above-described processing.

Note that a program for causing a personal computer to perform the function of the digital watermark encoding device E or the digital watermark decoding device D may be uploaded to a bulletin board (BBS) of a communication line and distributed via the communication line, for example. Further, the carrier wave may be modulated with a signal representing these programs, the obtained modulated wave may be transmitted, and the apparatus that receives the modulated wave may demodulate the modulated wave to restore these programs.
The above-described processing can be executed by starting up these programs and executing them under the control of the OS in the same manner as other application programs.

  When the OS shares a part of the processing, or when the OS constitutes a part of one component of the present invention, a program excluding the part is stored in the recording medium. May be. Also in this case, in the present invention, it is assumed that the recording medium stores a program for executing each function or step executed by the computer.

It is a block diagram which shows the structure of the digital watermark encoding apparatus which concerns on embodiment of this invention. It is a graph which shows the aspect by which a pitch component is processed. It is a block diagram which shows the structure of the digital watermark decoding apparatus concerning embodiment of this invention. It is a graph explaining the aspect from which a bit string is extracted. It is a graph which shows the other aspect by which a pitch component is processed. It is a graph explaining the other aspect from which a bit sequence is extracted.

Explanation of symbols

E Digital watermark encoding device E1 Audio data input unit E2 Pitch identification unit E3 BRF
E4 BPF
E5 Pitch shift unit E6 Addition unit D Digital watermark decoding device D1 Audio data input unit D2 Pitch identification unit D3 BPF
D4 Bit string extraction unit

Claims (8)

An audio signal acquisition means for acquiring an audio signal representing the audio;
Audio signal processing for processing the acquired audio signal so that the pattern on the time axis of the frequency shift of the pitch component of the audio represented by the audio signal represents the value of data to be embedded as a digital watermark Means,
A digital watermark encoding apparatus characterized by the above. The voice signal processing means is
Pitch component extraction means for extracting a pitch component signal representing the pitch component of the voice represented by the voice signal from the acquired voice signal;
A non-pitch component extracting means for extracting a non-pitch component signal representing a component other than the pitch component of the sound represented by the sound signal from the acquired sound signal;
Pitch component frequency shift means for acquiring the pitch component signal and shifting the frequency of the pitch component signal so as to have a pattern on the time axis that represents the value of data to be embedded as a digital watermark;
Adding a frequency component shifted pitch component signal and a non-pitch component signal, and adding them together to generate a digital watermarked audio signal;
The digital watermark encoding apparatus according to claim 1. The pitch component extraction means includes
Pitch frequency specifying means for specifying the frequency of the pitch component of the voice represented by the voice signal;
A band-pass filter that extracts a component in the vicinity of the identified frequency as the pitch component of the audio signal,
The non-pitch component extraction unit is configured by a band rejection filter that extracts a residual component as the non-pitch component by removing a component near the specified frequency from the audio signal.
The digital watermark encoding apparatus according to claim 2. An audio signal acquisition means for acquiring an audio signal to which a digital watermark is applied, which represents audio;
Pitch component extraction means for extracting a pitch component signal representing the pitch component of the voice represented by the voice signal from the acquired voice signal;
Means for specifying a frequency shift pattern on the time axis of the extracted pitch component signal, and specifying data embedded as a digital watermark in the audio signal based on the specified pattern. ,
A digital watermark decoding apparatus characterized by the above. An electronic watermark encoding method executed in an apparatus having an audio signal acquisition unit and an audio signal processing unit,
An audio signal acquisition step in which the audio signal acquisition unit acquires an audio signal representing the audio;
The audio signal processing unit, the acquired audio signal, and that the pattern on the time axis of the shift of the frequency of the pitch component of the speech to which the sound signal is represented, represents the value of the data to be embedded as an electronic watermark and the audio signal processing step of processing so that,
Watermarking encoding method characterized that you provided with. An electronic watermark decoding method executed in an apparatus having an audio signal acquisition unit, a pitch component extraction unit, and a specification unit,
The audio signal acquisition unit, representing sound, an audio signal acquiring step of acquiring speech signal subjected to digital watermark,
The pitch component extraction unit, wherein the acquired voice signal, the pitch component extraction step of extracting a pitch component signal representing a pitch component of the speech to which the sound signal is represented,
The specific portion, wherein a is extracted pitch component signal to identify the pattern of the frequency shift on the time axis, based on the specified pattern has been embedded as a digital watermark into the audio signal data Specific steps to identify ,
Electronic watermark decoding method characterized that you provided with. Computer
An audio signal acquisition means for acquiring an audio signal representing the audio;
Audio signal processing for processing the acquired audio signal so that the pattern on the time axis of the frequency shift of the pitch component of the audio represented by the audio signal represents the value of data to be embedded as a digital watermark Means,
Program to make it function. Computer
An audio signal acquisition means for acquiring an audio signal to which a digital watermark is applied, which represents audio;
Pitch component extraction means for extracting a pitch component signal representing the pitch component of the voice represented by the voice signal from the acquired voice signal;
Means for specifying a frequency shift pattern on the time axis of the extracted pitch component signal, and specifying data embedded as a digital watermark in the audio signal based on the specified pattern;
Program to make it function.

Images