JP6372066B2 - Synthesis information management apparatus and speech synthesis apparatus - Google Patents

Description

  The present invention relates to a technique for synthesizing speech.

  When a plurality of phonemes are continuously generated, a phenomenon (hereinafter referred to as “speech change”) occurs in which each phoneme changes due to the influence of preceding and following phonemes. Dropping (disappearing) where pronunciation of a specific phoneme is omitted and assimilation in which each phoneme changes to a phoneme similar to the preceding and following phonemes are typical examples of speech changes. For example, if “out” ([aU] [t]) and “door” ([dh] [O @]) are pronounced continuously, the phoneme [t] at the end of “out” is dropped And pronounced [aU] [dh] [O @]. As described above, in this specification, the phoneme symbol of each phoneme is described in a format compliant with X-SAMPA (eXtrended-Speech Assessment Methods Phonetic Alphabet).

  On the other hand, a speech synthesis technique for generating speech that pronounces an arbitrary character string has been proposed. In order to synthesize perceptually natural speech, it is important to reproduce the aforementioned speech changes that occur during actual pronunciation. Against the background described above, Patent Document 1 discloses that a phoneme that falls out or assimilated by omitting or changing a phoneme that satisfies a condition previously defined in the sound change rule dictionary among a plurality of phonemes in time series. A technique for synthesizing speech in which changes are reproduced is disclosed.

JP 2011-175074 A

  However, under the technique of Patent Document 1, a sound change is uniformly applied to phonemes that satisfy the conditions specified in the sound change rule dictionary, and a sound change is not uniformly applied to phonemes that do not satisfy the conditions. Therefore, an unnatural voice change that does not match the actual pronunciation tendency may be given. For example, voice changes are likely to occur when a plurality of phonemes are pronounced quickly at short intervals, but voice changes are unlikely to occur when a plurality of phonemes are pronounced late or at a sufficient interval. Tend. However, in the technique of Patent Document 1, when the sound change rule dictionary is matched, a sound change is given regardless of the speed and interval of pronunciation of each phoneme.

  A configuration in which the user instructs whether or not there is a voice change for each phoneme can be assumed, but in order to appropriately determine the phoneme to which the voice change is actually to be applied, specialized knowledge about the voice change is necessary. There is also a problem that the work load for instructing whether or not there is a voice change for each phoneme is excessive. In view of the above circumstances, an object of the present invention is to realize a natural voice change while suppressing a user's load.

  The synthesis information management device according to the first aspect of the present invention is a device that manages synthesis information that specifies the pronunciation content of a synthesis target speech, and is a synthesis target that is a speech change index that is an index of the likelihood of speech change. Index calculation means for calculating the phoneme of speech, whether to generate a speech change for the phoneme of the synthesis target speech, according to the speech change index of the phoneme, and for the phoneme determined to generate a speech change The information management means for not setting the occurrence of the voice change in the synthesis information for the phonemes determined not to generate the voice change. In the above configuration, the speech change index is calculated for the phoneme of the synthesis target speech specified by the synthesis information, and the presence or absence of the phoneme change of the phoneme is controlled according to the speech change index. Therefore, when compared with the technique of Patent Document 1 in which the presence or absence of a sound change is uniformly determined only depending on whether or not each phoneme meets a predetermined condition, for example, an operation for instructing the presence or absence of a sound change for each phoneme It is possible to realize a natural voice change that matches the actual pronunciation tendency, while suppressing the user's load due to.

  The speech synthesizer according to the second aspect of the present invention is a device that generates a speech signal of a synthesis target speech whose synthesis information specifies the pronunciation content, and a speech change index that is an index of the likelihood of speech change. Index calculation means for calculating the phoneme of the synthesis target speech and means for generating a speech signal corresponding to the synthesis information, and whether or not a speech change is generated for the phoneme of the synthesis target speech is used as the speech change index of the phoneme A sound signal that generates a sound signal for a phoneme that is determined to generate a sound change, and generates a sound signal that does not generate a sound change for a phoneme that is determined not to generate a sound change Synthesizing means. In the above configuration, the speech change index is calculated for the phoneme of the synthesis target speech specified by the synthesis information, and the presence or absence of the phoneme change of the phoneme is controlled according to the speech change index. Therefore, as in the first aspect, for example, it is possible to realize a natural voice change that matches the actual pronunciation tendency while suppressing the user's load due to the work of instructing the presence or absence of the voice change for each phoneme. It is. A specific example of the second mode will be described later as a third embodiment, for example.

  In a preferred example of the first aspect and the second aspect of the present invention, the index calculating means is a synthesis target based on reference information that designates a basic index that is an index of the likelihood of occurrence of speech change for each combination of two phonemes. A basic index corresponding to a combination of two phonemes of speech is specified, and a speech change index is calculated according to the basic index. In the above aspect, the basic index corresponding to the combination of the two phonemes of the synthesis target speech is specified from the reference information designating the basic index for each combination of two phonemes, and applied to the calculation of the speech change index. From the above, it reflects the tendency that the probability of speech change differs depending on the combination of two phonemes (there is a combination of phonemes that are likely to cause speech changes and a combination of phonemes that are unlikely to cause speech changes). It is possible to realize natural voice changes.

  In the preferred embodiments of the first and second aspects of the present invention, the synthesis information specifies the duration and the pronunciation content for each speech unit of the synthesis target speech, and the index calculation means is the phoneme duration of the synthesis target speech. The voice change index is calculated according to In the above aspect, since the voice change index is calculated according to the duration of the voice unit, it is possible to reproduce the actual pronunciation tendency that a voice change is likely to occur when a plurality of phonemes are quickly pronounced. It is.

  In a preferred example of the first aspect and the second aspect of the present invention, the synthesis information designates a pitch and a pronunciation content for each voice unit of the synthesis target voice, and the index calculation means includes the pitch of the phoneme of the synthesis target voice. The voice change index is calculated according to In the above aspect, since the voice change index is calculated according to the pitch of the voice unit, it is possible to realize a natural voice change reflecting the tendency that the voice change is more likely to occur as the pitch of the voice is higher. It is.

  In a preferred example of the first aspect and the second aspect of the present invention, the synthesis information includes control information representing a musical expression of the synthesis target speech, and the index calculation means is responsive to the phoneme control information of the synthesis target speech. Calculate voice change index. In the above aspect, since the voice change index is calculated according to the control information representing the musical expression of the synthesis target voice, it is possible to give a natural voice change suitable for the musical expression of the synthesis target voice. It is. In addition, the specific example of the above aspect is later mentioned as 4th Embodiment, for example.

  The preferred embodiments of the first aspect and the second aspect of the present invention include an instruction receiving unit that receives an instruction from a user, and the information management unit includes a voice change index calculated by the index calculating unit for the phoneme of the synthesis target voice, The instruction receiving unit determines whether or not to generate a sound change of the phoneme according to the magnitude of the threshold set according to the instruction received from the user. In the above aspect, since a threshold value to be compared with the voice change index for determining whether there is a voice change is set according to an instruction from the user, a voice change suitable for the user's intention and preference is realized. It is possible. In addition, the specific example of the above aspect is later mentioned, for example as 2nd Embodiment.

  The synthesis information management device and the speech synthesizer according to each aspect described above are realized by hardware (electronic circuit) such as a DSP (Digital Signal Processor) dedicated to editing synthesis information and generating a speech signal, as well as a CPU. It is also realized by cooperation between a general-purpose arithmetic processing device such as (Central Processing Unit) and a program. The program of the present invention can be provided in a form stored in a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and an optical recording medium (optical disk) such as a CD-ROM is a good example, but a known arbitrary one such as a semiconductor recording medium or a magnetic recording medium This type of recording medium can be included. For example, the program of the present invention can be provided in the form of distribution via a communication network and installed in a computer. The present invention is also specified as an operation method (synthesis information management method) of the synthesis information management apparatus and an operation method (voice synthesis method) of the speech synthesis apparatus according to each aspect described above.

1 is a block diagram of a speech synthesizer according to a first embodiment of the present invention. It is a schematic diagram of synthetic information. It is a schematic diagram of reference information. It is a schematic diagram of an edit image. It is a flowchart of operation | movement of a speech synthesizer. It is a flowchart of an edit process. It is a flowchart of an audio | voice change setting process. It is a specific example of calculation of a voice change index. It is explanatory drawing of the change of the score image according to an audio | voice change parameter | index. It is a schematic diagram of the edited image in 2nd Embodiment. It is a flowchart of the speech synthesis process in 3rd Embodiment. It is a schematic diagram of the synthesis information in the fourth embodiment.

<First Embodiment>
FIG. 1 is a configuration diagram of a speech synthesizer 100 according to the first embodiment of the present invention. The speech synthesizer 100 according to the first embodiment generates a speech signal V of a singing voice of an arbitrary piece of music (hereinafter referred to as “synthetic music”) by a unit connection type speech synthesis that connects a plurality of speech units. Device. When a plurality of phonemes are continuously pronounced, a speech signal V is generated that reproduces a phenomenon (speech change) in which each phoneme changes due to the influence of the preceding and following phonemes. In the first embodiment, generation of a voice signal V that reproduces a dropout in which pronunciation of a specific phoneme is omitted among various voice changes including assimilation and connection is exemplified.

  As illustrated in FIG. 1, the speech synthesizer 100 includes a computer system (for example, a mobile phone or a personal computer) that includes an arithmetic processing device 10, a storage device 12, a display device 14, an input device 16, and a sound emitting device 18. Information processing device). The display device 14 (for example, a liquid crystal display panel) displays an image instructed from the arithmetic processing device 10. The input device 16 is an operation device (for example, a pointing device such as a mouse or a keyboard) operated by the user for various instructions to the speech synthesizer 100, and includes a plurality of operators operated by the user, for example. Is done. Note that a touch panel configured integrally with the display device 14 may be employed as the input device 16. The sound emitting device 18 (for example, a speaker or headphones) reproduces sound according to the audio signal V.

  The storage device 12 stores a program executed by the arithmetic processing device 10 and various data used by the arithmetic processing device 10. A known recording medium such as a semiconductor recording medium or a magnetic recording medium or a combination of a plurality of types of recording media is arbitrarily employed as the storage device 12. The storage device 12 of the first embodiment stores a speech unit group L, synthesis information S, and reference information R as illustrated below.

  The speech segment group L is a set (speech library) of a plurality of speech segments collected in advance from recorded speech of a specific speaker. Each phoneme unit is a phoneme unit (for example, a vowel or a consonant) that is a minimum unit of distinction of linguistic meaning, or a phoneme chain in which a plurality of phonemes are connected. However, in the first embodiment, a phoneme chain (a diphone) in which two adjacent phonemes are connected is exemplified as a speech unit. Each speech segment is represented as a time series of a speech waveform sample sequence in the time domain or a frequency domain spectrum calculated for each frame of the speech waveform. In the following description, silence is positioned as one phoneme for the sake of convenience and is represented by the symbol “Sil”.

  The synthesis information S designates a voice that is a target of voice synthesis (hereinafter referred to as “synthesis target voice”). The synthesis information S of the first embodiment is time-series data that designates the singing voice of the synthesized music as the synthesis target voice, and as exemplified in FIG. 2, the pitch for each voice unit (every note) of the synthesis target voice. Designate X1, pronunciation period X2, and pronunciation content X3. The sound unit is a sound unit (segmental unit such as a syllable or a mora) assigned to one note of the synthesized music.

  The pitch X1 is the pitch of each note of the synthesized music, and is specified by, for example, a note number conforming to the MIDI (Musical Instrument Digital Interface) standard. The sound generation period X2 is a time length (note value) of a note, and is defined by, for example, a sound generation start time T1 and a continuation length (time length) T2. The continuation length T2 of the first embodiment is expressed by a numerical value of a tempo (BPM: Beats Per Minute) when a quarter note is a reference value (T2 = 120). For example, the duration T2 corresponding to an eighth note is set to 240, and the duration T2 corresponding to a half note is set to 60. That is, the continuation length T2 is set to a smaller numerical value as the duration of the sound generation period X2 is longer. A configuration in which the sound generation period X2 is defined by the start time T1 and the end time of sound generation (a configuration in which the duration T2 can be calculated according to the time length between the two times) is also suitable. The pronunciation content X3 is the content of pronunciation by the synthesis target speech, and includes a speech code (sounding character) YA representing a speech unit and a phoneme symbol YB of each phoneme constituting the speech unit. The voice code YA corresponds to a character (grapheme) constituting the lyrics of the synthesized music. As can be understood from the above description, the synthesis information S can be rephrased as data specifying the score of the synthesized music.

  The reference information R is data used to determine whether or not there is a speech change (phoneme dropout) in the synthesis target speech. As illustrated in FIG. 3, two reference phoneme arrays (hereinafter referred to as “phoneme pairs”). )), A pre-conversion pattern QA, a post-conversion pattern QB, and a basic index B are designated. That is, the reference information R can also be expressed as a data table that associates the pre-conversion pattern QA, the post-conversion pattern QB, and the basic index B with each phoneme pair. The reference information R of the first embodiment is a pre-conversion pattern QA and a post-conversion pattern QB for a phoneme pair that can cause a speech change (dropping) under the language of the synthesis target speech among a combination of two arbitrary phonemes. And basic indicator B.

  The pre-conversion pattern QA specifies a phoneme pair before the occurrence of a speech change, and the post-conversion pattern QB is after the occurrence of a speech change (that is, when a speech change is generated in the phoneme pair specified by the pre-conversion pattern QA). Specifies the phoneme pair. For example, if “out door” with “out” ([aU] [t]) and “door” ([dh] [O @]) is pronounced, the phoneme at the end of “out” [t ] Can be pronounced as [aU] [dh] [O @] due to a voice change (dropping). Therefore, as illustrated in FIG. 3, the phoneme pair [t-dh] in which the phoneme [t] at the end of “out” and the phoneme [dh] at the beginning of “door” are arranged is designated as the pre-conversion pattern QA. The phoneme pair [Sil-dh] obtained by converting the phoneme [t] into the silent phoneme [Sil] of the phoneme pair [t-dh] is designated as the post-conversion pattern QB. Similarly, for example, a phoneme pair [p-th] pre-conversion pattern QA included in “keep talk” ([kh] [i:] [p] [th] [o:] [k]) includes a phoneme. The converted pattern QB of the phoneme pair [Sil-th] obtained by converting [p] into a silent phoneme [Sil] corresponds to, for example, “pick part” ([ph] [I] [k] [ph] [Q @ ] [t]), the pattern QA before conversion of the phoneme pair [k-ph] includes the post-conversion pattern QB of the phoneme pair [Sil-ph] obtained by converting the phoneme [k] to the silent phoneme [Sil]. Corresponds.

  The basic index B in FIG. 3 is an index of the likelihood of a voice change from the pre-conversion pattern QA to the post-conversion pattern QB. In the first embodiment, the basic index B of the phoneme pair (pre-conversion pattern QA) that is likely to cause a voice change is set to a larger numerical value. Each basic index B is set experimentally or statistically by investigating the tendency of voice change from a large number of recorded voices (materials) recorded in advance. For example, the phoneme [t] of the phoneme pair [t-dh] is more likely to drop than the phoneme [k] of the phoneme pair [k-ph], and the phoneme [k] of the phoneme pair [k-ph] Tends to drop out compared to the phoneme [p] of the phoneme pair [p-th]. Considering the above trend, in the example of FIG. 3, the basic index B of the phoneme pair [t-dh] exceeds the basic index B of the phoneme pair [k-ph], and the basic index of the phoneme pair [k-ph]. B exceeds the basic index B of phoneme pair [p-th].

  The arithmetic processing unit 10 (CPU) in FIG. 1 executes a program stored in the storage unit 12 to execute a plurality of functions (instruction receiving unit 22, display) for editing the synthesis information S and generating the audio signal V. The control unit 24, the index calculation unit 26, the information management unit 32, and the speech synthesis unit 34) are realized. A configuration in which each function of the arithmetic processing device 10 is distributed to a plurality of devices, or a configuration in which a dedicated electronic circuit (for example, DSP) realizes a part of the functions of the arithmetic processing device 10 may be employed.

  The display control unit 24 displays various images on the display device 14. Specifically, the display control unit 24 of the first embodiment causes the display device 14 to display the edited image 40 of FIG. 4 for the user to confirm and edit the contents of the synthesized music designated by the synthesis information S. As illustrated in FIG. 4, the edited image 40 of the first embodiment includes a score image 42 that represents the content of the synthesized music.

  In the musical score image 42, a musical note image 44 representing each musical note designated by the synthesis information S is arranged in time series on a coordinate plane in which a time axis (horizontal axis) and a pitch axis (vertical axis) intersecting each other are set. This is a piano roll type image. The position of the note image 44 in the direction of the pitch axis is set according to the pitch X1 specified by the synthesis information S, and the position and display length of the note image 44 in the direction of the time axis are pronounced by the synthesis information S. It is set according to the period X2. Each musical note iconic image 44 is added with a voice code YA and a phoneme symbol YB of the pronunciation content X3 designated by the synthesis information S. Note that the positions where the voice code YA and the phoneme symbol YB are displayed are not limited to the example shown in FIG. 4 (inside the musical note image 44).

  The instruction receiving unit 22 in FIG. 1 receives an instruction from a user according to an operation on the input device 16. For example, the user can instruct the speech synthesizer 100 to edit the synthesis information S by appropriately operating the input device 16 while confirming the edited image 40. The instruction receiving unit 22 receives an instruction to edit the composite information S from the user.

  The index calculation unit 26 calculates the voice change index A for the phoneme corresponding to the pronunciation content X3 specified by the synthesis information S. The voice change index A is an index of the likelihood of occurrence of a voice change. In the first embodiment, a larger numerical value is set for the sound change index A of a phoneme pair in which a sound change is likely to occur. The index calculation unit 26 of the first embodiment changes the sound for each phoneme pair that matches any of the pre-conversion patterns QA registered in the reference information R among the time series of a plurality of phonemes specified by the pronunciation content X3. Indicator A is calculated.

  As described above, the basic index B specified by the reference information R is an index of the likelihood of a voice change from the pre-conversion pattern QA to the post-conversion pattern QB. Therefore, the index calculation unit 26 according to the first embodiment uses the speech change index A of the phoneme pair that matches the pre-conversion pattern QA among the time series of the plurality of phonemes of the synthesis target speech specified by the synthesis information S as the reference information R. Of these, calculation is performed according to the basic index B corresponding to the pre-conversion pattern QA. Specifically, the index calculation unit 26 sets the voice change index A according to the basic index B so that the voice change index A becomes a larger value as the basic index B is larger (the voice change is more likely to occur in the phoneme pair). Is calculated.

  When a plurality of phonemes are pronounced quickly and continuously (when the duration of each phoneme is short), there is a general tendency that a voice change is likely to occur for the phoneme. In consideration of the above tendency, the index calculating unit 26 of the first embodiment uses the synthesis information S for the phoneme pairs corresponding to the phoneme pairs as the speech change index A of each phoneme pair of the synthesis target speech specified by the synthesis information S. Is calculated according to the duration T2 of the pronunciation period X2 specified by. Specifically, the index calculation unit 26 corresponds to the duration T2 of the note to which the front phoneme (that is, the phoneme to be dropped) is assigned among the two phonemes of the phoneme pair that matches the pre-conversion pattern QA. The voice change index A is calculated. For example, the voice change index A is set to a larger value as the duration T2 is shorter in time (the value of the duration T2 is larger).

  Further, there is a general tendency that the higher the pitch of the voice, the more likely the voice change to occur. In consideration of the above tendency, the index calculating unit 26 of the first embodiment uses the synthesis information S for the phoneme pairs corresponding to the phoneme pairs as the speech change index A of each phoneme pair of the synthesis target speech specified by the synthesis information S. Is calculated according to the pitch X1 specified by. Specifically, the index calculation unit 26 calculates the speech change index A according to the pitch X1 of the note to which the front phoneme is assigned among the two phonemes of the phoneme pair that matches the pre-conversion pattern QA. For example, the voice change index A is set to a larger value as the pitch X1 is higher.

Specifically, the index calculation unit 26 of the first embodiment calculates the voice change index A by the calculation of the following formula (1) using the basic index B, the duration T2 and the pitch X1.
A = B + (T2 × 0.2) + (X1 × 0.1) (1)
That is, the weighted sum of the basic index B, the duration T2 and the pitch X1 is calculated as the voice change index A. The weight value of the basic index B exceeds the weight value of the duration T2 and the pitch X1. That is, the basic index B is preferentially reflected in the voice change index A as compared with the duration T2 and the pitch X1.

  The information management unit 32 in FIG. 1 manages the composite information S stored in the storage device 12. For example, the information management unit 32 updates the composite information S in accordance with the editing instruction received from the user by the instruction receiving unit 22. In addition, the information management unit 32 of the first embodiment sets the occurrence of a voice change in the synthesized information S according to the voice change index A calculated by the index calculation unit 26. Specifically, the information management unit 32 determines that the phoneme pair when the speech change index A of the phoneme pair that matches the pre-conversion pattern QA among the time series of the plurality of phonemes of the synthesis target speech exceeds a predetermined threshold value TH. In the reference information R to the phoneme pair of the post-conversion pattern QB corresponding to the pre-conversion pattern QA. For example, in the illustration of FIG. 3, when the speech change index A of the phoneme pair [t-dh] included in the synthesis target speech exceeds the threshold value TH, the phoneme pair [t-dh] is the phoneme pair of the converted pattern QB. Changed to [Sil-dh]. In other words, the phoneme [t] is converted to the silent phoneme [Sil], thereby dropping the phoneme (speech change).

  The speech synthesis unit 34 in FIG. 1 generates a speech signal V using the speech element group L and the synthesis information S stored in the storage device 12. Specifically, the speech synthesizer 34 sequentially selects a speech unit corresponding to the pronunciation content X3 (speech code YA and phoneme symbol YB) for each note specified by the synthesis information S from the speech unit group L, The speech signals V are generated by adjusting the speech segments to the pitch X1 and the sound generation period X2 specified by the synthesis information S and connecting them together. The voice signal V generated by the voice synthesizer 34 is supplied to the sound emitting device 18 so that the singing voice of the synthesized music is reproduced.

  FIG. 5 is a flowchart of a schematic operation of the speech synthesizer 100 according to the first embodiment. 5 is started in response to an instruction from the user to the input device 16. When the processing of FIG. 5 is started, the display control unit 24 causes the display device 14 to display the edited image 40 of FIG. 4 corresponding to the composite information S stored in the storage device 12 (SA1). Then, the instruction receiving unit 22 determines whether an editing instruction has been received from the user (SA2). When the instruction receiving unit 22 receives an editing instruction (SA2: YES), an editing process corresponding to the content of the editing instruction is executed (SA3). The editing process includes updating of the edited image 40 by the display control unit 24 and updating of the composite information S by the information management unit 32. On the other hand, when the instruction receiving unit 22 does not receive the editing instruction (SA2: NO), the editing process is not executed.

  When the above processing is completed, the instruction receiving unit 22 determines whether or not an instruction for voice synthesis (generation of the voice signal V) is received from the user (SA4). When the instruction reception unit 22 receives a voice synthesis instruction (SA4: YES), the voice synthesis unit 34 generates a voice signal V by voice synthesis processing using the voice element group L and the synthesis information S (SA5). ). On the other hand, when voice synthesis is not instructed (SA4: NO), the voice synthesis process is not executed. Further, the instruction receiving unit 22 determines whether or not an instruction to end the process has been received from the user (SA6). If the end of the process is not instructed (SA6: NO), the process transitions to step SA1, and the subsequent processes are repeated. If the end of the process is instructed (SA6: YES), the process in FIG. 5 ends. .

  FIG. 6 is a flowchart of specific contents of the editing process (SA3). When the editing process is started, the instruction receiving unit 22 determines whether an editing instruction for the musical note image 44 of the score image 42 has been received from the user (SB1). When the editing instruction for the musical note image 44 is received (SB1: YES), the update of the edited image 40 by the display control unit 24 and the update of the composite information S by the information management unit 32 are executed according to the content of the editing instruction ( SB2). For example, when the addition of the note image 44 to the score image 42 is instructed, the display control unit 24 adds the note image 44 to the position indicated by the user in the score image 42, and the information management unit 32 uses the note image 44. The note information (X1 to X3) instructed by the person is added to the synthesis information S. When the movement of the existing musical note image 44 is instructed, the display control unit 24 changes the position of the musical note image 44 in accordance with an instruction from the user, and the information management unit 32 performs the sound of the note to be edited. The high X1 and the sound generation period X2 (start time T1) are changed according to an instruction from the user. When a change in the display length of the musical note image 44 on the time axis is instructed, the display control unit 24 changes the display length of the musical note image 44 in accordance with an instruction from the user, and the information management unit 32 The note generation period X2 (continuation length T2) is changed in accordance with an instruction from the user. When the change of the voice code YA of each note is instructed, the display control unit 24 changes the voice code YA of the note to be edited in the synthesis information S in accordance with the instruction from the user and the phoneme symbol YB. Is updated in accordance with the changed voice code YA, and the display control unit 24 arranges the voice code YA and the phoneme symbol YB corresponding to the changed pronunciation content X3 in the vicinity of the musical note image 44.

  In addition, the user can instruct the application of a sound change by appropriately operating the input device 16. When the instruction receiving unit 22 receives an instruction to give a voice change from the user (SB3: YES), the voice change setting process of FIG. 7 is executed to set the voice change corresponding to the voice change index A in the synthesis information S. (SB4).

  When the voice change setting process is started, the information management unit 32 selects one phoneme from a time series of a plurality of phonemes (phonemes specified by phoneme symbols YB of the pronunciation contents X3) of the synthesis target speech specified by the synthesis information S. A pair (hereinafter referred to as “selected phoneme pair”) is selected (SC1). The information management unit 32 determines whether or not the selected phoneme pair matches any of the plurality of pre-conversion patterns QA registered in the reference information R (SC2). That is, it is determined whether or not the selected phoneme pair corresponds to a phoneme pair that can cause a voice change.

  When the selected phoneme pair matches any of the pre-conversion patterns QA (SC2: YES), the index calculation unit 26 calculates the speech change index A of the selected phoneme pair by the calculation of the above formula (1) (SC3). . For calculation of the voice change index A, the basic index B corresponding to the pre-conversion pattern QA and the pitch X1 and duration T2 of the note corresponding to the front phoneme among the two phonemes of the selected phoneme pair are applied. The

  The information management unit 32 determines whether or not the voice change index A calculated by the index calculation unit 26 exceeds the threshold value TH (SC4). That is, it is determined according to the voice change index A whether or not a voice change is generated in the selected phoneme pair. When the voice change index A exceeds the threshold TH (SC4: YES), the information management unit 32 sets the occurrence of the voice change for the selected phoneme pair in the synthesis information S (SC5). Specifically, the phoneme symbol YB of each phoneme of the selected phoneme pair in the synthesis information S is converted into the phoneme symbol YB of each phoneme specified by the post-conversion pattern QB corresponding to the pre-conversion pattern QA.

  On the other hand, when the voice change index A is lower than the threshold value TH (SC4: NO), the information management unit 32 does not execute Step SC5 for setting the occurrence of the voice change. That is, the occurrence of a voice change for the selected phoneme pair is not set in the synthesis information S. If the selected phoneme pair does not match any pre-conversion pattern QA (SC2: NO), that is, if no voice change can occur for the selected phoneme pair, the calculation of the voice change index A (SC3) or the selected phoneme No pair conversion (SC4, SC5) is performed.

  For example, as illustrated in FIG. 8, attention is paid to the lyrics “out door” and “keep talk”. The basic index B of the phoneme pair [t-dh] included in “out door” is 80, and the basic index B of the phoneme pair [p-th] included in “keep talk” is 20. The phoneme [t] in the phoneme pair [t-dh] has a note duration T2 of 240 (eighth note) and a pitch X1 of 50, and the phoneme [p] in the phoneme pair [p-th]. Assuming that the note duration T2 is 60 (half note) and the pitch X1 is 50, the voice change index A of the phone pair [t-dh] of “out door” is 133 (= 80 + (240 × 0.2) + (50 × 0.1)), and “keep talk” phoneme pair [p-th] speech change index A is 37 (= 20 + (60 × 0.2) + (50 × 0.1)) is there. Therefore, for example, assuming that the threshold TH is 110, for the phoneme pair [t-dh] (SC4: YES) of “out door”, the voice change to the phoneme pair [Sil-dh] specified by the post-conversion pattern QB is On the other hand, no voice change is set for the phoneme pair [p-th] (SC4: NO) of “keep talk”.

  When the above processing is executed, the information management unit 32 determines whether or not the processing from step SC1 to step SC5 has been executed for all phoneme pairs included in the time series of the plurality of phonemes of the synthesis target speech (SC6). ). If there is an unprocessed phoneme pair (SC6: NO), the information management unit 32 moves the process to step SC1 and selects a phoneme pair immediately after the currently selected phoneme pair as a new selected phoneme pair. The subsequent processing (SC2-SC6) is executed. On the other hand, when the process has been executed for all phoneme pairs (SC6: YES), the display control unit 24 updates the score image 42 displayed on the display device 14 to the content after the sound change is set (SC7).

  FIG. 9 is an explanatory diagram of the change of the score image 42 in conjunction with the setting of the voice change. In FIG. 9, it is assumed that “out” and “door” are designated as two musical notes (N1, N2). The part (A) in FIG. 9 shows a state before setting the voice change. Specifically, the phonetic code “out” and the phoneme symbol [aU t] are displayed together with the note image 44 of the note N1, and the phonetic symbol “door” and the phoneme symbol [dh O @] are shown as the note image 44 of the note N2. It is displayed with. When the user gives an instruction to change the voice (SB3: YES), and the voice change index A of the phoneme pair [t-dh] of “out door” exceeds the threshold TH as shown in the above example (SC4: YES). As illustrated in part (B) of FIG. 9, the phoneme symbol [aU t] of the “out” of the note N1 is changed to the phoneme symbol [aU Sil] obtained by converting the phoneme [t] to the silent phoneme [Sil]. It is changed (SC7). Therefore, the user can visually grasp the voice change. The synthesis information S after the execution of the voice change setting process described above is applied to the voice synthesis process by the voice synthesizer 34, thereby generating a voice signal V that reproduces the voice change (dropping). It is also possible to highlight (for example, add an underline) the phoneme symbol YB that has been changed due to a change in speech.

  In the editing process of FIG. 6, the user can instruct to cancel the voice change by appropriately operating the input device 16. When the instruction receiving unit 22 receives an instruction to cancel the voice change from the user (SB5: YES), the voice change set in the synthesis information S is canceled (SB6). Specifically, the information management unit 32 converts the phoneme symbol YB of each phoneme designated by the pronunciation content X3 of the synthesis information S to the initial content corresponding to the speech code YA (that is, the phoneme symbol to which no speech change is given). YB), and the display control unit 24 initializes the edited image 40 displayed on the display device 14 to the content corresponding to the synthesis information S after canceling the voice change (that is, the phoneme symbol YB of each note is initialized). Content). If cancellation of voice change is not instructed (SB5: NO), the editing process ends.

  As described above, in the first embodiment, for the phoneme pair of the synthesis target speech specified by the synthesis information S, the speech change index A indicating the likelihood of occurrence of a speech change is calculated, and the speech change of the phoneme pair is calculated. Presence / absence is controlled according to the voice change index A. Therefore, when compared with the technique of Patent Document 1 in which the presence or absence of a sound change is uniformly determined only depending on whether or not each phoneme meets a predetermined condition, a natural sound change that matches the actual pronunciation tendency is obtained. It is possible to realize.

  For example, in the first embodiment, since the voice change index A is calculated according to the basic index B set individually for each phoneme pair, the possibility of the voice change is different for each phoneme pair (the voice change is different). It is possible to realize a natural speech change that matches the actual pronunciation tendency of the phonemes that are likely to occur and those that are difficult to generate). Further, since the voice change index A is calculated according to the duration T2 of the voice unit (note), the actual pronunciation tendency that the voice change is likely to occur when a plurality of phonemes are pronounced quickly and continuously. It can be reproduced. Since the voice change index A is calculated according to the pitch X1 of the voice unit, the tendency that the voice change is more likely to occur as the voice pitch is higher is reproduced.

Second Embodiment
A second embodiment of the present invention will be described below. In the first embodiment, the threshold value TH to be compared with the voice change index A is fixed to a predetermined value. On the other hand, in the second embodiment, the threshold value TH is variably controlled. In addition, about the element which an effect | action and function are the same as that of 1st Embodiment in each form illustrated below, the reference | standard referred by description of 1st Embodiment is diverted, and each detailed description is abbreviate | omitted suitably.

  FIG. 10 is a schematic diagram of the edited image 40 in the second embodiment. As illustrated in FIG. 10, the edited image 40 of the second embodiment includes a score image 42 similar to that of the first embodiment and an operation image 46 for receiving an instruction from the user. In FIG. 10, an image of an operator (slider) that can be operated by the user is illustrated as an operation image 46. The user can instruct an arbitrary threshold value TH by appropriately operating the operation image 46 using the input device 16. The instruction receiving unit 22 receives an instruction for the threshold TH from the user. That is, the threshold value TH in the second embodiment is variably set according to an instruction from the user. In step SC4 of the voice change setting process (SB4), the information management unit 32 determines whether the voice change index A calculated by the index calculation unit 26 and the threshold TH corresponding to the instruction received from the user by the instruction receiving unit 22 are large or small. In response to this, it is determined whether or not a voice change is generated in the selected phoneme pair.

  As understood from the above description, the voice change is less likely to occur in the synthesis target voice as the threshold TH corresponding to the instruction from the user is larger. Specifically, focusing on the example of FIG. 8 described above, for example, in a state where the threshold value TH is set to 150, a voice change is given to both the phoneme pair [t-dh] and the phoneme pair [p-th]. However, when the threshold value TH is set to 110, a phoneme change is applied to the phoneme pair [t-dh], and when the threshold value TH is set to 30, the phoneme pair [t-dh] and the phoneme pair [p- A phoneme change is given to both of [th].

  In the second embodiment, the same effect as in the first embodiment is realized. In the second embodiment, since the threshold value TH applied to the determination of the presence / absence of a sound change is variably controlled, the frequency of the sound change in the sound signal V can be controlled in various ways. Particularly in the second embodiment, since the threshold value TH is set in accordance with an instruction from the user, there is an advantage that it is possible to realize a sound change adapted to the user's intention and preference.

<Third Embodiment>
In the first embodiment, the voice change is set for the synthesis information S before the voice synthesis process is executed. On the other hand, in the third embodiment, the voice change is reflected in the voice signal V in the voice synthesis process. That is, in the third embodiment, the voice change is not reflected in the synthesis information S. Therefore, in the editing process (FIG. 6) of the third embodiment, the voice change setting and canceling processes (SB3-SB6) are omitted.

  FIG. 11 is a flowchart of the speech synthesis process (SA5) executed by the speech synthesizer 34 of the third embodiment to generate the speech signal V. When the speech synthesis process is started, the speech synthesis unit 34 selects phoneme pairs corresponding to the pronunciation content X3 designated by the synthesis information S as the selected phoneme pairs in order from the beginning of the synthesized music (SD1). Then, the speech synthesizer 34 determines whether the selected phoneme pair matches any of the plurality of pre-conversion patterns QA registered in the reference information R (SD2).

  When the selected phoneme pair matches any of the pre-conversion patterns QA (SD2: YES), the index calculation unit 26 changes the voice of the selected phoneme pair by the calculation of the above formula (1), as in the first embodiment. Index A is calculated (SD3). The speech synthesizer 34 determines whether or not the speech change index A calculated by the index calculator 26 exceeds a threshold value TH (SD4). That is, as in the first embodiment, it is determined according to the voice change index A whether or not a voice change is generated in the selected phoneme pair.

  When the speech change index A is lower than the threshold TH (SD4: NO), that is, when no speech change is generated for the selected phoneme pair, the speech synthesizer 34 selects the speech unit corresponding to the selected phoneme pair selected from the synthesis target phonemes. Is selected from the speech segment group L (SD5). On the other hand, when the speech change index A exceeds the threshold TH (SD4: YES), that is, when a speech change is generated for the selected phoneme pair, the speech synthesizer 34 replaces the selected phoneme pair with the phoneme of the converted pattern QB. A speech unit corresponding to the pair is selected from the speech unit group L (SD6).

  For example, it is assumed that the selected phoneme pair is a phoneme pair [t-dh] included in “out door”. When the speech change index A of the selected phoneme pair is lower than the threshold TH, the speech synthesizer 34 selects a speech unit corresponding to the phoneme pair [t-dh] from the speech unit group L (SD5). On the other hand, when the speech change index A exceeds the threshold value TH, the speech synthesizer 34 drops the phoneme pair [Sil-dh] (ie, phoneme [t]) of the converted pattern QB corresponding to the phoneme pair [t-dh]. The speech unit corresponding to the selected phoneme pair) is selected from the speech unit group L (SD6).

  The speech synthesizer 34 generates a speech signal V using the speech segment selected in step SD5 or step SD6 (SD7). As in the first embodiment, a known technique is arbitrarily employed to generate the audio signal V using the speech element. As understood from the above description, the speech synthesizer 34 of the third embodiment determines whether or not to generate a speech change in the selected phoneme pair according to the speech change index A and determines to generate a speech change. For the selected phoneme pair, a voice signal V that generates a voice change is generated (SD6, SD7), while for a selected phoneme pair that is determined not to generate a voice change, a voice signal V that does not generate a voice change is generated ( SD5 and SD7) function as elements.

  As described above, in the third embodiment, the speech change index A is calculated for the phoneme pair specified by the synthesis information S, and the presence or absence of a speech change is controlled according to the speech change index A. Therefore, as in the first embodiment, there is an advantage that a natural voice change that matches the actual pronunciation tendency can be realized.

<Fourth embodiment>
FIG. 12 is a schematic diagram of the synthesis information S in the fourth embodiment. As illustrated in FIG. 12, the synthesis information S of the fourth embodiment includes control information XC in addition to the same elements (pitch X1, sound generation period X2, sound generation content X3) as in the first embodiment. The control information XC is a variable that designates a musical expression (expression) of the synthesis target voice. For example, accent, staccato, and volume are specified by the control information XC. The index calculation unit 26 according to the fourth embodiment performs a sound change of the phoneme by calculation using a variable specified by the control information XC of each phoneme of the synthesis target speech in addition to the basic index B, the duration T2 and the pitch X1. Indicator A is calculated (SC3). The control information XC is set according to the instruction received from the user by the instruction receiving unit 22.

  For example, a user tends to indicate an accent or staccato for a note intended for clear pronunciation (ie, a note to be emphasized). Considering the above tendency, the index calculation unit 26 calculates the voice change index A so that the voice change index A of the phoneme for which accent or staccato is instructed by the control information XC decreases (that is, the voice change is less likely to occur). The structure which does is suitable. In addition, for example, for notes intended for clear pronunciation, the user tends to set the volume to a large value. In consideration of the above tendency, a configuration in which the index calculation unit 26 calculates the voice change index A so that the voice change index A decreases as the volume specified by the control information XC increases. The application of the voice change using the voice change index A is the same as in the first embodiment.

  In the fourth embodiment, the same effect as in the first embodiment is realized. In the fourth embodiment, since the voice change index A is calculated according to the control information XC representing the musical expression of the synthesis target voice, natural voice change suitable for the musical expression of the synthesis target voice is obtained. Can be given. In the above example, the configuration based on the first embodiment is illustrated. However, the configuration of the fourth embodiment in which the control information XC is reflected in the voice change index A is also applied to the second embodiment and the third embodiment. It can be applied as well.

<Modification>
Each of the above-described embodiments can be variously modified. Specific modifications are exemplified below. Two or more modes arbitrarily selected from the following examples can be appropriately combined.

(1) In each of the above-described embodiments, the voice change index A is calculated according to the basic index B, the duration T2, and the pitch X2, but the variables applied to the calculation of the voice change index A are as described above. It is not limited. For example, the sound change index A can be calculated according to the tempo of the synthesized music. As described above, when a plurality of phonemes are pronounced quickly, there is a tendency that a sound change is likely to occur. Therefore, a configuration in which the voice change index A is set to a larger value (a value at which a voice change is likely to occur) as the synthesized music tempo is faster is preferable. The tempo of the synthesized music is variably instructed by the user according to an operation on the input device 16, for example.

  For example, one or two types of variables selected from the basic index B, the continuation length T2, and the pitch X2 can be applied to the voice change index A. In addition, a configuration in which only the control information XC of the fourth embodiment is applied to the calculation of the voice change index A, or one or two types of variables selected from the basic index B, the duration T2, and the pitch X2 and the control information. A configuration in which the combination with XC is applied to the calculation of the voice change index A can also be adopted. Further, the calculation method of the voice change index A to which each variable is applied is not limited to the above example (Formula (1)).

(2) In each of the above-described embodiments, the case where the voice change index A increases as the voice change easily occurs (the case where the voice change index A decreases as the voice change hardly occurs) is exemplified. The relationship between the level of the possibility of and the magnitude of the voice change index A is appropriately changed. That is, a configuration in which the voice change index A decreases as the voice change easily occurs (a configuration in which the voice change index A increases as the voice change hardly occurs) may be employed.

(3) In the first embodiment, the phoneme symbol YB of the phoneme that causes the voice change (dropping) in the synthesis information S is replaced with the phoneme symbol YB after the voice change, but the occurrence of the voice change is set in the synthesis information S The configuration for doing so is arbitrary. For example, a configuration in which auxiliary information (for example, symbol “_0” indicating devoicing) is added to the phoneme symbol YB of a phoneme that generates a change in speech. In addition, if attention is paid only to omission of voice changes, it is possible to delete the phoneme symbol YB of the phoneme causing the voice change.

(4) In the third embodiment, the speech unit corresponding to the post-conversion pattern QB is selected for the phoneme pair that causes the speech change. However, the method for generating the speech signal V according to the presence or absence of the speech change is exemplified above. It is not limited to. For example, it is possible to reproduce the loss of phonemes by setting the volume of the phoneme that causes a sound change (drop) in the phoneme pair to zero.

(5) In each of the above-described embodiments, the note duration T2 corresponding to the phoneme causing the speech change is applied to the calculation of the speech change index A. However, the phoneme is pronounced during the pronunciation period X2 (continuation length T2). It is also possible to apply the continuation length (time length) to the calculation of the voice change index A.

(6) In the first embodiment, the speech synthesis apparatus 100 that executes both management of the synthesis information S (display control unit 24 and information management unit 32) and generation of the speech signal V (speech synthesis unit 34) is exemplified. However, the present invention can also be specified as a composite information management apparatus that manages the composite information S. In the synthesis information management apparatus, the presence or absence of the speech synthesizer 34 is not questioned. It is also possible to realize the speech synthesizer 100 and the synthesized information management device with a server device that communicates with a terminal device such as a mobile phone. The instruction receiving unit 22 receives an instruction given by the user to the terminal device from the terminal device via the communication network, and the display control unit 24 transmits, for example, image data of the edited image 40 to the terminal device, thereby editing the edited image 40. Is displayed on the display device of the terminal device. In addition, the voice synthesizer 34 transmits the voice signal V generated by the voice synthesis process to the terminal device.

(7) In each of the above-described embodiments, the storage device 12 that stores the speech element group L and the synthesis information S is mounted on the speech synthesizer 100. However, an external device (for example, a server device) that is independent of the speech synthesizer 100. Is also used to store the speech element group L and the synthesis information S. The speech synthesizer 100 acquires the speech unit of the speech unit group L and the synthesis information S through, for example, a communication network, and executes editing processing and speech synthesis processing. As understood from the above description, the elements that store the speech element group L and the synthesis information S are not essential elements of the speech synthesizer 100.

(8) In each of the above-described embodiments, the generation of the voice signal V of the singing voice of the synthesized music has been exemplified. However, the present invention is also applied to the generation of the voice signal V of the voice other than the singing voice (for example, conversation sound). Is possible. Therefore, the pitch X1 suitable for singing voice synthesis can be omitted from the synthesis information S. As understood from the above description, the synthesis information S exemplified in each of the above aspects is comprehensively expressed as information specifying the pronunciation content of the speech to be synthesized. Further, the method of speech synthesis processing is arbitrary. Specifically, in addition to the unit-connected speech synthesis processing exemplified in the above-described embodiments, the filter processing corresponding to the pronunciation content X3 (phoneme) is performed on the pitch estimated by the HMM (Hidden Markov Model). It is also possible to generate the voice signal V by a statistical model type voice synthesis process to be executed.

(9) In each of the above-described embodiments, the synthesis of English speech has been exemplified, but the speech language to be synthesized is arbitrary. For example, the present invention can be applied to the case of generating speech in an arbitrary language such as Japanese, Spanish, Chinese, or Korean.

DESCRIPTION OF SYMBOLS 100 ... Voice synthesizer, 10 ... Arithmetic processing unit, 12 ... Memory | storage device, 14 ... Display device, 16 ... Input device, 18 ... Sound emission device, 22 ... Instruction reception part, 24 ... Display Control unit, 26... Index calculation unit, 32... Information management unit, 34.

Claims (4)

A device that manages synthesis information that specifies a pitch, duration, and pronunciation content for each voice unit of a synthesis target voice,
A variable voice change indicator is indicative of the occurrence ease of voice change, for each of a plurality of phonemes of the synthesis target speech, and index calculation means for calculating in accordance with the duration of the phoneme,
An instruction receiving means for receiving an instruction from the user;
Depending on the size of the speech change index calculated by the index calculation means for the phoneme of the synthesis target speech and the threshold value variably controlled according to the instruction received from the user by the instruction receiving unit, the speech of the phoneme It is determined whether or not to generate a change, and for the phonemes determined to generate a speech change, the generation of the speech change is set in the synthesis information, while for the phonemes determined not to generate the speech change, the speech change is generated. A combined information management apparatus comprising: information management means not set in the combined information. A device that manages synthesis information that specifies a pitch, duration, and pronunciation content for each voice unit of a synthesis target voice,
A variable voice change indicator is indicative of the occurrence ease of voice change, for each of a plurality of phonemes of the synthesis target speech, and index calculation means for calculating in accordance with the phoneme pitch,
An instruction receiving means for receiving an instruction from the user;
Depending on the magnitude of the voice change index calculated by the index calculation means for the phoneme of the synthesis target voice and the threshold set according to the instruction received from the user by the instruction receiving unit, the voice change of the phoneme is determined. It is determined whether or not to generate a voice change for the phoneme determined to generate a voice change in the synthesis information, while for the phoneme determined not to generate a voice change, the voice change occurrence is set to the synthesis A composite information management device comprising: information management means not set in the information. The index calculation means is a basic index corresponding to a combination of two phonemes of the synthesis target speech based on reference information that designates a basic index that is an index of the likelihood of speech change for each combination of two phonemes. The synthetic information management device according to claim 1 or 2 , wherein the voice change index is calculated according to the basic index. An apparatus for generating a voice signal of a synthesis target voice in which synthesis information specifies a pitch, duration, and pronunciation content for each voice unit ,
A variable voice change indicator is indicative of the occurrence ease of voice change, for each of a plurality of phonemes of the synthesis target speech and index calculation means for calculating in accordance with the duration of the phoneme,
An instruction receiving means for receiving an instruction from the user;
A means for generating a voice signal according to the synthesis information, the voice change index calculated by the index calculation means for the phoneme of the synthesis target voice, and variable according to an instruction received from a user by the instruction receiving unit Whether to generate a sound change of the phoneme according to the magnitude of the controlled threshold, and generating the sound signal in which the sound change is generated for the phoneme determined to generate the sound change. A speech synthesizer comprising: a speech synthesizer that generates the speech signal that does not cause a speech change for a phoneme determined not to cause a speech change.

Images