JP4306643B2 - Singing composition device and singing composition program - Google Patents

Description

  The present invention relates to a singing voice synthesizing apparatus and a singing voice synthesis program for synthesizing singing sounds.

Various song synthesizers have been proposed that store lyric data and note data, read out the lyric data in response to reading out the note data, and sing the lyrics by generating phonemes corresponding to the lyric data. This type of synthesizer is required to have a function of synthesizing natural human-like singing sounds. When observing the human singing sound, the pitch of the singing sound does not change stepwise according to the note data, but changes from a utterance corresponding to one note to a utterance corresponding to the next note. Draw and change continuously. This change is thought to contribute to the human nature of singing. Therefore, conventionally, in a synthesizer, studies have been made to give human-like changes to the pitch of the synthesized singing sound. For example, when synthesizing a singing sound corresponding to two consecutive notes, Patent Document 1 connects a pitch corresponding to each note with a straight line, and obtains a pitch curve provided with an overshoot part before and after the straight line. An apparatus for synthesizing a singing sound whose pitch changes according to this pitch curve is proposed.
JP 2003-323188 A

  By the way, when observing the actual singing voice, especially in the legato style of singing, the pitch of the subsequent note is higher than the pitch of the preceding note in two consecutive notes, and the length of the subsequent note is sufficiently long. When the pitch is long, the time at which the pitch starts to change is later than usual, and the duration of the pitch change is longer than usual. However, the above-described singing voice synthesizing apparatus disclosed in Patent Document 1 has a monotonous mechanical singing voice because the pitch trajectory is defined at the switching portion of the notes by a uniform method regardless of the singing. There was a problem that it was easy to become.

  The present invention has been made in view of the circumstances described above. When a specific chanting method is designated, the pitch trajectory at the switching portion between two consecutive notes is changed from the normal case. An object of the present invention is to provide a singing synthesis apparatus and a singing synthesis program that can synthesize natural singing sounds.

The present invention provides a transition start time from a pitch corresponding to a preceding note to a pitch corresponding to a succeeding note when sequentially synthesizing the singing sound of two consecutive notes when a specific singing method is specified. It is a means for setting the transition end time, and when the pitch of the subsequent note is higher than the pitch of the preceding note, the transition start time is delayed compared to the normal case where the specific chorus is not specified, A transition point setting means for extending the transition duration from the transition start time to the transition end time, and generating pitch data for changing the pitch of the singing sound from the transition start time set by the setting means to the transition end time. Pitch data generating means to be stored in the storage means, and the pitch data is read from the storage means in accordance with the progress of the song and output to the sound source for synthesizing the singing sound Providing a singing voice synthesizing apparatus and a computer, characterized by comprising an output unit singing synthesis program to function as the singing voice synthesizing apparatus.
According to this invention, when a specific singing method is specified, the start of portamento is delayed as compared to the normal case, and the duration of portamento is longer, for example, when singing a rising sound form by legato Human singing voice can be reproduced in singing synthesis, and natural singing sound can be obtained.

Embodiments of the present invention will be described below with reference to the drawings.
<First Embodiment>
FIG. 1 is a block diagram showing a configuration of a singing voice synthesizing apparatus according to the first embodiment of the present invention. This singing voice synthesizing apparatus is obtained by installing a singing voice synthesis program on a computer such as a personal computer having a function of synthesizing and outputting a voice. In FIG. 1, CPU1 is a control center which controls each part of this song synthesis apparatus. The ROM 2 is a read-only memory that stores a control program for controlling basic operations of the singing voice synthesizing apparatus such as a loader. The display unit 3 is a device for displaying an operation state of the device, input data, a message for the operator, and the like. The operation unit 4 is a means for receiving commands and various types of information from the user, and includes various types of operators such as a keyboard and a mouse. The interface group 5 includes a network interface for performing data communication with other devices via a network, a driver for transmitting / receiving data to / from an external storage medium such as a magnetic disk and a CD-ROM, and the like. It is comprised by. The HDD (hard disk device) 6 is a non-volatile storage device for storing information such as various programs and databases. The RAM 7 is a volatile memory used as a work area by the CPU 1. The CPU 1 loads a program in the HDD 6 into the RAM 7 and executes it in accordance with a command given via the operation unit 4.

  The formant sound source 8 is a sound source that synthesizes a singing sound under the control of the CPU 1. The formant sound source 8 is provided with formant center frequency data FFreq, formant level data Flevel and formant shape data FShape as information for specifying the formant of the singing sound, and pitch data PITCH specifying the pitch of the singing sound. It is done. The formant sound source 8 synthesizes a singing sound having a formant and pitch designated by these data, and outputs it from the sound system 9.

  Information stored in the HDD 6 includes a song editing program 61, song data 62, a phoneme database 63, and a song synthesis program 64. The song data 62 is data composed of note data representing a series of notes constituting the song and lyrics data representing lyrics uttered in accordance with the notes, and is edited for each song and stored in the HDD 6. The song editing program 61 is a program executed by the CPU 1 to edit song data. In a preferred embodiment, the song editing program 61 causes the display unit 3 to display a GUI (graphical user interface) including an image of a piano keyboard. The user can designate an image of a desired key on the keyboard displayed on the display unit 3 by operating the operation unit 4, and can input lyrics to be uttered in accordance with the note by operating the operation unit 4. In this way, the song editing program 61 receives information about the notes and the lyrics to be uttered in accordance with the notes from the user via the operation unit 4 and stores the note data and the lyrics data for each note in the HDD 6 as the song data 62. To store.

  The note data corresponding to one note includes information indicating the note generation time, pitch, and note length. The lyric data is data in which lyrics to be pronounced in accordance with the notes are defined for each note. The song data 62 is obtained by arranging the note data corresponding to each note and the lyrics data in time series in accordance with the generation order from the start of the song. In the song data, the note data and the lyrics data are the note data. Corresponds in units.

  The song synthesis program 64 is a program that causes the CPU 1 to execute a process of causing the formant sound source 8 to synthesize a song sound according to the song data 62. In a preferred embodiment, the song synthesis program 64 and the song editing program 61 are downloaded from a site in the Internet, for example, through an appropriate one in the interface group 5 and installed in the HDD 6. In another aspect, the song synthesis program 64 and the like are traded in a state stored in a computer-readable storage medium such as a CD-ROM or MD. In this aspect, the song synthesis program 64 and the like are read from the storage medium via an appropriate one in the interface group 5 and installed in the HDD 6. The phonological database 63 is a collection of phonological data groups referred to by the song synthesis program 64. In the phonological database 63, a group of phonological data is prepared for each type of generated voice, for example, male voice, female voice, or a specific singer. At the time of singing synthesis by the singing synthesis program 64, the user can select a group of phonological data to be used according to the voice quality to be uttered by operating the operation unit 4. The phoneme data constituting each group is a parameter for formant synthesis of each phoneme. For each phoneme, formant shape data FShape for designating the shape of each formant for generating the phoneme, the center frequency of each formant Are formant center frequency data FFreq and formant level data FLlevel for designating the output level of each formant.

  FIG. 2 shows the processing contents of the song synthesis program 64. The song synthesis program 64 is roughly divided into a song synthesis score generation process 64A and a song synthesis score output process 64B. The singing synthesis score generation process 64A is based on the song data 62 designated by the user and the phonological data of the group designated by the user in the phonological database 63, and the singing synthesis score comprising the pitch data track 71 and the phonological data track 72. This is a process of generating 70 in the work area in the RAM 7 (see FIG. 1). Here, the pitch data track 71 is a data stream in which pitch data PITCH specifying the pitch of the singing sound is arranged in time series as time passes from the start point of the song. The phonological data track 72 is a data stream in which formant shape data FShape characterizing the formant of the singing sound, formant center frequency data FFreq, and formant level data FLlevel are arranged in time series as time passes from the start of the song.

  The song synthesis score generation process 64 </ b> A includes a transition point setting process 641 and a pitch data generation process 642 as processes for generating the pitch data track 71. In this embodiment, basically, pitch data for changing the pitch of the singing sound is generated in accordance with the note data. However, in order to enrich the expression of the singing sound, the switching portion of the note is the same as the preceding note. Pitch data for changing the pitch of the synthesized singing sound along a smooth curved trajectory from the pitch to the pitch of subsequent notes is generated.

  In order to change the pitch of the singing sound in this way, in the transition point setting process 641, when the singing sounds of two consecutive notes are sequentially synthesized, the pitch corresponding to the preceding note is changed to the pitch corresponding to the subsequent note. Set the migration start time and migration end time. The processing content of the transition point setting processing 641 in the present embodiment differs depending on whether legato is specified as a chanting method, for example, by the operation of the operation unit 4, and precedes when legato is specified. It differs depending on the pitch relationship between the note to be played and the pitch of the following note. That is, it is as follows. First, in a normal case where legato is not specified as a chanting method, the transition point setting process 641 performs a transition start time and a transition end by a uniform method regardless of the pitch relationship between the preceding note and the subsequent note. Time is set. On the other hand, in the transition point setting process 641 when legato is specified, the transition start time and transition end time setting methods differ depending on the pitch relationship between the preceding note and the subsequent note. First, when the pitch of the subsequent note is higher than the pitch of the preceding note, the transition point setting process 641 delays the transition start time from the normal case, and the transition duration from the transition start time to the transition end time is , Make it longer than usual. On the other hand, when the pitch of the subsequent note is lower than the pitch of the preceding note, the transition point setting process 641 sets the transition start time earlier than the normal case, and the transition duration from the transition start time to the transition end time is , Make it longer than usual.

  In the pitch data generation process 642, a trajectory of a pitch that changes in accordance with the note data is obtained based on the note data and the transition start time and transition end time set by the transition point setting process 641, and the pitch is calculated along the trajectory. Pitch data to be changed is generated and stored in the pitch data track 71. More specifically, in the pitch data generation process 642, in a two-dimensional coordinate system in which the horizontal axis is time and the vertical axis is pitch, a straight line that maintains the pitch specified by the note data in a portion other than the note switching portion. The trajectory is the pitch trajectory, and a gentle curve from the pitch of the preceding note to the pitch of the subsequent note is obtained as the pitch trajectory during the note switching portion, that is, the period from the transition start time to the transition end time. Then, the pitch data obtained in this way and changing the pitch along the trajectory is generated.

  The singing synthesis score generation process 64 </ b> A includes a phonological extraction process 643 and a phonological data generation process 644 as processes for generating the phonological data track 72. The phoneme extraction process 643 is a process of sequentially extracting phonemes from the lyrics indicated by the lyrics data 62B. The phoneme data generation process 644 is phoneme data for synthesizing the phonemes extracted by the phoneme extraction process 643 from the groups specified by the user among the phoneme data in the phoneme database 63, specifically, formant shape. Data FShape, formant center frequency data FFreq, and formant level data FLlevel are read and stored in the phoneme data track 72.

The singing synthesis score output process 64B performs synchronous reproduction of the pitch data track 71 and the phonological data track 72 generated by the singing synthetic score generation process 64A described above, and the pitch data PITCH and the formant shape data FShape, which is phonological data, This is a process of supplying formant center frequency data FFreq and formant level data FLlevel to the formant sound source 8. More specifically, in the singing synthesis score output process 64B, the pitch data PITCH and phonological data corresponding to each note are read from the pitch data track 71 and the phonological data track 72 and supplied to the formant sound source 8 as the music progresses. In the singing synthesis score output process 64B, syllables consisting only of vowels are output from the formant sound source 8 at the note-on timing of notes, and syllables consisting of consonants and vowels are formed at the vowel part of the formant sound source 8 at the note-on timing of notes. The timing of supplying each phoneme data to the formant sound source 8 is controlled.
The above is the outline of the processing content of the song synthesis program.

  Next, the operation of this embodiment will be described. In addition, below, the operation | movement of singing composition which is the characteristics of this embodiment is demonstrated, and description of another operation | movement is abbreviate | omitted. When a predetermined instruction is given via the operation unit 4, the CPU 1 loads the song synthesis program 64 in the HDD 6 into the RAM 7 and executes it. In the process of executing the song synthesis program 64, when song data 62 to be synthesized and a group of phonological data used for synthesis are designated by the operation of the operation unit 4, and an instruction to start song synthesis is given by the user. First, the execution of the song synthesis score generation process 64A is started, and the execution of the song synthesis score output process 64B is started a little later.

  In the singing synthesis score generation process 64A, the pitch data track 71 is generated from the note data 62A by the execution of the transition point setting process 641 and the pitch data generation process 642, and the lyric data 62B by the execution of the phoneme extraction process 643 and the phoneme data generation process 644. From the phoneme data track 72 is generated.

  In the singing synthesis score output process 64B, the pitch data track 71 and the phonological data track 72 generated by the singing synthesis score generation process 64A are synchronously reproduced, and the pitch data PITCH and the phonological data are supplied to the formant sound source 8. As a result, a singing sound having a formant characterized by phonological data and having a pitch specified by the pitch data PITCH is sequentially synthesized by the formant sound source 8 and output from the sound system 9.

  In the present embodiment, a transition point setting process 641 and a pitch data generation process 642 are executed as means for controlling the pitch of the singing sound synthesized by the formant sound source 8. For this reason, the aspect of the change of the pitch of the singing sound synthesize | combined by the formant sound source 8 becomes a characteristic of this embodiment so that it may demonstrate below.

  3 and 4 show an execution example of the transition point setting process 641 and the pitch data generation process 642. In these figures, the horizontal axis is time, and the vertical axis is pitch. FIG. 3 shows an execution example of the transition point setting process 641 and the pitch data generation process 642 in the case of the rising note shape that switches from the note A to the note B having a higher pitch than this. First, in a normal case where no legato is specified, the transition point setting process 641 uses a time tA1 that is a predetermined time before the end time of the note A as a transition start time, and a time that is a predetermined time after the start time of the note B. Let tB1 be the transition end time. In the pitch data generation process 642, the pitch of the note A is maintained during the period before the transition start time tA1, and the period from the transition start time tA1 to the transition end time tB1 is determined from the pitch of the note A as indicated by a broken line. After descending slightly, draw a gentle curve, rise until it slightly exceeds the pitch of note B, find a trajectory that descends from it and reaches the pitch of note B, and change the pitch along this trajectory Generate pitch data. On the other hand, when legato is specified, in the transition point setting process 641, a time tA1L slightly delayed from the transition start time tA1 in the normal case is set as the transition start time. Further, the transition end time tB1L is greatly delayed from the transition end time tB1 in the normal case so that the transition continuation time from the transition start time tA1L to the transition end time tB1L becomes longer than the transition continuation time tB1-tA1 in the normal case. To set. In the pitch data generation processing 642, a solid curve trajectory from the pitch of the note A to the pitch of the note B is obtained from the transition start time tA1L to the transition end time tB1L set in this way, and the pitch along the trajectory is determined. The pitch data for changing the value is generated. In this way, when legato is specified, the start of portamento is delayed compared to the normal case, and the duration of portamento becomes longer, and human beings when singing a rising note form with legato Singing voice can be reproduced in singing synthesis.

  FIG. 4 shows an execution example of the transition point setting process 641 and the pitch data generation process 642 in the case of a falling note that switches from a note A to a note B having a lower pitch than this. In this figure, tA2 and tB2 are the transition start time and transition end time set by the transition point setting process 641 in a normal case where legato is not specified, and tA2L and tB2L are transition points when legato is specified. The transition start time and the transition end time set by the setting process 641. The curved trajectory indicated by a broken line is a pitch trajectory in a normal case where no legato is specified, and the curved trajectory indicated by a solid line is a pitch trajectory in a normal case where no legato is specified. As shown in this figure, when legato is specified in this embodiment, the transition duration tB2L-tA2L is set to the normal case by setting the transition start time tA2L earlier than the normal transition start time tA2. Longer than the transition duration tB2-tA2. In this way, when legato is specified, the start of portamento will be earlier than usual, and the duration of portamento will be longer. Singing voice can be reproduced in singing synthesis.

  As described above, according to the present embodiment, when legato is designated, the pitch transition start time and end time and transition duration in the note switching portion are changed from those in the normal case. Therefore, portamento in the case of legato can be brought close to an actual human song, and a natural human song can be reproduced in the song synthesis.

Second Embodiment
In the present embodiment, a restriction is added in a specific case to the movement of the transition start time and the transition end time and the extension of the transition duration performed when legato is specified in the first embodiment. .

  In the case of the rising note shape of FIG. 3, when the note length of the subsequent note B is shorter than the note length of the preceding note A, the transition start time tA1L and the transition end time tB1L are delayed to increase the transition duration time. , Singing becomes unnatural. Therefore, in the transition point setting process 641 (see FIG. 2) in the present embodiment, the ratio of the note length of the subsequent note B to the note length of the preceding note A is obtained when continuous note singing is performed. As the note length ratio between the notes decreases, the transition start time delay degree tA1L-tA1 is decreased, and the transition duration extension degree (tB1L-tA1L) / (tB1-tA1) is decreased. To do.

  In this way, instead of adjusting the transition start time and transition duration in accordance with the note length ratio between notes, the note length ratio (note length of note B / note length of note A) is a constant threshold (1.6). The transition start time, the transition end time, and the transition duration time in the case of the legato described in the first embodiment may be extended only when the degree is larger than (preferably).

  Even if the note length ratio is large, if the note length of the subsequent note B itself is short, if the transition start time, the transition end time are moved, and the transition duration time is extended, the singing is unnatural It becomes. Therefore, only when the note length of the subsequent note B is longer than a certain value, the transition start time, the transition end time, and the transition duration time described in the first embodiment may be extended. .

  In addition, when there is not much difference between the pitches of the preceding note A and the subsequent note B, the singing is unnatural when the transition start time, the transition end time are moved and the transition duration time is extended in the first embodiment. It will be a thing. Therefore, only when the difference between the pitches of the preceding note A and the succeeding note B is equal to or greater than a certain value (for example, 5 semitones or more), the transition start time, the shift of the transition end time described in the first embodiment, and The transition duration may be extended.

  In the case of the descending sound form of FIG. 4 described above, when the note length of the preceding note A is shorter than the note length of the subsequent note B, the transition start time and the transition end time shift and transition of the first embodiment. If the duration is extended, the singing becomes unnatural. Therefore, in a preferred embodiment, the ratio of the note length of the preceding note A to the note length of the subsequent note B (the note length of the note A / the note length of the note B) is obtained, and as the note length ratio between the notes becomes smaller. TA1-tA1L is reduced to the extent that the transition start time is advanced, and the extent (tB1L-tA1L) / (tB1-tA1) of the extension of the transition duration is reduced.

  In this way, instead of adjusting the transition start time and transition duration in accordance with the note length ratio between notes, the note length ratio (note length of note A / note length of note B) is a constant threshold (1.6). The transition start time, the transition end time, and the transition duration time in the case of the legato described in the first embodiment may be extended only when the degree is larger than (preferably). Further, only when the note length of the preceding note A is longer than a certain value, the transition start time, the transition end time, and the transition duration time described in the first embodiment may be extended. . Even in the case of a descending note, the transition start described in the first embodiment is performed only when the difference between the pitches of the preceding note A and the following note B is a certain value or more (for example, five semitones or more). The movement of the time, the transition end time, and the extension of the transition continuation time may be performed.

<Other embodiments>
While the first and second embodiments of the present invention have been described above, various embodiments other than this can be considered for the present invention. For example:
(1) The sound source for synthesizing the singing sound may not be a formant sound source. For example, a memory storing sample data of various phonological waveforms, or a memory storing spectral envelopes of harmonic components and anharmonic components of various phonological components, and storing the specified phonological sample data from this memory A sound source configured to read and synthesize a singing sound may be used in a singing synthesizer.
(2) The pitch trajectory obtained by the pitch data generation processing 642 may be displayed on the display unit 3 in response to a request given by the operation of the operation unit 4. In addition, the user who confirms the display may be configured to be able to correct the transition start time and the transition end time for a desired note switching portion by operating the operation unit 4.
(3) Regardless of the specified chanting method, whether the pitch trajectory is a curved trajectory as shown in FIGS. 3 and 4 or a rectangular trajectory determined by note data is used. You may comprise so that it can designate by operation.
(4) Various methods for calculating the pitch trajectory at the note switching portion are conceivable. For example, for each of a normal case and a case where legato is specified, data relating to one or a plurality of control points, which are target passage points of the pitch trajectory at the part where the note is switched, is stored in the HDD 6, and the legato is designated. In this case, a control point for legato may be selected, and a curve passing through the control point may be obtained as a pitch trajectory.
(5) A set of parameters for determining the pitch trajectory at the note switching portion is prepared in the HDD 6 for each type of singing such as “normal”, “sharp”, and “legato”. When the type of singing is selected by the operation, a set of parameters corresponding to the type may be read from the HDD 6 and given to the singing synthesis score generation process 64A. In this case, the set of parameters corresponding to “legato” is such that the transition start time and transition end time are moved and the transition duration is extended as described in the first embodiment. Prepare things. Further, in this case, whether the pitch trajectory is the trajectory determined by the parameter set or the rectangular trajectory determined by the note data can be designated by the operation of the operation unit 4. It may be configured. In this way, the user can set parameters for realizing his / her favorite song without knowing the meaning of the parameters, and the convenience for the user is improved.

It is a block diagram which shows the structure of the song synthesizing | combining apparatus which is 1st Embodiment of this invention. It is a figure which shows the content of the song synthesis | combination process in the embodiment. It is a figure which shows the execution example of the transfer setting process in the same embodiment, and a pitch data generation process. It is a figure which shows the execution example of the transition point setting process and pitch data generation process in the embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... CPU, 4 ... Operation part, 6 ... HDD, 7 ... RAM, 8 ... Formant sound source, 63 ... Phonological database, 64 ... Singing synthesis program, 70 ... Singing synthesis score, 71 ... Pitch data track, 72... Phoneme data track, 62 A .. Note data, 641... Transition point setting process, 642 .. Pitch data generation process, 64 A .. Singing synthesis score generation process, 64 B. .

Claims (6)

When sequentially singing the singing sounds of two consecutive notes when a specific singing method is specified, the transition start time and transition end time from the pitch corresponding to the preceding note to the pitch corresponding to the subsequent note are set. If the pitch of the subsequent notes is higher than the pitch of the preceding notes, the transition start time is delayed compared to the normal case where the specific singing method is not specified. A transition point setting means for extending the transition duration until the transition end time;
Pitch data generating means for generating pitch data for changing the pitch of the singing sound from the transition start time set by the setting means to the transition end time, and storing it in the storage means;
A singing synthesizing apparatus comprising: output means for reading out the pitch data from the storage means in accordance with the progress of a song and outputting the singing sound to a sound source.   The transition point setting means, when the pitch of the subsequent note is lower than the pitch of the preceding note, the transition start time is advanced compared to the normal case where the specific singing method is not specified. The singing voice synthesizing apparatus according to claim 1, wherein the transition continuation time from to the transition end time is lengthened.   The transition point setting means controls the degree of movement of the transition start time or transition end time and the degree of expansion of the transition duration according to the ratio of the note length of the preceding note and the note length of the subsequent note. The singing voice synthesizing apparatus according to claim 1.   The transition point setting means restricts movement of the transition start time or transition end time and expansion of the transition duration when the note length of the preceding note or the note length of the subsequent note is shorter than a threshold value. The singing voice synthesizing apparatus according to claim 1.   The transition point setting means restricts movement of the transition start time or transition end time and extension of the transition duration when the difference in pitch between the preceding note and the subsequent note is smaller than a threshold value. The singing voice synthesizing apparatus according to claim 1. When sequentially singing the singing sounds of two consecutive notes when a specific singing method is specified, the transition start time and transition end time from the pitch corresponding to the preceding note to the pitch corresponding to the subsequent note are set. If the pitch of the subsequent notes is higher than the pitch of the preceding notes, the transition start time is delayed compared to the normal case where the specific singing method is not specified. Transition point setting process to increase the transition duration until the transition end time,
Pitch data generation processing for generating pitch data for changing the pitch of the singing sound from the transition start time set by the setting means to the transition end time and storing it in the storage means;
A singing composition program that causes a computer to execute an output process of reading the pitch data from the storage unit in accordance with the progress of a tune and outputting the singing sound to a sound source.

Images