JP4192994B2 - Data input program for singing synthesis - Google Patents

Description

  The present invention relates to a technique for facilitating input of lyrics in a singing voice synthesizing apparatus.

Conventionally, in a speech synthesizer or a speech synthesis program, when a text is given, a phonetic symbol corresponding to the text is generated and a speech is synthesized according to the phonetic symbol. Some devices and programs of this type have a function of editing a generated phonetic symbol string. For this editing, these devices and programs are equipped with a text editor, and a user generally edits a phonetic symbol string directly using this text editor.
However, the generated phonetic symbol sequence is a sequence of symbols representing readings, symbols and positions of accents and poses, etc. If the user does not know the meaning of these symbols, the user edits the phonetic symbol sequence. Was very difficult.

  As a technique for overcoming this drawback, for example, Patent Document 1 introduces a symbol called utterance classification that is easy to understand for a general user who does not know a phonetic symbol string. This pronunciation division represents the position of the accent phrase, voice phrase, and exhalation paragraph, and can be easily edited by those who can speak the language without knowing the definition of the phonetic symbol string. Can do. Further, since the phonetic symbols are indirectly edited by the editing operation of the phonetic classification, according to Patent Document 1, the phonetic symbol string can be easily edited.

Today, music is composed using a computer. This music editing using a computer is also called DTM (desktop music). In this DTM, the user can play a favorite song by himself using MIDI (Musical Instruments Digital Interface) sound source, or the user himself can be a producer and think about the tone and melody. I enjoy making music.
Also, in DTM, using a voice synthesis technique, a computer is allowed to sing a song according to a song.
JP-A-5-11797

  The technique of Patent Document 1 described above proposes a phonetic symbol string generation device capable of easily generating an appropriate phonetic symbol string that gives a highly natural prosody, and a text-to-speech synthesis system using the same. . However, the technique of Patent Document 1 cannot be applied to a device or program for music editing work in which lyrics are assigned to musical notes of a musical score that has already been completed.

  In addition, in music editing, when adding lyrics and pronunciation to notes, it is difficult for a user who does not know phonetic symbols and syllable breaks to allocate one vowel to one note. Because, for example, when assigning English lyrics to a note, the user must know the syllable of the word. However, a general user must look at the dictionary to know the syllables of words, and it is troublesome to do so when inputting lyrics.

  The present invention has been made in view of the above-described circumstances, and an object thereof is to provide a mechanism that makes it easy to assign lyrics to musical notes.

  In order to achieve the above object, a singing synthesizing data input program according to the present invention includes singing score data composed of a plurality of note data including information for designating a sounding volume or a sounding period corresponding to a plurality of notes constituting a music piece. A storage process stored in the storage means, and input data input in association with a plurality of continuous note data stored in the storage means, a plurality of written characters constituting one word, and a continuous Input data including a notation character connection symbol for connecting between the two notation characters is acquired via an input device, and a word acquisition process for acquiring a word from the input data, acquired in the word acquisition process For a word, a word information acquisition process for acquiring a phonetic symbol indicating a pronunciation mode of the word and division information indicating the number of syllables constituting the word, and the word information acquisition The number represented by the classification information acquired in the process is compared with the number of note data associated with the word acquired in the word acquisition process, and if both match, one word is acquired in the word acquisition process. Are divided into syllables each including one vowel, and each syllable is a plurality of consecutive notes associated with the input data from which the word is acquired among the note data in the singing score data. When the syllable assigned to the data and the note data is a predetermined syllable and the sound volume or sound period specified by the note data is equal to or longer than a predetermined time, the phonetic symbol corresponding to the syllable is changed to another syllable. By changing to phonetic symbols, the singing score data updating process for updating the singing score data stored in the storage means is performed on the computer. Make.

  The singing synthesizing data input program according to the present invention includes a storing process of storing singing score data composed of a plurality of note data corresponding to a plurality of notes constituting a music piece in a storing unit, and a plurality of storing data stored in the storing unit. Input data that is input in association with consecutive note data, and includes a plurality of written characters that form one word and a written character connection symbol that connects two consecutive written characters. A word acquisition process for acquiring input data via an input device and acquiring a word from the input data, a phonetic symbol indicating the pronunciation of the word, and a word for the word acquired in the word acquisition process A word information acquisition process for acquiring classification information representing the number of syllables, a number represented by the classification information acquired in the word information acquisition process, and the word acquisition process. The number of note data associated with the word acquired in this way is compared, and if they match, the phonetic symbol obtained from one word in the word acquisition process each includes a vowel Each syllable is assigned to a plurality of continuous note data associated with the input data from which the word is acquired among the note data in the singing score data, and the syllable assigned to the note data is predetermined. The singing score data stored in the storage means is updated by correcting the phonetic symbol corresponding to the syllable based on the syllables assigned to the note data before and after the note data. Let the computer execute the singing score data update process.

  According to the present invention, if the user has no knowledge of correct syllable breaks and the number of vowels is correctly input even if lyrics are input incorrectly, the correct phonetic symbol is set.

A preferred embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of a computer 1 having a function as a singing synthesizing data input device according to an embodiment of the present invention. In the computer 1 shown in FIG. 1, a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, an operation unit 14, an HDD (Hard Disk Drive Device) 15, a display 16, and data input / output. The units 17 are connected via a bus BUS and can exchange data with each other. The sound source unit 18 and the speaker 19 are connected to the computer 1 as external devices, but may be configured as devices inside the computer 1.

  The CPU 11 is a microprocessor that performs general-purpose data processing. In addition to the computer 1 according to a control program such as BIOS (Basic Input / Output System) stored in the ROM 12 and an OS (operating system) stored in the HDD 15, The control processing of the constituent parts is performed.

  The ROM 12 is a non-volatile memory that stores a control program such as BIOS. The RAM 13 is a volatile memory for temporarily storing data used by the CPU 11 and other components. The BIOS in the ROM 12 is read by the CPU 11 and written in the RAM 13 when the computer 1 is turned on. The CPU 11 constructs a hardware usage environment according to the BIOS in the RAM 13. The operation unit 14 includes a keypad, a mouse, and the like, and transmits data reflecting operation contents performed by the user to the CPU 11. The HDD 15 is a nonvolatile memory having a large-capacity storage area, and data stored in the HDD 15 can be rewritten. The HDD 15 stores an OS, various applications, and data used by the various applications. The CPU 11 reads out the OS from the HDD 15 and writes it into the RAM 13 after constructing the hardware environment by the BIOS, and performs processing such as construction of a GUI (Graphical User Interface) environment and an application execution environment according to the OS. Among the applications stored in the HDD 15, there is a singing composition data input application.

  When the CPU 11 receives an instruction to execute the song synthesis data input application from the user by operating the mouse or the like, the CPU 11 reads the song synthesis data input application from the HDD 15 and writes it into the RAM 13, and performs various processes according to the song synthesis data input application. Build the environment to do. In this way, the computer 1 functions as a song synthesis data input device according to the present embodiment.

  The display 16 includes a liquid crystal display and a drive circuit that drives the liquid crystal display under the control of the CPU 11 and displays information such as characters and graphics.

The data input / output unit 17 is, for example, a USB (Universal Serial Bus) interface or an interface capable of inputting / outputting various data. The data input / output unit 17 receives data from an external device, transfers the received data to the CPU 11, or is generated by the CPU 11. Send data to external devices.
The sound source unit 18 generates a musical sound signal based on the input data and outputs it as a musical sound from the speaker 19.

  FIG. 2 is a block diagram showing a functional configuration of a singing voice synthesizing data input device provided by the CPU 11 of the computer 1 executing a singing voice data input application. As shown in the figure, the singing voice synthesizing data input device has an operation means 21, an external data input means 22, a data editing means 23, a display means 24 and a storage means 25. Among these, the operation unit 21 is the operation unit 14 of the computer 1, and the storage unit 25 is the RAM 13 and the HDD 15 of the computer 1. The external data input means 22 is the data input / output unit 17 of the computer 1, and the display means 24 is the display 16. The data editing means 23 is a software module that constitutes a singing synthesizing data input application.

  The external data input unit 22 acquires the singing score data 251 from an external device or the like, and transmits the acquired singing score data 251 to the data editing unit 23. The data editing unit 23 stores the received singing score data 251 in the storage unit 25, and edits the singing score data 251 in accordance with the operation of the operation unit 21. The display unit 24 displays the singing score data 251 stored in the storage unit 25.

  FIG. 3 illustrates the configuration of this singing score data 251. The singing score data 251 is time-series data composed of note data corresponding to each of a series of notes constituting the music. In FIG. 3, the data for one line arranged side by side indicates one note data. One piece of note data is composed of data of numbers, pitches, phonetic symbols, pronunciation periods, input characters, and display characters. Here, the number in each piece of note data indicates the number of the note represented by the note data from the beginning of the song. The pitch and sound generation period in each note data designate the pitch and sound generation period of the note corresponding to the note data. The phonetic symbols, input characters, and display characters are information that the data editing unit 23 assigns to each piece of note data in accordance with the operation of the operation unit 21 by the user. The feature of the present embodiment is in the assignment processing of phonetic symbols to each note data performed by the data editing means 23. The storage means 25 stores dictionary data 253 and editing rule data 252 that are referred to in the process of assigning phonetic symbols to the respective note data.

  FIG. 4 shows the structure of dictionary data. Dictionary data is a collection of data corresponding to each of a large number of written characters such as English words, and the data corresponding to one word represents the notation of the word and the manner in which the word is pronounced. It consists of a phonetic symbol string and information indicating the number of divisions into how many syllables the voice when the word is pronounced is divided into.

In the present embodiment, the user can input lyrics to be assigned to a series of note data by operating the operation means 21. When inputting the lyrics, there are cases where a word consisting of a plurality of syllables is assigned to a plurality of consecutive notes.
The present embodiment is intended for the convenience of the user in such a case. In the present embodiment, when the user wants to assign a word consisting of n syllables to n consecutive notes, only the following rule should be observed.
a. The first notation character in the word is input in association with the first note, and the last notation character in the word is input in association with the last note.
b. Data input in association with a note other than the last note must always end with a notation character connection symbol (for example, a hyphen).
c. The notation characters other than the first notation character and the last notation character in the word are input in the same order as the appearance order in the word.

When the data editing unit 23 receives data input according to this rule from the operation unit 21, the data editing unit 23 interprets the notation character string connected by the notation character connection symbol as one word. Then, the data editing unit 23 refers to the dictionary data 253 stored in the storage unit 25 to obtain a phonetic symbol string corresponding to this word, and uses it as n syllables each including one vowel. Divided into n pieces of note data.
The edit rule data 252 is data representing a special rule referred to in the assignment process described above. For example, in English, there is an ambiguous vowel (Schwa sound). An ambiguous vowel is used when the pronunciation is weak and the pronunciation period is short. When the volume of the corresponding note is large or the pronunciation period is long, it is converted into a normal vowel. As a more specific example, when a Schwa sound is assigned to a note lasting 0.5 seconds or more in the assignment process, the data editing means 23 converts this into a phonetic symbol “Q”.

  Another rule is to change the pronunciation of “h”. This is a rule that makes the pronunciation of “h” unvoiced or voiced according to the previous and next pronunciations. Specifically, when the phonetic symbol “h” is assigned to certain note data in the assignment process, the data editing means 23 assigns a vowel to the note data before the note data, and When there is no rest between the note data and the note data, the pronunciation of “h” is voiced. For example, if there is a rest between the words “She hits” and “She” and “hits”, the “h” in “hits” is silent, and if there is no rest, it is voiced.

  Next, the operation of the singing synthesizing data input device will be described. When the data input application for song synthesis is executed by the CPU 11, a piano roll as shown in FIG. Next, when an instruction to read the singing score data is given to the operation means 21, the singing score data is read from the outside using the external data input means 22.

  The read singing score data is stored in the storage means 25 via the data editing means 23. The singing score data stored in the storage means 25 is displayed on the display means 24 as a piano roll screen. FIG. 5 shows this piano roll screen. In this piano roll screen, the up and down direction corresponds to the arrangement direction of the piano keys, and the lower side represents the bass key and the upper side represents the treble key. That is, the vertical axis of the piano roll screen is the pitch axis. The horizontal axis of the piano roll screen is a time axis. In the drawing, a rectangle surrounded by a thick line represents individual note data included in the singing score data, and is called a note bar. The position of the note bar corresponding to each note data in the pitch axis direction indicates the pitch specified by the note data. The position of the note bar corresponding to each note data in the time axis direction indicates the sounding timing of the note designated by the note data. Further, the length of the note bar corresponding to each note data indicates the sound generation period designated by the note data.

  In the present embodiment, when a word is divided and assigned to a plurality of note data, the user inputs the word by the operation means 21 using a hyphen indicating that words are continuous as shown in FIG. That is, the word “amazing” in FIG. 6 is input as “a−”, “ma−”, and “zing”, but “−” attached to “a” and “ma” is the input of the word Is still in the middle, indicating that there are still characters to be assigned to the subsequent note bar. In the example shown in the figure, “a−” is connected to “ma−”, and the connected one is further connected to “zing”. Since “zing” does not have a hyphen, it is a written character string at the end of the word.

  How “amazing” input in this way is processed in the present embodiment will be described below.

  First, the user uses the operation means 21 to enter a lyric input waiting state in which lyrics can be input to the note bar by, for example, double-clicking the first note bar NB1 to which “amazing” is assigned. Then, “a” and “−” are input by the operating means 21 and the enter key is pressed.

  Thereby, the data editing means 23 acquires the input data “a−” in association with the note bar NB1. The input data “a−” is displayed in association with the note bar NB1, and is stored in a buffer area (not shown) secured in the storage means 25. Next, the user designates the note bar NB2 and inputs “ma−” by the same operation. Accordingly, the data editing unit 23 adds the newly acquired input data “ma−” to the existing input data “a−” in the buffer area in the storage unit 25. As a result, the input data in the buffer area is “a-ma-”. Next, the user designates the note bar NB3 and inputs “zing” by the same operation. Accordingly, the data editing unit 23 adds the newly acquired input data “zing” to the existing input data “a-ma-” in the buffer area in the storage unit 25. As a result, the input data in the buffer area is “a-ma-zing”. In this case, since the input “zing” does not end with “−” at the end, the data editing unit 23 detects that input of all the notation characters constituting the word is completed.

  Next, the data editing unit 23 removes the hyphen from the input data “a-ma-zing” in the buffer area in the storage unit 25 to obtain the word “amazing”.

Next, the data editing unit 23 searches for “amazing” in the dictionary data 253 in order to obtain the number of divisions and pronunciation symbols of “amazing”. Since the dictionary data 253 stores “3” and “@meIZIN” as the number of divisions of “amazing” and the phonetic symbols, respectively, the data editing means 23 obtains them.
Next, the data editing means 23 compares the number of divisions with the number of note data (note bar) to which the user has assigned words. If there is a discrepancy between the numbers, an error display is displayed on the display means 24 and the processing is stopped.

  Next, the data editing means 23 divides the obtained phonetic symbol string in order to assign it to a note. At this time, it is divided using a rule with the position of the vowel as a delimiter. This is because, in a normal score, one syllable corresponds to one note, and one syllable contains one vowel. Since the phonetic symbol of “amazing” is “@meIZIN”, it is divided into three syllables “@”, “meI”, and “ZIN”. Then, the data editing means 23 stores the divided syllables in the phonetic symbol column of each note data corresponding to the note bars NB1, NB2 and NB3 in the singing score data.

Next, the data editing unit 23 corrects the phonetic symbols in the singing score data in the storage unit 25 according to the rules stored in the editing rule data 252. FIG. 7 shows final singing score data that has undergone this processing. The phonetic symbols shown in FIG. 7 are phonetic alphabets according to a sampler (SAMPA) that specifies phonetic symbols so that they can be easily handled on a computer.
In the case of the data shown in FIG. 6, since there is no rule that applies to the edit rule data 252, nothing is changed.
In this way, final singing score data corresponding to “amazing” is obtained in the storage means 25.

Next, with reference to FIG. 8A, an operation when the user assigns and inputs the word “amazing” to four notes will be described. In this example, the user inputs notation characters in NB1 to NB4 of the note bar as shown in the figure. Since the number of syllables of the word “amazing” is “3”, this is an example in which the user misunderstands the input.
In this case, the data editing unit 23 stores “a-ma-zi-ng” input by the user in the buffer area, as in the above-described processing.
Next, the data editing means 23 obtains the word “amazing” by removing the hyphen from “a-ma-zi-ng” stored in the buffer area.

  Next, the data editing unit 23 searches for “amazing” in the dictionary data 253 in order to obtain the number of divisions and pronunciation symbols of “amazing”. Since the dictionary data 253 stores “3” and “@meIZIN” as the number of divisions of “amazing” and the phonetic symbols, respectively, the data editing means 23 obtains them. Next, the data editing means 23 compares the number of divisions with how many note bars the word is input over. In the case of “amazing”, the user is input across four note bars, and the number of note bars is different from the number of sections. Therefore, the data editing unit 23 gives an error display to the display unit 24 and stops the processing.

  Next, when the user correctly assigns and inputs the word “amazing” to the three notes, but as shown in FIG. 8B, the operation when the syllable of the word “amazing” is erroneously input Will be explained. In this case, “ama-zi-ng” input by the user is stored in the buffer area. Since hyphens are removed from the buffer area, the content of the buffer area is the word “amazing” as described above. . Accordingly, as described above, the three syllables “@”, “meI”, and “ZIN” constituting the phonetic symbol of this word are assigned to the three note data.

Next, as shown in FIG. 8C, an operation in the case where the word “amazing” is assigned to four notes and input will be described. This is not when the user misunderstands the number of syllables of the word “amazing” as 4, but when the user correctly recognizes the number of syllables and assigns the word “amazing” to four notes. .
Specifically, it is assumed that the user inputs to the note bars NB1 to NB4 as shown in FIG. 8C with the intention of extending the sound “ma−”.
In this case, the user inputs only “−” in the note bar next to the note bar in which “ma−” is input. The “-” input in this way is replaced with another special symbol, for example, “*”, and stored in the buffer area.

Therefore, when the user finishes inputting the four note bars, “a-ma- * zing” remains in the buffer area. The data editing unit 23 deletes the hyphen from “a-ma- * zing” stored in the buffer area to obtain a character string “ama * zing”. Next, when the data editing unit 23 detects that “ama * zing” in the buffer area includes “*”, the data editing unit 23 copies “ama * zing” to the second buffer area, The “*” is removed from “amai * zing” of the word to obtain the word “amazing”.
Next, the data editing unit 23 searches for “amazing” in the dictionary data 253 in order to obtain the number of divisions and pronunciation symbols of “amazing”. Since the dictionary data 253 stores “3” and “@meIZIN” as the number of divisions of “amazing” and the phonetic symbols, respectively, the data editing means 23 obtains them.

Next, the data editing means 23 compares the number of divisions with the number of note bars that are input across words. In this case, while the number of divisions of “amazing” is “3”, the number of note bars in which the user inputs “amazing” is “4”, and the number of note bars is more than the number of divisions. Only 1 ”.
However, one “*” is stored in the second buffer area, which means that one of the four note bars takes over the sound of the preceding note bar. . In this case, the data editing means 23 subtracts the number of “*” from the number of note bars and compares it with the number of sections. Since the number is the same here, the data editing means 23 continues the processing.

Then, the data editing unit 23 performs the following process. First, the data editing unit 23 sequentially extracts syllables from the head of “ama * zing” in the buffer area. Since “a” is extracted first, the phonetic symbol “@” corresponding to this syllable is assigned to the first note data among the respective note data corresponding to the four note bars to which “amazing” is input. .
Next, since “ma” is extracted, the phonetic symbol “meI” corresponding to the extracted syllable is assigned to the second note data.
Next, since “*” is extracted, the phonetic symbol “I” for extending the previous phonetic symbol “meI” is assigned to the third note data.
Finally, since “zing” is extracted, the phonetic symbol “ZIN” corresponding to the extracted syllable is assigned to the fourth note data.
Through the processing as described above, singing score data as shown in FIG. 9 is obtained.
Thus, according to the present embodiment, when a user inputs a word across a plurality of note bars, each syllable constituting the word is automatically assigned to each note data corresponding to each note bar, Singing score data is generated. Therefore, the user can easily edit the singing score data.

  In this embodiment, an example in which an English alphabet is used as a written character has been described. However, the written character may be a Chinese character or a symbol string as long as it is converted into a phonetic symbol by dictionary data.

It is a figure which shows the structure of the computer which implement | achieves the data input interface for song synthesis | combination by this invention. It is a figure which shows the functional block diagram of the data input interface for song synthesis by this invention. It is a figure which shows the structure of song score data. It is a figure which shows the structure of dictionary data. It is a figure which shows the example of a display of a piano roll. It is a figure which shows the specific example which inputs a lyrics into a piano roll. It is a figure which shows the structure of song score data. It is a figure which shows the specific example which inputs a lyrics into a piano roll. It is a figure which shows the structure of song score data.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Computer system, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Operation part, 15 ... HDD, 16 ... Display, 17 ... Data entry Output unit, 18 ... sound source unit, 19 ... speaker, 21 ... operating means, 22 ... external data input means, 23 ... data editing means, 24 ... display means, 25 ... -Memory means.

Claims (2)

A storage process of storing in the storage means singing score data composed of a plurality of note data including information specifying the sound production volume or the sound generation period corresponding to a plurality of notes constituting the music;
Input data that is input in association with a plurality of continuous note data stored in the storage means, and connects between a plurality of written characters constituting one word and the two continuous written characters A word acquisition process of acquiring input data including a notation character connection symbol via an input device, and acquiring a word from the input data;
For a word acquired in the word acquisition process, a word information acquisition process for acquiring a pronunciation symbol indicating a pronunciation mode of the word and division information indicating the number of syllables constituting the word;
If the number represented by the category information acquired in the word information acquisition process is compared with the number of note data associated with the word acquired in the word acquisition process, and the two match, the word acquisition process The phonetic symbol obtained from one word is divided into syllables each including one vowel, and a plurality of syllables are associated with input data from which the word is acquired among the note data in the singing score data. Assigned to the continuous note data, and the syllable assigned to the note data is a predetermined syllable, and the sound volume or sound generation period specified by the note data is equal to or longer than a predetermined time, the sound corresponding to the syllable A singing score data update process for updating the singing score data stored in the storage means by changing the symbol to another phonetic symbol; A data input program for singing synthesis characterized by being executed by a computer. A storage process of storing in the storage means singing score data composed of a plurality of note data corresponding to a plurality of notes constituting the music;
Input data that is input in association with a plurality of continuous note data stored in the storage means, and connects between a plurality of written characters constituting one word and the two continuous written characters A word acquisition process of acquiring input data including a notation character connection symbol via an input device, and acquiring a word from the input data;
For a word acquired in the word acquisition process, a word information acquisition process for acquiring a pronunciation symbol indicating a pronunciation mode of the word and division information indicating the number of syllables constituting the word;
When the number represented by the category information acquired in the word information acquisition process is compared with the number of note data associated with the word acquired in the word acquisition process, and the two match, the word acquisition process The phonetic symbol obtained from one word is divided into syllables each including one vowel, and a plurality of syllables are associated with input data from which the word is acquired among the note data in the singing score data. If the syllable assigned to the note data is a predetermined syllable, the phonetic symbol corresponding to the syllable is determined based on the syllables assigned to the note data before and after the note data. The singing score data updating process for updating the singing score data stored in the storage means, A data input program for singing synthesis characterized by being executed.

Images