JP7618983B2 - Electronic musical instrument, electronic musical instrument control method, and program - Google Patents

Description

The present invention relates to an electronic musical instrument that reproduces singing voices in response to the operation of controls such as a keyboard, a control method for an electronic musical instrument, and a program.

For keyboard instruments, a music training system is known that allows a user to take lessons on an electronic instrument based on music data that indicates the pitch to be played and the timing of the playing, and inputs signals from the electronic instrument to evaluate and score the user on various aspects (for example, the system described in Patent Document 1).

In recent years, application software for smart devices (hereafter referred to as "apps") has become known that is connected to an electronic keyboard instrument via MIDI (Musical Instrument Digital Interface), and that adds points by pressing the corresponding keys on the keyboard of the electronic keyboard instrument when a bar (piano roll) that falls from the top of the screen as the song progresses reaches the keys at the bottom of the screen. Some such apps can also display the progress of the scoring during the song, and the results of the scoring after the song has ended, allowing users to practice playing without getting bored (for example, the app described in non-patent document 1).

Japanese Patent Application Publication No. 10-187021

"Chordana Play for Piano: MIDI Player", Casio Computer Co., Ltd., [Retrieved July 20, 2020], Internet <URL https://web.casio.com/app/ja/piano/>

However, the above-mentioned conventional technology is based on the premise that the performance progress of the user's musical instrument performance and the scoring results are displayed on a display screen. Such displays are only possible with a display device such as a high-definition display mounted on an electronic musical instrument or a display on a smart device.

However, in general electronic musical instruments, particularly low-cost electronic musical instruments aimed at beginners that are the target of such performance practice functions, the display devices installed are small and low resolution, or in many cases no display device is installed at all. Therefore, even if the performance practice function itself can be implemented in such electronic musical instruments alone, it has been difficult to provide it to users as a function that is easy to understand and will not tire of.

Furthermore, even if the performance progress is shown on the display device, beginners in particular may be so distracted by their own performance that they do not have time to look at the display device other than the keyboard.

The present invention aims to enable users to practice playing in an easy-to-understand manner, without losing motivation or getting bored, even on electronic musical instruments that have only a low-quality display device or no display device, and to enable users to receive information about playing practice even without a display device.

An electronic musical instrument according to one embodiment of the present invention includes a performance information acquisition means for acquiring performance information of a performer, a performance evaluation means for evaluating the performance of the performer multiple times at different times while a piece of music is being played based on the performance guide data and the performance information, the performance information including at least lyric information, pitch information, and timing information, a singing voice production means for vocalizing the lyrics based on the performance information and the lyric information, and a voice quality change means for, when the evaluation of the performance evaluation means has changed from the previous evaluation, changing the voice quality of the singing voice in response to the changed evaluation.

The present invention makes it possible to allow a user to practice playing an instrument in an easy-to-understand manner, without losing motivation or becoming bored, even on electronic musical instruments that have only a low-quality display device or no display device, and also makes it possible to convey information about playing practice to the user even without a display device.

1 is a diagram showing an example of the appearance of an embodiment of an electronic keyboard instrument; 1 is a block diagram showing an example of a hardware configuration of an embodiment of a control system for an electronic keyboard instrument; FIG. 2 is a block diagram showing a configuration example of a voice synthesis LSI. FIG. FIG. 2 is a diagram illustrating an example of a data configuration according to an embodiment. 13 is a flowchart showing an example of the overall processing of a performance lesson. 13 is a flowchart showing a detailed example of a lesson process. 13 is a flowchart showing a detailed example of an automatic playback process. 13 is a flowchart showing a detailed example of a performance guide process. 11 is a flowchart showing a detailed example of a key pressing/releasing process. 13 is a flowchart showing a detailed example of a scoring process. 13 is a flowchart showing a detailed example of a voice quality updating process. 13 is a flowchart showing an example of a voice quality change process. 13 is a flowchart showing a detailed example of a noise mixing ratio interpolation process and a formant interpolation process. 13 is a block diagram showing another example of the configuration of the vocalization model unit 303 in the voice synthesis unit 300 in the voice synthesis LSI. FIG.

Hereinafter, the embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram showing an example of the appearance of an embodiment 100 of an electronic keyboard instrument. The electronic keyboard instrument 100 includes a keyboard 101 consisting of a plurality of keys as operators, a first switch panel 102 for instructing various settings such as specifying the volume, setting the tempo for automatic lyrics playback, and starting automatic lyrics playback, and a second switch panel 103 for selecting songs and selecting instrument tones. Each key of the keyboard 101 includes an LED (Light Emitting Diode) 104. This LED 104 lights up at maximum brightness when the key it includes is the key that should be specified next during automatic lyrics playback, and lights up at half the maximum brightness when the key is the key that should be specified after the next key during automatic lyrics playback. Furthermore, the electronic keyboard instrument 100 includes a speaker on the back, side, or rear part, etc., which emits musical tones generated by playing, although not shown in particular.

2 is a diagram showing an example of the hardware configuration of an embodiment of the control system 200 of the electronic keyboard instrument 100 of FIG. 1. In FIG. 2, the control system 200 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, a sound source LSI (Large Scale Integrated Circuit) 204, a voice synthesis LSI 205, a key scanner 206 to which the keyboard 101 of FIG. 1, the first switch panel 102 and the second switch panel 103 are connected, an LED controller 207 to which the LEDs 104 of each key on the keyboard 101 of FIG. 1 are connected, and a network interface 208 for exchanging MIDI data with an external network is connected to a system bus 209. A timer 210 for controlling the sequence of automatic playback of vocal data is also connected to the CPU 201. Furthermore, the musical tone output data 218 and singing voice output data 217 output from the sound source LSI 204 and the voice synthesis LSI 205, respectively, are converted by D/A converters 211 and 212 into an analog musical tone output signal and an analog singing voice output signal, respectively. The analog musical tone output signal and the analog singing voice output signal are mixed in a mixer 213, and the mixed signal is amplified in an amplifier 214 and then output from a speaker or an output terminal (not shown).

The CPU 201 executes the control program stored in the ROM 202 while using the RAM 203 as a work memory, thereby executing the control operation of the electronic keyboard instrument 100 in FIG. 1. The ROM 202 also stores the control program and various control data, as well as performance guide data, which will be described later, including lyric data.

The CPU 201 is equipped with a timer 210 used in this embodiment, which counts the progress of automatic playback of performance guide data in the electronic keyboard instrument 100, for example.

The sound source LSI 204 reads musical tone waveform data, for example from a waveform ROM (not shown), in accordance with sound generation control instructions from the CPU 201, and outputs the data to the D/A converter 211. The sound source LSI 204 has the ability to generate up to 256 voices simultaneously.

When the voice synthesis LSI 205 receives from the CPU 201 lyric information, which is text data of lyrics, and pitch information regarding pitch as singing voice data 215, it synthesizes singing voice output data 217, which is the voice data of the corresponding singing voice, and outputs it to the D/A converter 212.

The key scanner 206 constantly scans the key-on/key-off state of the keyboard 101 in FIG. 1, the switch operation state of the first switch panel 102, and the second switch panel 103, and notifies the CPU 201 of state changes by issuing an interrupt.

The LED controller 207 is an IC (integrated circuit) that controls the display state of each LED 104 provided on each key on the keyboard 101 in FIG. 1.

Figure 3 is a block diagram showing an example of the configuration of the voice synthesis unit 300 in this embodiment. Here, the voice synthesis unit 300 is built into the electronic keyboard instrument 100 as one function executed by the voice synthesis LSI 205 in Figure 2.

The voice synthesis unit 300 synthesizes and outputs singing voice output data 217 by inputting singing voice data 215 including lyric information and pitch information instructed by the CPU 201 in Fig. 2. At this time, the processor of the voice synthesis unit 300 executes vocalization processing to output singing voice output data 217 that infers the singer's singing voice based on the spectrum information 309 and sound source information 310 output by the acoustic model unit 301 in response to the singing voice data 215 including lyric information and pitch information input by the CPU 201, for the acoustic model set in the acoustic model unit 301. The voice synthesis unit 300 is implemented based on the technology described in the following patent document, for example.

(Patent Document): Japanese Patent No. 6610714 Although details of the operation of the speech synthesis unit 300 are disclosed in the above-mentioned patent document, an outline of the operation will be described below.

The speech synthesis unit 300 includes a text analysis unit 302, an acoustic model unit 301, a speech model unit 303, a formant interpolation processing unit 306, and a noise superimposition unit 307.

The voice synthesis unit 300 executes a statistical voice synthesis process to synthesize singing voice output data 321 corresponding to singing voice data 215 including lyric information, which is the text of lyrics, and sound source information 310, by predicting the data using a statistical model called an acoustic model set in the acoustic model unit 301.

The text analysis unit 302 inputs singing voice data 215 including lyric information related to the phonemes, pitch, etc. of the lyrics specified by the CPU 201 in FIG. 2, and analyzes the data. As a result, the text analysis unit 302 analyzes and outputs a linguistic feature sequence 308 expressing the phonemes, parts of speech, words, etc. corresponding to the singing voice data 215.

The acoustic model unit 301 receives the language feature sequence 308 and the pitch information in the singing data 215, and estimates and outputs the corresponding spectral information 309 and sound source information 310. That is, the acoustic model unit 301 outputs estimates of the spectral information 309 and sound source information 310 that maximize the generation probability, based on the language feature sequence 308 and the pitch information in the singing data 215 received from the text analysis unit 302, using an acoustic model set as a learning result by machine learning, for example.

The vocal model unit 303 receives the spectrum information 309 and the sound source information 310 to generate singing voice output data 321 corresponding to the singing voice data 215 including the lyric information and pitch information specified by the CPU 201. The singing voice output data 321 is output from the D/A converter 212 in FIG. 2 via the mixer 213 and the amplifier 214, and is output from a speaker (not shown).

The acoustic features output by the acoustic model unit 301 include spectral information 309 that models the human vocal tract and sound source information 310 that models the human vocal cords. For example, the parameters of the spectral information 309 can be line spectral pairs (LSP) or line spectral frequencies (LSF), which can efficiently model multiple formant frequencies that are human vocal tract characteristics. For the sound source information 310, a fundamental frequency (F0) and a power value that indicate the pitch frequency of human voice can be used. The vocalization model unit 303 includes a sound source generation unit 304 and a synthesis filter unit 305. The sound source generating unit 304 is a part that models the human vocal cords, and by sequentially inputting a series of sound source information 310 input from the acoustic model unit 301, generates a sound source signal consisting of, for example, a pulse train (in the case of a voiced phoneme) that is periodically repeated at the fundamental frequency (F0) and power value included in the sound source information 310, or a white noise (in the case of an unvoiced phoneme) having a power value included in the sound source information 310, or a signal that is a mixture of these. The synthesis filter unit 305 is a part that models the human vocal tract, and forms a digital filter that models the vocal tract based on the series of spectrum information 309 that is sequentially input from the acoustic model unit 301. The sound source signal input from the sound source generating unit 304 is used as an excitation source signal to excite the digital filter, and filter output data 313 of a digital signal that is the basis of singing voice output data 321 is output.

The sampling frequency for the singing voice output data 217 is, for example, 16 KHz (kilohertz). Furthermore, when LSF parameters obtained by, for example, LSP analysis processing are used as the parameters of the spectrum information 309, the update frame period is, for example, 5 milliseconds. Furthermore, in the case of LSF analysis processing, the analysis window length is, for example, 25 milliseconds, the window function is, for example, a Blackman window, and the analysis order is, for example, 10th order.

The operation of this embodiment will be described in outline with the configuration of Figs. 2 and 3. First, the CPU 201 operates as a performance guide means for guiding the performer in playing a piece of music based on performance guide data including at least lyric information, pitch information, and timing information. Specifically, in Fig. 2, the CPU 201 executes an automatic playback process in which it sequentially reads out a series of sets of performance guide data including at least lyric information, pitch information, and timing information for automatic playback stored in the ROM 202, which is a memory, and automatically plays back the lyric information and pitch information included in the set of performance guide data at a timing corresponding to the timing information included in the set of performance guide data. Details of this automatic playback process will be described later with reference to the flowchart of Fig. 8.

At this time, the CPU 201 executes a performance guide process that indicates the key on the keyboard 101 that corresponds to the pitch information to be automatically played back, and guides the user to perform a performance lesson (performance practice) by pressing the key in synchronization with the automatic playback. More specifically, in this performance guide process, the CPU 201 synchronizes with the timing of the automatic playback, and causes the LED 104 of the key (operator) corresponding to the pitch information to be automatically played back next to be lit with high brightness, for example, maximum brightness, and causes the LED 104 of the key corresponding to the pitch information to be automatically played back next to be lit with low brightness, for example, half the maximum brightness, as shown by the key with two lit LEDs 104 in FIG. 1. The details of this performance guide process will be described later using the flowchart in FIG. 9.

Next, the CPU 201 operates as a performance information acquisition means for acquiring performance information of the performer. Specifically, the CPU 201 acquires the performance operations of the performer pressing or releasing the keys on the keyboard 101 in FIG. 1 according to the performance guide.

The CPU 201 also operates as a performance evaluation means for evaluating the performer's performance from time to time while the piece of music is being played, based on the performance guide data and performance information. Specifically, the CPU 201 executes a scoring process for scoring the performance lesson by comparing the key press timing (operation timing) and key press pitch (operation pitch) in the performance lesson with the automatically reproduced timing information and pitch information. Details of this scoring process will be described later using the flowchart in FIG. 11.

Then, the CPU 201 operates as a singing voice producing means for vocalizing lyrics based on the performance information and lyric information. Specifically, when the timing (operation timing) of pressing a key on the keyboard 101 in the performance lesson and the key pressing pitch (operation pitch) of the key correctly correspond to the timing information and pitch information for automatic playback, the CPU 201 inputs the lyric information and pitch information to be automatically played back at the key pressing timing as singing voice data 215 to the acoustic model unit 301 via the text analysis unit 302 in FIG. 3, and outputs filter output data 313 by exciting the digital filter of the synthesis filter unit 305 formed based on the spectrum information 309 output from the acoustic model unit 301 by the sound source signal output by the sound source generation unit 304 in which the sound source information 310 output from the acoustic model unit 301 is set, and the filter output data 313 is output as the singing voice output data 217 in FIG. 2.

At this time, the CPU 201 operates as a voice quality change means for changing the voice quality of the singing voice in response to the performance evaluation. Specifically, the CPU 201 in FIG. 2 and the formant interpolation processing unit 306 and the noise superimposition unit 307 in the voice synthesis unit 300 in FIG. 3 execute a voice quality change process for changing the voice quality of the singing voice output data 217 output in the vocalization process in response to the intermediate scoring results in the scoring process described above.

At this time, the CPU 201, operating as a voice quality change means, interpolates between a plurality of voice qualities corresponding to a plurality of specific performance evaluations at a ratio according to the performance evaluation during the progression of the music piece. The CPU 201 also changes the voice quality, for example, by changing the ratio of the formant components of the human voice and the noise components mixed with the human voice.

More specifically, in the above-mentioned voice quality change process, the CPU 201 first calculates practice progress data 311 indicating the degree of progress in the performance lesson according to the intermediate scoring results in the above-mentioned scoring process. Then, the formant interpolation processing unit 306 in FIG. 3 executes an interpolation process at a ratio according to the practice progress data 311 given by the CPU 201 between one or more sets of spectrum information 309 output from the acoustic model unit 301 corresponding to the lyric information in the singing voice data 215 in FIG. 3 that has a pleasant voice quality including a female voice and is automatically played back, which corresponds to a case where the intermediate scoring result is good, and one or more sets of spectrum information 309 output from the acoustic model unit 301 corresponding to the lyric information in the singing voice data 215 that has a harsh voice quality including a male voice and is automatically played back, which corresponds to a case where the intermediate scoring result is bad, thereby calculating the target spectrum information 312 and inputting it to the synthesis filter unit 305 in the vocalization model unit 303.

In the above-mentioned voice quality change process, in addition to the operation of the formant interpolation processing unit 306, the noise mixing ratio interpolation processing unit 316 of Figure 3 performs an interpolation process in a ratio corresponding to the above-mentioned practice progress data 311 provided by the CPU 201 between one or more sets of noise mixing ratios output from the acoustic model unit 301 corresponding to the lyric information in the singing voice data 215 of Figure 3, which has a pleasant voice quality including a female voice and is automatically played back corresponding to a case where the intermediate scoring result is good, and one or more sets of noise mixing ratios output from the acoustic model unit 301 corresponding to the lyric information in the singing voice data 215, which has a harsh voice quality including a male voice and is automatically played back corresponding to a case where the intermediate scoring result is poor, to calculate a target noise mixing ratio 317 and input it to the noise superimposition unit 307. Then, the noise superimposing unit 307 generates noise data 315 having an amplitude value obtained by multiplying the maximum amplitude value of the singing voice output data 217 by the target noise mixing ratio 317 calculated by the noise mixing ratio interpolation processing unit 316, mixes it with the filter output data 313 output from the synthesis filter unit 305, and outputs it as the singing voice output data 217.

The above-mentioned two voice quality changing processes realize a function (hereinafter referred to as a "morphing function") that gradually changes the singing voice quality from one character (e.g., adult male) to another character (e.g., adult female) when the user makes the electronic keyboard instrument 100 sing during a performance lesson, as shown in FIG. 4(a), for example. In this embodiment, as shown in FIG. 4(a), the voice quality of the singing function at the start of the performance lesson is set to, for example, an adult male, and after the performance lesson begins, as the tasks are cleared and the intermediate scoring results improve, the voice quality gradually changes to an adult female voice quality, and conversely, if the score decreases, the voice quality changes to a slightly harsh voice, known as a hoarse voice.

As described above, the voice synthesis unit 300 in this embodiment produces a voice by passing an excitation source corresponding to the vibration of the human vocal cords through a filter corresponding to the characteristics of the human vocal tract. As shown in FIG. 4(b), the characteristics of the filter corresponding to the vocal tract characteristics correspond to the so-called formants of the human voice, and the character of the human voice depends heavily on these characteristics. Therefore, in this embodiment, in the formant interpolation processing unit 306 in FIG. 3, the characteristics in the synthesis filter unit 305 are gradually changed by the target spectrum information 312 obtained by interpolating the spectrum information 309 of multiple characteristics output by the acoustic model unit 301 based on the practice progress data 311 output by the CPU 201 based on the intermediate scoring results, thereby smoothly changing the character of one person's voice to the character of another person's voice.

In addition to the characteristics of the synthesis filter unit 305, a white noise component is added to the voice to make it more realistic. Therefore, in this embodiment, the noise mixing ratio interpolation processing unit 316 in FIG. 3 calculates a target noise mixing ratio 317 obtained by interpolation processing based on the practice progress data 311 output from the CPU 201 based on the intermediate scoring results, and the noise superimposition unit 307 calculates noise data 315 obtained by increasing or decreasing the amount of white noise added based on the target noise mixing ratio 317, and mixes the noise data 315 with the filter output data 313 output by the synthesis filter unit 305 to generate singing voice output data. This makes it possible to sing in a way that reflects the intermediate scoring results, which have expressive characteristics such as the characteristics of a so-called husky voice.

The operation of the electronic keyboard instrument 100 of this embodiment having the configuration shown in Figures 1, 2, and 3 will be described in detail below.

Figure 5 (a) is a diagram showing an example of the data structure of performance guide data loaded from ROM 202 in Figure 2 to RAM 203 in this embodiment. This data structure example conforms to the standard MIDI file format, which is one of the MIDI file formats. This song data is made up of data blocks called chunks. Specifically, the song data is made up of a header chunk at the beginning of the file, followed by track chunks in which lyric data for the lyric parts is stored. Note that there may also be a separate track chunk in which automatic performance data for the accompaniment parts is stored.

A header chunk consists of four values: ChunkID, ChunkSize, FormatType, NumberOfTrack, and TimeDivision. ChunkID is the 4-byte ASCII code "4D 54 68 64" (numbers are hexadecimal), which corresponds to the four half-width characters "MThd" that indicate that it is a header chunk. ChunkSize is 4-byte data that indicates the data length of the FormatType, NumberOfTrack, and TimeDivision parts of the header chunk, excluding ChunkID and ChunkSize, and the data length is fixed at 6 bytes: "00 00 00 06" (numbers are hexadecimal). In this embodiment, FormatType is 2-byte data "00 00" (numbers are hexadecimal), which means format 0 uses a single track. NumberOfTrack is 2-byte data "00 01" (numbers are hexadecimal), which means that one track corresponding to the lyrics part is used. TimeDivision is data indicating the time base value that indicates the resolution per quarter note, and in this embodiment, it is 2-byte data "01 E0" (numbers are hexadecimal), which indicates 480 in decimal.

A track chunk consists of a performance data set (0≦i≦L-1) consisting of ChunkID, ChunkSize, DeltaTime[i] and Event[i]. ChunkID is a 4-byte ASCII code "4D 54 72 6B" (numbers are hexadecimal) that corresponds to the four half-width characters "MTrk" that indicate a track chunk. ChunkSize is 4-byte data that indicates the data length of the portion of the track chunk excluding ChunkID and ChunkSize.

DeltaTime[i] is timing information indicating the wait time (relative time) from the execution time of the immediately preceding Event[i-1] (the beginning when i=0), and is variable length data of 1 to 4 bytes. Event[i] is a meta event that contains lyric information, which is the text data of the lyrics, and pitch information that indicates the pitch. In each performance guide data set DeltaTime1[i] and Event[i], Event[i] is executed after waiting DeltaTime[i] from the execution time of the immediately preceding Event[i-1], thereby realizing the progress of automatic playback (vocalization) of lyrics.

The data structure of Figures 5(b) and 5(c) will be described later.

Figure 6 is a flow chart showing an example of the overall processing of a performance lesson. This processing is realized as a process in which the CPU 201 loads the overall processing program of the performance lesson stored in the ROM 202 into the RAM 203 and executes it in Figure 2, and is called from a main processing program (not shown) that controls the electronic keyboard instrument 100.

First, the CPU 201 has the user operate the second switch panel 103 in FIG. 1 to select the vocal song for which the user wishes to practice playing lessons (step S601).

Next, the CPU 201 executes lesson processing (step S602). Details of this processing will be described later.

When the user's performance lesson is completed by the lesson process, the CPU 201 feeds back the evaluation score of the user's performance (step S603). Here, the CPU 201 provides the singing voice data 215 to the voice synthesis LSI 205 in FIG. 2, and vocalizes the evaluation score in a voice with a voice quality corresponding to the evaluation score finally achieved by the user. As described below, there are 11 scores ranging from 0 to 10, and a perfect score of 10 will result in the pronunciation of "jutten" in a female voice, and a score of 0 will result in the pronunciation of "reiten" in a hoarse voice.

Figure 7 is a flow chart showing a detailed example of the lesson process in step S602 in Figure 6. First, the CPU 201 executes an initialization process (step S701). In this process, the parameters required for executing the lesson process are initialized, and the LED 104 of the key on the keyboard 101 in Figure 1 that is to be pressed first is turned on at half the maximum brightness.

Next, the CPU 201 executes vocal song start processing to start playback of the performance guide data for the vocal song selected by the user in step S601 of FIG. 6 (step S702). In this vocal song start processing, the CPU 201 executes TickTime initialization processing. In this embodiment, the progression of lyrics progresses in units of time called TickTime. The time base value specified as the TimeDivision value in the header chunk of the song data in FIG. 5 indicates a quarter note resolution, and if this value is, for example, 480, a quarter note has a time length of 480 TickTime. The waiting time DeltaTime[i] value in the track chunk of the song data in FIG. 5 is also counted in units of TickTime. Here, how many seconds 1 TickTime actually is depends on the tempo specified for the song data. Now, assuming that the tempo value is Tempo [beats/minute] and the time base value is TimeDivision, the CPU 201 calculates TickTime [seconds] by performing an arithmetic process corresponding to the following formula (1).

Next, in the vocal song start process of step S702, the CPU 201 sets a timer interrupt for the timer 210 in Figure 2 with the calculated TickTime [seconds]. As a result, every time the TickTime [seconds] elapses in the timer 210, an interrupt for vocal song progression (hereinafter referred to as an "automatic playback interrupt") is generated for the CPU 201. Therefore, in the automatic playback process (Figure 8 described later) executed by the CPU 201 based on this automatic playback interrupt, a control process is executed to progress the set of performance guide data every TickTime.

In the initial state, the tempo value Tempo is stored in the ROM 202 in FIG. 2 as a predetermined value, for example, 60 beats per second. If the tempo value of the music piece is included in the performance guide data as an event in the performance guide data, that tempo value may be used. Alternatively, the tempo value at the time of the previous end may be stored in the non-volatile memory. The user can also change the tempo value by operating the first switch panel 102 in FIG. 1, and each time this is done, the TickTime [seconds] is calculated and a timer interrupt is set in the timer 210, although this is not shown in the figure.

Next, in the vocal song start process of step S702, the CPU 201 initializes the value of the variable DeltaT in the RAM 203, which counts the relative time from the occurrence time of the previous event in units of TickTime during the progress of the automatic playback process, to 0. Next, the CPU 201 initializes the value of the variable SongIndex in the RAM 203, which specifies the value of i of each of the performance data sets DeltaTime[i] and Event[i] (1≦i≦L-1) in the track chunk of the song data illustrated in FIG. 5, to 0. As a result, in the example of FIG. 5, the first performance data sets DeltaTime[0] and Event[0] in the track chunk are first referenced as the initial state. Furthermore, the CPU 201 initializes the value of the variable SongStart in the RAM 203, which indicates whether the lyrics will progress (=1) or not (=0), to 1 (progress).

Furthermore, in the vocal song start processing of step S702, the CPU 201 sets the value of the variable Margin in the RAM 203 to an integer value that is closest to 30% of the value of the above DeltaTime[0]. The Margin variable value will be described later. In addition, in the vocal song start processing of step S702, the CPU 201 resets the value of the correct answer flag, which is a variable in the RAM 203 and the variables NoteOn_in, NoteOn_out, and NoteOn, which will be described later, to 0.

After processing in step S702, the CPU 201 repeatedly executes the series of processes from steps S703 to S709 to proceed with automatic lyric playback processing and user performance lesson processing.

Figure 8 is a flow chart showing an example of automatic playback processing executed based on an automatic playback interrupt generated by timer 210 every TickTime [seconds] during the period of repeated processing of steps S703 to S709 in Figure 7 above. This processing is realized as a function in which, when an automatic playback interrupt occurs from timer 210, CPU 201 interrupts the processing being executed in the lesson processing of Figure 7 (any of steps S703 to S709) and executes an automatic playback processing program preloaded from ROM 202 to RAM 203.

First, the CPU 201 determines whether the value of the variable SongStart in the RAM 203 is 1, i.e., whether automatic playback of lyrics has been instructed (step S801).

If the CPU 201 determines that the automatic playback of lyrics has not been instructed (the determination in step S801 is NO), the CPU 201 does not progress the lyrics and ends the automatic playback process illustrated in the flowchart of FIG. 8.

If the CPU 201 determines that automatic playback of lyrics has been instructed (YES in step S801), it determines whether the DeltaT value, which indicates the relative time from the occurrence time of the previous event for the track chunk of the performance guide data having the data structure of FIG. 5(a) loaded in the RAM 203, has reached a value obtained by subtracting the Margin value from the waiting time DeltaTime[SongIndex] of the performance guide data set to be executed, which is indicated by the SongIndex value (step S802). If the determination in step S802 is NO, it further determines whether the DeltaT value has reached a value obtained by adding the Margin value to DeltaTime[SongIndex] (step S803).

In this embodiment, it is determined whether the user pressed the correct key pitch on the keyboard 101 in FIG. 1 at the timing when the lyrics of each set of performance guide data should be spoken, which matches the pitch indicated by the pitch information set for the set. In this case, in order to allow the user some leeway in their performance lessons, as shown in FIG. 4(c), the timing at which the user's key press is considered correct can be set to have a width before and after the timing that exactly matches the timing information = DeltaTime [SongIndex] set for each set of performance guide data. This time width is called the key press period. The key press period can be an absolute time, such as 100 milliseconds before and after the just timing, but in this embodiment, 30% of the note length is set as the key press period before and after the just timing. In other words, for a quarter note at a tempo of 60, the note length is 1 second, so the key press period is a section of 300 milliseconds before and after the just timing, that is, 600 milliseconds. Instead of setting the key-press period for the same amount of time before and after Just Timing, it is also possible to set it to 20% before and 40% after. In this embodiment, in step S809, a value of 30% of Just Timing DeltaTime[SongIndex] is calculated and set to the variable Margin in RAM 203. Then, in step S802, it is determined whether the variable value DeltaT indicating the relative time from the occurrence time of the previous event has reached "DeltaTime[SongIndex]-Margin" corresponding to the start time of the key-press period, and in the following step S803, it is determined whether DeltaT has reached "DeltaTime[SongIndex]+Margin" corresponding to the end time of the key-press period.

If the determinations in both steps S802 and S803 are NO, the CPU 201 increments the value of the variable DeltaT in the RAM 203, which indicates the relative time from the occurrence time of the previous event, by +1, and advances the time by 1 TickTime unit corresponding to the current interrupt (step S804). After that, the CPU 201 ends the automatic playback process shown in the flowchart of FIG. 8, and returns to the execution of the interrupted lesson process in FIG. 7.

If it is determined in step S802 that the variable value DeltaT, which indicates the relative time from the occurrence time of the previous event, has reached "DeltaTime[SongIndex]-Margin" corresponding to the start time of the key-press period, the CPU 201 sets the value of the variable NoteOn_in in the RAM 203, which indicates that the key-press period has begun, to 1, and further sets the value of the variable NoteOn in the RAM 203, which indicates that the key-press period is in progress, to 1 (step S805). Next, the CPU 201 proceeds to the process of step S804, increments the value of the variable DeltaT in the RAM 203, which indicates the relative time from the occurrence time of the previous event, by +1, and advances the time by 1 TickTime unit corresponding to the current interrupt, and then ends the automatic playback process shown in the flowchart of FIG. 8, and returns to the execution of the process that was interrupted in the lesson process of FIG. 7.

After the determination in step S802 is NO, if it is determined in step S803 that the variable value DeltaT indicating the relative time from the occurrence time of the previous event has reached "DeltaTime[SongIndex]+Margin" corresponding to the end time of the key press period, the CPU 201 sets the value of the variable NoteOn_out in RAM 203, indicating that the key press period is just about to end, to 1, and further sets the value of the variable NoteOn in RAM 203, indicating that the key press period is in progress, to the value 0 indicating that the key press period is no longer in progress (step S806).

Next, the CPU 501 sets the new value of the variable DeltaT to "DeltaT - DeltaTime[SongIndex] + 1", which is the point at which the value of the variable DeltaT has already advanced from DeltaTime[SongIndex] (step S807).

Furthermore, the CPU 201 increments the value of the variable SongIndex in the RAM 203 by +1 to advance the set of performance guide data used in the arrival determination by one (step S808).

Finally, the CPU 201 sets the Margin value in FIG. 4(c) used to determine whether the next set of performance guide data has been reached to "INT(DeltaTime[SongIndex]×0.3)" (where "INT()" indicates an operation to calculate the integer value of the value in parentheses) obtained by multiplying the new DeltaTime[SongIndex] value referenced by the newly updated SongIndex value by 0.3 (30%) (step S809). After that, the CPU 201 ends the automatic playback process shown in the flowchart in FIG. 8, and returns to the execution of the interrupted lesson process in FIG. 7.

Returning to the explanation of the flowchart in FIG. 7, in the repeated processing of steps S703 to S709, the CPU 201 first executes a performance guide process (step S703). In this performance guide process, the CPU 201 synchronizes with the timing of automatic playback, and causes the LED 104 of the key (operator) corresponding to the pitch information to be automatically played next to be lit with high brightness, for example, maximum brightness, as shown by the key with two lit LEDs 104 in FIG. 1, and causes the LED 104 of the key corresponding to the pitch information to be automatically played after that to be lit with low brightness, for example, half the maximum brightness. The details of this performance guide process will be described later using the flowchart in FIG. 9.

Next, the CPU 201 executes key pressing/release processing (step S704). In this key pressing/release processing, when the user presses a new key on the keyboard 101 in FIG. 1, if the CPU 201 determines that the key pressing timing (operation timing) of the key (operator) on the keyboard 101 in the performance lesson correctly corresponds to the timing information to be automatically played (entering the key pressing period in FIG. 4(c)), and that the key pressing pitch (operation pitch) correctly corresponds to (matches) the pitch information of the set of performance guide data, it generates a vocalization event for outputting the singing voice output data 217 from the voice synthesis LSI 205 in FIG. 2 at that key pressing timing.

In addition, in the key pressing/release processing of step S704, when the user presses a new key on the keyboard 101 in FIG. 1, if the CPU 201 determines that the key pressing timing in the performance lesson correctly corresponds to the timing information to be automatically played back (entering the key pressing period in FIG. 4(c)), but the key pressing pitch (operation pitch) does not correctly correspond (does not match) to the pitch information of the performance guide data set, it generates a sound generation event to output a predetermined musical sound output data 214 (for example, based on the instrument sound and the pitch of the performance guide data previously selected by the user on the second switch panel 103 in FIG. 1) from the sound source LSI 204 in FIG. 2 at the key pressing timing.

Furthermore, in the key pressing/release processing of step S704, if the user presses a new key on the keyboard 101 in FIG. 1 and the CPU 201 determines that the key pressing timing in the performance lesson does not correctly correspond to the timing information to be automatically played back (not within the key pressing period in FIG. 4(c)), it does not generate either an event for causing the voice synthesis LSI 205 to produce the singing voice output data 217 or an event for causing the sound source LSI 204 to produce the musical sound output data 214.

On the other hand, in the key press/release process of step S704, when the user releases any key on the keyboard 101 of FIG. 1, the CPU 201 generates a key release event to end the production of the corresponding singing voice output data 217 in the voice synthesis LSI 205 or the production of the corresponding musical tone output data 214 in the sound source LSI 204.

Details of the key press/release process in step S704 will be described later using the flowchart in Figure 10.

Next, the CPU 201 executes a scoring process (step S705). In this scoring process, the CPU 201 compares the key press timing (operation timing) and key press pitch (operation pitch) in the performance lesson with the automatically reproduced timing information and pitch information to score the performance lesson. Details of this scoring process will be described later using the flowchart in FIG. 11.

Next, the CPU 201 executes a voice quality update process (step S706). In this voice quality update process, the CPU 201 executes a process of setting a value indicating the voice quality of the singing voice output data 217 output in the vocalization/pronunciation process in step S708, which will be described later, according to the intermediate scoring results of the scoring process in step S705.

Next, the CPU 201 executes a practice progress calculation process (step S707). In this practice progress calculation process, the CPU 201 in FIG. 2 and the formant interpolation processing unit 306 and the noise superimposition unit 307 in the voice synthesis unit 300 in FIG. 3 calculate practice progress data 311 for changing the voice quality of the singing voice output data 217 output in the vocalization/pronunciation process in step S708, which will be described later, according to the intermediate scoring results in the scoring process in step S705. Details of the voice quality update process and the practice progress calculation process described above will be described later using the flowchart in FIG. 12.

Furthermore, the CPU 201 executes vocalization/pronunciation processing (step S708). In this vocalization/pronunciation processing, when the CPU 201 generates a vocalization event of the singing voice output data 217 in the key press/release processing of step S704, the CPU 201 issues the vocalization event stored in the issued event area on the RAM 203 to the voice synthesis LSI 205 of Fig. 2, thereby causing the voice synthesis LSI 205 to output the singing voice output data 217. In addition, in this vocalization/pronunciation processing, when the CPU 201 generates a pronunciation event of the musical tone output data 214 in the key press/release processing of step S704, the CPU 201 issues the pronunciation event stored in the issued event area on the RAM 203 to the sound source LSI 204 of Fig. 2, thereby causing the sound source LSI 204 to output the musical tone output data 214. Furthermore, in this vocalization/sound generation process, if the CPU 201 generates a key release event in the key press/key release process of step S704, it issues the key release event stored in the issued event area on the RAM 203 to the voice synthesis LSI 205 or sound source LSI 204 in FIG. 2, thereby stopping the output of the corresponding singing voice output data 217 or musical sound output data 214.

Finally, the CPU 201 determines whether or not there are no more sets of performance guide data to be read from the RAM 203 and the vocal piece has ended (step S709). If the determination in step S709 is NO, the process returns to step S703 and repeats the series of steps S703 to S709. If the determination in step S709 is YES, the CPU 201 ends the lesson process in step S602 of FIG. 6, which is shown in the flowchart of FIG. 7.

Figure 9 is a flowchart showing a detailed example of the performance guide process in step S703 of Figure 7.

First, the CPU 201 determines whether the value of the variable NoteOn_in in the RAM 203 is 1, i.e., whether the current timing of automatic playback has entered the key-press period (see FIG. 4C) (step S901). As described above, the value of this variable NoteOn_in is set to the value "1", indicating that the key-press period has begun, when it is determined in step S802 of the automatic playback process in FIG. 8 that the variable value DeltaT, which indicates the relative time from the occurrence time of the previous event, has reached "DeltaTime[SongIndex]-Margin", which corresponds to the start time of the key-press period.

When the determination in step S901 is YES, the CPU 201 lights up, via the LED controller 207 in Fig. 2, the LED 104 of the key on the keyboard 101 in Fig. 1 that corresponds to the pitch information contained in the performance guide data set Event[SongIndex] in the RAM 203 referenced by the current variable value SongIndex value in the RAM 203, at maximum brightness (step S902). This allows the user to recognize that the key whose LED 104 is lit at maximum brightness is the next key to be pressed.

Next, the CPU 201 determines whether the performance guide data set Event[SongIndex+1] referenced by the current variable value SongIndex+1 on the RAM 203 exists on the RAM 203 (step S903).

If the determination in step S903 is YES, the CPU 201 causes the LED 104 of the key on the keyboard 101 in FIG. 1 that corresponds to the pitch information contained in the performance guide data set Event[SongIndex+1] on the RAM 203 to light up at half the maximum brightness via the LED controller 207 in FIG. 2 (step S904). This allows the user to recognize that the key whose LED 104 is lit up at half the maximum brightness is the next key to be pressed.

If the determination in step S903 is NO, the CPU 201 does not execute the process of step S904. As a result, the user can recognize that the key with the LED 104 lit at maximum brightness is the last key of the lesson because there is no key with the LED 104 lit at half the maximum brightness.

After the process of step S904 or after the determination of step S903 becomes NO, the CPU 201 sets the value of the variable NoteOn_in in the RAM 203 to 0, thereby ending the state in which the current timing of automatic playback has entered the key-press period. The CPU 201 then ends the performance guide process of step S703 in FIG. 7, which is shown in the flowchart of FIG. 9.

If the determination in step S901 is NO, the CPU 201 next determines whether the value of the variable NoteOn_out in the RAM 203 is 1, i.e., whether the current timing of automatic playback is the timing of exiting the key-press period (see FIG. 4C) (step S906). As described above, the value of this variable NoteOn_out is set to the value "1", indicating the exit of the key-press period, when it is determined in step S803 of the automatic playback process in FIG. 8 that the variable value DeltaT, which indicates the relative time from the occurrence time of the previous event, has reached "DeltaTime[SongIndex]+Margin", which corresponds to the end time of the key-press period.

When the determination in step S906 is YES, the CPU 201 turns off the LED 104 of the key on the keyboard 101 in FIG. 1 that corresponds to the pitch information contained in the performance guide data set Event[SongIndex-1] on the RAM 203, which is referenced by the current variable value SongIndex value on the RAM 203 minus 1, via the LED controller 207 in FIG. 2 (step S907). This allows the user to recognize that the key-press period for this key has ended. Note that the reason SongIndex-1 is referenced instead of SongIndex is that when NoteOn_out=1 in step S806 in FIG. 8, the value of SongIndex is incremented by +1 in the following step S808, so that the LED 104 of the key in the previous key-press period can be turned off by referencing Event[SongIndex-1].

After the process of step S907 or the determination of step S906 is NO, the CPU 201 ends the performance guide process of step S703 in FIG. 7 shown in the flowchart of FIG. 9.

Figure 10 is a flowchart showing a detailed example of the key press/release process in step S704 of Figure 7.

First, the CPU 201 determines whether or not a new key has been pressed on the keyboard 101 in FIG. 1 by the user via the key scanner 206 in FIG. 2 (step S1001).

If the determination in step S1001 is NO, the CPU 201 proceeds to the key release determination process in step S1007.

If the determination in step S1001 is YES, the CPU 201 determines whether the value of the variable NoteOn in the RAM 203 is 1, i.e., whether the current timing of automatic playback is within the key press period (see FIG. 4(c)) (step S1002).

If the determination in step S1002 is NO, the CPU 201 proceeds to the key release determination process in step S1007. In this way, when the user presses a new key on the keyboard 101 in FIG. 1, if it is determined that the key pressing timing in the performance lesson does not correctly correspond to the timing information for automatic playback (not within the key pressing period in FIG. 4(c)), neither an event for causing the voice synthesis LSI 205 to vocalize the singing voice output data 217 nor an event for causing the sound source LSI 204 to sound the musical tone output data 214 is generated, and neither vocalization nor pronunciation is performed.

In this way, the user can recognize that he or she pressed the key at the wrong time because neither the singing voice output data 217 nor the musical sound output data 214 is produced despite the key being pressed.

If the determination in step S1002 is YES, the CPU 201 determines whether the pitch of the newly pressed key notified via the key scanner 206 matches the pitch information contained in Event[SongIndex] on RAM 203 referenced by the current SongIndex value on RAM 203 (step S1003).

If the determination in step S1003 is YES, the CPU 201 generates a vocal event having the lyrics information and pitch information contained in Event[SongIndex] on the RAM 203 referenced by the current SongIndex value on the RAM 203 as singing voice data 215, and sets it in the issued event area of the RAM 203 (step S1004). Next, the CPU 201 sets the value of the correct flag variable on the RAM 203 to 1 (step S1005).

On the other hand, if the determination in step S1003 is NO, the CPU 201 generates a sound generation event having tone color information of a predetermined tone color (e.g., piano tone) as sound generation control data 216 (see FIG. 2) together with pitch information contained in Event[SongIndex] on RAM 203 referenced by the current SongIndex value on RAM 203, and sets it in the issued event area of RAM 203 (step S1006).

As in steps S1004 or S1005 above, when the user newly presses any key on the keyboard 101 in FIG. 1 and the timing of the key press correctly corresponds to the timing information to be automatically played back, if the pressed key pitch correctly corresponds to the pitch information of the set of performance guide data, the voice synthesis LSI 205 can output singing voice output data 217 corresponding to the lyrics and pitch being automatically played back as a correct answer, and if the pressed key pitch is incorrect, the sound source LSI 204 can output musical sound output data 214 of a specified tone color corresponding to the pitch being automatically played back as an incorrect answer. In this way, the user can easily recognize whether the performance lesson is correct or incorrect by whether the singing voice output data 217 or the musical sound output data 214 is pronounced, without having to look at a display or the like each time.

If the determination in step S1001 is NO or after processing in step S1005 or S1006, the CPU 201 determines whether or not a new key has been released by the user on the keyboard 101 in FIG. 1 via the key scanner 206 in FIG. 2 (step S1007).

If the determination in step S1007 is YES, a key release event is generated to stop the output of the currently sounding singing voice output data 217 or musical tone output data 214 from the voice synthesis LSI 205 or the sound source LSI 204 in FIG. 2, and is set in the issued event area of the RAM 203 (step S1008).

After processing in step S1008 or if the determination in step S1007 is NO, the CPU 201 ends the key press/release processing in step S704 in FIG. 8 shown in the flowchart in FIG. 10.

Figure 11 is a flowchart showing a detailed example of the scoring process in step S705 in Figure 7. First, the CPU 201 determines whether the value of the variable NoteOn_out in the RAM 203 is 1, i.e., whether the current timing of automatic playback is the timing to exit the key-press period (see Figure 4 (c)) (step S1101).

If the determination in step S1101 is YES, the following scoring process is executed. That is, the CPU 201 first determines whether the variable value of the correct flag in the RAM 203 is 1 (step S1102). As described above, the variable value of this correct flag is set to the value "1" in step S1005 of FIG. 10 when the user newly presses any key on the keyboard 101 in FIG. 1 and the key pressing timing correctly corresponds to the timing information to be automatically played back, and the key pressing pitch correctly corresponds to the pitch information in the set of performance guide data.

If the determination in step S1102 is YES, the CPU 201 performs a point increment process (e.g., +1) on the variable value indicating the intermediate scoring result stored in the RAM 203 (step S1103). Next, the CPU 201 resets the variable value of the correct answer flag stored in the RAM 203 to 0 (step S1104).

On the other hand, if the determination in step S1102 is NO, the CPU 201 performs a subtraction process (e.g., subtracting 1) on the variable value indicating the intermediate scoring result stored in the RAM 203 (step S1105).

After processing step S1104 or S1105, the CPU 201 resets the value of the NoteOn_out variable on the RAM 203 to 0 (step S1106).

After processing in step S1106 or if the determination in step S1101 is NO, the CPU 201 ends the scoring process in step S705 of FIG. 7, which is shown in the flowchart of FIG. 11.

As described above, in this embodiment, the state of the correct flag during the key pressing period is checked when the key pressing period has elapsed, so even if the user presses a key outside of the key pressing period, it does not affect the score. Points are deducted if a key is not pressed during the key pressing period.

Figure 12 is a flow chart showing a detailed example of the voice quality update process in step S706 in Figure 7. As described above, in this embodiment, the voice quality is changed so that the initial state is a male voice, and as the intermediate scoring score increases, the voice quality is changed toward a female voice, and as the score decreases, the voice quality is changed toward a male hoarse voice. Voice quality is determined by the frequency characteristics of the vocal tract called formants as described above, as well as the frequency of the sound emitted by the vocal cords (voice pitch). As shown in Figure 4 (b), in general, in female voices and children, the formants after the first formant are located in the high frequency range, while in male voices, the formants are located in the low frequency range. In addition, women and children generally have high voice pitches (frequency of vibration of the vocal cords).

In light of the above trends, in this embodiment, as shown in FIG. 5(b), the voice quality is changed in 11 stages in total, five stages upward and five stages downward, centered on the initial male voice quality 5. The formants are changed to interpolate between female voices, male voices, and hoarse voices. In addition, in the third stage of voice quality change (voice quality 8 and voice quality 2), a process is performed to raise or lower the voice pitch by one octave.

In the process shown in the flowchart of FIG. 12, the CPU 201 compares the score of the previous intermediate scoring result stored in the RAM 203 with the score of the current intermediate scoring result (step S1201), and determines whether the score has increased (step S1202).

If the determination in step S1202 is YES (if the score has increased), the CPU 201 adds one level to the voice quality (step S1205) if the voice quality has not reached the maximum value of 10 (if the determination in step S1204 is NO). If the voice quality has reached the maximum value of 10 (if the determination in step S1204 is YES), the CPU 201 does not execute the addition process in step S1205.

Also, if the voice quality is at 8 or 2, which is the octave switching stage (if the determination in step S1206 is YES), the CPU 201 raises the pitch of the voice by one octave (step S1207). If the determination in step S1206 is NO, the CPU 201 maintains the octave. After that, the CPU 201 ends the voice quality update process in step S706 in FIG. 7, which is shown in the flowchart in FIG. 12.

On the other hand, if the determination in step S1202 is NO, the CPU 201 further determines whether the score has decreased as a result of the comparison process in step S1201 (step S1203). If the determination in step S1203 is also NO and the score has been maintained, the CPU 201 does nothing and ends the voice quality update process in step S706 in FIG. 7 shown in the flowchart in FIG. 12.

If the CPU 201 determines in step S1203 that the score has decreased (YES), and the voice quality has not reached the minimum value of 0 (NO in step S1208), it subtracts one level from the voice quality (step S1209). If the voice quality has reached the minimum value of 0 (YES in step S1208), the CPU 201 does not execute the subtraction process in step S1209.

Also, if the voice quality is at 8 or 2, which is the octave switching stage (if the determination in step S1210 is YES), the CPU 201 lowers the voice pitch by one octave (step S1211). If the determination in step S1210 is NO, the CPU 201 maintains the octave.

ここで、ｃｕｒＮｕｍは図７のステップＳ７０６の声質更新処理によってＲＡＭ２０３上の変数ｃｕｒＮｕｍに得られている現在の声質の値である。また、ｍａｌｅＮｕｍは、男性の声質の値であり、前述したように例えば５である。更に、ｆｅｍａｌｅＮｕｍは、女性の声質の値であり、例えば最高値１０である。従って、上記（２）式の演算処理により算出される練習進行度データ３１１の値ｘは、男性の声質の値ｍａｌｅＮｕｍに対する採点途中結果に対応する現在の声質の値ｃｕｒＮｕｍの差分値が、男性の声質の値ｍａｌｅＮｕｍに対する女性の声質の値ｆｅｍａｌｅＮｕｍの差分値に対して、どの程度の割合であるかを示している。例えば、現在の声質の値ｃｕｒＮｕｍ＝１０（女性の声質の値と同じ最高の声質の値）であれば、上記（２）式の演算結果は「ｘ＝（１０―５）÷（１０－５）＝１」となる。また例えば、現在の声質の値ｃｕｒＮｕｍ＝５（男性の声質の値と同じ値）であれば、上記（２）式の演算結果は「ｘ＝（５―５）÷（１０－５）＝０」となる。更に例えば、現在の声質の値ｃｕｒＮｕｍ＝０（最低の声質の値）であれば、上記（２）式の演算結果は「ｘ＝（０―５）÷（１０－５）＝－１」となる。即ち、上記（２）式の演算処理によって算出される練習進行度データ３１１の値ｘは、図７のステップＳ７０５の採点処理に基づく採点途中結果が最高値となって、図７のステップＳ７０６の声質更新処理によって算出される現在の声質の値ｃｕｒＮｕｍが女性の声質と同じ最高値１０になれば、練習進行度データ３１１の値ｘ＝１となる。また、図７のステップＳ７０５の採点処理に基づく採点途中結果が平均値となって、図７のステップＳ７０６の声質更新処理によって算出される現在の声質の値ｃｕｒＮｕｍが男性の声質と同じ平均値５になれば、練習進行度データ３１１の値ｘ＝０となる。更に、図７のステップＳ７０５の採点処理に基づく採点途中結果が最低値となって、図７のステップＳ７０６の声質更新処理によって算出される現在の声質の値ｃｕｒＮｕｍが最低値０になれば、練習進行度データ３１１の値ｘ＝－１となる。 After calculating the voice quality as described above, the CPU 201 stores the voice quality in the variable curNum on the RAM 203, and then ends the voice quality update process of step S706 in Fig. 7 shown in the flowchart of Fig. 12. Next, in the practice progress calculation process of step S707 in Fig. 7, the CPU 201 uses the voice quality value obtained for the variable curNum in step S706 to calculate the value x of the practice progress data 311 by the arithmetic process shown in equation (2).

Here, curNum is the current voice quality value obtained in the variable curNum on the RAM 203 by the voice quality update process in step S706 of FIG. 7. Furthermore, maleNum is the value of male voice quality, for example 5 as described above. Furthermore, femaleNum is the value of female voice quality, for example the highest value 10. Therefore, the value x of the practice progress data 311 calculated by the calculation process of the above formula (2) indicates the ratio of the difference value of the current voice quality value curNum corresponding to the intermediate scoring result for the male voice quality value maleNum to the difference value of the female voice quality value femaleNum for the male voice quality value maleNum. For example, if the current voice quality value curNum=10 (the highest voice quality value, the same as the female voice quality value), the calculation result of the above formula (2) is "x=(10-5)÷(10-5)=1". For example, if the current voice quality value curNum = 5 (the same value as that of a male voice quality), the calculation result of the above formula (2) is "x = (5 - 5) / (10 - 5) = 0." For example, if the current voice quality value curNum = 0 (the lowest voice quality value), the calculation result of the above formula (2) is "x = (0 - 5) / (10 - 5) = -1." That is, if the value x of the practice progress data 311 calculated by the calculation process of the above formula (2) is the highest value based on the intermediate scoring result based on the scoring process of step S705 in Fig. 7, and the current voice quality value curNum calculated by the voice quality update process of step S706 in Fig. 7 is the highest value 10, the same as that of a female voice quality, then the value x of the practice progress data 311 will be x = 1. Furthermore, if the intermediate scoring result based on the scoring process in step S705 of Fig. 7 becomes the average value and the current voice quality value curNum calculated by the voice quality updating process in step S706 of Fig. 7 becomes the same average value 5 as a male voice quality, the value x of the practice progress data 311 will be 0. Furthermore, if the intermediate scoring result based on the scoring process in step S705 of Fig. 7 becomes the minimum value and the current voice quality value curNum calculated by the voice quality updating process in step S706 of Fig. 7 becomes the minimum value 0, the value x of the practice progress data 311 will be -1.

Figure 5 (c) is a diagram showing an example of the data structure of the voice quality parameters (hereinafter referred to as the "voice quality parameter structure") stored in the acoustic model unit 301 in the voice synthesis LSI 205 in this embodiment. The "vowel flag" is a flag indicating whether the phoneme is a vowel or a consonant. If it is a vowel, it is true, and if it is a consonant, it is false. The "noise mixing ratio" is the amplitude ratio of the white noise mixed into the voice. This value is the ratio when the maximum amplitude of the singing voice output data 217 (32,768 for 16 bits) is set to 1. The "LSF1 array pointer" and "LSF2 array pointer", which are formant parameters, are array pointers to the LSF values obtained by converting the LSP parameter values obtained as a result of the LSP analysis into frequency parameter values. Since two LSFs make a pair of data, the above data structure stores the pairs of "LSF1 array pointer" and "LSF2 array pointer" for the number of orders of the LSP analysis. The data set of the above voice quality parameter structures, the number of which is equal to all phonemes x all note numbers, constitutes the entire voice quality parameter set for one human voice character. This voice quality parameter set is stored in a writable ROM (not shown) in the acoustic model unit 301 for each character, male voice, female voice, and hoarse voice.

Figure 13 is a flowchart showing an example of voice quality change processing executed by the processor of the voice synthesis LSI 205 in Figure 3 as the functions of the formant interpolation processing unit 306 and the noise mixing ratio interpolation processing unit 316 of the voice synthesis unit 300 during vocalization processing in the voice synthesis LSI 205. Here, a voice quality change between a male voice and a female voice is explained. A voice quality change between a male voice and a hoarse voice can also be implemented in the same way.

The processor of the voice synthesis LSI 205 obtains the phoneme number to be vocalized, which is set via the text analysis unit 302 of FIG. 3, based on the singing voice data 215 issued by the CPU 201 in the vocalization/pronunciation process of step S708 of FIG. 7 (step S1301), and also obtains the note number, which is pitch information specified in the form included in the singing voice data 215 (step S1302). The processor of the voice synthesis unit 300 also obtains the value x of the practice progress data 311 calculated by the CPU 201 in step S707 of FIG. 7 through the arithmetic process shown in the above-mentioned equation (2) (step S1303).

Next, the processor of the speech synthesis LSI 205 acquires the voice quality parameter structures for the male and female voices in the data format shown in FIG. 5(c) from the acoustic model unit 301 based on the phoneme number acquired in step S1301 and the note number acquired in step S1302 (steps S1304, S1305).

Next, the processor of the speech synthesis LSI 205 determines whether or not a vowel flag is set in the voice quality parameter structure acquired in step S1304 or S1305 (step S1306).

If the phoneme of the current phoneme number acquired in step S1301 is not a vowel and the judgment in step S1306 is NO, i.e., if it is a consonant, the phoneme is a noise voice without pitch. In this case, the processor of the speech synthesis LSI 205 extracts the "noise mixture ratio" (see FIG. 5(c)) from the male voice quality parameter structure acquired in step S1304 and sets it as the target noise mixture ratio 317 in the noise superimposition unit 307 of FIG. 3, and extracts the "LSF1 array pointer" and "LSF2 array pointer" (see FIG. 5(c)) and sets them as the target spectrum information 312 in FIG. 3 in the synthesis filter unit 305 of FIG. 3 (step S1310). It is of course possible to set data from a female voice quality parameter structure instead of a male voice as the target noise mixture ratio 317 and the target spectrum information 312.

If the phoneme of the current phoneme number acquired in step S1301 is a vowel and the determination in step S1306 is YES, the processor of the speech synthesis LSI 205 first executes a noise mixing ratio interpolation process as the function of the noise mixing ratio interpolation processing unit 316 in FIG. 3 (step S1307), and then executes a formant interpolation process as the function of the formant interpolation processing unit 306 in FIG. 3 (step S1308). The processor of the speech synthesis LSI 205 then sets the target noise mixing ratio 317 obtained by the noise mixing ratio interpolation process in step S1307 as a target parameter in the noise superimposition unit 307 in FIG. 3, and sets the target spectrum information 312 obtained by the formant interpolation process in step S1308 as a target parameter in the synthesis filter unit 305 in FIG. 3.

After processing of step S1309 or S1310, the processor of the voice synthesis LSI 205 ends the voice quality change processing shown in the flowchart of FIG. 13.

Figure 14 (a) is a flowchart showing a detailed example of the noise mixing ratio interpolation process of step S1307 in Figure 13, which is executed by the processor of the speech synthesis LSI 205 as a function of the noise superimposition unit 307 of the speech synthesis unit 300 in Figure 3.

The processor of the speech synthesis LSI 205 extracts the value of the male noise mixing ratio as maleRate from the male voice quality parameter structure acquired from the acoustic model unit 301 in step S1304 of FIG. 3, and also extracts the value of the female noise mixing ratio as femaleRate from the female voice quality parameter structure acquired from the acoustic model unit 301 in step S1305 of FIG. 3. Furthermore, using the value x of the practice progress data 311 acquired in step S1303 of FIG. 13, the processor calculates the value targetRate of the target noise mixing ratio 317 by performing the arithmetic processing shown in equation (3) below (step S1401).

As described above, the value x of the practice progress data 311 indicates the ratio of the difference value of the current voice quality value curNum corresponding to the intermediate scoring result for the male voice quality value maleNum to the difference value of the female voice quality value femaleNum for the male voice quality value maleNum. Therefore, as shown in the above formula (3), the difference value of the female noise mixture ratio for the male noise mixture ratio (femaleRate-maleRate) is multiplied by the value x of the practice progress data 311 corresponding to the scoring result, and the multiplication result is added to the male noise mixture ratio maleRate, thereby calculating the value targetRate of the target noise mixture ratio 317 corresponding to the voice quality interpolated in response to the scoring result.

The processor of the voice synthesis LSI 205 in FIG. 3, as a function of the noise mixing ratio interpolation processing unit 316, sets the target noise mixing ratio value targetRate calculated in step S1307 of FIG. 13 shown in the flowchart of FIG. 14(a) as the target noise mixing ratio 317, which is a target parameter, in the noise superimposition unit 307 in the voice synthesis LSI 205 in FIG. 13 in step S1309 of FIG. 13. Then, as a function of this noise superimposition unit 307, the processor of the voice synthesis LSI 205 generates noise data 315 having an amplitude value obtained by multiplying the maximum amplitude value of the signal by the value targetRate of the target noise mixing ratio 317, and mixes it with the filter output data 313.

Figure 14 (b) is a flowchart showing a detailed example of the formant interpolation process in step S1308 in Figure 13, which is executed by the processor of the speech synthesis LSI 205 as a function of the formant interpolation processing unit 306 of the speech synthesis unit 300 in Figure 3.

The processor of the speech synthesis LSI 205 repeatedly executes the processes of steps S1412 and S1413 for each parameter set of LSF1 and LSF2, incrementing the value of a variable i held, for example, as a register not specifically shown, from 0 (step S1411) by +1 (step S1414) until it is determined that the value has reached the parameter order N (step S1415).

First, the processor of the speech synthesis LSI 205 extracts the i-th male LSF1 array pointer from the male voice quality parameter structure acquired from the acoustic model unit 301 in step S1304 of FIG. 3, and acquires the value of the LSF1 parameter as maleLSF1[i] from a memory (not shown) in the acoustic model unit 301 referenced by the pointer. Also, in step S1305 of FIG. 3, the processor extracts the i-th female LSF1 array pointer from the female voice quality parameter structure acquired from the acoustic model unit 301, and acquires the value of the LSF1 parameter as femaleLSF1[i] from the memory in the acoustic model unit 301 referenced by the pointer. Then, using the value x of the practice progress data 311 acquired in step S1303 of FIG. 13, the processor executes the arithmetic processing shown in equation (4) below to calculate the i-th target LSF1 value targetLSF1[i], which is part of the target spectrum information 312 (step S1412).

As described above, the value x of the practice progress data 311 indicates the ratio of the frequency difference value of the current voice quality value curNum corresponding to the intermediate scoring result for the male voice quality value maleNum to the frequency difference value of the female voice quality value femaleNum for the male voice quality value maleNum. Therefore, as shown in the above formula (4), the frequency difference value of the female's i-th LSF1 parameter value for the male's i-th LSF1 parameter value (femaleLSF1[i] - maleLSF1[i]) is multiplied by the value x of the practice progress data 311 corresponding to the scoring result, and the multiplication result is added to the male's i-th LSF1 parameter value maleLSF1[i], thereby calculating the i-th target LSF1 parameter value targetLSF1[i] corresponding to the voice quality interpolated in response to the scoring result.

Next, the processor of the speech synthesis LSI 205 extracts the i-th male LSF2 array pointer from the male voice quality parameter structure acquired from the acoustic model unit 301 in step S1304 of FIG. 3, and acquires the value of the LSF2 parameter as maleLSF2[i] from the memory in the acoustic model unit 301 to which the pointer refers, and also extracts the i-th female LSF2 array pointer from the female voice quality parameter structure acquired from the acoustic model unit 301 in step S1305 of FIG. 3, and acquires the value of the LSF2 parameter as femaleLSF2[i] from the memory in the acoustic model unit 301 to which the pointer refers, and further calculates the i-th target LSF2 value targetLSF2[i], which is part of the target spectrum information 312, by performing the arithmetic processing shown in equation (5) below using the value x of the practice progress data 311 acquired in step S1303 of FIG. 13 (step S1413).

As in the case of the LSF1 parameter value in equation (4) above, as shown in equation (5) above, the frequency difference value (femaleLSF2[i] - maleLSF2[i]) of the female's i-th LSF2 parameter value relative to the male's i-th LSF2 parameter value is multiplied by the value x of the practice progress data 311 corresponding to the scoring result, and the multiplication result is added to the male's i-th LSF2 parameter value maleLSF2[i], thereby calculating the i-th target LSF2 parameter value targetLSF1[i] corresponding to the voice quality interpolated in accordance with the scoring result.

The processor of the speech synthesis LSI 205 in Fig. 3, as a function of the formant interpolation processing unit 306, sets a pair (0≦i≦N-1) of the target LSF1 parameter value targetLSF1[i] and the target LSF2 parameter value targetLSF2[i] for the LSP analysis order calculated in the formant interpolation processing in step S1308 in Fig. 13 shown in the flowchart in Fig. 14(b) described above, as target spectrum information 312, which is a target parameter, in the synthesis filter unit 305 in the speech synthesis LSI 205 in Fig. 13 in step S1309 described above in Fig. 13. As a result, the synthesis filter unit 305 forms a digital filter using the target spectrum information 312, and outputs filter output data 313 by inputting a sound source signal to this digital filter using the function of the sound source generation unit 304 in the vocalization model unit 303 in Fig. 3. Finally, this filter output data 313 is mixed with noise data 315 and output as singing voice output data 217.

By the voice quality change process described by the flowcharts of Figures 13 and 14 above, the voice quality of the singing function at the start of a performance lesson is set to, for example, an adult male voice, and as the intermediate score improves after the start of the performance lesson, the voice quality of the singing voice emitted from the speaker of the electronic keyboard instrument 100 based on the user's key press operation gradually changes to an adult female voice quality, and conversely, if the score decreases, the voice quality of the singing voice changes from an adult male voice quality to a voice quality that is a little harsh, so-called hoarse voice. Furthermore, it is possible to add a voice quality such as a husky voice to or not to depending on the intermediate score. This makes it possible for the user to easily check the level of their performance operation skill as the performance lesson progresses by the voice quality of the singing voice that is emitted, without having to check the display every time.

In the embodiment described above, the score for the intermediate scoring results was increased or decreased based on whether each note to be pressed was played or not, but the score could also be increased or decreased by taking the average score for a group of several notes (phrases) or several measures, or the score could be increased or decreased if a key was pressed or not several times in succession.

Furthermore, when scoring groups of notes as described above, bonus stages may be provided in which more points are added in more difficult sections, or penalty stages may be provided in which more points are deducted if a mistake is made in an easy section.

In this embodiment, the formants are shifted, and the mixing level and pitch of the white noise are changed, but it is also possible to add effects such as reverberation, or tremolo, which causes the pitch of the voice to fluctuate, or to add effects such as making the pitch more unstable as the score decreases.

In this embodiment, the LED corresponding to the next key to be pressed is lit at maximum brightness, and the LED corresponding to the key after that is lit at half the maximum brightness, but it is also possible to distinguish between the keys by keeping the LED brightness constant and lighting the LED corresponding to the next key to be pressed at the same brightness, and blinking the LED corresponding to the key after that.

In this embodiment, the electronic keyboard instrument 100 is used, but the present invention may also be implemented in other types of musical instruments, such as guitar-type or wind-type electronic musical instruments.

In this embodiment, the highest score is a female voice and the lowest score is a hoarse voice, but it goes without saying that there are many other possible combinations for selecting these voice qualities.

In this embodiment, the target voice is a typical female voice, but an acoustic model that models the voice of a specific person may also be used. For example, if an acoustic model that has been trained to sing the voice of a famous singer is used, it will be possible to produce a voice that approaches the voice of the singer you admire as your performance improves, making performance lessons even more effective.

In this embodiment, LSF is used as the voice quality parameter, but if the synthesis filter unit 305 in FIG. 3 is realized by a filter bank, it is also possible to regard the amplification factor of each filter constituting the filter bank as a formant shape and perform voice quality interpolation processing for the gain of each filter bank.

Furthermore, it is also possible to perform an interpolation process in the frequency domain by regarding the frequency amplitude outline generated by applying a moving average filter to the frequency amplitude components of the voice according to the method described in the following patent document as a voice quality parameter. Various other methods for changing the voice quality can be adopted regardless of the description in the embodiments.
(Patent Document): JP 2005-084661 A

In this embodiment, the sound source signal as the excitation source signal to the synthesis filter unit 305 in FIG. 3 is generated by the sound source generation unit 304 inside the speech synthesis LSI 205 in FIG. 3 based on the pitch information 310 output from the acoustic model unit 301. However, in another embodiment, the sound source signal input to the synthesis filter unit 305 may be supplied from the sound source LSI 204 in FIG. 2.

Figure 15 is a block diagram showing another example of the configuration of the vocalization model unit 303 in the voice synthesis unit 300 in the voice synthesis LSI to realize the above configuration. In step S1004 in Figure 14 during the key press/release process in step S704 in Figure 7 described above, the CPU 201 generates a vocalization event having lyrics information and pitch information contained in Event [SongIndex] on RAM 203 referenced by the current SongIndex value on RAM 203 as singing voice data 215, and sets this in the issued event area of RAM 203. At the same time, the CPU 201 generates a vocalization event for specifying a vocalization sound source having a predetermined vocalization channel specification as vocalization control data 216 (see Figure 2) together with the pitch information also contained in Event [SongIndex], and sets this in the issued event area of RAM 203.

In response to the above processing, in the speaking/pronunciation processing of step S708, the CPU 201 issues the speaking event stored in the issued event area on the RAM 203 to the voice synthesis LSI 205 of FIG. 2, and issues the pronunciation event for specifying the speaking sound source to the sound source LSI 204 of FIG. 2.

As a result, the sound source LSI 204 uses a specific sound source channel (or multiple channels) specified by the pronunciation event for specifying the vocal sound source to generate musical sound output data 1501 for the vocal sound source having a pitch corresponding to the pitch information included in the pronunciation event for specifying the vocal sound source, and inputs the data to the voice synthesis LSI 205 in FIG. 2 via a signal path not specifically shown in FIG. 2.

The vocal sound source musical sound output data 1501 generated and output by the sound source LSI 204 based on the performer's performance lessons is input to a digital filter formed by the synthesis filter unit 305 based on the target spectrum information 312 input from the acoustic model unit 301 via the formant interpolation processing unit 306 in the voice synthesis LSI 205 of Figure 3, and singing voice output data 217 is output from the synthesis filter unit 305.

The singing voice output data 217 thus generated and output uses the instrument sound generated by the sound source LSI 204 as the sound source signal. Therefore, compared to the singer's singing voice, the fidelity is slightly lost, but the atmosphere of the instrument sound set by the sound source LSI 204 is well preserved, and the vocal quality of the singer's singing voice is also well preserved, making it possible to output effective singing voice output data 217.

Furthermore, the vocal sound source musical sound output data 1501 can also operate polyphonically using multiple channels, making it possible to produce the effect of multiple singing voices harmonizing together.

Note that the vocal sound source musical sound output data 1501 may be any waveform signal, but due to its nature as a sound source signal, it is preferable to use instrument sounds such as brass, string, and organ sounds that contain many harmonic components and last for a long time. Of course, if an instrument sound that does not follow such standards at all, such as an animal's cry, is used to achieve a greater effect, a very interesting effect can be obtained. As a specific example, the vocal sound source musical sound output data 1501 using waveform data obtained by sampling the barking of a pet dog may be input to the synthesis filter unit 310. This will produce a very interesting effect, as if the pet dog is singing the lyrics.

Although the disclosed embodiments and their advantages have been described in detail above, those skilled in the art may make various modifications, additions, and omissions without departing from the scope of the present invention as clearly set forth in the claims.

In addition, the present invention is not limited to the above-described embodiment, and various modifications can be made in the implementation stage without departing from the gist of the invention. Furthermore, the functions performed in the above-described embodiment may be implemented in appropriate combinations as much as possible. The above-described embodiment includes various steps, and various inventions can be extracted by appropriate combinations of the multiple components disclosed. For example, if the effect can be obtained even if some components are deleted from all the components shown in the embodiment, then the configuration from which these components are deleted can be extracted as an invention.

The following supplementary notes are further disclosed regarding the above-described embodiment.
(Appendix 1)
A performance information acquisition means for acquiring performance information of a performer;
a performance evaluation means for evaluating the performance of a performer multiple times at different timings during the progress of a piece of music based on performance guide data including at least lyric information, pitch information, and timing information and the performance information;
a singing voice producing means for producing lyrics by singing voice based on the performance information and the lyrics information;
a voice quality changing means for changing the voice quality of the singing voice in response to a change in the evaluation of the performance evaluation means when the evaluation of the performance evaluation means has changed from a previous evaluation;
An electronic musical instrument comprising:
(Appendix 2)
2. The electronic musical instrument according to claim 1, further comprising a performance guide means for guiding a performer in playing a piece of music based on the performance guide data.
(Appendix 3)
Further comprising a plurality of light emitting elements,
the performance guide means causes a light emitting element corresponding to pitch information included in the performance guide data to emit light at a timing corresponding to timing information included in the performance guide data;
3. The electronic musical instrument according to claim 2.
(Appendix 4)
4. The electronic musical instrument according to claim 1, wherein the voice quality changing means interpolates between a plurality of voice qualities corresponding to a plurality of specific evaluations by the performance evaluation means at a ratio according to the evaluation by the performance evaluation means during the progression of the musical piece.
(Appendix 5)
5. The electronic musical instrument according to claim 1, wherein the voice quality changing means changes a ratio of a formant component of a human voice to a noise component mixed with the human voice.
(Appendix 6)
Electronic musical instrument processors,
A performance information acquisition process for acquiring performance information of a performer;
a performance evaluation process for evaluating the performance of a performer multiple times at different timings during the progression of a piece of music based on performance guide data including at least lyrics information, pitch information, and timing information and the performance information;
a singing voice production process for producing lyrics by singing voice based on the performance information and the lyrics information;
a voice quality change process for changing the voice quality of the singing voice in response to a change in the evaluation of the performance evaluation means when the evaluation of the performance evaluation means has changed from a previous evaluation;
A method for controlling an electronic musical instrument to execute the above.
(Appendix 7)
Electronic musical instrument processors,
A performance information acquisition process for acquiring performance information of a performer;
a performance evaluation process for evaluating the performance of a performer multiple times at different timings during the progression of a piece of music based on performance guide data including at least lyrics information, pitch information, and timing information and the performance information;
a singing voice production process for producing lyrics by singing voice based on the performance information and the lyrics information;
a voice quality change process for changing the voice quality of the singing voice in response to a change in the evaluation of the performance evaluation means when the evaluation of the performance evaluation means has changed from a previous evaluation;
A program for executing.

100 Electronic keyboard instrument 101 Keyboard 102 First switch panel 103 Second switch panel 104 LED
200 Control system 201 CPU
202 ROM
203 RAM
204 Sound source LSI
205 Voice synthesis LSI
206 Key scanner 207 LED controller 208 Network interface 209 System bus 210 Timer 211, 212 D/A converter 213 Mixer 214 Amplifier 215 Singing voice data 216 Sound generation control data 217 Singing voice output data 218 Musical sound output data 300 Voice synthesis unit 301 Acoustic model unit 302 Text analysis unit 303 Vocalization model unit 304 Sound source generation unit 305 Synthesis filter unit 306 Formant interpolation processing unit 307 Noise superimposition unit 308 Language feature sequence 309 Spectral information 310 Sound source information 311 Practice progress data 312 Target spectral information 313 Filter output data 314 Noise mixture ratio 315 Noise data 316 Noise mixture ratio interpolation processing unit 317 Target noise mixture ratio 1501 Musical tone output data for vocal sound source

Claims (7)

A performance information acquisition means for acquiring performance information of a performer;
a performance evaluation means for evaluating the performance of a performer multiple times at different timings during the progress of a piece of music based on performance guide data including at least lyric information, pitch information, and timing information and the performance information;
a singing voice producing means for producing lyrics by singing voice based on the performance information and the lyrics information;
a voice quality changing means for changing the voice quality of the singing voice in response to a change in the evaluation of the performance evaluation means when the evaluation of the performance evaluation means has changed from a previous evaluation;
An electronic musical instrument comprising: The electronic musical instrument according to claim 1, further comprising a performance guide means for guiding the performer in playing a piece of music based on the performance guide data. Further comprising a plurality of light emitting elements,
the performance guide means causes a light emitting element corresponding to pitch information included in the performance guide data to emit light at a timing corresponding to timing information included in the performance guide data;
3. The electronic musical instrument according to claim 2. The electronic musical instrument according to any one of claims 1 to 3, wherein the voice quality change means interpolates between a plurality of voice qualities corresponding to a plurality of specific evaluations by the performance evaluation means at a ratio according to the evaluation by the performance evaluation means during the progression of the piece of music. The electronic musical instrument according to any one of claims 1 to 4, wherein the voice quality change means changes the ratio of formant components of the human voice and noise components mixed with the human voice. Electronic musical instrument processors,
A performance information acquisition process for acquiring performance information of a performer;
a performance evaluation process for evaluating the performance of a performer multiple times at different timings during the progression of a piece of music based on performance guide data including at least lyrics information, pitch information, and timing information and the performance information;
a singing voice production process for producing lyrics by singing voice based on the performance information and the lyrics information;
a voice quality change process for changing the voice quality of the singing voice in response to a change in evaluation of the performance evaluation process from a previous evaluation, when the evaluation of the performance evaluation process has changed from a previous evaluation;
A method for controlling an electronic musical instrument to execute the above. Electronic musical instrument processors,
A performance information acquisition process for acquiring performance information of a performer;
a performance evaluation process for evaluating the performance of a performer multiple times at different timings during the progression of a piece of music based on performance guide data including at least lyrics information, pitch information, and timing information and the performance information;
a singing voice production process for producing lyrics by singing voice based on the performance information and the lyrics information;
a voice quality change process for changing the voice quality of the singing voice in response to a change in evaluation of the performance evaluation process from a previous evaluation, when the evaluation of the performance evaluation process has changed from a previous evaluation;
A program for executing.

Images