JP7660987B2 - Karaoke equipment - Google Patents

Description

The present invention relates to a karaoke device.

Among the songs that the karaoke device plays are duet songs each having a singing section (singing part) for two singers.
2. Description of the Related Art In karaoke devices, a technique is known in which singing voice data of two singers singing karaoke along with a karaoke performance of a duet piece is input and analyzed individually, and the harmony singing is evaluated based on the analysis results.

For example, according to Patent Document 1, a karaoke scoring device includes a storage means for storing melody part reference data for evaluating the singing of the melody part and chorus part reference data for evaluating the singing of the chorus part, an extraction means for extracting singing pitches independently from the singing voice signals of the melody part and the chorus part, and a chorus part scoring means for calculating a singing harmony pitch, which is the pitch difference between the singing pitch of the chorus part and the singing pitch of the melody part, calculating a reference harmony pitch, which is the pitch difference between the pitch of the chorus part reference data and the pitch of the melody part reference data, and scoring the singing of the chorus part based on the degree of agreement between the singing harmony pitch and the reference harmony pitch, thereby scoring harmony singing.

Also, according to Patent Document 2, the karaoke scoring device compares the volume transitions and pitch transitions of the singing signal from the first microphone (e.g., the male part) with the volume transitions and pitch transitions of the singing signal from the second microphone (e.g., the female part), and outputs scoring data that correlates with the temporal similarity of the volume transitions and the temporal similarity of the pitch transitions, thereby scoring the synchronicity when the same melody is sung in singing sections of the male and female parts that do not overlap in time (i.e., singing sections where two singers alternate).

JP 2016-085393 A JP 2006-259237 A

When singing a duet song using a karaoke machine, the two singers do not necessarily take turns singing the same melody, but may take turns singing different melodies during the dialogue sections. However, conventional karaoke machines are unable to evaluate the karaoke singing during the dialogue sections where the singers take turns singing different melodies. For example, even if two singers perform a good karaoke performance during the dialogue sections, they may not receive a score that takes this into account, which can leave them dissatisfied.

The object of the present invention is to provide a karaoke device that can evaluate karaoke singing in a dialogue section where different melodies are sung alternately.

In order to solve the above problem, the karaoke device of the present invention is a karaoke device that performs karaoke of a duet song having a dialogue section in which two singers alternately sing karaoke, and is characterized by having a storage unit that stores, in association with the duet song, first reference data used to evaluate the singing voices of the two singers who sing the duet song and second reference data indicating information on the opening and closing vocals of the two original singers in the dialogue section of the duet song, an acquisition unit that acquires information on the opening and closing vocals of the two singers in the dialogue section based on a comparison between the singing voices of the two singers in the dialogue section and the first reference data, and an evaluation unit that compares the information on the opening and closing vocals of the two singers in the dialogue section with the second reference data and evaluates the karaoke singing of the duet song by the two singers based on the comparison result.

The present invention makes it possible to evaluate karaoke singing in a dialogue section where different melodies are sung alternately.

1 is a block diagram showing a karaoke device according to a first embodiment of the present invention; 4 is a table showing an example of a configuration of a duet song to be played by the karaoke device according to the first embodiment of the present invention. 5 is a table showing an example of second reference data in the karaoke apparatus according to the first embodiment of the present invention. 5 is a flowchart showing an operation of evaluating karaoke singing of a duet piece in the karaoke apparatus according to the first embodiment of the present invention. 5 is a table showing an example of an evaluation result of karaoke singing of a duet song in the karaoke device according to the first embodiment of the present invention. 5 is a table showing an example of an evaluation result of karaoke singing of a duet song in the karaoke device according to the first embodiment of the present invention. FIG. 11 is a block diagram showing a karaoke apparatus according to a second embodiment of the present invention. 10 is a flowchart showing the operation of the karaoke device according to the second embodiment of the present invention. FIG. 11 is a block diagram showing a karaoke device according to a third embodiment of the present invention. 10 is a flowchart showing the operation of the karaoke device according to the third embodiment of the present invention.

[First embodiment]
A karaoke device 1 according to a first embodiment of the present invention will now be described.

[Karaoke equipment]
Fig. 1 is a block diagram showing the configuration of a karaoke machine 1 according to the first embodiment. As shown in Fig. 1, the karaoke machine 1 includes a karaoke main unit 2, a first microphone 3, a second microphone 4, a speaker 5, a display device 6 such as a monitor, and a remote control device 7. The karaoke machine 1 is installed in an installation location such as a karaoke room.

The karaoke main unit 2 is connected by wire or wirelessly to a first microphone 3, a second microphone 4, a speaker 5, a display device 6, and a remote control device 7. The first microphone 3 and the second microphone 4 convert the user's singing voice into an audio signal and input it to the karaoke main unit 2. The speaker 5 emits the user's voice and a karaoke performance of a song based on the sound emission signal from the karaoke main unit 2. The display device 6 displays various screens based on the video signal etc. from the karaoke main unit 2, and for example displays lyrics captions together with a background image in time with the karaoke performance.

The remote control device 7 is mainly composed of a touch panel. The remote control device 7 displays various screens and various operation buttons on the touch panel, and also accepts login operations and song selection (reservation) operations via the touch panel. The remote control device 7 and the karaoke main unit 2 are paired via wireless communication, allowing various information to be transmitted and received between them.

The remote control device 7 transmits user identification information (user ID) and other information input in response to the user's login operation to the karaoke unit 2. The remote control device 7 accepts song searches and song selections (reservations), and transmits song identification information (song ID) of the song selected in response to the user's song selection operation to the karaoke unit 2 as reserved song information.

The karaoke unit 2 also has a function for evaluating the singing voice of the singer, in addition to the function for performing karaoke of the selected song.

The karaoke main unit 2 comprises a control unit 10, a memory unit 11, a sound source 12, and an audio input/output unit 13. The control unit 10 is configured with a CPU (Central Processing Unit) and other components, and controls the karaoke main unit 2. The memory unit 11 is configured with storage media such as a ROM (Read Only Memory), RAM (Random Access Memory), and HDD (Hard Disc Drive), and stores programs and data for controlling each part and function of the karaoke device 1.

For example, the storage unit 11 stores the user ID of each user of the karaoke device 1 to identify each user. The storage unit 11 also stores basic information such as the song ID, song title, singer name, and genre for each song that the karaoke device 1 can play. The storage unit 11 also stores song data for each song, which includes accompaniment data, reference data, and lyric data. The song data also includes performance section information that identifies non-sung sections such as introduction sections, interlude sections, and postlude sections in the song, as well as singing sections. The accompaniment data is MIDI format data that is the source of accompaniment sounds for karaoke performance. The reference data is pitch information that is used as a standard for evaluating a singer's karaoke singing. The lyric data is data that is the source of lyric subtitles that are displayed on the display device 6.

The songs that the karaoke device 1 plays include duet songs that have a singing section (singing part) for each of the two singers. A duet song includes a first singing part (e.g., a female part) sung by one of the two singers and a second singing part (e.g., a male part) sung by the other singer, and includes, for example, a solo section in which each singer sings solo and a choral section in which the two singers sing together.

In a duet song, the solo section includes a call-and-response section in which two singers alternate (e.g., a male and female singing section that do not overlap in time), and in this embodiment, for example, refers to a section in which a first singing part by one singer is followed by a second singing part by the other singer, and these singing parts have a corresponding relationship like a call-and-response. The chorus section includes a harmony singing section in which the first and second singing parts of the two singers have a pitch difference. The song data for a duet song includes, as performance section information, information on the first and second singing parts of each call-and-response section, and information on each harmony singing section.

Figure 2 shows an example of a duet song X1 consisting of 48 bars with a male part (male voice) and a female part (female voice), and the breakdown of each bar will be explained below. Bars 01-04 are the introductory section (non-singing section). Bars 05-08 are the first dialogue section, with bars 05-06 being the female part and bars 07-08 being the male part. Bars 09-12 are the second dialogue section, with bars 09-10 being the female part and bars 11-12 being the male part. Bars 13-17 are the interlude section (non-singing section). Measures 18-25 are the third dialogue section, with bars 18-21 being the female part and bars 22-25 being the male part. Measures 26-27 are the fourth dialogue singing section, with measure 26 being the female part and measure 27 being the male part. Measures 28-29 are the fifth dialogue singing section, with measure 28 being the female part and measure 29 being the male part. Measures 30-31 are the first harmony singing section, which has both male and female parts. Measures 32-35 are the sixth dialogue singing section, with measures 32-33 being the female part and measures 34-35 being the male part. Measures 36-39 are the seventh dialogue singing section, with measures 36 being the female part and measures 37-39 being the male part. Measures 40-43 are the second harmony singing section, which has both male and female parts. Measures 44-48 are the postlude section (non-singing section).

In particular, in this embodiment, the reference data for a duet song includes first reference data used to evaluate the singing voices of the two singers who sing the duet song karaoke, and second reference data that indicates information on the opening and closing vocalizations of the two original singers in the exchange section of the duet song. The memory unit 11 stores the first reference data and second reference data for a specific duet song in association with the duet song (e.g., its song identification information).

The first reference data includes reference data corresponding to a first singing part sung by one of the two singers and reference data corresponding to a second singing part sung by the other singer, and is created so that, for example, the vocal timing of each phrase matches the musical score of the original song.

The second reference data indicates the relationship between the vocal timing of each original singer in the voice-over singing section and the vocal timing based on the first reference data. For example, if the vocal timing of the original singer in the singing part that constitutes the voice-over singing section (e.g., the vocal timing of the first sound of each phrase) is earlier than the first reference data by a predetermined running time threshold (e.g., 50 msec) or more, the second reference data of the singing part is set to "running singing", and if it is later than the first reference data by a predetermined accumulation time threshold (e.g., 50 msec) or more, the second reference data of the singing part is set to "accumulating singing", and if it does not fall under either of the above cases (e.g., if the difference from the vocal timing based on the first reference data is less than 50 msec), the second reference data of the singing part is set to "neither".

The second reference data may be created by determining the vocal timing of each original singer during the call-and-response section based on the result of listening to the original singers singing the duet song (e.g., on a CD). Alternatively, the second reference data may be created by analyzing the singing voices of the duet song by each original singer using a computer with any of a variety of analysis methods such as frequency analysis to detect the vocal timing of each original singer during the call-and-response section and comparing it with the first reference data.

In the second reference data, the information on the quick and slow singing of each interlocutor section by the two original singers A1 and A2 includes nine patterns such as "A1: quick singing, A2: slow singing", "A1: slow singing, A2: slow singing", "A1: neither, A2: quick singing", etc., each of which is closely related to the atmosphere of the song and the content of the lyrics.

For example, duet songs with male and female parts often express the feelings of men and women, and in the dialogue section, if the leading female part sings "fast singing" and the following male part sings "build-up singing", karaoke singing with an adult atmosphere is expressed. Also, if both singers sing "build-up singing" in the dialogue section, karaoke singing with a heavy atmosphere is expressed, and for example, it is karaoke singing suitable for a song that wants to express sadness. On the other hand, if both singers sing "fast singing" in the dialogue section, karaoke singing with a light atmosphere is expressed, and for example, it is karaoke singing suitable for a song with comical lyrics. Professional artists (original singers) are skilled in singing techniques that allow for a variety of expressions, so the karaoke device 1 configures the second reference data based on information on the original singer's "fast singing" and "build-up singing".

Figure 3 shows an example of the second reference data based on the singing of duet song X1 by two original singers A1 and A2, and the details of each singing section are explained below.

In the first call-and-sing section, if the information on the fast singing and build-up singing of original singers A1 and A2 is "fast singing" and "build-up singing", the second reference data is set as "first call-and-sing section, original singer A1: fast singing, original singer A2: build-up singing", in this case, it is original singer A1 who is singing relatively faster. In the second call-and-sing section, if the information on the fast singing and build-up singing of original singers A1 and A2 is "fast singing" and "build-up singing", the second reference data is set as "second call-and-sing section, original singer A1: fast singing, original singer A2: build-up singing", in this case too, it is original singer A1 who is singing relatively faster.

In addition, in the third call-and-sing section, if the information on the fast singing and build-up singing of original singers A1 and A2 is "build-up singing" and "neither," the second reference data is set as "third call-and-sing section, original singer A1: build-up singing, original singer A2: neither," and in this case, it is original singer A2 who is singing relatively faster. In the fourth call-and-sing section, if the information on the fast singing and build-up singing of original singers A1 and A2 is "neither" and "fast singing," the second reference data is set as "fourth call-and-sing section, original singer A1: neither, original singer A2: fast singing," and in this case too, it is original singer A2 who is singing relatively faster. In the fifth call-and-response section, if the information on the fast singing and build-up singing of original singers A1 and A2 is "neither" and "fast singing," the second reference data is set as "fifth call-and-response section, original singer A1: neither, original singer A2: fast singing," and in this case too, it is original singer A2 who is singing relatively faster.

In addition, in the sixth call-and-sing section, if the information on the fast singing and the build-up singing of original singers A1 and A2 is "fast singing" and "build-up singing", the second reference data is set as "sixth call-and-sing section, original singer A1: fast singing, original singer A2: build-up singing", in which case it is original singer A1 who is singing relatively fast. In the seventh call-and-sing section, if the information on the fast singing and the build-up singing of original singers A1 and A2 is "build-up singing" and "build-up singing", the second reference data is set as "seventh call-and-sing section, original singer A1: build-up singing, original singer A2: build-up singing", in which case both original singers A1 and A2 are "build-up singing", so there is no original singer who is singing relatively fast (A1 = A2).

In particular, in this embodiment, the lyric data of a duet song is stored by identifying the singing parts of the two singers who sing the duet song karaoke. For example, the lyric data of a duet song is stored so that the first and second singing parts of the two singers in the dialogue singing section and the harmony singing section can be identified.

The sound source 12 is, for example, a MIDI sound source, and is a karaoke performance means that generates an accompaniment sound signal for a karaoke performance based on the accompaniment data of the music piece data.

The audio input/output unit 13 is composed of a mixer (digital mixer), an amplifier, etc., and mixes the accompaniment sound signals of the karaoke performance and the audio signals of the karaoke singing, and amplifies this mixed signal so that it can be emitted from the speaker 5. For example, the audio input/output unit 13 is communicatively connected to the first microphone 3 and the second microphone 4, and inputs audio signals from each of the first microphone 3 and the second microphone 4.

The control unit 10 controls each part and each function of the karaoke device 1 by executing calculations based on the programs and data stored in the memory unit 11. For example, the control unit 10 of this embodiment has the function of operating as an acquisition unit 20, an evaluation unit 21, and a presentation unit 22 by executing the programs stored in the memory unit 11, and evaluating the karaoke singing of a duet song.

The acquisition unit 20 is configured to acquire information on the initial singing and the build-up singing of the two singers in the exchange singing section based on a comparison between the singing voices of the two singers in the exchange singing section and the first reference data when the karaoke device 1 is performing a karaoke performance of a duet song. To acquire information on the initial singing and the build-up singing, the publicly known technology disclosed in JP 2007-072316 A and JP 2005-107330 A can be used.

The acquisition unit 20 inputs the first audio signal and the second audio signal, which indicate the singing voices of the two singers input from the first microphone 3 and the second microphone 4, respectively, via the audio input/output unit 13. The acquisition unit 20 determines the call-and-response section based on the performance section information of the music data of the duet song, and acquires the singing voices of the two singers corresponding to the call-and-response section from the first audio signal and the second audio signal to obtain the first call-and-response section and the second call-and-response section. The acquisition unit 20 also compares the first call-and-response section and the second call-and-response section with the first reference data to acquire information on the early singing and the late singing of the two singers in the call-and-response section.

For example, when a duet song is composed of a first singing part and a second singing part, the case will be described where singing using the first microphone 3 corresponds to the first singing part and singing using the second microphone 4 corresponds to the second singing part. The acquisition unit 20 acquires a first vocalization standard and a second vocalization standard as the vocalization timing of the first singing part and the second singing part in the corresponding exchange singing section in the first reference data, and also acquires a first vocalization timing and a second vocalization timing as the vocalization timing of the first exchange singing and the second exchange singing of the two singers in the corresponding exchange singing section.

The acquisition unit 20 then compares the first vocalization criterion of the first singing part with the first vocalization timing of the first call-and-response singing, and if the first vocalization timing is earlier than the first vocalization criterion by a predetermined running time threshold or more, judges the first call-and-response singing to be "running singing", if the first vocalization timing is later than the first vocalization criterion by a predetermined accumulation time threshold or more, judges the first call-and-response singing to be "accumulating singing", and if neither of these apply (if the first vocalization timing is later than the predetermined running time threshold and earlier than the predetermined accumulation time threshold), judges the first call-and-response singing to be "neither".

Similarly, the acquisition unit 20 compares the second vocalization standard of the second singing part with the second vocalization timing of the second call singing, and if the second vocalization timing is earlier than the second vocalization standard by a predetermined running time threshold or more, it judges the second call singing to be "running singing", if the second vocalization timing is later than the second vocalization standard by a predetermined accumulation time threshold or more, it judges the second call singing to be "accumulating singing", and if neither of these apply (if the second vocalization timing is later than the predetermined running time threshold and earlier than the predetermined accumulation time threshold), it judges the second call singing to be "neither".

When determining whether the first audio signal or the second audio signal obtained from the first microphone 3 or the second microphone 4 corresponds to the first singing part or the second singing part of the duet song, the acquisition unit 20 may make the determination based on the vocalization timing of the first audio signal and the second audio signal and the vocalization timing of the first singing part and the second singing part. In other words, of the first audio signal and the second audio signal, the audio signal closer to the vocalization timing of the first singing part may be determined to correspond to the first singing part, and the audio signal closer to the vocalization timing of the second singing part may be determined to correspond to the second singing part.

The evaluation unit 21 is basically configured to evaluate the singing voice sung by the singer in time with the karaoke performance of the song, and calculate score information as the evaluation result. Specifically, the evaluation unit 21 acquires the first and second audio signals input from the first and second microphones 3 and 4 by the karaoke main unit 2 via the audio input/output unit 13, and evaluates the singing voice by comparing them with the reference data of the song stored in the storage unit 11.

In particular, in this embodiment, the evaluation unit 21 is configured to evaluate the karaoke singing of a duet song by two singers when the karaoke device 1 performs a karaoke performance of the duet song. At this time, the evaluation unit 21 compares the first and second audio signals input from the first and second microphones 3 and 4 with reference data corresponding to the first and second singing parts in the first reference data, and evaluates and scores the singing voices of the first and second audio signals based on the comparison result.

For example, the evaluation unit 21 evaluates the pitch accuracy and singing technique of the singing voice indicated by the first audio signal, and scores the first singing part based on the first audio signal on a scale of 40 points, evaluates the pitch accuracy and singing technique of the singing voice indicated by the second audio signal, and scores the second singing part based on the second audio signal on a scale of 40 points, and further scores the harmony singing section including both the first and second singing parts on a scale of 10 points.

The evaluation unit 21 also compares the information on the early and late singing of the two singers in the dialogue section acquired by the acquisition unit 20 with the second reference data, and evaluates and scores the singing voices in the dialogue section based on the comparison result. For example, the evaluation unit 21 scores the dialogue section including both the first and second singing parts on a scale of 10 points.

Then, the evaluation unit 21 calculates overall scoring information for the karaoke singing of the duet song based on the scoring information for the first audio signal (first singing part), the scoring information for the second audio signal (second singing part), the scoring information for the harmony singing section, and the scoring information for the call-and-response singing section, and scores the karaoke singing of the duet song, for example, out of a maximum of 100 points.

The presentation unit 22 is configured to display the scoring information of the karaoke singing evaluated by the evaluation unit 21 on the display device 6 or the remote control device 7 and present it to the user. In this case, the presentation unit 22 may display not only the overall scoring information of the karaoke singing of the duet song, but also the scoring information of the first audio signal (first singing part), the scoring information of the second audio signal (second singing part), the scoring information of the harmony singing section, and the scoring information of the call-and-response singing section individually, and may also display the scoring results of multiple scoring items (e.g., pitch accuracy, rhythm, vibrato, etc.) for the scoring information of the first audio signal (first singing part), the scoring information of the second audio signal (second singing part), and the scoring information of the harmony singing section.

[Evaluation Operation of the First Embodiment]
Next, in the first embodiment, when two singers U1 and U2 sing the karaoke song X1 shown in Fig. 2, the karaoke device 1 performs an evaluation operation using the second reference data shown in Fig. 3. The evaluation operation of the karaoke device 1 in the first embodiment is described below with reference to Fig. 4. Fig. 4 is a flow chart showing the evaluation operation of the karaoke device 1 in the first embodiment.

First, when one of the two singers U1 and U2 operates the remote control device 7 to reserve duet song X1, the control unit 10 of the karaoke main unit 2 registers the reserved song information of duet song X1 received from the remote control device 7 in the reservation management table of the storage unit 11. The control unit 10 reads the reserved song information of duet song X1 from the reservation management table, and reads the song data of duet song X1 corresponding to the song ID of this reserved song information from the storage unit 11.

The karaoke main unit 2 starts the karaoke performance of the duet song X1 (step S1), reads out the song data of the duet song X1 from the storage unit 11, performs the karaoke performance of the duet song X1 using the sound source 12 and the audio input/output unit 13, and emits the accompaniment sound of the duet song X1 from the speaker 5. When the singers U1 and U2 sing karaoke in time with the karaoke performance using the first microphone 3 and the second microphone 4, respectively, the singing voice is emitted from the speaker 5 together with the accompaniment sound of the duet song X1.

The karaoke main unit 2 also displays lyric captions on the display device 6 based on the lyric data of the duet song X1, changes the display mode (e.g., color, etc.) of the lyric captions in accordance with the performance section information of the song data, and further displays the lyric captions for the first singing part (e.g., female part) and the second singing part (e.g., male part) of the duet song X1 in a manner that makes them distinguishable by color, etc. For example, the display mode of the lyric captions displays the sung parts and unsung parts in a manner that makes them distinguishable by color, etc., and determines in real time the timing when the lyrics have been sung in accordance with the performance section information, and changes the lyric captions for the unsung parts to the sung parts based on this timing. In other words, the display switch timing (e.g., color change timing) at which the unsung parts of the lyric captions are switched to the sung parts is based on the song data.

When performing karaoke for duet song X1, the introductory section (measures 01-04) is played first (step S2: NO).

Next, when the first call-and-response section (measures 05-08) is performed (step S2: YES), the acquisition unit 20 acquires the first call-and-response singing corresponding to the first singing part (measures 05-06) of the first call-and-response section from the first audio signal indicating the singing voice of singer U1. The acquisition unit 20 compares the first call-and-response singing of singer U1 with the first reference data in the first singing part (measures 05-06) of the first call-and-response section, and acquires the vocalization timing of the first call-and-response singing relative to the first reference data. As an example, if the difference in vocalization timing between singer U1 and the first reference data is less than a predetermined running time threshold and less than a predetermined accumulation time threshold, the acquisition unit 20 determines that the first singing part (measures 05-06) of the first call-and-response section by singer U1 is "neither."

The acquisition unit 20 also acquires a second call-and-response singing corresponding to the second singing part (bars 07-08) of the first call-and-response section from the second audio signal indicating the singing voice of singer U2. The acquisition unit 20 compares the second call-and-response singing of singer U2 in the second singing part (bars 07-08) of the first call-and-response section with the first reference data to acquire the vocalization timing of the second call-and-response singing relative to the first reference data. As an example, if the vocalization timing of singer U2 is slower than the first reference data by a predetermined hold-up time threshold or more, the acquisition unit 20 judges the second singing part (bars 07-08) of the first call-and-response section by singer U2 to be "hold-up singing". The acquisition unit 20 then acquires the information on the accelerated singing and sustained singing in the first call-and-response section, as shown in FIG. 5, as "first call-and-response section, singer U1: neither, singer U2: sustained singing, relatively accelerated: U1" (step S3).

Next, the evaluation unit 21 compares the information on the initial singing and build-up singing of the first exchange singing section of the two singers U1 and U2 with the second reference data, and evaluates the singing of the first exchange singing section based on the comparison result (step S4). As an example, in the first inter-singing section, if the information on the early and late singing of the two singers U1 and U2 is "first inter-singing section, singer U1: neither, singer U2: late singing, relatively early: U1" and the second reference data is "first inter-singing section, original singer A1: early singing, original singer A2: late singing", i.e., "relatively early: A1", then the information on the early and late singing of the first singing part of both singers does not match, but the information on the early and late singing of the second singing part of both singers matches, and the relative relationship in which the singing in the first singing part is relatively early and the singing in the second singing part is relatively late matches. In this case, the evaluation unit 21 scores the evaluation of the first inter-singing section as 5.0 points and stores it in the memory unit 11.

Next, when the second call-and-response section (measures 09-12) is performed (step S2: YES), the acquisition unit 20 acquires the first call-and-response singing corresponding to the first singing part (measures 09-10) of the second call-and-response section from the first audio signal indicating the singing voice of singer U1. The acquisition unit 20 compares the first call-and-response singing of singer U1 with the first reference data in the first singing part (measures 09-10) of the second call-and-response section, and acquires the vocalization timing of the first call-and-response singing relative to the first reference data. As an example, if the vocalization timing of singer U1 is earlier than the first reference data by a predetermined running time threshold or more, the acquisition unit 20 judges the first singing part (measures 09-10) of the second call-and-response section by singer U1 to be "running singing".

The acquisition unit 20 also acquires a second call-and-response singing corresponding to the second singing part (bars 11-12) of the second call-and-response section from the second audio signal indicating the singing voice of singer U2. The acquisition unit 20 compares the second call-and-response singing of singer U2 in the second singing part (bars 11-12) of the second call-and-response section with the first reference data to acquire the vocalization timing of the second call-and-response singing relative to the first reference data. As an example, if the vocalization timing of singer U2 is slower than the first reference data by a predetermined hold-up time threshold or more, the acquisition unit 20 judges the second singing part (bars 11-12) of the second call-and-response section by singer U2 to be "hold-up singing." The acquisition unit 20 then acquires, as information on the accelerated singing and sustained singing in the second call-and-response section, "second call-and-response section, singer U1: accelerated singing, singer U2: sustained singing, relatively accelerated: U1," as shown in FIG. 5 (step S3).

Next, the evaluation unit 21 compares the information on the initial singing and build-up singing of the second exchange singing section of the two singers U1 and U2 with the second reference data, and evaluates the singing of the second exchange singing section based on the comparison result (step S4). As an example, in the second inter-singing section, if the information on the fast singing and the build-up singing of the two singers U1 and U2 is "second inter-singing section, singer U1: fast singing, singer U2: build-up singing, relatively fast: U1" and the second reference data is "second inter-singing section, original singer A1: fast singing, original singer A2: build-up singing", i.e., "relatively fast: A1", then the information on the fast singing and the build-up singing of the first singing part of both singers is the same, and the information on the fast singing and the build-up singing of the second singing part of both singers is the same, and the relative relationship in which the singing in the first singing part is relatively fast and the singing in the second singing part is relatively build-up is also the same. In this case, the evaluation unit 21 scores the evaluation of the second inter-singing section as 10.0 points and stores it in the memory unit 11.

Next, after the interlude section (measures 13-17) is performed (step S2: NO), when the third call-and-response section (measures 18-25) is performed (step S2: YES), the acquisition unit 20 acquires the first call-and-response singing corresponding to the first singing part (measures 18-21) of the third call-and-response section from the first audio signal indicating the singing voice of singer U1. In the first singing part (measures 18-21) of the third call-and-response section, the acquisition unit 20 compares the first call-and-response singing of singer U1 with the first reference data, and acquires the vocalization timing of the first call-and-response singing relative to the first reference data. As an example, if the difference in vocalization timing between singer U1 and the first reference data is less than a predetermined running time threshold and less than a storage time threshold, the acquisition unit 20 determines that the first singing part (measures 18 to 21) of the third call-and-response singing section by singer U1 is "neither."

The acquisition unit 20 also acquires a second call-and-response singing corresponding to the second singing part (bars 22-25) of the third call-and-response section from the second audio signal indicating the singing voice of singer U2. The acquisition unit 20 compares the second call-and-response singing of singer U2 with the first reference data in the second singing part (bars 22-25) of the third call-and-response section to acquire the vocalization timing of the second call-and-response singing relative to the first reference data. As an example, if the vocalization timing of singer U2 is earlier than the first reference data by a predetermined running time threshold or more, the acquisition unit 20 judges the second singing part (bars 22-25) of the third call-and-response section by singer U2 to be "running singing." The acquisition unit 20 then acquires the information on the accelerated singing and build-up singing in the third call-and-response section, as shown in FIG. 5, as "third call-and-response section, singer U1: neither, singer U2: accelerated singing, relatively accelerated: U2" (step S3).

Next, the evaluation unit 21 compares the information on the initial singing and build-up singing of the third exchange singing section of the two singers U1 and U2 with the second reference data, and evaluates the singing of the third exchange singing section based on the comparison result (step S4). As an example, in the third inter-singing section, if the information on the early singing and the build-up singing of the two singers U1 and U2 is "third inter-singing section, singer U1: neither, singer U2: early singing, relatively early: U2" and the second reference data is "third inter-singing section, original singer A1: build-up singing, original singer A2: neither," i.e., "relatively early: A2," then the information on the early singing and the build-up singing of the first singing part of both singers is inconsistent, and the information on the early singing and the build-up singing of the second singing part of both singers is inconsistent, but the relative relationship in which the singing in the first singing part is relatively early and the singing in the second singing part is relatively early is consistent. In this case, the evaluation unit 21 scores the evaluation of the third inter-singing section as 5.0 points and stores it in the memory unit 11.

Next, when the fourth call-and-response section (measures 26-27) is performed (step S2: YES), the acquisition unit 20 acquires the first call-and-response singing corresponding to the first singing part (measure 26) of the fourth call-and-response section from the first audio signal indicating the singing voice of singer U1. The acquisition unit 20 compares the first call-and-response singing of singer U1 with the first reference data in the first singing part (measure 26) of the fourth call-and-response section, and acquires the vocalization timing of the first call-and-response singing relative to the first reference data. As an example, if the vocalization timing of singer U1 is earlier than the first reference data by a predetermined running time threshold or more, the acquisition unit 20 judges the first singing part (measure 26) of the fourth call-and-response section by singer U1 to be "running singing".

The acquisition unit 20 also acquires a second call-and-response singing corresponding to the second singing part (bar 27) of the fourth call-and-response singing section from the second audio signal indicating the singing voice of singer U2. The acquisition unit 20 compares the second call-and-response singing of singer U2 with the first reference data in the second singing part (bar 27) of the fourth call-and-response singing section to acquire the vocalization timing of the second call-and-response singing relative to the first reference data. As an example, if the difference in vocalization timing between singer U2 and the first reference data is less than a predetermined running time threshold and less than a predetermined accumulation time threshold, the acquisition unit 20 determines that the second singing part (bar 27) of the fourth call-and-response singing section by singer U2 is "neither." The acquisition unit 20 then acquires, as information on the accelerated singing and build-up singing in the fourth call-and-response section, "fourth call-and-response section, singer U1: accelerated singing, singer U2: neither, relatively accelerated: U1," as shown in FIG. 5 (step S3).

Next, the evaluation unit 21 compares the information on the early singing and the build-up singing of the two singers U1 and U2 in the fourth exchange singing section with the second reference data, and evaluates the singing of the fourth exchange singing section based on the comparison result (step S4). As an example, if the information on the early singing and the build-up singing of the two singers U1 and U2 in the fourth exchange singing section is "fourth exchange singing section, singer U1: early singing, singer U2: neither, relatively early: U1" and the second reference data is "fourth exchange singing section, original singer A1: neither, original singer A2: early singing", i.e., "relatively early: A2", then The information on the early and late singing of the first singing part does not match, and the information on the early and late singing of the second singing parts of both singers does not match, so that in the singing of the two singers U1 and U2, singer U1 (first singing part) tends to be relatively early, while in the singing of the two original singers A1 and A2 (second reference data), original singer A2 (second singing part) tends to be relatively early, and the relative relationship between the two is reversed. In this case, the evaluation unit 21 scores the fourth exchange singing section as 0.0 points and stores it in the memory unit 11.

Next, when the fifth call-and-response section (measures 28-29) is performed (step S2: YES), the acquisition unit 20 acquires the first call-and-response singing corresponding to the first singing part (measure 28) of the fifth call-and-response section from the first audio signal indicating the singing voice of singer U1. The acquisition unit 20 compares the first call-and-response singing of singer U1 with the first reference data in the first singing part (measure 28) of the fifth call-and-response section, and acquires the vocalization timing of the first call-and-response singing relative to the first reference data. As an example, if the vocalization timing of singer U1 is earlier than the first reference data by a predetermined running time threshold or more, the acquisition unit 20 judges the first singing part (measure 28) of the fifth call-and-response section by singer U1 to be "running singing".

The acquisition unit 20 also acquires a second call-and-response singing corresponding to the second singing part (bar 29) of the fifth call-and-response singing section from the second audio signal indicating the singing voice of singer U2. The acquisition unit 20 compares the second call-and-response singing of singer U2 with the first reference data in the second singing part (bar 29) of the fifth call-and-response singing section to acquire the vocalization timing of the second call-and-response singing relative to the first reference data. As an example, if the vocalization timing of singer U2 is earlier than the first reference data by a predetermined running time threshold or more, the acquisition unit 20 judges the second singing part (bar 29) of the fifth call-and-response singing section by singer U2 to be "running singing". The acquisition unit 20 then acquires "5th call-and-response section, singer U1: rushing singing, singer U2: rushing singing, relatively rushing: U1=U2" as information on the rushing and building up singing in the 5th call-and-response section, as shown in Figure 5 (step S3). In this case, since both singers U1 and U2 are "rushing singing," no singer is singing in a relatively rushed manner, as described above.

Next, the evaluation unit 21 compares the information on the initial singing and build-up singing of the fifth exchange singing section of the two singers U1 and U2 with the second reference data, and evaluates the singing of the fifth exchange singing section based on the comparison result (step S4). As an example, in the fifth call-and-response singing section, the information on the fast singing and build-up singing of the two singers U1 and U2 is "Fifth call-and-response singing section, singer U1: fast singing, singer U2: fast singing, relatively fast: U1 = U2", and the second reference data is "Fifth call-and-response singing section, original singer A1: neither, original singer A2: fast singing", i.e., "relatively fast: A2", then the information on the fast singing and build-up singing of the first singing parts of both singers is inconsistent, but the information on the fast singing and build-up singing of the second singing parts of both singers is inconsistent, and while neither of the singers U1 and U2 is in a fast manner, the singing of the two original singers A1 and A2 (second reference data) shows that original singer A2 (second singing part) is relatively fast, and although the relative relationship between the two singers is not reversed, they are inconsistent. In this case, the evaluation unit 21 scores the fifth vocal exchange section at 2.5 points and stores the score in the memory unit 11.

Next, after the first harmony singing section (measures 30-31) is performed (step S2: NO), when the sixth call-and-response singing section (measures 32-35) is performed (step S2: YES), the acquisition unit 20 acquires the first call-and-response singing corresponding to the first singing part (measures 32-33) of the sixth call-and-response singing section from the first audio signal indicating the singing voice of the singer U1. The acquisition unit 20 compares the first call-and-response singing of the singer U1 with the first reference data in the first singing part (measures 32-33) of the sixth call-and-response singing section, and acquires the vocalization timing of the first call-and-response singing relative to the first reference data. As an example, if the vocalization timing of the singer U1 is earlier than the first reference data by a predetermined running time threshold or more, the acquisition unit 20 judges the first singing part (measures 32-33) of the sixth call-and-response singing section by the singer U1 as "running singing".

The acquisition unit 20 also acquires a second call-and-response singing corresponding to the second singing part (bars 34-35) of the sixth call-and-response singing section from the second audio signal indicating the singing voice of singer U2. The acquisition unit 20 compares the second call-and-response singing of singer U2 with the first reference data in the second singing part (bars 34-35) of the sixth call-and-response singing section to acquire the vocalization timing of the second call-and-response singing relative to the first reference data. As an example, if the vocalization timing of singer U2 is slower than the first reference data by a predetermined hold-up time threshold or more, the acquisition unit 20 judges the second singing part (bars 34-35) of the sixth call-and-response singing section by singer U2 to be "hold-up singing." The acquisition unit 20 then acquires, as information on the accelerated singing and sustained singing in the sixth call-and-response section, "sixth call-and-response section, singer U1: accelerated singing, singer U2: sustained singing, relatively accelerated: U1," as shown in FIG. 5 (step S3).

Next, the evaluation unit 21 compares the information on the initial singing and build-up singing of the six exchange singing section of the two singers U1 and U2 with the second reference data, and evaluates the singing of the sixth exchange singing section based on the comparison result (step S4). As an example, in the sixth inter-singing section, if the information on the fast singing and the build-up singing of the two singers U1 and U2 is "sixth inter-singing section, singer U1: fast singing, singer U2: build-up singing, relatively fast: U1" and the second reference data is "sixth inter-singing section, original singer A1: fast singing, original singer A2: build-up singing", i.e., "relatively fast: A1", then the information on the fast singing and the build-up singing of the first singing part of both singers is the same, and the information on the fast singing and the build-up singing of the second singing part of both singers is the same, and the relative relationship in which the singing in the first singing part is relatively fast and the singing in the second singing part is relatively build-up is also the same. In this case, the evaluation unit 21 scores the evaluation of the sixth inter-singing section as 10.0 points and stores it in the memory unit 11.

Next, when the seventh call-and-response section (measures 36-39) is performed (step S2: YES), the acquisition unit 20 acquires the first call-and-response singing corresponding to the first singing part (measure 36) of the seventh call-and-response section from the first audio signal representing the singing voice of singer U1. The acquisition unit 20 compares the first call-and-response singing of singer U1 with the first reference data in the first singing part (measure 36) of the seventh call-and-response section to acquire the vocalization timing of the first call-and-response singing relative to the first reference data. As an example, if the difference in vocalization timing between singer U1 and the first reference data is less than a predetermined running time threshold and less than a predetermined accumulation time threshold, the acquisition unit 20 determines that the first singing part (measure 36) of the seventh call-and-response section by singer U1 is "neither."

The acquisition unit 20 also acquires a second call-and-response singing corresponding to the second singing part (bars 37-39) of the seventh call-and-response singing section from the second audio signal indicating the singing voice of singer U2. The acquisition unit 20 compares the second call-and-response singing of singer U2 with the first reference data in the second singing part (bars 37-39) of the seventh call-and-response singing section to acquire the vocalization timing of the second call-and-response singing relative to the first reference data. As an example, if the difference in vocalization timing between singer U2 and the first reference data is less than a predetermined running time threshold and less than a predetermined accumulation time threshold, the acquisition unit 20 determines that the second singing part (bars 37-39) of the seventh call-and-response singing section by singer U2 is "neither." The acquisition unit 20 then acquires "7th call-and-response section, singer U1: neither, singer U2: neither, relatively quick: U1=U2" as information on the quick singing and build-up singing in the 7th call-and-response section, as shown in Figure 5 (step S3). In this case, since both singers U1 and U2 are "neither," as described above, no singer is singing in a relatively quick manner.

Next, the evaluation unit 21 compares the information on the initial singing and build-up singing of the seventh exchange singing section of the two singers U1 and U2 with the second reference data, and evaluates the singing of the seventh exchange singing section based on the comparison result (step S4). As an example, in the seventh call-and-response section, the information on the fast singing and the slow singing of the two singers U1 and U2 is "seventh call-and-response section, singer U1: neither, singer U2: neither, relatively fast: U1=U2" and the second reference data is "seventh call-and-response section, original singer A1: slow singing, original singer A2: slow singing", i.e., "relatively fast: A1=A2", then the information on the fast singing and the slow singing of the first singing part of both singers is inconsistent, and the information on the fast singing and the slow singing of the second singing part of both singers is inconsistent, but the relative relationship in which neither of the singing in the first singing part and the second singing part is fast is consistent. In this case, the evaluation unit 21 scores the evaluation of the seventh call-and-response section as 5.0 points and stores it in the memory unit 11.

Next, the second harmony singing section (measures 40-43) is performed (step S2: NO), and then the postlude section (measures 44-48) is performed (step S2: NO), and the karaoke performance of duet song X1 ends (step S5: YES).

When the karaoke performance of duet song X1 is completed, the evaluation unit 21 compares the first audio signal representing the singing voice of singer U1 with the first reference data to evaluate the singing voice of singer U1, scoring it as 35.0 points, for example, as shown in FIG. 6, and also compares the second audio signal representing the singing voice of singer U2 with the first reference data to evaluate the singing voice of singer U2, scoring it as 37.5 points, for example. The evaluation unit 21 also compares the first audio signal and the second audio signal of the first harmony singing section and the second harmony singing section with the first reference data to evaluate the singing voice of the harmony singing section, scoring it as 7.2 points, for example.

The evaluation unit 21 further evaluates the singing voice of the call-and-response section based on the evaluation results of the first call-and-response section to the seventh call-and-response section, and as an example, averages the scores for the first call-and-response section to the seventh call-and-response section to give a score of 5.4 points, as shown in FIG. 6. The evaluation unit 21 then evaluates the karaoke singing of duet song X1 based on the evaluation results of singer U1's singing voice, singer U2's singing voice, the evaluation results of the harmony singing section, and the evaluation results of the call-and-response section, and as an example, gives a score of 85.1 points by adding up the scores.

Then, the presentation unit 22 displays and presents the evaluation result of the karaoke singing of the duet song X1 by the evaluation unit 21 on the display device 6 and the remote control device 7 (step S6).

As described above, according to the first embodiment of the present invention, the karaoke device 1 is a karaoke device 1 for performing karaoke of a duet song having a call-and-response section in which two singers alternately sing karaoke, and includes a storage unit 11 for storing, in association with the duet song, first reference data used to evaluate the singing voices of the two singers singing the duet song and second reference data indicating information on the opening and closing vocals of the two original singers in the call-and-response section of the duet song, an acquisition unit 20 for acquiring information on the opening and closing vocals of the two singers in the call-and-response section based on a comparison between the singing voices of the two singers in the call-and-response section and the first reference data, and an evaluation unit 21 for comparing the information on the opening and closing vocals of the two singers in the call-and-response section with the second reference data and evaluating the karaoke singing of the duet song by the two singers based on the comparison result.

In this way, when a duet song is sung using the karaoke device 1, the karaoke singing of each singing part can be evaluated, and the karaoke singing of the back-and-forth singing section in which the singers alternately sing different melodies can be evaluated. Therefore, for example, if two singers perform karaoke well in the back-and-forth singing section, the score can take that into consideration, which can liven up the atmosphere and encourage the singing of duet songs.
[Modification]

In addition, in the karaoke device 1 of the first embodiment, as a modified example, the presentation unit 22 may be configured to present advice regarding fast singing and slow singing in the exchange singing section of the duet song to at least one of the two singers who sang the duet song based on the evaluation result of the karaoke singing of the duet song by the evaluation unit 21.

The presentation unit 22 may present, as advice, that the opening and closing stanzas of the singer's karaoke singing in the call-and-response section are different from those of the original singer, and may present recommended opening and closing stanzas. The presentation unit 22 may also present, as advice, that the opening and closing stanzas of the singer's karaoke singing in the call-and-response section are highly rated because they are the same as those of the original singer. The presentation unit 22 may present the advice by displaying a message on the display device 6 or the remote control device 7, or may present the advice by creating a sound that is uttered through the speaker 5.

For example, in the first singing section of duet song X1, the second reference data for the first singing part is "fast singing", while the information on fast singing and build-up singing for singer U1 is "neither". On the other hand, in the first singing section, the second reference data for the second singing part is "build-up singing", while the information on fast singing and build-up singing for singer U2 is "build-up singing".

In this case, the presentation unit 22 creates and presents advice such as "U1 is singing '(the first singing part of the first dialogue section)' at the timing written in the music score. Singer A1 is singing in a rush. Singing in a rush will better match the atmosphere of song X1," or "U2 is singing '(the second singing part of the first dialogue section)' with a build-up. Singer A2 is singing in a rush. The atmosphere of song X1 comes out well."

As a result, based on the evaluation results of the karaoke singing in the duet song's duet singing section, the presentation unit 22 not only presents the karaoke singing evaluation results, but also provides advice on fast singing and building up singing in the duet singing section, thereby supporting the singer in singing duets and encouraging duet singing.

In the above first embodiment, an example was described in which the second reference data indicating information on the early singing and the build-up singing of the duet song's original singer's part in the duet song and the information on the early singing and the build-up singing of the duet song's part acquired from the singer's karaoke singing by the acquisition unit 20 are classified into three levels, "early singing," "build-up singing," and "neither," with respect to vocalization standards based on the first reference data; however, the present invention is not limited to this example.

In another example, with regard to the information on early singing and delayed singing in the call and dialogue section set as the second reference data, or the information on early singing and delayed singing in the call and dialogue section obtained from the singer singing karaoke, "early singing", which is early timing, may be further classified into multiple stages, and "delayed singing" may be further classified into multiple stages, relative to the vocalization standard based on the first reference data.

In this case, a number of different running time thresholds are preset using the vocalization timing of the call-and-response section of the first reference data as the vocalization standard, and a number of different retention time thresholds are also preset. The vocalization timing of the call-and-response section by the original singer's singing or the singer's karaoke singing is expressed as a time amount relative to the vocalization standard, and this time amount is compared with the running time threshold or the retention time threshold to determine the degree of "rushing singing" or the degree of "retention singing". As an example, the information on the rushing singing and retention singing in the call-and-response section is classified as one of "rushing singing 5", "rushing singing 4", "rushing singing 3", "rushing singing 2", "rushing singing 1", "0 (neither)", "retention singing 1", "retention singing 2", "retention singing 3", "retention singing 4", and "retention singing 5". In addition, by setting upper limits for the running time threshold and the accumulation time threshold, if the vocalization timing exceeds the upper limits relative to the vocalization standard, the singing will not be judged or evaluated as "running singing" or "accumulating singing", and points will be deducted as a singing error in the scoring information for each singing part.

Second Embodiment
A karaoke device 1 according to a second embodiment of the present invention will now be described. Fig. 7 is a block diagram showing the configuration of the karaoke device 1 according to the second embodiment. In the description of the second embodiment, the same components as those in the first embodiment are given the same reference numerals as those in the first embodiment, and the description thereof will be omitted.

In addition to the configuration of the first embodiment, the karaoke device 1 of the second embodiment is configured to store the evaluation results of the karaoke singing of a duet song, and correct the timing of switching the display of the lyric subtitles based on the evaluation results during the next karaoke performance of the duet song.

In the second embodiment, the control unit 10, like the first embodiment, operates as an acquisition unit 20, an evaluation unit 21, and a presentation unit 22 by executing a program stored in the memory unit 11, and further operates as a first correction unit 30.

In the second embodiment, after the karaoke performance of a duet song sung by two singers is completed, the storage unit 11 stores the evaluation result of the karaoke singing of the duet song evaluated by the evaluation unit 21 in association with the two singers (user identification information) and the duet song (song identification information). The user identification information of the two singers is information input when logging in to the karaoke device 1, and the song identification information of the duet song is information input when selecting (reserving) the duet song.

The first correction unit 30 is configured to correct the timing of switching between the unsung and sung parts of the lyrics caption (e.g., the timing of changing the color) based on the evaluation results of the karaoke singing of the duet song.

Figure 8 is a flow chart showing the operation of the karaoke device 1 of the second embodiment. First, when two singers are logged in to the karaoke device 1 and one of the two singers selects (reserves) a duet song (step S11), the first correction unit 30 checks whether the evaluation results of a previous karaoke performance of the duet song by the two singers are stored in the memory unit 11 based on the user identification information of the two singers and the song identification information of the duet song (step S12).

If such an evaluation result is stored in the memory unit 11 (step S12: YES), the first correction unit 30 corrects the timing of switching between the unsung and sung parts of the lyrics caption during the current karaoke performance of the duet song based on the evaluation result (step S13). At this time, if the early singing and delayed singing information of the past evaluation result differs from the early singing and delayed singing information indicated by the second reference data in a given singing exchange section, the first correction unit 30 corrects the timing of switching between the lyrics caption so that the vocal timing of the singer in the singing exchange section approaches the second reference data.

As a result, when the karaoke device 1 performs the karaoke of the selected (reserved) duet song (step S14), when displaying lyrics captions on the display device 6 based on the lyrics data of the duet song, the karaoke device 1 can display the lyrics captions with a display switching timing corrected based on the past evaluation results (step S15).

Specifically, when a past evaluation result of the karaoke singing of duet song X1 as shown in FIG. 5 is stored in the memory unit 11 for the second reference data as shown in FIG. 2, the first correction unit 30 refers to this evaluation result.

At this time, in the first singing part of the first inter-singing section, the information on the rushing singing and the build-up singing indicated by the second reference data is "running singing", whereas the information on the rushing singing and the build-up singing of the singer U1 is "neither". Therefore, in order to encourage the singer U1 to sing in a relatively rushing manner, the first correction unit 30 corrects the timing of the display switch of the lyrics caption of the first singing part of the first inter-singing section to be advanced by a predetermined time (for example, 50 msec). Also, in the second singing part of the first inter-singing section, the information on the rushing singing and the build-up singing indicated by the second reference data is "build-up singing", and the information on the rushing singing and the build-up singing of the singer U2 is also "build-up singing". In this way, when the information on the rushing singing and the build-up singing of both parties matches, the first correction unit 30 does not correct the timing of the display switch of the lyrics caption of the second singing part of the first inter-singing section.

In the second call-and-response section, the information on the early singing and the late singing indicated by the second reference data matches the information on the early singing and the late singing of singers U1 and U2 in both the first singing part and the second singing part, so the first correction unit 30 does not correct the timing of switching the display of the lyric caption for the first singing part and the second singing part in the second call-and-response section.

In the first singing part of the third singing section, the information on fast singing and build-up singing indicated by the second reference data is "build-up singing", whereas the information on fast singing and build-up singing of singer U1 is "neither". Therefore, in order to encourage singer U1 to sing with a relatively build-up style, the first correction unit 30 corrects the timing of switching the display of the lyrics caption for the first singing part of the third singing section to be delayed by a predetermined time. Also, in the second singing part of the third singing section, the information on fast singing and build-up singing indicated by the second reference data is "neither", whereas the information on fast singing and build-up singing of singer U2 is "fast singing". Therefore, in order to encourage singer U2 to sing with a relatively long pause, the first correction unit 30 corrects the timing of switching the display of the lyrics caption for the second singing part in the third singing exchange section by delaying it by a predetermined time.

In the first singing part of the fourth singing section, the information on the fast singing and the build-up singing indicated by the second reference data is "neither", whereas the information on the fast singing and the build-up singing of the singer U1 is "running singing". Therefore, the first correction unit 30 corrects the timing of the display switch of the lyrics caption of the first singing part of the fourth singing section to delay by a predetermined time in order to encourage the singer U1 to sing in a relatively build-up style. Also, in the second singing part of the fourth singing section, the information on the fast singing and the build-up singing indicated by the second reference data is "running singing", whereas the information on the fast singing and the build-up singing of the singer U2 is "neither". Therefore, the first correction unit 30 corrects the timing of the display switch of the lyrics caption of the second singing part of the fourth singing section to advance by a predetermined time in order to encourage the singer U2 to sing in a relatively build-up style.

In the first singing part of the fifth exchange singing section, the information on the rushing and building up singing indicated by the second reference data is "neither", whereas the information on the rushing and building up singing of singer U1 is "running up". Therefore, in order to encourage singer U1 to sing with a relatively building up tone, the first correction unit 30 corrects the timing of switching the display of the lyrics caption for the first singing part of the fifth exchange singing section to delay it by a predetermined time. In addition, in the second singing part of the fifth exchange singing section, the information on the rushing and building up singing indicated by the second reference data is "running up", and the information on the rushing and building up singing of singer U2 is also "running up". In this way, when the information on the rushing and building up singing of both singers matches, the first correction unit 30 does not correct the timing of switching the display of the lyrics caption for the second singing part of the fifth exchange singing section.

In the sixth call-and-response section, the information on the early singing and the build-up singing indicated by the second reference data matches the information on the early singing and the build-up singing of singers U1 and U2 in both the first singing part and the second singing part, so the first correction unit 30 does not correct the timing of switching the display of the lyric caption for the first singing part and the second singing part in the sixth call-and-response section.

In the first singing part of the seventh call-and-sing section, the information on the rushing and building up singing indicated by the second reference data is "building up singing", whereas the information on the rushing and building up singing of singer U1 is "neither". Therefore, in order to encourage singer U1 to sing in a relatively building up style, the first correction unit 30 corrects the timing of switching the display of the lyrics caption for the first singing part of the seventh call-and-sing section to be delayed by a predetermined time. Also, in the second singing part of the seventh call-and-sing section, the information on the rushing and building up singing indicated by the second reference data is "building up singing", whereas the information on the rushing and building up singing of singer U2 is "neither". Therefore, in order to encourage singer U2 to sing with a relatively long pause, the first correction unit 30 corrects the timing of switching the display of the lyrics caption for the second singing part in the seventh singing exchange section by delaying it by a predetermined time.

According to the second embodiment, the karaoke device 1 reflects the evaluation results of the duet song's previous singing of the duet song in the timing of switching the display of the lyrics subtitles when the duet song is currently being performed karaoke. This allows the singer's tendency to accelerate and build up when singing the duet song's karaoke section to be closer to the second reference data (i.e., the original singer).

In the second embodiment, the storage unit 11 that stores the evaluation results of karaoke singing of a duet song may always store the most recent evaluation results, or may update and store the evaluation results each time a higher evaluation result is obtained, or may allow the singer to select whether or not to store the evaluation results when they are obtained.

[Third embodiment]
A karaoke machine 1 according to a third embodiment of the present invention will now be described. Fig. 9 is a block diagram showing the configuration of the karaoke machine 1 according to the third embodiment. In the description of the third embodiment, the same components as those in the first embodiment are given the same reference numerals as those in the first embodiment, and the description thereof will be omitted.

The karaoke device 1 of the third embodiment is configured to store singing voices of duet songs sung as karaoke, and to correct the vocal timing of the stored singing voices in the call-and-response singing sections based on the second reference data, in addition to the configuration of the first embodiment.

In the third embodiment, the control unit 10, like the first embodiment, operates as an acquisition unit 20, an evaluation unit 21, and a presentation unit 22 by executing a program stored in the memory unit 11, and further operates as a second correction unit 40.

In the third embodiment, the storage unit 11 acquires a first audio signal and a second audio signal, each of which indicates the singing voices of two singers singing a duet song at karaoke, via the audio input/output unit 13, records them as recorded voices, and stores them in association with the two singers (user identification information) and the duet song (song identification information). The user identification information of the two singers is information input when logging in to the karaoke device 1, and the song identification information of the duet song is information input when the duet song is selected (reserved).

The second correction unit 40 is configured to correct the opening and closing vocals of the dialogue section in a recorded audio of karaoke singing of a duet song by two singers, based on the evaluation results of the karaoke singing of the duet song and the second reference data.

Figure 10 is a flow chart showing the operation of the karaoke device 1 of the third embodiment. First, when one of the two singers U1 and U2 operates the remote control device 7 to set the recording mode with correction and reserve the duet song X1, the karaoke main unit 2 starts the karaoke performance of the duet song X1 (step S21) and emits the accompaniment sound of the duet song X1 from the speaker 5 via the audio input/output unit 13.

When the karaoke performance of the duet song X1 begins, the karaoke main unit 2, being in the correction recording mode, starts recording the karaoke singing of the two singers U1 and U2 (step S22), acquires the first audio signal and the second audio signal representing the singing voices of the two singers U1 and U2 input from the first microphone 3 and the second microphone 4, respectively, via the audio input/output unit 13, records them as recorded audio, and stores them in the memory unit 11.

The karaoke main unit 2 performs the karaoke performance of the duet song X1 in the same manner as the evaluation operation (steps S2 to S6) in the flowchart of the first embodiment shown in FIG. 4. When the karaoke performance of the duet song X1 is completed, the karaoke singing of the duet song X1 by the two singers U1 and U2 is evaluated by the evaluation unit 21, and the evaluation result is presented by the presentation unit 22 (steps S23 to S27).

Next, when one of the two singers U1 and U2 operates the remote control device 7 to instruct playback of the recorded audio of the karaoke singing of the duet song X1, the second correction unit 40 acquires from the storage unit 11 the recorded audio corresponding to the karaoke singing of the duet song X1 by the two singers U1 and U2. Since the second correction unit 40 is in the correction recording mode, it corrects the initial singing and the build-up singing in the call-and-response section of the acquired recorded audio based on the evaluation result of the karaoke singing that is the basis of the recorded audio and the second reference data (step S28). The karaoke device 1 outputs the recorded audio, the timing of which has been corrected by the second correction unit 40, to the speaker 5 via the audio input/output unit 13 to emit the sound (step S29).

The second correction unit 40 may correct the recorded audio and store it in the storage unit 11 when the karaoke singing of the duet song is completed, or may store the recorded audio as is in the storage unit 11 when the karaoke singing of the duet song is completed and correct the recorded audio when it is played back.

Specifically, the second correction unit 40 corrects the recorded voice when the karaoke singing of the duet song X1 is evaluated as shown in FIG. 5, using the second reference data as shown in FIG. 2.

At this time, in the first singing part of the first call-and-sing section, the information on the rushing and building up singing indicated by the second reference data is "running singing", whereas the information on the rushing and building up singing of singer U1 is "neither". Therefore, the second correction unit 40 corrects the timing of vocalization to be earlier so that the recorded voice of singer U1 in the first singing part of the first call-and-sing section becomes "running singing" similar to the second reference data. Also, in the second singing part of the first call-and-sing section, the information on the rushing and building up singing indicated by the second reference data is "building up singing", and the information on the rushing and building up singing of singer U2 is also "building up singing". In this way, when the information on the rushing and building up singing of both parties matches, the second correction unit 40 does not correct the recorded voice of singer U2 in the second singing part of the first call-and-sing section.

In the second call-and-response section, the information on the early singing and the late singing indicated by the second reference data matches the information on the early singing and the late singing of singers U1 and U2 in both the first singing part and the second singing part, so the second correction unit 40 does not correct the recorded voices of singers U1 and U2 in the first singing part and the second singing part of the second call-and-response section.

In the first singing part of the third call-and-sing section, the information on the rushing and building up singing indicated by the second reference data is "building up singing", whereas the information on the rushing and building up singing of singer U1 is "neither". Therefore, the second correction unit 40 corrects by delaying the vocalization timing so that the recorded voice of singer U1 becomes "building up singing" similar to the second reference data in the first singing part of the third call-and-sing section. Also, in the second singing part of the third call-and-sing section, the information on the rushing and building up singing indicated by the second reference data is "neither", whereas the information on the rushing and building up singing of singer U2 is "running up singing". Therefore, the second correction unit 40 corrects by delaying the vocalization timing so that the recorded voice of singer U2 becomes "neither" similar to the second reference data in the second singing part of the third call-and-sing section.

In the first singing part of the fourth singing section, the information on the rushing and building up singing indicated by the second reference data is "neither", whereas the information on the rushing and building up singing of singer U1 is "running singing". Therefore, the second correction unit 40 corrects by delaying the vocalization timing so that the recorded voice of singer U1 in the first singing part of the fourth singing section becomes "neither" like the second reference data. Also, in the second singing part of the fourth singing section, the information on the rushing and building up singing indicated by the second reference data is "running singing", whereas the information on the rushing and building up singing of singer U2 is "neither". Therefore, the second correction unit 40 corrects by advancing the vocalization timing so that the recorded voice of singer U2 in the second singing part of the fourth singing section becomes "running singing" like the second reference data.

In the first singing part of the fifth call-and-sing section, the information on the rushing and building up singing indicated by the second reference data is "neither", whereas the information on the rushing and building up singing of singer U1 is "running singing". Therefore, the second correction unit 40 corrects the timing of vocalization by delaying it so that the recorded voice of singer U1 in the first singing part of the fifth call-and-sing section becomes "neither" like the second reference data. Also, in the second singing part of the fifth call-and-sing section, the information on the rushing and building up singing indicated by the second reference data is "running singing", and the information on the rushing and building up singing of singer U2 is also "running singing". In this way, when the information on the rushing and building up singing of both singers matches, the second correction unit 40 does not correct the recorded voice of singer U2 in the second singing part of the fifth call-and-sing section.

In the sixth call-and-response section, the information on the early singing and the late singing indicated by the second reference data matches the information on the early singing and the late singing of singers U1 and U2 in both the first singing part and the second singing part, so the second correction unit 40 does not correct the recorded voices of singers U1 and U2 in the first singing part and the second singing part of the sixth call-and-response section.

In the first singing part of the seventh call-and-sing section, the information on the early singing and the build-up singing indicated by the second reference data is "build-up singing", whereas the information on the early singing and the build-up singing of singer U1 is "neither". Therefore, the second correction unit 40 corrects by delaying the vocalization timing so that the recorded voice of singer U1 becomes "build-up singing" similar to the second reference data in the first singing part of the seventh call-and-sing section. Also, in the second singing part of the seventh call-and-sing section, the information on the early singing and the build-up singing indicated by the second reference data is "build-up singing", whereas the information on the early singing and the build-up singing of singer U2 is "neither". Therefore, the second correction unit 40 corrects by delaying the vocalization timing so that the recorded voice of singer U2 becomes "build-up singing" similar to the second reference data in the second singing part of the seventh call-and-sing section.

According to the third embodiment, the karaoke device 1 corrects the recorded voice of the karaoke singing of a duet song based on the karaoke singing evaluation result and the second reference data, so that the tendency of the early singing and the delayed singing in the call-and-response section sounded by playing back the recorded voice can be made to be closer to the second reference data (i.e., the original singer). This allows the singer to refer to and learn more preferable early singing and delayed singing when singing a duet song, so the karaoke device 1 can encourage duet singing.

In the third embodiment, an example has been described in which the karaoke device 1 records and corrects the karaoke singing of a duet song when the recording mode with correction is set, and the corrected recorded singing can be played back, but the present invention is not limited to this example. In another example, when the recording mode without correction is set, the karaoke device 1 records the karaoke singing of a duet song as is, and when playing back the recorded singing, advice may be provided at points where the opening and closing parts of the call-and-response section differ from the second reference data (i.e., the original singer).

The present invention may be modified as appropriate without going against the gist or concept of the invention as can be read from the claims and the entire specification, and karaoke devices with such modifications are also included in the technical concept of the present invention.

Reference Signs List 1 Karaoke device 2 Karaoke main unit 3 First microphone 4 Second microphone 6 Display device 10 Control unit 11 Storage unit 20 Acquisition unit 21 Evaluation unit 22 Presentation unit 30 First correction unit 40 Second correction unit

Claims (4)

A karaoke apparatus for performing a karaoke performance of a duet song having a dialogue singing section in which two singers sing karaoke alternately,
a storage unit that stores, in association with the duet song, first reference data used for evaluating the singing voices of the two singers who sing the duet song as karaoke, and second reference data indicating information on the opening and closing vocal parts of the two original singers in the exchange singing section of the duet song;
an acquisition unit that acquires information on the early singing and the late singing of the two singers in the exchange singing section based on a comparison between the singing voices of the two singers in the exchange singing section and the first reference data;
an evaluation unit that compares information on the initial singing and the build-up singing of the two singers in the exchange singing section with the second reference data, and evaluates the karaoke singing of the duet song by the two singers based on the comparison result;
A karaoke device comprising: The karaoke device according to claim 1, further comprising a presentation unit that presents advice to at least one of the two singers regarding fast singing and slow singing in the singing exchange section based on the evaluation result by the evaluation unit. the storage unit stores the evaluation result by the evaluation unit in association with the two singers and the duet song;
3. The karaoke apparatus according to claim 1, further comprising a first correction unit which, when a karaoke performance of the duet song is reserved by either of the two singers, corrects the timing of switching the display of lyric subtitles in the duet song's duet singing section based on the evaluation results corresponding to the two singers and the duet song. the storage unit records and stores the singing voices of the two singers who sing the duet song as karaoke;
The karaoke apparatus according to any one of claims 1 to 3, further comprising a second correction section for correcting the accelerated singing and the accumulated singing in the interlocutor singing section of the recorded singing voices of the two singers based on the second reference data.

Images