JP6864571B2 - Lyrics telop color change data creation device - Google Patents

Description

The present invention relates to a lyrics telop color change data creation device.

In the karaoke device, the lyrics of the karaoke song to be sung can be superimposed and displayed on the background image for singing support.

Furthermore, in a karaoke device, the color of the displayed lyrics (lyrics telop) may be sequentially changed (color change) as the karaoke song is played so that the singer can easily grasp the lyrics to be sung at a certain point in time. It is done.

The color change of the lyrics telop is performed by using the data for controlling the color change of the lyrics (hereinafter, "lyric telop color change data") created in advance.

Such lyrics telop color change data can be used for the work (authoring) in which an operator changes the lyrics by using an input means (for example, a mouse of a personal computer) while listening to the played music and watching the corresponding lyrics. It is common to create based on.

For example, Patent Document 1 discloses a digital karaoke software editing device that can input data of lyrics color change timing with a simple operation. Further, Patent Document 2 discloses a technique of displaying lyrics on a touch panel and generating and editing display control data indicating a color change timing based on the acquired touch information in order to facilitate manual work by an operator. Has been done.

Japanese Unexamined Patent Publication No. 10-268878 Japanese Unexamined Patent Publication No. 2015-176379

However, authoring requires a very high level of skill and is difficult to carry out without an expert. Further, even if the techniques disclosed in Patent Documents 1 and 2 are used, it is difficult to easily create the lyrics telop color change data because the manual work by the operator is required.

An object of the present invention is to provide a lyrics telop color change data creating device capable of easily generating data for controlling color change of lyrics displayed when playing a karaoke music.

The main invention for achieving the above object is based on a singing voice signal obtained by singing a certain karaoke song, uttered text information corresponding to a plurality of narrated pseudonyms, and uttering each of the plurality of pseudonyms. The lyrics of the karaoke song based on the acquisition unit that acquires the vocal data including the pseudonym vocal timing information corresponding to the timing, the acquired vocal data, and the lyrics text data corresponding to the lyrics of the karaoke song. A calculation unit that calculates color change information corresponding to the time required for color change or the speed of color change when is displayed for each character included in the lyrics, and a setting unit that sets the time to start the color change. , To control the color change of the lyrics displayed based on the lyrics text data at the time of playing the certain karaoke song by associating the calculated color change information with the set time to start the color change. It is a lyrics telop color change data creation device having a lyric telop color change data creation unit and a lyrics telop color change data creation unit.
Other features of the present invention will be clarified by the description of the specification and drawings described later.

According to the present invention, it is possible to easily generate data for controlling the color change of the lyrics displayed when the karaoke music is played.

It is a figure which shows the hardware configuration example of the karaoke apparatus which concerns on 1st Embodiment. It is a figure which shows the software structure example of the karaoke main body which concerns on 1st Embodiment. It is a figure which shows the example of the singing voice and utterance text information which concerns on 1st Embodiment. It is a figure which shows the example of the utterance text information and the pseudonym utterance timing information which concerns on 1st Embodiment. It is a figure which shows the example of the utterance text information and lyrics text data which concerns on 1st Embodiment. It is a figure which shows the example of the lyrics text data and the color change information which concerns on 1st Embodiment. It is a figure which shows the example of the lyrics text data which concerns on 1st Embodiment, and the time to start the color change. It is a flowchart which shows the process of the karaoke apparatus which concerns on 1st Embodiment. It is a figure which shows the hardware configuration example of the karaoke apparatus which concerns on 2nd Embodiment. It is a figure which shows the software structure example of the karaoke main body which concerns on 2nd Embodiment. It is a flowchart which shows the process of the karaoke apparatus which concerns on 2nd Embodiment.

In addition to the above-mentioned main invention, at least the following matters will be clarified from the description of the specification and the drawings described later.

That is, it becomes clear that the lyrics telop color change data creating device sets a time in which the setting unit sets a time earlier than the timing at which the pseudonym is uttered by a predetermined value as a time for starting the color change. Further, the predetermined value is preferably in the range of about 200 msec to about 300 msec. With such a device, it is possible to create lyrics telop color change data suitable for actual karaoke singing.

Further, when the setting unit sets a time earlier than the timing at which the pseudonym is uttered as the time to start the color change, if the performance tempo of the certain karaoke music is faster than the predetermined tempo, the pseudonym If the time earlier than the timing at which the karaoke song is uttered is set longer than the predetermined value within the range of approximately 200 msec to approximately 300 msec, and the performance tempo of the karaoke song is slower than the predetermined tempo, the timing at which the pseudonym is uttered is set. A device for creating a music telop color change data that sets an early time shorter than a predetermined value within a range of about 200 msec to about 300 msec becomes clear. According to such a device, the time for starting the color change can be set according to the tempo of the music.

Further, based on the display control unit that displays the lyrics based on the lyrics text data according to the performance of the certain karaoke song, and the line-of-sight detection result of the singer who visually recognizes a certain character included in the displayed lyrics. It has a specific unit that specifies the timing at which the singer visually recognizes the certain character, and the setting unit starts the color change of the specific character based on the timing at which the specified character is visually recognized. Lyrics telop color change data creation device that sets the time to do. According to such a device, the line-of-sight detection result can be used to create lyrics telop color change data suitable for actual karaoke singing.

<First Embodiment>
The karaoke device 1 according to the first embodiment will be described with reference to FIGS. 1 to 4.

== Karaoke device ==
The karaoke device 1 is a device for performing karaoke performance and singing karaoke by a singer. The karaoke device 1 registers the karaoke songs selected by the singer in the reservation queue, and performs karaoke performances in order. As shown in FIG. 1, the karaoke device 1 includes a karaoke body 10, a speaker 20, a display device 30, a microphone 40, and a remote control device 50.

The speaker 20 is configured to emit sound based on the sound emitted signal from the karaoke main body 10. The display device 30 is configured to display an image or an image on the screen based on the signal from the karaoke main body 10. The microphone 40 is configured to convert the singing voice of the singer (input voice to the microphone 40) into an analog singing voice signal and input it to the karaoke main body 10. The remote control device 50 is a device for performing various operations on the karaoke main body 10. The singer can use the remote control device 50 to select (reserve) a karaoke song that he / she wishes to sing. On the display screen of the remote controller device 50, icons and the like for inputting instructions for various operations are displayed.

The karaoke main body 10 performs various controls related to karaoke singing, such as performance control of selected karaoke songs, display control of lyrics and background images, and processing of singing voice signals input through a microphone 40. As shown in FIG. 1, the karaoke main body 10 includes a control unit 11, a communication unit 12, a storage unit 13, an audio processing unit 14, a display processing unit 15, and an operation unit 16. Each configuration is connected to bus B via an interface (not shown).

The control unit 11 includes a CPU 11a and a memory 11b. The CPU 11a realizes various control functions by executing an operation program stored in the memory 11b. The memory 11b is a storage device that stores a program to be executed in the CPU 11a and temporarily stores various information when the program is executed.

The communication unit 12 provides an interface for connecting the karaoke body 10 to the communication line via a router (not shown).

The storage unit 13 is a large-capacity storage device that stores various types of data, such as a hard disk drive. The storage unit 13 stores a plurality of music data for performing karaoke performance by the karaoke device 1.

The music data is given a music ID for identifying each karaoke music. The music data includes accompaniment data, reference data, and the like. The accompaniment data is the data that is the source of the karaoke performance sound. The accompaniment data includes information indicating the tempo when playing karaoke. A predetermined value is set for the tempo for each song. Reference data is data used as a reference when scoring karaoke singing by a singer.

Further, the storage unit 13 includes lyrics telop data for displaying lyrics corresponding to each karaoke song on the display device 30 or the like, background image data such as a background image displayed on the display device 30 or the like during karaoke performance, and each song. The performance time data indicating the karaoke performance time and the attribute information of the music (information about the music such as singer name, lyrics / composer name, genre, etc.) are stored.

The lyrics telop data includes lyrics text data and lyrics telop color change data.

The lyrics text data is data corresponding to lyrics (characters included in the lyrics) displayed on the display device 30 or the like. Specifically, the lyrics text data is data for sequentially displaying lyrics for each singing section (for example, "Why seagulls cry", "Seagulls in the sea", ...). That is, the lyrics text data includes timing information indicating the timing of starting the display and the timing of erasing the display for each singing section, and layout information indicating the position of the text in the display screen. In addition, when the lyric text contains Chinese characters or a foreign language (for example, "squeal", "sea"), the lyric text data includes ruby data (for example, "sea" for "sea").

The lyrics telop color change data is data for controlling the color change of the lyrics displayed based on the lyrics text data when the karaoke music is played. The color change means, for example, changing the color of the lyrics displayed based on the lyrics text data (for example, white) sequentially with the performance of the karaoke music (for example, from white to red). The lyrics telop color change data includes the time required for the color change when the lyrics of the karaoke song are displayed, the color change information corresponding to the speed of the color change, and the time for starting the color change (details will be described later).

Based on the control of the control unit 11, the sound processing unit 14 controls the performance of the karaoke music and processes the singing voice signal input through the microphone 40. The display processing unit 15 performs processing related to various displays in the display device 30 and the remote control device 50 based on the control of the control unit 11. For example, the display processing unit 15 controls the display device 30 to display an image in which lyrics telops and various icons are superimposed on a background image when a karaoke song is played. Alternatively, the display processing unit 15 displays various icons for operation input on the display screen of the remote controller device 50. The operation unit 16 includes a panel switch, a remote control reception circuit, and the like, and sends operation signals such as a music selection signal and a performance stop signal to the control unit 11 in response to the operation of the panel switch of the karaoke device 1 or the remote control device 50 by the singer. Output. The control unit 11 detects the operation signal from the operation unit 16 and executes the corresponding process.

Here, the karaoke device 1 according to the present embodiment corresponds to a "lyrics telop color change data creation device" that creates lyrics telop color change data. The lyrics telop color change data is created by an arbitrary karaoke device, for example, when distributing a new karaoke song. In addition, the lyrics telop color change data is a singing voice signal obtained by singing a professional singer who actually sings a karaoke song or a person who has the same singing ability as a professional singer (hereinafter, "model singer"). Created using. By using the singing voice signal of the model singer, it is possible to create lyrics telop color change data suitable for the karaoke music. The singing of the karaoke song by the model singer is performed at the stage where the lyrics telop color change data has not been created yet. Therefore, the model singer sings a karaoke song while looking at a score or a lyrics card containing lyrics, for example.

In the following example, the processing for two sections of "Why the seagull sings" and "The seagull is in the sea" out of the plurality of singing sections of the karaoke song X will be described, but the lyrics telop color change data of the karaoke song X will be described. At the time of creation, the same processing is performed for all the singing sections included in the karaoke music X. Karaoke music X is an example of "a certain karaoke music".

It is also possible to create lyrics telop color change data on another computer (for example, a personal computer owned by an operator) using the singing voice signal input by the microphone 40. In this case, another computer corresponds to the "lyric telop color change data creation device".

(Software configuration)
FIG. 2 is a diagram showing a software configuration example of the karaoke main body 10. The karaoke body 10 includes an acquisition unit 100, a calculation unit 200, a setting unit 300, and a creation unit 400. The acquisition unit 100, the calculation unit 200, the setting unit 300, and the creation unit 400 are realized by the CPU 11a executing a program stored in the memory 11b.

[Acquisition department]
The acquisition unit 100 acquires vocalization data based on the singing voice signal obtained by singing the karaoke music X. The singing audio signal is obtained by the singing of the model singer as described above. The utterance data includes utterance text information and kana utterance timing information.

The vocalized text information corresponds to a plurality of kana (texts) actually uttered in the singing of the model singer. The acquisition unit 100 converts the singing voice signal into text by performing a known voice recognition process on the singing voice signal. For example, the acquisition unit 100 processes the singing voice signals of the lyrics "Why the seagull sings" and "The seagull is in the sea" obtained by singing the karaoke song X by the model singer, and texts those lyrics. Acquire the converted information (spoken text information) (see FIG. 3A). In the example of FIG. 3A, the utterance text information is converted into text by a pseudonym in Roman characters.

The kana utterance timing information corresponds to the timing at which each of the plurality of kana included in the utterance text information is uttered. The acquisition unit 100 acquires the timing at which each pseudonym is uttered by performing a known waveform analysis process on the singing audio signal. In the above example, the acquisition unit 100 acquires the timing at which each pseudonym is uttered by processing the singing audio signals obtained by singing the lyrics "Why the seagull crows" and "The seagull is in the sea" (Fig. See 3B). In the example of FIG. 3B, the kana utterance timing information of each kana corresponds to the elapsed time from the time (0.0 sec) of singing the first kana (“ka” in FIG. 3A) to the singing of a certain kana. The tempo of the singing at this time is BPM = 60 (the speed of 60 beats in quarter notes per minute).

The acquisition unit 100 outputs the acquired utterance data to the calculation unit 200.

[Calculation unit]
The calculation unit 200 calculates color change information for each character included in the lyrics based on the acquired vocal data and the lyrics text data corresponding to the lyrics of the karaoke song X.

As the lyrics text data, those stored in the storage unit 13 as described above are used. Alternatively, lyrics text data may be acquired from a server device (not shown). The color change information corresponds to the time required for color change or the speed of color change when the lyrics of the karaoke song X are displayed.

As a specific example, the calculation unit 200 reads the lyrics text data of the karaoke music X from the storage unit 13 when calculating the color change information. The karaoke music X corresponds to the music sung when the singing voice signal that is the source of the acquired vocal data is obtained. The calculation unit 200 collates the utterance text information included in the utterance data with the lyrics text data of the karaoke music X read from the storage unit 13. At this time, when the lyrics include kanji or a foreign language, the calculation unit 200 collates the ruby data included in the lyrics text data with the uttered text information. For example, the karaoke song X includes the character "sea" in the lyrics. In this case, the calculation unit 200 collates the ruby "sea" of the "sea" with the uttered text information (see FIG. 3C). Similarly, the particle "ha" included in the lyrics is collated with the utterance text information.

Next, the calculation unit 200 obtains the time required for color change of each character based on the collation result of the utterance text information and the lyrics text data and the kana utterance timing information included in the utterance data.

For example, in the examples of FIGS. 3B and 3C, the character "ku" indicates the elapsed time indicated by the kana utterance timing information of 5.5 sec, and the kana utterance timing information of the next character "no" indicates. The elapsed time is 6.0 sec. Therefore, it takes 0.5 sec for the character "ku" to sing the next character. Therefore, the calculation unit 200 calculates "0.5 sec" as the color change information of the character "ku". On the other hand, the elapsed time indicated by the kana utterance timing information for the character "umi (u, mi)" is 10.0 sec for "u" and 11.0 sec for "mi". Further, the elapsed time indicated by the kana utterance timing information of the character "ni (ni)" next to "umi (u, mi)" is 12.0 sec. Therefore, the calculation unit 200 calculates "2.0 sec", which is the sum of the elapsed times of the two kana characters, as the color change information of the characters "sea".

In this way, the calculation unit 200 calculates the color change information for each character included in the lyrics text data (see FIG. 3D). The calculation unit 200 outputs the calculated color change information to the creation unit 400. The calculation unit 200 may calculate the speed of color change for each character based on the calculated time for each character and the tempo of the karaoke music X, and use the speed as the color change information.

[Settings]
The setting unit 300 sets the time for starting the color change. For the time to start the color change, for example, the kana utterance timing information can be used as it is. For example, in the above example, the setting unit 300 sets the time to start the color change of the first character "ka" to "0.0sec" based on the kana vocalization timing information, and changes the color of the next character "mo". After the start time is "1.0 sec", the time to start the color change of the character "me" is "2.0 sec" ... The time to start the color change of the character "sea" is "10.0 sec" It can be set later (see FIG. 3E).

The time for starting the color change may be set for each character, for each singing section, or for each phrase. When setting is performed for each singing section or phrase, the setting unit 300 needs to specify the first character of the singing section or phrase and set a time for starting color change based on the character. The first character of the singing section or phrase may be specified in advance by including information that can identify the first character in the music data, or may be specified each time by using a known automatic translation function. The setting unit 300 outputs the time for starting the set color change to the creating unit 400.

On the other hand, in actual karaoke singing, it is often the case that the timing of singing is grasped while referring to the color change of the characters displayed on the display device 30. Therefore, it is preferable that the character color change is started a little before the actual singing timing. Therefore, the setting unit 300 can set the time to start the color change so that the color change of the characters can be started a little before the actual singing timing.

Specifically, the setting unit 300 sets a time earlier than the timing at which the model singer utters the pseudonym by a predetermined value as the time to start the color change.

The predetermined value is a threshold value set in consideration of the ease of singing karaoke (easiness of visually recognizing the characters to be changed in color) and the balance of the entire performance of the karaoke song. Specifically, the predetermined value is preferably in the range of 200 msec to 300 msec.

For example, in the example of FIG. 3B, the timing at which the pseudonym of "no" is uttered is 6.0 sec. If the color change is started at this timing, a general singer may not be able to sing along with the karaoke performance as a result of the delay in the actual vocalization. Therefore, the setting unit 300 sets the time "5.75 sec", which is 250 msec earlier than 6.0 sec, as the time to start the color change of the character "(no)".

Alternatively, it is also possible to set the time to start the color change in consideration of the performance tempo of each karaoke song. Generally, when the performance tempo of a karaoke song is fast, the speed of color change is felt to be slow, and when the performance tempo is slow, the speed of color change is felt to be fast. May be difficult to refer to the color change. Therefore, when setting a time earlier than the timing when the pseudonym is uttered as the time to start color change, for a karaoke song with a fast performance tempo, a time earlier than the timing when the pseudonym is uttered is set longer and the performance tempo is slower. For karaoke songs, setting a shorter time than the timing at which the pseudonym is uttered makes it easier for the singer to sing karaoke while referring to the color change.

Therefore, when the setting unit 300 sets a time earlier than the timing at which the pseudonym is uttered as the time for starting the color change, if the performance tempo of a certain karaoke song is faster than the predetermined tempo, the timing at which the pseudonym is uttered. If the earlier time is set longer than the predetermined value within the range of approximately 200 msec to approximately 300 msec and the performance tempo of a certain karaoke song is slower than the predetermined tempo, the time earlier than the timing at which the pseudonym is uttered is approximately 200 msec. Set shorter than a predetermined value within the range of about 300 msec.

For example, an example in which a predetermined tempo is BPM = 100 and a predetermined value is 250 msec will be described. The setting unit 300 compares the performance tempo of a certain karaoke song with a predetermined tempo. When the playing tempo is slower than the predetermined tempo (for example, BPM = 60), the color change start time setting unit 300 sets the time earlier than the timing at which the pseudonym is uttered to be shorter than the predetermined value (for example, 200 msec). On the contrary, when the playing tempo is faster than the predetermined tempo (for example, BPM = 140), the setting unit 300 sets a time earlier than the timing at which the pseudonym is uttered longer than the predetermined value (for example, 300 msec).

The predetermined tempo and the predetermined value are determined according to the display conditions such as the size of the characters to be color-changed, the type of font, the brightness of the display, the age group of the singer assumed for each karaoke song, and the like. It is possible to set the value. However, in any case, from the viewpoint of ease of singing, it is preferable to set the predetermined value within the range of 200 to 300 msec.

[Creation department]
The creation unit 400 creates the lyrics telop color change data by associating the calculated color change information with the set time to start the color change.

The lyrics telop color change data is data for controlling the color change of the lyrics displayed based on the lyrics text data when playing a certain karaoke song.

The creation unit 400 creates lyrics telop color change data for each karaoke song by associating the color change information output from the calculation unit 200 with the time for starting the color change output from the setting unit 300. The creation unit 400 can store the created lyrics telop color change data as lyrics telop data by associating it with the lyrics text data (data used when calculating the color change information) stored in the storage unit 13. it can.

Alternatively, the creation unit 400 may transmit the lyrics telop data in which the lyrics text data and the created lyrics telop color change data are associated with each other to the server device. The server device can distribute the received lyrics telop data to another karaoke device.

== About the operation of the karaoke device 1 ==
Next, a specific example of the operation of the karaoke device 1 in the present embodiment will be described with reference to FIG. FIG. 4 is a flowchart showing an operation example of the karaoke device 1. Here, an example of creating lyrics telop color change data of karaoke music X will be described.

The karaoke device 1 plays the karaoke music X based on the accompaniment data. The model singer sings along with the performance of the karaoke song X. The karaoke device 1 acquires the singing audio signal of the model singer (acquisition of the singing audio signal. Step 10).

The acquisition unit 100 includes utterance text information corresponding to a plurality of uttered pseudonyms based on the singing voice signal obtained in step 10, and pseudonym utterance timing information corresponding to the timing of uttering each of the plurality of pseudonyms. Acquire vocalization data (acquisition of vocalization data. Step 11).

The calculation unit 200 has color change information corresponding to the time required for color change when displaying the lyrics of the karaoke song X, based on the vocal data acquired in step 11 and the lyrics text data corresponding to the lyrics of the karaoke song X. Is calculated for each character (calculation of color change information. Step 12).

The setting unit 300 sets the time to start the color change based on the kana utterance timing information (setting the time to start the color change. Step 13).

The creation unit 400 associates the color change information calculated in step 12 with the time for starting the color change set in step 13, so that the lyrics displayed based on the lyrics text data at the time of playing the karaoke song X Create lyrics telop color change data for controlling color change (creation of lyrics telop color change data. Step 14).

Further, the creation unit 400 creates the lyrics telop data of the karaoke song X by associating the lyrics text data with the lyrics telop color change data created in step 14 (creation of the lyrics telop data. Step 15).

As described above, the karaoke device 1 according to the present embodiment has an acquisition unit 100, a calculation unit 200, a setting unit 300, and a creation unit 400. The acquisition unit 100 acquires utterance data including utterance text information and kana utterance timing information. The calculation unit 200 calculates color change information corresponding to the time required for color change when displaying the lyrics of the karaoke song for each character included in the lyrics, based on the utterance data and the lyrics text data. The setting unit 300 sets the time to start the color change based on the kana utterance timing information. The creation unit 400 creates the lyrics telop color change data by associating the calculated color change information with the set time to start the color change.

According to such a karaoke device 1, the lyrics telop color change data is automatically generated only by acquiring the singing voice signal sung by the model singer for the karaoke music for which the lyrics telop color change data has not been created yet. Can be created in. Therefore, it is not necessary for an expert to manually create the lyrics telop color change data as in the conventional case. That is, according to the karaoke device 1 according to the present embodiment, it is possible to easily generate data for controlling the color change of the lyrics displayed when the karaoke music is played.

<Second Embodiment>
Next, the karaoke device 1 according to the second embodiment will be described with reference to FIGS. 5 to 7. In the present embodiment, a configuration for setting a time for starting color change will be described by using a known line-of-sight detection technique. A detailed description of the same configuration as that of the first embodiment will be omitted.

== Karaoke device ==
As shown in FIG. 5, the karaoke device 1 according to the present embodiment includes a karaoke main body 10, a speaker 20, a display device 30, a microphone 40, a remote control device 50, and a line-of-sight detector 60.

The line-of-sight detector 60 is a device that detects the line of sight of a singer who sings a karaoke song. The line-of-sight detector 60 includes a known non-contact small line-of-sight detection system (for example, small line-of-sight detection system "EyeExpert (registered trademark)", Fujitsu Limited), a goggle-type eye tracking system (mobile eye tracking system "ViewPoint"), and the like. (Creact Co., Ltd.) can be used. When singing a karaoke song, the singer looks at the characters that he / she intends to sing from the displayed lyrics. The line-of-sight detector 60 detects the line of sight of such a singer. The singer in the present embodiment is the model singer described in the first embodiment. Further, the line-of-sight detector 60 may be provided independently of the karaoke device 1. In this case, the line-of-sight detector 60 transmits the detection result to the karaoke device 1. Further, for example, when a small line-of-sight detection system is used as the line-of-sight detector 60, a light source and an optical sensor for the system are arranged in the display device 30 as in a known system.

(Software configuration)
FIG. 6 is a diagram showing a software configuration example of the karaoke main body 10. The karaoke body 10 includes an acquisition unit 100, a calculation unit 200, a setting unit 300, a creation unit 400, a display control unit 500, and a specific unit 600. The acquisition unit 100, the calculation unit 200, the setting unit 300, the display control unit 500, and the specific unit 600 are realized by the CPU 11a executing a program stored in the memory 11b.

[Display control unit]
The display control unit 500 displays the lyrics based on the lyrics text data according to the performance of a certain karaoke song.

The lyrics text data is created separately from the lyrics telop color change data. Therefore, even if the lyrics telop color change data has not been created yet, the lyrics based on the lyrics text data can be displayed according to the performance of the karaoke music. Therefore, the singer can sing karaoke while referring to the displayed lyrics. However, the color of the displayed lyrics is not changed.

For example, when creating lyrics telop color change data for karaoke music X, the display control unit 500 reads the lyrics text data of karaoke music X from the storage unit 13 and sings the lyrics for each singing section (in accordance with the performance of karaoke music X). For example, "Why the seagull sings", "The seagull is in the sea", ...) are displayed in sequence. The display control unit 500 switches the display of the lyrics based on the timing information included in the lyrics text data.

[Specific part]
The identification unit 600 specifies the timing at which the singer visually recognizes the certain character based on the line-of-sight detection result of the singer who visually recognizes the certain character included in the displayed lyrics.

As described above, when the display control unit 500 displays the lyrics of the karaoke music X on the display device 30 or the like, the singer can refer to the lyrics. On the other hand, since the color is not changed, the singer will look at the character that he / she wants to sing. The identification unit 600 identifies the timing at which the singer turns his / her line of sight to a certain character (that is, the timing at which the certain character is visually recognized) based on the detection result of the line-of-sight detector 60.

As a specific example, when the lyrics "seagull is in the sea" are displayed on the display device 30, the singer visually recognizes the character "ka" to be sung and then sings the character "ka". In this case, the line-of-sight detector 60 detects the line of sight for the character "ka". The line-of-sight detector 60 outputs a signal to the effect that the line-of-sight has been detected to the specific unit 600. The identification unit 600 specifies the time when the signal is received (the elapsed time since the first character is sung) as the timing when the character "ka" is visually recognized. The line-of-sight detection may be performed for each character included in the lyrics, or for each singing section or each phrase (first character).

[Settings]
The setting unit 300 according to the present embodiment sets the time for starting the color change of the specified character based on the timing at which the specified character is visually recognized.

For example, in the above example, it is assumed that the timing at which the character "ka" is visually recognized is 7.75 seconds after the first character is sung. In this case, the color change start time setting unit 300 sets the time as the time to start the color change of the character "ka".

== About the operation of the karaoke device 1 ==
Next, a specific example of the operation of the karaoke device 1 in the present embodiment will be described with reference to FIG. 7. FIG. 7 is a flowchart showing an operation example of the karaoke device 1. Here, an example of creating lyrics telop color change data of karaoke music X will be described.

The karaoke device 1 plays the karaoke music X based on the accompaniment data. The display control unit 500 causes the display device 30 to display the lyrics of the karaoke music X according to the accompaniment data of the karaoke music X (displays the lyrics according to the performance. Step 20). The model singer sings along with the performance of the karaoke song X while referring to the displayed lyrics. The karaoke device 1 acquires the singing audio signal of the model singer (acquisition of the singing audio signal. Step 21). Further, the line-of-sight detector 60 detects the line of sight of the model singer who refers to the displayed lyrics, and outputs the detection result to the setting unit 300 (line-of-sight detection, step 22).

The acquisition unit 100 includes utterance text information corresponding to a plurality of uttered pseudonyms based on the singing voice signal obtained in step 21, and pseudonym utterance timing information corresponding to the timing of uttering each of the plurality of pseudonyms. Acquire vocalization data (acquisition of vocalization data. Step 23).

The calculation unit 200 is based on the vocal data acquired in step 23 and the lyrics text data corresponding to the lyrics of the karaoke song X, and the color change information corresponding to the time required for the color change when the lyrics of the karaoke song X are displayed. Is calculated for each character (calculation of color change information. Step 24).

On the other hand, the specifying unit 600 specifies the timing at which the model singer visually recognizes a certain character based on the line-of-sight detection result in step 22 (identification of the visually recognized timing. Step 25).

The setting unit 300 sets the time to start the color change based on the timing when the certain character specified in step 25 is visually recognized (setting the time to start the color change. Step 26).

The creation unit 400 associates the color change information calculated in step 24 with the time for starting the color change set in step 26, so that the lyrics displayed based on the lyrics text data at the time of playing the karaoke song X Create lyrics telop color change data for controlling color change (creation of lyrics telop color change data. Step 27).

Further, the creation unit 400 creates the lyrics telop data of the karaoke song X by associating the lyrics text data with the lyrics telop color change data created in step 27 (creation of lyrics telop data, step 28).

As described above, the karaoke device 1 according to the present embodiment visually recognizes the display control unit 500 for displaying the lyrics based on the lyrics text data according to the karaoke performance of a certain karaoke song, and the displayed lyrics. Based on the line-of-sight detection result of the singer, the singing unit 600 has a specific unit 600 that specifies the timing when the singer visually recognizes the character. Further, the setting unit 300 sets the time for starting the color change of the specified character based on the timing at which the specified character is visually recognized.

As mentioned above, the singer looks at the characters he or she is trying to sing. That is, it can be said that the timing at which the line of sight is directed is a timing suitable for starting the color change of the characters. Therefore, the karaoke device 1 according to the present embodiment detects the line of sight and sets the time for starting the color change. Therefore, it is possible to create lyrics telop color change data suitable for actual karaoke singing.

<Others>
The above embodiment is presented as an example and does not limit the scope of the invention. The above configurations can be implemented in appropriate combinations, and various omissions, replacements, and changes can be made without departing from the gist of the invention. The above-described embodiments and modifications thereof are included in the scope and gist of the invention, as well as in the scope of the invention described in the claims and the equivalent scope thereof.

1 Karaoke device 10 Karaoke body 11 Control unit 100 Acquisition unit 200 Calculation unit 300 Setting unit 400 Creation unit

Claims (5)

Based on the singing voice signal obtained by singing a certain karaoke song, it includes utterance text information corresponding to a plurality of uttered pseudonyms and pseudonym utterance timing information corresponding to the timing of uttering each of the plurality of pseudonyms. The acquisition department that acquires vocalization data,
Based on the acquired vocal data and the lyrics text data corresponding to the lyrics of the certain karaoke song, the color change corresponding to the time required for the color change or the speed of the color change when the lyrics of the certain karaoke song are displayed. A calculation unit that calculates information for each character included in the lyrics,
A setting unit that sets the time to start the color change, and
By associating the calculated color change information with the set time to start the color change, the lyrics for controlling the color change of the lyrics displayed based on the lyrics text data when the certain karaoke song is played. A creation department that creates telop color change data, and
Lyrics telop color change data creation device with. The lyrics telop color change data creating device according to claim 1, wherein the setting unit sets a time earlier than the timing at which the pseudonym is uttered by a predetermined value as a time for starting the color change. The lyrics telop color change data creating device according to claim 2 , wherein the predetermined value is in the range of 200 msec to 300 msec. When the setting unit sets a time earlier than the timing at which the pseudonym is uttered by a predetermined value as the time to start the color change, if the performance tempo of the certain karaoke music is faster than the predetermined tempo, 2 If the predetermined value is changed to be longer in the range of 00 msec to 300 msec and the performance tempo of the certain karaoke song is slower than the predetermined tempo , the predetermined value becomes longer in the range of 200 msec to 300 msec. The song telop color change data creating device according to any one of claims 1 to 3, wherein the device is changed so as to be shorter. A display control unit that displays lyrics based on the lyrics text data according to the performance of the certain karaoke song,
Based on the line-of-sight detection result of the singer who visually recognizes a certain character included in the displayed lyrics, a specific unit that specifies the timing when the singer visually recognizes the certain character, and
Have,
The lyrics telop color change data creation device according to claim 1, wherein the setting unit sets a time for starting the color change of the certain character based on the timing at which the specified character is visually recognized. ..

Images