JP7632413B2 - Karaoke program and karaoke system - Google Patents

Description

The present invention relates to a karaoke program and a karaoke system that can be executed on a terminal device.

Traditionally, karaoke, where people enjoy singing along to musical accompaniment, has been performed in social settings. Karaoke is usually enjoyed by gathering at karaoke booths or snack bars, where people eat and drink. However, in the event of an infectious disease outbreak, it may be difficult to gather and enjoy karaoke. For this reason, some karaoke lovers enjoy karaoke by using online web conferences. In karaoke using web conferences, users use their personal computers to connect to a web conference server that is hosting the web conference, and download songs to play and enjoy singing.

When enjoying karaoke in a remote location, such as karaoke using a web conference, delays in the transmission line can become a problem. Patent Document 1 discloses a karaoke system that deals with such transmission delays. Specifically, the system measures the transmission delay time of the received singing voice between the karaoke performance device of the remote singing partner, and generates a substitute playback voice to be played from the speaker in accordance with the measured transmission delay time of the singing voice.

JP 2016-206575 A

Patent Document 1 discloses a method to reduce the sense of discomfort caused by delays when singing at multiple locations. However, no consideration is given to listeners who are not expected to sing, and so viewers hear substitute audio at terminals with large delays, unable to hear the actual singer's voice, and unable to get the feeling of enjoying karaoke in the same karaoke space.

The present invention has been developed in consideration of such circumstances, and provides a mechanism that suppresses the mismatch between the music and the singing voice, and does not cause discomfort to the listener, even when communication is performed between multiple terminal devices.

For this reason, the karaoke program according to the present invention employs the following configuration.
A karaoke program executable on a plurality of terminal devices connected to each other via a network, the program comprising:
A connection process for connecting to a server device;
A discrimination process for discriminating whether the terminal is a singer terminal or a listener terminal;
a first receiving process for receiving music information from the server device;
If the terminal is determined to be a listener terminal in the determination process ,
a second receiving process for receiving singing voice information transmitted from the singer terminal in a streaming manner;
A determination process for determining a delay amount of the second reception process;
A reproduction process is executed to delay the reproduction of the music information with respect to the streaming-reproduced singing voice, based on the reproduction timing of the music information determined based on the delay amount of the second reception process.

Furthermore, in the karaoke program according to the present invention,
The playback timing used in the playback process is determined based on the delay amounts determined at the plurality of listener terminals.

Furthermore, in the karaoke program according to the present invention,
The playback timing used in the playback process is determined based on the maximum delay amount among the delay amounts determined at the multiple listener terminals.

Furthermore, in the karaoke program according to the present invention,
The determination process is executed before the start of reproduction of the music information or multiple times during the execution of the reproduction process.

Furthermore, in the karaoke program according to the present invention,
In the reproduction process, if the reproduction timing has changed from the previous time, the reproduction speed of the music information is changed.

Furthermore, in the karaoke program according to the present invention,
The reproduction of the music information in the reproduction process is delayed during a non-singing section.

Further, the karaoke program according to the present invention comprises:
A karaoke program executable on a plurality of terminal devices connected via a network,
A connection process for connecting to a server device;
A discrimination process for discriminating whether the terminal is a singer terminal or a listener terminal;
a first receiving process for receiving music information from the server device;
If the discrimination means judges that the terminal is a listener terminal,
a second receiving process for receiving singing voice information transmitted from the singer terminal in a streaming manner;
a determination process for determining a first delay amount of the first reception process and a second delay amount of the second reception process;
A playback process is executed to delay the playback of the music information relative to the streaming-played singing voice, based on a first delay amount of the first receiving process and a playback timing of the music information determined based on a second delay amount of the second receiving process.

Further, the karaoke system according to the present invention comprises:
A karaoke system having a plurality of terminal devices communicatively connected via a network,
The terminal device
A connection process for connecting to a server device;
A discrimination process for discriminating whether the terminal is a singer terminal or a listener terminal;
a first receiving process for receiving music information from the server device;
If the terminal is determined to be a listener terminal in the determination process ,
a second receiving process for receiving singing voice information transmitted from a singer terminal in a streaming manner;
A determination process for determining a delay amount of the second reception process;
A reproduction process is executed to delay the reproduction of the music information with respect to the streaming-reproduced singing voice, based on the reproduction timing of the music information determined based on the delay amount of the second reception process.

The karaoke program and karaoke system according to the present invention can reduce the sense of discomfort felt by listeners when karaoke is performed between multiple terminal devices by suppressing the mismatch between the singing voice and the music on the terminal device of the listener who listens to the singing.

FIG. 1 shows a system configuration according to an embodiment of the present invention. FIG. 1 is an external view of a mobile terminal according to an embodiment of the present invention; FIG. 1 is a block diagram showing a configuration of a mobile terminal according to an embodiment of the present invention. FIG. 1 is a flow diagram showing room processing according to the present embodiment. 1 is a flow chart showing the karaoke process of the present embodiment; FIG. 1 is a flow diagram showing a music playback process A according to an embodiment of the present invention. FIG. 1 is a flow diagram showing a music playback process B according to the present embodiment. 1 is a timing chart illustrating the amount of delay in a mobile terminal according to an embodiment of the present invention. A timing chart illustrating the amount of delay in a mobile terminal according to another embodiment. 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention;

Figure 1 is a diagram showing the system configuration of this embodiment. In this embodiment, the system allows karaoke to be performed between mobile terminals 6, and therefore communication between the mobile terminals 6 is necessary. Figure 1 shows various examples of communication formats between the mobile terminals 6. For example, in the case of a private home, a router 12a connected to the Internet is provided, and a wireless router 12b is connected to the router 12a. The mobile terminal 6 can be connected to the Internet via the wireless router 12b.

In addition to these communication formats, it is also possible to communicate with the Internet using a mobile communication network. The mobile communication network is a communication format in which the mobile terminal 6 is connected to the Internet by performing wireless communication between the mobile terminal 6 and a wireless base station 13.

By using these various communication methods, the mobile terminal 6 can be connected to the Internet. In addition, a management server 11a and a distribution server 11b for managing the karaoke system are also connected to the Internet. A karaoke program is installed on the mobile terminal 6 that is connected to the Internet, and by communicating with the management server 11a and the distribution server 11b, it becomes possible to perform karaoke between the mobile terminals 6.

In this embodiment, a form of karaoke using a mobile terminal 6 as a terminal device will be described. Note that, in addition to the mobile terminal 6, a home game machine, a personal computer, or a karaoke device dedicated to karaoke may also be used as the terminal device. Furthermore, a terminal device in which a karaoke program is installed will function as a karaoke device.

Fig. 2 is a diagram showing the appearance (front view) of a mobile terminal 6 according to an embodiment of the present invention, and Fig. 3 is a block diagram showing the configuration of the mobile terminal 6 according to an embodiment of the present invention. This mobile terminal 6 can be a smartphone, a tablet terminal, a mobile phone, or the like. By installing a karaoke program in this mobile terminal 6, it becomes possible to perform karaoke between mobile terminals 6.

The portable terminal 6 of this embodiment has a control means including a CPU 61, a RAM 62, a storage unit 63, an image processing unit 64, and an audio processing unit 66. A non-volatile storage means such as a NAND type flash memory can be used for the storage unit 63. The storage unit 63 stores the karaoke program of this embodiment and various data used in the karaoke program.

The CPU 61 is a means for controlling the entire mobile terminal 6, and executes various controls based on the programs and data stored in the memory unit 63. The RAM 62 temporarily stores data generated under the control of the CPU 61.

The sound processing unit 66 is a means for inputting and outputting sound related to the portable terminal 6. It is a means for emitting sound from the speaker 67a under the control of the CPU 61. The sound processing unit 66 is also connected to the microphone 67b, and can pick up singing voices, telephone conversation voices, and surrounding sounds from the microphone 67b. External devices such as a headset may be used for the speaker 67a and microphone 67b.

The touch panel display unit 68 is composed of a display unit 68m that displays an image and a touch panel 68n that is arranged on the front or back of the display unit 68m. The touch panel 68n can be of various types that can recognize the position of the user's touch, such as a capacitance type. The CPU 61 displays on the display unit 68m and can determine which part of the displayed image has been touched by touch input from the touch panel 68n.

The image processing unit 64 is a means for inputting and outputting image-related data for the mobile terminal 6. The image processing unit 64 displays and outputs images formed by the CPU 61 on the display unit 68m. It also has a function of capturing images, or video (if video), from the front camera 65F arranged on the front side (the touch panel display unit 68 side) of the mobile terminal 6, or the rear camera 65R arranged on the rear side of the mobile terminal 6, and storing the captured images or video in the memory unit 63 as image or video information.

The mobile terminal 6 of this embodiment includes a first communication unit 69a and a second communication unit 69b as communication means for communicating with the outside. The first communication unit 69a is a communication means for connecting to a mobile communication network, and by wirelessly connecting to a mobile base station 140, it is possible to perform various communications via the Internet network using the communication lines of the mobile communication network. The second communication unit 69b is a communication means for connecting via Wi-Fi. The mobile terminal 6 can use the second communication unit 69b to wirelessly connect to an access point 110 and communicate with a local area network in which the access point 110 participates. Note that, in addition to the first communication unit 69a and the second communication unit 69b, the communication means of the mobile terminal 6 may include an infrared communication unit, a short-range communication means such as Bluetooth (registered trademark), and the like.

The mobile terminal 6 of this embodiment is configured to include an input switch 70 in addition to the touch panel 68n as an input means. This input switch 70 (input switches 70a, 70b, etc. in FIG. 2) can be provided with various switches such as a power switch, a volume adjustment switch, and a switch whose input purpose is changed depending on the program.

Furthermore, the mobile terminal 6 of this embodiment has a vibration unit 72 and an LED 73 to provide various notifications to the user who holds the mobile terminal 6. The vibration unit 72 is a member formed by attaching an eccentric weight to the shaft of a motor. By driving the motor to rotate, it is possible to transmit vibrations that can notify the user. The LED 73 is located above the front of the mobile terminal 6 as shown in FIG. 2, and can provide various notifications to the user by blinking.

In this embodiment, by installing a karaoke program on such a mobile terminal 6, it becomes possible to perform karaoke between mobile terminals 6. Now, we will explain the details of the various processes that are performed when performing karaoke using a mobile terminal 6.

Figure 4 is a flow diagram showing the room processing of this embodiment. Here, a case where karaoke is performed using mobile terminals 6a and 6b will be described. In this embodiment, by gathering in a virtually created room, users in the same room can perform karaoke by sharing audio (which may also include video).

Here, we will explain the case where a user using mobile terminal 6a creates a room, and a user using mobile terminal 6b participates in that room. It is assumed that the users using mobile terminals 6a and 6b already have accounts on management server 11a. Note that users who do not have an account can access management server 11a and create a new account.

First, the mobile terminal 6a issues a login instruction (S11) to the management server 11a. The login instruction is issued by inputting account information (user ID, password, etc.). Upon receiving the login instruction (S11), the management server 11a executes a login process (S21). In the login process, authentication is performed based on the account information, and if the authentication is correct, the user is allowed to use various functions of the karaoke program.

Figure 10 (A) shows the room selection screen that is displayed after logging in. The room selection screen displays a room selection field 68a and a create new room button 68b. The room selection field 68a is a room that has already been created, and displays the status of that room. The room selection field displays the room name, icons of users participating in the room, the room's holding status ("OPEN", "In preparation"), the date the room will be held, etc. Also, a lock symbol displayed in the upper left corner of the room selection field 68a indicates that the room requires a password or that only users who have been invited in advance can participate in the room.

A user can participate in a room by operating the room selection field 68a. The new room creation button 68b is a button for creating a new room. By operating the new room creation button 68b, the mobile terminal 6a can issue a room creation instruction (S12) to the management server 11a. The management server 11a executes a room creation process (S22) based on the room creation instruction (S12). When the room creation process (S22) is completed, the room selection field 68a shown in FIG. 7 (A) is displayed.

On the other hand, mobile terminal 6b that wants to join the room first issues a login instruction (S11) to management server 11a by inputting account information. As with mobile terminal 6a, management server 11a performs authentication based on the account information, and if the authentication is correct, allows mobile terminal 6b to log in. The user refers to the room selection screen shown in FIG. 10(A) and issues a room participation instruction by selecting the room selection field 68a that the user wants to enter (S13). In the example of FIG. 4, when the user of mobile terminal 6a is participating in a room created by the user, management server 11a executes a user addition process (S23) that allows the user of mobile terminal 6b to participate in the room created by the user of mobile terminal 6a.

When the users of mobile terminals 6a and 6b enter the same room, it becomes possible to execute karaoke processing (S12) between mobile terminals 6a and 6b.

The karaoke process (S30) mainly comprises reservation management and music playback processing. Reservation management is a process for reserving songs to be sung, and music playback processing is a process for playing songs registered in the reservation list in order based on the reservation processing. When playing songs, performance sounds are played based on the performance information included in the music information, and lyrics are displayed based on the lyrics information included in the music information.

Figure 5 is a flow diagram showing the karaoke process (S30) of this embodiment. In Figure 5, the mobile terminal 6a is the singer (the person who reserved the song) and the mobile terminal 6b is the listener, but the singer and listener may be interchangeable. Also, in Figure 5, for ease of explanation, two mobile terminals 6a and 6b are shown as an example participating in the room, but three or more mobile terminals may participate in the room.

First, the mobile terminals 6a and 6b each communicate with the management server 11a and start a session for reservation management (S101, S121). The management server 11a starts a session for reservations, music distribution, etc. for the mobile terminals 6a and 6b participating in the same room (S161).

Next, the mobile terminals 6a and 6b each communicate with the distribution server 11b and start a session for the audio stream (S102, S122). After the session for the audio stream starts, audio communication of singing voice information and the like can be performed between the mobile terminals 6a and 6b. The distribution server 11b distributes the singing voice and the like between the mobile terminals 6a and 6b using a streaming method (S151).

After the session for the audio stream starts, the mobile terminals 6a and 6b each communicate with the distribution server 11b to measure the amount of delay in audio upload/download (S103, S123). The amount of delay measured here is used to align the timing of the song and singing voice played on the mobile terminal 6b used by the listener.

The singer's mobile terminal 6a executes a song reservation process (S104). In the reservation process (S104), a song is searched for and selected to be sung, allowing the song to be added to a reservation list managed by the management server 11a. A user who wants to reserve a song searches for a song using the song search screen shown in FIG. 10(B).

The song search screen has a search category selection field 68c and a song selection field 68d. The search category selection field has search categories such as "Favorites," "Songs," "Artists," and "Albums." By selecting one of these search categories, the user can perform a song search according to the selected search category. The state in FIG. 10(B) is when "Songs" has been selected as the search category, and the song selection field 68d is displayed in alphabetical order.

The song selection field 68d displays an icon (such as album art), song title, and artist name for each song. When the user selects the desired song selection field 68d, a song confirmation screen for that song is displayed.

Figure 11 (C) is a diagram showing the song confirmation screen of this embodiment. The song confirmation screen displays a song confirmation field 68e. In the song confirmation field, for the selected song, an icon (such as album art), song name, artist name, a reservation button 68g, and a cancel button 68f are displayed. The user checks the information displayed on this song confirmation screen, and when making a reservation, operates the reservation button 68g to reserve the song. When a reservation is made, reservation information is sent to the management server 11a. The reservation information includes a user ID indicating the user who made the reservation, and a song ID indicating the song. The management server 11a registers the received reservation information in a reservation list (S162).

Then, when the next song to be played arrives in the reservation list, the management server 11a notifies the mobile terminals 6a and 6b belonging to the same room of a reservation response (S163). The reservation response includes information that can identify the person who reserved the song, and by referring to the reservation response, the mobile terminals 6a and 6b can determine whether the terminal is a singer terminal (a mobile terminal 6 that has reserved a song) or a listener terminal (a mobile terminal 6 that has not reserved a song).

In the case of FIG. 5, since the song reservation was made by mobile terminal 6a, mobile terminal 6a determines that it is a singer terminal and sets itself to singer mode (S106). On the other hand, mobile terminal 6b determines that it is a listener terminal and sets itself to listener mode (S125).

After receiving the reservation response, the management server 11a transmits music information to the mobile terminals 6a and 6b participating in the same room (S164). In this embodiment, the music information is distributed using a storage-type distribution method rather than a streaming method. Even with a storage-type distribution method, it is possible to start playback by storing part of the music information. When transmission from the management server 11a begins, the mobile terminals 6a and 6b start receiving the music information (S107, S126), and execute music playback process A (S130) and music playback process B (S140), respectively.

Figure 6 is a flow diagram showing music playback process A (S130) executed by the mobile terminal 6a set to singer mode. In music playback process A, playback of music information is started (S132) on the condition that a predetermined amount of music information has been downloaded (S131: Yes). In addition, during the period until playback of the music information ends (S134: Yes), the amount of upload delay for the singing voice information from the mobile terminal 6a to the distribution server 11b is measured at appropriate timing (S133).

Figure 12 (E) shows a music playback screen that is displayed during music playback on the mobile terminal 6 of this embodiment. As explained in Figure 11 (D), the room name and a music search button 68k are displayed at the top of the music playback screen. By operating the music search button 68k, the music search screen explained in Figure 10 (B) is displayed, and it is possible to reserve a song.

In addition, the music playback screen is broadly divided into a music-related display section 68h, an upper display section 68i, and a lower display section 68j. The music-related display section 68h includes a lyrics display section 681 and a progress bar 682. The lyrics display section 681 displays the lyrics to be sung for the music being played. Note that the lyrics display section 681 may also display a background image. The progress bar 682 indicates the progress of music playback.

The area occupied by the upper display column 68i and the lower display column 68j can be changed. Normally, the state shown in FIG. 11(D) is displayed, that is, both the upper display column 68i and the lower display column 68j are displayed. When the comment tab 686, reservation confirmation tab 687, or remote control tab 688 provided in the lower display column 68j is operated, the lower display column 68j is displayed using the area where the upper display column 68i was displayed, as shown in FIG. 13(F). FIG. 13(F) shows the state where the remote control tab 688 has been operated. Operating the comment tab 686, reservation confirmation tab 687, or remote control tab 688 again returns to the state shown in FIG. 12(E).

The upper display area 68i displays a status display/selection button 683, a microphone selection button 684, and a user icon 685. The status display/selection button 683 is a button that indicates the status of the mobile terminal 6. When the mobile terminal 6 is set to the singer mode, "singing" is marked, and when the mobile terminal 6 is set to the listener mode, "listening" is marked. The state in FIG. 11(D) is a state in which "singing" is marked. The user can manually switch between "singing", "listening", and "away". When the user switches to "singing", the setting is changed to the singer mode, and when the user switches to "listening" or "away", the setting is changed to the listener mode. In this embodiment, the three states can be switched, but the two statuses "singing" and "listening" may also be used, and various display forms can be adopted.

The microphone selection button 684 can be manually operated by the user to switch the microphone 67b on and off. In this embodiment, when the mode is set to singer mode, it is automatically set to on, and when the mode is set to listener mode, it is automatically set to off. The user icons 685 show all users participating in the room. In the example of FIG. 11(D), four users have participated. The user whose icon is shown with a thick line ("JOY" in the example of FIG. 11(D)) indicates that he is singing (set to singer mode). Also, the icon with a speaker and a cross mark ("Monkichi" in the example of FIG. 11(D)) indicates that he is away. These display forms are merely examples, and status such as singing, listening, and away may be shown by other display forms.

The lower display field 68j has three tabs (a comment tab 686, a reservation confirmation tab 687, and a remote control tab 688). In the state shown in FIG. 11(D), the comment tab 686 is selected, and the lower display field 68j displays a comment display field 689 and a comment input field 690. The comment display field 689 displays user comments entered by the user using the comment input field 690, as well as system comments associated with operations related to the music playback process. In FIG. 11(D), the comment display field 689 displays system comments associated with the reservation process, including the user name and the reserved music/artist name. The comment display field 689 displays the system comments and user comments in chronological order, allowing users to communicate smoothly by viewing these system comments and user comments.

Figure 13 (F) shows the state when the remote control tab 688 is operated. As mentioned above, the lower display field 68j is displayed by expanding it to the area where the upper display field 68i is displayed. When the remote control tab 688 is selected, the lower display field 68j displays the singer volume 691, speaker volume 692, pitch change button 693, BGM volume 694, playback position adjustment button 695, re-sing button 696, pause button 697, and stop performance button 698.

In this embodiment, the various controls and buttons displayed on the remote control tab 688 can only be operated from a mobile terminal 6 set to the singer mode. Therefore, operations related to music playback are prohibited from a mobile terminal 6 set to the listener mode, making it possible to prevent erroneous input by the listener.

The singer volume 691 is an operator that adjusts the volume of the singing voice input from the microphone 67b of the mobile terminal 6 that is set to the singer mode. The speaker volume 692 is an operator that adjusts the volume of the singing voice input from the microphone 67b of the mobile terminal 6 that is set to the listener mode. The singer volume 691 and speaker volume 692 are set to the default state for each mode when the mobile terminal 6 is set to the singer mode or listener mode, but can only be operated from the mobile terminal 6 that is set to the singer mode.

The pitch change button 693 is a button for changing the pitch of the song being played. The BGM volume 694 is an operator for adjusting the volume of the BGM played between songs. The playback position adjustment button 695 is an operator for changing the playback position. The current playback time and total playback time are displayed to the left of the operators. The sing again button 696 can be operated to play the currently played song from the beginning. The pause button 697 is a button for temporarily pausing the currently played song. The stop performance button 698 is a button for stopping the currently played song, and when the stop performance button 698 is operated, playback processing moves to the next reserved song.

Next, we will explain the music playback process B executed by the mobile terminal 6a set to the listener mode. FIG. 7 is a flow diagram showing the music playback process B (S140). In the music playback process B, the playback of the music information is also started (S144) under the conditions that a predetermined amount of music information has been downloaded (S141: Yes) and that the playback start timing has arrived (S143: Yes).

Here, the playback start timing is the timing calculated (S142) using the amount of delay measured using measurements (S103, S123) on the mobile terminals 6a and 6b. The singing voice information input on the mobile terminal 6a is uploaded to the distribution server 11b, downloaded from the distribution server 11b to the mobile terminal 6b, and played on the listener's mobile terminal 6b. For this reason, it is conceivable that a delay will occur in the singing voice information between the mobile terminal 6a and the mobile terminal 6b. In this embodiment, taking such circumstances into consideration, one of the objectives is to suppress any discrepancy that may occur between the singing voice and the song played on the listener's mobile terminal 6b.

Figure 8 is a timing chart illustrating the amount of delay in a mobile terminal of this embodiment. Like Figure 5, Figure 8 assumes two mobile terminals 6a and 6b. On the singer's side, the singer sings in sync with the playback of the music information, so there is no delay between the music and the singing voice. On the other hand, on the listener A's side, as explained above, a delay amount D1 occurs in the singing voice information. Therefore, if the music is played at the same timing as the singing company's side, the music will end up being played ahead of the music by the delay amount D1.

Therefore, in this embodiment, the start of playback of the music information is delayed by a delay amount D1 to synchronize the music and the singing voice on the listener side. The delay amount D1 is calculated, for example, by adding the delay amount measured by the mobile terminal 6a (S103) when uploading the singing voice information and the delay amount measured by the mobile terminal 6b (S123) when downloading the singing voice information. The distribution server 11b receives the delay amount from the mobile terminals 6a and 6b to calculate the delay amount D1, and transmits the playback timing St_a received from the mobile terminal 6a on the singer side to the mobile terminal 6b on the listener side. The mobile terminal 6b determines the playback timing St_b on the listener A side based on the received delay amount D1 and playback timing St_a (S142).

In addition, in this embodiment, the listener's mobile terminal 6b corrects the timing of the playback of the singing voice and the music during the playback of the music information. The delay in the singing voice information that occurs between the mobile terminals 6a and 6b is likely to change depending on the communication conditions. Therefore, the mobile terminals 6a and 6b measure the amount of delay in the singing voice information at appropriate times during the playback of the music information, thereby suppressing the occurrence of a discrepancy between the played singing voice and the music.

As explained in FIG. 6, in the music playback process A executed on the singer's mobile terminal 6a, the amount of delay in uploading singing voice information during the music playback period is measured (S133) and transmitted to the distribution server 11b. Meanwhile, in the music playback process B executed on the listener's side, the amount of delay in downloading singing voice information is measured (S145) and transmitted to the distribution server 11b. Then, in the music playback process B, based on the amount of delay received from the distribution server 11b, a correction amount for the playback of the music information is determined (S146). The amount of correction may be to speed up, delay, or maintain the timing of playback of the music information.

The mobile terminal 6b corrects the playback timing of the music information based on the determined correction amount (S147). One possible correction method would be to change the playback timing to the appropriate timing all at once, but in that case, the continuity of the music playback would be lost, resulting in an unnatural playback sound. For this reason, this embodiment employs a method of correcting the playback speed of the music information until the appropriate playback timing is reached. For example, if the playback timing of the music information is late, the playback speed of the music information is increased, and if the playback timing of the music information is early, the playback speed of the music information is decreased.

The measurement of the delay amount and the correction of the playback timing are repeated at appropriate timings until the playback of the music information ends (S148: Yes). By performing correction based on the delay amount during the playback of the music information in this way, it is possible to prevent a discrepancy from occurring between the singing voice information input on the singer's mobile terminal 6a and the music information played on the listener's mobile terminal 6b.

Figure 9 is a timing chart explaining the amount of delay in a mobile terminal in another embodiment. In the embodiment of Figure 8, as in Figure 5, a state in which two mobile terminals 6a and 6b participate in a room is explained. However, it is also possible for three or more mobile terminals 6 to participate in a room. Figure 9 shows a state in which three mobile terminals 6 participate in the same room, with one mobile terminal 6a being used by the singer and two mobile terminals 6b and 6c being used by listeners.

Figure 9 (A) shows an embodiment in which the same processing as in Figure 8 is performed individually in each of the mobile terminals 6b and 6c. It is considered that the delay amount is different between the mobile terminals 6b and 6c. Each of the mobile terminals 6b and 6c uses the respective delay amounts D1 and D2, and uses the playback timings St_b and St_c to achieve consistency between the audio and the music.

In addition to enjoying karaoke by listening to the singing voice, mobile terminals 6b and 6c can also be used to actively participate and enjoy themselves. For example, it is possible to exchange messages using the comment input field 690 described in FIG. 12. Listeners can enter comments or other messages to support the singer, or communicate with each other by sharing their impressions of the singing.

As shown in Figure 9 (A), if the timing of playback of music information differs between the listeners' mobile terminals 6b, 6c, communication between listeners may not be smooth, particularly due to misalignment of the timing of comments about the music being played or the singing voice.

For this reason, in this embodiment, the listener's mobile terminals 6a and 6b use the same playback timing, rather than using different playback timings St_b and St_c.

As shown in FIG. 9(B), the singing voice information is delayed by D1 in the mobile terminal 6b, and delayed by D2 in the mobile terminal 6c. Note that the delay D1 is smaller than the delay D2. In this case, the playback timings of the listener's mobile terminals 6b and 6c are synchronized by unifying the playback timing St_c using the larger delay D in each mobile terminal 6b and 6c. The distribution server 11b decides which of the delays D1 and D2 to use, or the information required for the decision, such as the delays D1 and D2, can be received via the distribution server 11b.

In this way, by unifying the playback timing on the listeners' mobile terminals 6b and 6c, multiple listeners can listen to the music and singing voice at the same time, which facilitates smooth communication between listeners.

As described above, in this embodiment, when enjoying karaoke in a remote location, it is possible to suppress the mismatch between the reproduced music information and the singing voice information, and to suppress the sense of discomfort felt by the listener. Note that the present invention is not limited to the above-described embodiment. Various modified examples are described below.

[First Modification]
In the above-mentioned embodiment, the timing of reproducing the music information is determined taking into consideration the delay of the singing voice information. In the above-mentioned embodiment, since a storage type music information distribution is used, the possibility of a delay occurring in the download of the music information is low. Meanwhile, the music information can also be distributed using a streaming method, as in the case of the singing voice information. In this case, it is considered that a delay occurs in the music information distributed from the management server 11a, and in some cases, the delay may be greater than that of the singing voice information.

In light of this situation, the timing of playback of music information on the listener's mobile terminal 6b may take into account the delay amount of music information at the time of distribution. In this case, the listener's mobile terminal 6b measures the delay amount (first delay amount) for downloading from the management server 11a that distributes the music information. If the delay amount (first delay amount) of the music information is greater than the delay amount (second delay amount) of the singing voice information transmitted from the mobile terminal 6a, the timing of playback of the music information is determined based on the delay amount (first delay amount) of the music information. On the other hand, if the delay amount (first delay amount) of the music information is equal to or less than the delay amount (second delay amount) of the singing voice information transmitted from the mobile terminal 6a, the timing of playback of the music information is determined based on the delay amount (second delay amount) of the singing voice information.

In this way, even if the distribution of music information is delayed due to a streaming method or the like, it is possible to synchronize the playback timing of music information and singing voice information.

When there are multiple listeners, the listeners' mobile terminals 6b and 6c measure the amount of delay (first delay) for downloading from the management server 11a. By determining the playback timing using the largest amount of delay between the singing voice information delay (second delay) and the music information delay (first delay) measured by the listeners' mobile terminals 6b and 6c, it becomes possible to unify the playback timing between the listeners' mobile terminals 6b and 6c.

[Second Modification]
In the above embodiment, as explained in the music playback process B of FIG. 7, the playback timing is corrected when a deviation occurs during music playback. In the above embodiment, the playback speed is changed to perform the correction (S147) to reproduce a natural playback sound. Instead of this form, the correction may be performed at a timing other than the singing section (non-singing section). For example, by performing the correction during an interlude, it is possible to perform the correction without interfering with the listening of the singing. Furthermore, the non-singing section may be not only an interlude but also a short section between singing phrases.

The method of detecting the non-singing section may involve setting information about the singing section in the music information beforehand and detecting it by reading it out in synchronization with the playback of the music information, or detecting periods when no singing voice is being input as the non-singing section.

[Third Modification]
In the above embodiment, the singing voice information is transmitted from the portable terminal 6a of the singer to the portable terminals 6b and 6c of the listeners via the distribution server 11b. Instead of this form, the singing voice information may be transmitted from the portable terminal 6a of the singer to the portable terminals 6b and 6c of the listeners in a peer-to-peer manner without the distribution server 11b.

6 (6a, 6b): Portable terminal 68m: Display unit 11a: Management server 68n: Touch panel 11b: Distribution server 69a: First communication unit 12a: Router 69b: Second communication unit 12b: Wireless router 70 (70a, 70b): Input switch 13: Wireless base station 72: Vibration unit 61: CPU 73: LED
62: RAM 110: Access point 63: Storage unit 140: Mobile base station 64: Image processing unit 681: Lyrics display field 65F: Front camera 682: Progress bar 65R: Rear camera 683: Selection button 66: Sound processing unit 684: Microphone selection button 67a: Speaker 685: User icon 67b: Microphone 686: Comment tab 68: Touch panel display unit 687: Reservation confirmation tab 68a: Room selection field 688: Remote control tab 68b: New room creation button 689: Comment display field 68c: Search category selection field 690: Comment input field 68d: Song selection field 691: Singer volume 68e: Song confirmation field 692: Speaker volume 68f: Cancel button 693: Pitch change button 68g: Reservation button 694: BGM volume 68h: Song related display field 695: Playback position adjustment button 68i: Upper display area 696: Button 68j: Lower display area 697: Pause button 68k: Song search button 698: Play stop button

Claims (7)

A karaoke program executable on a plurality of terminal devices connected to each other via a network, the program comprising:
A connection process for connecting to a server device;
A discrimination process for discriminating whether the terminal is a singer terminal or a listener terminal;
a first receiving process for receiving music information from the server device;
If the terminal is determined to be a listener terminal in the determination process ,
a second receiving process for receiving singing voice information transmitted from the singer terminal in a streaming manner;
A determination process for determining a delay amount of the second reception process;
a reproduction process for delaying reproduction of the music information with respect to the streaming-reproduced singing voice, based on a reproduction timing of the music information determined based on the delay amount of the second reception process. 2. The karaoke program according to claim 1, wherein the playback timing used in the playback process is determined based on the delay amounts determined by the plurality of listener terminals. 3. The karaoke program according to claim 2, wherein the playback timing used in the playback process is determined based on a maximum delay amount among delay amounts determined in the plurality of listener terminals. The karaoke program according to claim 1 , wherein the determination process is executed before playback of the music information starts or multiple times during the playback process. The karaoke program according to claim 4 , wherein the reproduction process changes a reproduction speed of the music information when a reproduction timing has changed from a previous time. The karaoke program according to claim 4 , wherein the delay in the reproduction of the music information in the reproduction process is performed during a non-singing section. A karaoke system having a plurality of terminal devices communicatively connected via a network,
The terminal device
A connection process for connecting to a server device;
A discrimination process for discriminating whether the terminal is a singer terminal or a listener terminal;
a first receiving process for receiving music information from the server device;
If the terminal is determined to be a listener terminal in the determination process ,
a second receiving process for receiving singing voice information transmitted from a singer terminal in a streaming manner;
A determination process for determining a delay amount of the second reception process;
a reproduction process for delaying the reproduction of the music information with respect to the streaming-reproduced singing voice, based on a reproduction timing of the music information determined based on the delay amount of the second reception process.