JP7639740B2 - Karaoke program and karaoke device - Google Patents

Description

The present invention relates to a karaoke program that can be executed on a terminal device, and a karaoke device.

Traditionally, karaoke, where people enjoy singing along to musical accompaniment, has been performed at social gatherings. Currently, with the spread of infectious diseases, it can be difficult to gather in a karaoke booth to sing. For this reason, some karaoke lovers enjoy karaoke by using online web conferences. In karaoke using web conferences, users use their personal computers to connect to a web conference server that is hosting the web conference, and download songs to play and enjoy singing.

In karaoke, where multiple users gather, various effects are used to liven up the singing atmosphere. Patent Document 1 discloses a karaoke device that can play back interjections (recorded information) posted by users during non-singing sections. Patent Document 2 discloses a karaoke transmission system that generates karaoke background images using captured images uploaded when using karaoke, thereby providing a wide variety of karaoke background images.

JP 2015-191162 A JP 2011-75701 A

Usually, when family or friends gather to enjoy karaoke at a karaoke booth, they cheer on the singer and liven up the atmosphere by joining in, clapping, or playing tambourines or maracas. The singer can feel the listeners' emotions through such reactions and direct conversation, which allows them to become even more immersed in their singing.

On the other hand, when singing karaoke over a web conference, it is difficult for the singer to know whether the listeners are listening or not, and it is also difficult to sense how they are feeling. It is also difficult for the listeners to convey to the singer that they are listening, which can make them feel uneasy.

The effects described in Patent Documents 1 and 2 are different from karaoke, which is usually performed in a karaoke booth or the like, in that listeners' reactions are not real-time, and do not fully resolve the problems with karaoke using web conferences described above.

The present invention was developed in consideration of these circumstances, and one of its objectives is to facilitate smooth communication between users who enjoy karaoke via internet.

For this reason, the karaoke program according to the present invention employs the following configuration.
A karaoke program executable on a terminal device,
The device is capable of performing music playback processing, audio processing, communication processing, and display processing;
The music playback process plays and outputs music information.
The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit,
The communication process enables voice communication with other terminal devices;
The display process forms a lyrics display area on a display unit of the terminal device,
in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed on at least another terminal device, an effect image is displayed based on the effect display instruction;
The display process displays, on a display unit of the terminal device, an icon of a user set in a terminal device participating in the communication process, and when an effect display instruction operation is performed in at least another terminal device, displays an effect image that moves to the lyrics display area starting from the icon of the user who performed the effect display instruction operation.
Furthermore, the karaoke program according to the present invention comprises:
A karaoke program executable on a terminal device,
The device is capable of performing music playback processing, audio processing, communication processing, and display processing;
The music playback process plays and outputs music information.
The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit,
The communication process enables voice communication with other terminal devices;
The display process forms a lyrics display area on a display unit of the terminal device,
in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed at least on another terminal device, a first effect image or a second effect image, which has a different pattern from the first effect image and serves as a background for the displayed lyrics , is displayed based on the effect display instruction;
The first effect image and the second effect image are not prohibited from being displayed simultaneously, but a plurality of second effect images are prohibited from being displayed simultaneously.

Further, in the karaoke program according to the present invention,
In the lyrics display area, the display position of the effect image is determined based on the position of the lyrics being displayed or a preset lyrics display position.

Further, in the karaoke program according to the present invention,
The display process displays an effect image for a certain period of time after displaying an effect based on an effect display instruction.

The karaoke device according to the present invention further comprises:
A karaoke device capable of reproducing and outputting music information,
The device is capable of performing music playback processing, audio processing, communication processing, and display processing;
The music playback process plays and outputs music information.
The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit,
The communication process enables voice communication with other terminal devices;
The display process forms a lyrics display area on a display unit of the terminal device,
in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed on at least another terminal device, an effect image is displayed based on the effect display instruction;
The display process displays, on a display unit of the terminal device, an icon of a user set in a terminal device participating in the communication process, and when an effect display instruction operation is performed in at least another terminal device, displays an effect image that moves to the lyrics display area starting from the icon of the user who performed the effect display instruction operation.
The karaoke device according to the present invention further comprises:
A karaoke device capable of reproducing and outputting music information,
The device is capable of performing music playback processing, audio processing, communication processing, and display processing;
The music playback process plays and outputs music information.
The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit,
The communication process enables voice communication with other terminal devices;
The display process forms a lyrics display area on a display unit of the terminal device,
in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed on at least another terminal device, an effect image is displayed based on the effect display instruction;
The display process displays, on a display unit of the terminal device, an icon of a user set in a terminal device participating in the communication process, and when an effect display instruction operation is performed in at least another terminal device, displays an effect image that moves to the lyrics display area starting from the icon of the user who performed the effect display instruction operation.
The karaoke device according to the present invention further comprises:
A karaoke device capable of reproducing and outputting music information,
The device is capable of performing music playback processing, audio processing, communication processing, and display processing;
The music playback process plays and outputs music information.
The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit,
The communication process enables voice communication with other terminal devices;
The display process forms a lyrics display area on a display unit of the terminal device,
in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed at least on another terminal device, a first effect image or a second effect image, which has a different pattern from the first effect image and serves as a background for the displayed lyrics , is displayed based on the effect display instruction;
The first effect image and the second effect image are not prohibited from being displayed simultaneously, but a plurality of second effect images are prohibited from being displayed simultaneously.

The karaoke program and karaoke device of the present invention enable good communication between multiple terminal devices (karaoke devices) when singing karaoke, allowing users to fully enjoy karaoke.

FIG. 1 shows a system configuration according to an embodiment of the present invention. FIG. 1 is an external view of a mobile terminal according to an embodiment of the present invention; FIG. 1 is a block diagram showing a configuration of a mobile terminal according to an embodiment of the present invention. FIG. 1 is a flow diagram showing room processing according to the present embodiment. A flow diagram showing reservation processing according to the present embodiment. A flow diagram showing processing between mobile terminals according to the present embodiment. 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; FIG. 2 is a flow diagram showing the effect display process of the present embodiment. 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; FIG. 13 is a diagram showing a result screen based on effect history information of the present embodiment. FIG. 13 is a diagram showing a display screen of the tally result according to another embodiment;

Figure 1 is a diagram showing the system configuration of this embodiment. In this embodiment, the system allows karaoke to be performed between mobile terminals 6, and therefore communication between the mobile terminals 6 is necessary. Figure 1 shows various examples of communication formats between the mobile terminals 6. For example, in the case of a private home, a router 12a connected to the Internet is provided, and a wireless router 12b is connected to the router 12a. The mobile terminal 6 can be connected to the Internet via the wireless router 12b.

In addition to these communication formats, it is also possible to communicate with the Internet using a mobile communication network. The mobile communication network is a communication format that connects the mobile terminal 6 to the Internet by performing wireless communication between the mobile terminal 6 and a wireless base station 13.

By using these various communication methods, the mobile terminal 6 can be connected to the Internet. A management server 11 for managing the karaoke system is also connected to the Internet. A karaoke program is installed on the mobile terminal 6 that is connected to the Internet, and by communicating with the management server 11, karaoke can be performed between the mobile terminals 6.

In this embodiment, a form of karaoke using a mobile terminal 6 as a terminal device will be described. Note that, in addition to the mobile terminal 6, a home game machine, a personal computer, or a karaoke device dedicated to karaoke may also be used as the terminal device. Furthermore, a terminal device in which a karaoke program is installed will function as a karaoke device.

Fig. 2 is a diagram showing the appearance (front view) of a mobile terminal 6 according to an embodiment of the present invention, and Fig. 3 is a block diagram showing the configuration of the mobile terminal 6 according to an embodiment of the present invention. This mobile terminal 6 can be a smartphone, a tablet terminal, a mobile phone, or the like. By installing a karaoke program in this mobile terminal 6, it becomes possible to perform karaoke between mobile terminals 6.

The portable terminal 6 of this embodiment has a control means including a CPU 61, a RAM 62, a storage unit 63, an image processing unit 64, and an audio processing unit 66. The storage unit 63 can be a non-volatile storage means such as a NAND type flash memory. The program of this embodiment and various data used by the program are stored in this storage unit 63.

The CPU 61 is a means for controlling the entire mobile terminal 6, and executes various controls based on the programs and data stored in the memory unit 63. The RAM 62 temporarily stores data generated under the control of the CPU 61.

The audio processing unit 66 is a means for inputting and outputting audio related information for the mobile terminal 6. It is a means for emitting sound from the speaker 67a under the control of the CPU 61. The audio processing unit 66 is also connected to the microphone 67b, and can pick up telephone conversation voice and surrounding sounds from the microphone 67b. External devices such as a headset may be used for the speaker 67a and microphone 67b.

The touch panel display unit 68 is composed of a display unit 68m that displays an image and a touch panel 68n that is arranged on the front or back of the display unit 68m. The touch panel 68n can be of various types that can recognize the position of the user's touch, such as a capacitance type. The CPU 61 displays on the display unit 68m and can determine which part of the displayed image has been touched by touch input from the touch panel 68n.

The image processing unit 64 is a means for inputting and outputting image-related data for the mobile terminal 6. The image processing unit 64 displays and outputs images formed by the CPU 61 on the display unit 68m. It also has a function of capturing images, or video (if video), from the front camera 65F arranged on the front side (the touch panel display unit 68 side) of the mobile terminal 6 or the rear camera 65R arranged on the rear side of the mobile terminal 6, and storing the captured images or video in the memory unit 63 as image or video information.

The mobile terminal 6 of this embodiment includes a first communication unit 69a and a second communication unit 69b as communication means for communicating with the outside. The first communication unit 69a is a communication means for connecting to a mobile communication network, and by wirelessly connecting to a mobile base station 140, it is possible to perform various communications via the Internet network using the communication lines of the mobile communication network. The second communication unit 69b is a communication means for connecting via Wi-Fi. The mobile terminal 6 can use the second communication unit 69b to wirelessly connect to an access point 110 and communicate with a local area network in which the access point 110 participates. Note that, in addition to the first communication unit 69a and the second communication unit 69b, the communication means of the mobile terminal 6 may include an infrared communication unit, a short-range communication means such as Bluetooth (registered trademark), and the like.

The mobile terminal 6 of this embodiment is configured to include an input switch 70 in addition to the touch panel 68n as an input means. This input switch 70 (input switches 70a, 70b, etc. in FIG. 2) can be provided with various switches such as a power switch, a volume adjustment switch, and a switch whose input purpose is changed depending on the program.

Furthermore, the mobile terminal 6 of this embodiment has a vibration unit 72 and an LED 73 to provide various notifications to the user who holds the mobile terminal 6. The vibration unit 72 is a member formed by attaching an eccentric weight to the shaft of a motor. By driving the motor to rotate, it is possible to transmit vibrations that can notify the user. The LED 73 is located above the front of the mobile terminal 6 as shown in FIG. 2, and can provide various notifications to the user by blinking.

In this embodiment, a karaoke program is installed on such a mobile terminal 6, making it possible to perform karaoke between the mobile terminals 6. Now, we will explain the details of the various processes that are performed when performing karaoke using the mobile terminal 6.

Figure 4 is a flow diagram showing the room processing of this embodiment. Here, a case where karaoke is performed using mobile terminals 6a and 6b will be described. In this embodiment, by gathering in a virtually created room, users in the same room can perform karaoke by sharing audio (which may also include video).

Here, we will explain the case where a user using mobile terminal 6a creates a room, and a user using mobile terminal 6b participates in that room. It is assumed that the users using mobile terminals 6a and 6b already have accounts on the management server 11. Note that users who do not have an account can access the management server 11 and create a new account.

First, the mobile terminal 6a issues a login instruction (S11) to the management server 11. The login instruction is issued by inputting account information (user ID, password, etc.). Upon receiving the login instruction (S11), the management server 11 executes a login process (S21). In the login process, authentication is performed based on the account information, and if the authentication is correct, the user is allowed to use various functions of the karaoke program.

Figure 7 (A) shows the room selection screen that is displayed after logging in. The room selection screen displays a room selection field 68a and a new room creation button 68b. The room selection field 68a is a room that has already been created, and displays the status of that room. The room selection field displays the room name, icons of users participating in the room, the room's holding status ("OPEN", "In preparation"), the date the room will be held, and so on. Additionally, a lock symbol displayed in the upper left corner of the room selection field 68a indicates that the room requires a password or that only users who have been invited in advance can participate in the room.

A user can participate in a room by operating the room selection field 68a. The new room creation button 68b is a button for creating a new room. By operating the new room creation button 68b, the mobile terminal 6a can issue a room creation instruction (S12) to the management server 11. The management server 11 executes a room creation process (S22) based on the room creation instruction (S12). When the room creation process (S22) is completed, the room selection field 68a shown in FIG. 7 (A) is displayed.

On the other hand, mobile terminal 6b that wants to join the room first issues a login instruction (S11) to management server 11 by inputting account information. As with mobile terminal 6a, management server 11 performs authentication based on the account information, and if the authentication is correct, allows mobile terminal 6b to log in. The user refers to the room selection screen shown in FIG. 7(A) and issues a room participation instruction by selecting the room selection field 68a that the user wants to enter (S13). In the example of FIG. 4, when the user of mobile terminal 6a is participating in a room created by the user, management server 11 executes a user addition process (S23) that allows the user of mobile terminal 6b to participate in the room created by the user of mobile terminal 6a.

When the users of mobile terminals 6a and 6b enter the same room, it becomes possible to execute karaoke processing (S12) between mobile terminals 6a and 6b.

The karaoke process (S12) mainly comprises a reservation process and a song playback process. The reservation process is a process for reserving a song to be sung, and the song playback process is a process for playing songs registered in the reservation list in order based on the reservation process. When playing a song, the performance sound is played based on the performance information included in the song information, and the lyrics are displayed based on the lyrics information included in the song information.

Figure 5 is a flow diagram showing the reservation process of this embodiment. In the reservation process, a song is searched for and a song to be sung is selected, allowing the song to be added to a reservation list managed by the management server 11. A user who wishes to reserve a song searches for a song using the song search screen shown in Figure 7 (B) (S101). The song search screen in Figure 7 (B) can be displayed by operating the song search button 68k shown in Figures 8 (C) and 8 (D) described later.

The song search screen has a search category selection field 68c and a song selection field 68d. The search category selection field has search categories such as "Favorites," "Songs," "Artists," and "Albums." By selecting one of these search categories, the user can perform a song search according to the selected search category. The state in FIG. 7(B) is when "Songs" has been selected as the search category, and the song selection field 68d is displayed in alphabetical order.

The song selection field 68d displays an icon (such as album art), song title, and artist name for each song. When the user selects the desired song selection field 68d (S102: Yes), a song confirmation screen for that song is displayed (S103).

Figure 8 (C) is a diagram showing the song confirmation screen of this embodiment. A song confirmation field 68e is displayed on the song confirmation screen. In the song confirmation field, for the selected song, an icon (album art, etc.), song name, artist name, a reservation button 68g, and a cancel button 68f are displayed. The user checks the information displayed on this song confirmation screen, and when making a reservation, operates the reservation button 68g (S104: Yes) to reserve the song. When a reservation is made, reservation information is sent to the management server 11. The reservation information includes a user ID indicating the user who made the reservation, and a song ID indicating the song. The management server 11 registers the received reservation information in a reservation list.

Figure 6 is a flow diagram showing the processing between mobile terminals in this embodiment. Here, an example is explained in which two mobile terminals 6a and 6b are in the same room. In reality, the number of mobile terminals 6 that can be in a room can be two or more.

First, the mobile terminal 6a executes a reservation process (S100) to reserve a desired song. The reservation process (S100) is as described in FIG. 5, and the mobile terminal 6a transmits reservation information to the management server 11. The management server 11, which has received the reservation information, registers the reservation information in a reservation list (S201). In this embodiment, a system comment is displayed on all mobile terminals 6a and 6b that are in the room in response to the reservation operation (S101, S151).

The management server 11 that receives the reservation information transmits a system comment to all mobile terminals 6a, 6b in the room in which the user who made the reservation is present (S202), and the mobile terminals 6a, 6b that receive the system comment display the system comment (S101, S151). Based on this configuration, it is possible for users who sing karaoke at a distance to be notified of operations related to the music playback process (in this case, the reservation process). This allows each user to know who is performing what operation, enabling smooth communication.

The management server 11 checks the reservation list (S203), and if there is a song to be played next (S204), executes the song playback process for that song, as enclosed by the dashed line in FIG. 6. In the song playback process, the user who made the reservation is first identified as the singer based on the user ID included in the reservation information (S205). Then, an instruction is given to the mobile terminal 6a used by the user identified as the singer to change to singer mode (S208). The mobile terminal 6a that has received the instruction to change changes to singer mode (S102).

Meanwhile, the management server 11 identifies users other than the singer who are in the same room as the singer as listeners, and instructs the mobile terminal 6b used by the user identified as the singer to change to listener mode (S207). The mobile terminal 6a that receives the change instruction changes to singer mode (S152). In each mobile terminal 6, a function change process is executed to change at least one of the image display function and the sound output function executed on the mobile terminal 6 based on the change instruction to listener mode or singer mode.

Now, the singer mode and listener mode will be explained. When the mobile terminal 6 is set to the singer mode, the volume of the microphone 67b is increased sufficiently to make it easier to capture the user's singing voice. When the mobile terminal 6 is set to the singer mode, the singing voice captured by the microphone 67b is transmitted to the network (such as the management server 11) without adding any sound effects such as echo. When the mobile terminal 6 is set to the singer mode, a communication mode with lower latency is set than the communication processing in the listener mode. By setting this communication mode, the singing voice in the singer mode is transmitted to other mobile terminals 6 with reduced delay.

On the other hand, in a portable terminal 6 set to the listener mode, the microphone 67b is turned off or the volume of the microphone 67b is lowered so as not to interfere with the singing. In addition, the sound processor 66 adds sound effects to the singing voice transmitted via the network, and the sound is emitted from the speaker 67a. Note that music is played on both the portable terminal 6 set to the singer mode and the listener mode.

In this way, by adjusting the volume in the singer mode and listener mode, it is possible to create an environment that is easy to listen to. In addition, by applying sound effects to the singing voice using the sound processing unit 66 of the mobile terminal 6 set to the listener mode, it is possible to obtain the following effects.

When transmitting audio over a network, if there is a problem with the acoustic environment that causes a delay in the singer's voice, it is possible that communication between the singer and the listener will not go well. Furthermore, when singing voice to which sound effects (especially delay-type sound effects such as echo, delay, and reverb) have been applied is transmitted over a network, the singing voice may be significantly degraded. In this embodiment, by applying sound effects to the singing voice by the sound processing unit 66 of the mobile terminal 6 set to the listener mode, it is possible to reproduce high-quality singing voice and provide a good acoustic environment on the mobile terminal 6 set to the listener mode.

Returning to the flow diagram of FIG. 6, once the settings for the singer mode and listener mode are complete, the management server 11 starts transmitting music to each of the mobile terminals 6a, 6b (S208). In this embodiment, music is transmitted by streaming playback, but it is also possible to use an accumulation type in which music information is stored in advance in the mobile terminals 6a, 6b and then played back. In either case, music playback is synchronized on each of the mobile terminals 6a, 6b.

Each mobile terminal 6a, 6b plays the music (S103, S153). Then, in the mobile terminal 6a set to singer mode, the singing voice is collected by the microphone 67b (S104). The collected singing voice is sent to the management server 11, which then transfers it to the other mobile terminal 6b in the same room (S209). The mobile terminal 6b that receives the singing voice applies sound effects to the received singing voice in the sound processing unit 66 (S154), and emits it from the speaker 67a together with the music playback sound.

Figure 9 (E) shows a music playback screen that is displayed during music playback on the mobile terminal 6 of this embodiment. As shown in Figure 8 (D), the room name and a music search button 68k are displayed at the top of the music playback screen. By operating the music search button 68k, the music search screen described in Figure 7 (B) is displayed, and it is possible to reserve a song.

In addition, the music playback screen is broadly divided into a music-related display section 68h, an upper display section 68i, and a lower display section 68j. The music-related display section 68h includes a lyrics display section 681 and a progress bar 682. The lyrics display section 681 displays the lyrics to be sung for the music being played. Note that the lyrics display section 681 may also display a background image. The progress bar 682 indicates the progress of music playback.

The area occupied by the upper display column 68i and the lower display column 68j can be changed. Normally, the state shown in FIG. 8(D) is displayed, that is, both the upper display column 68i and the lower display column 68j are displayed. When the comment tab 686, reservation confirmation tab 687, or remote control tab 688 provided in the lower display column 68j are operated, the lower display column 68j is displayed using the area where the upper display column 68i was displayed, as shown in FIG. 10. FIG. 10 shows the state where the remote control tab 688 has been operated. Operating the comment tab 686, reservation confirmation tab 687, or remote control tab 688 again returns to the state shown in FIG. 9(E).

The upper display area 68i displays a status display/selection button 683, a microphone selection button 684, and a user icon 685. The status display/selection button 683 is a button that indicates the status of the mobile terminal 6. When the mobile terminal 6 is set to the singer mode described above, "singing" is marked, and when the mobile terminal 6 is set to the listener mode, "listening" is marked. The state in FIG. 8(D) is a state in which "singing" is marked. The user can also manually switch between "singing", "listening", and "away". When the user switches to "singing", the setting is changed to the singer mode, and when the user switches to "listening" or "away", the setting is changed to the listener mode. In this embodiment, the three states can be switched, but the two statuses "singing" and "listening" may also be used, and various display forms can be adopted.

The microphone selection button 684 can be manually operated by the user to switch the microphone 67b on and off. In this embodiment, when the mode is set to singer mode, it is automatically set to on, and when the mode is set to listener mode, it is automatically set to off. The user icons 685 indicate all users participating in the room. In the example of FIG. 8(D), four users have participated. The user whose icon is indicated by a thick line ("JOY" in the example of FIG. 8(D)) indicates that he/she is singing (set to singer mode). Also, the icon with a speaker and a cross mark ("Monkichi" in the example of FIG. 8(D)) indicates that he/she is away. These display forms are merely examples, and status such as singing, listening, and away may be indicated by other display forms.

The lower display field 68j has three tabs (a comment tab 686, a reservation confirmation tab 687, and a remote control tab 688). In the state shown in FIG. 8(D), the comment tab 686 is selected, and a comment display field 689 and a comment input field 690 are displayed in the lower display field 68j. In addition to user comments entered by the user using the comment input field 690, the comment display field 689 displays system comments associated with operations related to the music playback process. In FIG. 8(D), the comment display field 689 displays system comments associated with the reservation process, including the user name and the reserved music/artist name. The comment display field 689 displays the system comments and user comments in chronological order, and users can communicate smoothly by viewing these system comments and user comments.

The comment input field 690 is also provided with a first button 690a and a second button 690b for displaying various effects in the area including the lyrics display field 681. When the first button 690a is operated, a first effect selection field for selecting a first display effect image is displayed on the operated mobile terminal 6. When the second button 690b is operated, a second effect selection field for selecting a second display effect image is displayed on the operated mobile terminal 6. The first display effect image and the second display effect image will be described in detail later.

Figure 10 (F) shows the state when the remote control tab 688 is operated. As mentioned above, the lower display column 68j is displayed by expanding it to the area where the upper display column 68i is displayed. When the remote control tab 688 is selected, the lower display column 68j displays the singer volume 691, speaker volume 692, pitch change button 693, BGM volume 694, playback position adjustment button 695, re-sing button 696, pause button 697, and stop performance button 698.

In this embodiment, the various controls and buttons displayed on the remote control tab 688 can only be operated from a mobile terminal 6 set to the singer mode. Therefore, operations related to music playback are prohibited from a mobile terminal 6 set to the listener mode, making it possible to prevent erroneous input by the listener.

The singer volume 691 is an operator that adjusts the volume of the singing voice input from the microphone 67b of the mobile terminal 6 that is set to the singer mode. The speaker volume 692 is an operator that adjusts the volume of the singing voice input from the microphone 67b of the mobile terminal 6 that is set to the listener mode. The singer volume 691 and speaker volume 692 are set to the default state for each mode when the singer mode or listener mode is set, but can only be operated from the mobile terminal 6 that is set to the singer mode.

The pitch change button 693 is a button for changing the pitch of the song being played. The BGM volume 694 is an operator for adjusting the volume of the BGM played between songs. The playback position adjustment button 695 is an operator for changing the playback position. The current playback time and total playback time are displayed to the left of the operators. The sing again button 696 can be operated to play the currently played song from the beginning. The pause button 697 is a button for temporarily pausing the currently played song. The stop performance button 698 is a button for stopping the currently played song, and when the stop performance button 698 is operated, playback processing moves to the next reserved song.

While a song is being played, a user who has been set to singer mode can use the various controls and buttons displayed on the remote control tab 688 in FIG. 10 (F) to perform various setting operations related to the song playback. In the flow diagram of FIG. 6, if a setting operation is performed while a song is being played (S105: Yes), the mobile terminal 6a sends a setting change instruction to the management server 11. The management server 11 executes the setting change in accordance with the received setting change instruction (S210).

Setting changes can be made not only by the management server 11, such as changing the pitch of a song, but also by changing settings on various mobile terminals 6a and 6b. For example, when a mobile terminal 6a set to singer mode changes the sound settings of the sound effects (echo, delay, etc.) on another mobile terminal 6b (set to listener mode), the mobile terminal 6a set to singer mode transmits sound setting information related to the sound effects to the management server 11 (S106). The management server 11 that receives the sound setting information transfers the sound setting information to the other mobile terminal 6b. The mobile terminal 6b that receives the sound setting information imparts sound effects to the singing voice signal or the played music sound based on the received sound setting information.

The management server 11 transmits (S211) a system comment based on the setting change (S210) to each of the mobile terminals 6a, 6b. On each of the mobile terminals 6a, 6b, the system comment is displayed in the comment display field 689 described in FIG. 8 (D) (S107, S155). Therefore, users participating in the room can check the operation related to the music playback process from the system comment displayed in the comment display field 689. Therefore, users participating in the room can communicate smoothly by checking the system comment.

When the management server 11 completes the music transmission (S212: Yes), it checks the reservation list to play the next music (S203). In addition, as the music playback transmission ends, each mobile terminal 6a, 6b ends the music playback (S108, S156: Yes).

Figure 11 is a flow diagram showing the effect display process of this embodiment. In this embodiment, in order to facilitate communication between users, it is possible to display user comments in the comment display field 689 described in Figure 9 (E). Users can convey messages to other users by inputting text in the comment input field 690. In this embodiment, based on the operation of the first button 690a and the second button 690b provided in the comment input field 690, effect images are displayed in the lyrics display field 681 etc., and similarly to messages, it is possible to convey emotions to other users and facilitate communication.

FIG. 12(G) is a display screen when the first button 690a is operated on the mobile terminal 6 of this embodiment. When the first button 690a is operated, the first effect selection field 710 is displayed. In the first effect selection field 710, a plurality of first effect images 711 are displayed, and the user can select from these a first effect image 711 that matches his/her own emotions when listening to the singing of other users. Since the first effect image 711 also has a text message attached, the user can convey his/her emotions about the singing to other users by selecting the first effect image 711 that matches his/her own emotions. Here, a case where a first effect image 711 with a dashed circular frame is selected in the first effect selection field 710 will be described.

When the first effect image 711 is selected, as shown by the dashed arrow in FIG. 12(H), an effect is created in which the selected first effect image 711 moves from the user icon 685 (in this case, "Pirori") of the selected user toward the lyrics display field 681. FIG. 12(H) shows the screen of the mobile terminal 6 of the selected user, but the effect of the selected first effect image 711 moving from the user icon 685 (in this case, "Pirori") toward the lyrics display field 681 is also created on the screens of the mobile terminals 6 of other users who are in the same room.

In this way, in this embodiment, by displaying the first effect image 711 moving from the position of the user icon 685 that selected the first effect image 711 into the lyrics display field 681, it becomes possible to convey to other users what emotion each user is expressing, thereby facilitating communication.

In this embodiment, the first effect image 711 is moved and displayed using the position of the lyrics displayed in the lyrics display field 681, or a lyrics display position that is set in advance in the lyrics display field 681. For example, by displaying the first effect image 711 so as to avoid these lyrics display positions, it is possible to prevent the visibility of the lyrics from being impaired by the first effect image 711. Note that when moving to avoid these positions, the first effect image 711 and the displayed lyrics may be partially overlapped to the extent that the visibility of the lyrics can be confirmed.

The first effect image 711 is displayed in the lyrics display field 681 for a predetermined period of time from the start of display, and then disappears from the lyrics display field 681 with a predetermined disappearance effect (e.g., gradually fading, etc.). The disappearance effect may be different for each first effect image 711. In this embodiment, if the same user selects the first effect image 711 multiple times within a short period of time, or if multiple users select the first effect image 711 multiple times within the same period of time, the selected multiple first effect images 711 are displayed.

Figure 13 (I) shows the display screen when the second button 690b is operated on the mobile terminal 6 of this embodiment. When the second button 690b is operated, the second effect selection field 720 is displayed. In the second effect selection field 720, samples of multiple second effect images 721 are displayed, and the user can select from these a second effect image 721 that matches his or her own emotions when listening to the singing of other users. Here, a case will be described where a second effect image 721 with a dashed circular frame is selected in the second effect selection field 720.

When the second effect image 721 is selected, as shown in FIG. 13(J), the second effect image 721 is displayed in the lyrics display field 681 as the background of the lyrics to be displayed. In this embodiment, the second effect image 721 is in an animated format with movement, and is displayed for a predetermined period of time after the user makes a selection.

Since the second effect image 721 is an effect image that is displayed using the entire area of the lyrics display field 681, if multiple second effect images 721 are displayed simultaneously, it is conceivable that the screen will become cluttered. For this reason, in this embodiment, the simultaneous display of multiple second effect images 721 is prohibited. Note that in this embodiment, it is possible to simultaneously display the first effect image 711 and the second effect image 721.

Returning to the flow diagram of the effect display process in FIG. 11, the display process of the first effect image 711 and the second effect image 721 described in FIG. 12 and FIG. 13 will be described. Note that the music playback period from S110 to S120 in the effect display process corresponds to the music playback period from S103 to S108 and S153 to S156 on the mobile terminals 6a and 6b described in FIG. 6.

When music playback begins (S110), the mobile terminal 6 confirms the selection of the first effect image 711 using the first effect selection field 710 (S111: Yes), the receipt of an operation instruction for the first effect image 711 from the other mobile terminal 6 (S114: Yes), the selection of the second effect image 721 using the second effect selection field 720 (S115: Yes), or the receipt of an operation instruction for the second effect image 721 from the other mobile terminal 6 (S119: Yes).

When a selection operation of the first effect image 711 is performed (S111: Yes), the mobile terminal 6 transmits an operation instruction to the management server 11 (S112). This operation instruction includes information about which user selected which first effect image 711. The management server 11 that receives the operation instruction transmits an operation instruction for the first effect image 711 to the mobile terminals 6 of other users who are in the same room.

Then, on the mobile terminal 6 on which the selection operation of the first effect image 711 has been performed, the first effect image 711 is displayed (S113). Similarly, when an operation instruction for the first effect image 711 is received from another mobile terminal 6 via the management server 11 (S114: Yes), the first effect image 711 is also displayed (S113).

On the other hand, when the second effect image 721 is selected (S115: Yes), the mobile terminal 6 first checks whether or not it is the period during which the second effect image 721 can be operated (S116). As described in FIG. 13, the second effect image 721 is an effect image that is displayed over the entire lyrics display field 681, and therefore in this embodiment, overlapping display is prohibited. For example, when the second effect image 721 is displayed during the selection operation (S115: Yes), it is determined that it is not the period during which the image can be selected (S116: Yes). On the other hand, when it is the period during which the image can be selected (S116: Yes), an operation instruction is sent to the management server 11, as in the case of the first effect image (S117). The management server 11 that has received the operation instruction sends an operation instruction for the second effect image 721 to the mobile terminals 6 of other users who are in the same room.

Whether or not it is the selection period for the second effect image 721 may be determined not only by the mobile terminal 6 but also by checking with the management server 11. If an operation instruction has already been received from another mobile terminal 6, it may be determined that it is not the selection period, thereby preventing the second effect image 721 from being selected multiple times due to delays in the communication line, etc.

The second effect image 721 is displayed on the mobile terminal 6 on which the selection operation of the second effect image 721 has been performed (S118). Similarly, when an operation instruction for the second effect image 721 is received from another mobile terminal 6 via the management server 11 (S119: Yes), the second effect image 721 is also displayed (S118).

In this embodiment, after the first effect image 711 is displayed (S113) and after the second effect image 721 is displayed (S118), information associated with the display of the first effect image 711 and the second effect image 721 is recorded and updated as effect history information (S120). In this embodiment, the effect history information is stored and managed on the management server 11. The effect history information includes information such as the target song, the user who sang (reserved), and which user selected the first effect image 711 or the second effect image 721 and at what point in time. By subsequently aggregating this effect history information, it is possible to perform an evaluation for each song or each user.

When the playback of the song ends (S121: Yes), a compilation process (S122) is performed on the effect history information recorded and updated during the song playback period. In the compilation process (S122) of this embodiment, the number of times the first effect image 711 was selected is compiled for the played song and the user who made the reservation (or the user who sang). In this embodiment, a type is set for the first effect image 711, and by compiling by type, it is possible to form an index of the emotions the user felt about the played song.

When the tallying process (S122) is completed, the effect history information and a result screen based on the tallying results are displayed on the touch panel display unit 68.

Figure 14 is a diagram showing a result screen based on the effect history information of this embodiment. In this embodiment, the played music is divided into unit intervals in chronological order, and the number of times the first effect image 711 was selected for each unit interval is displayed in a graph. A unit interval that is selected many times can be said to be the interval that has attracted the attention of users. In addition, the unit interval displays the first effect image 711 that was selected the most and the user icon 685 of the user who performed the most selection operations. These first effect image 711 and user icon 685 make it possible to confirm what emotions users had in the unit interval and which user was most interested.

Then, in the upper right corner of the result screen, a user ranking 731 and a singing ranking 732 based on the results of the tallying process (S122) are displayed. The user ranking 731 indicates a ranking for the users who sang the song (in this case, "Pirori") based on the total number of times the selected first effect image 711 was selected in the room created this time. The singing ranking 732 indicates a ranking for the song sung this time (in this case, "Inu no Omawarisan") in the room created this time based on the total number of times each type of first effect image 711 was selected.

In this way, in this embodiment, by displaying the results screen, it is possible to notify the user of various situations and rankings based on the specification of the first effect image 711 performed during song playback, and the user can review the state of their singing based on the results, or devote themselves to singing in the future in order to compete for rankings.

In this embodiment, the result screen described in FIG. 14 is displayed when the song ends, but it may be displayed in response to a user operation performed while participating in a room or at an appropriate time other than while participating in a room.

Figure 15 is a diagram showing a display screen of the tally results in another embodiment. In Figure 14, a ranking of the songs sung or the users who sang is displayed, but the tally results may be displayed and output in an appropriate format. Figure 15 (a) is a display screen of the tally results showing the "Today's Like! Ranking" tallied for the first effect image 711 displayed in the upper right. In Figure 15 (a), the number of times each sung song was liked for the type to which the first effect image 711 belongs (in this case, "Like!") is tallied, and a ranking of songs that were felt to be "Like!" by each song and user is displayed.

Fig. 15(b) is a display screen showing the tally results of the "Today's Good! Ranking" tallied for the first effect images 711 displayed in the upper right. In Fig. 15(b), for the type to which the two first effect images 711 belong (in this case, "Good!"), the number of times each song was sung is tallied, and a ranking of songs that the user felt were "Good!" is displayed.

Figure 15(c) shows a ranking of the number of times the first effect image 711 was selected when the user was singing, for each user participating in the room. Since all first effect images 711 are the subject of the ranking, the first effect image 711 that was the subject of the count is not displayed, as in Figures 15(a) and 15(b). Also, as in Figure 15(c), instead of ranking the users who were cheered on, cheering users, i.e., users participating in the room, may compete in a ranking format based on the number of times they selected the first effect image 711.

As described above, in this embodiment, by displaying user comments entered by user operations and system comments formed based on operations related to music playback processing in chronological order, good communication between users is possible, allowing users to fully enjoy karaoke.

This embodiment can employ various modified examples, which are described below. The modified examples are described below.

[First Modification]
The tallying process in the above embodiment is targeted only at the first effect image 711, but it may also be targeted at the second effect image 721. Also, in the above embodiment, the tallying process targets participants in a certain room and songs sung in that room, but the tallying process may not be limited to this form and may be targeted at all rooms held in the karaoke system, or any other suitable form may be adopted.

[Second Modification]
In the above embodiment, the selection operation of the first effect image 711 and the second effect image 721 can be performed regardless of the singer or the listener. However, a restriction may be set so that the selection operation of the first effect image 711 and the second effect image 721 can be performed only by the listener, that is, only by the mobile terminal 6 set in the listener mode. If the singer himself performs the selection operation of the first effect image 711 and the second effect image 721, his/her operation may be included in the tally. By setting the above restriction, it becomes possible to perform a fair evaluation.

Even if the singer is able to select the first effect image 711 and the second effect image 721, the singer's selection operation is not tallied, so that the singer can enjoy the generation of the effect image by the selection operation and a fair evaluation can be made.

[Third Modification]
In the effect display process described in the flow chart of FIG. 11, a selectable period (S116) is provided for the second effect image 721, so that the second effect image 721 is not displayed overlappingly. Instead of this form, the second effect image 721 may be displayed overlappingly like the first effect image 711. Also, instead of providing a selectable period, a reservation system may be used. The second effect image 721 selected first is displayed, and after the display of the second effect image 721 is completed, the second effect image 721 selected next is displayed, so that the second effect images 721 are displayed without overlapping.

[Fourth Modification]
Each user may be limited in the number of times of selection operation of the first effect image 711 and the second effect image 721. For example, by limiting the number of times of selection operation of each of the first effect image 711 and the second effect image 721 or the total number of both during the playback period of one song, the user can use the first effect image 711 and the second effect image 721 at an appropriate timing without selecting them indiscriminately. In addition, for various tabulations such as rankings, each user will carefully select the images while the number of times of selection is limited, and it is expected that the accuracy of the tabulations will be high.

6 (6a, 6b): Portable terminal 70b: Input switch 11: Management server 72: Vibration unit 12a: Router 73: LED
12b: Wireless router 110: Access point 13: Wireless base station 140: Mobile base station 61: CPU 681: Lyrics display field 62: RAM 682: Progress bar 63: Storage section 683: Selection button 64: Image processing section 684: Microphone selection button 65F: Front camera 685: User icon 65R: Rear camera 686: Comment tab 66: Sound processing section 687: Reservation confirmation tab 67a: Speaker 688: Remote control tab 67b: Microphone 689: Comment display field 68: Touch panel display section 690: Comment input field 68a: Room selection field 690a: First button 68b: New room creation button 690b: Second button 68c: Search category selection field 691: Singer volume 68d: Song selection field 692: Speaker volume 68e: Song confirmation field 693: Pitch change button 68f: Cancel button 694: BGM volume 68g: Reservation button 695: Playback position adjustment button 68h: Song-related display area 696: Button 68i: Upper display area 697: Pause button 68j: Lower display area 698: Performance stop button 68k: Song search button 710: First effect selection area 68m: Display section 711: First effect image 68n: Touch panel 720: Second effect selection area 69a: First communication section 721: Second effect image 69b: Second communication section 731: User ranking 70: Input switch 732: Singing ranking 70a: Input switch

Claims (6)

A karaoke program executable on a terminal device,
The device is capable of performing music playback processing, audio processing, communication processing, and display processing;
The music playback process plays and outputs music information.
The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit,
The communication process enables voice communication with other terminal devices;
The display process forms a lyrics display area on a display unit of the terminal device,
in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed on at least another terminal device, an effect image is displayed based on the effect display instruction;
The display process displays on the display unit of the terminal device an icon of a user set in the terminal device participating in the communication process, and when an effect display instruction operation is performed in at least another terminal device, displays an effect image that moves from the icon of the user who performed the effect display instruction operation to the lyrics display area. A karaoke program executable on a terminal device,
The device is capable of performing music playback processing, audio processing, communication processing, and display processing;
The music playback process plays and outputs music information.
The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit,
The communication process enables voice communication with other terminal devices;
The display process forms a lyrics display area on a display unit of the terminal device,
in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed at least on another terminal device, a first effect image or a second effect image, which has a different pattern from the first effect image and serves as a background for the displayed lyrics , is displayed based on the effect display instruction;
A karaoke program in which the first effect image and the second effect image are not prohibited from being displayed simultaneously, and a plurality of second effect images are prohibited from being displayed simultaneously. 3. The karaoke program according to claim 1, wherein the display position of the effect image in the lyrics display area is determined based on the position of the lyrics being displayed or a preset lyrics display position. 4. The karaoke program according to claim 1 , wherein the display process displays an effect image for a certain period of time after displaying an effect based on an effect display instruction. A karaoke device capable of reproducing and outputting music information,
The device is capable of performing music playback processing, audio processing, communication processing, and display processing;
The music playback process plays and outputs music information.
The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit,
The communication process enables voice communication with other terminal devices;
The display process forms a lyrics display area on a display unit of the terminal device,
in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed on at least another terminal device, an effect image is displayed based on the effect display instruction;
The display process displays on the display unit of the terminal device an icon of a user set in the terminal device participating in the communication process, and when an effect display instruction operation is performed in at least another terminal device, displays an effect image that moves from the icon of the user who performed the effect display instruction operation to the lyrics display area as a starting point in this karaoke device. A karaoke device capable of reproducing and outputting music information,
The device is capable of performing music playback processing, audio processing, communication processing, and display processing;
The music playback process plays and outputs music information.
The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit,
The communication process enables voice communication with other terminal devices;
The display process forms a lyrics display area on a display unit of the terminal device,
in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed at least on another terminal device, a first effect image or a second effect image, which has a different pattern from the first effect image and serves as a background for the displayed lyrics , is displayed based on the effect display instruction;
A karaoke device in which the first effect image and the second effect image are not prohibited from being displayed simultaneously, and a plurality of second effect images are prohibited from being displayed simultaneously.