JP4286446B2 - A computer with karaoke that uses a voice input melody search to select a song and make a reservation at a karaoke performance terminal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the melody retrieval technology of a sound input system by a client server system, which guides a user to sound-input correct melody, prevents a wasteful retrieval instruction from being given to a melody database retrieval processing means as much as possible and can usefully use the function of the system at the most. SOLUTION: The sound input analysis means of a client analyzes a sound signal with timing made by a metronome as a reference in a sound input reception process and calculates the synchronous degree of the pause of a musical time sequential change and metronome timing and gives an instruction to reinput sound to the user when the synchronous degree does not satisfy the reference. When the synchronous degree reaches the reference, a query including melody piece data obtained by expressing the musical time sequential change of the sound signal in an electronic music system is generated. The server retrieves a melody database with melody piece data in the query as a key, detects music and generates a retrieval report.

Description

[0001]
[Technical field to which the invention belongs]
The present invention relates to a karaoke accessory computer used together with a karaoke performance terminal in a karaoke store, and more particularly to a karaoke accessory computer having a function of selecting a song by voice input type melody search and making a reservation in a karaoke performance terminal.
[0002]
[Prior art]
For example, as described in detail in Japanese Patent Application Laid-Open No. 2000-172893, the following system for searching for music based on a melody inputted by voice has been recently developed. In this system, the melody data of a large number of songs is stored in a database, the humming and scat audio signals uttered by the user toward the microphone are analyzed by a computer, and the melody is extracted from the musical time-series changes of the audio signals. Then, the database is searched by using the melody as a clue, and a song including a similar melody is found. Combining this melody search system with a karaoke device makes it possible to request a song that has a distinctive melody but does not know the song name by voice input.
[0003]
[Problems to be solved by the invention]
In the online karaoke system, 20,000 songs can be played even in the present situation. A computer system that provides such a melody database with a large number of songs and realizes high-precision and high-speed melody search by voice input is a fairly advanced system, and is expensive in terms of both hardware and software. For this reason, it is difficult to attach this type of melody search system to each karaoke device, and it can be said that a client server system in which an advanced computer system is shared by many terminal devices is suitable.
[0004]
Analyzing Hamming and Scat voice input, extracting musical time-series changes to create melodic piece data, and using it as a clue to search a melody database with an enormous amount of data, the process is very complex Even if a client-server system is adopted, the load distribution design method similar to a general database search system concentrates the load on the server and cannot respond to search requests from a large number of clients at high speed.
[0005]
Unlike professional musicians, it is difficult for amateurs to use accurate melody. In many cases, the original melodies are not accurately remembered. In addition, the pitch and rhythm of the scat of a person who is timid will be quite crazy. Since many such inaccurate melodic voice inputs will be performed, as a result of searching the melodic database, it is often the case that music with high melodic similarity cannot be found. If all the processing that is wasted as a result is executed by the server, it is difficult to efficiently process meaningful processing with high efficiency.
[0006]
The present invention has been made in view of the above problems, and its purpose is to guide the user to input the most accurate melody as much as possible, and to useless search for the melody database search processing means. It is an object of the present invention to provide a voice input type melody search technique using a client server system in which instructions are given as little as possible and the functions of the system can be utilized to the maximum and meaningful.
[0007]
[Means for Solving the Problems]
The present invention relates to a karaoke-attached computer that selects a melody by voice input type melody search and reserves it at a karaoke performance terminal, and has a configuration described as the following items (1) to (10).
(1) It is a karaoke accessory computer provided with a first communication means, a second communication means, a display means, an operation input means, a voice input / output means, a music index database, and a control means. (1) The communication means can transmit the reservation command to the karaoke performance terminal. (3) The second communication means can communicate with the search server on the network. (4) The display means is used under the control of the control means. (5) The operation input means is capable of accepting user input. (6) The voice input / output means is configured to output the sound emitted by the user while outputting sound to the user. be detectable (7) music index database is a music ID, a title, aggregating the correspondence of music selection number for karaoke (8) control means, upon receiving a user input of a melody retrieval request The metronome sound is output from the voice input / output means, and the voice input from the voice input / output means is digitally recorded, and music score data in the form of electronic score is generated based on the recorded data, and the music search including the melody piece data is performed Sending the query to the search server (9) When the control means receives a search report including a song ID list from the search server responding to the song search query, each of the corresponding songs from the song index database based on the song ID list The song name is extracted and the song name list is displayed on the display means so that the user input of the performance reservation instruction specifying one song from the song name list can be accepted. (10) The control means can receive 1 from the song list being displayed. When the user input of the performance reservation instruction specifying the song is accepted, the music selection number corresponding to the song name is extracted from the song index database. And transmitting the music selection number and the reservation command to the karaoke performance terminal.
In the present invention, preferably, the voice input / output means has the same form as a telephone handset.
[0009]
In the present invention, preferably, in the process of digitally recording the input voice from the voice input / output means, the control means generates a graph image that can identify the total length of the recording period and the recorded section and displays it on the display means. I will let you.
[0010]
In the present invention, preferably, the control means is capable of accepting a user input of a redo command in a process of digitally recording an input voice from the voice input / output means. The process of digitally recording the input sound from the output means will be restarted from the beginning.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
=== System configuration example ===
FIG. 1 shows an example of a system configuration in which the present invention is applied to a karaoke group system installed in a karaoke store having a large number of guest rooms. A karaoke performance terminal 1 and a guest room versatile terminal 2 installed in each guest room and a single store server 3 installed in the store are connected via a LAN 8 to constitute a client server system.
[0012]
Each karaoke performance terminal 1 is connected to a public telephone line 4 directly or via a private branch exchange, and an inter-host communication control means for communicating with an external karaoke host device 5 connected to the public telephone line 4, and between the hosts Information processing means for storing karaoke data (including data for generating accompaniment music and data for displaying lyrics) stored in a storage device and managed as a karaoke music database by the communication control means from the karaoke host device 5; , A user interface for receiving the requested music and managing the queue, and a performance control means for extracting the karaoke data of the requested music from the karaoke music database according to the queue and outputting the accompaniment music and lyrics in synchronism with the audio video .
[0013]
The store server 3 is connected to an external network 6 such as the Internet and functions as a proxy server, and is operated on the external network 6 with each karaoke performance terminal 1 and each guest room versatile terminal 2 connected to the LAN. 7 relays TCP / IP communication with the network.
[0014]
Each guest room versatile terminal 2 includes an interactive GUI (graphical user interface) using a display with a touch panel, voice input / output means such as a telephone handset (handset), and the guest room versatile terminal 2. Means for presenting usable content (information service provided by the external server 7) to the user through an interactive GUI, and means for accessing the external server 7 providing the content selected by the user via the store server 3; Is provided. The store server 3 records the charging information of the paid content used in each guest room multipurpose terminal 2 and presents it to the manager in a timely manner.
[0015]
=== Outline of melody search process ===
In the “client server system for searching for music based on a melody inputted by voice” of this embodiment, each guest room versatile terminal 2 in the system configuration example of FIG. 1 is the client computer, and on the external network 6. A specific external server 7 in operation corresponds to the server computer (this is referred to as a search server 7). Each guest room versatile terminal 2 performs TCP / IP communication with the search server 7 via the in-store LAN 8, the store server 3, and the external network 6 to exchange music search queries and search reports.
[0016]
The search server 7 includes a melody database in which melody data of singing parts of a large number of songs expressed in an electronic score format such as MIDI is aggregated using song IDs as keys. Each guest room versatile terminal 2 has various information about the karaoke music to be searched (song name, composer, lyricist, genre, used movie, TV drama and commercial name, song selection number for karaoke performance terminal 1, etc.) Has a music index database in which music IDs are collected as keys.
[0017]
The user utters and inputs a melody by humming or scat using the handset according to the operation guide by the interactive GUI of the cabin multi-use terminal 2. The guest room versatile terminal 2 takes the input audio signal into the DSP, converts it into digital data, temporarily stores it in the memory, analyzes it, extracts the musical time-series change, and expresses the data in the form of an electronic musical score. Data is generated and a music search query including the data is delivered to the search server 7.
[0018]
The search server 7 that has received the music melody query searches the melody database using the melody piece data as a clue, lists music pieces including a melody pattern having a high similarity to the melody piece data, in order of similarity, A search report including a list of these music IDs is created and delivered to the query-use guest room multi-use terminal 2.
[0019]
Upon receiving the search report, the guest room multi-use terminal 2 extracts necessary information (song name, composer, songwriter, song selection number, etc.) from the song index database based on the list of song IDs reported in the report, and is interactive. A list of corresponding songs is displayed on the GUI screen in an appropriate layout. In this search result report screen, they are listed in the order of similarity of each music reported in the search report.
[0020]
When the user finds the intended song in the song names listed on the search result report screen, the user moves the cursor to the song name and enters a reservation command. Then, the user enters the karaoke performance terminal 1 in the same room from the guest room versatile terminal 2. A music selection number and a reservation command are transmitted, and a performance reservation is made.
[0021]
=== Voice input acceptance process ===
The processing procedure of the voice input reception process executed in the cabin multipurpose terminal 2 is shown in the flowchart of FIG. When the melody search menu is selected, a metronome video as illustrated in FIG. 3 is displayed on the interactive GUI screen. The metronome operates at an initial tempo, and a beeping electronic metronome sound is generated in synchronization with the video from the speaker of the handset. An operation guide for changing the tempo is also displayed on the screen of the metronome on the interactive GUI, and the tempo of the metronome can be arbitrarily changed by the user touching the screen. That is, the computer of the terminal 2 changes the tempo of the metronome video according to the user's touch input (steps 100, 101, and 108 in FIG. 2).
[0022]
The user appropriately adjusts the tempo of the metronome, and then utters the melody of the music to be searched by humming or scat in accordance with the timing of the metronome and inputs it to the microphone of the handset. The computer of the terminal 2 recognizes this and starts the analysis process while digitally recording the input voice for a time of 16 beats at the tempo engraved by the metronome. At the same time, as the recording progresses, a bar graph image or the like for showing to the user how far the recording has progressed is displayed on the screen of the interactive GUI. If the user thinks the melody utterance is wrong, he enters a redo command and returns to the top of the process. (The operation of this portion corresponds to steps 102, 103, and 104 in FIG. 2).
[0023]
The computer of the terminal 2 analyzes the audio signal for 16 beats, extracts the musical time-series change, and performs rhythm sound determination and pitch sound determination in the process of generating melodic piece data expressed in electronic score format. (Steps 105 and 106).
[0024]
In the rhythm sound detection process, the voice signal is analyzed based on the timing ticked by the metronome in the voice input acceptance process, and the degree of synchronization between the musical time-series change of the voice signal and the metronome timing is calculated. Then, it is determined whether or not the degree of synchronization meets the standard. Of course, a timing difference up to a certain extent is allowed, but if the deviation at each utterance timing equivalent to one note or the total deviation exceeds a standard, it is determined that the rhythm is unsatisfactory, and the interactive GUI is used to perform voice input again to the user. (Step 109).
[0025]
In the pitch sound determination process, it is determined based on a predetermined standard whether or not the pitch (pitch) change of the voice signal acquired from the user in the voice input reception process is suitable for a musical scale. As is well known, the pitch of music expressed by a score is a discrete value having a predetermined regularity. If the input voice contains a pitch that is too large to be normalized to the discrete value of such a scale, it is determined that this is a pitch sound, and the interactive GUI is used to instruct the user to repeat the voice input. (Step 110).
[0026]
If neither rhythm nor pitch is determined, the process is advanced and a song search including melodic piece data generated based on the musical time-series change of the audio signal is waited for input of a search command from the user. The query is delivered to the search server 7 (steps 107 and 112). Note that the user can also select to redo voice input without inputting a search command (steps 107 and 111).
[0027]
【The invention's effect】
(A) The voice signal (digital recording data) uttered by the user is not passed to the server computer, but the melodic piece data generated by the client computer based on the musical time-series change of the voice signal is passed to the server computer. Therefore, the load on the server computer is greatly reduced, the amount of data transmitted from the client to the server is remarkably reduced, and the system as a whole is highly efficient.
[0028]
(B) When a user utters a melody to be searched for by humming or scat, a video of a metronome that serves as a guide is presented, and the user can freely change the tempo, so voice input is extremely easy. This increases the probability that the intended melody can be correctly input. Therefore, the probability of requesting a useless search to the server is reduced.
[0029]
(C) At the processing stage of the client computer, if the input speech is determined to be rhythmic based on the timing of the metronome, the user is prompted to input the speech without issuing a query based on the input speech to the server. Let me start over. Therefore, the probability of requesting a useless search to the server is reduced, and the server is consequently more likely to execute a meaningful search.
[0030]
(D) If the pitch change of the input voice of the user deviates significantly from the musical scale characteristic at the processing stage of the client computer, the query is not issued to the user without issuing a query based on the input voice to the server. Retry voice input. Therefore, the probability of requesting a useless search to the server is reduced, and the server is consequently more likely to execute a meaningful search.
[Brief description of the drawings]
FIG. 1 is a diagram showing a system configuration example in which the present invention is applied to a karaoke collective system installed in a karaoke store having a large number of guest rooms.
FIG. 2 is a flowchart showing a control procedure of a voice input reception process executed by a client computer in the melody search system of the present invention.
FIG. 3 is a diagram showing an example of a metronome video displayed on a screen of an interactive user interface of a client computer in the present invention.
[Explanation of symbols]
1 Karaoke performance terminal 2 Guest room versatile terminal (client computer of the present invention)
3 store server 4 public telephone line 5 karaoke host device 6 external network 7 external server (one of which is the server computer of the present invention)
8 In-store LAN

Claims (4)

A karaoke-attached computer comprising a first communication means, a second communication means, a display means, an operation input means, a voice input / output means, a music index database, and a control means,
The first communication means can transmit a reservation command to the karaoke performance terminal,
The second communication means can communicate with a search server on the network,
The display means displays information to the user under the control of the control means,
The operation input means can accept user input,
The voice input / output means can detect the voice emitted by the user while outputting sound toward the user,
Music index database, aggregation and music ID, the song title, the association of music selection numbers for karaoke,
When receiving the user input of the melody search request, the control means outputs the metronome sound from the voice input / output means and digitally records the input voice from the voice input / output means, and generates an electronic score format based on the recorded data. Generate melodic piece data, send a music search query containing the melodic piece data to the search server,
When the control means receives the search report including the music ID list from the search server responding to the music search query, the control means extracts the music name of each corresponding music from the music index database based on the music ID list and displays the music name list on the display means. And accepting user input for performance reservation instructions that specify one song from the song name list,
When the control unit accepts a user input of a performance reservation instruction specifying one song from the displayed song list, the control unit extracts the song selection number corresponding to the song name from the song index database, and the song selection number and the reservation command are extracted from the karaoke. A computer with karaoke that is sent to the performance terminal. The karaoke-attached computer according to claim 1, wherein the voice input / output means has the same form as a telephone handset. The control means generates a graph image capable of identifying the total length of the recording period and the recorded section in the process of digitally recording the input voice from the voice input / output means and displays the graph on the display means. Computer with karaoke as described. In the process of digitally recording the input voice from the voice input / output means, the control means can accept the user input of the redo command, and when receiving the user input of the redo command, the control means receives the input voice from the voice input / output means. The karaoke-attached computer according to any one of claims 1 to 3, wherein the digital recording process is restarted from the beginning.

Images