JP6073145B2 - SINGING VOICE DATA GENERATION DEVICE AND SINGING MOVIE DATA GENERATION DEVICE - Google Patents

Description

  The present invention relates to a singing voice data generating device for generating singing voice data during karaoke singing. The present invention also relates to a singing moving image data generating device for generating singing moving image data from singing sound data generated by a singing sound data generating device and moving image data at the time of karaoke singing taken by a user video camera.

  In recent years, in the karaoke industry, services are provided for recording video and audio during karaoke singing and publishing them as singing video content. In this service, shooting is performed with a video camera installed in a karaoke room, but the singing video content obtained is similar because the installation position and number of video cameras are limited.

  In order to obtain unique singing video content, it may be possible to publish the user video content shot by the user in the karaoke room, but the sound quality is far from the singing video content obtained by the above service, and it is satisfactory It was not a thing. Further, as described in Patent Document 1, as a technique for using a user video or a still image taken by a user with a commercial karaoke system, the user video is used only as a background video at the time of karaoke singing. It was.

Japanese Patent Laid-Open No. 2006-3843

  Thus, in the conventional karaoke system, it is difficult to obtain satisfaction in terms of sound quality when trying to obtain unique singing moving image content, and it is difficult to obtain unique singing moving image content when emphasizing sound quality. Here, if a new singing video data can be generated by synthesizing the video portion of the user video data and the singing voice data acquired by the karaoke device, unique and high-quality singing video content can be obtained. An effective method for synthesizing singing voice data has not been proposed.

  The present invention has been made in view of such circumstances, and its purpose is to obtain singing voice data suitable for synthesis with user video data, and to provide singing video content with unique and high-quality sound. There is to get.

  In order to achieve the above-mentioned object, the present invention is a singing voice data generating device for generating singing voice data at the time of karaoke singing synthesized with moving picture data at the time of karaoke singing shot by a user video camera. A synchronization sound output means for outputting a synchronization sound having a frequency outside the audible frequency range and recordable by the user video camera; and recording the singing sound during the karaoke singing together with the synchronization sound; And singing voice data generating means for generating.

  According to the present invention, since each of the moving image data photographed by the user video camera and the singing sound data generated by the singing sound data generating device includes the synchronizing sound, the moving image data is based on these synchronizing sounds. The time of the video portion and the singing voice data can be synchronized. Thereby, the singing voice data suitable for the synthesis with the user moving image data can be acquired.

  In the above singing voice data generation apparatus, it is preferable that the synchronization sound output means repeatedly outputs the synchronization sounds set in different output patterns at intervals. In this configuration, the user moving image data and the singing voice data can be synchronized at a plurality of times in the karaoke song according to the pattern of the synchronization sound.

  In the above singing voice data generating device, the singing voice data generating means includes a mixing unit that mixes a singing voice signal obtained by converting the singing voice with a microphone and a synchronizing sound signal that is a basis of the synchronizing sound, and is mixed. The singing voice data is preferably generated based on a later signal. In this configuration, since the singing voice signal and the synchronization sound signal are mixed by the mixing unit, the synchronization sound signal can be mixed at an appropriate level.

  In addition, the singing video data generating apparatus of the present invention acquires the singing voice data generated by the above singing voice data generating apparatus and the video data photographed by the user video camera at the time of karaoke singing corresponding to the singing voice data. Recognizing the output timing of the synchronized sound in the moving image data based on the audio portion of the moving image data and recognizing the output timing of the synchronized sound in the singing sound data based on the singing sound data Output timing recognition means, the output timing of the synchronized sound in the moving image data, and the output timing of the synchronized sound in the singing audio data, the video portion of the moving image data and the singing audio data It has video data generation means to synthesize and generate new song video data It is characterized in.

  According to the present invention, a singing moving image content having a unique and high sound quality can be obtained by synthesizing a video portion of moving image data and singing sound data.

  The singing video data generating device of the present invention includes the singing voice data generated by the singing voice data generating device, and a plurality of video data shot by a user video camera during karaoke singing corresponding to the singing voice data. Based on the sound part of the plurality of moving image data, and recognizing the output timing of the synchronized sound in each moving image data, and based on the singing sound data, the plurality of the singing sound data In a state where the output timing recognition means for recognizing the output timing of the synchronization sound, the output timing of the synchronization sound in the plurality of moving image data, and the output timing of the plurality of synchronization sounds in the singing voice data are aligned, The video part of the video data is connected and synthesized with the singing voice data Singing video data generating apparatus characterized by having a motion picture data generating means for generating a new singing video data.

  According to the present invention, individual and high-quality singing moving image content can be obtained by synthesizing video portions of a plurality of moving image data and singing voice data.

  According to the present invention, singing voice data suitable for synthesis with user moving image data can be acquired. In addition, it is possible to obtain a unique and high-quality singing moving image content.

It is a conceptual diagram explaining schematic structure of a karaoke system. It is a block diagram explaining the structure of a host apparatus (server). It is a block diagram explaining the structure of a karaoke apparatus. It is a block diagram explaining the structure of a karaoke main body. It is a figure explaining an acoustic processing part. It is a block diagram explaining the structure of a user terminal. (A) is a conceptual diagram explaining the relationship between a recordable zone | band and an audible zone zone | band. (B) is a conceptual diagram explaining a synchronous sound signal and a singing voice signal. It is a figure explaining the outline | summary of a process of a karaoke system. It is a conceptual diagram explaining the synthesis | combination of the video part of moving image data, and singing voice data. (A) is a conceptual diagram explaining the synchronous sound signal and singing voice signal in 2nd Embodiment. (B) is a conceptual diagram explaining the detail of the synchronous sound signal in 2nd Embodiment. It is a conceptual diagram explaining the synthesis | combination of the image | video part of moving image data and song audio | voice data in 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described. The karaoke system shown in FIG. 1 has a host device 1, a karaoke device 2, and a user terminal 3. And these are connected via the communication line 4 so that communication is possible. In this karaoke system, singing video data (a set of singing video data and singing voice data) acquired by karaoke singing is uploaded from the karaoke device 2 to the host device 1 (server) and can be viewed through the user terminal 3. Data is generated. Moreover, according to a user's request | requirement, the video part of the user moving image data image | photographed with the user video camera and the song audio | voice data acquired with the karaoke apparatus 2 are synthesize | combined, and new song moving image data may be produced | generated. it can. Hereinafter, each apparatus 1-3 which comprises a karaoke system is demonstrated.

  First, the host device 1 will be described. The host device 1 functions as a server and accumulates and manages various types of information such as customer information and singing moving image data. As illustrated in FIG. 2, the host device 1 includes a host-side control unit 11, a host-side communication unit 12, and a host-side storage unit 13.

  The host-side control unit 11 is a central part of control in the host device 1 and includes a CPU 11a and a memory 11b. The CPU 11a executes various controls according to the operation program stored in the memory 11b. The memory 11b is a storage element that stores a program to be executed by the CPU 11a and temporarily stores various types of information when the program is executed. The host-side communication unit 12 provides an interface for connecting the host device 1 to the communication line 4.

  The host-side storage unit 13 is a storage device that stores a large amount of information, and includes a hard disk drive or the like. A partial area of the host-side storage unit 13 is used as a customer information storage area, a singing voice data storage area, a singing video data storage area, a user moving image data storage area, and a browsing moving image data storage area.

  In the customer information storage area, personal information such as gender, age, date of birth, address, etc., use store, use date, singing history, etc. in a state associated with the user ID which is identification information for each singer The history data is stored.

  The singing voice data storage area stores singing voice data acquired during karaoke singing, and the singing video data storage area stores singing video data acquired during karaoke singing. In this embodiment, MP3 format data is used for singing voice data, and MPEG2 format data is used for singing video data. These singing voice data and singing video data are singing moving image data that is the basis of the singing moving image content, and are uploaded from the karaoke apparatus 2 when the karaoke singing ends. And each data is memorize | stored in the state matched with various information, such as music ID and user ID which are identification information for every karaoke music.

  In the user moving image data storage area, user moving image data shot by a user video camera at the time of singing karaoke is stored. In the present embodiment, the user moving image data is transmitted through the user terminal 3 or transmitted from the user video camera UC (see FIG. 3). In addition, if user video data can be isolate | separated into each data of a song video part and a song voice part by conversion, the data of arbitrary formats can be utilized.

  In the browsing moving image data storage area, browsing moving image data for allowing the user terminal 3 to browse is stored. This browsing moving image data is data that is the basis of the singing moving image content reproduced by the user terminal 3, and is, for example, moving image data in WMV format or MP4 format. In this embodiment, two types of browsing moving image data are stored in the browsing moving image data storage area. Specifically, the video data for browsing created based on the singing video data and the singing voice data acquired by the karaoke device 2, the user video data captured by the user video camera UC, and the karaoke device 2 Movie data for browsing created based on singing voice data is stored.

  Next, the karaoke apparatus 2 will be described. This karaoke device 2 is a device for recording video and audio during karaoke performance and singing, and is installed in each karaoke room RM of a karaoke store KB as shown in FIG. As shown in FIG. 3, the karaoke apparatus 2 includes a karaoke main body 21, a speaker 22, a monitor 23, a microphone 24, a video camera 25, and a remote control device 26.

  The karaoke main body 21 is a part that performs various controls relating to karaoke singing, such as performance control of selected karaoke music, display control of lyrics and background video, and processing of a singing voice signal input through the microphone 24. The karaoke main body 21 will be described in detail later.

  The speaker 22 is connected to the karaoke main body 21 and emits sound based on the sound emission signal from the karaoke main body 21. The monitor 23 is also connected to the karaoke main body 21 and displays video on the screen based on the video signal from the karaoke main body 21. The microphone 24 is also connected to the karaoke main body 21, and converts the voice of the singer into an analog singing voice signal and causes the karaoke main body 21 to input it. The microphone 24 having such a function corresponds to singing voice input means. The video camera 25 is also connected to the karaoke main body 21, and the video being sung is converted into a singing video signal and input to the karaoke main body 21. The video camera 25 having such a function corresponds to a singing video photographing means.

  The remote control device 26 includes a short-range wireless communication unit capable of bidirectional communication for transmitting and receiving information to and from the karaoke main body 21 and is operated when a karaoke song is reserved. When reserving karaoke music, the remote control device 26 transmits an operation signal including a music ID for identifying the music to be played. When a registered user logs in through the remote control device 26, the user ID of the user is transmitted to the host device 1. The remote controller 26 displays various types of information registered in My Page.

  Further, the remote control device 26 of the present embodiment also functions as the user terminal 3. In this case, the remote control device 26 is connected to the host device 1 via a nearby access point (not shown) and the communication line 4. For this reason, the user can access My Page while staying at a karaoke shop.

  FIG. 3 also shows a user video camera UC in addition to the karaoke apparatus 2. This user video camera UC is owned by a karaoke user, and records video and audio during karaoke singing as user video data. The user video camera UC only needs to be capable of recording video and audio during karaoke singing and recording in a band exceeding the human audible band (20 Hz to 15 kHz). For this reason, a digital video camera UC1 and a mobile phone UC2 with a moving image shooting function are included. Also, a digital still camera with a moving image shooting function may be used.

  Next, the karaoke main body 21 will be described in detail. As shown in FIG. 4, the karaoke main body 21 includes a main body side control unit 31, a main body side communication unit 32, a main body side storage unit 33, an acoustic processing unit 34, a display processing unit 35, and a video input unit 36. And an operation unit 37. These units are connected to a communicable state via the data bus BU.

  The main body side control unit 31 is a central part of control in the karaoke main body 21, and includes a CPU 31a and a memory 31b. The CPU 31a executes various controls according to the operation program stored in the memory 31b. For example, an operation input process for receiving an operation from the operation unit 37 and a sequencer process that operates as a sequencer are performed. The memory 31b is a storage element that stores a program to be executed by the CPU 31a and temporarily stores various types of information when the program is executed. The main body side communication unit 32 provides an interface for connecting the karaoke main body 21 (the karaoke apparatus 2) to the communication line 4. The operation of the main body side communication unit 32 is controlled by the main body side control unit 31.

  The main body side storage unit 33 is a large-capacity storage device that stores various types of data, and includes, for example, a hard disk drive. The main body side storage unit 33 includes, for example, a music data storage area, a synchronization sound data storage area, a background video data storage area, a singing video data storage area, a singing voice data storage area, and a collaboration video data storage area.

  The song data storage area stores song data for playing the karaoke song selected by the remote control device 26, and the synchronization sound data storage area stores the synchronization sound for synchronizing the user moving image data and the singing voice data. Is stored, and background video data for displaying the background video on the monitor 23 is stored in the background video data storage area. Here, the synchronization sound will be briefly described. This synchronized sound is a sound that is set to a frequency that is outside the audible band and can be recorded by the microphone of the user video camera UC.

  Singing video data taken by the video camera 25 is stored in the singing video data storage area, and singing voice data is stored in the singing voice data storage area. The singing voice data of the present embodiment includes digitalized singing voice and digitalized singing voice with karaoke accompaniment. In the collaboration video data storage area, the song video data received from the host device 1 for the collaboration song is stored. This song moving image data is reproduced when a collaboration song is performed.

  The acoustic processing unit 34 is a part that controls performance of karaoke music, or performs control such as converting a singing voice signal input through the microphone 24 into digital singing voice data. As shown in FIG. 5, the acoustic processing unit 34 includes a first mixing unit 34a and a second mixing unit 34b.

  The first mixing unit 34a receives music data (MIDI data), synchronized sound data, and a singing voice signal from the microphone 24, a musical sound signal based on the music data, a synchronized sound based on the synchronized sound data, and a singing The audio signal is mixed with an appropriate balance and output to the speaker 22 as a sound emission signal. Thereby, the accompaniment and sound accompanying karaoke singing are output from the speaker 22. In addition, the above-described synchronization sound is also output from the speaker 22.

  Thus, the set of the main body side storage unit 33, the sound processing unit 34 (first mixing unit 34a), and the speaker 22 stores the music data and the synchronization sound data, and stores the musical tone signal and the synchronization sound data based on the music data. This is equivalent to a synchronization sound output means because a sound emission signal is output by synthesizing a synchronization sound based on the sound output, and a synchronization sound and karaoke accompaniment are emitted based on the sound emission signal.

  The music data, the synchronization sound data, and the singing voice signal from the microphone 24 are also input to the second mixing unit 34b. And from the 2nd mixing part 34b, the singing voice data based only on a singing voice signal and the above-mentioned musical sound signal, a synchronous sound, and the singing voice data with accompaniment based on a singing voice signal are output. Accordingly, the singing voice data storage area stores singing voice data in which singing voice is recorded, and singing voice data with accompaniment in which karaoke accompaniment, synchronization sound, and singing voice are recorded.

  In this way, the acoustic processing unit 34 (second mixing unit 34b) mixes the musical tone signal, the synchronization sound signal, and the singing voice signal, and generates the singing voice data with accompaniment, so the singing voice at the time of karaoke singing Is recorded together with the synchronization sound and corresponds to singing voice data generating means for generating voice data.

  The display processing unit 35 shown in FIG. 4 performs control such as display of a background video during karaoke performance. At the time of karaoke performance, background video data is input to the display processing unit 35, and the background video data is decoded. Then, the display processing unit 35 combines the lyrics telop with the video signal of the background video generated by the decoding, and outputs the combined video signal to the monitor 23. As a result, the monitor 23 displays a video in which lyrics telop is superimposed on the background video.

  The video input unit 36 is a part that performs a process of taking a singing video signal from the video camera 25, and is constituted by a so-called video card. The video input unit 36 includes, for example, an external interface for connecting to the video camera 25, an internal interface for connecting to the main body side control unit 31 and the like through the data bus BU, and a video memory as a frame buffer. (Both not shown).

  The operation unit 37 includes a panel switch, a remote control receiving circuit, and the like, and outputs an operation signal corresponding to the operation of the panel switch or the remote control device 26 by the user to the main body side control unit 31. The main body side control unit 31 detects an operation signal by performing an operation input process, and executes a corresponding process. Note that the panel switch and the remote control device 26 include various key switches (not shown) for selecting an operation.

  And it can instruct | indicate the recording of the song moving image at the time of karaoke singing through the login process through the remote control device 26. By instructing the recording of the singing video, the singing video data and the singing voice data acquired along with the singing of the target karaoke song are stored in the singing video data storage area and the singing voice data storage area of the main body side storage unit 33. Is done. Furthermore, after the music ends, the stored singing video data and singing voice data are uploaded to the host device 1.

  Next, the user terminal 3 will be described. The user terminal 3 is a communication terminal operated by a karaoke user, and is operated when browsing the singing video content. As shown in FIG. 1, the user terminal 3 includes a personal computer 3A, a tablet computer 3B, a mobile phone 3C, and the like. Further, as described above, the remote control device 26 is also included in the user terminal 3.

  As illustrated in FIG. 6, the user terminal 3 includes a terminal-side control unit 41, a terminal-side communication unit 42, a terminal-side storage unit 43, a display unit 44, and an input unit 45.

  The terminal-side control unit 41 is a central part of control in the user terminal 3, and includes a CPU 41a and a memory 41b. The CPU 41a executes various controls according to the operation program stored in the memory 41b. The memory 41b is a storage element that stores a program to be executed by the CPU 41a and temporarily stores various types of information when the program is executed. The terminal side communication unit 42 provides an interface for connecting the user terminal 3 to the communication line 4. The terminal-side storage unit 43 is a storage device that stores various types of information, and includes a nonvolatile memory, a hard disk drive, and the like.

  The display unit 44 and the input unit 45 are parts that provide a user interface. The display unit 44 is a part that displays various menus and the like, and is composed of, for example, a liquid crystal display device or a cathode ray tube. The input unit 45 is a part that captures an operation performed by a user as operation information, and includes, for example, a keyboard, a mouse, and a touch panel. In the tablet computer 3B and the remote control device 26 that require portability, a liquid crystal display panel with a touch panel in which the display unit 44 and the input unit 45 are integrated is preferably used.

  Next, the operation of the karaoke system having the above configuration will be described. In this karaoke system, singing voice data suitable for synthesis with user video data photographed by the user video camera UC is generated, and new singing video data is synthesized by synthesizing the singing voice data and the video portion of the user video data. It has the feature in the point which produces | generates.

  Hereinafter, a series of processes will be described with reference to the flowchart of FIG. First, when recording of a singing moving image is instruct | indicated by operation of the operation part 37 by the user of karaoke (S1), the karaoke apparatus 2 manages the performance of the said karaoke music in a queue. Then, when the order of the karaoke music comes, the karaoke device 2 starts karaoke accompaniment (S2). At that time, the karaoke apparatus 2 outputs a synchronized sound at an arbitrary timing in the karaoke accompaniment. In this embodiment, the synchronized sound is included and output immediately after the performance of the karaoke accompaniment starts. Thereby, the karaoke accompaniment including the synchronization sound is emitted from the speaker 22. Further, the synchronization sound signal is mixed with the singing voice signal from the microphone 24.

  As described above, the frequency of the synchronization sound is set to a value that is outside the audible band and can be recorded by the microphone of the user video camera UC. For this reason, the synchronized sound emitted from the speaker 22 is difficult to hear from the singer and is recorded in the user moving image data photographed by the video camera 25. The synchronized sound is also recorded in the singing voice data acquired by karaoke singing.

  When the karaoke performance is finished (S3), various data (singing video data, singing voice data, accompaniment) stored in the main body side storage unit 33 (singing video data storage area, singing voice data storage area) of the karaoke apparatus 2 are provided. Singing voice data) is uploaded to the host device 1 (S4) and stored in the host-side storage unit 13 (singing video data storage area, singing voice data storage area).

  Thereafter, user moving image data shot by the user video camera UC is registered in the host-side storage unit 13 (user moving image storage area) of the host device 1 (S5). For example, when the user video camera UC is a digital video camera UC1 or a digital still camera (not shown), user video data stored in the user video camera UC is transmitted via the user terminal 3 (personal computer 3A). To the host device 1.

  If the user video camera UC is a tablet computer 3B or mobile phone UC2 (3C) having a camera function and a communication function, a two-dimensional barcode is displayed on the monitor 23, and the two-dimensional barcode is photographed. By doing so, the user moving image data may be transferred to the designated transmission destination address. Further, the transmission destination URL may be transmitted by e-mail, and the user moving image data may be transferred through the web page of the transmission destination URL.

  Next, the host device 1 acquires various data, synthesizes the video portion of the user moving image data and the singing voice data with accompaniment, and generates new singing moving image data (S6). At this time, the host-side control unit 11 functions as a data acquisition unit, and acquires the singing voice data generated by the karaoke apparatus 2 and the moving image data shot by the user video camera UC. Next, the host-side control unit 11 functions as output timing recognition means, recognizes the output timing of the synchronized sound in the user moving image data based on the sound portion of the user moving image data, and sings sound data based on the singing sound data. Recognize the output timing of the sync tone. Next, the host-side control unit 11 functions as moving image data generating means, and the video of the user moving image data is aligned with the synchronized sound output timing in the user moving image data and the synchronized sound output timing in the singing sound data. The part and the singing voice data are synthesized to generate new singing video data. Then, the generated new song moving image data is stored in the browsing moving image data storage area of the host-side storage unit 13.

  Next, a description will be given of the synchronization sound used in the processing described above, and the synthesis of the video portion of the user moving image data and the singing voice data with accompaniment.

  First, the synchronization sound will be described. As shown in FIG. 8A, the synchronization sound is set to a frequency outside the human audible frequency band and within the frequency band that can be recorded by the user video camera UC. For example, when the audible frequency band is set to 20 Hz to 15 kHz and the recordable frequency band is set to 10 Hz to 20 kHz, the synchronization sound has a frequency F1 of 10 Hz to less than 20 Hz on the low frequency side, and a frequency of 20 kHz to 20 kHz on the high frequency side. F2 is determined respectively. Similarly, when the audible frequency band is set to 15 Hz to 20 kHz and the recordable frequency band is set to 5 Hz to 30 kHz, the synchronization sound has a frequency F1 of 5 Hz to less than 15 Hz on the low frequency side and higher than 20 kHz and lower than 30 kHz on the high frequency side. It is determined for each frequency F2. From the viewpoint of shortening the output time of the synchronization sound as much as possible, it is preferable to use the high frequency side frequency.

  And the karaoke apparatus 2 outputs a synchronous sound immediately after the accompaniment start in the target karaoke music in the case of recording a song animation. In this case, the acoustic processing unit 34 reads the music data and the synchronization sound data from the main body side storage unit 33, and mixes the musical sound signal based on the music data and the synchronization sound based on the synchronization sound data. As a result, as shown in FIG. 8B, the sound emission signal WV immediately after the start of accompaniment includes the synchronization signal SY and the musical sound signal AC.

  From the speaker 22, a synchronized sound and a karaoke accompaniment corresponding to the sound emission signal WV are emitted. Here, since the frequency of the synchronization sound is outside the audible frequency band and within the recordable frequency band, it is difficult for the singer to hear, but is recorded as a synchronization sound by the user video camera UC. The In addition, the acoustic processing unit 34 mixes the singing voice signal input through the microphone 24 with the musical sound signal AC based on the music data and the synchronized sound signal SY based on the synchronized sound data. Thereby, in addition to the karaoke accompaniment and the singing voice, the synchronized sound is also recorded in the singing voice data with accompaniment output from the acoustic processing unit 34.

  Next, the synthesis of the video portion of the user moving image data and the singing voice data with accompaniment will be described. FIG. 9 is a conceptual diagram illustrating the synthesis of the video portion of the user moving image data and the singing voice data performed in the host device 1. The host-side storage unit 13 stores user moving image data and accompaniment song voice data. At the time of synthesis, the host-side control unit 11 reads the user moving image data and the accompaniment singing voice data from the host-side storage unit 13. The user moving image data is divided into video data VD and audio data WV1. In this embodiment, user moving image data is converted into video data VD in MPEG2 format and audio data WV1 in MP3 format. This is because the song video data uploaded from the karaoke apparatus 2 is in the MPEG2 format and the song audio data is in the MP3 format.

  If the user moving image data is converted into the video data VD and the audio data WV1, the host-side control unit 11 recognizes the output timing of the synchronization sound. As described above, since the synchronization sound is defined at a specific frequency (for example, frequency F2), the host-side control unit 11 adds the audio data WV1 based on the user moving image data and the singing audio data WV2 with accompaniment, respectively. On the other hand, a synchronized sound can be extracted by applying a bandpass filter (program) corresponding to a specific frequency. Thereby, the output timing of the synchronous sound in user moving image data and the output timing of the synchronous sound in accompaniment song voice data WV2 can be recognized.

  If the output timing of the synchronized sound is recognized, the host-side control unit 11 synthesizes the video data VD1 based on the user video data and the singing voice data WV2 with accompaniment to generate new singing video data. At the time of synthesis, the host-side control unit 11 matches the output timing of the synchronization sound signal SY1 in the video data with the output timing of the synchronization sound signal SY2 in the accompaniment singing sound data WV2. Thereby, new song moving image data is generated in a state in which the time of the video portion and the audio portion are synchronized. Then, the host side control unit 11 stores the generated new song moving image data in the browsing moving image data storage area of the host side storage unit 13.

  In the singing moving image data obtained by the above procedure, the video portion (video data VD) of the user moving image data photographed by the user and the singing voice data WV2 with accompaniment acquired by the karaoke apparatus 2 are synthesized. Therefore, a high-quality singing moving picture content with a high degree of freedom can be obtained for the video.

  As is apparent from the above description, according to the karaoke system of the present embodiment, the synchronized sound signal is transmitted to each of the user moving image data photographed by the user video camera UC and the accompaniment singing voice data WV2 generated by the karaoke apparatus 2. Since SY1 and SY2 are included, the time of the video data VD based on the user video data and the singing voice data WV2 with accompaniment can be synchronized based on the synchronization sound signals SY1 and SY2.

  In addition, the acoustic processing unit 34 of the karaoke apparatus 2 includes a second mixing unit 34b that mixes a singing voice signal obtained by converting the singing voice by the microphone 24 and a synchronizing sound signal SY that is a basis of the synchronizing sound. Singing voice data is generated based on the later signal. Thereby, the synchronization sound signal SY can be mixed into the singing voice signal at an appropriate level.

  The above description of the embodiment is for facilitating the understanding of the present invention, and does not limit the present invention. The present invention can be changed and improved without departing from the gist thereof, and the present invention includes equivalents thereof.

  For example, the synchronization sound in the first embodiment described above was output immediately after the start of karaoke accompaniment. However, it is not limited to this configuration. For example, the synchronization sound may be repeatedly output at intervals. Hereinafter, the second embodiment configured as described above will be described. Since the system configuration of the second embodiment is the same as that of the first embodiment, the description thereof is omitted. The difference from the first embodiment is in the output mode of the synchronization sound signal and the use of the synchronization sound. Therefore, the following description will be made on these differences.

  As shown in FIG. 10 (a), the sound emission signal WV in the second embodiment is obtained by repeatedly synthesizing a plurality of synchronized sound signals SYa and SYb with the musical sound signal AC. In the present embodiment, the synchronization sound signal SY is synthesized at a predetermined synchronization interval (30 seconds interval). And as shown in FIG.10 (b), each synchronous sound signal SYa-SYc is each set to the different output pattern. In this example, the number of large amplitude portions is different for each of the synchronization sound signals SYa to SYc. For this reason, each sync signal SYa to SYc can be identified based on the number of large amplitude portions. Note that the output patterns of the synchronization sound signals SYa to SYc may be in other forms as long as the host side control unit 11 can identify them. For example, the frequency may be different for each synchronization sound signal.

  And in this embodiment, as shown in FIG. 11, based on four types of synchronous sound signals SY1a to SY1d and SY2a to SY2d included in each audio data, images of four user moving image data A to D having different shooting periods are used. Composite video data is generated by combining the parts. That is, the host-side control unit 11 aligns the output timings of the synchronized sound signals SY1a to SY1d in the four user moving image data A to D with the output timings of the synchronized sound signals SY2a to SY2d in the accompaniment singing sound data WV21. Thus, the video data VD1 to VD4 based on the respective user moving image data A to D are connected. And based on synthetic | combination video data VD11 and accompaniment song voice data WV21, the new song animation data are produced | generated.

  In 2nd Embodiment which performs such a process, according to the output pattern of a synchronizing sound, user moving image data and singing voice data can be synchronized at the some time in a karaoke song. In addition, it is possible to synthesize a plurality of user video data and singing voice data having different shooting times. As a result, it is possible to obtain a singing moving image content that is more unique and has high sound quality.

DESCRIPTION OF SYMBOLS 1 ... Host apparatus; 2 ... Karaoke apparatus; 3 ... User terminal; 4 ... Communication line; 11 ... Host side control part; 12 ... Host side communication part; 13 ... Host side memory | storage part; 23 ... monitor; 24 ... microphone; 25 ... video camera; 26 ... remote control device; 31 ... main body side control section; 32 ... main body side communication section; 33 ... main body side storage section; 34 ... acoustic processing section; 34b ... second mixing unit; 35 ... display processing unit; 36 ... video input unit; 37 ... operating unit; 41 ... terminal side control unit; 42 ... terminal side communication unit; 43 ... terminal side storage unit; 44 ... display 45: Input unit; KB ... Karaoke store; RM ... Karaoke room; UC1 ... Digital video camera; UC2 ... Mobile phone; BU ... Data bus; WV ... Sound emission signal; Voice data based on the data; WV2, WV21 ... Singing voice data with accompaniment; SY, SY1a to SY1d, SY2a to SY2d ... Synchronous sound signal; VD, VD1 to VD4 ... Video data based on user video data; VD11 ... Synthetic video data; AC ... Music signal

Claims (4)

A singing voice data generating device for generating singing voice data at the time of karaoke singing synthesized with video data at the time of karaoke singing shot by a user video camera,
Synchronization sound output means for outputting a synchronization sound having a frequency outside the audible frequency range and recordable by the user video camera;
Record the singing voice during the karaoke singing along with the synchronization sound, possess a singing voice data generating means for generating the singing voice data,
The singing voice data generating apparatus, wherein the synchronizing sound output means repeatedly outputs the synchronizing sounds set in different output patterns at intervals . The singing voice data generating means includes a mixing unit that mixes a singing voice signal obtained by converting the singing voice with a microphone and a synchronizing sound signal that is a basis of the synchronizing sound, and the singing voice data is based on the signal after mixing. singing voice data generating apparatus according to claim 1 Symbol placement and generates the data. Data acquisition means for acquiring singing voice data generated by the singing voice data generating device according to claim 1 or 2 , and moving picture data photographed by a user video camera during karaoke singing corresponding to the singing voice data;
Output timing recognition means for recognizing the output timing of the synchronized sound in the moving image data based on the audio portion of the moving image data and recognizing the output timing of the synchronized sound in the singing audio data based on the singing audio data; ,
In a state where the output timing of the synchronized sound in the moving image data and the output timing of the synchronized sound in the singing sound data are aligned, the video portion of the moving image data and the singing sound data are synthesized, and a new singing moving image A singing moving image data generating device comprising moving image data generating means for generating data. Data acquisition means for acquiring singing voice data generated by the singing voice data generating device according to claim 1 and a plurality of video data photographed by a user video camera at the time of singing karaoke corresponding to the singing voice data When,
Recognizing the output timing of the synchronized sound in each moving image data based on the sound portions of the plurality of moving image data, and recognizing the output timing of the synchronized sounds in the singing sound data based on the singing sound data. Output timing recognition means;
In a state where the output timing of the synchronized sound in the plurality of moving image data and the output timing of the plurality of synchronized sounds in the singing sound data are aligned, the video portions of the plurality of moving image data are joined together and the singing sound data And a singing moving image data generating device for generating new singing moving image data.

Images