JP4354337B2 - Karaoke device with sign language learning function - Google Patents

Description

  The present invention relates to an improved technique for using a karaoke apparatus for sign language learning.

=== Background of this technology ===
In recent years, the keyword “barrier-free” has become established in society. Barrier-free is to improve the social environment to improve the quality of life (QOL) of people with disabilities somewhere in the mind and body. And in order to improve the social environment, it is important for people with disabilities to actively communicate with those who do not.

  The sign language handled by the present invention is effective as a means of communication between a person with hearing impairment and a person who does not, and as the above barrier-free concept is established, many people learn sign language.

=== Sign Language Learning ===
Sign language is not a simple language expression of words or spoken language (hereinafter, written language). Even a sign language in a country is a foreign language with its own grammar and vocabulary system that is different from the written language of that country. Therefore, it is most effective to learn sign language directly from a sign language speaker (a person who is a native speaker of sign language or a person who is proficient in sign language).

  Certainly, for those who are actively trying to “learn” sign language, it is better to learn sign language directly from the sign language speaker. However, this learning method is hard for those who want easy and natural communication with people with hearing impairments. And since the communication in the daily conversation is possible only by translating the words and example sentences in the sign language language into the sign language, the sign language is often learned by using commercially available teaching materials. As a form of teaching material, a videotape in which a sign language action is recorded is common because the sign language expresses a language by a body action mainly having a finger. That is, a videotape in which a video of the sign language action itself and a subtitle indicating the vocabulary (sign language word) of each sign language action is reproduced by a videotape player and learned while watching it.

  Recently, a sign language learning material that can be used on a personal computer such as a CD-ROM has also been provided. This computer-compatible teaching material is a dictionary in which a sign language and a character language are associated with each other for a large number of words and example sentences, and the sign language is expressed by a moving image by computer graphics (CG). When a word or example sentence in a character language is selected so as to draw an ordinary dictionary, the corresponding sign language is displayed on a computer display by CG.

  As described above, sign language is the same as foreign language for ordinary people. Therefore, a certain amount of determination and effort are required to learn sign language. In addition, the learning motivation is often temporary unless necessary to learn sign language.

  However, the number of words and example sentences that can be included in one volume is small with conventional videotape teaching materials. Moreover, if the language (sign language) ability of daily conversation is acquired, teaching materials that simply translate words or example sentences in a written language lack application skills. It is necessary to handle various situations such as conversations that are assumed in daily life. However, this is also difficult to record on a single video tape. Therefore, unless a variety of video materials are prepared, the willingness to learn will soon decline. If you have many teaching materials, the cost of teaching materials will increase. In this way, sign language cannot be easily learned and it is difficult to spread sign language. Computer-aided teaching materials have many recorded vocabularies, but they are just dictionaries. Therefore, practical learning that assumes each scene of conversation cannot be expected.

  SUMMARY OF THE INVENTION An object of the present invention is to realize a sign language learning method for providing an environment in which a variety of conversation scenes can be assumed, and a sign language can be easily learned by touching a natural sign language.

  The inventors of the present application considered the conditions and environment for learning sign language casually, and focused on karaoke. Karaoke is an act of speaking lyrics in accompaniment with words. As described above, since sign language is also a language (word), this karaoke act is directly applied to sign language. I was convinced that karaoke could be a learning material for sign language if it was “speaking in sign language” along with the accompaniment music.

  A karaoke apparatus that plays karaoke can output accompaniment music and lyric character strings for tens of thousands of karaoke songs. The lyrics of each song can be used as a story-like conversation. Therefore, we thought that if there was a karaoke device with a sign language learning function, everyone could naturally become familiar with sign language and helped popularize sign language.

The karaoke apparatus according to the present invention is specified by the following items (1) to (6).
(1) It is a karaoke apparatus with sign language learning function provided with a karaoke database, a sign language video database, a performance means, and a control means. Storing karaoke data (3) The lyric rendering data has an identification code indicating that it is a word to be translated into sign language among the words in the lyric string (4) The sign language video database is (5) The performance means outputs karaoke accompaniment music based on the music generation data and synchronizes with the music progression based on the lyric rendering data according to the control of the control means. (6) When the control means displays a lyric image containing a word with an identification code, the lyric image is displayed on the display means. Be displayed on the display means extracts the sign language video associated with the word from the sign language video database

  According to the present invention, a sign language learning function can be attached to a karaoke apparatus. As a result, a wide variety of scenes included in the lyrics of each huge karaoke piece can be reflected in sign language learning. Therefore, the learner can continue to learn without getting bored. In addition, since the karaoke apparatus is used by a wide variety of people, it can provide an environment where the sign language can be naturally contacted and can be easily learned. Therefore, it contributes to the spread of sign language.

=== Schematic configuration and basic operation of karaoke equipment ===
FIG. 1 shows a functional block diagram of a karaoke apparatus with a sign language learning function according to an embodiment of the present invention. The hardware configuration of this karaoke apparatus with sign language learning function (hereinafter referred to as karaoke apparatus) is almost the same as that of a karaoke apparatus of a type that processes MIDI data to generate karaoke accompaniment music.

  The central control unit 2 is a microcomputer including a CPU, a RAM, and a ROM inside, and controls the peripheral components to control the karaoke apparatus 1. The hard disk device 3 generates music generation data that is the origin of karaoke accompaniment music described in MIDI format for karaoke songs corresponding to a large number of songs, and lyrics rendering data that is the origin of the generation of lyrics images that are rendered in synchronization with the performance of the music Are stored as karaoke data. Video script data for designating a mood video on which a lyrics image is superimposed is also associated with each piece of music.

  When the central control unit 2 receives the music number (music ID) of the requested music designated and input through the user interface such as the remote control transmitter 5 or the operation panel 6 via the operation control unit 7, the input order and the music number are received. Are stored in the RAM in association with each other to perform performance reservation processing for the music. When the performance order of the music is reached, the corresponding music generation data, lyrics rendering data, and video script data are extracted from the hard disk device 3 and processed.

  The music generation data is sequentially transferred to the synthesizer 8, and the audio signal of the accompaniment music generated here and the audio signal input from the microphone 11 are input to the mixing amplifier 9, and the mixed audio signal is output to the speaker 10. The The lyric rendering data is processed to generate a lyric character string synchronized with the accompaniment music. Lyrics rendering data is divided into appropriate measures for all lyrics characters into character strings, and information such as the display / erase order of the character strings (lyric rendering sequence) and the display position of the character strings, and display / erase of each character string Each information such as timing and color change timing (singing time instruction) for each character in the character string is described. The central control unit 2 generates a lyric image including a specified lyric character string at a specified timing, and sequentially generates a lyric image that changes the color of each character in the lyric character string at a specified timing, The image is sequentially developed in the video RAM 12 as a bitmap.

  The optical disk changer 13 divides and stores video data for a long time, which is a generation origin of a mood video used as a background of a lyrics image, into a large number of optical disks. In this embodiment, a CD-ROM is used as the optical disk, and each CD-ROM records MPEG1 format video data corresponding to a video having a playback time of 74 minutes in the same manner as a known video CD. A medium storage unit for storing a large number of CD-ROMs, a player including a CD-ROM drive and a video data decoding circuit, and a transport mechanism for taking out an appropriate CD-ROM from the medium storage unit and loading it into the player Built in.

  Video script data is data that describes when, on which disk, and where video data is recorded. In this example, the elapsed time from the start of the performance of the karaoke song, the optical disc changer's A CD-ROM storage position (disk number) in the medium storage unit and a sector address assigned to each data sector of the CD-ROM are described. As the sector address, minutes / seconds / frames corresponding to the elapsed time when the video recorded on the CD-ROM is reproduced from the beginning are described.

  The central control unit 2 controls the optical disc changer 13 in accordance with the video script data, drives a designated CD-ROM with a built-in player, decodes video data from the designated recording position of the CD-ROM, and mood video. Output as a video signal. As a result, video data for a short time is connected while being sequentially decoded and edited into a series of mood videos. The video control unit 14 superimposes a lyric image developed in a bitmap on the video RAM 12 into a video signal of an appropriate mood video reproduced by the optical disc changer 13, and displays and outputs a mood video containing the lyric image on the display 15.

  Further, the optical disc changer 13 of this example incorporates two CD-ROM drive devices, and the video script data is created in advance so that video data recorded on the same CD-ROM is not designated continuously. . The optical disc changer 13 alternately outputs video data read and decoded from the CD-ROM loaded in each of the two systems of players. This prevents the video from being interrupted when the video for a short time changes. In addition to the configuration of this example, a configuration in which video data is stored in a hard disk device or video data is decoded by software is also conceivable.

=== Sign language video database ===
The karaoke apparatus 1 of the present embodiment realizes a sign language learning function by devising the software executed by the central control unit 2 and the database structure used by the software while having the same hardware configuration as the conventional karaoke apparatus. is doing.

  Sign language actions corresponding to words and example sentences in the sign language are stored as video data (sign language moving image data) in the CD-ROM as in the mood video. In this embodiment, the optical disk changer 13 stores a plurality of CD-ROMs in which sign language moving image data is recorded exclusively. In addition, a word / example sentence collection in which words and example sentences in a character language are aggregated and associated with the recording locations of sign language video data corresponding to each word and example sentence is stored and prepared in the hard disk device 3. FIG. 2 schematically shows the data structure of the word / example sentence collection. A disk number for identifying the CD-ROM and a data recording position (sector address) on the disk are associated with a large number of character strings.

  In many karaoke songs, music that seems to be suitable for sign language learning is registered in advance as sign language learning music. In this embodiment, a music piece having a relatively slow tempo and having only a Japanese character language included in the lyrics character string is selected as a sign language learning music piece. A list of song numbers of these sign language learning songs is stored in the hard disk device 3 and prepared. Therefore, by updating this list, other karaoke songs can be registered as new sign language learning songs. Of course, a sign indicating that the music is a sign language learning music is added to the music generation data and the lyrics rendering data, and the central control unit 2 recognizes this code to identify a normal Karao music and a sign language learning music. May be.

=== Operation of Karaoke Device in Sign Language Learning Function ===
The user appropriately operates the user interface (5, 6) to switch the operation mode to the sign language learning function. And a user inputs a music number into a karaoke apparatus 1 via a user interface (5, 6) like a general karaoke music, and a user reserves a performance reservation for this music. In the sign language learning mode, the central control unit 2 takes out the corresponding music generation data and lyrics rendering data when the performance order of the sign language learning music is reached.

  The central control unit 2 processes the music generation data of the sign language learning tune to output the accompaniment music, and processes the lyric rendering data to display the lyric character string. In the present embodiment, the color change control for the lyrics character string is not performed, and only the display / erasure of the character string is performed in accordance with the progress of the accompaniment music. Then, a word-by-word character or character string is extracted from the lyric character string being displayed and output, and the extracted character string is referred to the word / example sentence collection. When the corresponding sign language moving image data is associated, the optical disc changer 13 is controlled, and the sign language moving image data corresponding to the character string is decoded and displayed on the display. At this time, a lyric image in which the character portion of the lyric corresponding to the sign language operation being displayed has a color different from that of other characters is generated, and the bitmap is developed in the video RAM. This allows the user to know which words in the lyrics string are translated into sign language. Of course, apart from the character strings in the lyric rendering data, words and example sentence character strings corresponding to sign language actions are recorded as an integrated sign language video data on a CD-ROM, and the sign language actions containing the character strings are recorded as images. It is also possible to display. When the sign language moving image is not displayed, the lyrics character string is displayed with a single color screen such as a blue background as a background.

  In this example, the music generation data transfer to the synthesizer 8 is paused while the sign language video is displayed, and the accompaniment music is paused. When a sign language moving image of a certain word is displayed in a row, the pause state of the accompaniment music is released. Thereby, after the sign language action of a certain word is displayed reliably, it is possible to shift to the sign language action of the next word. Of course, the music generation data transfer rate to the synthesizer 8 may be slowed to slow the accompaniment music progress tempo. FIG. 3 is a schematic screen view showing the operating state of the sign language learning mode in the present embodiment when a sign language moving image is displayed on the display.

=== Other Embodiments ===
In the above embodiment, when the lyrics character string of the sign language learning song is displayed, the words are extracted in real time, and the corresponding sign language moving image data is decoded and output. Not limited to this control method, for example, prior to performance processing of a sign language learning song, the lyrics rendering data of the song is examined in advance, the word is referred to the word / example sentence collection, and the corresponding sign language video data is recorded. A schedule describing the location and the display timing of the sign language video data may be created. Then, when processing the music generation data of the sign language learning song, a method of decoding the corresponding sign language video data at a specified timing according to the schedule is also conceivable. If the created schedule is stored, the schedule creation process can be omitted when the same music is designated as a sign language learning music next time.

  Of course, an appropriate identification code may be added to the lyric rendering data so that the character string portion of the word to be translated into sign language can be identified, and a table in which the word to be translated is associated with the music number is prepared. When a musical piece is played using the sign language learning function, if a word with an identification code or a word specified by a table is extracted from the displayed lyrics character string A sign language video can also be displayed.

  When a word in a lyrics character string is examined in advance, only a predetermined number of words having a high appearance frequency among words associated with the sign language may be translated into the sign language. Thereby, it is possible to intensively learn the sign language operation of a predetermined number of words.

  A word to be translated into sign language can be appropriately selected according to the sign language proficiency of the user. For example, the user interface accepts learning level setting inputs such as beginner / intermediate / advanced, and a sign indicating the level is added to each character string recorded in the word / example sentence collection. Alternatively, different word / example sentence collections are prepared for each level. The words corresponding to the set level are translated into sign language.

  Also, the playback speed may be variably controlled as appropriate, for example, the sign language image displayed and output in accordance with the learning level is turned forward or slow motion.

  In the above embodiment, when the sign language moving image is not displayed, the lyrics character string is superimposed on the monochrome screen. Not limited to this configuration, the lyrics image is superimposed on the mood video in the same way as when playing normal karaoke music, and the sign language video is displayed on the display screen by a known video insertion method such as two-screen display or sub-screen display. You may display simultaneously.

It is a functional block diagram of the karaoke apparatus with a sign language learning function in the Example of this invention. It is a schematic data structure figure of the word / example sentence collection processed in the said Example. It is the schematic which shows the sign language moving image screen displayed on the display of the said Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Karaoke apparatus 2 Central control part 3 Hard disk apparatus 13 Optical disk changer 15 Display

Claims (1)

A karaoke apparatus with a sign language learning function comprising a karaoke database, a sign language video database, a performance means, and a control means,
The karaoke database stores music generation data and karaoke data for each song including lyrics rendering data.
In the lyrics rendering data, an identification code indicating that it is a word to be translated into sign language is attached to the words in the lyrics string,
The sign language video database stores multiple words and sign language videos in association with each other.
The performance means outputs the karaoke accompaniment music based on the music generation data according to the control of the control means, and sequentially displays the lyric image on the display means in synchronization with the music progression based on the lyric rendering data,
A karaoke apparatus with a sign language learning function, wherein when a lyric image including a word with an identification code is displayed, the control means extracts a sign language video associated with the word from the sign language video database and displays it on the display means.

Images