JP5282230B2 - Learning materials and teaching material information output device - Google Patents

Description

  The present invention relates to a learning material and a learning material information output device, and more particularly to a learning material and a learning material information output device that enable a learner to understand the pronunciation of a language more accurately and efficiently.

  In conventional learning materials for language learning, phonetic symbols (or phonetic symbols) are used when writing pronunciations of learning languages. In particular, the International Phonetic Alphabet (IPA), which is a unified symbol capable of comprehensively expressing pronunciation in all languages, may be used.

  In addition, the pronunciation of the learning language is expressed by phonetic characters in the native language that are easy to understand (for example, hiragana or katakana if the learner's native language is Japanese) without using special symbols. There are many things.

  Furthermore, for example, in Patent Document 1, in English words, the syllable type with a strong accent is made bold and bold so that the learner can visually understand the position of the accent, An English word learning material is disclosed that enables memorization.

JP 2003-005625 A

  However, in order to use teaching materials whose pronunciation is expressed by phonetic symbols such as international phonetic symbols, it is necessary to learn the phonetic symbols themselves. Therefore, it is difficult for beginners to learn pronunciation with phonetic symbols.

  In addition, the method of presenting the position of a strong accent by notating the pronunciation of a phonetic character or highlighting the type of a teaching material cannot provide information on important elements such as intonation and rhythm in the pronunciation of a language.

  The present invention has been made in view of such a situation, and enables even a beginner who cannot understand phonetic symbols to understand the pronunciation of a language more accurately and efficiently.

One aspect of the present invention includes an original text display section that displays an original text in a predetermined language, and a score display section that displays a musical score showing a pronunciation corresponding to the original text displayed by the original text display section with music symbols. The relative length and height of the pronunciation are represented by notes of the music symbols shown in the score, with the clef and change symbols omitted, the breaks of the pronunciation are represented by rest symbols, Is a learning material that represents the strength of the pronunciation .

  A portion of the musical symbol that is pronounced in an integrated manner can be represented by a consecutive digit.

  Of the musical symbols, a slur or tie may represent a portion that is pronounced by connecting the pronunciations.

  Corresponding to the score displayed by the score display unit using a translated text display unit that displays a translation in another language corresponding to the original text displayed by the source text display unit, and a phonetic character in the other language And a pronunciation display unit for displaying the pronunciation of the pronunciation.

According to another aspect of the present invention, there is provided an original score acquisition means for acquiring an original text in a predetermined language, and a score acquisition for acquiring score information of a score in which a pronunciation corresponding to the original text acquired by the original text acquisition means is indicated by a music symbol. Means, storage means for storing the score information acquired by the score acquisition means, and the original sentence acquired by the original sentence acquisition means, a score generated from the score information stored in the storage means, and the Output means for outputting teaching material information associated with the correspondence with the original sentence, the output means relative to the pronunciation by the note of the musical symbol shown in the musical score in which the clef and the change symbol are omitted The musical score representing the length and height, the break of the pronunciation by the rest symbol, the strength of the pronunciation by the strength symbol is generated, and the correspondence between the score and the original text is related A teaching information output device for outputting wood information.

  The output means can generate the musical score representing a portion of the musical symbol that is pronounced integrally by a continuous digit.

  Using the translation acquisition means for acquiring a translation in another language corresponding to the original sentence acquired by the original sentence acquisition means, and using the phonetic characters in the other language, the score information acquired by the score acquisition means Pronunciation acquisition means for acquiring the corresponding pronunciation pronunciation, and the storage means further stores the translation acquired by the translation acquisition means and the pronunciation reading acquired by the pronunciation acquisition means The output means can output teaching material information that associates the correspondence between the score, the original sentence, the translation, and the pronunciation reading generated from the score information stored in the storage means.

  Voice acquisition means for acquiring voice data of the pronunciation in the predetermined language; and a score conversion means for converting the voice data acquired by the voice acquisition means into the score information, wherein the score acquisition means The score information converted by the score generating means can be acquired.

  Voice recognition means for recognizing the original sentence in the predetermined language from the voice data acquired by the voice acquisition means; and the original sentence acquisition means acquires the original sentence recognized by the voice recognition means. Can do.

In the learning material according to one aspect of the present invention, an original text in a predetermined language is displayed, a musical score showing a pronunciation corresponding to the displayed original text is displayed , and clef and change symbols are omitted. The relative length and height of the pronunciation are represented by notes of the musical symbols shown in the score, the break of the pronunciation is represented by the rest symbol, and the strength of the pronunciation is represented by the strength symbol.

In the teaching material information output apparatus according to one aspect of the present invention, an original text in a predetermined language is acquired, and musical score information of a musical score in which a pronunciation corresponding to the acquired original text is indicated by a music symbol is acquired. The acquired musical score information and the original text are accumulated, and teaching material information that associates the correspondence between the musical score generated from the accumulated musical score information and the original text is output. Note that the relative length and height of the pronunciation are indicated by notes among the musical symbols shown in the score without the clef and change symbols, the break of the pronunciation is indicated by the rest symbol, and the pronunciation is indicated by the strength symbol. A score representing strength is generated from the score information.

  As described above, according to one aspect of the present invention, a learner can understand the pronunciation of a language more accurately and efficiently.

It is a figure which shows the structure of 1st Embodiment of the learning material for learning of this invention. It is a figure which shows the structure of 2nd Embodiment of the learning material for learning of this invention. It is a figure which shows the structure of 3rd Embodiment of the learning material of this invention. It is a figure which shows the structure of 4th Embodiment of the learning material for learning of this invention. It is a block diagram which shows the structure of 1st Embodiment of the teaching material information output device of this invention. It is a flowchart explaining operation | movement of 1st Embodiment of the teaching material information output device of this invention. It is a block diagram which shows the structure of 2nd Embodiment of the teaching material information output device of this invention. It is a flowchart explaining operation | movement of 2nd Embodiment of the teaching material information output device of this invention. It is a figure which shows the score which represented the pitch absolutely. It is a block diagram which shows the structure of 3rd Embodiment of the teaching material information output device of this invention. It is a flowchart explaining operation | movement of 3rd Embodiment of the teaching material information output device of this invention.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram showing a configuration of a first embodiment of a learning material to which the present invention is applied. In the present embodiment, a learning material in which a learner whose native language is Japanese learns English is exemplified. The learning material 1 includes an original text display unit 11, a score display unit 12, a translated text display unit 13, and a pronunciation display unit 14.

  The original sentence display unit 11 displays a sentence (sentence) or a phrase (sentence) as one unit as a language to be learned by the learner. In the example of FIG. 1, an English sentence “What did you make?” Is displayed.

  The musical score display unit 12 displays a syllable when the sentence of the original text display unit 11 is pronounced by a native speaker as a musical symbol. That is, pronunciation (such as intonation (ie, relative fluctuations in pitch), rhythm (ie, temporal slowness of sounds), and articulation (ie, dynamics and expressions in sounds and sound connections) Features are expressed by associating music symbols with the sound recognized as a unit as a unit. As music symbols, for example, notes, rests, accent symbols, tenuto symbols, staccato symbols, slur symbols, tie symbols, and equidistant horizontal lines can be used.

  The score displayed by the score display unit 12 is displayed in a format according to the staff notation, and the pitch of the sound in the pronunciation (that is, the pitch) is indicated by the vertical position of the note, and the arrangement of notes and rests. Indicates the rhythm of pronunciation. In the display of the score, an arbitrary length can be adopted as the length of one beat of the quarter note (ie, tempo). For example, in the case of English pronunciation, it is possible to employ a tempo that hits 80 to 120 beats of quarter notes per minute.

  The pitch information of pronunciation cannot be expressed accurately on the staff. This is because, on the staff, it is only possible to express a half unit of 1 / 12th of an octave in 12 equal temperament as a minimum unit, but in order to express a pitch difference when a person speaks a language, it is a unit finer than a semitone. Because it becomes necessary.

  Therefore, in this embodiment, the pitch difference is displayed relatively. For example, when the pitch is set to four levels of “low”, “medium”, “high”, and “very high”, the pitch is expressed by placing the note at any one of the four levels. Is done. In this case, since the pitch difference is expressed by a relative amount, it is not necessary to display the equidistant horizontal lines together with the score displayed on the score display unit 12.

  However, in order to enable the learner to easily recognize the difference in the level of each note, it is also possible to display a horizontal line with equal intervals. In the embodiment of FIG. 1, three horizontal lines are displayed. However, this horizontal line is not a staff of the notation that is the absolute pitch reference. For example, in the embodiment of FIG. 1, the fifth note is arranged between the top horizontal line and the second horizontal line, and the sixth note is arranged on the second horizontal line. This does not mean that the sound corresponding to the sixth note is lower than the pitch corresponding to the fifth note by the whole tone. It simply means that the sixth pitch is relatively lower than the fifth pitch. Of course, the absolute pitch of the pitches corresponding to the fifth and sixth notes, that is, the frequency is not specified.

  The musical score display unit 12 visually displays various articulations by music symbols. For example, an accent symbol when you pronounce the sound strongly, a tenuto symbol when you pronounce it while keeping the pitch, a staccato symbol when you pronounce the sound separately, and a slur when you pronounce the sound smoothly Symbols and tie symbols can also be displayed. In addition, a group of pronunciations can be expressed by a series of digits that are connected and displayed with notes having a note value finer than an eighth note.

  In the example of FIG. 1, the pronunciation of the sentence “What did you make?” Displayed in the original text display unit 11 is the sixteenth note, the sixteenth rest, the eighth note, the sixteenth note, the sixteenth note, and the eighth minute. It is expressed by notes and sixteenth notes. The first sixteenth note to the fourth sixteenth note are made into a series of digits, and the third and fourth sixteenth notes are connected by a tie symbol. The fifth eighth note and the sixth sixteenth note are arranged in series. A horizontal line is displayed on the score. That is, the first sixteenth note is the highest note, the second eighth note is lower than that, and the third and fourth sixteenth notes are second. Lower than the pitch of an eighth note. The pitches of the fifth eighth note and the sixth sixteenth note are lower than the first sixteenth note and higher than the second eighth note. The pitch of the fifth eighth note is higher than the sixth sixteenth note. Accent symbols are attached to the first sixteenth note and the fifth eighth note.

That is, in this example, the relative height of each phrase “What”, “did you”, and “make?” Is represented by the vertical position of the note, and the relative length of the pronunciation is the note length. Expressed by type. “What” is represented by a sixteenth note, “did you” is represented by an eighth note and two sixteenth notes, and “make?” Is represented by an eighth note and a sixteenth note. Corresponds to the vertical position of each note. In this example, since three horizontal lines are added, it is easy to understand the relative height of each note.

The length of the sound is represented by the type of each note. In this example, the pronunciation is expressed by eighth notes and sixteenth notes. Associate an eighth note with “di (d)” or “ma” and associate a sixteenth note with “What”, “(di) d you”, and “(ma) ke”. It is expressed that is relatively longer than the latter sound. The length of the sound break between “What” and “did you” is represented by a 16-minute rest, which is a rest symbol.

  The continuous beam indicates that the portion is a portion that is pronounced integrally. In the example of FIG. 1, the “What did you” portion from the first 16th note to the fourth 16th note, the fifth eighth note, and the sixth 16th note. The parts of “make” to which the notes are associated are shown as continuous parts, which are parts that are pronounced integrally or continuously. In other words, it is expressed that this sentence should be pronounced in two parts: a “What did you” part and a “make” part.

When the display is purely musical, for example, the first sixteenth note of the “What did you” portion can be expressed as a single note without being a digit. However, if this is done, it cannot be expressed that the “What did you” part should be pronounced as a unit, that is, the “What” part should be pronounced as a “did you” part. Therefore, the part that is pronounced as a unit is expressed as a continuous beam.

To be more precise, the “(di) d you” part of “What did you” represented by two sixteenth notes is shown as being a part of a string and pronounced as a unit. ing. Similarly, the part of “di (d)” displayed with eighth notes and the part of “you” displayed with two sixteenth notes are a part of a string that is pronounced as a unit. Is represented.

In addition, the “ma” and “ke” portions of “make” represented by eighth notes and sixteenth notes are also represented as consecutive digits, and are represented as integrally pronounced portions.

  Furthermore, the Thai symbols indicate that the pronunciation of the last “d” and “you” in “did” should be concatenated to form one syllable. If the display is purely musical, the third and fourth sixteenth notes can be represented by one eighth note. However, it becomes difficult to understand that there are two syllables in the pronunciation, “d” and “you”. Therefore, in the present embodiment, the number of notes corresponding to the number of syllables is displayed, and tie symbols (slur symbols when the pitches are different) are added to them. In this way, the “do” part of “Dido” and the next “you” part should not be connected as two syllables but should be connected so that they become one syllable like “Duyu”. That is, it is expressed that the sound should be pronounced so that the number of syllables is substantially reduced.

In addition, the fact that “ma” in “What” and “make?” Is pronounced strongly is indicated by an accent symbol, which is a dynamic symbol.

  The score of the score display unit 12 is displayed below or above the corresponding word of the source text display unit 11 so that the learner can easily understand the relationship between the score of the score display unit 12 and the spelling of the source text display unit 11. The

  The translated sentence display unit 13 displays a translated sentence in another language corresponding to the original sentence display unit 11. In the example of FIG. 1, when the learner's native language is Japanese, a translated sentence “What did you make?” Is displayed. In this example, the translated text in the translated text display section 13 is displayed on the right side of the original text display section 11 so that it can be understood that it corresponds to the original text in the original text display section 11. The display position of the translated sentence display unit 13 is arbitrary as long as the correspondence with the original sentence display unit 11 can be easily understood by the learner.

  The pronunciation display unit 14 displays the reading of the pronunciation of the original text in the original text display unit 11 by phonograms that the learner can understand. In the example of FIG. 1, the pronunciation reading “Waddy Do You Make?” Is displayed by Katakana, which is one of the Japanese phonetic characters. Further, the pronunciation display unit 14 is displayed so that each sound corresponds to the note of the score display unit 12, and the upper side or the lower side of the score display unit 12 so that the learner can easily understand the relation with the note. Is displayed next to

  When learning using the learning material 1, the learner learns the meaning of the English sentence displayed on the original text display unit 11 from the translated text displayed on the translated text display unit 13, and is displayed on the pronunciation display unit 14. Learn pronunciation by reading different pronunciations. The learning material 1 includes a compact disc in which the pronunciation of the original text in the original text display unit 11 recorded by a native speaker is recorded, and the learner listens to the reproduced sound and displays the score displayed on the score display unit 12. Learn how to pronounce, such as intonation, rhythm, and articulation.

  FIG. 2 is a diagram showing a configuration of a second embodiment of a learning material to which the present invention is applied. In the present embodiment, a learning material in which a learner whose native language is Japanese learns English is exemplified. The learning material 3 includes original text display units 31 and 41, musical score display units 32 and 42, translation display units 33 and 43, pronunciation display units 34 and 44, face illustration display units 35 and 45, and an illustration display unit 50. Is done.

  The original text display units 31 and 41 correspond to the original text display unit 11 in FIG. 1, respectively. In the example of FIG. 2, the learning target sentences “What color do you like?” And “Pink.” Are respectively displayed. doing.

  The score display sections 32 and 42 correspond to the score display section 12 in FIG.

  In the example of FIG. 2, the score display unit 32 includes a sixteenth note, a sixteenth rest, a sixteenth note, a sixteenth note, a sixteenth note, a sixteenth note, a sixteenth note, a sixteenth note, and a sixteenth note. It is expressed. The first to third sixteenth notes, the fourth and fifth sixteenth notes, and the sixth to eighth sixteenth notes are arranged in series. In this example, no horizontal line is displayed on the score. The pitch of the second sixteenth note is higher than the first sixteenth note, the third sixteenth note is lower than the pitch of the second sixteenth note, and the first sixteenth note is lower than the first sixteenth note. It is higher than the pitch of the half note. The fourth 16th note has a lower pitch than the first 16th note, and the fifth 16th note has a lower pitch than the fourth 16th note. The pitch of the sixth sixteenth note is higher than that of the fifth sixteenth note and is about the same as the third sixteenth note. The pitch of the seventh sixteenth note is lower than the sixth sixteenth note, and the eighth sixteenth note is lower than the seventh sixteenth note.

  In the example of FIG. 2, the relative height of each phrase “What color”, “do you”, “like?” Is expressed by the vertical position of the note, and the relative length of the pronunciation is Expressed by the type of note. In this example, “What color”, “do you”, and “like?” Are all represented by sixteenth notes. The pitch of each note is represented by the vertical position of each note. In the case of this example, no horizontal line is added, but the vertical position of each note is arranged at a position where the relative height is known.

“What”, “color”, “do you”, “like?” Pronunciations are one 16th note, two 16th notes, two 16th notes, and three 16th notes respectively. It is expressed. Thereby, it is expressed that the length of each pronunciation is the same. The length of the sound break between “What” and “color” is represented by a 16-minute rest, which is a rest symbol.

The “What color” portion represented by the first to third sixteenth notes, the “do you” portion represented by the fourth and fifth sixteenth notes, and the sixth to sixth portions Each “like?” Portion represented by the eighth sixteenth note is a continuous beam. Accordingly, the “What color”, “do you”, and “like?” Portions are each pronounced integrally.

  The musical score display unit 42 is expressed by eighth notes and sixteenth notes, and the first eighth note and the second sixteenth note are arranged in a series. There are no horizontal lines on the score. The pitch of the second sixteenth note is lower than the first eighth note. Also, an accent symbol is attached to the first eighth note.

  The “Pink.” Portion is represented by eighth notes and sixteenth notes. The pitch of each note is represented by the vertical position of each note. In the case of this example, no horizontal line is added, but the vertical position of each note is arranged at a position where the relative height is known.

By associating an eighth note with the “Pin” portion and a sixteenth note with the “k” portion, it is expressed that the former sound is longer than the latter sound. In addition, the fact that the “Pin” portion is pronounced strongly is indicated by an accent symbol which is a strength symbol.

  By using an eighth note of “Pin” and a sixteenth note of “k” as digits, it is expressed that “Pink” should be pronounced integrally.

  The translation display units 33 and 43 correspond to the translation display unit 13 in FIG. In the example of FIG. 2, the translated sentence display unit 33 displays a translated sentence “What color do you like?” And the translated sentence display unit 43 displays the translated sentence “Pink.”.

  The pronunciation display sections 34 and 44 correspond to the pronunciation display section 14 in FIG. In the example of FIG. 2, the pronunciation display section 34 displays the pronunciation reading “Waccoal Do You Like?” And the pronunciation display section 44 displays the pronunciation reading “Pink.”.

  The face illustration display sections 35 and 45 display the speaker as an illustration to identify the speaker. In the example of FIG. 2, the face illustration display unit 35 displays the childcare person's face illustration, and the face illustration display unit 45 displays the child's face illustration attached to the original text of the original text display units 31 and 41. . This shows that the sentence “What color do you like?” In the original text display section 31 is spoken by a nursery teacher, and the sentence “Pink.” In the original text display section 41 is spoken by a kindergarten.

  The illustration display unit 50 displays the state of conversation with an illustration. In the example of FIG. 2, in the illustration display section 50, the nursery teacher picks up the color list paper and asks “What color do you like?” An illustration showing how to answer “Pink.” Is displayed.

  When learning using the learning material 3, the learner uses the illustrations displayed on the face illustration display units 35, 45 and the illustration display unit 50 for the conversation displayed on the original text display units 31, 41. And the meaning is learned from the translations displayed on the translation display units 33 and 43. In addition, the learner learns pronunciation by reading the pronunciation displayed on the pronunciation display units 34 and 44 and the reproduced sound from the attached compact disc, and intonation, rhythm, and the like by the score displayed on the score display units 32 and 42. And learn how to pronounce in conversations like articulation.

  FIG. 3 is a diagram showing a configuration of a third embodiment of a learning material to which the present invention is applied. In the present embodiment, a learning material in which a learner whose native language is Japanese learns English is exemplified. The learning material 6 includes an original text display unit 61 and a score display unit 62.

  The learning material 6 is for learning how to pronounce English, for example. The original text display unit 61 corresponds to the original text display unit 11 in FIG. 1, and in the example of FIG. 3, the sentence “Good morning.” To be learned is displayed.

  The score display section 62 corresponds to the score display section 12 in FIG. In the example of FIG. 3, the score display unit 62 is expressed by a sixteenth note, a sixteenth rest, an eighth note, and a dotted eighth note. The second eighth note and the third dotted eighth note are arranged in series. There are no horizontal lines on the score. The pitch of the second eighth note is higher than the first sixteenth note, the pitch of the third dotted eighth note is lower than the second eighth note, the first Higher than the sixteenth notes.

  That is, the relative height of the pronunciation of each phrase “Good” and “morning” is expressed by the vertical position of the note, and the relative length of the pronunciation is expressed by the type of the note. In the example of FIG. 3, “Good” is represented by a sixteenth note, and “morning” is represented by two eighth notes. The last eighth note of the two eighth notes is marked with a dot, indicating that it is 1.5 times longer than the first eighth note. Of course, this length is also relative and need not be exactly 1.5 times as long. The length of the sound break between “Good” and “morning” is represented by a 16-minute rest, which is a rest symbol.

  The two eighth notes representing “morning” are assumed to be a series of digits, indicating that they should be pronounced as a unit.

In the case of this example, no horizontal line is added, but the vertical position of each note is arranged at a position where the relative height is known.

  When learning using the learning material 6, the learner describes how to pronounce the original text displayed in the original text display section 61, such as intonation, rhythm, and articulation, and the score displayed in the score display section 62. Learning by playing sound from a compact disc.

  FIG. 4 is a diagram showing the configuration of the fourth embodiment of the learning material to which the present invention is applied. This embodiment illustrates a learning material in which a learner whose native language is Japanese learns Chinese. The learning material 7 includes an original sentence display unit 71, a score display unit 72, a translated sentence display unit 73, and a pronunciation display unit 74.

  The original text display unit 71 corresponds to the original text display unit 11 in FIG. 1, and in the example of FIG. 4, the learning target sentence “Yoshio Hayami.” Is displayed.

  The score display section 72 corresponds to the score display section 12 in FIG. In the example of FIG. 4, the score display unit 72 is expressed by three eighth notes. A horizontal line is displayed on the score. The second eighth note is higher and the third eighth note is lower than the pitch of the first eighth note.

  In the example of FIG. 4, “early”, “upper”, and “good” are all represented by eighth notes. The height of each pronunciation is represented by the vertical position of each note. In this example, since four horizontal lines are added, it is easy to understand the relative height of each note.

In addition, since the “early”, “upper”, and “good” pronunciations are each expressed in eighth notes, the length of each pronunciation is the same.

  The translation display unit 73 corresponds to the translation display unit 13 in FIG. In the example of FIG. 4, the translated sentence display section 73 displays the translated sentence “Good morning.”

  The pronunciation display unit 74 corresponds to the pronunciation display unit 14 in FIG. In the example of FIG. 4, the pronunciation display section 74 displays the pronunciation reading “Zao Xiang Hao.”

  When learning using the learning material 7, the learner learns the meaning of the Chinese sentence displayed on the original sentence display unit 71 from the translated sentence displayed on the translated sentence display unit 73 and displays it on the pronunciation display unit 74. The pronunciation is learned by reading the pronunciation, and the pronunciation such as intonation, rhythm, and articulation is learned from the score displayed on the score display unit 72. Also in this case, the reproduction sound of the attached compact disc is used.

  In the above embodiments, English and Chinese have been studied. However, the learning material of the present invention can be applied to any language as long as it is a language with sound.

  When learning pronunciation, in addition to visual information based on the score, the pronunciation can be visualized by using auditory learning using a compact disk, memory, etc. that records the pronunciation of the score as audio data. You can learn from both the auditory side and expect synergistic efficiency in language acquisition.

  FIG. 5 is a diagram showing the configuration of the first embodiment of the teaching material information output apparatus to which the present invention is applied. The teaching material information output apparatus 100 according to the present embodiment includes a voice acquisition unit 101, a score generation unit 102, a score acquisition unit 103, a voice recognition unit 104, a source sentence acquisition unit 105, a translation acquisition unit 106, a pronunciation acquisition unit 107, and a storage unit 108. , And an output unit 109.

  The voice acquisition unit 101 acquires voice data produced by a native speaker or the like of a language to be learned (for example, English) using a sentence or phrase to be learned as a unit, and the acquired voice data is converted into a score unit 102 and This is supplied to the voice recognition unit 104.

  The score coding unit 102 converts the intonation, rhythm, and articulation of pronunciation from the voice data acquired by the voice acquisition unit 101 into score information encoded as music symbols, and the converted score information is a score acquisition unit. 103. The score information encoded by the score encoding unit 102 corresponds to the score displayed by the score display unit 12 in FIG. The score acquisition unit 103 acquires the score information converted by the score conversion unit 102 and supplies the acquired score information to the storage unit 108.

  The voice recognition unit 104 recognizes the generated voice from the voice data acquired by the voice acquisition unit 101 and converts it into corresponding character data, and supplies the converted character data to the original text acquisition unit 105. The original text acquisition unit 105 acquires the character data converted by the voice recognition unit 104 as an original text, and supplies the acquired text data of the original text to the storage unit 108.

  The translation acquisition unit 106 acquires character data of a translation (for example, Japanese) corresponding to the original sentence acquired by the original sentence acquisition unit 105 and supplies the acquired character data of the translation to the storage unit 108. The pronunciation acquisition unit 107 acquires pronunciation readings corresponding to the score information acquired by the score acquisition unit 103 as character data of phonetic characters (for example, katakana) in a language (for example, Japanese) corresponding to the translation. The acquired pronunciation character data is supplied to the storage unit 108.

  The accumulating unit 108 is acquired by the musical score information acquired by the musical score acquisition unit 103, the original character data acquired by the original text acquisition unit 105, the translated character data acquired by the translation acquisition unit 106, and the voice acquisition unit 107. Character data of the pronunciation of the pronunciation is accumulated and supplied to the output unit 109.

  The output unit 109 generates a score from the score information accumulated by the accumulation unit 108. Then, teaching material information in which four character data of the generated score, the original sentence, the translated sentence, and the pronunciation pronunciation stored by the storage unit 108 are associated is output.

  Next, the learning material information output processing by the learning material information output apparatus 100 of FIG. 5 will be described with reference to the flowchart of FIG.

  In step S121, the voice acquisition unit 101 acquires voice data regarding the pronunciation of the original text (in this embodiment, English). In the process of step S121, the voice acquisition unit 101 acquires voice data when a native speaker speaks an English sentence or phrase to be output as teaching material information.

  In step S122, the score generating unit 102 converts the audio data acquired by the process of step S121 into score information.

  The musical score information is obtained by encoding pronunciation intonation, rhythm, and articulation with musical symbols. Of these, the pronunciation intonation is converted into musical score information by the score encoding unit 102 as follows, for example. That is, the sound data acquired by the sound acquisition unit 101 is converted into time frequency information by Fourier transform, and converted into the pitch of a note based on the frequency of each sound in the time frequency information. The score generating unit 102 has a memory in which a conversion table defining the relationship between the pitch on the score and the corresponding frequency range is recorded, so that the frequency of each sound in the audio data is converted to the pitch information on the score. Can be converted.

  The rhythm of pronunciation is converted into musical score information by the musical score conversion unit 102 as follows, for example. That is, the time amplitude information such as the peak position and duration of the amplitude of the voice data acquired by the voice acquisition unit 101 is converted into the type and time position of the note or rest. Since the score generating unit 102 has a memory in which a conversion table defining the relationship between time amplitude information and score information is recorded, the peak position of the amplitude of the voice data is converted to the time position of the note on the score, and the amplitude The duration can be converted to a note type. Further, if voice data generated at a predetermined tempo can be used, it can be converted into more accurate rhythm score information.

  The articulation of pronunciation is converted into musical score information by the musical score conversion unit 102 as follows, for example. That is, the time amplitude of the sound data acquired by the sound acquisition unit 101 is compared with the first threshold value, and an accent symbol is added to a sound having an amplitude peak equal to or greater than the first threshold value.

  The duration of a frequency indicating a predetermined sound is compared with a second threshold, and a tenuto symbol is added to a sound having a duration equal to or greater than the second threshold. In addition, if the duration of the sound frequency is a duration equal to or smaller than a third threshold value that is smaller than the second threshold value, a staccato symbol used when the sound is sounded separately is added.

  If the amplitude does not become the fourth threshold value or less while a plurality of adjacent sounds are sounded, a tie symbol is added when the plurality of sounds have the same pitch, and a slur symbol is added when the pitches are different. For example, in the embodiment of FIG. 1, the third and fourth notes of the score display unit 12 correspond to “do” and “you” sounds of the pronunciation display unit 14, respectively. Since these two sounds are sounded smoothly without interruption, the amplitude does not fall below the fourth threshold value even between the amplitude peak of “Do” and the amplitude peak of “You”. In other words, since the amplitude between the end of the pronunciation of “Do” and the start of the pronunciation of “Yu” does not drop below the fourth threshold value, a tie symbol is added to the third and fourth notes. Yes.

  Assuming a fifth threshold value that is smaller than the fourth threshold value, the amplitude is not less than or equal to the fifth threshold value and the note value of the multiple notes is finer than an eighth note while a plurality of consecutive sounds are produced. , Those notes are made into a series of digits and expressed as a group of pronunciations.

  As described above, the score coding unit 102 converts the sound data into the score information based on the time amplitude information and the time frequency information of the sound data. Note that the method for converting voice data into musical score information is not limited to the above-described method, and an arbitrary algorithm or a combination of a plurality of algorithms can be used.

  The information format of the musical score information may be in accordance with a data format such as an electronic musical instrument digital interface (MIDI) that has already been standardized, or a unique information format may be used. The musical score information may be in a format that can be edited as necessary.

  In step S123, the score acquisition unit 103 acquires the score information converted in the process of step S122. In step S124, the voice recognition unit 104 recognizes the voice data obtained in step S121 and converts it into character data. As the speech recognition algorithm of the speech recognition unit 104, for example, an arbitrary algorithm such as a Hidden Markov Model (HMM) can be adopted. In step S125, the original text acquisition unit 105 acquires the character data recognized in the process of step S124.

  In step S126, the translation acquisition unit 106 acquires character data about the translation input from the outside. In step S127, the pronunciation acquisition unit 107 acquires character data about pronunciation reading input from the outside. In the processing of steps S126 and S127, character data input by the operator operating the keyboard or mouse is acquired, or character data already stored in the storage medium is acquired.

  In step S128, the storage unit 108 stores the data acquired by the score acquisition unit 103, the original sentence acquisition unit 105, the translation acquisition unit 106, and the pronunciation acquisition unit 107. That is, the storage unit 108 acquires the musical score information data acquired in step S123, the original character data acquired in step S125, the translated character data acquired in step S126, and the step S127 processing. Accumulate four character data of pronunciation readings.

  In step S129, the output unit 109 outputs teaching material information from the data accumulated in the process of step S128. In the process of step S129, a score is generated from the score information stored by the storage unit. Then, the generated musical score, the original sentence stored by the storage unit 108, the translated sentence, and the pronunciation reading are associated with each other and output as teaching material information. For example, the original text data, score information, translated text data, and pronunciation reading text data are associated with the original text display section 11, the score display section 12, the translation display section 13, and the pronunciation display section 14 in FIG. Teaching material information is output. In addition, the output unit 109 displays a uniform horizontal line indicating a pitch difference on the score based on an instruction from the operator. A specific form of output is display on a display unit such as an LCD or printing on paper. Thereby, for example, the learning material 1 as shown in FIG. 1 is generated.

  Through the above teaching material information output processing, the teaching material information output device 100 automatically recognizes the score information and the original text from the pronunciation voice data, and can efficiently output the teaching material information for learning the meaning and pronunciation of the language.

  FIG. 7 is a diagram showing the configuration of the second embodiment of the teaching material information output apparatus to which the present invention is applied. The learning material information output apparatus 200 includes a score acquisition unit 201, an original sentence acquisition unit 202, a translation acquisition unit 203, a pronunciation acquisition unit 204, a storage unit 205, and an output unit 206.

  The apparatus of FIG. 7 does not have a configuration corresponding to the voice acquisition unit 101, the score generating unit 102, and the voice recognition unit 104 in the apparatus of FIG. 5 described above, and directly receives the musical score information of the original text and the text data of the original text. To get. That is, the score acquisition unit 201 acquires score information of the pronunciation of the original sentence and stores it in the storage unit 205. The original text acquisition unit 202 acquires text data of the original text corresponding to the pronunciation and stores it in the storage unit 205.

  The translation acquisition unit 203, the pronunciation acquisition unit 204, the storage unit 205, and the output unit 206 correspond to the translation acquisition unit 106, the pronunciation acquisition unit 107, the storage unit 108, and the output unit 109 in FIG. Description is omitted as appropriate.

  Next, processing of the learning material information output apparatus 200 will be described with reference to the flowchart of FIG.

  In step S221, the score acquisition unit 201 acquires score information about the pronunciation of the original text (for example, English). Here, for example, a music score creator having an absolute pitch listens to an English sentence pronounced by a native speaker and grasps its intonation, rhythm, and articulation. Then, the score creator inputs the grasped pronunciation information into the score acquisition unit 201 by the score creator himself. The score acquisition unit 201 acquires score information input by the score creator by operating an input device such as a keyboard or a mouse. Of course, other operators can input musical score information based on a musical score created by a musical score creator having an absolute pitch.

  A score creator can input score information to the score acquisition unit 201 as a score. While the pitch in human speech is continuous, the pitch notation on the musical score is discontinuous with a semitone as the minimum unit. In addition, a score creator having an absolute pitch has the ability to identify a pitch of about 1 / 100th of a semitone (ie, 1 cent) in 12 equal temperament. Therefore, a music score creator having an absolute pitch grasps the pitch of pronunciation of voice data in units smaller than the notation of the score.

  The music score creator can grasp each sound of the pronunciation as an absolute pitch, grasp the first pitch of the pronunciation as an absolute pitch, and the pitch of the following nth tone is the nth pitch. You may grasp | ascertain as a relative pitch on the basis of -1st pitch. In any case, when inputting the pitch information of the acquired pronunciation as a musical score, the score creator rounds down or rounds up the pitch acquired in cents to a semitone or full pitch. Enter the pitch on the staff as semitone or full pitch.

  As for the rhythm and articulation of pronunciation, the contents obtained from the pronunciation of the native speaker by the score creator are input to the score acquisition unit 201 as musical scores. Grasping the pronunciation depends on the subjectivity of the score creator, and different score information may be input even if the score is different even if the pronunciation is the same. However, the most important thing here is to convert the pronunciation to information that can be visually understood as a musical score and to be easy to grasp, and the accuracy of the details in the musical score information is not a big problem.

  FIG. 9 shows an example in which a music score creator having an absolute pitch listens to an utterance “What color do you like?” From a native speaker and expresses it on a staff. In this state, although the truncation or round-up process is performed, the pitch is expressed absolutely. However, because of the nature of the language, the pitch must absolutely match the pitch of the native speaker. Therefore, in the present embodiment, as displayed in the score display section 32 of FIG. 2, the clef (for example, treble clef) is omitted and the relative pitch is input. The ups and downs of the semitone by the change symbol (for example, sharp or flat) is also omitted because its strictness is not necessary.

  In step S222, the original sentence acquisition unit 202 acquires character data regarding the original sentence corresponding to the pronunciation of the native speaker. In step S <b> 223, the translation acquisition unit 203 acquires character data about the translation corresponding to the original sentence acquired by the original sentence acquisition unit 202. In step S <b> 224, the pronunciation acquisition unit 204 acquires character data regarding pronunciation reading corresponding to the score information acquired by the score acquisition unit 201. In the processing of steps S221 to S224, score information and character data input by the user operating an input device such as a keyboard or a mouse are acquired.

  In step S225, the storage unit 205 stores the data acquired by the score acquisition unit 201, the original sentence acquisition unit 202, the translation acquisition unit 203, and the pronunciation acquisition unit 204. That is, the storage unit 205 acquires the musical score information data acquired in the process of step S221, the original text data acquired in the process of step S222, the translated text data acquired in the process of step S223, and acquired in the process of step S224. Accumulate four character data of pronunciation readings.

  In step S226, the output unit 206 outputs teaching material information from the data accumulated in the process of step S225. In the process of step S226, a score is generated from the score information stored by the storage unit 205. Then, the generated musical score, the original sentence accumulated by the accumulation unit 205, the translated sentence, and the pronunciation reading are associated with each other and output as teaching material information. For example, the original text data, score information, translated text data, and pronunciation reading text data are associated with the original text display section 11, the score display section 12, the translation display section 13, and the pronunciation display section 14 in FIG. Teaching material information is output. The output unit 206 can also generate equidistant horizontal lines indicating relative pitch differences as necessary. Thereby, for example, a learning material 1 as shown in FIG. 1 is generated.

  Through the above teaching material information output processing, the teaching material information output device 200 directly acquires the score information about the pronunciation from the input of the score creator having the absolute pitch, so that the teaching material information reflecting the delicate expression in the pronunciation is output to the score information. can do.

  FIG. 10 is a diagram showing the configuration of the third embodiment of the teaching material information output apparatus to which the present invention is applied. The teaching material information output device 300 includes a voice acquisition unit 301, a score generation unit 302, a score acquisition unit 303, a voice recognition unit 304, an original sentence acquisition unit 305, a storage unit 306, and an output unit 307.

  The apparatus of FIG. 10 does not have a configuration corresponding to the translation acquisition unit 106 and the pronunciation acquisition unit 107 in the apparatus of FIG. 5 described above, and outputs only the original text of the language to be learned and the score indicating the pronunciation as teaching material information. It is.

  That is, the storage unit 306 stores the musical score information acquired by the score acquisition unit 303 and the character data about the original text acquired by the original text acquisition unit 305. The output unit 307 generates a musical score using a staff notation from the musical score information stored in the storage unit 306. Then, teaching material information associating two elements of the generated score and the original sentence stored in the storage unit 306 is output.

  The voice acquisition unit 301, the score conversion unit 302, the score acquisition unit 303, the voice recognition unit 304, and the original text acquisition unit 305 are the voice acquisition unit 101, the score conversion unit 102, the score acquisition unit 103, the voice recognition unit 104, FIG. And the original text acquisition unit 105, which are repeated and will not be described as appropriate.

  Next, processing of the learning material information output apparatus 300 will be described with reference to the flowchart of FIG.

  In step S321, the voice acquisition unit 301 acquires voice data regarding the pronunciation of the original text. In the process of step S321, for example, original voice data of a native speaker is acquired. In step S322, the score generating unit 302 converts the audio data acquired in the process of step S321 into score information. The process of step S322 is the same process as the process of step S122 of FIG.

  In step S323, the score acquisition unit 303 acquires the score information converted in the process of step S322. In step S324, the voice recognition unit 304 recognizes the voice data acquired in step S321 and converts it into character data. In step S325, the original sentence acquisition unit 305 acquires character data regarding the original sentence converted in the process of step S324. The processes in steps S323 to S325 are the same as the processes in steps S123 to S125 in FIG.

  In step S326, the storage unit 306 stores the data acquired by the score acquisition unit 303 and the original text acquisition unit 305. That is, the accumulation unit 306 accumulates two pieces of data of musical score information acquired in the process of step S323 and original text character data acquired in the process of step S325.

  In step S327, the output unit 307 outputs teaching material information from the data accumulated in the process of step 326. In the process of step S327, a score is generated from the score information stored by the storage unit 306. Then, the generated musical score and the teaching material information relating the two elements of the original sentence accumulated by the accumulation unit 306 are output. At this time, equidistant horizontal lines indicating relative pitch differences are added as necessary. Thereby, for example, teaching material information in which the text data of the original text and the score information are associated with each other is output to the original text display section 61 and the score display section 62 as shown in FIG.

  Through the above teaching material information output process, the teaching material information output device 300 automatically recognizes both the score information and the original text necessary for the teaching material information from the pronunciation sound data, so that the teaching material information for learning the pronunciation of the language is efficiently obtained. Can be output.

  In the present specification, each step includes not only the processing performed in time series according to the described order but also the processing executed in parallel or individually even if the processing is not necessarily performed in time series.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

  DESCRIPTION OF SYMBOLS 11 Original text display part, 12 Musical score display part, 13 Translation display part, 14 Pronunciation display part, 31 Original text display part, 32 Musical score display part, 33 Translation display part, 34 Pronunciation display part, 41 Original text display part, 42 Musical score display part, 43 translation display section, 44 pronunciation display section, 61 original text display section, 62 musical score display section, 71 original text display section, 72 musical score display section, 73 translated text display section, 74 pronunciation display section

Claims (9)

An original text display section for displaying an original text in a predetermined language;
A score display section that displays a score that shows the pronunciation corresponding to the original text displayed by the original text display section with music symbols ;
The relative length and height of the pronunciation are represented by notes among the musical symbols shown in the score where the clef and change symbols are omitted, the break of the pronunciation is represented by a rest symbol, A learning material representing the strength of pronunciation . The Beam of the musical symbols, learning materials according to claim 1 which represents the integrally pronounced in part of the sound. The learning material according to claim 2 , wherein a portion of the music symbol that is pronounced by connecting the pronunciation is represented by slur or tie. A translation display unit for displaying a translation in another language corresponding to the original text displayed by the original text display unit;
The learning material according to claim 3 , further comprising: a pronunciation display unit that displays a pronunciation of the pronunciation corresponding to the score displayed by the score display unit using a phonetic character in the other language. An original text acquisition means for acquiring an original text in a predetermined language;
A score acquisition means for acquiring score information of a score in which the pronunciation corresponding to the original text acquired by the original text acquisition means is indicated by music symbols;
Storage means for storing the score information acquired by the score acquisition means and the original text acquired by the original text acquisition means;
Output means for outputting teaching material information associating the correspondence between the score and the original text generated from the score information stored in the storage means ,
The output means indicates the relative length and height of the pronunciation by the notes of the music symbols shown in the score where the clef and change symbols are omitted, and the break of the pronunciation by the rest symbol. A learning material information output device that generates the musical score that expresses the strength of the pronunciation using dynamic symbols and outputs the educational material information that associates the correspondence between the musical score and the original text . The teaching material information output device according to claim 5 , wherein the output unit generates the score that represents a portion of the musical symbol that is pronounced in an integrated manner by consecutive digits. A translation acquisition means for acquiring a translation in another language corresponding to the original text acquired by the original text acquisition means;
Pronunciation acquisition means for acquiring readings of the pronunciation corresponding to the score information acquired by the score acquisition means using phonograms in the other languages; and
The storage means further stores the translation acquired by the translation acquisition means and the pronunciation reading acquired by the pronunciation acquisition means;
The teaching material information according to claim 6 , wherein the output means outputs teaching material information in which correspondence between the score, the original sentence, the translation, and the pronunciation pronunciation generated from the score information stored in the storage means is associated. Output device. Voice acquisition means for acquiring voice data of the pronunciation in the predetermined language;
Further comprising: a musical score converting means for converting the voice data acquired by the voice acquisition means into the musical score information,
The teaching material information output device according to claim 6 , wherein the score acquisition unit acquires the score information converted by the score conversion unit. Voice recognition means for recognizing the original text in the predetermined language from the voice data acquired by the voice acquisition means;
The teaching material information output device according to claim 8 , wherein the original text acquisition unit acquires the original text recognized by the voice recognition unit.

Images