JP7806833B2 - Electronic device, learning support system, learning processing method and program - Google Patents

Description

The present invention relates to an electronic device, a learning support system, a learning processing method, and a program.

Some electronic devices, such as electronic dictionaries and personal computers, are equipped with learning functions aimed at learning foreign languages. Language learning includes, for example, a function for learning vocabulary using a dictionary (see, for example, Patent Document 1), a function for learning grammar using example sentences and the like included in a dictionary, a listening learning function for listening to sentences read aloud in the foreign language, and a speaking learning function for learning to pronounce words in the foreign language correctly.

Japanese Patent Application Laid-Open No. 2021-047769

As such, conventional electronic devices have multiple learning functions for language study, but even when studying the same language, different learning methods are used depending on the purpose. Therefore, users have to select a function from multiple learning functions that uses content that suits their learning purpose, specify the subject to be studied (e.g., words), and then execute the learning function. As a result, when studying a language, users have tended to study content individually.

The present invention was made in consideration of the above-mentioned problems, and aims to provide an electronic device, learning support system, learning processing method, and program that enables language learning by effectively using multiple learning functions.

In order to solve the above problems, an electronic device according to an embodiment of the present invention has a control unit that performs a speech recognition process on the voice generated when a user reads a sentence aloud, determines whether the pronunciation of words contained in the sentence is correct, extracts from the sentence any mispronounced words whose pronunciation is determined to be incorrect as a result of the accuracy determination, changes the extracted mispronounced words to root forms using the mispronounced words as their root forms, outputs the root forms into which the mispronounced words have been changed, outputs a learning method list that lists learning methods that can be performed on the root forms of the words, and executes a learning function using a learning method selected from the learning method list.

The present invention provides an electronic device, learning support system, learning processing method, and program that allows users to effectively use multiple learning functions to learn a language.

1 is a schematic diagram of a learning support system according to an embodiment of the present invention. FIG. 1 is a front view showing the external configuration of an electronic dictionary according to an embodiment of the present invention. FIG. 2 is a functional block diagram showing the configuration of an electronic circuit of another electronic device that is connected to the electronic dictionary via communication when used according to the embodiment of the present invention. 4 is a flowchart showing the operation of the electronic dictionary according to the present embodiment. 4 is a flowchart showing the operation of the tablet PC (digital textbook) in this embodiment. 4 is a diagram showing an example of a textbook screen displayed on a touch panel display unit of a tablet PC according to the embodiment. FIG. FIG. 3 is a diagram showing an example of an extracted word list displayed in the electronic dictionary according to the embodiment. 4 is a diagram showing an example of a textbook screen displayed on a touch panel display unit of a tablet PC according to the embodiment. FIG. FIG. 4 is a diagram showing an example of a learning method list displayed on the electronic dictionary according to the embodiment. FIG. 10 is a diagram showing an example of a learning function execution screen when "vocabulary learning" is selected from the learning method list in this embodiment. FIG. 10 is a diagram showing an execution screen of a wordbook registration function executed in accordance with the learning method "word learning" in this embodiment. FIG. 10 is a diagram showing an example of a learning function execution screen when "pronunciation training" is selected from the learning method list in this embodiment. FIG. 10 is a diagram showing an example of an execution screen of the example sentence search function when "Example Sentence Study" is selected from the learning method list in this embodiment. 1 is a schematic diagram of a learning support system according to an embodiment of the present invention.

The following describes an embodiment of the present invention with reference to the drawings.

Figure 1 is a schematic diagram of a learning support system according to an embodiment of the present invention. Learning support system 1 includes an electronic dictionary 10 as an electronic device (first electronic device) and a tablet PC 20 as an electronic device (second electronic device). Furthermore, learning support system 1 may include a server 30. Server 30 provides images and test data, including digital textbook data, to tablet PC 20 and the like via network N. Figure 1 shows a functional block diagram illustrating the configuration of the electronic circuitry of electronic dictionary 10.

In this embodiment, an example is shown in which the electronic device is configured as an electronic dictionary 10. Note that the electronic device can be realized as various electronic devices such as an electronic dictionary 10, a personal computer, a smartphone, a tablet PC, a game device, etc.

The electronic dictionary 10 stores multiple types of dictionary content as dictionary data. In the dictionary content, information on at least one word meaning is registered in association with each of multiple headwords and other words. The dictionary content included in the electronic dictionary 10 is generally created by publishers and includes content that has also been published in paper form, making it highly reliable. Therefore, by effectively utilizing highly reliable dictionary content for study, you can expect to achieve correct and effective learning results.

Dictionary content is not limited to dictionaries related to foreign languages such as English or Japanese languages, but also includes content such as encyclopedias, study guides, workbooks, literary works, and commentaries.

Dictionary content can be used not only through the dictionary search function, but also as learning content for language learning.

The electronic dictionary 10 has the configuration of a computer that reads programs recorded on various recording media or transmitted programs and whose operation is controlled by the read programs, and its electronic circuitry is equipped with a CPU (central processing unit) 11.

The CPU 11 functions as a control unit that controls the entire electronic dictionary 10. The CPU 11 controls the operation of each circuit component in accordance with a control program pre-stored in the memory 12, a control program loaded into the memory 12 from a recording medium 13 such as a ROM card via the recording medium reader 14, or a control program downloaded from an external device (such as a server) and loaded into the memory 12 via a network N such as the Internet.

The control program stored in memory 12 is activated in response to input signals from the key input unit 16 in response to user operation, input signals from the touch panel display unit 17 in response to user operation, or connection and communication signals with an external recording medium 13 such as an EEPROM (registered trademark), RAM, or ROM connected via the recording medium reading unit 14.

Connected to the CPU 11 are a memory 12, a recording medium reading unit 14, a communication unit 15, a key input unit 16, a touch panel display unit 17 (display), an audio input unit (microphone) 18, an audio output unit (speaker) 19, and the like. In one embodiment, the display unit does not need to have touch panel functionality. For example, an LCD (Liquid Crystal Display) may be used as the display unit instead of the touch panel display unit 17.

The control programs stored in memory 12 include a dictionary control program 12a, a voice recognition program 12b, and a learning method processing program 12c. Dictionary control program 12a is a program for controlling the overall operation of electronic dictionary 10. Dictionary control program 12a also realizes a dictionary search function that searches dictionary content and displays information based on character strings entered through the input units (key input unit 16, touch panel display unit 17, voice input unit 18).

The dictionary control program 12a not only provides a dictionary search function that searches dictionary content and displays information, but also implements multiple different learning functions that can be used to learn foreign languages. Foreign languages can include multiple languages, such as English, Chinese, Korean, German, and French. The multiple learning functions include, for example, a word search function that is effective for vocabulary learning, a sentence search function that is ideal for learning sentences, a listening learning function that allows users to listen to sentences and words read aloud in the foreign language, a speaking learning function (pronunciation training function) that allows users to practice correctly pronouncing sentences and words in the foreign language, and a vocabulary book registration function that registers specific words that need to be learned. Furthermore, functions for studying grammar and long passages, a function for viewing reference books for foreign language qualification exams, and a test question function that displays and allows users to answer test questions can be provided. The above-mentioned learning functions are merely examples, and other functions that can be used to learn foreign languages can also be provided.

Each learning function can process multiple pieces of content. For example, the word search function can use multiple dictionary contents as search targets, and can search all of them or select one of them. The speaking learning function can selectively use sentence content to practice sentence pronunciation and word content to practice word pronunciation. The test question function also has multiple test question contents with different test types and question levels, and can process any of them.

The speech recognition program 12b is a program that realizes a speech recognition function that recognizes the speech of a learner reading aloud sentences, etc. input from the speech input unit 18, and outputs the sentences, etc. read aloud by the learner. The speech recognition program 12b is capable of speech recognition for multiple languages, such as English, Chinese, Korean, German, and French. Speech recognition by the speech recognition program 12b involves, for example, performing acoustic analysis on the speech data of the learner reading aloud, and detecting the spoken sentences, etc. by comparing the acoustic analysis results with a speech recognition dictionary (including an acoustic model, a language model, a pronunciation dictionary, etc.) prepared for each language.

In the voice recognition program 12b of this embodiment, when data such as a sentence to be read aloud is available, the results of voice recognition of the voice of the sentence to be read aloud can be compared with the data of the sentence to be read aloud to determine whether the pronunciation of words included in the sentence to be read aloud is correct. Furthermore, when data such as a sentence to be read aloud is not available, if a word obtained from the results of voice recognition of the voice of the sentence to be read aloud is not included in pre-registered words to be recognized (for example, a database in which words to be recognized are registered), it can be determined that the word is pronounced incorrectly.

The learning method processing program 12c uses the results of the speech recognition processing by the speech recognition program 12b to, for example, identify learning functions (learning methods) that can be used for learning from the multiple learning functions provided by the dictionary control program 12a, and provides a learning method list that lists the learning functions (learning methods).

Memory 12 stores dictionary data 12d, extracted word data 12e, learning method search data 12f, learning content data 12g, etc.

Dictionary data 12d includes a database that compiles dictionary content, such as multiple dictionaries, including an English-Japanese dictionary, a Japanese-English dictionary, an English-English dictionary, an English-Chinese dictionary, and a Japanese dictionary, as well as multiple types of encyclopedias. It may also include dictionary content in other languages in addition to English-language content. Dictionary data 12d associates semantic information that explains the meaning (semantics) of each headword with each dictionary. Dictionary data 12d may not be built into the electronic dictionary 10 itself, but may instead be stored in an external device (such as a server) accessible via network N.

Extracted word data 12e is data indicating words that have been extracted from the results of speech recognition by speech recognition program 12b and that have been determined to be incorrect in a pronunciation accuracy assessment. Note that extracted word data 12e may also be data indicating words extracted regardless of accuracy assessment, obtained by speech recognition of speech in which a sentence or the like is read aloud.

The learning method search data 12f is data indicating learning functions that can be executed for words determined to be incorrectly pronounced, as indicated by the extracted word data 12e. The learning functions indicated by the learning method search data 12f are processed by the learning method processing program 12c to be listed as learning methods that can be used to learn words, and are output by displaying them on the touch panel display unit 17, etc.

Learning content data 12g is data related to content used by various learning functions of the dictionary control program 12a.

The communication unit 15 controls communications with other electronic devices via a network N such as the Internet or a local area network (LAN), or controls communications for short-range wireless communications such as Bluetooth (registered trademark) or Wi-Fi (registered trademark) with other electronic devices in the vicinity (e.g., tablet PC 20).

The electronic dictionary 10 can perform processing in cooperation with other electronic devices (e.g., tablet PC 20) by connecting via communication with the communication unit 15.

In the electronic dictionary 10 configured in this way, the CPU 11 controls the operation of each circuit component in accordance with commands written in various programs, such as the dictionary control program 12a, the voice recognition program 12b, and the learning method processing program 12c, and the software and hardware work together to achieve the functions described in the operational explanation below.

Figure 2 is a front view showing the external configuration of the electronic dictionary 10 in this embodiment.

In the case of the electronic dictionary 10 in Figure 2, the lower section of the device body, which can be opened and closed, houses the CPU 11, memory 12, recording medium reading unit 14, communication unit 15, voice input unit 18, voice output unit 19, as well as a key input unit 16, while the upper section houses the touch panel display unit 17.

The key input section 16 includes character input keys 16a, a dictionary selection key 16b that allows you to select various dictionaries and functions, a [Translate/Confirm] key 16c, a [Back] key 16d, cursor keys (up, down, left, right) 16e, a delete key 16f, a power button, and various other function keys.

The touch panel display unit 17 displays various menus and buttons 17a depending on the execution of various functions. The touch panel display unit 17 can be used, for example, with a pen to perform touch operations to select various menus and buttons 17a, and to input characters by hand.

Figure 3 is a functional block diagram showing the configuration of the electronic circuitry of another electronic device (second electronic device) that is connected to and used via communication with the electronic dictionary 10 according to this embodiment. In this embodiment, an example is shown in which the other electronic device (second electronic device) is configured as, for example, a tablet PC 20. The tablet PC 20 is a device used, for example, as a digital textbook. Note that the electronic device is not limited to a tablet PC 20, and can be realized by various electronic devices such as a personal computer, smartphone, game device, etc.

The tablet PC 20 can access a server 30 connected via a network N, including the Internet, and display web pages and text files containing digital textbook data provided by the server 30, allowing the user to view them.

The tablet PC 20 has the configuration of a computer that reads programs recorded on various recording media or transmitted programs and whose operation is controlled by the read programs, and its electronic circuitry is equipped with a CPU (central processing unit) 21.

The CPU 21 functions as a control unit that controls the entire tablet PC 20. The CPU 21 controls the operation of each circuit component in accordance with a control program pre-stored in the memory 22, a control program loaded into the memory 22 from a recording medium 23 such as a ROM card via the recording medium reader 24, or a control program downloaded from an external device (such as a server) and loaded into the memory 22 via a network N such as the Internet.

The control program stored in memory 22 is activated in response to an input signal from the button input unit 26 in response to a user operation, an input signal from the touch panel display unit 27 in response to a user operation, or a connection/communication signal with an external recording medium 23 such as an EEPROM (registered trademark), RAM, or ROM connected via the recording medium reading unit 24.

Connected to the CPU 21 are a memory 22, a recording medium reading unit 24, a communication unit 25, a button input unit 26, a touch panel display unit 27 as a display unit, an audio input unit (microphone) 28, an audio output unit (speaker) 29, and the like.

The control programs stored in memory 22 include a basic program 22a (OS (Operating System)) that controls the overall operation of the tablet PC 20, and a textbook processing program 22b for displaying textbook screens (including text, images, etc.) based on textbook data.

By executing the basic program 22a and the textbook processing program 22b, the tablet PC 20 transmits, via communication, sentence data (text data) contained in the textbook data to be displayed on the tablet PC 20 to the electronic dictionary 10.

Memory 22 stores textbook data 22c, extracted word data 22d, and other data to be displayed by textbook processing program 22b.

The textbook data 22c includes text data and image data representing various textbooks. The textbook data 22c is displayed on the touch panel display unit 27 under the control of the textbook processing program 22b. When the tablet PC 20 is connected to the electronic dictionary 10 and executes linked processing, the data to be processed (e.g., sentence data) displayed on the touch panel display unit 27 is sent to the electronic dictionary 10.

Extracted word data 22d is data received from electronic dictionary 10 when tablet PC 20 is connected to electronic dictionary 10 and executes linked processing. When sentence data to be processed is sent to electronic dictionary 10, extracted word data 22d is data indicating, from among the words included in this sentence data, words that have been extracted from the speech recognition results of speech recognition in electronic dictionary 10 and determined to be incorrect by a pronunciation accuracy determination. Based on extracted word data 22d, under control of textbook processing program 22b, words indicated by extracted word data 22d in the sentence displayed on touch panel display unit 27 are displayed in a display format different from the others.

The communication unit 25 controls communications with other information processing devices (e.g., server 30) via a network N such as the Internet or a local area network (LAN), or controls communications for short-range wireless communication such as Bluetooth (registered trademark) or Wi-Fi (registered trademark) with other electronic devices (e.g., electronic dictionary 10) in the vicinity.

The audio input unit 28 inputs audio such as a learner reading aloud a sentence. The audio output unit 29 outputs various sounds in accordance with processing by the CPU 21.

Next, we will explain the operation of the electronic dictionary 10 and the tablet PC 20 (digital textbook) as the second electronic device in this embodiment.

First, we will explain the case where the electronic dictionary 10 and tablet PC 20 operate in conjunction with each other.

Figure 4 is a flowchart showing the operation of the electronic dictionary 10 in this embodiment. Figure 5 is a flowchart showing the operation of the tablet PC 20 (digital textbook) in this embodiment.

First, the user initiates processing to connect (set up a link) the electronic dictionary 10 and tablet PC 20 via wireless communication. When the CPU 11 of the electronic dictionary 10 receives an instruction to set up a link (step A1, Yes), it executes a communication program to set up wireless communication with the tablet PC 20 via the communication unit 15. When the CPU 21 of the tablet PC 20 receives an instruction to set up a link (step B1, Yes), it executes a communication program to set up wireless communication with the electronic dictionary 10 via the communication unit 25. As a result, the electronic dictionary 10 and tablet PC 20 each execute processing to set up a communication link (steps A2, B2), and become ready for data communication.

When the tablet PC 20 is instructed to execute the textbook processing program 22b in response to a user operation, it reads the textbook data 22c from the memory 22 (step B3) and displays a textbook screen (text, images, etc.) on the touch panel display unit 27 based on the textbook data 22c (step B4). The tablet PC 20 changes the display content of the textbook screen in response to an operation on the touch panel display unit 27 or the button input unit 26.

Figure 6 shows an example of a textbook screen displayed on the touch panel display unit 27 of the tablet PC 20 in this embodiment. The textbook screen shown in Figure 6 (A) displays a sentence representing the content of a conversation in English, such as "Did you enjoy your homestay in London?"

When the tablet PC 20 displays a sentence as shown in Figure 6 (A), it can use the voice recognition function of the electronic dictionary 10 to determine whether the sentence data is read aloud.

For example, when a text range to be read aloud is specified by touching the text displayed on the touch panel display unit 27 (step B5, Yes), the CPU 21 of the tablet PC 20 transmits the text data (text data) to the electronic dictionary 10 via the communication unit 25 (step B6).

On the other hand, when the CPU 11 of the electronic dictionary 10 receives text data to be read aloud from the tablet PC 20 via the communication unit 15 (step A3, Yes), it temporarily stores this text data in the memory 12 (step A4).

In response to receiving text data from the tablet PC 20, the CPU 11 starts the voice recognition program 12b and transitions to a state in which voice recognition using the voice recognition function for voice input from the voice input unit 18 is possible.

Here, the user instructs the electronic dictionary 10 to start reading aloud, for example, by operating the key input unit 16, and the tablet PC 20 starts reading aloud the sentence selected as the subject of reading aloud (text data sent from the tablet PC 20).

When an instruction to start reading aloud is input (step A5), the CPU 11 performs speech recognition processing on the speech data (the speech of the user reading aloud) input from the speech input unit 18 (step A6). The speech recognition processing is performed according to the language to be read aloud. For example, if an English text is to be read aloud, speech recognition processing may be performed using a speech recognition dictionary for English by specifying English as the language in advance. Similarly, speech recognition processing for each language may be performed using a speech recognition dictionary for each language by specifying languages such as Chinese, Korean, German, or French.

Speech recognition processing for each language may also be performed without specifying the language in advance. For example, acoustic analysis may be performed on the speech data, and the language may be determined by comparing the acoustic analysis results with a speech recognition dictionary for each language (including an acoustic model, language model, pronunciation dictionary, etc.) prepared in advance, and speech recognition processing may be performed using the speech recognition dictionary for the determined language. Also, by providing a speech recognition function that uses AI (Artificial Intelligence) technology and having it learn multiple languages, it may be possible to perform speech recognition processing for each language without specifying the language in advance.

The CPU 11 performs voice recognition processing on the input voice data, outputting text data as the voice recognition result. Furthermore, the CPU 11 compares the text data as the voice recognition result with the sentence data received from the tablet PC 20 to determine whether the pronunciation of words contained in the sentence is correct. That is, the text data as the voice recognition result and the text data of the sentence are compared word by word, and if they match, the word is determined to be correctly pronounced, and if they differ, the word is determined to be incorrectly pronounced.

The CPU 11 extracts the words determined to be incorrectly pronounced as mispronounced words, associates the mispronounced words with the original words in the sentence data, and records the associations as extracted word data 12e in memory 12 (step A7). The CPU 11 transmits the extracted word data 12e indicating the words determined to be incorrectly pronounced to the tablet PC 20 as the speech recognition results for the reading of the sentence (step A8).

When the CPU 21 of the tablet PC 20 receives the extracted word data 12e from the electronic dictionary 10 (step B7, Yes), it stores it in the memory 22 (extracted word data 22d). In accordance with the extracted word data 22d, the CPU 21 displays, for the sentence selected for reading aloud and currently displayed on the touch panel display unit 27, words corresponding to the mispronounced words indicated by the extracted word data 22d, i.e., words that could not be pronounced correctly, in a display format different from the display of the rest of the sentence (step B8).

The textbook screen shown in Figure 6 (B) shows an example in which the display format has been changed for the words "enjoy" and "homestay." Note that examples of different display formats that can be applied to words include changing the text color, highlighting, and underlining. Other display formats can also be used.

In this way, by linking with the electronic dictionary 10, the tablet PC 20 uses the voice recognition function of the electronic dictionary 10 to change the display format of words that could not be pronounced correctly (words determined to be incorrectly pronounced) in the sentence being displayed on the touch panel display unit 27 (performs a reading determination). This allows the user to easily visually confirm words that have been determined to be incorrectly pronounced.

When an instruction to change the display content of the textbook screen is input in response to an operation on the touch panel display unit 27 (step B9, No), the CPU 21 reads the textbook data 22c from the memory 22 (step B3) and displays the changed textbook screen on the touch panel display unit 27 based on the textbook data 22c (step B4).

Furthermore, when an instruction to end the display of the textbook screen is received (step B9, Yes), the CPU 21 ends the processing.

Meanwhile, the electronic dictionary 10 records the extracted word data 12e in the memory 12, and also creates a list (extracted word list) of the misread words indicated by the extracted word data 12e and displays it on the touch panel display unit 17 (step A9).

In this embodiment, an extracted word list is created and displayed, but it is also possible to register words using the vocabulary book registration function of the dictionary control program 12a and display a screen that lists the words registered in the vocabulary book registration function.

Figure 7 shows an example of an extracted word list displayed in the electronic dictionary 10 in this embodiment. The extracted word list shown in Figure 7 shows an example in which two words, "enjoy" and "homestay," are displayed.

In the example shown in Figure 7, the word "enjoy" (see Figure 6(A)) in the text data received from the tablet PC 20 is recognized as, for example, "enjoying," and similarly, the word "homestay" (see Figure 6(A)) is recognized as, for example, "hamster," and each pronunciation is determined to be incorrect.

In the examples shown in Figures 6 and 7, when creating the extracted word list, the words are listed as they are spelled in the text data ("enjoy", "homestay"), but as explained below, it is also possible to list words whose spelling has been changed from the words in the text data.

Figure 8 (A) shows an example of a textbook screen displayed on the touch panel display unit 27 of the tablet PC 20 in this embodiment. The textbook screen shown in Figure 8 (A) displays a sentence representing the content of a conversation in English, such as "Are you enjoying your homestay in London?"

In the same manner as above, the sentence shown in Figure 8 (A) is read aloud, and the speech recognition function of the electronic dictionary 10 is used to determine whether the speech read aloud is correct. As a result, the word "enjoying" in the sentence data is recognized as "join," for example, and the pronunciation is determined to be incorrect (the pronunciation of the word "homestay" is determined to be correct). On the textbook screen, the display format is changed and the word "enjoying" is displayed as a word that was not pronounced correctly, as shown in Figure 8 (B).

In this case, rather than listing the word "enjoying" in the sentence data as it is, for example, the word "enjoying" in the sentence data is converted to its root form and listed. In other words, similar to Figure 7, the root form of the word "enjoying" is listed as "enjoy" and the extracted word list is displayed.

By not displaying duplicates of the same word, or even different spellings of the same word (e.g., "enjoying" and "enjoy"), the extracted word list makes it easier to select words from the extracted word list.

Note that the sentence shown in Figure 8(A) shows an example of the word "enjoying" written in the present participle form, but this is not limited to the present participle form, and the target word can also be written in a different form from its original form.

For example, you can do the same for words in the third person singular present notation, past tense, past participle, plural, and notations with prefixes/suffixes added, changing them to their original form to create an extracted word list.

If the relationship between the root form of a word and a word that is spelled differently from the root form is included in dictionary data 12d, the spelling of the word in the sentence data can be changed to the root form by referencing dictionary data 12d. Furthermore, table data that shows the relationship between the root form of a word and a word that is spelled differently from the root form can be prepared separately from dictionary data 12d, and the spelling of the word in the sentence data can be changed to the root form by referencing this table data.

Here, when an instruction to select a word from the extracted word list displayed on the touch panel display unit 17 is input (for example, by touching the check button corresponding to the word) (step A10), the CPU 11 searches for a learning method corresponding to the word and displays a list of learning methods on the touch panel display unit 17 (step A11).

In other words, the CPU 11 searches for a learning function from among the multiple learning functions provided by the dictionary control program 12a that can learn a word selected from the extracted word list. For example, if the content that each learning function processes contains a word selected from the extracted word list, it can determine that the learning function is capable of learning (executable).

Alternatively, an index may be created in advance in which executable learning functions (learning methods) are registered for each word, and this index may be searched based on a word selected from the extracted word list to determine the executable learning functions corresponding to the word.

Figure 9 shows an example of a learning method list displayed on the electronic dictionary 10 in this embodiment. The learning method list shown in Figure 9 is an example of a case where the word "enjoy" is selected from the extracted word list.

As shown in Figure 9, the learning method list makes it easy to see that the word "enjoy" can be learned through "word learning," "example sentence learning," "pronunciation training," and "grammar learning." The learning method list does not include learning methods that cannot be used to learn the word "enjoy." Therefore, by using the learning method list, it is possible to efficiently select a learning method that can be used to learn from multiple learning methods. Furthermore, by selecting and executing multiple learning methods included in the learning method list, it is possible to effectively learn words selected from the extracted word list (i.e., words determined to be incorrectly pronounced).

When a learning method is selected from the learning method list (Step A12, Yes), the CPU 11 executes the selected learning function (Step A13) with the word selected from the extracted word list as the processing target (based on the selected word), and displays an execution screen for the learning function on the touch panel display unit 17 (Step A14). In one embodiment, the execution screen includes at least a portion of the content to be processed by the learning function. Specifically, the execution screen may include a portion of the content to be processed that includes the word to be processed (for example, an explanation of the word in dictionary content, or, when the example sentence search function described below is executed, a sentence including the word to be processed).

Figure 10 shows an example of the execution screen for the learning function when "Word Learning" is selected from the learning method list in this embodiment.

Figure 10 shows the execution screen of the word search function executed in response to the "Word Study" learning method. When the word search function is normally executed, it is launched with the headword input area 40 blank, but when it is launched when selected from the learning method list, a screen is displayed showing the results of a word search performed using a word selected from the extracted word list as the headword.

This allows for efficient learning, as there is no need to launch the learning function and then specify the words to be learned (enter them as headwords).

The execution screen for the word search function shown in Figure 10 displays search results for multiple different dictionaries (such as "G English-Japanese," "W English-Japanese," and "O English-Japanese") included in dictionary data 12d. In other words, the word search function processes multiple pieces of content. Here, instead of executing the word search function on multiple pieces of content (dictionaries), it is also possible to select any one dictionary (content) and execute the word search function.

Also, in the above explanation, when "Word Study" is selected from the study method list, the word search function is executed, but multiple different study functions may be associated with the study method "Word Study."

For example, the learning method "vocabulary learning" can be associated with a word search function as well as a vocabulary book registration function. This allows users to select one of multiple learning functions that process different content for a single learning method, depending on the learning content they need.

Figure 11 shows the execution screen of the vocabulary book registration function executed in accordance with the "vocabulary learning" learning method in this embodiment. As shown in Figure 11, the execution screen of the vocabulary book registration function displays an execution screen in which the word "enjoy" selected from the extracted word list has been registered.

Figure 12 shows an example of the execution screen for the learning function when "Pronunciation Training" is selected from the learning method list in this embodiment.

The execution screen of the speaking learning function executed in accordance with the learning method "Pronunciation Training" shown in Figure 12 shows the execution screen of the function for practicing pronunciation by word. The speaking learning function can execute not only the function for practicing pronunciation by word shown in Figure 12, but also the function for practicing pronunciation by sentence unit containing the word "enjoy" selected from the extracted word list.

Figure 13 is a diagram showing an example of the execution screen of the example sentence search function when "Example Sentence Study" is selected from the learning method list in this embodiment. As shown in Figure 13, the execution screen of the example sentence search function displays a list of example sentences obtained by searching for example sentences in dictionaries included in dictionary data 12d (e.g., "G Dictionary") based on the word "enjoy" selected from the extracted word list.

If, for example, the [Back] key 16d on the key input unit 16 is operated (step A15, No) after selecting a learning method from the learning method list and executing one of the learning functions, the CPU 11 returns to the state in which the learning method list before the learning function was executed is displayed on the touch panel display unit 17 (in this case, there is no need to search for a learning method). Thereafter, the CPU 11 executes the same processing as described above (steps A11 to A14).

This allows for effective learning by sequentially selecting and executing different learning functions from the learning method list for a single word selected from the extracted word list. Furthermore, the learning support system of this embodiment allows users to efficiently learn words that they were unable to pronounce correctly, i.e., words that are highly in need of learning.

When the CPU 11 receives an instruction to end the process (step A15, Yes), it ends the learning process that uses the results of the voice recognition process.

Next, we will explain the case where the electronic dictionary 10 and tablet PC 20 are not linked.

In this case, when the tablet PC 20 is instructed to execute the textbook processing program 22b in response to a user operation, it reads the textbook data 22c from the memory 22 (step B10) and displays a textbook screen (text, images, etc.) on the touch panel display unit 27 based on the textbook data 22c (step B11). The tablet PC 20 changes the display content of the textbook screen in response to the operation on the touch panel display unit 27.

When using the voice recognition function of the electronic dictionary 10, the user instructs the electronic dictionary 10 to start reading aloud, for example, by operating the key input unit 16, and the electronic dictionary 10 begins reading aloud a sentence or the like displayed on the touch panel display unit 27 of the tablet PC 20.

When a command to start reading aloud is input (step A5), the CPU 11 inputs the voice input from the voice input unit 18 (step A6) and performs voice recognition processing on this voice data (step A7). In the voice recognition processing, processing according to the language to be read aloud is performed in the same manner as described above.

However, when not linked to the tablet PC 20, the text data to be read aloud is not received, so the CPU 11 refers to, for example, a database in which the words to be recognized are registered for the text data that is the result of voice recognition, and if the text data contains a word that does not exist in this database, it determines that the pronunciation of that word is incorrect. If the word that has been voice recognized is not included in the database, the word obtained as a result of that voice recognition can also be said to be a non-existent word.

In this case, the CPU 11 may compare the word obtained as a result of the speech recognition with a plurality of pre-registered words to be the target of speech recognition, and determine as an incorrectly pronounced word a pre-registered word whose degree of match (recognition accuracy, i.e., recognition likelihood) with the word obtained as a result of the speech recognition is below a predetermined reference value. Furthermore, in one embodiment, the CPU 11 may determine as an incorrectly pronounced word the pre-registered word with the greatest degree of match.

The degree of match is calculated, for example, by converting input data (audio data) of speech read aloud by the user into pronunciation data (phonetic symbols), comparing the sequence of the pronunciation data in the audio data with the sequence of the pronunciation data of pre-registered words, and deriving the ratio of the number of correctly pronounced characters to the total number of characters that make up the words being compared. The pronunciation data of pre-registered words is registered in a database in advance in association with the words. The degree of match may also be used to determine whether the pronunciation is correct or incorrect, as described above.

Furthermore, the CPU 11 can display all words extracted by speech recognition processing of the read aloud on the touch panel display unit 17 and allow the user to select any word to generate extracted word data 12e. In other words, the user designates any misread (misrecognized) word from the words displayed on the touch panel display unit 17 as a word to be pronounced incorrectly.

Thereafter, the CPU 11 stores the extracted word data 12e indicating the word determined to be incorrectly pronounced in the memory 12, and executes the same processes as described above (steps A9 to A15).
In this way, even when the electronic dictionary 10 is not linked to the tablet PC 20, it can use the voice recognition function of the electronic dictionary 10 to perform a reading aloud judgment on a sentence read aloud by the user, and display a learning method list based on words that are judged to be incorrectly pronounced as a result of the judgment, thereby achieving the same effect as when linked to the tablet PC 20.

In the above explanation, the electronic dictionary 10 and tablet PC 20 are described as independent electronic devices, but as long as the functions of the electronic dictionary 10 and tablet PC 20 (digital textbook) described above are present, the functions of the electronic dictionary 10 and tablet PC 20 described above can be realized in a single electronic device.

In the above explanation, the electronic dictionary 10 determines whether the pronunciation of words included in a sentence is correct by comparing the text data resulting from the voice recognition with the sentence data received from the tablet PC 20, but the tablet PC 20 may also perform the pronunciation determination. In this case, the tablet PC 20 has a voice recognition function (not shown), and transmits word data to the electronic dictionary 10 indicating words determined to be incorrectly pronounced as a result of the accuracy determination of the voice input from the voice input unit 28. The electronic dictionary 10 creates a list of misread words (creates an extracted word list) based on the word data received from the tablet PC 20. In this case, the tablet PC 20 does not need to transmit the sentence data to be read aloud to the electronic dictionary 10.

Furthermore, in the above explanation, the learning support system is described as having the tablet PC 20 connected to the electronic dictionary 10 and performing cooperative processing, but the electronic dictionary 10 and the server may work together to perform the same processing as described above, or processing may be performed solely in cooperation with the server instead of the electronic dictionary 10.

Figure 14 is a schematic diagram of a learning support system according to another embodiment of the present invention. The learning support system 1 of this embodiment further includes a server 40. Figure 14 shows a functional block diagram illustrating the configuration of the electronic circuitry of the server 40. The configuration and operation of this embodiment that are not specifically described are the same as those of the above-described embodiment, so redundant description will be omitted.

The server 40 is provided as an electronic device (external electronic device) in the same way as the electronic dictionary 10, and is used, for example, in cooperation with the electronic dictionary 10 or in place of the electronic dictionary 10. By providing the server 40 with a voice recognition function (voice recognition program) in the same way as the electronic dictionary 10, it is possible to perform reading aloud determination using the voice recognition function (including determining whether a word is pronounced incorrectly) realized by the electronic dictionary 10 and the server 40 working together, or using only the voice recognition function of the server 40.

The server 40 has the configuration of a computer that reads programs recorded on various recording media or transmitted programs and whose operation is controlled by the read programs, and its electronic circuitry is equipped with a CPU (central processing unit) 41.

The CPU 41 functions as a control unit that controls the entire server 40. The CPU 41 controls the operation of each circuit component in accordance with a control program pre-stored in the memory 42, a control program loaded into the memory 42 from a recording medium 43 such as a ROM card via the recording medium reader 44, or a control program downloaded from an external device (such as a server) and loaded into the memory 42 via a network N such as the Internet.

Connected to the CPU 41 are a memory 42, a recording medium reader 44, a communication unit 45, and the like. Control programs stored in the memory 42 include a dictionary control program 42a, a voice recognition program 42b, and a learning method processing program 42c. Note that the programs and data stored in the memory 42 shown in Figure 14 are essentially the same as the programs and data of the same names shown in Figure 1, and therefore will not be described here.

Furthermore, each part of the server 40 has essentially the same function as each part of the electronic dictionary 10 with the same name shown in Figure 1, so a description of each part will be omitted.

First, we will explain the case where the electronic dictionary 10 and the server 40 work together. That is, the CPU 11 (controller) of the electronic dictionary 10 and the CPU 41 (controller) of the server 40 work together based on their respective programs to function as the control unit of the learning support system.

For example, in the above description, speech recognition is performed using the speech recognition function (speech recognition program 12b) of the electronic dictionary 10. However, speech recognition may also be performed using the speech recognition function (speech recognition program 42b) of the server 40. In this case, the learner's speech to be recognized is input through the speech input unit 18 of the electronic dictionary 10, as described above. The speech data and text data to be read aloud received from the tablet PC 20 can be transmitted from the electronic dictionary 10 to the server 40, where speech recognition processing (including determining whether words are pronounced incorrectly) can be performed. Alternatively, the tablet PC 20 may input speech reading aloud from the speech input unit 28 along with specifying the text range to be read aloud (FIG. 5, steps B5 and B6). In this case, the speech data of the speech input from the speech input unit 28, together with the text data to be read aloud, can be transmitted to the server 40 via the electronic dictionary 10, where speech recognition processing can be performed.

Generally, the server 40 has higher processing power than electronic devices such as the electronic dictionary 10. Therefore, by having the server 40 perform speech recognition processing, which involves a heavy processing load, the load on the electronic dictionary 10 can be significantly reduced, improving the overall processing efficiency.

In addition to speech recognition processing, for example, the function of providing a learning method list for the electronic dictionary 10 (processing of the learning method processing program 12c) may be executed using the function of the learning method processing program 42c of the server 40. In this case, the results of the processing executed by the learning method processing program 42c of the server 40 are transmitted to the tablet PC 20 via the electronic dictionary 10, and the learning method list as the processing result is displayed in the same manner as described above.

Next, we will explain the case where the tablet PC 20 works with only the server 40 instead of the electronic dictionary 10. In this case, the tablet PC 20 works with an external electronic device 50 (e.g., a personal computer) and establishes a link with the server 40 via the electronic device 50 and the network N. The electronic device 50 functions in the same way as the electronic dictionary 10 working with the server 40 described above, and displays a screen corresponding to the results of processing executed by the server 40.

The tablet PC 20 specifies the range of text to be read aloud (Figure 5, steps B5 and B6), and inputs the learner's voice to be recognized from the voice input unit 28. The voice data of the voice is then sent to the server 40 along with the text data of the specified range of text. This allows the server 40 to perform voice recognition processing on the voice spoken by the learner.

Note that other processing by the server 40 (CPU 11) is performed in the same manner as in the flowchart shown in Figure 4, and therefore will not be described here.

In addition, in the above explanation, the learner's voice is input into the tablet PC 20, but if the electronic device 50 is equipped with a voice input unit (microphone), the voice of the sentence specified on the tablet PC 20 may be input from the electronic device 50, similar to the electronic dictionary 10 described above. In this case, the electronic device 50 transmits the input voice data to the server 40 along with the sentence data received from the tablet PC 20.

In addition, although the above explanation describes the tablet PC 20 linking with the server 40 via the external electronic device 50, the tablet PC 20 may also link with the server 40 without going through the electronic device 50. In this case, the server 40 executes processing in the same manner as the electronic dictionary 10 described above, and displays the screen resulting from the processing on the display unit (touch panel display unit 27) of the tablet PC 20.

In this way, the learning support system of this embodiment can effectively use the learning functions to learn a language by using the electronic dictionary 10, by having the electronic dictionary 10 work in cooperation with the server 40, or by using the server 40 instead of the electronic dictionary 10 to execute support processes. Note that the methods described in the embodiments, i.e., the methods such as the processes shown in the flowcharts, can be stored and distributed as programs that can be executed by a computer on recording media such as memory cards (ROM cards, RAM cards, etc.), magnetic disks (flexible disks, hard disks, etc.), optical disks (CD-ROMs, DVDs, etc.), and semiconductor memories. The computer can then read the programs stored on the external recording media and, by having its operation controlled by these programs, can realize processes similar to the functions described in the embodiments.

In addition, the program data for implementing each method can be transmitted over a network (Internet) in the form of program code, and the program data can be imported from a computer connected to this network to achieve functions similar to those of the above-mentioned embodiments.

The present invention is not limited to the embodiments, and various modifications can be made in the implementation stage without departing from the gist of the invention. Furthermore, the embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining the multiple disclosed constituent elements. For example, even if some constituent elements are deleted from all of the constituent elements shown in the embodiments, or some constituent elements are combined, if the problem stated in the "Problem to be Solved by the Invention" section can be solved and the effect stated in the "Effects of the Invention" section can be obtained, then the configuration in which these constituent elements are deleted or combined can be extracted as an invention.

The inventions described in the original claims of this application are as follows:

[1]
outputting a list of learning methods that can be implemented for words that are determined to be incorrectly pronounced among the words included in the speech when the user reads the sentence aloud;
An electronic device having a control unit that executes a learning function using a learning method selected from the learning method list.

[2]
The control unit
A speech recognition process is performed on the speech to determine whether the pronunciation of words included in the sentence is correct or incorrect;
Extract words that are judged to be incorrectly pronounced,
The electronic device according to [1], wherein the learning method list is output for the extracted words.

[3]
The control unit
receiving text data of a text to be read aloud from another device;
By comparing the sentence data with the results of speech recognition of the speech read aloud, the pronunciation of words is determined to be correct;
The electronic device according to [1], wherein the electronic device transmits words determined to be incorrect to the other device.

[4]
The control unit
causing an external electronic device to perform a speech recognition process including determining whether a word in the speech is mispronounced;
The electronic device according to [1], wherein the learning method list is output based on the determination result by the external electronic device.

[5]
The control unit
The electronic device according to [1], wherein a learning function using a learning method selected from the learning method list is executed on the word determined to be mispronounced.

[6]
The control unit
The electronic device according to any one of [1] to [5], which executes a learning function for one of a plurality of contents included in the learning method selected from the learning method list.

[7]
The control unit
The electronic device according to [6], wherein when the learning function is executed, a part of the content including the word whose pronunciation is determined to be incorrect is displayed on a display device.

[8]
The control unit
If the words obtained by speech recognition of the speech read aloud are not included in the pre-registered words targeted for speech recognition,
The electronic device described in [1] determines that a pre-registered word whose degree of match with the word obtained by the speech recognition is below a predetermined reference value is an incorrectly pronounced word.

[9]
The speech recognition process includes:
The electronic device according to either [2] or [4], further comprising a process of performing speech recognition for each of the speeches pronounced in a plurality of different languages.

[10]
In an electronic device having a control unit,
The control unit
Displaying the text to be read aloud on a display device;
Transmitting the sentence data of the sentence to another device;
receiving word data indicating words in the sentence that have been determined to be incorrectly pronounced by the other device;
The electronic device displays the word determined to be incorrectly pronounced based on the word data in a display format different from other words.

[11]
outputting a list of learning methods that can be implemented for words that are determined to be incorrectly pronounced among the words included in the speech when the user reads the sentence aloud;
A learning support system having a control unit that executes a learning function according to a learning method selected from the learning method list.

[12]
a first electronic device having a first control unit;
a second electronic device having a second control unit;
Equipped with
The first control unit
outputting a list of learning methods that can be implemented for words that are determined to be incorrectly pronounced among the words included in the speech when the user reads the sentence aloud;
execute a learning function according to a learning method selected from the learning method list;
The second control unit is
Displaying the text to be read aloud on a display device;
Transmitting text data of the text to the second electronic device;
receiving word data indicating words in the sentence that have been determined to be incorrectly pronounced by the second electronic device;
displaying the word determined to be incorrectly pronounced based on the word data in a display form different from other words;
Learning support system.

[13]
A learning processing method in an electronic device having a control unit,
The control unit
outputting a list of learning methods that can be implemented for words that are determined to be incorrectly pronounced among the words included in the speech when the user reads the sentence aloud;
a learning processing method for executing a learning function according to a learning method selected from the learning method list;

[14]
Computer,
outputting a list of learning methods that can be implemented for words that are determined to be incorrectly pronounced among the words included in the speech when the user reads the sentence aloud;
a program for causing the program to function to execute a learning function according to a learning method selected from the learning method list;

1...Learning support system, 10...Electronic dictionary, 11, 21...CPU, 12, 22...Memory, 12a...Dictionary control processing program, 12b...Speech recognition program, 12c...Learning method processing program, 12d...Dictionary data, 12e...Extracted word data, 12f...Learning method search data, 12g...Learning content data, 13, 23...External recording medium, 14, 24...Recording medium reading unit, 15, 25...Communication unit, 16...Key input unit, 26...Button input unit, 17, 27...Touch panel display unit, 18...Speech input unit, 19...Speech output unit, 20...Tablet PC, 22a...Basic program, 22b...Textbook processing program, 22c...Text data, 22d...Extracted word data, 30, 40...Server.

Claims (12)

performing a speech recognition process on the speech when the user reads the sentence aloud, and then determining whether the pronunciation of words included in the sentence is correct;
extracting from the sentence mispronounced words determined to have the pronunciation incorrect from among the words included in the sentence as a result of the accuracy determination;
The extracted misread words are changed to original words having the misread words as their original forms;
outputting the original word obtained by changing the misread word;
outputting a learning method list that lists learning methods that can be implemented for the original word;
An electronic device having a control unit that executes a learning function using a learning method selected from the learning method list. The control unit
Accepting a selection of any one of the root words;
The electronic device according to claim 1 , wherein a learning method list is output, which lists feasible learning methods from among a plurality of learning methods for the selected original word. The control unit
receiving text data of a text to be read aloud from another device;
By comparing the sentence data with the results of speech recognition of the voice when the sentence is read aloud, the pronunciation of words included in the sentence is determined to be correct or incorrect;
The electronic device according to claim 1 , wherein the misread word is transmitted to the other device. The control unit
causing an external electronic device to perform a speech recognition process including determining whether a word in the speech is mispronounced;
The electronic device according to claim 1 , wherein the learning method list is output based on a result of determination by the external electronic device. The control unit
The electronic device according to claim 1 , wherein a learning function according to a learning method selected from the learning method list is executed on the misread words as a processing target. The control unit
5. The electronic device according to claim 1, wherein a learning function is executed for one of a plurality of contents included in a learning method selected from the learning method list. The control unit
The electronic device according to claim 6 , wherein when the learning function is executed, a portion including the mispronounced word is displayed on a display device. The control unit
8. The electronic device according to claim 7, wherein, when a word obtained by speech recognition of the speech of the sentence read aloud is a pre-registered word to be the target of speech recognition and the degree of match with the word obtained by the speech recognition is equal to or less than a predetermined reference value, the pre-registered word to be the target of speech recognition is determined to be the misread word. The speech recognition process includes:
The electronic device according to claim 2 or 4, further comprising a process for performing speech recognition for each of the speeches pronounced in a plurality of different languages. a first electronic device having a first control unit;
a second electronic device having a second control unit;
Equipped with
The first control unit
performing a speech recognition process on the speech when the user reads the sentence aloud, and then determining whether the pronunciation of words included in the sentence is correct;
extracting from the sentence mispronounced words determined to have the pronunciation incorrect from among the words included in the sentence as a result of the accuracy determination;
The extracted misread words are changed to original words having the misread words as their original forms;
outputting the original word obtained by changing the misread word;
outputting a learning method list that lists learning methods that can be implemented for the original word;
execute a learning function according to a learning method selected from the learning method list;
transmitting word data indicating the misread word to the second electronic device;
The second control unit is
Displaying the text to be read aloud on a display device;
Transmitting text data of the text to the first electronic device;
receiving the word data indicating the misread word from the first electronic device;
The learning support system displays the mispronounced words in a display format different from other words based on the word data. A learning processing method in an electronic device having a control unit,
The control unit
performing a speech recognition process on the speech when the user reads the sentence aloud, and then determining whether the pronunciation of words included in the sentence is correct;
extracting from the sentence mispronounced words determined to have the pronunciation incorrect from among the words included in the sentence as a result of the accuracy determination;
The extracted misread words are changed to original words having the misread words as their original forms;
outputting the original word obtained by changing the misread word;
outputting a learning method list that lists learning methods that can be implemented for the original word;
a learning processing method for executing a learning function according to a learning method selected from the learning method list; Computer,
performing a speech recognition process on the speech when the user reads the sentence aloud, and then determining whether the pronunciation of words included in the sentence is correct;
extracting from the sentence mispronounced words determined to have the pronunciation incorrect from among the words included in the sentence as a result of the accuracy determination;
The extracted misread words are changed to original words having the misread words as their original forms;
outputting the original word obtained by changing the misread word;
outputting a learning method list that lists learning methods that can be implemented for the original word;
a program for causing the program to function to execute a learning function according to a learning method selected from the learning method list;