JP5020593B2 - Foreign language learning communication system - Google Patents

Description

  The present invention is a system that supports learning a foreign language by listening to the pronunciation of a native speaker, and particularly relates to a foreign language learning communication system for allowing an infant who has not reached school age to naturally learn a foreign language. Is.

  Human language skills are known to develop significantly in early childhood. In particular, the ability to listen to language is naturally acquired by listening to the conversations and conversations of mothers and parents during early childhood. Therefore, it is extremely efficient for a child to learn a foreign language by listening to the pronunciation of a native speaker during early childhood and learning from the ear. However, it is difficult for teachers who are underdeveloped psychologically to teach in the traditional manner in which teachers give students knowledge as is done at school. If it is desirable to hire native speakers and learn from the ears, it is physically and economically difficult to hire a dedicated teacher who can teach.

  Under these circumstances, there are many cases where the opportunity to conduct foreign language education in early childhood that is most suitable for language learning is missed. In recent years, a system for one-to-one foreign language lessons has already been put into practical use for students who have reached school-age with a remote teacher using a videophone. Since this system allows students to take classes at home at a convenient time, physical constraints can be eased. However, for preschoolers who have not developed the concept of willingness to learn and whose concentration tends to be interrupted, the form of classes cannot be established as described above, and the foreign language education system only provides one-way classes via videophone. So it is difficult for young children to learn foreign languages efficiently.

As mentioned above, learning the infant's native language as a model, it is natural to learn a foreign language through talking in daily life, reading and talking about picture books and nursery rhymes, or conversations between family members and children. They are efficient and can be achieved by interpreting their native language into a foreign language and listening to it at the same time. For this purpose, simultaneous interpretation is necessary. In order to perform simultaneous interpretation economically, it is desirable to pool simultaneous interpreters and allocate simultaneous interpretation only when necessary. Patent Document 1 or Patent Document 2 has been proposed as a system for pooling simultaneous interpreters and managing the performance of interpretation work based on reservations or managing the work of interpreters.
JP 2003-36304 A Japanese Patent Laid-Open No. WO 01/084402

In the daily life of infants who are not well-developed, both mother and guardian talks and family conversations in their native language are translated into foreign languages in real time and both the native language and foreign language are heard simultaneously. The main purpose of this system is to support language development.
Foreign languages that are simply translated and spoken can be caught as noise by young children. Therefore, there is a need for a technique for naturally bringing an infant's attention to the speech uttered after translation.

  Infants can play freely using educational material media such as picture books and nursery rhymes, and they can always connect and converse with an interpreter who is a native speaker when they want to speak in a foreign language. It is necessary to have a technology that solves the constraints of physical distance and time by sharing the information in real time.

It is required to comprehensively evaluate the achievement status of children's ability to use language from the viewpoint of developmental psychology and feed back to future learning methods and contents. For this purpose, technology is needed to efficiently acquire and manage basic data.
A system for managing reservations for simultaneous interpreters and managing work for interpreters must also be provided. In consideration of user convenience, it is desirable that reservation management can be performed by the WEB system.

  In view of the above situation, the present invention translates a mother's or guardian's talk or family conversation in their native language into a foreign language in real time in the daily life of an infant who has not developed concentration. An object of the present invention is to provide a foreign language learning communication system that can support the development of operational skills in both languages by listening to foreign languages simultaneously.

In order to solve the above problems, the present invention provides the following configurations.
A user computer installed on the user side who learns a foreign language and an interpreter computer installed on the interpreter side who interprets to learn a foreign language communicate with the center computer installed in the center via the Internet. A foreign language learning communication system that is freely connected,
The user computer is connected to a user stuffed robot provided on the user side, receives a voice signal transmitted from the user stuffed robot, and a transmission means sends the received voice signal to the center computer. When,
Receiving means for receiving an audio signal from the center computer;
Transmission means for transmitting the audio signal received from the center computer to the user plush robot;
The user stuffed toy robot has transmission means for transmitting the audio signal collected by the sound collection means to the user computer;
Receiving means for receiving an audio signal transmitted from the user computer;
Audio reproduction means for reproducing the received audio signal;
The interpreter computer receives a voice signal transmitted from the center computer;
An audio reproduction means for reproducing the received audio signal;
Sound collecting means for collecting the sound interpreted by the interpreter based on the reproduced sound;
Transmission means for transmitting the collected audio signal to the center computer,
The center computer has a receiving means for receiving an audio signal from the user computer;
Transmitting means for transmitting an audio signal received from the user computer to the interpreter computer;
Receiving means for receiving an audio signal from the interpreter computer;
Transmission means for transmitting the audio signal received from the interpreter computer to the user computer,
The interpreter stuffed robot is provided on the interpreter side, the interpreter stuffed robot is a measuring means for measuring changes in the hand, foot, or facial expression of the interpreter stuffed robot due to the operation of the interpreter;
Transmission means for transmitting the measurement signal of the measurement means to the interpreter computer,
The interpreter computer comprises a receiving means for receiving the measurement signal from the interpreter plush robot;
A transmission means for transmitting the received measurement signal to the center computer;
The center computer includes receiving means for receiving the measurement signal from the interpreter computer;
Robot control relay means for generating a control signal for relaying the movement of the interpreter stuffed robot to the user stuffed robot based on the measurement signal;
Transmission means for transmitting the control signal to the user computer,
The user computer has receiving means for receiving the control signal from the center computer;
A transmission means for transmitting the received control signal to the user plush robot;
The user stuffed robot has a receiving means for receiving the control signal from the user computer;
An object of the present invention is to provide a foreign language learning communication system comprising an actuator that changes the hand, foot, or facial expression of a user stuffed robot based on a received control signal.

In the invention according to claim 2 , a user computer installed on the user side who learns a foreign language and an interpreter computer installed on the interpreter side who interprets to learn a foreign language are installed in the center. A foreign language learning communication system connected to a center computer via the Internet,
The user computer includes sound collecting means for collecting a conversation on the user side,
Transmitting means for transmitting the sound signal collected by the sound collecting means to the center computer;
Receiving means for receiving an audio signal from the center computer;
Audio reproduction means for reproducing the received audio signal;
The interpreter computer receives a voice signal transmitted from the center computer;
An audio reproduction means for reproducing the received audio signal;
Sound collecting means for collecting the sound interpreted by the interpreter based on the reproduced sound;
Transmission means for transmitting the collected audio signal to the center computer,
The center computer has a receiving means for receiving an audio signal from the user computer;
Transmitting means for transmitting an audio signal received from the user computer to the interpreter computer;
Receiving means for receiving an audio signal from the interpreter computer;
Transmission means for transmitting the audio signal received from the interpreter computer to the user computer,
The interpreter stuffed robot is provided on the interpreter side, the interpreter stuffed robot is a measuring means for measuring changes in the hand, foot, or facial expression of the interpreter stuffed robot due to the operation of the interpreter;
Transmission means for transmitting the measurement signal of the measurement means to the interpreter computer,
The interpreter computer comprises a receiving means for receiving the measurement signal from the interpreter plush robot;
A transmission means for transmitting the received measurement signal to the center computer;
The center computer includes receiving means for receiving the measurement signal from the interpreter computer;
Robot control relay means for generating a control signal for relaying the movement of the interpreter plush robot to the user computer based on the measurement signal;
Transmission means for transmitting the control signal to the user computer,
The user computer includes a liquid crystal screen,
A stuffed toy exposing the liquid crystal screen and encapsulating the user computer;
Receiving means for receiving the control signal from the center computer;
Display control means for changing the control signal into a display signal;
Display means for displaying the display signal on the liquid crystal screen,
The sound collecting means and sound reproducing means is to provide a language learning communication system, characterized in that provided in said stuffed animal.

The invention according to claim 3, wherein the user computer according to claim 1 or 2, characterized in that it comprises a voice conversion means for converting the voice quality of speech signals received from the center computer to a predetermined voice quality A foreign language learning communication system is provided.

The invention according to claim 4 is characterized in that the user computer transmits a position signal of the teaching material being referred to by the user computer to the center computer;
Receiving means for receiving, from the center computer, a position signal of the teaching material being referred to by the interpreter computer;
Teaching material sharing display means for displaying the same teaching material position as the teaching material position referred to by the interpreter based on the received position signal,
The center computer includes teaching material relay means for relaying and transmitting the position signal received from the user computer to the interpreter computer and relaying and transmitting the position signal received from the interpreter computer to the user computer,
The interpreter computer has transmitting means for transmitting a position signal of the teaching material referred to by the interpreter computer to the center computer;
Receiving means for receiving a position signal of the teaching material referred to by the user from the center computer;
Based on the received position signal, foreign according to any one of claims 1 to 3, characterized in that it comprises a materials shared display means for displaying the same materials position as teaching position where the user computer to see A language learning communication system is provided.

The invention according to claim 5 is characterized in that the center computer records an interpreter work database that records the work situation of the interpreter;
Receiving means for receiving the interpreter's work status from the interpreter computer;
Registration means for registering the received work status in the interpreter work database;
Receiving means for receiving a reservation for interpretation from the user computer;
Based on the book of the received interpretation, wherein from the interpretation work database, in any one of claims 1 to 4, characterized in that a selection means for selecting the interpreter and time that matches the reserved Provide a foreign language learning communication system.

The invention according to claim 6 is characterized in that the center computer is provided with a calculation means for periodically calculating the work results of the interpreter and the charge to the user from the interpreter work database. The foreign language learning communication system according to 5 , is provided.

In the invention according to claim 7 , the center computer includes a vocabulary list in which vocabularies are registered in advance,
A learning achievement management database for registering or referring to the learning achievement status of the learning target person,
A recognition means for recognizing the speech of the person being studied,
A cut-out means for cutting out a word using the recognized voice of the person to be learned as input,
A comparison means for comparing the extracted word with the vocabulary registered in the vocabulary list;
If there is a matched word, a counting means for counting the number of matches,
Recording means for recording the counted number of matches in the learning achievement management database;
Registration means for registering the learning achievement degree evaluation information received from the interpreter computer and the learning achievement degree evaluation information input to the center computer by the center instructor in the learning achievement degree management database;
The interpreter computer is connected to the center computer, and includes reference means for freely referencing learning achievement level evaluation information in the learning achievement level management database,
The said user computer is connected to the said center computer, The learning achievement degree evaluation information in the said learning achievement degree management database is provided with the reference means which can refer freely on predetermined conditions. A foreign language learning communication system according to any one of 1 to 6 is provided.

According to the first aspect of the present invention, a foreign language can be heard in real time with pronunciation of a native speaker at the same time as a native language in an infant or the like that has been physically and economically difficult. Infants who have excellent ability to learn languages through ears compared to adults, but have difficulty in learning through classes because they are not well-developed, so they can learn foreign language skills naturally in daily life and play Become.
The interpreter can work in the center or work at home. Therefore, because interpreters can be recruited not only from domestic residents but also from overseas residents, it is possible to improve the working conditions of interpreters at night by requesting interpreters from overseas.
Furthermore, it is possible to attract the child's attention effectively by producing a sound from the stuffed animal. In addition, since sound is collected by the sound collecting means provided for the stuffed toy, effective sound collection is possible.
And, by operating the interpreter plush robot according to the content of the conversation, the movement of the user plush toy robot can be controlled, and the movement of the user plush toy robot can effectively attract the attention of the infant it can.

According to the invention described in claim 2 , a robot or a stuffed toy that moves according to the movement of the interpreter plush robot on the liquid crystal screen of the user's personal computer by operating the interpreter plush robot according to the content of the conversation. Can be displayed and the attention of the infant can be attracted effectively by the displayed video.
Further, the display means can be provided at a relatively low cost than the actuator. Furthermore, the user plush toy robot provided with the actuator becomes unnecessary, and the cost can be reduced.

According to the invention described in claim 3 , for example, it can be converted into a voice quality having characteristics such as a male voice, a female voice, a high voice and a low voice in accordance with the characteristics of the character robot. It can be attracted effectively.

According to the fourth aspect of the present invention, an infant can freely play using teaching material media prepared in advance, and an interpreter who is a play partner shares teaching material media with the user in real time so that the interpreter's It can reduce the restraint time and increase the learning efficiency.

According to the fifth aspect of the present invention, it is possible to register and manage the working situation, and it is possible to easily select an interpreter and time that match the reservation based on the reservation of the interpreter.

According to the invention described in claim 6 , since the center computer periodically calculates the working results of the interpreter and the charge to the user, it becomes easy to pay the interpreter and charge the user. .

According to the invention of claim 7 , the learning achievement level of the learning subject can be managed and referred to, and the expert can learn the language ability, intelligence, and mental development status of the child who is the learning subject. Because it can monitor and give appropriate feedback, it is possible to respond appropriately and provide guidance for parents who are concerned about childcare.

Hereinafter, embodiments of the present invention will be described with reference to the drawings showing examples.
The overall configuration of the present invention is shown in FIG. The center computer 101 installed in the center, the user personal computer 102 which is a user computer installed in the user's home, and the interpreter personal computer 103 which is an interpreter computer are connected using a network 104 such as the Internet.

  The user's family conversation is transmitted to the interpreter's personal computer 103 via the user's personal computer 102, the network 104 and the center computer 101. The interpreter uses the interpreter's personal computer 103 to listen to the conversation in the user's native language, translates it into a foreign language, and sends it back to the user's personal computer 102 via the network 104 and the center computer 101. At the user's home, educated infants and others can listen to conversations that are interpreted in real time in their native language and in a foreign language, so the operational ability of both languages can be developed simultaneously.

  The translated conversation is monitored by the center staff from the computer terminal 105 and simultaneously stored in the external storage device 106. A user stuffed robot (child device) 107 connected to the user personal computer 102 operates in conjunction with an interpreter stuffed robot (parent device) 108 on the interpreter side. The interpreter can attract the child's attention effectively by operating the interpreter plush toy robot 108 according to the content of the conversation at the same time as the interpretation work.

  FIG. 2 shows a hardware configuration example of the user side system. A user's conversation is collected by a microphone 202 which is a sound collecting means, and analog voice information is converted and compressed into digital information by a user personal computer 102. The voice information converted into digital information is transmitted from the user personal computer 102 to the center computer 101 and the interpreter personal computer 103 via the network 104. Reservation registration at the center of interpreter request time and selection of interpreters are performed using the WEB system. The voice information resulting from the interpretation by the interpreter is also returned to the user personal computer 102 via the network 104 as digital information. This audio information is converted into analog information using the user's personal computer 102 and listened to from the speaker 203 which is an audio reproduction means.

  Instead of connecting the microphone 202 and the speaker 203 directly to the personal computer, the user stuffed robot 107 incorporating them may be connected to the user personal computer 102 by wire or wireless to enhance the learning effect. Assume that the user plush toy robot 107 can be remotely operated by an interpreter from a remote location by a mechanism described later. The purpose of using the user plush toy robot 107 is to gather the attention of the infant to be trained in the voice of the interpreter.

Instead of using the user plush toy robot 107, a user plush toy 1501 with a liquid crystal screen in which a personal computer main body, a liquid crystal monitor, a microphone and a speaker are integrated with the plush toy as shown in FIG.
It is assumed that teaching materials such as fairy tales and nursery rhymes can be used online or using teaching material media 205 using a user personal computer 102 installed in the user's home.

  FIG. 3 shows a hardware configuration example of the interpreter side system. The voice information requested by the user for interpretation is received by the interpreter personal computer 103 via the network 104. The audio information sent as compressed digital information is expanded and converted into analog information by the interpreter personal computer 103 and listened to by the headphones 303 as the audio reproducing means of the headset 302 connected to the interpreter personal computer 103. The interpretation result is sent to the interpreter's personal computer 103 through the microphone 304 which is a sound collecting means, converted into digital information, compressed and compressed, and transmitted to the user's personal computer 102 via the center computer 101 using the network 104. There is an interpreter plush robot 108 on the interpreter side, and when the interpreter operates the limbs and facial expressions of the interpreter plush robot 108, the operation content is transmitted to the user plush toy robot 107 on the user side. The user stuffed toy robot 107 imitates the operation of the interpreter stuffed toy robot 108 by the action of a robot control relay program, which will be described later, which is programmed to follow the operation of the interpreter stuffed toy robot 108.

  When an interpreter starts his daily work, he / she registers that he / she has started work using a center WEB system which will be described later. At the same time, the scheduled interpretation time for the day will be confirmed using the WEB system. When a call lamp 305 can be connected to the interpreter's personal computer 103 and there is a business contact from the interpreter center or an interpretation request from the user, the interpreter uses the blinking of the call light 305 and the beep sound of the interpreter's personal computer 103 to inform the interpreter. A notification system is used.

  FIG. 4 shows a configuration example of the center system. The interpretation center system includes a center computer 101 for performing interpretation support work processing, a network router 402 for connection to a network 104 such as the Internet, an external storage device 106, a printing device 404, and a computer terminal 105. As main business programs, a voice communication control program 406 for relaying voice information between the user personal computer 102 and the interpreter personal computer 103, and a communication log recording program for recording communication contents between the user personal computer 102 and the interpreter personal computer 103. 407, a robot control relay program 408, a teaching material relay program 409, and a learning achievement automatic evaluation program 4010, which are programs for performing real-time processing.

  The voice communication control program 406 has a function of relaying voice information between the user personal computer 102 and the interpreter personal computer 103. Simultaneously with the recording of the communication log, the center staff can monitor the voice information during the communication with the computer terminal 105. The communication log record is a communication record between the user personal computer 102 and the interpreter personal computer 103, and is used later for quality assurance, quality improvement, and learning achievement survey.

  The robot control relay program 408 generates a control signal for relaying the movement of the interpreter stuffed robot 108 operated by the interpreter to the user stuffed robot 107. The control signal is sent from the center computer 101 to the user stuffed robot 107 via the user personal computer 102.

  The learning material relay program 409 notifies the interpreter personal computer 103 of the position of the educational material being referred to by the user personal computer 102 and relays the position of the educational material designated by the interpreter personal computer 103 to the user personal computer 102.

  As shown in FIG. 19, the learning achievement level automatic evaluation program automatically records the vocabulary that can recognize and speak the speech of the person to be learned in the learning achievement level management database described later in the following procedure. is there. A word is cut out using the voice from the user personal computer 102 monitored by the center computer 101 as an input (1901). A vowel formant position of the extracted word is obtained, and it is confirmed whether the speaker is a learning target (1902). At this time, in the case of family utterance, the following processing is not performed. Here, it is assumed that the formant of the target person's speech is registered in the system in advance, and the speaker is obtained by extracting from speech data to higher-order formant components using a digital filter program and comparing it with a sample collected in advance. Can be identified. Next, the uttered word is recognized, and it is checked whether there is a vocabulary list stored in the external storage device 106 that matches a word of a vocabulary registered in advance (1905). If there is a word that matches the vocabulary list, the number of utterances of the management target vocabulary in the learning achievement degree management database is counted up as an utterance record by the learner (1906).

  The technology for identifying words from speech data is a digital filter as shown in the non-patent document “Real-time speech recognition” (edited by IEICE Akio Ando) and the non-patent document “Speech recognition system” (edited by IPSJ Kiyohiro Shikano). -Although it can be implemented by a technology applying a program, recently there is a product that has been commercialized as a commercial LSI, and such LSI may be adopted and implemented as firmware. Such mechanical speech recognition is a technology in the development stage, and the recognition rate is slightly lower than that of humans, and the recognition accuracy is limited. Therefore, the learning achievement status other than vocabulary such as syntax and pronunciation is evaluated by an interpreter or a center instructor listening to the ear and registered in the learning achievement management database using the learning achievement evaluation WEB program. For vocabulary data, the machine processing result may be corrected using the WEB system.

  In FIG. 4, an interpreter reservation WEB system 4011, an interpreter work management WEB system 4012, an interpreter work schedule WEB system 4013, and a learning achievement level evaluation WEB system 4014 are implemented as a WEB system with emphasis on operability. The interpreter reservation WEB system 4011 is a system for accepting a user's interpreter reservation. The interpreter / employment management WEB system 4012 accesses an interpreter / employee database corresponding to the interpreter's attendance book and registers the work status. The interpreter work schedule WEB system 4013 has a function in which the interpreter's work schedule is pre-registered in the interpreter work database. As shown in FIG. 20, the learning achievement degree evaluation WEB system has a function of accessing the learning achievement degree management database 2002 and registering or referring to the learning achievement status of the person to be learned, and when interpreters and center staff register learning evaluations. Used for. Further, it is assumed that some data can be referred from the user personal computer 102 side, and the reference range of the user data is limited to the child of the user, and the learning achievement evaluation web system 4014 based on the user register. Control. The evaluation items are designed so that summary display and detailed display 2004 (see FIG. 20) are performed.

The charging program 4015 and the communication log reference program 4016 are batch processing programs. The billing program 4015 is a batch processing program that aggregates the interpreter's working hours and the user's usage hours and bills salaries and charges. Center staff can refer to the communication log using the communication log reference program for quality assurance. These center operations mainly operate the computer terminal 105 to start / stop programs and refer to data. A plurality of computers that process these center operations may be installed in accordance with the scale of the center, and the load may be distributed by sharing a database.
Various data files are stored in the external storage device 106, and the database is shared among the programs. A list of main data files is shown in FIG.

  FIG. 5 shows an outline of the user side system. The user-side system uses the user personal computer 102 described above. The user personal computer 102 includes a CPU 502, an audio interface HW 503, a network interface HW 504, and hardware (HW) of a robot control device HW 5011. The audio interface HW converts the analog audio signal input from the microphone into a digital signal having a predetermined sampling frequency by the PCM method. Conversely, a digital signal having a predetermined sampling frequency is input from a personal computer to generate an analog audio signal, and audio data can be sent to an amplifier / speaker. Further, the network interface HW 504 is connected to the modem device 505 and implements a protocol of the OSI model communication protocol layer 3 or lower. As the hardware (102, 502, 503, 504, 505, 5011), commercially available products may be used.

  The programs constituting the user side system are composed of a voice compression / decompression processing program 506, a voice communication program 507, a network communication program 508, a WEB browser 509, a robot control program 5010, a liquid crystal display robot control program 5012, and a teaching material sharing program 5013. .

  The audio compression / decompression processing program 506 is an ITU-T G. 723.1 or G.I. The audio data compression / decompression is performed using the hybrid audio codec standardized in H.729. G. 723.1 or G. The 729 codec can be implemented on a personal computer by applying a digital filter program. Since the audio compression / decompression processing program 506 can be distributed as a C language program / source code from the ITU, it may be used.

The compressed voice data is packetized / depacketized by the voice communication program 507. The packetized data is transmitted and received according to the RTP (Real-time Transport Protocol) protocol. The packet format is IETF (Internet Engineering) as shown in FIG.
Task Force) may conform to RFC3551. The packetized data is transmitted and received using the UDP protocol, which is one of the OSI model layer 4 functions provided by the network communication program 508 of the commercial operating system.

  Interpretation work is started by establishing a voice transmission path between the user personal computer 102 and the interpreter via the center when the voice communication program 507 is activated. In this case, connection control may use a protocol based on IETF RFC3261 called SIP (Session Initial Protocol). Although it is possible to program the protocol in accordance with the RFC, recent commercial operating systems for personal computers often provide an RTP protocol and a SIP protocol, which may be used.

As described above, the operation of the user plush toy robot 107 on the user side is controlled by following the operation of the user plush toy robot 107 to the operation of the interpreter plush toy robot 108 on the interpreter side. A technique is used that makes it appear as if the stuffed robot 107 is moving independently. As shown in FIG. 16, a sensor 1601, which is a measuring means for measuring the rotation angle, is incorporated in the limb joint of the interpreter plush robot 108, and the limb joint angle of the interpreter plush robot 108 moved by the interpreter's manual operation is digitized. The data is periodically transmitted to the user plush toy robot 107. The interpreter plush toy robot 108 prepares the robot to express the emotional and emotional expression pattern in the form of eyes, eyebrows, mouths, etc., so that the emotional and emotional pattern can be changed with a switch. In the figure, 1602 and 1605 are eyebrow movable parts as actuators, 1603 and 1606 are mouth movable parts as actuators, and 1604 are limb joints and servomotors as actuators.
Combining the robot's limb joint angles and facial expression patterns, for example, can convey simple emotions with facial expressions and gestures as shown in FIG.

  The limb joint angles (numerical data) and emotional expression patterns (numerical data) of the interpreter plush robot 108 are transmitted from the interpreter personal computer 103 to the robot control program 5010 of the user personal computer 102. The robot control program 5010 outputs a control signal to the robot control device 5011 so as to match the limbs and expressions of the robot with the interpreter plush robot 108 from the transmitted data.

  FIG. 18 shows a robot control data transmission method. The digitized limb joint angle data and emotional expression pattern data of the robot are transmitted by the RTP protocol after establishing a transmission path from the interpreter side to the user side according to the SIP protocol (1806) in the same manner as the voice data (1807). ). The voice transmission path identifies data by dividing the port number.

  Instead of using the mechanic robot as described above, as shown in FIG. 15, the user personal computer is provided with a liquid crystal screen 1501, and the user personal computer is covered with a stuffed toy with the liquid crystal screen 1501 exposed. In addition, a liquid crystal display robot control program 5012 is used to display a graphical robot on the liquid crystal screen in accordance with the interpreter's operation, and the operation of this graphical robot is displayed so as to follow the operation of the parent robot. Good. When this method is used, there is an economic advantage because the robot control device HW and the robot body are unnecessary.

  The user uses the WEB browser 509 to access the interpreter reservation WEB system for users provided by the interpreter center and make an interpreter reservation.

  As shown in FIG. 7, a user interpreting reservation web system 701 provides a user with a web screen 703 compliant with the HTTP protocol, and is responsible for interpreting reservation reservation management. The interpreting reservation web system 701 refers to the interpreter profile database and interpreter work database as shown in the flow 704 in accordance with the user input, and selects the interpreter and time that meet the user's request.

  The interpreting reservation web system 701 updates the reservation information in the interpreting schedule database 702 using the interpreter ID, date / time of the interpreter selected by the user as a key. If the interpreter is unable to work as scheduled due to circumstances, the center system will automatically select the desired order when selecting an interpreter according to the desired order. Can be used. The WEB browser 509 may use a commercial operating system accessory conforming to the HTTP protocol.

  If the user's age and language skills increase and he / she can see and listen to educational materials such as picture books and nursery rhymes, learning using educational media is also effective in addition to listening to interpreted conversations. At this time, in addition to listening to the educational materials on the user's side and playing, interactive communication such as asking questions from the interpreter according to the level of the infant or conversely speaking from the infant to the interpreter is more effective for learning. It is. In that case, it is important for the interactive communication that the interpreter knows the teaching material that the infant is accessing and the page that is being accessed or viewed.

  The educational material sharing program 5013 displays the educational material that the user is viewing on the interpreter's personal computer 103, and conversely, the educational material that the interpreter wants to show to the user can be displayed on the personal computer 102 without the user's operation. It is a program to control as follows. The teaching material sharing program 5013 on both the user side and the interpreter side exchanges the document number as the position signal of the teaching material and the corresponding page number, and displays the teaching material on each personal computer.

  FIG. 21 shows a protocol for sharing teaching materials. Moreover, you may implement the function which transfers and displays the graphical figure data handwritten on the ad lib by the user side or the interpreter side mutually. The educational material sharing data is also transmitted from the interpreter side to the user side in accordance with the RTP protocol and the SIP protocol in the same manner as the voice data. The voice transmission path identifies data by dividing the port number.

The voice communication program 507, the robot control program 5010, and the teaching material sharing program 5013 communicate with each other by setting a session with the corresponding program on the interpreter side by assigning different port numbers (P1, P2, P3) to the same IP address.
The audio data sent from the interpreter is expanded by the audio compression / decompression processing program 506 as described above, and is reproduced by the speaker via the audio interface HW 503.

  FIG. 8 shows an overview of the interpreter side system. The interpreter side system also uses the interpreter's personal computer 103. As with the user-side system, voice data transmission uses a voice communication path built on IP, and the hardware and software configurations (803 to 809) for voice communication are the same as the user-side system, so the explanation is omitted. To do.

  There is a call lamp 305 as hardware unique to the interpreter. The purpose of use of the call lamp 305 is used when the interpreter is called before the interpreter's personal computer 103 in order to contact the interpreter from the center. The call lamp controller 8013 controls on / off of the call lamp 305. The call light controller 8013 may be automatically activated when the operating system is activated. When a SIP call request is received from the center, the call light controller 8013 uses the serial IF of the interpreter personal computer 103 to call the call light control relay SW (switch ) Turn 8014 ON. At the same time, using the media IF provided by the commercial operating system of the interpreter's personal computer 103, a beep sound is generated from the speaker built in the interpreter's personal computer 103, and the interpreter is also notified by a sound.

  The interpreter-side robot control program 8010 and the robot controller 8011 have different functions from the above-described user-side robot control program 8010 and the robot controller 8011, and are generated by the interpreter operating the robot limbs and facial expression switches. The change in the angle of the limbs and the change in the facial expression switch are captured by the sensor and sent to the user side as numerical data. The transmission protocol is as described above.

  When the interpreter's voice is transmitted to the user, the voice quality may be converted by the voice quality conversion program 8016. It is converted into voice quality with characteristics such as male voice, female voice, high voice and low voice according to the characteristics of the character robot. The principle of voice quality conversion is non-patent literature (SP-87-68 LSP "Voice quality conversion using analysis and synthesis system" Tadaaki Shimizu and Kazuhiro Hamada). It has been reported that the characteristics can be realized by an application program of a digital filter. Patents that apply these principles to the voice change of a karaoke apparatus have also been proposed, as shown in the patent document “Patent Publication 2004-177984” and the patent document “Patent Publication H11-338480”.

  The interpreter accesses the interpreter work schedule WEB system 4012 for interpreters provided by the center using the WEB browser 809, and registers and confirms the work schedule.

  As shown in FIG. 9, the interpreter work schedule WEB system provides the interpreter with a WEB screen 903 compliant with the HTTP protocol. According to the interpreter's WEB screen 903 input, as shown in the flow 904, the interpreter work database and the interpreter reservation database 702 are accessed using the scheduled work date / time as a key to update the interpreter work database. As shown in the figure, the scheduled work time may be displayed as a white circle, the time when reservations are filled during the scheduled work time may be displayed as a black circle, or the black circle may have a hyperlink function and the name of the reservation person may be displayed.

  FIG. 10 shows the positioning of each communication program on the user side, center side, and interpreter side in the OSI model. FIG. 11 shows the connection relationship of the OSI model layer 4 and higher. As described above, between the user, the center, and the interpreter, business processing such as interpreting reservation, interpreting work management, and interpreting reservation management work is performed using the web. On the other hand, for voice data, the center implements a connection / relay function 11014 between the user and the interpreter.

  This function is shown in FIG. When the interpreter reservation time is reached and the user requests interpretation, the user activates the voice communication program 507 from the user personal computer 102. As shown in FIG. 12, when this program is started, it requests the voice communication control program 406 on the center side to start voice communication with the interpreter according to the SIP protocol (1201). The voice communication control program 406 searches the interpreter reservation database (1202), and makes a call request to the interpreter's personal computer according to the user's desired priority (1203). If there is no interpretation response after waiting for a certain period of time, a call is made to the next priority interpreter. Upon receiving an off-hook response 1204 from the interpreter, a voice transmission path is opened between the user and the interpreter (1205), and the interpreting operation is started. At the same time, the communication log recording program 407 is activated to record the communication contents between the user and the interpreter in a file (1206). The voice communication control program updates the record of the interpreter / date and time in the interpreter work database and records the start time of the interpreter work (1207).

  The voice communication control program detects the end of the interpretation work by the on-hook response 1208 and closes the voice transmission path (1209). At the same time, the communication log recording program is terminated. At the same time, the record of the interpreter and the corresponding date / time in the interpreter work database is updated to record the interpreter work end time (1210).

  The center staff can refer to the contents of past interpreter communication using the communication log reference program 1301 shown in FIG. The communication log file 1303 may be managed by using the communication log directory 1302, and the user ID, the interpreter ID, the interpreter start date / time, and the interpreter end date / time are registered in the directory. Can also be made easier. Since the communication log records the data of the voice communication channel serially, the data is compressed and the file capacity when saving the log can be saved. When referring to the log, the compressed data can be expanded and digital-analog conversion can be heard as described above. When the decompressed data is stored in a temporary file and referenced, the reference position on the file is moved forward or backward to allow fast forward and rewind reference.

  As shown in FIG. 14, the interpreter work database 1402 is aggregated to periodically calculate the work results of interpreters and billing to users. The total results are reported as an employment report 1403 and a billing report 1404, respectively. The center may make salary payments and billing based on this report.

1 is an overall configuration diagram of a foreign language learning communication system according to the present invention. It is a hardware structural example figure of the user side system by this invention. It is a hardware structural example figure of the interpreter side system by this invention. It is a structural example figure of the interpretation center system by this invention. It is a schematic diagram of the user side system by this invention. FIG. 4 is an RTP packet format diagram according to the present invention. It is explanatory drawing of the interpreter reservation WEB system for users by this invention. It is a schematic diagram of the interpreter side system by this invention. It is explanatory drawing of the interpreter work reservation WEB system for interpreters by this invention. It is explanatory drawing of the positioning in the OSI model of the communication program by this invention. It is a connection relation figure between communication programs by this invention. It is explanatory drawing of the interpretation center connection relay function by this invention. It is explanatory drawing of the interpretation center communication log reference program by this invention. It is explanatory drawing of the interpretation center accounting program by this invention. It is explanatory drawing of the stuffed type | mold liquid crystal screen by this invention. It is explanatory drawing of the joint and expression component of a robot by this invention. It is explanatory drawing of the example of the facial expression gesture of the robot by this invention. It is explanatory drawing of the robot control method by this invention. It is a processing flow of the learning achievement level automatic evaluation program by this invention. It is explanatory drawing of the learning achievement level evaluation WEB system by this invention. It is explanatory drawing of the teaching material sharing process by this invention. It is a main data file list by this invention.

Explanation of symbols

101 center computer 107 user plush toy robot 108 interpreter plush toy robot 2002 learning achievement management database 1402 interpreter employment database 1501 LCD screen

Claims (7)

A user computer installed on the user side who learns a foreign language and an interpreter computer installed on the interpreter side who interprets to learn a foreign language communicate with the center computer installed in the center via the Internet. A foreign language learning communication system that is freely connected,
The user computer is connected to a user stuffed robot provided on the user side, receives a voice signal transmitted from the user stuffed robot, and a transmission means sends the received voice signal to the center computer. When,
Receiving means for receiving an audio signal from the center computer;
Transmission means for transmitting the audio signal received from the center computer to the user plush robot;
The user stuffed toy robot has transmission means for transmitting the audio signal collected by the sound collection means to the user computer;
Receiving means for receiving an audio signal transmitted from the user computer;
Audio reproduction means for reproducing the received audio signal;
The interpreter computer receives a voice signal transmitted from the center computer;
An audio reproduction means for reproducing the received audio signal;
Sound collecting means for collecting the sound interpreted by the interpreter based on the reproduced sound;
Transmission means for transmitting the collected audio signal to the center computer,
The center computer has a receiving means for receiving an audio signal from the user computer;
Transmitting means for transmitting an audio signal received from the user computer to the interpreter computer;
Receiving means for receiving an audio signal from the interpreter computer;
Transmission means for transmitting the audio signal received from the interpreter computer to the user computer,
The interpreter stuffed robot is provided on the interpreter side, the interpreter stuffed robot is a measuring means for measuring changes in the hand, foot, or facial expression of the interpreter stuffed robot due to the operation of the interpreter;
Transmission means for transmitting the measurement signal of the measurement means to the interpreter computer,
The interpreter computer comprises a receiving means for receiving the measurement signal from the interpreter plush robot;
A transmission means for transmitting the received measurement signal to the center computer;
The center computer includes receiving means for receiving the measurement signal from the interpreter computer;
Robot control relay means for generating a control signal for relaying the movement of the interpreter stuffed robot to the user stuffed robot based on the measurement signal;
Transmission means for transmitting the control signal to the user computer,
The user computer has receiving means for receiving the control signal from the center computer;
A transmission means for transmitting the received control signal to the user plush robot;
The user stuffed robot has a receiving means for receiving the control signal from the user computer;
A foreign language learning communication system comprising an actuator that changes a hand, a foot, or a facial expression of a user stuffed robot based on a received control signal.
A user computer installed on the user side who learns a foreign language and an interpreter computer installed on the interpreter side who interprets to learn a foreign language communicate with the center computer installed in the center via the Internet. A foreign language learning communication system that is freely connected,
The user computer includes sound collecting means for collecting a conversation on the user side,
Transmitting means for transmitting the sound signal collected by the sound collecting means to the center computer;
Receiving means for receiving an audio signal from the center computer;
Audio reproduction means for reproducing the received audio signal;
The interpreter computer receives a voice signal transmitted from the center computer;
An audio reproduction means for reproducing the received audio signal;
Sound collecting means for collecting the sound interpreted by the interpreter based on the reproduced sound;
Transmission means for transmitting the collected audio signal to the center computer,
The center computer has a receiving means for receiving an audio signal from the user computer;
Transmitting means for transmitting an audio signal received from the user computer to the interpreter computer;
Receiving means for receiving an audio signal from the interpreter computer;
Transmission means for transmitting the audio signal received from the interpreter computer to the user computer,
The interpreter stuffed robot is provided on the interpreter side, the interpreter stuffed robot is a measuring means for measuring changes in the hand, foot, or facial expression of the interpreter stuffed robot due to the operation of the interpreter;
Transmission means for transmitting the measurement signal of the measurement means to the interpreter computer,
The interpreter computer comprises a receiving means for receiving the measurement signal from the interpreter plush robot;
A transmission means for transmitting the received measurement signal to the center computer;
The center computer includes receiving means for receiving the measurement signal from the interpreter computer;
Robot control relay means for generating a control signal for relaying the movement of the interpreter plush robot to the user computer based on the measurement signal;
Transmission means for transmitting the control signal to the user computer,
The user computer includes a liquid crystal screen,
A stuffed toy exposing the liquid crystal screen and encapsulating the user computer;
Receiving means for receiving the control signal from the center computer;
Display control means for changing the control signal into a display signal;
Display means for displaying the display signal on the liquid crystal screen,
The sound collecting means and sound reproducing means, foreign language learning communication system, characterized in that provided in said stuffed animal. 3. The foreign language learning communication system according to claim 1, wherein the user computer comprises voice quality conversion means for converting the voice quality of the voice signal received from the center computer into a predetermined voice quality. The user computer has a transmitting means for transmitting a position signal of the teaching material being referred to by the user computer to the center computer;
Receiving means for receiving, from the center computer, a position signal of the teaching material being referred to by the interpreter computer;
Teaching material sharing display means for displaying the same teaching material position as the teaching material position referred to by the interpreter based on the received position signal,
The center computer includes teaching material relay means for relaying and transmitting the position signal received from the user computer to the interpreter computer and relaying and transmitting the position signal received from the interpreter computer to the user computer,
The interpreter computer has transmitting means for transmitting a position signal of the teaching material referred to by the interpreter computer to the center computer;
Receiving means for receiving a position signal of the teaching material referred to by the user from the center computer;
Based on the received position signal, foreign according to any one of claims 1 to 3, characterized in that it comprises a materials shared display means for displaying the same materials position as teaching position where the user computer to see Language learning communication system. The center computer includes an interpreter employment database that records the work status of interpreters;
Receiving means for receiving the interpreter's work status from the interpreter computer;
Registration means for registering the received work status in the interpreter work database;
Receiving means for receiving a reservation for interpretation from the user computer;
Based on the book of the received interpretation, wherein from the interpretation work database, in any one of claims 1 to 4, characterized in that a selection means for selecting the interpreter and time that matches the reserved Foreign language learning communication system. 6. The foreign language learning communication system according to claim 5, wherein the center computer is provided with a calculation means for periodically calculating a work record of the interpreter and a charge to the user from the interpreter work database. . The center computer includes a vocabulary list in which vocabularies are registered in advance,
A learning achievement management database for registering or referring to the learning achievement status of the learning target person,
A recognition means for recognizing the speech of the person being studied,
A cut-out means for cutting out a word using the recognized voice of the person to be learned as input,
A comparison means for comparing the extracted word with the vocabulary registered in the vocabulary list;
If there is a matched word, a counting means for counting the number of matches,
And recording means for recording the counted match count in the learning achievement management database,
And a registration means for registering the learning achievement evaluation information instructor learning achievement evaluation information received and the center is input to the center computer from the interpreter's computer to the learning achievement management database,
The interpreter computer is connected to the center computer, and includes reference means for freely referencing learning achievement level evaluation information in the learning achievement level management database,
The said user computer is connected to the said center computer, The learning achievement degree evaluation information in the said learning achievement degree management database is provided with the reference means which can refer freely on predetermined conditions. The foreign language learning communication system according to any one of 1 to 6 .

Images