JP4836885B2 - Non-contact input device for mobile communication terminal, mobile communication terminal and mail creation system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a noncontact input device which enhances operability when a user creates email by means of a mobile communication terminal in a hands-free mode. <P>SOLUTION: The noncontact input device comprises a terminal I/F 1a for communicating with a mobile communication terminal 10; a terminal information acquiring part 2 for acquiring operation key information from the mobile communication terminal 10; a key layout information storage part 3a for storing key layout information; a key layout extracting part 3 for referring to the key layout information based on the operation key information, and extracting key layout corresponding to the operation key information; a display control part 3 for displaying the key layout on a hands-free operation screen 22a; an image pickup device 23 for creating an animation by photographing a subject; a gaze position identifying part 5 for detecting the movement of the subject's eyes through the analysis of the animation, and identifying a gaze position inside the hands-free operation screen 22a based on the detection result; and an operation input part 6 for identifying an operation key corresponding to the gaze position based on the result of identification by the gaze position identifying part 5 and the key layout, and creating operation input information. <P>COPYRIGHT: (C)2009,JPO&amp;INPIT

Description

  The present invention relates to a contactless input device for a mobile communication terminal, a mobile communication terminal, and a mail creation system. More specifically, the present invention relates to a contactless input capable of hands-free operations such as mail creation in a mobile communication terminal Relates to the device.

  2. Description of the Related Art Conventionally, an in-vehicle hands-free e-mail creating apparatus is known as an e-mail creating apparatus for assisting e-mail creation when creating an e-mail while driving in a car (for example, Patent Document 1). The hands-free e-mail creation device described in Patent Document 1 is a terminal device for enabling e-mail creation even if a user who is driving a car does not operate by hand. Done.

Specifically, the terminal device is controlled based on a command designated by voice, and an e-mail having voice data input as a message as attached data is created. The e-mail created in this way is transmitted from the terminal device using the mobile phone as a modem. That is, in this hands-free mail creation device, the terminal device is operated by voice, and the message to be sent is sent as voice data. In the communication terminal that has received such an e-mail, the message is transmitted to the user by reproducing the voice data attached to the e-mail.
JP 2005-222410 A

  In the conventional mail creating apparatus as described above, since the mobile phone is used only as a modem, it is considered that an application program for creating mail and the created mail data exist on the terminal device. For this reason, there is a case where the operation procedure at the time of creating the mail is greatly different from that of the mobile phone normally used, and there is a problem that the operability is not good. In addition, since mail data created using the mail creation device exists only on the terminal device and does not exist on the mobile phone, it is created using the mail data created using the mail creation device and the mobile phone. Mail data is distributed and held. For this reason, there has been a problem that management of mail data becomes complicated.

  The present invention has been made in view of the above circumstances, and provides a non-contact input device, a mobile communication terminal, and a mail creation system capable of hands-free operations such as mail creation in a mobile communication terminal. It is aimed. In particular, it is an object of the present invention to provide a non-contact type input device with improved operability when a user creates a hands-free e-mail using a mobile communication terminal. It is another object of the present invention to provide a non-contact input device for a mobile communication terminal that improves convenience when managing created mail data.

A non-contact type input device for a portable communication terminal according to the first aspect of the present invention is a communication means for communicating with a portable communication terminal having two or more operation keys, and the portable communication terminal via the communication means. A terminal information acquisition unit that acquires operation key information including identification information of a communication carrier in order to specify the key arrangement of each operation key, and a key that holds two or more different key arrangements in association with the operation key information. Key arrangement information storage means for storing arrangement information and key arrangement extraction for referring to the key arrangement information based on the operation key information acquired by the terminal information acquisition means and extracting a key arrangement corresponding to the operation key information Means for displaying a key arrangement extracted by the key arrangement extracting means on the hands-free operation screen, and photographing a subject. Imaging means for generating an image; gaze position determination means for analyzing the moving image to detect eye movement of a subject; and determining a gaze position in the hands-free operation screen based on the detection result; and the gaze Based on the determination result by the position determination means and the key arrangement extracted by the key arrangement extraction means, the operation key corresponding to the gaze position is determined, operation input information corresponding to the operation key is generated, and the communication means is passed through the communication means. Operation input means for sending to the mobile communication terminal.

  In this non-contact type input device, key arrangement information for storing a plurality of different key arrangements in association with operation key information is stored, and the key arrangement is determined based on the operation key information acquired from the mobile communication terminal. Extracted. The extracted key arrangement is displayed on the hands-free operation screen, and the gaze position in the hands-free operation screen is determined based on the detection result of the eye movement. Then, based on the determination result and the key arrangement, an operation key corresponding to the gaze position is determined, and operation input information corresponding to the operation key is generated.

  According to such a configuration, the movement of the eyes is detected to determine the gaze position in the hands-free operation screen, and the operation key operation input information corresponding to the gaze position is generated. By gazing at the key arrangement on the screen, a desired operation key can be operated. Therefore, operations such as mail creation in the mobile communication terminal can be performed in a hands-free manner, and operability can be improved. In addition, since data such as mail data created in this manner exists on the mobile communication terminal, the convenience in data management can be improved.

  In addition to the above configuration, the non-contact type input device for a mobile communication terminal according to the second aspect of the present invention includes a gaze movement detection unit in which the gaze position determination unit detects a gaze movement due to eyeball rotation based on the moving image. A blink detection unit that detects blinks based on the moving image, determines a gaze position based on a detection result of eye movement, and the operation input unit is based on a detection result by the blink detection unit, Operation input information is generated as if the user operated an operation key corresponding to the gaze position when the user blinked.

  According to a third aspect of the present invention, there is provided a non-contact input device for a mobile communication terminal, wherein the mobile communication terminal has a terminal screen, and the terminal information acquisition means is connected to the mobile communication terminal via the communication means. The display information for the input device generated based on the display information of the terminal screen is acquired from the terminal, and the display means displays the key arrangement in the key arrangement display area on the hands-free operation screen, and the terminal The display information for the input device is displayed in the screen display area, and the gaze position determining means is configured to determine the gaze position in the key arrangement display area.

  According to a fourth aspect of the present invention, there is provided a non-contact type input device for a mobile communication terminal, in addition to the above configuration, a voice input unit that collects a user's voice and generates voice data, Voice analysis means for generating message data consisting of the following and sending it as a mail text to the portable communication terminal via the communication means. According to such a configuration, message data obtained by analyzing the user's voice is sent as an email text to the mobile communication terminal via the communication means, so that only character input can be performed using the movement of the eyes. The body of the mail can be entered by voice.

According to a fifth aspect of the present invention, there is provided a mobile communication terminal comprising: a communication means for communicating with a mobile communication terminal having two or more operation keys; and a key arrangement of the operation keys from the mobile communication terminal via the communication means. Terminal information acquisition means for acquiring operation key information consisting of identification information of a telecommunications carrier in order to identify the key arrangement, and key arrangement information for storing two or more different key arrangements in association with the operation key information, respectively Information storage means, key arrangement extraction means for referring to the key arrangement information based on the operation key information acquired by the terminal information acquisition means, and extracting a key arrangement corresponding to the operation key information, and a hands-free operation screen Display means for displaying the key arrangement extracted by the key arrangement extraction means on the hands-free operation screen, and imaging for shooting a subject and generating a moving image Means for detecting eye movement of the subject by analyzing the moving image and determining a gaze position in the hands-free operation screen based on the detection result, and discrimination by the gaze position determination unit Based on the result and the key arrangement extracted by the key arrangement extraction means, the operation key corresponding to the gaze position is determined, operation input information corresponding to the operation key is generated, and the portable communication terminal is connected via the communication means. A portable communication terminal having communication means for communicating with a non-contact input device having an operation input means for sending to the operation key , based on an acquisition request from the non-contact input device Operation key information sending means for sending information to the non-contact type input device is provided.

In addition to the above configuration, a mobile communication terminal according to a sixth aspect of the present invention generates a display for generating input device display information to be displayed on the hands-free operation screen based on the terminal screen and display information on the terminal screen. Information generation means, input device display information sending means for sending the input device display information to the non-contact type input device based on an acquisition request from the non-contact type input device, and the non-contact type input device Mail generating means for generating an e-mail based on the operation input information received from the voice, and voice analyzing means for analyzing the voice data received from the non-contact input device and generating message data consisting of character strings the mail generating means, said message data to generate electronic mail as the mail body, the display information generating means, an operation input received from the non-contact input device Based on the information and updates the display information for the input apparatus, the terminal information transmitting unit is configured to deliver the input device display the updated information via the communication means to the non-contact type input device . According to such a configuration, since voice data from the non-contact type input device is analyzed and message data is generated, and an e-mail is generated as the mail body, the eye movement is utilized by the non-contact type input device. In addition to inputting text, you can also enter the text of a mail by voice. In this portable communication terminal, the display information for the input device and the operation key information are transmitted based on the acquisition request from the non-contact type input device, and the input device is based on the operation input information from the non-contact type input device. Display information is updated, and the updated display information is sent to the non-contact input device. According to such a configuration, since operation input such as mail creation is performed based on operation input information from the non-contact type input device, the mobile communication terminal is operated hands-free using the non-contact type input device. be able to.

A mail creation system according to a seventh aspect of the present invention includes a mobile communication terminal having two or more operation keys , a display device having a hands-free operation screen, an imaging device that captures a subject and generates a moving image, and a main unit. A mail composing system comprising: a main body unit that communicates with the mobile communication terminal; and a key arrangement of the operation keys from the mobile communication terminal via the main body communication means. Key arrangement information that stores terminal information acquisition means for acquiring operation key information including identification information of a communication carrier for identification, and key arrangement information that holds two or more different key arrangements in association with the operation key information. The key arrangement information is referred to based on the operation key information acquired by the storage means and the terminal information acquisition means, and corresponds to the operation key information. Key arrangement extracting means for extracting key arrangement; display control means for controlling the display device; and displaying the key arrangement extracted by the key arrangement extracting means on the hands-free operation screen ; and analyzing the moving image. The eye movement of the subject is detected and the gaze position discriminating means for discriminating the gaze position in the hands-free operation screen based on the detection result, the discrimination result by the gaze position discrimination means and the key arrangement extracting means An operation input unit that determines an operation key corresponding to the gaze position based on the arranged key position, generates operation input information corresponding to the operation key, and sends the operation input information to the mobile communication terminal via the main body communication unit. provided, the mobile communication terminal, a terminal communication means for communicating with the main unit, based on the acquisition request from the main unit Configured to include a terminal information sending means for sending the operation key information to the main unit, based on the operation input information received from the main unit, and a mail generating unit which generates an e-mail.

  According to the non-contact input device for a mobile communication terminal, the mobile communication terminal, and the mail creation system according to the present invention, an eye movement is detected to determine a gaze position in the hands-free operation screen, and an operation corresponding to the gaze position Since key operation input information is generated, the user can operate a desired operation key by paying attention to the key arrangement on the hands-free operation screen. Therefore, operations such as mail creation in the mobile communication terminal can be performed in a hands-free manner, and operability can be improved. In addition, since data such as mail data created in this manner exists on the mobile communication terminal, the convenience in data management can be improved.

Embodiment 1 FIG.
FIG. 1 is a system diagram showing a configuration example of a system including a non-contact input device according to Embodiment 1 of the present invention, and shows a non-contact input device for operating a mobile communication terminal 10 in a hands-free manner. An email creation system 100 is shown as an example of a system that includes the email. The mail creation system 100 includes a mobile communication terminal 10 and a non-contact input device connected to the mobile communication terminal 10, and is a system that allows hands-free operations such as mail creation in the mobile communication terminal 10. .

  The portable communication terminal 10 is a portable communication device such as a mobile phone or a PHS (Personal Handyphone System), and a thin rectangular housing has an LCD (Liquid Crystal Display) 11 having a terminal screen. Is provided. The terminal screen is a display screen on which mail data and the like are displayed. As such a mobile communication terminal 10, one that is normally used by a user can be used.

  The non-contact type input device includes a main unit 20, a communication cable 21, a display device 22, an imaging device 23, and a microphone 24. The main unit 20 is a control unit for controlling the mobile communication terminal 10, the display device 22, the imaging device 23, and the microphone 24.

  The communication cable 21 is a transmission line for transmitting a data signal, and the data signal is transmitted and received between the mobile communication terminal 10 and the main unit 20 via the communication cable 21. The communication cable 21 is, for example, a serial communication cable, and communication data is serially transmitted. The mobile communication terminal 10 is usually provided with a connector for connecting to an external device such as a headphone, a display device, or a personal computer, and an audio signal or a video signal is output through the connector, or communication data is transmitted. Sent and received. The communication cable 21 is detachably connected to such a connector.

  The display device 22 is a display device having a hands-free operation screen 22a, and screen display is performed based on an image signal from the main unit 20. The hands-free operation screen 22a is a display screen that is used when the mobile communication terminal 10 is operated hands-free using eye movements. The same screen as the mobile communication terminal 10 is displayed, and the mobile communication terminal Ten key layouts are displayed.

  The hands-free operation screen 22a has, for example, a horizontally long rectangular shape. This hands-free operation screen 22a is easy to see even when the distance between the user and the display device 22 is longer than the distance between the user and the mobile communication terminal 10 when operating the mobile communication terminal 10. The screen size is larger than the terminal screen 11.

  When the non-contact type input device of the mail creation system 100 is installed in an automobile, the display device 22 is installed with a screen upright on a dashboard, for example. In this case, a head-up display that displays information on a transparent plate-like member such as a windshield is suitable as the display device 22 from the viewpoint of not obstructing the view of the user during driving.

  The imaging device 23 is a camera that captures a subject, generates a moving image, and outputs the moving image to the main unit 20, and is disposed around the display device 22. In this example, the imaging device 23 is attached to the upper surface of the housing of the display device 22.

  The microphone 24 is a sound collecting device that collects a user's voice, generates a sound signal, and outputs the sound signal to the main unit 20. For example, the microphone 24 is disposed around the user.

  FIG. 2 is a block diagram showing a configuration example of the main unit 20 in the mail creation system 100 of FIG. The main unit 20 includes a terminal I / F 1a, a display I / F 1b, a camera I / F 1c, a microphone I / F 1d, a terminal information acquisition unit 2, a key arrangement extraction unit 3, a display control unit 4, and a gaze position determination. A unit 5, an operation input unit 6, and an audio data generation unit 7.

  The terminal I / F (interface) 1 a is a communication unit for transmitting and receiving data signals to and from the mobile communication terminal 10, and the mobile communication terminal 10 is connected via the communication cable 21.

  The terminal information acquisition unit 2 sends an acquisition request as necessary, and performs an operation of acquiring input device display information and operation key information as terminal information from the mobile communication terminal 10 via the terminal I / F 1a. . The input device display information acquired by the terminal information acquisition unit 2 is display information for the main unit corresponding to the display information on the terminal screen, and is generated in the mobile communication terminal 10.

  The display information for input device includes, for example, image data necessary for performing the same screen display on the display device 22 as the mobile communication terminal 10 connected to the main unit 20. Specifically, image data composed of color components RGB for screen display in the mobile communication terminal 10 is acquired as display information for input device.

  The operation key information is information necessary for displaying the key arrangement of the mobile communication terminal 10 connected to the main unit 20 on the hands-free operation screen 22 a of the display device 22. For example, information that can identify the communication carrier to which the user of the mobile communication terminal 10 is subscribed, the manufacturer of the mobile communication terminal 10, or the model of the mobile communication terminal is acquired as the operation key information. Specifically, the communication carrier name, manufacturer name, and model name are acquired as operation key information.

  In general, a basic rule is set by a communication carrier regarding which function key is to be arranged at which position of a mobile communication terminal, and there is little difference in key arrangement among manufacturers. In particular, the layout of the operation keys that are normally used and the function assignments for the operation keys are strictly standardized for each telecommunications carrier, and the manufacturers and models differ between telecommunications terminals belonging to the same telecommunications carrier. There are only design differences. Therefore, by acquiring at least the identification information of the communication carrier as the operation key information, it is possible to determine the operation key arrangement and function assignment, and display the key arrangement of the mobile communication terminal 10 on the hands-free operation screen 22a. be able to.

  The terminal information acquisition unit 2 performs an operation of electrically detecting that the mobile communication terminal 10 and the main unit 20 are connected to each other and automatically sending an acquisition request to the mobile communication terminal 10.

  The key arrangement information storage unit 3a is a memory that stores key arrangement information that holds a plurality of different key arrangements in association with operation key information. The key arrangement extraction unit 3 refers to the key arrangement information based on the operation key information acquired by the terminal information acquisition unit 2 and performs an operation of extracting a key arrangement corresponding to the operation key information.

  For example, key type information that associates the carrier name, manufacturer name, and model name with the assignment type information of the function assignment for the operation key; and key layout information that associates the position information of the operation key with the function assignment for each assignment type information; Are prepared in advance as key arrangement information. When the operation key information is acquired by the terminal information acquisition unit 2, the key type information is referred to based on the operation key information, and the assignment type of the function assignment is determined. Then, the key layout information is extracted by referring to the key layout information based on the determination result and determining the position and function assignment of the operation keys.

  The display I / F 1 b is a communication unit for transmitting and receiving data signals to and from the display device 22. The display control unit 4 performs an operation of controlling the display device 22 via the display I / F 1b. Specifically, the key arrangement extracted by the key arrangement extraction unit 3 is displayed in the key arrangement display area on the hands-free operation screen 22a, and the input device display information acquired by the terminal information acquisition unit 2 is displayed on the terminal. An operation of displaying in the screen display area is performed.

  The camera I / F 1 c is a communication unit for transmitting and receiving data signals to and from the imaging device 23. The gaze position determination unit 5 analyzes the moving image obtained from the imaging device 23 via the camera I / F 1c to detect the eye movement of the subject, and based on the detection result, the hands-free operation screen 22a An operation to determine the gaze position is performed.

  The gaze position determination unit 5 includes a line-of-sight movement detection unit 5a that detects line-of-sight movement due to eyeball rotation based on a moving image, and a blink detection unit 5b that detects blinks due to eyelid movement based on a moving image. The position detection unit 5c determines the gaze position based on the detection result of the eye movement.

  In the eye movement detection unit 5a and the blink detection unit 5b, for example, the user's face is determined based on the contour extraction of the still image for each frame, and the eye movement and blink due to the rotation of the eyeball are detected based on the motion analysis between the frames. Is done. The position detection unit 5c determines the gaze position in the key arrangement display area on the hands-free operation screen 22a.

  The operation input unit 6 determines an operation key corresponding to the gaze position based on the determination result by the gaze position determination unit 5 and the key arrangement extracted by the key arrangement extraction unit 3, and generates operation input information corresponding to the operation key. The operation to be performed. Specifically, based on the detection result by the blink detection unit 5b, the operation input information is generated assuming that the user operates the operation key corresponding to the gaze position when the user blinks. The generated operation input information is sent to the mobile communication terminal 10 via the terminal I / F 1a.

  The microphone I / F 1 d is receiving means for receiving an audio signal from the microphone 24. The voice data generation unit 7 performs an operation of generating voice data by digitizing a voice signal obtained by collecting a user's voice and sending the voice data to the mobile communication terminal 10 via the terminal I / F 1a. . This voice data is generated, for example, for e-mail attachment data.

  FIG. 3 is a plan view showing an example of the appearance of the mobile communication terminal 10 in the mail creation system 100 of FIG. In addition to the LCD 11, the mobile communication terminal 10 is provided with an operation key 12, an antenna 13 for cellular communication, a receiver 14 for reception, and a microphone 15 for transmission.

  The LCD 11 is a display device having a terminal screen having a vertically long rectangular shape, and displays a mail creation screen and the like.

  The operation key 12 is composed of, for example, a push-type switch, and a predetermined operation input is performed by pressing it with a finger. Specifically, the operation keys 12 include a direction key 16a, a CSK key 16b, a function key 17 such as an LSK key, an RSK key, an off-hook key and an on-hook key, and character type change keys 18a and 18c. A clear key 18b, a numeric key 18d, and a view key 18e are provided.

  The direction key 16a is a multi-function key that allows different operation inputs depending on the up / down / left / right pressed positions, and has an annular shape. By operating the direction key 16a, it is possible to scroll the screen display, move the cursor, or change the focusing target.

  The CSK key 16b is a determination key arranged at the center of the direction key 16a. The LSK key is a function key arranged on the left side of the direction key 16a, and is assigned a function for displaying a mail creation screen. The RSK key is a function key arranged on the right side of the direction key 16a, and is assigned a browser function for connecting to a website. The view key 18e is an operation key to which a predetermined function is assigned. For example, a function for browsing telephone book data and mail data is assigned by an operation during a call.

  The receiver 14 is an audio output device for reproducing the received audio signal, and is disposed on the opposite side of the operation key 12 with respect to the LCD 11. The microphone 15 is a voice input device that collects a user's voice and generates a voice signal. The microphone 15 is disposed at the end of the operation key 12 opposite to the LCD 11.

  FIG. 4 is a block diagram showing an example of a schematic configuration of the mobile communication terminal 10 in the mail creation system 100 of FIG. The mobile communication terminal 10 includes a cellular communication unit 31, a main control unit 32, a speaker 33, and an external device I / F 34 in addition to the LCD 11, the operation key 12, the antenna 13, the receiver 14, and the microphone 15. Each unit is controlled by the unit 32.

  The cellular communication unit 31 is a mobile communication unit that performs wireless communication with a base station via the antenna 13. Calls and electronic mails are exchanged with other communication terminals via the cellular communication unit 31. The speaker 33 is an audio output device for reproducing a ringtone.

  The external device I / F 34 is a communication unit for communicating with an external device such as a headphone, a display device, or a personal computer. The communication cable 21 of the main unit 20 is connected to the external device I / F 34.

  FIG. 5 is a block diagram illustrating a configuration example of a main part of the mobile communication terminal 10 of FIG. 4, in which an example of a functional configuration in the main control unit 32 is illustrated. The main control unit 32 includes an operation key information storage unit 41, a terminal information transmission unit 42, an input device display information storage unit 43, a display information generation unit 44, a mail generation unit 45, and a mail data storage unit 46.

  The operation key information storage unit 41 is a non-volatile memory for storing operation key information, and the name of the communication carrier to which the user of the mobile communication terminal 10 subscribes, the name of the manufacturer and model of the mobile communication terminal 10 are the operation keys. It is retained as information.

  The display information generation unit 44 performs an operation of generating input device display information for screen display on the display device 22 based on display information on the terminal screen. The input device display information storage unit 43 is a memory for holding the input device display information generated by the display information generation unit 44 in a rewritable manner. For example, image data for screen display at the time of mail creation is held as input device display information.

  The terminal information sending unit 42 includes an operation key information sending unit 42 a that sends operation key information to the main unit 20, and a display information sending unit 42 b that sends input device display information to the main unit 20.

  The operation key information sending unit 42 a performs an operation of reading the operation key information from the operation key information storage unit 41 based on the acquisition request from the main unit 20 and transmitting it to the main unit 20 via the external device I / F 34. . The display information sending unit 42b performs an operation of reading the display information from the input device display information storage unit 43 based on an acquisition request from the main unit 20 and transmitting it to the main unit 20 via the external device I / F 34. .

  The display information generation unit 44 performs a process of updating the display information in the input device display information storage unit 43 based on the operation input information received from the main unit 20 via the external device I / F 34. The display information sending unit 42b performs an operation of transmitting the updated display information for the input device to the main unit 20 via the external device I / F 34.

  The mail generation unit 45 performs an operation of generating an e-mail based on the operation input information received from the main unit 20 via the external device I / F 34. The e-mail generated by the mail generation unit 45 is stored in the mail data storage unit 46. When the audio data generated for the attached data by the main unit 20 is received, an e-mail with the audio data as the attached data is created.

  As the audio data to be attached to the e-mail, for example, the audio data input by the microphone 24 is encoded as a file such as AMR (Advanced Multi Rate codec) or AAC (Advanced Audio Coding) in the mail generation unit 45. The data converted into the voice compression data can be considered.

  FIG. 6 is a diagram showing an example of the operation in the mail creation system 100 of FIG. 1, and shows an example of the screen display on the hands-free operation screen 22 a of the display device 22. In the hands-free operation screen 22a, a terminal screen display area 51 and a key arrangement display area 52 are provided. In this example, the display area 51 is arranged on the left side, and the display area 52 is arranged on the right side.

  The display area 51 is a display area in which screen display is performed based on input device display information acquired from the mobile communication terminal 10. In the display area 51, the same screen display as the terminal screen of the mobile communication terminal 10 connected to the main unit 20 is performed.

  The display area 52 is a display area in which key arrangement is displayed based on operation key information acquired from the mobile communication terminal 10. In this display area 52, function assignments are displayed as key layouts in the same positional relationship as the operation keys on the mobile communication terminal 10 connected to the main unit 20.

  As such a key arrangement, specifically, a direction key is arranged at the center of the upper stage, and an LSK key and an off-hook key are arranged on the left side of the direction key. An RSK key and an on-hook key are arranged on the right side of the direction key. A CSK key is arranged at the center of the direction key.

  Further, a character type change key, a clear key, a numeric keypad, and a view key are arranged below the direction keys. In this example, the mail function is assigned to the LSK key, and the browser function is assigned to the RSK key.

  FIG. 7 is a diagram showing an example of key layout information held by the key arrangement information storage unit 3a of FIG. 2. Even if the function assignment assignment types for the operation keys 12 are different, the positional relationship between the operation keys 12 is different. The case where it does not change is shown. The key layout information includes a key number, function assignment, X coordinate, and Y coordinate. The key number is identification information for managing information related to the operation keys 12 in the main unit 20, and a serial number for each operation key 12 is assigned.

  The function assignment is information indicating the function assigned to the operation key, and is held for each assignment type. In this example, function allocation is held for each allocation type from “key type 1” to “key type N”.

  The X and Y coordinates are two-dimensional position information for specifying the relative positional relationship of the operation keys 12.

  Specifically, for example, a mail function is assigned as the key type 1 and a function for starting a predetermined application is assigned as the key type 2 to the operation key with the key number “01”. The X coordinate of this operation key is X01, and the Y coordinate is Y01. Further, a browser function is assigned as the key type 1 and a browsing function for telephone book data is assigned as the key type 2 to the operation key with the key number “02”. The X coordinate of this operation key is X02, and the Y coordinate is Y02.

  Further, the off-hook function is assigned as the key type 1 and the mail function is assigned as the key type 2 to the operation key having the key number “03”. The X coordinate of this operation key is X03, and the Y coordinate is Y03. Further, an on-hook function is assigned as the key type 1 and a browser function is assigned as the key type 2 to the operation key with the key number “04”. The X coordinate of this operation key is X04, and the Y coordinate is Y04.

  Further, the view function is assigned as the key type 1 to the operation key with the key number “25”, and the key type 2 is invalid. The X coordinate of this operation key is X25, and the Y coordinate is Y25.

  In the operation input unit 6, for example, the above-described key number is notified to the mobile communication terminal 10 as operation input information based on the gaze position determination result. That is, the key number corresponding to the operation key selected on the hands-free operation screen 22a by the eye movement is notified. In the mobile communication terminal 10, the same key event as that when the operation key is directly operated by hand is executed based on the key number notified from the main unit 20. Therefore, the user can operate the mobile communication terminal 10 without touching it directly by moving the line of sight while viewing the key arrangement on the hands-free operation screen 22a.

  Here, an example has been described in which the positional relationship of each operation key 12 does not change even if the function assignment assignment type for the operation key 12 is different. However, if the assignment type is different, the positional relationship of each operation key 12 may change. In such a case, key layout information may be held for each allocation type. Also, if the manufacturer or model of the mobile communication terminal 10 is different, the function assignment information and position information are held for each carrier, manufacturer, and model when the function assignments and positional relationships with the operation keys 12 change. You may let them.

  FIG. 8 is a diagram showing an example of the operation in the mail creation system 100 of FIG. 1, and shows another example of the screen display on the hands-free operation screen 22 a of the display device 22. In the hands-free operation screen 22a in FIG. 6, the key arrangement corresponding to the key type 1 is displayed in the display area 52, whereas in the hands-free operation screen 22a in FIG. The key layout to be displayed is displayed.

  That is, in the display area 52 of the hands-free operation screen 22a, a direction key is arranged in the upper center, and an LSK key and a mail key are arranged on the left side of the direction key. On the right side of the direction key, an RSK key and a browser key are arranged. A CSK key is arranged at the center of the direction key.

  Further, an off-hook key, a clear key, an on-hook key, and a numeric keypad are disposed below the direction key. In this example, a function for starting a predetermined application is assigned to the LSK key, and a function for browsing telephone directory data is assigned to the RSK key.

  As described above, in the mail creation system 100, the assignment type of the function assignment is determined based on the operation key information acquired from the mobile communication terminal 10, and the key arrangement of the mobile communication terminal 10 connected to the main unit 20 is determined as the display device. 22 is displayed appropriately.

  FIGS. 9A and 9B are diagrams showing an example of the operation in the mail creation system 100 of FIG. 1. FIG. 9A shows a photographed image 53 by the imaging device 23, and FIG. (B) shows the movement of the gaze positions B1 and B2 corresponding to the rotation of the eyeball A3.

  The moving image photographed by the imaging device 23 is analyzed by the gaze position determination unit 5, and the movement of the eye A2 is detected to determine the gaze position in the hands-free operation screen 22a. When detecting the movement of the eye A2 from the moving image, for example, the contour of the user's face A1 is extracted from the captured image by contour extraction, and the eye movement due to the rotation of the eyeball A3 and the blink due to the movement of the eyelid A4 are determined. The

  The user moves the line of sight to a desired operation key displayed in the display area 52 while looking at the key arrangement on the hands-free operation screen 22a, and blinks (closes 瞼 A4) to thereby operate the operation key. Can be operated. For example, by moving the line of sight from the gaze position B1 on the clear key to the gaze position B2 on the LSK key and blinking, such movement of the eye A2 is detected based on the moving image and corresponds to the gaze position B2. It is determined that the operation key, that is, the LSK key is operated.

  In this way, the operation input unit 6 generates operation input information on the assumption that the operation key corresponding to the gaze position when blinking is operated. In other words, blinking works in the same way as a mouse click operation.

  Here, if the relative positional relationship between the user and the imaging device 23 is different, it is considered that the line-of-sight direction with respect to the hands-free operation screen 22a is changed. Therefore, prior to the operation input by the line-of-sight movement as described above, the eye A2 It is assumed that alignment for determining the gaze position from the movement of the camera is performed. This alignment is performed, for example, based on the position of the eye A2 when a predetermined object is displayed on the hands-free operation screen 22a and the user is looking at the object.

  FIGS. 10A and 10B are diagrams showing an example of the operation in the mail creation system 100 of FIG. 1, and an example of the terminal screen displayed in the display area 51 of the hands-free operation screen 22a is shown. ing. FIG. 10A shows a mail creation screen 61, and FIG. 10B shows a destination selection screen 71.

  The mail creation screen 61 is the same screen display that is displayed on the mobile communication terminal 10 when the mail key is operated. The mail creation screen 61 includes a destination display field 62 for displaying a destination, a subject display field 63 for displaying a subject, a text display field 64 for displaying a mail text, and guide keys 65a to 65c.

  In the destination display field 62, the mail address of the destination of the e-mail is displayed. The guide keys 65a to 65c correspond to an LSK key, a CSK key, and an RSK key, respectively.

  The mail creation screen 61 is focused on any of the destination display column 62, the subject display column 63, and the text display column 64, and the focusing target can be changed by moving the line of sight. For example, by selecting a direction key and blinking, the focusing target is changed and the destination display column 62 is selected. Then, the terminal screen can be changed to the destination selection screen 71 by selecting the CSK key and blinking.

  Further, for example, a function for switching to a menu selection screen is assigned to the guide key 65a, and the menu selection screen can be displayed by selecting the LSK key and blinking. Further, for example, a mail transmission function is assigned to the guide key 65c, and an electronic mail being displayed on the mail creation screen 61 can be transmitted by selecting the RSK key and blinking.

  The destination selection screen 71 is a screen display for selecting a destination input method. In this example, an address book search button 72, a transmission / reception history button 73, and an address direct input button 74 are arranged.

  The address book search button 72 is an icon for designating a destination mail address by selecting a desired mail address from mail addresses registered in advance as telephone book data.

  The transmission / reception history button 73 is an icon for designating a mail address as a transmission destination by selecting a desired mail address from mail addresses held as a transmission history and a reception history. The address direct input button 74 is an icon for designating a mail address as a transmission destination by inputting characters.

  11 (a) and 11 (b) are diagrams showing an example of the operation in the mail creation system 100 of FIG. 1. FIG. 11 (a) shows an address list screen 81, and FIG. 11 (b). Shows an address input screen 83. The address list screen 81 is a screen display when the focus target is changed to either the address book search button 72 or the transmission / reception history button 73 on the destination selection screen 71 and the CSK key is operated by line-of-sight input.

  On this address list screen 81, a list of mail addresses 82 stored as telephone directory data or transmission / reception history is displayed. By changing the focusing object from these e-mail addresses 82 to a desired e-mail address and blinking with the line of sight to the CSK key, the e-mail address can be designated as a destination. After the e-mail address is determined in this way, the original e-mail creation screen 61 is restored.

  The address input screen 83 is a screen display when the focus target is changed to the address direct input button 74 on the destination selection screen 71 and the CSK key is operated by line-of-sight input.

  On this address input screen 83, a character type display field 84a, a character number display field 84b, and an input character display area 85 are arranged.

  In the character type display field 84a, the type of the character being input, for example, kanji, katakana, and numbers are displayed. In the character number display field 84b, the number of characters already input and the total number of characters that can be input are displayed. In the display area 85, input characters are displayed.

  The user can input a desired character by operating the operation key on the hands-free operation screen 22a by moving the line of sight. After the input of the mail address, the original mail creation screen 61 is restored.

  In the mail creation screen 61, the direct address input button 74 is selected when the subject is displayed by selecting the subject display field 63 and when the mail text is input by selecting the text display field 64. Characters can be entered in the same manner as when an email address is entered on the address entry screen 83.

  12 (a) and 12 (b) are diagrams showing an example of the operation in the mail creation system 100 of FIG. 1. FIG. 12 (a) shows a menu selection screen 91, and FIG. Shows an attached data selection screen 93. The menu selection screen 91 is a screen display when the operation key is operated by moving the line of sight to the LSK key corresponding to the guide key 65a on the mail creation screen 61 and blinking.

  The menu selection screen 91 displays a list of mail functions 92 that can be selected for creation, browsing, and transmission / reception of electronic mail. By changing the focusing target from these mail functions 92 to a desired mail function, moving the line of sight to the CSK key and blinking, the mail function can be executed.

  The attached data selection screen 93 is a screen display when a function for attaching attached data to an electronic mail is selected from the mail functions 92 on the menu selection screen 91.

  The attached data selection screen 93 displays a list of identification information 94 of data that can be selected as attached data. By changing the focusing target from the identification information 94 to the desired identification information and moving the line of sight to the CSK key to blink, the data associated with the identification information is designated as attached data.

  Steps S101 to S111 in FIG. 13 are flowcharts showing an example of the operation of the non-contact input device in the mail creation system 100 in FIG. First, when detecting that the mobile communication terminal 10 is connected, the terminal information acquisition unit 2 transmits a terminal information acquisition request to the mobile communication terminal 10 (step S101). The terminal information acquisition part 2 repeats transmission of an acquisition request until the terminal information from the mobile communication terminal 10 is normally received (step S102).

  Next, when the terminal information is received, the key arrangement extraction unit 3 determines the assignment type of the function assignment, that is, the key type based on the terminal information (step S103). At this time, if the key type information and the key layout information corresponding to the mobile communication terminal 10 that provided the terminal information are not registered as registration data, it is determined that determination of the key type is difficult, and this process ends. (Step S104).

  On the other hand, if the key type information and key layout information corresponding to the mobile communication terminal 10 are registered and the key type is normally determined, the key layout is specified based on the determination result, and the key arrangement is determined by the display device 22. It is displayed above (step S105).

  Next, the gaze position determination unit 5 analyzes the moving image obtained from the imaging device 23 to detect eye movement, and determines the gaze position on the hands-free operation screen 22a (steps S106 and S107). The processing procedures of steps S106 and S107 are repeated until a click operation by blinking is performed, and when the click operation by blinking is performed, operation input information is generated by determining the gaze position at the time of blinking, and the mobile communication terminal 10 (Steps S108 and S109).

  The screen display on the display device 22 is automatically updated when the mobile communication terminal 10 updates the input device display information based on the operation input information (step S110). The processing procedure from step S106 to step S110 is repeated until the operation is completed (step S111).

  According to the present embodiment, the eye movement is detected, the gaze position in the hands-free operation screen 22a is determined, and the operation input information of the operation key corresponding to the gaze position is generated. By gazing at the key layout on the operation screen 22a, a desired operation key can be operated. Therefore, since operations such as mail creation in the mobile communication terminal 10 can be performed hands-free, operability can be improved. Further, since data such as mail data created in this manner exists on the mobile communication terminal 10, the convenience in data management can be improved.

Embodiment 2. FIG.
In the first embodiment, an example in which characters such as a mail text are input by operating the operation keys on the hands-free operation screen 22a by moving the line of sight has been described. On the other hand, in this embodiment, a case will be described in which character input such as a mail text is performed by analyzing voice data.

  FIG. 14 is a block diagram showing a configuration example of the main unit 120 in the non-contact input device according to the second embodiment of the present invention. The main unit 120 differs from the main unit 20 of FIG. 2 in that it includes a voice analysis unit 121. The voice analysis unit 121 performs an operation of analyzing the voice data from the voice data generation unit 7 to generate message data including a character string and sending the message data to the mobile communication terminal 10 via the terminal I / F 1a. ing.

  For example, by analyzing the frequency of voice data, a character string spoken by the user is recognized and message data is created. That is, the user can input the character string by voice by speaking the character string to be input when inputting the mail text or the like.

  According to such a configuration, the message data obtained by analyzing the user's voice is sent as a mail text to the mobile communication terminal 10 via the terminal I / F 1a. Not only can you enter it, you can also enter the email text by voice.

Embodiment 3 FIG.
In the second embodiment, an example in which the main body unit 120 analyzes voice data to create a mail text has been described. On the other hand, in the present embodiment, a case where voice data is analyzed in the mobile communication terminal 10 will be described.

  FIG. 15 is a block diagram illustrating a configuration example of a main part of the mobile communication terminal according to the third embodiment of the present invention, in which an example of a functional configuration in the main control unit 132 is illustrated. This main control unit 132 is different from the main control unit 32 of FIG. 5 in that it includes a voice analysis unit 133. The voice analysis unit 133 performs an operation of analyzing voice data from the main unit 20 and generating message data including character strings.

  For example, by analyzing the frequency of voice data, a character string spoken by the user is recognized and message data is created. That is, the user can input the character string by voice by speaking the character string to be input when inputting the mail text or the like.

  The mail generation unit 45 performs processing for generating an electronic mail using the message data from the voice analysis unit 133 as a mail text.

  According to such a configuration, the voice data from the main unit 20 is analyzed and message data is generated, and an e-mail is generated as the mail body. Therefore, the main unit 20 uses the eye movement to input characters. Not only can you do it, you can also enter the body of your email by voice.

  Here, the voice data input by the microphone 24 is converted into message data and inserted into the mail body, or as described in the first embodiment, the voice data is converted into voice compression data and attached to the email. It is assumed that the user can select arbitrarily.

  FIG. 16 is a diagram showing the suitability of input and voice input by line-of-sight movement on the hands-free operation screen 22a for each operation target. Since the input by moving the line of sight on the hands-free operation screen 22a is accompanied by a click operation by blinking, it is suitable for a simple operation input such as changing a focusing target. On the other hand, voice input is more suitable for character input with a large number of operations than input by eye movement.

  For example, in the address input at the time of mail creation, when the input method is selection from a list of mail addresses held as phone book data or transmission / reception history, input by line of sight movement is preferable.

  On the other hand, when the input method is direct input of a mail address, voice input is preferable. Also, voice input is preferable when inputting a subject and a mail text.

  Further, when selecting attached data to be attached to an e-mail or selecting a mail function on a menu selection screen, input by line of sight movement is preferable.

  In the first to third embodiments, an example in which the same screen display as that of the mobile communication terminal 10 connected to the main unit 20 is performed on the display device 22 has been described, but the present invention is not limited thereto. is not. For example, a part of the display information for input device may be different from the display information on the terminal screen. Specifically, the screen display displayed on the hands-free operation screen 22a of the display device 22 as the terminal screen of the mobile communication terminal 10 is different from the actual screen display on the mobile communication terminal 10 in terms of the icon shape, The arrangement may be partially different.

  In the first to third embodiments, the example in which the operation input information is generated by determining the gaze position in the key arrangement display area has been described, but the present invention is not limited to this. For example, the gaze position in the terminal screen display area may be determined. By determining the gaze position in the terminal screen display area and generating operation input information, the cursor on the terminal screen can be moved by moving the line of sight, or the position of the cursor can be designated by blinking.

  Moreover, although Embodiment 1-3 demonstrated the example in case terminal information is transmitted via the communication cable 21, this invention is not limited to this. For example, an acquisition request or terminal information may be wirelessly transmitted between the mobile communication terminal 10 and the main unit 20. In this case, near field communication using radio waves and communication using infrared rays can be considered.

  In Embodiments 1 to 3, an example in which the imaging device 23 is mounted on the display device 22 has been described. However, the present invention is not limited to this. For example, the imaging device 23 may be attached to the user's head or the like, and the relative positional relationship between the imaging device 23 and the user's face may be fixed. In this way, since the position of the subject in the captured image is fixed, the process for extracting the user's face can be simplified.

1 is a system diagram showing a configuration example of a system including a contactless input device according to Embodiment 1 of the present invention, and shows a mail creation system 100. FIG. FIG. 2 is a block diagram showing a configuration example of a main unit 20 in the mail creation system 100 of FIG. 1. FIG. 2 is a plan view showing an example of the appearance of a mobile communication terminal 10 in the mail creation system 100 of FIG. 1. It is the block diagram which showed an example of schematic structure of the mobile communication terminal 10 in the mail preparation system 100 of FIG. FIG. 5 is a block diagram illustrating a configuration example in a main part of the mobile communication terminal 10 in FIG. 4, in which an example of a functional configuration in the main control unit 32 is illustrated. It is the figure which showed an example of the operation | movement in the mail preparation system 100 of FIG. 1, and an example of the screen display in the hands-free operation screen 22a is shown. It is the figure which showed an example of the key layout information hold | maintained by the key arrangement | positioning information storage part 3a of FIG. It is the figure which showed an example of the operation | movement in the mail preparation system 100 of FIG. 1, and another example of the screen display in the hands-free operation screen 22a is shown. It is the figure which showed an example of the operation | movement in the mail preparation system 100 of FIG. It is the figure which showed an example of the operation | movement in the mail preparation system 100 of FIG. 1, and an example of the terminal screen displayed in the display area 51 of a terminal screen is shown. FIG. 2 is a diagram illustrating an example of an operation in the mail creation system 100 of FIG. 1, in which an address list screen 81 and an address input screen 83 are illustrated. It is a figure showing an example of operation in mail creation system 100 of Drawing 1, and menu selection screen 91 and attached data selection screen 93 are shown. 3 is a flowchart showing an example of the operation of the non-contact input device in the mail creating system 100 of FIG. It is the block diagram which showed the structural example of the main body unit 120 in the non-contact-type input device by Embodiment 2 of this invention. It is the block diagram which showed the structural example in the principal part of the portable communication terminal by Embodiment 3 of this invention, and an example of the function structure in the main-control part 132 is shown. It is the figure which showed the suitability of the input by the eyes | visual_axis movement on the hands-free operation screen 22a, and the voice input for every operation target.

Explanation of symbols

1a Terminal I / F
1b Display I / F
1c Camera I / F
1d Microphone I / F
2 terminal information acquisition unit 3 key arrangement extraction unit 4 display control unit 5 gaze position determination unit 5a gaze movement detection unit 5b blink detection unit 5c position detection unit 6 operation input unit 7 audio data generation unit 10 portable communication terminal 11 LCD
12 Operation Key 13 Antenna 14 Receiver 15 Microphone 20 Main Unit 21 Communication Cable 22 Display Device 22a Hands-Free Operation Screen 23 Imaging Device 24 Microphone 31 Cellular Communication Unit 32 Main Control Unit 33 Speaker 34 External Device I / F
41 operation key information storage unit 42 terminal information transmission unit 42a operation key information transmission unit 42b display information transmission unit 43 display information storage unit for input device 44 display information generation unit 45 mail generation unit 46 mail data storage unit 100 mail creation system A1 face A2 Eye A3 Eyeball A4 Acupuncture

Claims (7)

Communication means for communicating with a mobile communication terminal having two or more operation keys;
Terminal information acquisition means for acquiring operation key information consisting of identification information of a communication carrier in order to specify the key arrangement of each operation key from the portable communication terminal via the communication means;
Key arrangement information storage means for storing key arrangement information for holding two or more different key arrangements in association with operation key information,
Key arrangement extraction means for referring to the key arrangement information based on the operation key information acquired by the terminal information acquisition means and extracting a key arrangement corresponding to the operation key information;
A display unit having a hands-free operation screen and displaying the key arrangement extracted by the key arrangement extraction unit on the hands-free operation screen;
Imaging means for capturing a subject and generating a moving image;
Gaze position determining means for analyzing the moving image to detect the movement of the subject's eyes and determining the gaze position in the hands-free operation screen based on the detection result;
Based on the determination result by the gaze position determination unit and the key arrangement extracted by the key arrangement extraction unit, the operation key corresponding to the gaze position is determined, operation input information corresponding to the operation key is generated, and the communication unit A non-contact type input device for a mobile communication terminal, comprising operation input means for sending to the mobile communication terminal via The gaze position determination unit includes a line-of-sight movement detection unit that detects line-of-sight movement due to rotation of an eyeball based on the moving image, and a blink detection unit that detects blinking based on the moving image. Determine the gaze position based on the result,
The operation input means generates the operation input information based on a detection result by the blink detection means, assuming that an operation key corresponding to a gaze position when the user blinks is operated. The non-contact type input device for portable communication terminals as described. The mobile communication terminal has a terminal screen,
The terminal information acquisition means acquires input device display information generated based on display information of the terminal screen from the mobile communication terminal via the communication means,
The display means displays the key arrangement in the key arrangement display area on the hands-free operation screen, and displays the input device display information in the terminal screen display area.
The non-contact type input device for a mobile communication terminal according to claim 1, wherein the gaze position determining means determines a gaze position in the key arrangement display area. Voice input means for collecting voice of the user and generating voice data;
2. The voice analysis unit according to claim 1, further comprising: a voice analysis unit that analyzes the voice data to generate message data including a character string and sends the message data to the portable communication terminal as the mail text via the communication unit. Contactless input device for portable communication terminals. Communication means for communicating with a mobile communication terminal having two or more operation keys, and identification information of a communication carrier for specifying the key arrangement of each operation key from the mobile communication terminal via the communication means Terminal information acquisition means for acquiring operation key information comprising: key arrangement information storage means for storing key arrangement information for holding two or more different key arrangements in association with operation key information; and the terminal information acquisition means Reference is made to the key arrangement information based on the obtained operation key information, and a key arrangement extraction unit that extracts a key arrangement corresponding to the operation key information and a hands-free operation screen are extracted by the key arrangement extraction unit. Display means for displaying the arranged key arrangement on the hands-free operation screen, imaging means for photographing a subject and generating a moving image, and analyzing the moving image and subject Gaze position discriminating means for detecting eye movement and discriminating the gaze position in the hands-free operation screen based on the detection result; the discrimination result by the gaze position discrimination means; and the key extracted by the key arrangement extracting means Non-contact type having an operation input unit that determines an operation key corresponding to the gaze position based on the arrangement, generates operation input information corresponding to the operation key, and sends the operation input information to the mobile communication terminal via the communication unit A mobile communication terminal having a communication means for communicating with an input device,
Based on the acquisition request from the non-contact input device, a mobile communication terminal that the operation key information, comprising the operation key information sending means for sending to the non-contact input device. A terminal screen,
Display information generating means for generating display information for an input device for displaying on the hands-free operation screen based on display information of the terminal screen;
Input device display information sending means for sending the input device display information to the non-contact type input device based on an acquisition request from the non-contact type input device;
Based on the operation input information received from the non-contact type input device, mail generating means for generating an email,
Analyzing voice data received from the non-contact type input device, and comprising voice analysis means for generating message data consisting of character strings,
The mail generation means generates an email using the message data as a mail body ,
The display information generating means updates the input device display information based on the operation input information received from the non-contact type input device,
6. The portable communication terminal according to claim 5, wherein the terminal information sending unit sends the updated display information for the input device to the non-contact type input device via the communication unit. A mail creation system comprising a mobile communication terminal having two or more operation keys , a display device having a hands-free operation screen, an imaging device for photographing a subject to generate a moving image, and a main unit,
The main unit is above a body communication means for communicating with the mobile communication terminal, the identification of the operators in order to identify the key arrangement of the operation keys from the mobile communication terminal via the body communication unit Terminal information acquisition means for acquiring operation key information comprising information, key arrangement information storage means for storing key arrangement information for holding two or more different key arrangements in association with operation key information, and the terminal information acquisition means The key arrangement extracting means for extracting the key arrangement corresponding to the operation key information, referring to the key arrangement information based on the operation key information acquired by the above, and controlling the display device, and extracting by the key arrangement extraction means by a display control means for displaying on said hands-free operation screen key arrangement was to detect the movement of the eye of the subject by analyzing the moving picture, this A gaze position determining means for determining a gaze position in the hands-free operation screen on the basis of the detection result, the fixation position on the basis of the determination result and key arrangement extracted by the key arrangement extracting means by the gaze position determining means An operation input unit that determines the corresponding operation key, generates operation input information corresponding to the operation key, and sends the operation input information to the mobile communication terminal via the main body communication unit;
The portable communication terminal, a terminal communication means for communicating with the main unit, based on the acquisition request from the main unit, a terminal information sending means for sending the operation key information to the main unit, the based on the operation input information received from the main unit, mail preparation system characterized in that a mail generating unit which generates an e-mail.

Images