JP5865597B2 - Portable electronic devices - Google Patents

Description

  The present invention relates to a portable electronic device including a touch panel.

  In recent years, as an input device that enables intuitive operation and does not include a device that requires a physically large area such as a keyboard, an operation for an image displayed on the display unit and an image displayed on the display unit There is an apparatus that uses a touch panel configured with a touch sensor that detects the above. In an electronic device including a touch panel as an input device, it is proposed to correct a positional relationship between detection by a touch sensor and an image displayed on a display unit in order to improve input accuracy (Patent Document 1, Patent). Reference 2).

JP 2001-67186 A JP 2004-280590 A

  However, although the correction methods described in Patent Documents 1 and 2 perform correction based on learning based on information input by the user, learning is performed based on the input even when the user inputs erroneously. There is a limit to the accuracy of correction. In addition, in order to perform correction using the learning function, it is necessary to accumulate the input history, and a storage area and a processing area are used for this process, and the storage function and calculation function are used. Will be burdened. Further, in some cases, the deviation from the user's recognition cannot be corrected by correcting the deviation between the display position and the contact position.

  The present invention has been made in view of the above, and an object thereof is to provide a portable electronic device that can detect an operation input by a user with high accuracy.

  In order to solve the above-described problems and achieve the object, the present invention relates to an electronic device, a housing, a display unit disposed in the housing, and a touch disposed to overlap the display unit. A touch panel including a sensor, a relative position detection unit that detects a relative position between the user and the touch panel, and a relative position between the user and the touch panel detected by the relative position detection unit. And a control unit that adjusts a correspondence relationship between an image to be displayed on the display unit and a detection area of the touch sensor corresponding to the image.

  Further, the relative position detection means has a camera that is arranged in the housing and shoots an image of an area facing the touch panel, and extracts the position of the user's face included in the image taken by the camera, It is preferable to detect a relative position between the user and the touch panel based on the position of the face.

  Further, it is preferable that the relative position detection unit extracts a position of a user's eyes as the user's face and detects a relative position between the user and the touch panel based on the position of the eyes.

  In addition, the relative position detection unit includes an attitude detection sensor that detects an orientation of the casing, and based on the orientation of the casing detected by the attitude detection sensor and a preset relationship with the user. It is preferable to detect a relative position with respect to the touch panel.

  In addition, when the control unit is in a state where an image is displayed from a state where no image is displayed on the display unit of the touch panel, the image displayed on the display unit of the touch panel and the touch corresponding to the image are displayed. It is preferable to adjust the correspondence relationship with the detection area of the sensor.

  The image processing apparatus further includes an acceleration sensor that detects acceleration of the housing, and the control unit detects an image that is displayed on the display unit of the touch panel when detecting an acceleration equal to or greater than a value set by the acceleration sensor, and the image It is preferable to adjust the correspondence relationship with the detection area of the touch sensor corresponding to.

  In addition, the control unit displays on the display unit of the touch panel in the same direction as the direction in which the user is inclined with respect to the front surface of the touch panel with respect to a position where the user is the front surface of the touch panel. It is preferable to shift the detection area of the touch sensor corresponding to the image to be moved.

  Moreover, it is preferable that the said control part detects the frequency of the operation mistake input by the said user, and further adjusts the deviation | shift amount of an operation target and a detection area based on the said frequency of the operation mistake.

  It is preferable that the control unit changes a deviation amount between the operation target and the detection area according to a type of application that is activated and displays the operation target on the display unit.

  It is preferable that the control unit further adjusts the correspondence relationship based on a correction operation input by the user.

  The portable electronic device according to the present invention has an effect that an operation input by a user can be detected with high accuracy.

FIG. 1 is a front view showing an appearance of a mobile phone terminal. FIG. 2 is a block diagram showing a schematic configuration of functions of the mobile phone terminal. FIG. 3 is a perspective view showing an example of the relationship between the mobile phone terminal and the user. FIG. 4 is a schematic diagram illustrating an example of a relative position between the mobile phone terminal and the user. FIG. 5 is a schematic diagram showing an example of an image taken by the mobile phone terminal. FIG. 6 is an explanatory diagram illustrating an example of a relationship between an image displayed on the display unit and a detection area of the touch sensor. FIG. 7 is a schematic diagram illustrating an example of a relative position between the mobile phone terminal and the user. FIG. 8 is a schematic diagram illustrating an example of an image captured by a mobile phone terminal. FIG. 9 is an explanatory diagram illustrating an example of a relationship between an image displayed on the display unit and a detection area of the touch sensor. FIG. 10 is a flowchart showing an example of the adjustment operation processing of the mobile phone terminal. FIG. 11 is a flowchart showing an example of the adjustment operation processing of the mobile phone terminal. FIG. 12 is a schematic diagram illustrating an example of an image captured by a mobile phone terminal. FIG. 13 is an explanatory diagram illustrating an example of a relationship between an image displayed on the display unit and a detection area of the touch sensor. FIG. 14 is a schematic diagram illustrating an example of an image captured by a mobile phone terminal. FIG. 15 is an explanatory diagram illustrating an example of a relationship between an image displayed on the display unit and a detection area of the touch sensor. FIG. 16 is a schematic diagram illustrating an example of an image captured by a mobile phone terminal. FIG. 17 is an explanatory diagram illustrating an example of a relationship between an image displayed on the display unit and a detection area of the touch sensor. FIG. 18 is an explanatory diagram illustrating an example of a relationship between an image displayed on the display unit and a detection area of the touch sensor.

  Hereinafter, the present invention will be described in detail with reference to the drawings. The present invention is not limited to the following description. In addition, constituent elements in the following description include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those in a so-called equivalent range. In the following, a mobile phone terminal will be described as an example of an electronic device. However, the application target of the present invention is not limited to a mobile phone terminal, and various devices including a touch panel, for example, a PHS (Personal Handy-phone System) The present invention can also be applied to PDAs, portable navigation devices, personal computers, game machines, and the like.

  FIG. 1 is a front view showing an appearance of a mobile phone terminal 1 that is an embodiment of the mobile electronic device of the present invention. The mobile phone terminal 1 has a thin plate-like casing 12. In the mobile phone terminal 1, a touch panel 2, an input unit 3 including a button 20, a button 22, and an input device 24, a receiver 7, a microphone 8, and a camera 16 are disposed on the surface of a housing 12. . The touch panel 2 is arranged on the surface having the widest area of the plate-like housing 12. The input unit 3 is also disposed at one end in the longitudinal direction of the surface of the housing 12 on which the touch panel 2 is disposed. The input unit 3 is arranged in the order of the button 20, the input device 24, and the button 22 from one side to the other side in the short direction. The receiver 7 is disposed at the other end in the longitudinal direction of the surface of the housing 12 where the touch panel 2 is disposed, that is, at the end opposite to the end where the input unit 3 is disposed. Yes. The microphone 8 is disposed at one end in the longitudinal direction of the surface of the housing 12 where the touch panel 2 is disposed, that is, the end where the input unit 3 is disposed. The camera 16 is disposed on the surface on which the touch panel 2 is disposed. More specifically, the camera 16 is disposed in a state where a photographing window for photographing an image is exposed at an end portion where the receiver 7 is disposed on the surface where the touch panel 2 is disposed.

  The touch panel 2 displays characters, figures, images, and the like, and is performed on the touch panel 2 using a finger, a stylus, a pen (a tip of a pen, a tip of a rod-like member) or the like (hereinafter simply referred to as “finger”). Detect various actions. For example, the mobile phone terminal 1 displays a virtual keyboard on the touch panel 2 in order to accept input of characters from the user. The mobile phone terminal 1 detects various operations input to the touch panel 2 with a finger in a state where the virtual keyboard is displayed on the touch panel 2, detects which key of the virtual keyboard is pressed or touched, Character input can be performed by using a key that has been detected as pressed or touched as an input key. Moreover, the touch panel 2 detects the input of various operations based on the displayed image and various operations performed on the touch panel 2 with a finger regardless of the input of characters, and based on the input operations. Various controls.

  When the buttons 20 and 22 are pressed, the input unit 3 activates a function corresponding to the pressed button. The input unit 3 also detects an operation input to the input device 24 as an operation, and performs various controls based on the input operation. For example, the input device 24 detects a direction instruction operation or a determination operation. The input device 24 includes a touch pad, an optical input device, an input device having buttons on the four sides and the center, and the like.

  The camera 16 is a photographing device that photographs an image. The photographing window is disposed so as to be exposed on the surface of the housing 12 on which the touch panel 2 is disposed. The camera 16 obtains an image of a predetermined angle region (photographing region) among the surfaces facing the touch panel 2 by reading light of the photographing region incident from the photographing window.

  Next, the relationship between the function of the mobile phone terminal 1 and the control unit will be described. FIG. 2 is a block diagram showing a schematic configuration of functions of the mobile phone terminal 1 shown in FIG. As shown in FIG. 2, the mobile phone terminal 1 includes a touch panel 2, an input unit 3, a power supply unit 5, a communication unit 6, a receiver 7, a microphone 8, a storage unit 9, a main control unit 10, A RAM (Random Access Memory) 11, a camera 16, and an acceleration sensor 18 are included. In the present embodiment, the combination of the photographing function of the camera 16 and the corresponding function of the control unit that processes an image photographed by the camera is the relative position detecting means.

  The touch panel 2 includes a display unit 2B and a touch sensor 2A superimposed on the display unit 2B. The touch sensor 2A detects various operations performed on the touch panel 2 using a finger together with the position on the touch panel 2 where the operation is performed. The operations detected by the touch sensor 2 </ b> A include an operation of bringing a finger into contact with the surface of the touch panel 2, an operation of moving a finger while keeping the surface in contact with the surface of the touch panel 2, and an operation of moving the finger away from the surface of the touch panel 2. It is. The touch sensor 2A may employ any detection method such as a pressure-sensitive method or an electrostatic method. The display unit 2B includes, for example, a liquid crystal display (LCD), an organic EL (Organic Electro-Luminescence) panel, and the like, and displays characters, figures, images, and the like.

  The input unit 3 includes the buttons 20 and 22 and the input device 24 as described above. The buttons 20 and 22 receive a user operation through physical input (pressing), and transmit a signal corresponding to the received operation to the main control unit 10. In addition, the input device 24 also accepts a user operation and transmits a signal corresponding to the accepted operation to the main control unit 10.

  The power supply unit 5 supplies power obtained from a storage battery or an external power supply to each functional unit of the mobile phone terminal 1 including the main control unit 10. The communication unit 6 establishes a radio signal line by a CDMA system or the like with a base station via a channel assigned by the base station, and performs telephone communication and information communication with the base station. The receiver 7 outputs the other party's voice, ringtone, and the like in telephone communication. The microphone 8 converts the voice of the user or the like into an electrical signal.

  The storage unit 9 is, for example, a nonvolatile memory or a magnetic storage device, and stores programs and data used for processing in the main control unit 10. Specifically, the storage unit 9 includes a mail program 9A for sending / receiving and browsing mail, a browser program 9B for browsing a WEB page, a camera program 9C for capturing an image with the camera 16, a camera 16 includes a face recognition program 9D for extracting a face included in the image photographed in 16, an operation detection position adjustment program 9E for adjusting the correspondence between the image displayed on the display unit 2B and the detection area of the touch sensor 2A, and various programs. A processing condition table 9F that associates various conditions used for execution is stored. The storage unit 9 also stores other programs and data such as an operating system program that realizes basic functions of the mobile phone terminal 1 and address book data in which names, telephone numbers, mail addresses, etc. are registered. The The storage unit 9 also stores a program for determining a control operation and processing based on an input operation input to the touch panel 2. The control operations and processes include various operations and processes executed by the mobile phone terminal 1. For example, a cursor, a pointer movement, a screen display switching, a character input process, various application activation processes, and an end process are included. is there.

  The main control unit 10 is, for example, a CPU (Central Processing Unit), and comprehensively controls the operation of the mobile phone terminal 1. Specifically, the main control unit 10 executes the program stored in the storage unit 9 while referring to the data stored in the storage unit 9 as necessary, so that the touch panel 2, the input unit 3, Various processes are executed by controlling the communication unit 6, the camera 16, the acceleration sensor 18, and the like. The main control unit 10 expands the program stored in the storage unit 9 and the data acquired / generated / processed by executing the process to the RAM 11 that provides a temporary storage area as necessary. The program executed by the main control unit 10 and the data to be referred to may be downloaded from the server device by wireless communication by the communication unit 6.

  As described above, the camera 16 is an imaging mechanism in which the shooting window for shooting an image is disposed so as to be exposed from the housing 12 and acquires an image of the shooting area.

  The acceleration sensor 18 is a detector that detects acceleration applied to the housing 12. Here, as the acceleration sensor 18, a detector that detects acceleration by various methods can be used. For example, detection that detects acceleration by a change in capacitance, a change in piezoresistance, a change in relative position, or the like. Can be used. The acceleration sensor 18 detects the direction of acceleration in addition to the magnitude of acceleration. The acceleration sensor 18 detects acceleration acting on the housing 12 when the operator moves or shakes the housing 12, and sends the detected acceleration information to the main control unit 10. The main control unit 10 executes the operation detection position adjustment program 9E, analyzes the detection result of the acceleration sensor 18, and specifically integrates the detected acceleration information to change the posture of the housing 12. The movement of the mobile phone terminal 1 (housing 12) can be detected by calculation.

  Next, an example of a method for adjusting the operation detection position will be described with reference to FIGS. FIG. 3 is a perspective view showing an example of the relationship between the mobile phone terminal and the user. FIG. 4 is a schematic diagram showing an example of the relative position between the mobile phone terminal and the user, FIG. 5 is a schematic diagram showing an example of an image taken by the mobile phone terminal, and FIG. It is explanatory drawing which shows an example of the relationship between the image to display, and the detection area of a touch sensor. FIG. 7 is a schematic diagram showing an example of the relative position between the mobile phone terminal and the user, FIG. 8 is a schematic diagram showing an example of an image taken by the mobile phone terminal, and FIG. It is explanatory drawing which shows an example of the relationship between the image to display, and the detection area of a touch sensor.

  The operation detection position adjustment processing shown in FIGS. 3 to 9 is repeatedly executed based on the functions provided by the camera program 9C, the face recognition program 9D, and the operation detection position adjustment program 9E. More specifically, the cellular phone terminal 1 uses various detection results and images acquired by a function (image shooting function) provided by the camera program 9C and a function (face extraction function) provided by the face recognition program 9D as operation detection positions. Analysis is performed based on the function provided by the adjustment program 9E, and the correspondence between the image displayed on the display unit 2B and the detection area of the touch sensor 2A is adjusted. That is, the main control unit 10 of the mobile phone terminal 1 controls each unit to adjust the relative positional relationship between the position of the image displayed on the display unit 2B and the position of the detection area detected as an operation on the image.

  First, as illustrated in FIG. 3, when the user H operates the touch panel 2 of the mobile phone terminal 1, the user H basically operates in a state of facing the touch panel 2. When the mobile phone terminal 1 detects an operation on the touch panel 2 input by the user H, the mobile phone terminal 1 activates the camera 16 and acquires an image of the imaging region 40 facing the camera 16 as shown in FIG. Here, the camera 16 has a photographing window disposed on the same surface as the surface on which the touch panel 2 is disposed. Accordingly, the shooting area 40 is an area facing the touch panel 2.

  The image of the imaging region 40 that the mobile phone terminal 1 captures with the camera 16 at the relative position shown in FIG. 4 is an image including the face of the user H. It should be noted that the relative position between the user H and the mobile phone terminal 1 in FIG. The center 42 extends in a direction orthogonal to the surface of the touch panel 2. When the mobile phone terminal 1 takes an image with the camera 16 at the relative position shown in FIG. 4, an image 50 shown in FIG. 5 is acquired. The image 50 shows the user Ha in the center of the image 50. Further, when the image 50 is viewed in the opposite direction, the right eye Er of the user Ha is displayed on the left side of the screen, and the left eye El is displayed on the right side of the screen.

  When the mobile phone terminal 1 acquires the image 50 illustrated in FIG. 5, the mobile phone terminal 1 analyzes the acquired image 50 and extracts the user Ha included in the image 50. After extracting the user Ha, the mobile phone terminal 1 identifies the position of the user Ha in the image 50, in this embodiment, the position of the eyes of the user Ha. The mobile phone terminal 1 sets the correspondence between the image to be displayed on the display unit 2B and the detection area of the touch sensor 2A based on the identified eye position.

  When the user Ha is in the center of the image 50 as shown in FIGS. 4 and 5 and the center 42 and the eyes of the user H overlap each other, the mobile phone terminal 1 is displayed on the touch panel 2 as shown in FIG. The operation target 60 of the image and the detection area 62 that detects an operation on the operation target 60 are set at the same position. That is, in the mobile phone terminal 1, the position of the operation target 60 on the surface of the touch panel 2 and the position of the detection area 62 are the same position, and the center of the operation target 60 and the center of the detection area 62 overlap each other. The operation target 60 is an image that the user displays on the display unit 2B of the touch panel 2 with an operation as an input target, and is a button, an item, or a window.

  Next, the case where the relative position between the user H and the mobile phone terminal 1 has the relationship shown in FIG. 7 will be described. It should be noted that the relative position between the user H and the mobile phone terminal 1 in FIG. 7 is a line segment 44 that is rotated by a predetermined angle about the camera 16 and below the center 42 of the imaging region 40. There will be a position. When the mobile phone 1 captures an image with the camera 16 at the relative position shown in FIG. 7, an image 50a shown in FIG. 8 is acquired. The image 50a shows the user Ha in the area below the center. Further, when the image 50a is viewed in the opposite direction, the right eye Er of the user Ha is displayed on the left side of the screen, and the left eye El is displayed on the right side of the screen.

  When the mobile phone terminal 1 acquires the image 50a shown in FIG. 8, the mobile phone terminal 1 analyzes the acquired image 50a, extracts the user Ha included in the image 50a, and specifies the position of the eye of the user Ha in the image 50a. . The mobile phone terminal 1 sets the correspondence between the image to be displayed on the display unit 2B and the detection area of the touch sensor 2A based on the identified eye position.

  As shown in FIG. 9, when the user Ha is below the center of the image 50 and the eyes of the user H are below the center as shown in FIGS. Further, the detection area 62a is set to the lower position on the screen with respect to the operation target 60. That is, the mobile phone terminal 1 sets the detection area 62 at a position lower than the center of the operation target 60. That is, when the cellular phone terminal 1 shows the positional relationship between the operation target 60 and the detection area 62a as shown in FIG. 9 when the touch panel 2 is viewed from the front of the touch panel 2, a part of the upper side of the operation target 60 is The detection area 62a does not overlap, and a part of the lower side of the detection area 62a has a positional relationship in which the operation target 60 does not overlap.

  As described above, the mobile phone terminal 1 detects the relative position between the touch panel 2 and the user H based on the image captured by the camera 16, and detects the detection region 62 with respect to the operation target 60 based on the detected relative position. By shifting, it is possible to set the detection region 62 in accordance with the relative position between the touch panel 2 and the user H. Thereby, the input intended by the user can be detected with higher accuracy.

  In addition, since the mobile phone terminal 1 detects the relative position between the touch panel 2 and the user H using the camera 16, the detection area 62 for the operation target 60 is used regardless of the posture of the user H and the mobile phone terminal 1. Can be set. That is, by controlling according to the relative relationship between the user H (the face) and the touch panel 2, it is possible to control similarly whether the user is sleeping, sitting, or standing. .

  In addition, the mobile phone terminal 1 learns and stores a user's operation history and the like in order to adjust the position of the detection area 62 with respect to the operation target 60 based on the detected relative position between the touch panel 2 and the user H. In addition, the position of the detection region 62 relative to the position of the operation target 60 can be adjusted. Thereby, the cellular phone terminal 1 can appropriately adjust the position of the detection region 62 with respect to the position of the operation target 60 by a simple process.

  It is preferable that the mobile phone terminal 1 shifts the detection area 62 with respect to the operation target 60 so that the area connecting the user's eyes and the operation target 60 becomes the detection area 62. Accordingly, the user can touch the detection area 62 by touching the operation target 60 as seen with eyes. Thereby, the input intended by the user can be detected with higher accuracy.

  As shown in FIGS. 4 to 6, the mobile phone terminal 1 uses the front position of the touch panel 2 as a reference, and the case where the center 42 of the imaging region 40 and the eyes of the user H overlap with each other as a reference position. It is preferable to shift the detection region 62 with respect to the operation target 60 based on the amount of deviation from the operation target 60. By setting the reference position, the amount by which the detection area 62 is displaced with respect to the operation target 60 can be easily calculated.

  Next, the adjustment operation processing of the touch panel 2 by the main control unit 10 of the mobile phone terminal 1 will be described with reference to FIGS. 10 and 11. Here, FIG. 10 and FIG. 11 are flowcharts showing examples of the adjustment operation processing of the mobile phone terminal. 10 and 11 are repeatedly executed based on the functions provided by the camera program 9C, the face recognition program 9D, and the operation detection position adjustment program 9E as described above. More specifically, the cellular phone terminal 1 uses various detection results and images acquired by a function (image shooting function) provided by the camera program 9C and a function (face extraction function) provided by the face recognition program 9D as operation detection positions. Analysis is performed based on the function provided by the adjustment program 9E, and the correspondence between the image displayed on the display unit 2B and the detection area of the touch sensor 2A is adjusted. In addition, when the touch panel 2 is OFF, that is, when no operation is detected on the touch panel 2, the main control unit 10 repeatedly executes the process illustrated in FIG.

  The main control unit 10 determines whether the touch panel is turned on as step S12. If the main control unit 10 determines in step S12 that the touch panel 2 is not ON (No), that is, remains OFF, the main control unit 10 ends this processing. If the main control unit 10 determines that the touch panel 2 has been turned on (Yes) in step S12, the main control unit 10 performs a startup process of the camera 16 in step S14.

  When the camera 16 is activated in step S14, the main control unit 10 calculates the position of the face and eyes by face recognition in step S16. That is, the main control unit 10 captures an image with the camera 16, analyzes the captured image by processing the face recognition program 9D, and calculates the position of the user's face and eyes included in the image.

  After calculating the position of the face and eyes in step S16, the main control unit 10 calculates the angle in step S18 by comparison with the true front data. In other words, the main control unit 10 compares the position of the user's face and eyes acquired in step S16 with the position of the user's face and eyes of the straight front data serving as a reference as step S18, and step S16. The angle of the user's face and eye position detected in step 1 with respect to the reference is detected. The frontal data may be set by using a representative sample image or registering the user image in front of the setting operation and initial operation.

  When the main control unit 10 detects the angle in step S18, the main control unit 10 determines whether it is viewed from the front as step S20. That is, it is determined whether the angle calculated in step S18 is 0 degrees or an angle that can be approximated to 0 degrees. If the main control unit 10 determines that it is viewed from the front (Yes) in step S20, the process is terminated. That is, this process is terminated without performing the process of shifting the detection area 62 with respect to the operation target 60. In this case, the center of the detection region 62 is associated with the center of the operation target 60 in a state that matches.

  If the main control unit 10 determines that it is not viewed from the front in step S20 (No), that is, the angle calculated in step S18 is not an angle that can be approximated to 0 degrees, the main control unit 10 corrects the deviation based on the angle in step S22. That is, based on the angle calculated in step S18, processing for shifting the detection area 62 with respect to the operation target 60 is performed. In this case, the center of the detection area 62 is shifted from the center of the operation target 60 by a predetermined distance or a predetermined angle. After performing the process of step S22, the main control unit 10 ends this process.

  Next, adjustment processing in a state where the touch panel 2 is ON will be described with reference to FIG. When the touch panel 2 is turned on, that is, when an operation is detected on the touch panel 2, the main control unit 10 repeatedly executes the process shown in FIG.

  In step S32, the main control unit 10 determines whether a change in the attitude of the terminal (change in inclination) is detected by the acceleration sensor 18. If the main control unit 10 determines in step S32 that the terminal posture has not changed (No), the main control unit 10 ends this processing. That is, the main control unit 10 sets the detection area 62 for the operation target 60 on the touch panel 2 with the current settings. If the main control unit 10 determines in step S32 that the attitude of the terminal has changed (Yes), the main control unit 10 executes a process of resetting the deviation in step S33. That is, the relationship between the operation target 60 and the detection area 62 is returned to the initial state. After performing the process of step S33, the main control unit 10 performs the startup process of the camera 16 as step S34.

  When the camera 16 is activated in step S34, the main control unit 10 calculates the position of the face and eyes by face recognition in step S36. That is, the main control unit 10 captures an image with the camera 16, analyzes the captured image by processing the face recognition program 9D, and calculates the position of the user's face and eyes included in the image.

  After calculating the position of the face and eyes in step S36, the main control unit 10 calculates an angle by comparing with the frontal data in step S38. In other words, the main control unit 10 compares the position of the user's face and eyes acquired in step S36 with the position of the user's face and eyes of the straight front data serving as a reference as step S38, and step S36. The angle of the user's face and eye position detected in step 1 with respect to the reference is detected.

  When the main control unit 10 detects the angle in step S38, the main control unit 10 determines whether it is viewed from the front as step S40. That is, it is determined whether the angle calculated in step S38 is 0 degrees or an angle that can be approximated to 0 degrees. If the main control unit 10 determines that it is viewed from the front (Yes) in step S40, the process is terminated. That is, this process is terminated without performing the process of shifting the detection area 62 with respect to the operation target 60. In this case, the center of the detection region 62 is associated with the center of the operation target 60 in a state that matches.

  If the main control unit 10 determines that it is not viewed from the front in step S40 (No), that is, the angle calculated in step S38 is not an angle that can be approximated to 0 degrees, the main control unit 10 corrects the deviation based on the angle in step S42. That is, based on the angle calculated in step S38, a process for shifting the detection area 62 with respect to the operation target 60 is performed. In this case, the center of the detection area 62 is shifted from the center of the operation target 60 by a predetermined distance or a predetermined angle. After performing the process of step S42, the main control unit 10 ends this process.

  The mobile phone terminal 1 performs the processing of the flowcharts of FIGS. 10 and 11, so that the operation target is detected every time a change in the orientation of the mobile phone terminal 1 is detected when the touch panel 2 is activated or in use. The relative relationship between 60 and the detection region 62 is adjusted to an appropriate positional relationship. Thereby, the input intended by the user can be detected with higher accuracy.

  In the above embodiment, the timing for readjusting the relative position is detected based on the detection result of the acceleration sensor 18, but the present invention is not limited to this. For example, the mobile phone terminal 1 may readjust the relative position at regular time intervals. Further, the relative position may be readjusted when a specific operation is input to the user.

  In the example shown in FIGS. 4 to 9, the case where the mobile phone terminal 1 is used in the vertical direction has been described, but the present invention is not limited to this. Hereinafter, the case where the cellular phone terminal 1 is used in the horizontal direction will be described with reference to FIGS. Here, FIG. 12 is a schematic diagram illustrating an example of an image photographed by the mobile phone terminal, and FIG. 13 is an explanatory diagram illustrating an example of a relationship between an image displayed on the display unit and a detection area of the touch sensor. . FIG. 14 is a schematic diagram illustrating an example of an image captured by the mobile phone terminal, and FIG. 15 is an explanatory diagram illustrating an example of a relationship between an image displayed on the display unit and a detection area of the touch sensor.

  Even when the user H operates the touch panel 2 of the mobile phone terminal 1 sideways, the user H basically performs the operation in a state facing the touch panel 2. When the mobile phone terminal 1 detects an operation on the touch panel 2 input by the user H, the mobile phone terminal 1 activates the camera 16 and acquires an image.

  First, the case where the mobile phone terminal 1 acquires the image 50b shown in FIG. The image 50b shows the user Ha in the center of the image 50b. Further, when the image 50b is viewed in the opposite direction, the right eye Er of the user Ha is displayed on the left side of the screen, and the left eye El is displayed on the right side of the screen. When the mobile phone terminal 1 acquires the image 50b shown in FIG. 12, the mobile phone terminal 1 analyzes the acquired image 50b and extracts the user Ha included in the image 50b. After extracting the user Ha, the mobile phone terminal 1 specifies the position of the user Ha in the image 50b, in this embodiment, the position of the eye of the user Ha. The mobile phone terminal 1 sets the correspondence between the image to be displayed on the display unit 2B and the detection area of the touch sensor 2A based on the identified eye position.

  When the user Ha is in the center of the image 50b as shown in FIG. 12, the mobile phone terminal 1 controls the operation target 70 of the image displayed on the touch panel 2 and the operation on the operation target 70 as shown in FIG. The detection area 72 to be detected is set at the same position. That is, in the mobile phone terminal 1, the position of the operation target 70 on the surface of the touch panel 2 and the position of the detection area 72 are the same position, and the center of the operation target 70 and the center of the detection area 72 overlap each other.

  Next, the case where the mobile phone terminal 1 acquires the image 50c shown in FIG. In the image 50c, the user Ha is shown in a region below the center. Further, when the image 50c is viewed in the opposite direction, the right eye Er of the user Ha is displayed on the left side of the screen, and the left eye El is displayed on the right side of the screen. When the mobile phone terminal 1 acquires the image 50c shown in FIG. 14, the mobile phone terminal 1 analyzes the acquired image 50c, extracts the user Ha included in the image 50c, and specifies the position of the eye of the user Ha in the image 50c. . The mobile phone terminal 1 sets the correspondence between the image to be displayed on the display unit 2B and the detection area of the touch sensor 2A based on the identified eye position.

  When the user Ha is located below the center of the image 50c as shown in FIG. 14 and the eyes of the user Ha are located below the center as shown in FIG. The detection area 72a is set to the lower position of the screen with respect to the object 70. That is, the mobile phone terminal 1 sets the detection area 72 at a position lower than the center of the operation target 70. That is, when the cellular phone terminal 1 shows the positional relationship between the operation target 70 and the detection area 72a as shown in FIG. 15 when the touch panel 2 is viewed from the front of the touch panel 2, a part of the upper side of the operation target 70 is The detection area 72a does not overlap, and a part of the lower side of the detection area 72a has a positional relationship in which the operation target 70 does not overlap.

  Even when the mobile phone terminal 1 uses the touch panel 2 sideways as described above, the mobile phone terminal 1 detects the relative position between the touch panel 2 and the user H based on the image captured by the camera 16, and based on the detected relative position. By shifting the detection area 72 with respect to the operation target 70, the detection area 72 can be set in accordance with the relative position between the touch panel 2 and the user H. Thereby, the input intended by the user can be detected with higher accuracy.

  Moreover, although the portable electronic device 1 of the said embodiment demonstrated the case where the user H was moving to the up-down direction of the reference position with respect to the touch panel 2, it is not limited to this. The same applies to the portable electronic device 1 when the user H is moving in the horizontal direction of the reference position with respect to the touch panel 2. The left-right direction needs to be reversed with respect to the left-right direction on the captured screen and the left-right direction in which the detection area 72 is shifted with respect to the operation target 70.

  Next, the case where the mobile phone terminal 1 acquires the image 50d shown in FIG. In the image 50d, the user Ha is shown in an area on the left side of the center. Also, when the image 50b is viewed in the opposite direction, the right eye Er of the user Ha is displayed at the left end of the screen, and the left eye El is displayed at the right side of the screen rather than the right eye Er. When the mobile phone terminal 1 acquires the image 50d shown in FIG. 16, the mobile phone terminal 1 analyzes the acquired image 50d, extracts the user Ha included in the image 50d, and specifies the position of the eye of the user Ha in the image 50d. . The mobile phone terminal 1 sets the correspondence between the image to be displayed on the display unit 2B and the detection area of the touch sensor 2A based on the identified eye position.

  When the user Ha is on the left side of the center of the image 50d as shown in FIG. 16 and the eyes of the user Ha are on the left side of the center as shown in FIG. 16, as shown in FIG. In contrast, the detection area 62b is set to the position on the right side of the screen. That is, the mobile phone terminal 1 sets the center of the detection area 62 to the right side of the center of the operation target 60. That is, when the mobile phone terminal 1 shows the positional relationship between the operation target 60 and the detection region 62b as shown in FIG. 17 when the touch panel 2 is viewed from the front of the touch panel 2, a part of the left side of the operation target 60 is The detection area 62b does not overlap, and a part of the right side of the detection area 62b has a positional relationship where the operation target 60 does not overlap.

  As described above, the mobile phone terminal 1 detects the relative position between the touch panel 2 and the user H based on the image captured by the camera 16 even when the relative position is shifted in the left-right direction with respect to the reference position. By shifting the detection area 62 with respect to the operation target 60 based on the detected relative position, the detection area 62 can be set in accordance with the relative position between the touch panel 2 and the user H. Thereby, the input intended by the user can be detected with higher accuracy.

  Next, FIG. 18 is an explanatory diagram illustrating an example of a relationship between an image displayed on the display unit and a detection area of the touch sensor. In the above embodiment, the shift amount of the detection area 62 with respect to the operation target 60 is constant in the entire area of the touch panel 2, but the present invention is not limited to this. For example, as shown in FIG. 18, the shift amount of the detection region 62 with respect to the operation target 60 may be changed according to the position of the touch panel 2 in the vertical direction. Here, in the touch panel 2 shown in FIG. 18, a total of 12 operation objects 60 are arranged in a lattice pattern of 4 vertically and 3 horizontally. The relative position between the user and the touch panel 2 is a relationship in which the user is positioned below the center of the touch panel 2. The cellular phone terminal 1 reduces the shift amount of the detection area 82 corresponding to each of the three lowermost operation objects 60 in the up-down direction of the screen among the 12 target areas, so that the vertical direction of the screen The shift amount of the detection area 84 corresponding to each of the three operation objects 60 in the lower second stage is made larger than the shift amount in the detection area 82, and the three operations in the lower third stage in the vertical direction of the screen are performed. The detection area 88 corresponding to each of the three operation objects 60 in the uppermost stage in the vertical direction of the screen is made larger than the deviation amount of the detection area 84 corresponding to each of the objects 60. Is set to be larger than the shift amount of the detection area 86. That is, the touch panel 2 shown in FIG. 18 is set to increase the amount of deviation as it moves from bottom to top in the vertical direction of the screen.

  As described above, the cellular phone terminal 1 adjusts the shift amount at each position on the screen of the touch panel 2 in accordance with the shift of the relative position between the user and the touch panel 2, so that the input intended by the user is performed. It can be detected with higher accuracy. Specifically, the positional relationship between the operation target 60 of the touch panel 2 and the detection region as seen from the user by increasing the shift amount in the region far from the user and decreasing the shift amount in the region close to the user. It becomes. For this reason, the input intended by the user can be detected with higher accuracy.

  Further, the mobile phone terminal 1 uses the camera 16 as relative position detection means for detecting the relative position between the user and the touch panel 2, so that the relative position can be obtained with simple processing, an inexpensive apparatus configuration, and an existing apparatus configuration. Can be suitably detected. Note that the relative position detection means is not limited to the camera 16. If the state of the area corresponding to the touch panel 2 can be detected, the detection device is not particularly limited as long as the person in the area corresponding to the touch panel 2 can be detected. For example, using an infrared sensor, Time-of-Flight method (a method of calculating the distance to the object from the time and speed of light until the light emitted from the light source is reflected by the object and reaches the sensor) The detection apparatus etc. which were used can be used.

  The relative position detection means preferably uses a device that detects the state of the area corresponding to the touch panel 2, but is not limited thereto. The relative position detection means has a posture detection sensor for detecting the orientation of the housing, and detects the relative position between the user and the touch panel based on the orientation of the housing detected by the posture detection sensor and a preset relationship. It is good also as composition to do. Even with such a configuration, the relative position can be detected by a simple process although the accuracy is lowered.

  Further, the mobile phone terminal 1 may accumulate the operation history input by the user and correct the shift amount of the detection area corresponding to the operation target set based on the characteristics of the user input. That is, the shift amount of the detection area corresponding to the operation target may be set based on the relative position between the user and the touch panel 2, and the shift amount set based on the input characteristics of the user may be further corrected. The user input characteristics include, for example, characteristics such as how much the contact position is deviated from the center of the detection area and in which direction the contact position is liable to deviate from the center of the detection area. Thereby, the amount of deviation can be set in consideration of the user's habit and habit. In addition, by detecting the characteristics of the user's input divided into the conditions of the relative position between the user and the touch panel 2, it is possible to correct an appropriate shift amount adapted to the relative position between the user and the touch panel 2. it can.

  In addition, the mobile phone terminal 1 may detect the frequency of the operation mistake input by the user and further adjust the amount of deviation between the operation target and the detection area based on the frequency of the operation error. Note that the determination of a pressing error can be performed by various methods. For example, the determination may be made based on previous and subsequent operations. Specifically, when the input operation is canceled and a nearby operation target is operated, it is only necessary to determine that the operation is a pressing error. Further, a setting mode may be provided, an operation target may be specified in the setting mode, and it may be determined whether the pressing is wrong using a relative relationship with the actually input operation. In this case, it is possible to determine whether or not there is a pressing error based on the relationship between the operation target designated by the mobile phone terminal 1 and the operation target actually input.

  In addition, the mobile phone terminal 1 may change the amount of deviation between the operation target and the detection area in accordance with the type of application that is activated and displays the operation target on the display unit 2B of the touch panel 2. That is, even if the operation target is the same size displayed at the same position, the amount of displacement of the detection area with respect to the operation target may be different based on the type of application that displays the operation target. Thus, appropriate correction can be performed for each application to be used by changing the amount of deviation depending on the application. For each application to be activated, the mobile phone terminal 1 stores the relationship of the amount of deviation between the operation target and the detection area in the storage unit 9 and adjusts the amount of deviation based on the relationship stored in the storage unit 9. Good. Further, the relationship between the amount of deviation between the operation target and the detection area may be set according to the type of the operation target. For example, when the application has a browser function realized by processing the browser program 9A, the amount of detection area shift may be set for each of the operation target icons, links, and toolbars. Note that the mobile phone terminal 1 may store the relationship of the size of the detection area with respect to the operation area in addition to the shift amount. For example, the toolbar may be set to enlarge the detection area with respect to the operation area.

  In addition, the mobile phone terminal 1 may allow the user to manually correct the amount of deviation between the set operation target and the detection area. In this case, the user may perform correction processing as initial settings or processing conditions, or may set correction processing while an image is displayed.

DESCRIPTION OF SYMBOLS 1 Mobile phone terminal 2 Touch panel 2A Touch sensor 2B Display part 3 Input part 5 Power supply part 6 Communication part 7 Receiver 8 Microphone 9 Memory | storage part 9A Mail program 9B Browser program 9C Camera program 9D Face recognition program 9E Operation detection position adjustment program 9F Processing conditions Table 10 Main controller 11 RAM
12 Housing 16 Camera 18 Acceleration sensor 20, 22 Button 24 Input device

Claims (8)

A housing,
A display unit disposed in the housing;
A touch panel including a touch sensor arranged to be superimposed on the display unit;
A relative position detecting means for detecting a relative position between the user and each position on the touch panel ;
Based on the relative positions of the respective position on the said user detected by said relative position detecting means touch panel, the position of each image to be displayed on the display unit of the touch panel, farther from the user, each A control unit that greatly shifts each detection area of the touch sensor corresponding to an image;
A portable electronic device characterized by comprising: The relative position detecting means includes a camera that is disposed in the housing and captures an image of an area facing the touch panel.
The position of the user's face contained in the image image | photographed with the said camera is extracted, and the relative position of the said user and the said touch panel is detected based on the position of the said face. Portable electronic device.   The relative position detection unit extracts a position of a user's eyes as the user's face, and detects a relative position between the user and the touch panel based on the position of the eyes. 2. The portable electronic device according to 2. The relative position detection means has an attitude detection sensor for detecting the orientation of the housing,
The portable electronic device according to claim 1, wherein a relative position between the user and the touch panel is detected based on a direction of the casing detected by the posture detection sensor and a preset relationship.   When the control unit is in a state where an image is displayed from a state where no image is displayed on the display unit of the touch panel, the touch corresponding to the image is displayed on the image displayed on the display unit of the touch panel. The portable electronic device according to any one of claims 1 to 4, wherein a detection region of the sensor is shifted. An acceleration sensor for detecting the acceleration of the housing;
The control unit shifts a detection area of the touch sensor corresponding to the image with respect to an image to be displayed on the display unit of the touch panel when detecting an acceleration equal to or higher than a value set by the acceleration sensor. The portable electronic device according to any one of claims 1 to 5.   2. The control unit according to claim 1, further comprising: detecting a frequency of a pressing error of an operation input by the user, and further adjusting a deviation amount between the operation target and the detection area based on the frequency of the pressing error. The portable electronic device as described in any one of 1-6.   The said control part changes the deviation | shift amount of an operation target and a detection area according to the kind of application which starts and displays an operation target on the said display part, The any one of Claim 1 to 7 characterized by the above-mentioned. The portable electronic device as described in.

Images