JP6892142B2 - Method of operation - Google Patents

Description

The present invention relates to a small input terminal device and an operation input method used by being worn on a wrist or the like.

In recent years, mobile information terminals such as smartphones that perform processing such as searching for various information, sending / receiving e-mails, and managing schedules in addition to telephone calls have rapidly become widespread. However, these mobile information terminals are often stored in bags or pockets when moving, and it is necessary to take out the mobile information terminals from the bags or pockets one by one in order to check received mails. For this reason, smaller input terminal devices such as wristwatches connected to these mobile information terminals by wireless communication have been developed.

In a mobile information terminal such as a smartphone, information can be displayed and input with a finger using a touch panel, but in a small input terminal device, the touch panel is also small, which makes it difficult to perform detailed operation input by touching the finger. .. Therefore, a non-contact input method using the back of the user's hand as an input surface has been proposed.

For example, in the wristband type input device described in Patent Document 1, characters can be input by providing a light emitting unit and a light receiving unit in the main body and detecting the positions of the fingertips on the back of the hand or the palm in a non-contact manner. It is composed.

JP-A-2014-106765

In the conventional character input method using a touch panel or the like, when the user's finger is in contact with the panel surface, the character is drawn according to the movement of the finger, and when the user's finger is away from the panel surface, the character is drawn. It is possible to distinguish between the valid period and the invalid period of the input so that there is no such thing. As a result, when inputting characters composed of multiple "pictures" such as line segments and dots, the valid "picture" part and the invalid part that moves between the "pictures" and the "pictures". By distinguishing between, the desired characters can be entered correctly.

On the other hand, in the non-contact input method such as Patent Document 1, a plurality of grid points on the back of the hand through which the fingertip has passed are detected, and the input characters are recognized from the dot matrix information. However, regarding the passing position of the fingertip, the valid part for the "picture" constituting the character and the invalid part moving between the "picture" and the "picture" are not distinguished. That is, there is a problem that the input character is easily recognized by mistake by detecting an invalid part unrelated to the "picture" as a part of the character. In other words, the start and end points of character drawing are unclear. Even if this invalid part is predicted and excluded in advance, if the stroke order is different at the time of input, or if characters including dakuten or characters with a large number of strokes are input, the trajectory of the invalid part is not constant. It seems that erroneous input is inevitable.

An object of the present invention is an input terminal device that, when a finger position is detected in a non-contact manner and an input operation is performed, does not react to an invalid movement of the finger movement and correctly executes a process according to an intended input operation. Is to provide.

The present invention is an input terminal device that inputs a user's operation via a position input object, wherein the position detection unit that detects the position of the position input object operated by the user in a non-contact manner and the position detection unit. The operation process control unit includes a display unit that displays a cursor based on the detected position and an operation process control unit that executes a corresponding operation process based on the position detected by the position detection unit. It has a plurality of operation modes related to execution, and one of the operation modes includes a state in which no operation process other than the movement of the cursor according to the position of the position input object is executed, and the operation process control The unit switches the plurality of operation modes when the user performs a specific operation via the position input object.

Further, according to the present invention, in an operation input method for inputting a user's operation via a position input object, a position detection step for non-contactly detecting the position of the position input object operated by the user and the position detection step. The operation processing step includes a display step for displaying the cursor on the display unit based on the position detected in the above step and an operation processing step for executing the corresponding operation processing based on the position detected in the position detection step. It has a plurality of operation modes related to execution of processing, and one of the operation modes includes a state in which no operation processing other than the movement of the cursor according to the position of the position input object is executed. The processing step switches between the plurality of operation modes when the user performs a specific operation via the position input object.

According to the present invention, when a finger position is detected in a non-contact manner and an input operation is performed, an input terminal device that does not react to an invalid movement of the finger movement and correctly executes a process according to an intended input operation. Can be provided.

The block diagram which shows the structure of the input terminal apparatus (Example 1). The figure which shows an example of the software structure of an input terminal device. The figure which shows the appearance of an input terminal device. The figure which shows the detection principle of a finger operation mode. The figure which shows the display example of the display part in the handwriting character input processing. A flowchart showing operation switching of the entire input terminal device. The figure which shows the example of the operation mode and the cursor display corresponding to it. The figure which shows the example of the screen display in the handwriting character input processing. The figure which shows the time change of the operation mode and the drawing position in FIG. 7. A flowchart showing handwritten character input processing in the finger operation mode. The flowchart which shows the determination process S605 of the operation mode in FIG. The figure which shows the structure of the display part for the key character input processing (Example 2). The figure which shows the example of the key character input processing by a finger operation mode. A flowchart showing a key character input process in a finger operation mode. The flowchart which shows the determination process S1105 of the operation mode in FIG. The figure which shows the example of the operation mode and the cursor display used in the mouse operation input processing (Example 3). The figure which shows the example of the screen display in the mouse operation input processing. A flowchart showing mouse operation input processing in the finger operation mode. The flowchart which shows the determination process S1504 of the operation mode in FIG. The figure which shows the detection principle of the 3D finger operation mode (Example 4). The figure which shows the example of the screen display in a 3D finger operation mode. The flowchart which shows the mouse operation input processing by the 3D finger operation mode. The figure which shows the example of the screen display in the key character input processing. The figure which shows the example of the screen display when the transition area is provided between layers (Example 5). A flowchart showing mouse operation input processing when a transition area is provided between layers.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

In the first embodiment, the configuration of the input terminal device and the case where the finger position is detected and the handwritten character is input will be described.

FIG. 1A is a block diagram showing the configuration of the input terminal device 1 according to the first embodiment, and the internal configuration (hardware configuration) thereof will be described. The input terminal device 1 is used by connecting to a mobile information terminal 2 such as a smartphone. The input terminal device 1 of this embodiment has a finger operation mode (non-contact detection method) using light reflection and a touch operation mode (contact detection method) as operation input functions.

The main control unit 101 is composed of a CPU (Central Processing Unit) or the like, and controls the entire input terminal device 1 according to various operation programs and data stored in the ROM (Read Only Memory) 102 or the RAM (Random Access Memory) 103. Control. The system bus 100 is a data communication path for transmitting and receiving data between the control unit 101 and each unit in the input terminal device 1.

The ROM 102 is a memory in which various programs for controlling the input terminal device 1 are stored, and a rewritable ROM such as an EEPROM (Electrically Erasable Programmable ROM) or a flash ROM is used. The RAM 103 is used as a temporary storage area for executing a program stored in the ROM 102, or as a temporary storage of finger position information detected by the finger position detection sensor 106. The storage unit 104 stores information such as operation setting values of the input terminal device 1, and uses a non-volatile and rewritable device such as a flash ROM or SSD (Solid State Drive). The ROM 102 and the RAM 103 may be integrated with the main control unit 101. Further, the ROM 102 does not have an independent configuration, but may be configured to use a part of the storage area in the storage unit 104.

The acceleration sensor 105 is a sensor that detects the acceleration of the input terminal device 1. The finger position detection sensor 106 is a sensor that non-contactly detects the position of a position input object such as a user's finger or a pen by reflected light. The short-range wireless communication unit 107 connects to the mobile information terminal 2 by wireless communication and transmits the information input to the input terminal device 1. As the communication method, for example, Bluetooth (registered trademark) or WiFi (registered trademark) is used.

The display control unit 108 controls the display of input characters, various setting images of the input terminal device 1, and the like on the display unit 109a. The touch panel 109 includes a display unit 109a and a touch operation input unit 109b. The display unit 109a is a display device such as a liquid crystal panel, and displays characters, images, and the like under the control of the display control unit 108. The touch operation input unit 109b is arranged on the display surface of the display unit 109a.

The touch operation input unit 109b is a touch pad type input device such as a capacitance type, and detects a contact operation with a finger, a touch pen, or the like as an operation input. Specifically, for example, a gesture called swipe that moves a finger in a specific direction while touching the touch panel with a finger, or a finger after touching the touch panel with a finger and then moving the finger at a speed equal to or higher than a predetermined value. Detects gestures called flicks that release, and gestures called taps that release quickly after touching the touch panel with a finger. This enables operation input to the input terminal device 1 by a touch method.

The timekeeping unit 110 measures, for example, the elapsed time from the date and time set by the user using an RTC (Real Time Clock) circuit, and outputs the date and time information. The main control unit 101 performs processing such as a timer by reading the date / time / time information output from the timekeeping unit 110. The operation input unit 111 is an instruction input unit for inputting an operation instruction to the input terminal device 1, and has a button for turning on / off the power supply and setting an operation mode.

As described above, the input terminal device 1 of the present embodiment has a finger position detection sensor 106 that detects the position of the user's finger in a non-contact manner and inputs a character or the like, and a touch that operates and inputs by touching the finger to the touch panel 109. It is configured to include an operation input unit 109b and a display unit 109a for displaying characters, images, and the like. This enables two operation input functions, a finger operation mode and a touch operation mode.

FIG. 1B is a diagram showing an example of the software configuration of the input terminal device 1, and shows the software configuration in the ROM 102, the RAM 103, and the storage unit 104.

The main control unit 101 constitutes the basic operation processing control unit 1021, the finger position information acquisition unit 1022, and the touch position information acquisition unit 1023 by executing the program stored in the ROM 102. Further, the RAM 103 includes a temporary storage area 1031 that temporarily holds data as needed when executing a program stored in the ROM 102. The storage unit 104 stores the setting information 1041 for executing the program stored in the ROM 102. In the following description, for the sake of simplicity, it is assumed that the main control unit 101 controls each operation block by executing the program stored in the ROM 102.

The basic operation processing control unit 1021 controls various settings of the input terminal device 1 and overall operations. The finger position information acquisition unit 1022 acquires the position information of the position input object such as the user's finger or pen based on the output of the finger position detection sensor 106. The touch position information acquisition unit 1023 acquires the position information in which a finger, a touch pen, or the like touches the touch panel 109 based on the output of the touch operation input unit 109b.

The finger operation processing control unit 1024 determines what kind of operation has been input based on the position information of the position input target object acquired from the finger position information acquisition unit 1022, and controls the processing corresponding to the determination result. Do. The touch operation processing control unit 1024 determines what kind of operation has been input based on the information of the touch position of the touch panel 109 acquired from the touch position information acquisition unit 1023, and controls the processing corresponding to the determination result. Do.

FIG. 2 is a diagram showing the appearance of the input terminal device 1. It is a wristwatch-type input terminal device that is worn on the user's wrist for use. A touch panel 109 (display unit 109a) is arranged on the surface side of the device. Two sensors 106a and 106b are arranged as finger position detection sensors 106 on the back side of the user's hand on the side surface of the device. These sensors consist of, for example, light emitting / receiving elements. Further, an operation button is arranged as an operation input unit 111 on the arm side of the user on the side surface of the device.

FIG. 3 is a diagram showing the detection principle of the finger operation mode, and shows the relationship between the position of the finger and the display position of the display unit. The user's finger (position input object) PS moves, for example, within the positions ABCD on the back side of the user's hand. The finger position detection sensors 106a and 106b are composed of an infrared light emitting element and a light receiving element. Infrared light emitted from the light emitting element is reflected by the finger PS, and the reflected light is detected by the light receiving element. The level of the detection signal in the light receiving element corresponds to the distances La and Lb from the sensors 106a and 106b to the finger PS.

The output signal of the finger position detection sensor 106 is sent to the finger position information acquisition unit 1022 to acquire the position F of the finger PS in the position input detection range Pa represented by the positions ABCD. Further, the positions ABCD of the position input detection range Pa correspond to the positions abcd of the display range Da of the display unit 109a of the touch panel 109, respectively. From this, the display position f of the cursor or the like in the display range Da is determined corresponding to the position F of the finger PS in the position input detection range Pa.

FIG. 4 is a diagram showing a display example of the display unit 109a in the handwritten character input process. The area 501 displays the result of recognizing the input handwritten character, and the area 502 displays the handwritten character input by the finger. As the button 503 selected by the user for character input processing, the recognition processing instruction button 503a for selecting the process for recognizing handwritten characters, the character type switching processing instruction button 503b for switching the type of characters to be input such as hiragana and alphabet, and the input handwriting. Erase processing instruction button 503c to erase the display of characters, character conversion processing instruction button 503d to select the processing to convert the character string of the recognition result displayed in the area 501 to kanji, etc., end processing instruction to end the handwritten character input processing Buttons 503e and the like are arranged.

FIG. 5 is a flowchart showing operation switching of the entire input terminal device 1. Here, the power is turned on / off and the operation input mode (finger operation mode / touch operation mode) is switched by pressing a button on the operation input unit 111.

In S301, the output of the operation input unit 111 is taken in, and in S302, it is determined whether or not the operation input unit 111 is pressed. When the operation input unit 111 is pressed (Yes), it branches to S306. In S306, the time during which the operation input unit 111 is pressed is determined, and the process proceeds to the power on / off switching or the operation input mode switching process.

When the operation input unit 111 is pressed by Tp or more for a predetermined time in the determination process S306 (Yes), the process branches to S313 and the on / off switching process is performed depending on whether or not the power supply is on. If the power is on in S313 (Yes), it branches to S314, turns off the power, and returns to S301. When the power is turned off, in order to reduce power consumption, the supply of power to parts other than the main control unit 101, the timekeeping unit 110, the operation input unit 111, and the like is stopped. When the power supply is off in S313 (No), it branches to S315, turns on the power, and returns to S301. When the power is turned on, power is supplied to each part constituting the input terminal device 1.

When the operation input unit 111 is not pressed for Tp or more for a predetermined time in the determination process S306 (No), it branches to S307 and determines whether or not the power supply is on. If the power is off (No), the process returns to S301. When the power is on (Yes), it branches to S308.

In S308, the operation input mode switching process is performed. That is, when the current operation input mode is set to the finger operation mode (Yes), the touch operation mode is switched and set in S309, and the cursor is not displayed on the display unit 109a of the touch panel 109 in S310. Set to. If the current operation input mode is not set to the finger operation mode (No) (that is, if it is set to the touch operation mode), switch to the finger operation mode in S311 and set, and in S312, the touch panel 109. The display unit 109a of is set to display the cursor.

If the operation input unit 111 is not pressed in the determination process S302 (No), the process proceeds to S303 and processing is performed according to the set operation input mode. In the case of the finger operation mode (Yes), the process proceeds to S304, it is determined what kind of operation has been input based on the finger position information acquired from the finger position information acquisition unit 1022, and the process corresponding to the determination result is performed. Execute. If it is not in the finger operation mode (No), the process proceeds to S305 (that is, in the case of the touch operation mode), and what kind of operation is input based on the touched position information acquired from the touch position information acquisition unit 1023. Is determined, and the process corresponding to the determination result is executed. After executing the process in the finger operation execution process S304 or the touch operation execution process S305, the process returns to S301.

Hereinafter, the handwritten character input process in the finger operation mode will be specifically described. In this embodiment, by giving a plurality of operation states (operation modes) to the finger operation, it is possible to correctly and easily input characters by handwriting.

FIG. 6 is a diagram showing an example of an operation mode and a cursor display corresponding thereto. The operation mode M0 is a state in which drawing for character input other than cursor movement is not performed (drawing stopped) even if a finger is operated. The operation mode M1 is a state in which drawing for character input is executed according to the position when the finger is operated. In addition, the cursor display (for example, white / black display) is switched according to the operation mode so that the user can easily identify the current operation mode. As will be described later, the operation mode can be switched from M0 to M1 or vice versa by performing a specific operation of holding the finger being operated at a certain position on the operation surface for a predetermined time of t0 or more. Can be switched from M1 to M0.

As a result, drawing is possible only when the operation mode is M1. Therefore, when inputting a character composed of a plurality of "pictures" such as line segments and dots, the operation mode is set to M1 for the valid "picture" part, and the "picture" and the "picture" are combined. By setting the operation mode to M0 for the invalid part that moves between, the desired character can be easily and correctly input.

FIG. 7 is a diagram showing an example of screen display in the handwritten character input process. Here, the case of inputting the hiragana "a" is taken as an example. The character input procedure is shown in (S1) to (S9), and the operation mode (M0 / M1) at that time is added.

In the following explanation, it is expressed that the cursor is "held" at a certain position, but even if the cursor position moves (blurrs) within a predetermined distance r0, this is allowed and it is regarded as the "holding" state. And. This r0 is due to an unintended movement (camera shake, etc.) of the user, and is set to a value of, for example, about several mm. Further, the time t0 for determining the "holding" state is, for example, a value of about several seconds. These threshold values may be set according to the user and the usage environment.

(S1): In the state where the operation mode is M0 (drawing stopped), the cursor is moved to the position 7a of the character input display area 502 by moving the finger PS within the position input detection range Pa.
(S2): Holds the cursor at the position 7a for a predetermined time t0 or more. As a result, the operation mode is switched to M1 (drawing execution), and the cursor display is changed.

(S3): The finger is moved when the display of the cursor changes, and the cursor is moved to the position 7b of the character input display area 502. As a result, a line from position 7a to position 7b is drawn.
(S4): The cursor is held at the position 7b for a predetermined time t0 or more. As a result, the operation mode is switched to M0 (drawing stop), and the cursor display is changed.

(S5): The finger is moved when the display of the cursor changes, and the cursor is moved to the position 7c of the character input display area 502.
(S6): Hold at position 7c for a predetermined time t0 or more. As a result, the operation mode is switched to M1 and the cursor display is changed.

Hereinafter, the drawing operation is continued in the same manner. That is, when drawing is performed, the operation mode is set to M1 and the cursor is moved, and when drawing is not performed, the operation mode is set to M0 and the cursor is moved. In addition, handwritten character input is performed by switching the operation mode by holding the cursor at the moved position for a predetermined time t0 or more.

(S7): When the drawing of the hiragana "a" is completed, the hiragana "a" is held at the position 7d for a predetermined time t0 or more. As a result, the operation mode is switched to M0, and the cursor display is changed.
(S8): Move the cursor to the recognition process instruction button 503a.
(S9): The recognition process is executed by holding the cursor on the recognition process instruction button 503a for a predetermined time t0 or more, and the recognition result is displayed in the display area 501.

FIG. 8 is a diagram showing the time change of the operation mode and the drawing position in FIG. 7. (A) indicates the position of the position input object (user's finger) PS, (b) indicates the operation mode, and (c) indicates the display unit drawing position. The symbols (S1) to (S6) and the symbols (7a) to (7c) on the horizontal axis correspond to the symbols in FIG. 7. In this way, drawing is performed only during the period when the operation mode is M1, and the operation mode is switched by holding the position input object PS at the same position for a time t0 or more.

FIG. 9 is a flowchart showing a handwritten character input process in the finger operation mode. Hereinafter, the contents of each process will be described in order.

In S601, the finger position information acquisition unit 1022 acquires the position F of the finger PS, and in S602, the acquired position F of the finger PS is converted into the position f in the display range Da as described with reference to FIG. Next, in S603, it is determined whether or not the distance r between the position f and the previously acquired position fd stored in the temporary storage area of the RAM 103 is equal to or less than a predetermined distance r0.

In the determination process S603, when the distance r is a predetermined distance r0 or more (No), the process branches to S617. In S617, it is determined whether or not the position f is within the display range of the character input display area 502. When the position f is within the range of the character input display area 502 (Yes), it branches to S618. When the position f is outside the range of the character input display area 502 (No), the process branches to S620. In S620, the operation mode is set to M0, and the process proceeds to S615.

In S618, the set operation mode is determined. When the operation mode is M0, the drawing process is not performed and the process proceeds to S615. When the operation mode is M1, it branches to S619, performs drawing processing from the position fd to the position f in the display range Da, and proceeds to S615.

On the other hand, in the determination process S603, when the distance r is equal to or less than the predetermined distance r0 (Yes), the process branches to S604. In S604, it is determined whether or not the position f is within the display range of the predetermined area 502 for inputting characters in the display range Da, or within the display range of the processing instruction button 503. If the position f is within the display range of the character input display area 502 or the processing instruction button 503 (Yes), the process proceeds to S605. If the position f is outside the display range of the character input display area 502 or the processing instruction button 503 (No), the process proceeds to S615.

In S605, the operation mode is determined by a timer. That is, the operation mode is switched by comparing the holding time with the predetermined time t0. The details of the determination process of S605 will be described with reference to FIG. In S606, branch processing is performed according to the determined operation mode. When the operation mode is M0, the process proceeds to S615, and when the operation mode is M1, the process proceeds to S607. In S607, it is determined whether or not the position f is within the display range of the processing instruction button 503. If the position f is within the display range of the processing instruction button 503 (Yes), the process proceeds to S608, and if the position f is outside the display range of the processing instruction button 503 (No), the process proceeds to S615.

In S608, branch processing is performed depending on which processing instruction button display range the position f is from 503a to 503e. If the position f is within the display range of the recognition process instruction button 503a, the process proceeds to S609 to execute the recognition process. If the position f is within the display range of the character type switching processing instruction button 503b, the process proceeds to S610 and the character type switching processing is executed. If the position f is within the display range of the erase processing instruction button 503c, the process proceeds to S611 to execute the erase process. If the position f is within the display range of the conversion process instruction button 503d, the process proceeds to S612 to execute the conversion process. After executing each process, the process proceeds to S613 to reset / stop the timer, and in S614, the operation mode is set to M0. When the position f is within the display range of the end processing instruction button 503e, the handwritten character input processing is ended.

In S615, for example, a cursor as shown in FIG. 6 is displayed at the position f of the display range Da according to the set operation mode. Next, in S616, the current position f is replaced with the previous position fd, stored in the temporary storage area of the RAM 103, and returned to S601.

FIG. 10 is a flowchart showing the operation mode determination process S605 in FIG.

In S701, it is determined whether or not the timer is started. If the timer has not been started (No), the process proceeds to S702 to start the timer and end the process. If the timer is started (Yes), the process proceeds to S703.

In S703, it is determined whether or not the timer exceeds the predetermined time t0. If the predetermined time t0 is not exceeded (No), the process ends. If the predetermined time t0 is exceeded (Yes), the process proceeds to S704.

In S704, branch processing is performed according to the operation mode. If the operation mode is M0, the process proceeds to S705, and the operation mode is switched to M1 for setting. If the operation mode is M1, the process proceeds to S706, and the operation mode is switched to M0 for setting. After that, the process proceeds to S707, and the timer is reset / stopped to end the process.

As described above, in the first embodiment, the operation mode is switched and drawing is performed by holding the position on the display screen of the position input object such as the user's finger within a predetermined distance r0 for a predetermined time t0 or more. The cursor is moved with the operation mode set to M1, and the cursor is moved with the operation mode set to M0 when drawing is not performed. As a result, even when entering characters with multiple stroke orders, the movement between strokes can be distinguished as a non-drawing section, and the intended characters can be easily and correctly entered even if the stroke order is different. Is possible. In addition, more complicated characters such as Chinese characters can be input.

Further, the processing corresponding to the processing instruction button can be performed by moving the cursor to the area where the processing instruction button is displayed and holding the position within a predetermined distance r0 from the position where the cursor is moved for a predetermined time t0 or more. It is possible to perform the same operation as clicking by moving the cursor to the position where the processing instruction button is displayed by an input means such as a mouse.

In this embodiment, the recognition process instruction button 503a, the character type switching process instruction button 503b, the erase process instruction button 503c, the character conversion process instruction button 503d, and the end process instruction button 503e are shown as the process instruction buttons. , Not limited to this, other processing instruction buttons may be provided.

Further, in this embodiment, the operation input mode (touch operation mode) by the touch panel 109 and the operation input mode (finger operation mode) by the finger position detection sensor 106 are switched by the operation input unit 111, but another method is used. You may switch. For example, when the back of the hand is tapped with a finger, the acceleration sensor 105 detects the vibration and the touch panel 109 does not detect the touch. Therefore, it is determined from the detection result of the touch panel 109 and the detection result of the acceleration sensor 105 that the back of the hand has been tapped. However, when the back of the hand is tapped with a finger, the operation input mode may be switched.

Further, in this embodiment, as a specific operation for switching the operation mode, an operation of holding the position on the display screen of a position input object such as a user's finger within a predetermined distance r0 for a predetermined time t0 or more is adopted. However, it is not limited to this. As another specific operation, for example, the cursor may be moved by moving the position input object, and the operation mode may be switched by tapping the back of the hand at the moved position.

In the second embodiment, a case where input is performed by selecting a key displayed on the screen like a keyboard will be described. In a small portable information terminal, the keys displayed on the display unit 109a are also small, and it becomes difficult to input by touching the touch panel 109 with a finger. Therefore, the key character is input in the finger operation mode.

FIG. 11 is a diagram showing a configuration of a display unit 109a for key character input processing. The area 1001 is a portion for displaying the key character input character, and the character input key 1002 is a key for selecting the character to be input. The key 1003 is for instructing various processes, and is a character type switching process instruction key 1003a for switching the type of characters to be input such as hiragana and alphabet, and a deletion process instruction key for deleting the input characters displayed in the area 1001. It is composed of 1003b, an end processing instruction key 1003c for ending the key character input processing, and the like.

FIG. 12 is a diagram showing an example of key character input processing in the finger operation mode, and here, the case of inputting the alphabet "Y" is taken as an example. The input procedure will be described in (S1) to (S4).

(S1): In the state where the operation mode is M0, by moving the finger PS within the position input detection range Pa, the cursor is moved to the position 12a where the character input key "WXYZ" is displayed.
(S2): The cursor is held at the position 12a for a predetermined time t0 or more (again, movement within a predetermined distance r0 is permitted. The same applies hereinafter). As a result, the operation mode is switched to M1 and the characters "W", "X", "Y", and "Z" are selected.

(S3): When the character selection is displayed, the finger is moved and the cursor is moved to the position 12b where the character "Y" is displayed.
(S4): The cursor is held at the position 12b for a predetermined time t0 or more. As a result, the character "Y" is determined as the input character, and the character "Y" is displayed in the input character display area 1001. When the character input is completed, the operation mode is switched to M0.

FIG. 13 is a flowchart showing a key character input process in the finger operation mode.

In S1101, the finger position information acquisition unit 1022 acquires the position F of the finger PS, and in S1102, the acquired position F of the finger PS is converted into the position f in the display range Da. Next, in S1103, it is determined whether or not the distance r between the position f and the previously acquired position fd is equal to or less than the predetermined distance r0.

In the determination process S1103, when the distance r is a predetermined distance r0 or more (No), the process branches to S1114. When the distance r is equal to or less than the predetermined distance r0 (Yes), the process branches to S1104. In S1104, it is determined whether or not the position f is within the display area of the character input key 1002 or the processing instruction key 1003. In the determination process S1104, when the position f is within the display area of the character input key 1002 or the process instruction key 1003 (Yes), the process branches to S1105, and when the position f is outside the display area of the character input key 1002 or the process instruction key 1003 (No). ) Branches to S1114.

In S1105, the operation mode is determined by a timer. That is, the operation mode is switched by comparing the holding time with the predetermined time t0. The details of the determination process of S1105 will be described with reference to FIG. In S1106, branch processing is performed according to the determined operation mode. When the operation mode is M0, the process proceeds to S1114, and when the operation mode is M1, the process proceeds to S1107. In S1107, it is determined whether or not the character selection display is on.

In the determination process S1107, if the character selection display is on (Yes), the process proceeds to S1116, and if the character selection display is off (No), the process proceeds to S1108. In S1116, it is determined whether or not the position f is within the character selection display area. If the position f is within the character selection area (Yes), the process proceeds to S1117, and if the position f is outside the character selection display area (No), the process proceeds to S1114. In S1117, the character displayed at the position f is determined as an input character and displayed in the input character display area 1001. In S1118, the character selection display is turned off, and the process proceeds to S1112.

In S1108, branch processing is performed depending on whether the position f is within the display area of either the character input key 1002 or the processing instruction key 1003. When the position f is within the display area of the character input key 1002, the process proceeds to S1109, the character selection display is turned on, and the character selection display corresponding to the key displayed at the position f is performed. If the position f is within the display area of the character type switching processing instruction key 1003a, the process proceeds to S1110 and the character type switching processing is executed. If the position f is within the display area of the deletion processing instruction key 1003b, the process proceeds to S1111 and the deletion processing is executed. After executing each process, the process proceeds to S1112 to reset / stop the timer, and in S1113, the operation mode is set to M0. When the position f is within the display area of the end processing instruction key 1003c, the key character input processing is ended.

In S1114, for example, a cursor as shown in FIG. 6 is displayed at the position f of the display range Da according to the set operation mode. Next, in S1115, the current position f is replaced with the previous position fd, stored in the temporary storage area of the RAM 103, and returned to S1101.

FIG. 14 is a flowchart showing the operation mode determination process S1105 in FIG.

In S1201, it is determined whether or not the timer is started. If the timer is not started (No), the process proceeds to S1202 to start the timer and end the process. If the timer is started (Yes), the process proceeds to S1203.

In S1203, it is determined whether or not the timer exceeds the predetermined time t0. If the predetermined time t0 is not exceeded (No), the process ends. If the predetermined time t0 is exceeded (Yes), the process proceeds to S1204.

In S1204, the operation mode is switched to M1 and set. In this case, in S1113 of FIG. 13, since the operation mode after the key character input is executed is switched to M0, it is not necessary to switch from M1 to M0. After that, the process proceeds to S1205, the timer is reset / stopped, and the process ends.

As described above, in the second embodiment, the cursor is moved to the position where the character input key or the processing instruction key is displayed, and the position within a predetermined distance r0 from the moved cursor position is held for a predetermined time t0 or more for input. It is possible to easily select the character to be performed or instruct the process to be executed. As a result, it is possible to perform the same operation as clicking by moving the cursor to the position where the character input key or the processing instruction key is displayed by an input means such as a mouse.

In this embodiment, an example is shown in which the character type switching processing instruction key 1003a, the deletion processing instruction key 1003b, and the end processing instruction key 1003c are used as the processing instruction keys, but the present invention is not limited to this, and other processing instructions can be used. A corresponding key may be provided. Further, as the character input key, a keyboard in which one character of the alphabet or hiragana corresponds to one key or a keyboard corresponding to the input of numbers may be displayed and input.

In the third embodiment, operations such as mouse click, double click, and drag and drop (hereinafter referred to as mouse operation input processing) are realized in the finger operation mode. In order to deal with this, the types of operation modes are added in this embodiment.

FIG. 15 is a diagram showing an example of an operation mode and a cursor display used in the mouse operation input process. Here, six types of operation modes M0 to M5 are used. The operation mode M0 is a state in which no input operation other than cursor movement is performed even if a finger is operated. In the operation modes M1 to M5, mouse operations of click processing (M1), double-click processing (M2), start of drag processing (M3), end of drag processing (M4), and drop processing (M5) are performed, respectively. is there. In addition, the cursor display (shape and / or color display) is switched according to each operation mode. As will be described later, the operation mode is switched by comparing the length of time for holding the finger during operation at a certain position on the operation surface with the three threshold values t0, t1, and t2.

FIG. 16 is a diagram showing an example of screen display in mouse operation input processing. On the display unit 109a, the icon group 1401 corresponding to the applications A to D is displayed as a window screen (home screen). The user can select a desired application by clicking the icon, execute the application by double-clicking, and move the display position of the icon by dragging and dropping. The flow of a series of processes will be described in (S1) to (S7).

(S1): In the state where the operation mode is M0, the cursor is moved to the position 14a where the icon [D] is displayed by moving the finger within the position input detection range.
(S2): The cursor is held at the position 14a for a predetermined time (range of t0 to t1). As a result, the operation mode is switched to M1 (click processing), and the icon "D" is selected as the mouse operation target. Further, the display of the cursor changes to the display corresponding to M1 in FIG. After that, by arbitrarily moving the finger until the predetermined time t1 elapses (in this case, the cursor is moved to a position separated by a predetermined distance r0 or more from the position 14a), the selected icon "D" is Click processing is executed.

(S3): When the cursor is held at the position 14a for a predetermined time t1 or more, the operation mode is switched to M2 (double-click processing), and the cursor changes to the display corresponding to M2 shown in FIG. After that, by arbitrarily moving the finger until the predetermined time t2 elapses (moving the cursor from the position 14a by a predetermined distance r0 or more), the double-click process is executed for the selected icon "D". ..

(S4): When the cursor is held at the position 14a for a predetermined time t2 or more, the operation mode is switched to M3 (drag processing start), and the cursor changes to the display corresponding to M3 shown in FIG.
(S5): After that, by moving the finger to move the cursor to the position 14b separated from the position 14a by a predetermined distance r0 or more, the operation mode is switched to M4 (drag processing end). As a result, the cursor changes to the display corresponding to M4 shown in FIG. Further, the icon "D" selected as the mouse operation target is moved and displayed at the position 14b where the cursor is moved, and the drag process for moving the position of the selected icon is executed.

(S6): Further move the finger to move the cursor and the selected icon [D] to the position 14c, and hold the cursor at the position 14c for a predetermined time t0 or more. As a result, the operation mode is switched to M5 (drop processing), and the drop processing is executed at the position 14c.
(S7): The display position of the selected icon "D" is set to 14c by the drop process. After that, the operation mode is switched to M0, and the cursor is displayed corresponding to M0.

FIG. 17 is a flowchart showing mouse operation input processing in the finger operation mode.

In S1501, the finger position information acquisition unit 1022 acquires the position F of the finger PS, and in S1502, the acquired position F of the finger PS is converted into the position f in the display range Da. Next, in S1503, it is determined whether or not the distance r between the position f and the previously acquired position fd is equal to or less than the predetermined distance r0.

In the determination process S1503, when the distance r is equal to or less than the predetermined distance r0 (Yes), the process branches to S1504, and in S1504, the operation mode is determined by the timer. That is, the operation mode is switched by comparing the holding time with the predetermined times t0, t1, and t2. The details of the determination process of S1504 will be described with reference to FIG. After that, in S1505, it is determined whether or not the operation mode determined in S1504 is M5. When the operation mode is M5 (Yes), it branches to S1508, and when the operation mode is not M5 (No), it branches to S1506. In S1508, the drop process of setting the display position of the icon selected by the drag process to the position f is executed, and the process proceeds to S1509.

In S1506, it is determined whether or not the position f is within the display area of the icon. When the position f is within the icon display area (Yes), it branches to S1507, and when it is outside the icon display area (No), it branches to S1509. In S1507, the icon whose position f is in the display area is selected as the mouse operation target, and the process proceeds to S1518. In S1509, the operation mode is set to M0, and the process proceeds to S1510. In S1510, the timer is reset / stopped, and the process proceeds to S1518.

On the other hand, in the determination process S1503, when the distance r is a predetermined distance r0 or more (No), the process branches to S1511. In S1511, the following branch processing is performed according to the operation mode.
When the operation mode is M0, the process proceeds to S1518 without processing.
When the operation mode is M1, the process proceeds to S1515, the click process is performed on the icon selected as the mouse operation target in S1507, and the process proceeds to S1516.
When the operation mode is M2, the process proceeds to S1514, the icon selected as the mouse operation target in S1507 is double-clicked, and the process proceeds to S1516.
When the operation mode is M3, the process proceeds to S1513, the operation mode is set to M4, and the process proceeds to S1517.
When the operation mode is M4, the process proceeds to S1512, and the icon selected as the mouse operation target in S1507 is displayed at the position f. As a result, a drag process for moving the position of the selected icon is performed, and then the process proceeds to S1518.

In S1516, the operation mode is set to M0, and in S1517, the timer is reset / stopped and the process proceeds to S1518. In S1518, a cursor as shown in FIG. 15 is displayed at the position f of the display range Da according to the current operation mode. Next, in S1519, the current position f is replaced with the previous position fd, stored in the temporary storage area of the RAM 103, and returned to S1501.

FIG. 18 is a flowchart showing the operation mode determination process S1504 in FIG.

In S1601, it is determined whether or not the timer is started. If the timer has not been started (No), the process proceeds to S1602 to start the timer and end the process. If the timer is started (Yes), the process proceeds to S1603.

In S1603, it is determined whether or not the operation mode is M4. If the operation mode is M4 (Yes), the process proceeds to S1604. In S1604, it is determined whether or not the timer exceeds the predetermined time t0, and if the timer does not exceed the predetermined time t0 (No), the process ends. If the predetermined time t0 has been exceeded (Yes), the process proceeds to S1605, the operation mode is set to M5, and the process ends. In the determination process S1603, if the operation mode is not M4 (No), the process proceeds to S1606.

In S1606, the following branch processing is performed according to the elapsed time of the timer. The following time thresholds have a relationship of t0 <t1 <t2.
If the timer does not exceed the predetermined time t0 (timer ≤ t0), the process ends.
When the timer is a time between t0 and t1 (t0 <timer ≦ t1), the process proceeds to S1609, the operation mode is set to M1, and the process ends.
When the timer is a time between t1 and t2 (t1 <timer ≦ t2), the process proceeds to S1608, the operation mode is set to M2, and the process ends.
When the timer exceeds the predetermined time t2 (timer> t2), the process proceeds to S1607, the operation mode is set to M3, and the process ends.

As described above, in the third embodiment, the operation mode is switched between M0 and M5 according to the length of time for holding the position within a predetermined distance r0 from the position where the cursor is moved. Then, after the operation mode is switched, by moving the cursor to a position separated by a predetermined distance r0 or more like a flick operation in the touch operation of the smartphone, click with the mouse, double-click, drag and drop, etc. The operation can be realized in the finger operation input mode. In addition, it is possible to confirm that the operation mode has been switched by changing the display of the cursor, and it is possible to easily perform the mouse operation intended by the user.

In the fourth embodiment, a case where the position of the user's finger is three-dimensionally detected and the input operation is performed (hereinafter, referred to as a 3D finger operation mode) will be described.

FIG. 19 is a diagram showing the detection principle of the 3D finger operation mode, and shows the relationship between the position of the finger and the display position of the display unit. The position of the finger PS is detected not only in the direction parallel to the display unit 109a of the input terminal device 1 (XY-axis direction) but also in the vertical direction (height Z-axis direction), and the input operation is performed. The figure shows the relationship between the position F of the detected finger (position input object) PS and the display position f of the display unit 109a.

Three finger position detection sensors 106a, 106b, and 106c are provided to detect the finger position three-dimensionally. Each of the sensors 106a to 106c is composed of, for example, an infrared light emitting element and a light receiving element, and each sensor 106a to 106c outputs a signal having a level corresponding to the distances La, Lb, and Lc to the finger PS. In the finger position information acquisition unit 1022, the position input detection range (space) surrounded by the positions ABCD-A'-B'-C'-D'based on the output signals from the sensors 106a to 106c. The position F of the finger PS in Pa is acquired. Corresponding to the acquired position F in the XY axis direction, the position f in the display range Da surrounded by the positions abc-d of the display unit 109a is determined, and for example, a cursor or the like is displayed. On the other hand, a plurality of display screens are switched according to the acquired position F in the Z-axis direction, as will be described later.

FIG. 20 is a diagram showing an example of screen display in the 3D finger operation mode. Here, a window screen (home screen) in which icons are arranged is displayed as a screen for operation. The position in the height Z direction with respect to the display unit 109a is divided into a plurality of layers L0, L1 and L2 with Z0 to Z3 as boundaries. The window screen is switched and displayed according to which layer the finger PS is located on. The display states of each layer are indicated by (S1) to (S3).

(S1): When the position F0 of the finger PS is in the range of the height direction Z0 to Z1 (layer L0), the icon group 1901 corresponding to the applications A to D is displayed on the window screen.
(S2): When the position F1 of the finger PS is in the range Z1 to Z2 in the height direction (layer L1), the icon group 1902 corresponding to the applications E to H is displayed on the window screen.
(S3): When the position F2 of the finger PS is in the range Z2 to Z3 in the height direction (layer L2), the icon group 1903 corresponding to the applications I to L is displayed on the window screen.

After that, the mouse operation input process described in the third embodiment is executed. That is, the application is selected by clicking the displayed icon, the application is executed by double-clicking the icon, and the display position of the icon is moved by dragging and dropping the icon.

As described above, according to the 3D finger operation mode, the screen display for scanning can be switched for each of a plurality of layers, so that mouse operation input processing can be easily performed for a large number of applications (icons).

FIG. 21 is a flowchart showing mouse operation input processing in the 3D finger operation mode. The same processing as in Example 3 (FIG. 17) is designated by the same reference numerals, and the repeated description will be omitted.

In S1501, the finger position information acquisition unit 1022 acquires the three-dimensional position F of the finger PS, and in S1502, the position of the acquired position F in the XY direction is converted to the position f in the display range Da as described in FIG. .. Next, in S1520, it is determined which layer the acquired position F is in the Z direction. In S1521, it is determined whether or not the determined layer LN is the same as the previously determined layer LNd stored in the temporary storage area of the RAM 103. As a result of the determination, if the determined layer LN is the same as the previously determined layer LNd (Yes), the process proceeds to S1503. The processing after S1503 is the same as that in FIG. 17, and the description thereof will be omitted. As a result of the determination, if the determined layer LN is different from the previously determined layer LNd (No), the process proceeds to S1522.

In S1522, it is determined whether or not the operation mode is M3 or M4. If the operation mode is M3 or M4 (Yes), the process proceeds to S1523, and if the operation mode is not M3 or M4 (No), the process proceeds to S1525. In S1525, the operation mode is set to M0, and the process proceeds to S1526. In S1526, the timer is reset / stopped, and the process proceeds to S1523.

In S1523, the discriminated layer LN is replaced with the previously discriminated layer LNd, stored in the temporary storage area of the RAM 103, and the process proceeds to S1524. In S1524, the screen of the display unit 109a is updated so as to display the window screen (home screen) corresponding to the determined layer LN, and the process proceeds to S1511. The processing after S1511 is mouse operation input processing according to each operation mode M0 to M4, but it is the same as that of FIG. 17, and the description thereof will be omitted.

As described above, by detecting the three-dimensional position of the user's finger and inputting the operation, it is divided into a plurality of layers according to the position in the height Z direction, and the window screen for operation corresponding to each layer ( Home screen) can be displayed. It is also possible to perform mouse operations such as clicking, double-clicking, and dragging and dropping the icons displayed in the windows of each layer.

In the above example, icons corresponding to different applications for each of a plurality of layers are displayed on the window screen, but the present invention is not limited to this, and it is possible to display different operation surfaces for each layer.

For example, FIG. 22 is a diagram showing an example of screen display in the key character input process. In the key character input process, a keyboard with a different character type for each layer is displayed on the display unit.

(S1): When the position F0 of the finger PS is within the range of the layer L0, the keyboard for inputting alphabets and symbols is displayed.
(S2): When the position F1 is within the range of the layer L1, the keyboard for inputting hiragana is displayed.
(S3): When the position F2 is within the range of the layer L2, the keyboard for inputting the numbers is displayed.
As a result, the user can easily change the character type to be input by changing the position of the finger PS in the height Z direction.

As described above, according to the fourth embodiment, since the screen display and the operation contents can be switched for each of a plurality of layers by the 3D finger operation mode, various kinds of operations can be easily realized. The user may appropriately set the number of layers depending on the type of operation to be used.

In the fifth embodiment, a configuration in which a transition region is provided between the layers will be described in order to smoothly switch between the plurality of layers in the 3D finger operation mode.

FIG. 23 is a diagram showing an example of screen display when a transition region is provided between layers. The position of the finger PS in the height Z direction is divided into a plurality of layers L0 to L2, and layer transition areas T01 and T12 are provided at the boundary of each layer. Similar to the fourth embodiment, window screens (icon groups) for different operations are displayed corresponding to each layer L0 to L2, but in the layer transition areas T01 and T12, any adjacent layer is supported. Display the window screen. Hereinafter, changes in the screen display will be described along with the movement of the finger PS in the height direction.

First, the case where the position of the finger PS is moved from F0 to F4 (that is, layers L0 to L2) will be described in (S1) to (S5).

(S1): The position F0 of the finger PS is within the range of the layer L0, and the icon group 2101 corresponding to the applications A to D is displayed on the window screen.
(S2): When the position F1 of the finger PS moves to the layer transition area T01 between the layers L0 and L1, the window screen (icon group 2101) displayed on the immediately preceding layer L0 is continuously displayed. At that time, the screen background is changed to the display 109T indicating the transition area.

(S3): The position F2 of the finger PS is within the range of the layer L1, and the icon group 2102 corresponding to the applications E to H is switched and displayed on the window screen.
(S4): When the position F3 of the finger PS moves to the layer transition area T12 between the layers L1 and L2, the window screen (icon group 2102) displayed on the immediately preceding layer L1 is continuously displayed. At that time, the screen background is changed to the display 109T indicating the transition area.

(S5): The position F4 of the finger PS is within the range of the layer L2, and the icon group 2103 corresponding to the applications I to L is switched and displayed on the window screen.

On the other hand, the case where the position of the finger PS is moved from F4 to F0 (that is, from layers L2 to L0) will be described in (S5') to (S1').

(S5'): The position F4 of the finger PS is within the range of the layer L2, and the icon group 2103 corresponding to the applications I to L is displayed on the window screen.
(S4'): When the position F3 of the finger PS moves to the layer transition area T12 between the layers L1 and L2, the window screen (icon group 2103) displayed on the immediately preceding layer L2 is continuously displayed. At that time, the screen background is changed to the display 109T indicating the transition area.

(S3'): The position F2 of the finger PS is within the range of the layer L1, and the icon group 2102 corresponding to the applications E to H is switched and displayed on the window screen.
(S2'): When the position F1 of the finger PS moves to the layer transition area T01 between the layers L0 and L1, the window screen (icon group 2102) displayed on the immediately preceding layer L1 is continuously displayed. At that time, the screen background is changed to the display 109T indicating the transition area.

(S1'): The position F0 of the finger PS is within the range of the layer L0, and the icon group 2101 corresponding to the applications A to D is switched and displayed on the window screen.

As described above, when the finger PS is located within the range of each layer, the window screen corresponding to each layer is displayed, and when the finger PS moves to the layer transition area, the window screen of the layer located immediately before is displayed. Is displayed continuously, and the background is changed and displayed.

In this way, a transition area is provided between each layer, and when the finger PS moves to the transition area, the background of the window screen is changed to notify the user that the transition area has been moved. At that time, since the window screen displayed immediately before is continued as it is, the operation is not hindered. This prevents the user from unintentionally moving to another layer. That is, when the background of the window screen changes, the finger PS may be moved in the direction of the original layer.

FIG. 24 is a flowchart showing a mouse operation input process when a transition region is provided between layers. The same processes as in Example 3 (FIG. 17) and Example 4 (FIG. 21) are designated by the same reference numerals, and repeated description thereof will be omitted.

In S1501, the three-dimensional position F of the finger PS is acquired, and in S1502, the position of the position F in the XY direction is converted into the position f in the display range Da. Next, in S1520, it is determined whether the position of the position F in the Z direction is within the range of which layer or the layer transition region. In S1530, branch processing is performed according to the layer discrimination result.

In the determination process S1530, when the determined layer LN is within the layer transition region (LN = between layers), the process branches to S1531. In S1531, the background of the window display screen is set to the display 109T corresponding to the layer transition area, and the process proceeds to S1503.

In the determination process S1530, if the determined layer LN is the same as the previously determined layer LNd (LN = LNd), the process proceeds to S1503. The processing after S1503 is the same as that in FIG. 17, and the description thereof will be omitted.

In the determination process S1530, if the determined layer LN is different from the previously determined layer LNd (LN ≠ LNd), the process proceeds to S1522. The processing after S1522 is the same as that in FIG. 21, and the description thereof will be omitted.

As described above, according to the fifth embodiment, not only the three-dimensional position of the finger PS is detected and the windows corresponding to the plurality of layers corresponding to the positions in the height direction Z are displayed, but also between the layers. A transition area is provided. Then, when the finger PS moves to the transition area, the background of the window screen is changed to notify the user, and it is possible to prevent the user from unintentionally moving to another layer. As a result, switching between each layer can be performed smoothly.

In this embodiment, the background of the window screen is changed when the finger PS moves to the layer transition area, but the present invention is not limited to this. For example, a method that can be recognized by the user, such as changing the display of the icon, changing the display of the cursor, or generating a sound or vibration, may be used. Further, when it is located in the transition area, the direction (up / down) of the original layer may be displayed.

In the above description of each embodiment, an example in which the finger position detection sensor 106 is composed of a light emitting / receiving element is shown, but the present invention is not limited to this, and a user's finger such as a sensor using ultrasonic waves or an image sensor such as a camera is used. Anything that can detect the position of the position input object such as a pen or a pen is sufficient. Further, although the wristwatch type input terminal device has been described as an example, in a smartphone, tablet, personal computer, etc., the position of a position input object such as a user's finger or pen is detected to input handwritten characters or operate a mouse. You may do so.

Further, in each of the above embodiments, as a condition of a specific operation for switching the operation mode, the position of the position input object such as the user's finger on the display screen is held within a predetermined distance for a predetermined time or longer. However, it is not limited to this. For example, the condition may be that the amount of movement of the position on the display screen of the position input object within a predetermined time is within a predetermined distance. Alternatively, the condition may be that the moving speed of the position on the display screen of the position input object is equal to or less than a predetermined speed.

Although the embodiments of the present invention have been described above with reference to some examples, it goes without saying that the configuration for realizing the technique of the present invention is not limited to the above examples, and various modifications can be considered. .. For example, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. All of these belong to the category of the present invention. In addition, numerical values and messages appearing in sentences and figures are merely examples, and the effects of the present invention may not be impaired even if different ones are used. Further, the programs described in each processing example may be independent programs, or a plurality of programs may constitute one application program. Further, the order in which each process is performed may be changed and executed.

The above-mentioned functions of the present invention and the like may be realized by hardware, for example, by designing a part or all of them by an integrated circuit. Further, the microprocessor unit or the like may realize it by software by interpreting and executing an operation program that realizes each function or the like. Hardware and software may be used together.

In addition, the control lines and information lines shown in the drawings are those considered necessary for explanation, and do not necessarily indicate all the control lines and information lines on the product. In practice, it can be considered that almost all configurations are interconnected.

1: Input terminal device, 2: Mobile information terminal, 101: Main control unit, 102: ROM, 103: RAM, 104: Storage unit, 105: Acceleration sensor, 106, 106a, 106b, 106c: Finger position detection sensor, 108 : Display control unit, 109: Touch panel, 109a: Display unit, 109b: Touch operation input unit, 110: Measuring unit, 111: Operation input unit, 1022: Finger position information acquisition unit, 1023: Touch position information acquisition unit, 1024: Finger operation processing control unit 1024: Touch operation processing control unit, PS: User's finger (position input object).

Claims (7)

It is an operation method for operating a terminal device via a position input object.
A position detection step that non-contactly detects the position of the position input object, and
A display step of displaying a cursor on the display unit of the terminal device capable of selecting a target based on the position detected in the position detection step, and a display step.
It has an operation processing step for causing the terminal device to execute a corresponding operation process based on the position detected in the position detection step.
In the operation processing step, switching between a plurality of operation modes related to execution of the operation processing including an operation mode in which no operation processing other than the movement of the cursor is executed as an operation according to the position of the position input object. It can be executed by holding the position of the position input object within a predetermined range for a predetermined time or longer .
The plurality of operation modes include a first operation mode in which the cursor is moved and drawn on the display unit according to the movement of the position detected in the position detection step, and the position detected in the position detection step. An operation method including a second operation mode in which only the cursor is moved on the display unit in response to the movement. The operation method according to claim 1.
In the display step, at least one of the shapes or colors of the cursor is displayed on the display unit in a different shape according to the plurality of operation modes. The operation method according to claim 1.
Further, an operation method comprising a transmission step of transmitting information from the terminal device to a mobile information terminal wirelessly connected to the terminal device based on the operation process in the operation processing step. It is an operation method for operating a terminal device via a position input object.
A position detection step that detects the position of the position input object in a non-contact manner, and
A display step of displaying a cursor on the display unit of the terminal device capable of selecting a target based on the position detected in the position detection step, and a display step.
It has an operation processing step for causing the terminal device to execute a corresponding operation process based on the position detected in the position detection step.
In the operation processing step, switching between a plurality of operation modes related to execution of the operation processing including an operation mode in which no operation processing other than the movement of the cursor is executed as an operation according to the position of the position input object. It can be executed by performing a specific operation via the position input object.
Further, in the operation processing step, the processing instruction button is displayed on the display unit, and the processing is performed when the position of the cursor is held within a predetermined range for a predetermined time or longer at the position where the processing instruction button is displayed. An operation method characterized by executing a process corresponding to an instruction button. The operation method according to claim 4.
Further, an operation method comprising a transmission step of transmitting information from the terminal device to a mobile information terminal wirelessly connected to the terminal device based on the operation process in the operation processing step. It is an operation method for operating a terminal device via a position input object.
A position detection step that detects the position of the position input object in a non-contact manner, and
A display step of displaying a cursor on the display unit of the terminal device capable of selecting a target based on the position detected in the position detection step, and a display step.
It has an operation processing step for causing the terminal device to execute a corresponding operation process based on the position detected in the position detection step.
In the operation processing step, switching between a plurality of operation modes related to execution of the operation processing including an operation mode in which no operation processing other than the movement of the cursor is executed as an operation according to the position of the position input object. It can be executed by performing a specific operation via the position input object.
Further, in the operation processing step, the position in the vertical direction with respect to the display unit is divided into a plurality of layers, and the position of the position input object detected in the position detection step is within the range of which layer of the plurality of layers. Switch the operation screen to be displayed on the display unit depending on the presence .
In the operation processing step, a transition region is further set at the boundary of the plurality of layers, and when the position of the position input object moves within the transition region, the operation displayed immediately before is displayed on the display unit. screen and displays to continue the use, operation wherein the Rukoto change the background of the screen. The operation method according to claim 6.
Further, an operation method comprising a transmission step of transmitting information from the terminal device to a mobile information terminal wirelessly connected to the terminal device based on the operation process in the operation processing step.

Images