JP6823988B2 - Screen operation method and screen operation device - Google Patents

Description

The present invention relates to a screen operation method and a screen operation device for operating the position of a cursor.

Conventionally, for example, a screen operation method has been known in which a gesture by an operator is detected and the position of a cursor on the screen is operated according to the detection result. Further, there is also known a method of detecting the line of sight of the operator and manipulating the position of the cursor on the screen according to the detection result (see, for example, Patent Document 1).

Special Table 2014-514661

However, the conventional screen operation method may not be able to accurately operate the cursor position, and has a problem of low reliability.
For example, in the method of manipulating the position of the cursor according to a gesture, the operating range of the operator is narrower than the operating range on the screen, and detailed operations cannot be performed.

For example, as shown in FIG. 12, buttons 102 of the function menu are arranged at 100 pixel intervals (reference numeral l) on the screen 101 displayed on the 100-inch display device, and are operated by the cursor 103. In this case, the distance between the buttons 102 on the screen 101 is about 11.5 cm, and when operating the distance by the movement of the hand, it is necessary to move the hand by, for example, about 4 cm. However, the movement of the arm that aims at the position of the hand floating in the air and moves it by 4 cm is a heavy physical burden, and since the movement is small, a detection error occurs and the position of the cursor 103 cannot be operated accurately. The probability is high.

Further, in the method of manipulating the position of the cursor according to the line of sight, a detection error or a detection error is likely to occur at the edge of the screen due to the method of detecting the direction of the line of sight, and it is difficult to guide the cursor to the intended place.

That is, in the line-of-sight direction detection, for example, as shown in FIG. 13, from the eyeball image of the operator who captured infrared rays as a light source using an infrared filter, the center point 104 of the pupil and the corneal reflex point (infrared ray reflection point) 105 Is detected, and the line-of-sight direction is obtained from the relative position.
However, for example, as shown in FIG. 14, when the movement of the line of sight becomes large and the center point 104 of the pupil deviates from the center position of the eyeball image, the corneal reflex point 105 shifts from the pupil portion to the white eye portion. In this case, the corneal reflex point 105 cannot be detected, or a plurality of reflections are disturbed and reflected at the edge of the pupil and the white of the eye, resulting in erroneous recognition.

In contrast to the above, a method of operating the position of the cursor by the operation of the mouse using a mouse is also generally known. However, especially when manipulating the position of the cursor on the screen having a large screen and high resolution, the method using a mouse is not preferable because it is complicated to switch the input on and off and it is difficult to move the cursor greatly.

The present invention has been made to solve the above problems, and an object of the present invention is to provide a screen operation method capable of accurately operating the position of a cursor.

In the screen operation method according to the present invention, the first action acquisition unit acquires the first action by the operator who operates the screen, and the operation range designation unit obtains the first action acquired by the first action acquisition unit. A part of the area on the screen is designated as the operation range, the second action acquisition unit acquires the second action by the operator, and the cursor position operation unit is from the second action acquired by the second action acquisition unit. , The feature is that the position of the cursor in the operation range is operated with an operation amount smaller than the operation amount which is the movement amount of the cursor with respect to the screen according to the ratio of the operation range specified by the operation range specification unit to the screen. And.

According to the present invention, since the configuration is as described above, the position of the cursor can be accurately manipulated.

It is a figure which shows the configuration example of the screen operation system including the screen operation device which concerns on Embodiment 1 of this invention. It is a flowchart which shows the operation example of the screen operation apparatus which concerns on Embodiment 1 of this invention. 3A and 3B are diagrams for explaining an operation example of the screen operation device according to the first embodiment of the present invention. 4A and 4B are diagrams for explaining another operation example of the screen operation device according to the first embodiment of the present invention. It is a figure which shows the configuration example of the screen operation system including the screen operation device which concerns on Embodiment 2 of this invention. 6A to 6C are diagrams for explaining an operation example of the screen operation device according to the second embodiment of the present invention. It is a figure which shows the structural example of the screen operation system including the screen operation device which concerns on Embodiment 3 of this invention. 8A and 8B are diagrams for explaining an operation example of the screen operation device according to the third embodiment of the present invention. It is a figure which shows the configuration example of the screen operation system including the screen operation device which concerns on Embodiment 4 of this invention. 10A and 10B are diagrams for explaining an operation example of the screen operation device according to the fourth embodiment of the present invention. 11A and 11B are diagrams showing a hardware configuration example of the screen operation device according to the first embodiment of the present invention. It is a figure which shows an example of the screen displayed on a display device. It is a figure explaining the line-of-sight detection method. It is a figure explaining the problem by the conventional screen operation method.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Embodiment 1.
FIG. 1 is a diagram showing a configuration example of a screen operation system including the screen operation device 3 according to the first embodiment of the present invention.
The screen operation system includes a display device 1, a detection device 2, and a screen operation device 3. The figure shows a case where the display device 1, the detection device 2, and the screen operation device 3 are separately configured, but the present invention is not limited to this, and the display device 1, the detection device 2, and the screen operation device 3 are other. It may be integrated with the device of. Further, the display device 1 and the screen operation device 3 and the detection device 2 and the screen operation device 3 are connected by wire or wirelessly, respectively, and are configured to be communicable.

The display device 1 displays a screen. Examples of the display device 1 include a PC display, a television, a head-mounted display, and the like.

The detection device 2 detects the first and second actions by the operator who operates the screen. Even when a plurality of persons are present facing the display device 1, the detection device 2 can detect the person by detecting, for example, a person holding a marker or a person performing a specific action. It can be recognized as an operator. The first and second actions include, for example, gestures (movement of body parts such as hands and face), movement of eyes, movement of mouse, keys or buttons using a keyboard or the like placed at the operator's hand, and the like. Operation, vocalization, etc. It should be noted that the first action and the second action may be any action that can be distinguished and are predetermined. The data indicating the first and second actions detected by the detection device 2 is output to the screen operation device 3.

The detection device 2 has, for example, one or more input devices such as a camera, a gesture recognition device for recognizing gestures, a line-of-sight recognition device for recognizing a line of sight, a mouse, a keyboard, and a microphone, and is obtained by this input device. The first and second actions are detected using the data.
For example, when the first action is the movement of the face and the second action is the movement of the hand, it can be realized by using one visible camera as an input device. On the other hand, when the first action is the movement of the face and the second action is the movement of the line of sight, it can be realized with one visible camera, but when improving the accuracy of the line of sight detection, an infrared light source and an infrared camera are used. Used separately. Since the resolution of the infrared camera is low, the distance between the display device 1 and the operator is limited. Therefore, a head-mounted display may be used as the display device 1, and an infrared light source and an infrared camera may be mounted on the display to detect the line of sight. The specific method for detecting the first and second actions by the detection device 2 is the same as that of the existing technology, and the description thereof will be omitted.

The screen operation device 3 operates the screen displayed on the display device 1. In the following, among the functions of the screen operation device 3, only the function of operating the position of the cursor will be described. As shown in FIG. 1, the screen operation device 3 includes a first action acquisition unit 301, an operation range designation unit 302, a second action acquisition unit 303, and a cursor position operation unit 304 as functions for operating the cursor position. ing.

The first action acquisition unit 301 acquires the first action by the operator. In the example of FIG. 1, the first action acquisition unit 301 acquires the first action by receiving the data output from the detection device 2. The first action acquisition unit 301 starts the process, for example, when the operator stands at a predetermined position, or when a display prompting the first action is displayed on the screen.

The operation range designation unit 302 designates a part of the area on the screen as the operation range from the first action acquired by the first action acquisition unit 301. At this time, for example, when the position on the screen corresponding to the first action is located within the threshold range for a specified time, the operation range designation unit 302 designates the area including the position as the operation range. .. The area may be set in advance by dividing the screen into a plurality of screens, or may be set each time around the position on the screen corresponding to the first action. Further, the size of the area is set to an appropriate size depending on, for example, the size of the entire screen, the distance between the display device 1 and the operator, the standing position of the operator, and the like.

The second action acquisition unit 303 acquires the second action by the operator. In the example of FIG. 1, the second action acquisition unit 303 acquires the second action by receiving the data output from the detection device 2. The second action acquisition unit 303 may start processing when, for example, the operation range is designated by the operation range designation unit 302, or may perform processing in parallel with the first action acquisition unit 301. May be good.

The cursor position operation unit 304 is an operation amount corresponding to the ratio of the operation range specified by the operation range designation unit 302 to the screen from the second action acquired by the second action acquisition unit 303, and the cursor position in the operation range Manipulate the position.

In FIG. 1, the first and second action acquisition units 301 and 303 acquire the first and second actions by receiving the data output from the detection device 2. However, the present invention is not limited to this, and the detection processing of the first and second actions based on the data obtained by the input device by the detection device 2 may be performed by the first and second action acquisition units 301 and 303.

Next, an example of operating the cursor position by the screen operating device 3 configured as described above will be described with reference to FIG. It is assumed that the display device 1 displays the screen and the detection device 2 constantly detects the first and second actions.
In the operation example of the cursor position by the screen operation device 3, as shown in FIG. 2, the first action acquisition unit 301 first receives the data output from the detection device 2 to perform the first action by the operator. (Step ST201).

Next, the operation range designation unit 302 designates a part of the area on the screen as the operation range from the first action acquired by the first action acquisition unit 301 (step ST202). At this time, for example, when the position on the screen corresponding to the first action is located within the threshold range for a specified time, the operation range designation unit 302 sets the area including the position as the operation range. .. As a result, the operation range in which the operator operates the cursor position can be narrowed down from the entire screen to a part of the screen. Also, the cursor is moved within the operating range.
Further, the operation range designation unit 302 may display a frame indicating the designated operation range on the screen. As a result, the operator can confirm the designated operation range.

Next, the second action acquisition unit 303 acquires the second action by the operator by receiving the data output from the detection device 2 (step ST203).

Next, the cursor position operation unit 304 performs an operation amount in the operation range according to the ratio of the operation range specified by the operation range designation unit 302 to the screen from the second action acquired by the second action acquisition unit 303. Manipulate the position of the cursor (step ST204). That is, by limiting the operation range from the entire screen to a part of the screen, the operation range with respect to the operation range of the operator becomes smaller, and the operator can perform the operation with a larger operation.

If the first action acquisition unit 301 acquires the first action again after the operation range designation unit 302 specifies the operation range and before the second action acquisition unit 303 acquires the second action, the operation range The designation unit 302 redesignates the operation range.

Further, in the above, the case where the second action acquisition unit 303 starts the process after the designation by the operation range designation unit 302 is shown. However, the present invention is not limited to this, and the second action acquisition unit 303 may perform processing in parallel with the first action acquisition unit 301.

Next, as a specific example, a case where the operator rotates the face as the first action and moves the hand as the second action will be described.
In this case, first, the operator stands at a position facing the display device 1 and rotates his face so as to look in a certain direction on the screen displayed on the display device 1. Then, the first action acquisition unit 301 acquires the orientation of the face, and the operation range designation unit 302 designates a part of the area on the screen as the operation range from the orientation of the face.
Next, the operator moves his / her hand so as to move the position of the cursor to the operation target. Then, the second action acquisition unit 303 acquires the movement of the hand, and the cursor position operation unit 304 converts the movement of the hand into an operation amount corresponding to the ratio of the operation range to the screen, and then the operation range. Manipulate the cursor position in.

In this way, by limiting the operation range from the entire screen to a part of the screen based on the orientation of the face, the ratio between the distance on the operation range and the distance on the movement range of the hand can be changed.
Here, as shown in FIG. 3A, it is assumed that the buttons 12 of the function menu are arranged at 100 pixel intervals (reference numeral l1) on the screen 11 displayed on the 100-inch display device 1 and operated by the cursor 13. In this screen 11, the operation range 14 is limited to a range of, for example, 300 pixels × 300 pixels from the entire screen. In this case, the movement range (reference numeral l2) of the hand required to move between the buttons 12 shown in FIG. 3B changes from, for example, the conventional 4 cm to 24 cm. By increasing the distance required to move the hand in this way, the detection error is suppressed and more detailed operation becomes possible.

Next, as another specific example, a case where the operator rotates the face as the first action and moves the line of sight as the second action will be described.
In this case, first, the operator stands at a position facing the display device 1 and rotates his face so as to look in a certain direction on the screen displayed on the display device 1. Then, the first action acquisition unit 301 acquires the orientation of the face, and the operation range designation unit 302 designates a part of the area on the screen as the operation range from the orientation of the face.
Next, the operator moves the line of sight so as to move the position of the cursor to the operation target. Then, the second action acquisition unit 303 acquires the movement of the line of sight, and the cursor position operation unit 304 converts the movement of the line of sight into an operation amount corresponding to the ratio of the operation range to the screen, and then the operation range. Manipulate the cursor position in.

By limiting the operation range from the entire screen to a part of the screen based on the orientation of the face in this way, as shown in FIG. 4, the operation range is narrowed from l3 to l4 and the orientation of the face is also the direction of the operation target. Since it faces toward, the amount of movement of the pupil (reference numeral l5) naturally decreases. Therefore, the occurrence of detection error and detection error can be suppressed, and the reliability of screen operation is improved. Further, even if a detection error occurs, the cursor does not deviate from the center of the field of view obtained from the direction of the face, and the stability of the screen operation using the movement of the line of sight is improved.

Next, as another specific example, a case where the operator rotates the face as the first action and moves the mouse as the second action will be described.
In this case, first, the operator stands at a position facing the display device 1 and rotates his face so as to look in a certain direction on the screen displayed on the display device 1. Then, the first action acquisition unit 301 acquires the orientation of the face, and the operation range designation unit 302 designates a part of the area on the screen as the operation range from the orientation of the face.
The operator then moves the mouse so that the position of the cursor is moved to the operation target. Then, the second action acquisition unit 303 acquires the movement of the mouse, and the cursor position operation unit 304 converts the movement of the mouse into an operation amount corresponding to the ratio of the operation range to the screen, and then the operation range. To operate the position of the cursor.

Here, in the operation of the cursor position using the mouse, especially when the cursor position on the screen having a large screen and high resolution is operated, the cursor may be lost or the moving distance of the cursor may be long and the work efficiency may decrease. To do. However, as in the method of the first embodiment, by limiting the operation range to a part of the screen area and then operating the cursor position, it is possible to prevent the cursor from being lost and to move the cursor. Work efficiency is improved because the amount can be reduced.

Further, as in the above three specific examples, by making the first action and the second action a movement along the time series of the human action, the operator does not distinguish between the first action and the second action. , It can operate in a series of flows, and operability is improved.

As described above, according to the first embodiment, the first action acquisition unit 301 acquires the first action, and the operation range designation unit 302 is based on the first action acquired by the first action acquisition unit 301. A part of the area on the screen is designated as the operation range, the second action acquisition unit 303 acquires the second action, and the cursor position operation unit 304 receives the second action acquired by the second action acquisition unit 303. Since it is configured to operate the cursor position in the operation range with the operation amount corresponding to the ratio of the operation range specified by the operation range designation unit 302 to the screen, the cursor position can be operated accurately. As a result, the reliability of screen operation can be improved.
Further, conventionally, only a single action is detected, and the position of the cursor is operated according to the detection result. Therefore, it is necessary to use a high-performance, high-priced, large-sized detection device. On the other hand, in the first embodiment, the detection device 2 detects two actions, narrows down the operation range, and then operates the cursor position, so that the detection device has low performance, low cost, and a small size. Even if it is 2, the reliability of the image operation can be improved.

Embodiment 2.
In the second embodiment, the case where the size of the operation range is changed after the operation range is specified is shown.
FIG. 5 is a diagram showing a configuration example of a screen operation system including the screen operation device 3 according to the second embodiment of the present invention. In the screen operation device 3 according to the second embodiment shown in FIG. 5, a third action acquisition unit 305 and a size change unit 306 are added to the screen operation device 3 according to the first embodiment shown in FIG. Other configurations are the same, and only different parts will be described with the same reference numerals.

The detection device 2 detects a third action by the operator in addition to the function in the first embodiment. Here, examples of the third action include gestures, movement of the line of sight, movement of the mouse, operation of keys or buttons using a keyboard or the like placed at the operator's hand, vocalization, and the like. The third action may be an action that can be distinguished from the first and second actions, and is predetermined. The data indicating the first to third actions detected by the detection device 2 is output to the screen operation device 3.

The third action acquisition unit 305 acquires the third action. In the example of FIG. 5, the third action acquisition unit 305 acquires the third action by receiving the data output from the detection device 2. The third action acquisition unit 305 starts processing when, for example, the operation range is designated by the operation range designation unit 302.
The size change unit 306 changes (expands or reduces) the size of the operation range designated by the operation range designation unit 302 from the third action acquired by the third action acquisition unit 305.

The cursor position operation unit 304 is designated by the operation range specification unit 302, and the cursor position is set in the operation range by the operation amount according to the ratio of the operation range changed in size by the size change unit 306 to the screen. Manipulate.

Further, in FIG. 5, the third action acquisition unit 305 acquires the third action by receiving the data output from the detection device 2. However, the present invention is not limited to this, and the detection process of the third action based on the information obtained by the input device by the detection device 2 may be performed by the third action acquisition unit 305.

Next, as a specific example, a case where the operator rotates the face as the first action, moves the face back and forth as the third action, and moves the mouse as the second action will be described.
In this case, first, the operator stands at a position facing the display device 1 and rotates his face so as to look in a certain direction on the screen displayed on the display device 1. Then, the first action acquisition unit 301 acquires the orientation of the face, and the operation range designation unit 302 sets a part of the area on the screen 11 as the operation range 14 from the orientation of the face, as shown in FIG. 6A. specify.
Next, the operator moves the face forward or backward so as to change the size of the operation range designated by the operation range designation unit 302. Then, the third action acquisition unit 305 acquires the movement of the face, and the size change unit 306 changes the size of the operation range 14 from the movement of the face, as shown in FIG. 6B. FIG. 6B shows a case where the operation range 14 is expanded by, for example, the operator moving his / her face backward.
Next, the operator moves the mouse so as to move the position of the cursor 13 to the operation target. Then, the second action acquisition unit 303 acquires the movement of the mouse, and the cursor position operation unit 304 converts the movement of the mouse into an operation amount corresponding to the ratio of the operation range 14 to the screen 11. The position of the cursor 13 is operated in the operation range 14 (FIG. 6C).

In this way, the operator can change the size of the operation range by, for example, moving his face back and forth. Thereby, for example, when the mouse is moved as the second action, the movement amount of the cursor relative to the movement amount of the mouse can be changed, and the operability can be further improved. As a result, not only is it possible to prevent the cursor from being lost, but it is also possible to operate the position of the cursor without moving the mouse away from the desk many times, which is expected to reduce wrist fatigue associated with long-term mouse operation.

Embodiment 3.
In the third embodiment, after the operation range is specified, the display magnification of the area on the screen corresponding to the operation range is changed.
FIG. 7 is a diagram showing a configuration example of a screen operation system including the screen operation device 3 according to the third embodiment of the present invention. In the screen operation device 3 according to the third embodiment shown in FIG. 7, a fourth action acquisition unit 307 and a magnification change unit 308 are added to the screen operation device 3 according to the first embodiment shown in FIG. Other configurations are the same, and only different parts will be described with the same reference numerals.

The detection device 2 detects the fourth action by the operator in addition to the function in the first embodiment. Here, examples of the fourth action include gestures, movement of the line of sight, movement of the mouse, operation of keys or buttons using a keyboard or the like placed at the operator's hand, vocalization, and the like. The fourth action may be determined in advance as long as the first and second actions are distinguishable actions. The data indicating the first, second, and fourth actions detected by the detection device 2 is output to the screen operation device 3.

The fourth action acquisition unit 307 acquires the fourth action. In the example of FIG. 7, the fourth action acquisition unit 307 acquires the fourth action by receiving the data output from the detection device 2. The fourth action acquisition unit 307 starts processing when, for example, the operation range is designated by the operation range designation unit 302.
The magnification changing unit 308 changes the display magnification of the area on the screen corresponding to the operation range designated by the operation range designation unit 302 from the fourth action acquired by the fourth action acquisition unit 307.

Further, in FIG. 7, the fourth action acquisition unit 307 is assumed to acquire the fourth action by receiving the data output from the detection device 2. However, the present invention is not limited to this, and the detection process of the fourth action based on the information obtained by the input device by the detection device 2 may be performed by the fourth action acquisition unit 307.

For example, as shown in FIG. 8, the operability is improved by increasing the display magnification of the area on the screen corresponding to the operation range 14 designated by the operation range designation unit 302 according to the fourth action by the operator. You can expect it. FIG. 8 shows a case where the area on the screen 11 corresponding to the operation range 14 is enlarged.

In the above, the case where the fourth action acquisition unit 307 and the magnification change unit 308 are added to the screen operation device 3 according to the first embodiment is shown. However, the present invention is not limited to this, and the fourth action acquisition unit 307 and the magnification change unit 308 may be added to the screen operation device 3 according to the second embodiment shown in FIG.

Embodiment 4.
In the fourth embodiment, after the operation range is specified, the operation range is moved to an arbitrary position on the screen.
FIG. 9 is a diagram showing a configuration example of a screen operation system including the screen operation device 3 according to the fourth embodiment of the present invention. In the screen operation device 3 according to the fourth embodiment shown in FIG. 9, the area moving unit 309 is added to the screen operation device 3 according to the first embodiment shown in FIG. Other configurations are the same, and only different parts will be described with the same reference numerals.

The area moving unit 309 moves the area on the screen corresponding to the operation range on the screen.
Here, for example, as shown in FIG. 10A, when the operation range 14 designated by the operation range designation unit 302 is located above the screen 11, the operator will perform the operation while looking upward, and the body. The burden is heavy. Therefore, as shown in FIG. 10B, the above burden can be reduced by moving the operation range 14 to the center or downward on the screen 11.

In the above, the case where the area moving unit 309 is added to the screen operation device 3 according to the first embodiment is shown. However, the present invention is not limited to this, and the area moving unit 309 may be added to the screen operation device 3 according to the second and third embodiments shown in FIGS. 5 and 7.

Finally, with reference to FIG. 11, a hardware configuration example of the screen operation device 3 according to the first to fourth embodiments will be described. Hereinafter, a hardware configuration example of the screen operation device 3 according to the first embodiment will be shown, but the same applies to the screen operation device 3 according to the second embodiment.
Each function of the first action acquisition unit 301, the operation range designation unit 302, the second action acquisition unit 303, and the cursor position operation unit 304 in the screen operation device 3 is realized by the processing circuit 51. As shown in FIG. 11A, the processing circuit 51 is a CPU (Central Processing Unit, central processing unit, processing unit) that executes a program stored in the memory 53, even if it is dedicated hardware, as shown in FIG. 11B. , Arithmetic logic unit, microprocessor, microprocessor, processor, DSP (also referred to as Digital Signal Processor) 52.

When the processing circuit 51 is dedicated hardware, the processing circuit 51 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate). Array) or a combination of these is applicable. The functions of the first action acquisition unit 301, the operation range designation unit 302, the second action acquisition unit 303, and the cursor position operation unit 304 may be realized by the processing circuit 51, or the functions of each unit may be collectively implemented by the processing circuit. It may be realized by 51.

When the processing circuit 51 is the CPU 52, the functions of the first action acquisition unit 301, the operation range designation unit 302, the second action acquisition unit 303, and the cursor position operation unit 304 are realized by software, firmware, or a combination of software and firmware. Will be done. Software and firmware are described as programs and stored in the memory 53. The processing circuit 51 realizes the functions of each part by reading and executing the program stored in the memory 53. That is, the screen operation device 3 includes a memory 53 for storing a program in which, for example, each step shown in FIG. 2 is eventually executed when executed by the processing circuit 51. Further, it can be said that these programs cause the computer to execute the procedures and methods of the first action acquisition unit 301, the operation range designation unit 302, the second action acquisition unit 303, and the cursor position operation unit 304. Here, the memory 53 is, for example, a non-volatile or volatile semiconductor such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Program ROM), or an EPROM (Electrically EPROM). , Magnetic discs, flexible discs, optical discs, compact discs, mini discs, DVDs (Digital Versailles Disc) and the like.

For each function of the first action acquisition unit 301, the operation range designation unit 302, the second action acquisition unit 303, and the cursor position operation unit 304, some of them are realized by dedicated hardware, and some of them are by software or firmware. It may be realized. For example, the function of the first action acquisition unit 301 is realized by the processing circuit 51 as dedicated hardware, and the processing circuit 51 of the operation range specification unit 302, the second action acquisition unit 303, and the cursor position operation unit 304 is used. The function can be realized by reading and executing the program stored in the memory 53.

As described above, the processing circuit 51 can realize each of the above-mentioned functions by hardware, software, firmware, or a combination thereof.

It should be noted that, within the scope of the invention, the present invention can be freely combined with each embodiment, modified from any component of each embodiment, or omitted from any component in each embodiment. ..

1 Display device, 2 Detection device, 3 Screen operation device, 51 Processing circuit, 52 CPU, 53 Memory, 301 1st action acquisition unit, 302 Operation range specification unit, 303 2nd action acquisition unit, 304 Cursor position operation unit, 305 3rd action acquisition unit, 306 size change unit, 307 4th action acquisition unit, 308 magnification change unit, 309 area movement unit.

Claims (6)

The first action acquisition unit acquires the first action by the operator who operates the screen, and obtains the first action.
The operation range designation unit designates a part of the area on the screen as the operation range from the first action acquired by the first action acquisition unit.
The second action acquisition unit acquires the second action by the operator and obtains the second action.
The cursor position operation unit is the amount of movement of the cursor with respect to the screen according to the ratio of the operation range specified by the operation range designation unit to the screen from the second action acquired by the second action acquisition unit. A screen operation method characterized in that the position of the cursor is operated in the operation range with an operation amount smaller than the operation amount. The operation range designation unit is characterized in that when the position on the screen corresponding to the first action is located within the threshold range for a specified time, the area including the position is designated as the operation range. The screen operation method according to claim 1. The third action acquisition unit acquires the third action by the operator and obtains the third action.
The size change unit changes the size of the operation range specified by the operation range designation unit from the third action acquired by the third action acquisition unit.
The cursor position operation unit operates the cursor position in the operation range with an operation amount corresponding to the ratio of the operation range designated by the operation range designation unit and changed in size by the size change unit to the screen. The screen operation method according to claim 1 or 2, wherein the screen operation method is described. The fourth action acquisition unit acquires the fourth action by the operator, and obtains the fourth action.
A claim that changes the display magnification of an area on the screen included in the operation range specified by the operation range designation unit from the fourth action acquired by the fourth action acquisition unit. The screen operation method according to any one of claims 1 to 3. The screen operation method according to any one of claims 1 to 4, wherein the area moving unit moves an area on the screen included in the operation range on the screen. The first action acquisition unit that acquires the first action by the operator who operates the screen,
From the first action acquired by the first action acquisition unit, an operation range designation unit that designates a part of the area on the screen as an operation range, and
The second action acquisition unit that acquires the second action by the operator, and
From the second action acquired by the second action acquisition unit , an operation smaller than the operation amount which is the movement amount of the cursor with respect to the screen according to the ratio of the operation range specified by the operation range designation unit to the screen. A screen operation device including a cursor position operation unit that operates the cursor position in the operation range by the amount.

Images