JP3605690B2 - Object selection confirmation method and storage medium storing object selection confirmation program - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a method for confirming selection of an object using a pointing device and a storage medium storing a program for confirming selection of an object.
[0002]
[Prior art]
With the spread of the Internet in recent years, the use of personal computers as communication means has increased. Usually, an input device such as a mouse or a keyboard is required to use a computer, but a special means is required for a disabled person who has difficulty using an existing input device. In particular, various devices have been developed as input means of personal computers for persons with physical disabilities, and methods to substitute the mouse operation using the remaining functions of the body, for example, obstacles that can voluntarily move the head A pointing device has been developed that allows a user to specify a button or icon on a screen using the movement.
[0003]
Pointing devices that use the movement of the head include products such as headspace pointers and head master plus that have various sensors attached to the head and have a mechanism for detecting up, down, left and right movements. There are products such as a method in which a reflective marker is attached to the head like a mouse and a marker area is extracted by image processing from a moving image captured by a CCD camera or the like.
In addition, the mounting position of the sensor or the marker in these products may be a movable position of the body in principle, and the operation is possible even if the product is mounted on a hand or a foot.
[0004]
In either method, the extracted sensor position or marker coordinate position is displayed as a pointer on a monitor of a personal computer after being appropriately coordinate-converted. Thus, an object such as an icon or a button on the screen can be indicated by the vertical or horizontal movement of the marker or the sensor.
[0005]
The application of these pointing devices to persons other than the physically handicapped is also being studied. For example, in the fields of VR (Virtual Reality) and IR (Augmented Reality), the virtual world created by the computer and the activities in the real world of humans have been studied. It is expected to be used as a pointing method to support the interaction of the Internet.
[0006]
Conventionally, as a method for determining the selection of an object such as a button or an icon specified by the pointing device, an automatic determination is performed after a lapse of a predetermined time, that is, an arbitrary object is continuously specified by the pointing device for a predetermined time. Then, a method of automatically determining the selection of the object or a method of confirming the selection using an external switch such as an exhalation switch or a push button has been used.
[0007]
[Problems to be solved by the invention]
However, although the method based on the automatic determination described above can be realized relatively easily without requiring a special operation from the operator by using a timer or the like, the input is determined unconditionally in a fixed time by the automatic determination. As a result, there may be a sense of incongruity that the computer operates against the will. In addition, in the method using an external switch, it is necessary to mount and fix the switch in accordance with the movable position and the movable range of the body each time it is used, especially when using a physically handicapped person, the burden on the operator and the caregiver. Become.
[0008]
Accordingly, the present invention has been made to solve the above-described problem, and realizes the selection of an object by a pointing device with a relatively simple configuration, and easily transmits its own intention to a computer. The purpose is to:
[0009]
[Means for Solving the Problems]
In order to solve the above-mentioned problem, the present inventors have conducted intensive research, input the movement trajectory of the pointer as coordinates on a two-dimensional plane, and selected an object when the movement trajectory matched a predetermined movement pattern. The present invention has been completed by paying attention to the fact that self-will can be reliably transmitted to a computer by being determined. That is, according to the object selection method according to claim 1 of the present invention which solves the above problem, a pointer which can indicate an arbitrary object by a pointer from one or more objects displayed on a plane or a space.IThe information processing apparatus uses an operating device to select and confirm the object pointed by the pointer. In the object selection determination method, the movement trajectory of the pointer is input as a position on a two-dimensional plane, and when the pattern of the movement trajectory matches a predetermined pattern, the selection of the designated object is determined. It is characterized by.
[0010]
According to this configuration, the operation using the pointing device is only to move the pointer in a predetermined pattern, and the object is selected and confirmed (selection is confirmed) only by the movement of the pointer.
[0011]
The plane or space is, for example, a computer screen or a virtual reality space. An object is, for example, an icon or button on a computer screen, a menu displayed on a menu bar or the like, or an object to be operated or processed (copy, cut, paste, delete, transform, drag and drop, etc.) on the computer. And figures.
Incidentally, the information processing apparatus in which the object selection and determination method of the present invention is implemented is, in an embodiment of the invention described later, configured integrally with a personal computer in which document creation software (word processing software), spreadsheet software, and the like are incorporated. You. The information processing apparatus is not limited to a personal computer, and may be integrally formed with an information-related device, an amusement device, and the like, such as a PDA (Personal Data Assistants) and a game machine.
[0012]
Also, OhHow to select the object,PreviousDisplaying an auxiliary symbol for the object according to the movement trajectory;DebaEncourage selection by chairMay be.
[0013]
According to this configuration, selection of an object is facilitated by displaying the auxiliary symbol. The auxiliary symbols are used for navigation (operation instruction guidance) of the operation of the pointing device so that the movement locus of the pointer becomes a predetermined pattern.
[0014]
Also, OhHow to select the object,PreviousWhen the pointer passes through the object and a departure angle between the approach direction and the departure direction of the pointer with respect to the object takes a value in a specific range, it is determined that the movement trajectory matches the predetermined pattern.May be.
[0015]
The object has a predetermined area. In this configuration, the selection of the object is determined by using the departure angle between the approach direction and departure direction of the pointer with respect to the object area. The use of an angle in this way is based on the finding that the departure angle when the pointer simply passes through the object and the departure angle when the object is selected by the pointer are clearly different. In an embodiment of the invention to be described later, when the departure angle is 90 ° or less, the selection of the object is determined as having the intention of selection.
[0016]
Also,Claim 1How to confirm the selection of objects,furtherA small area for input confirmation is displayed at a specific position based on the entry position of the pointer with respect to the object, and when the movement path when the pointer leaves the object passes through the small area, the movement path It is characterized in that it is determined that the pattern matches the predetermined pattern.
[0017]
When the operator of the pointer selects an object, the pointer enters the object (object area) and then leaves. On the other hand, even when the object is not selected, the pointer sometimes enters the object and then leaves the object. This claim1According to the configuration, when the pointer enters the object, a small area for input confirmation is displayed at a specific position based on the entry position. The pointer operator operates the pointer using the small area for input determination as a mark. When the pointer leaves the object and passes through the small area for input determination, the selection of the object is determined. On the other hand, when the pointer leaves the object and does not pass through the small area for input determination, the selection of the object is not determined. The input confirmation small area has a role of a guide for confirming the selection of the object. It goes without saying that not passing through the small area for input determination may be used as the object selection determination condition.
[0018]
Also, OhHow to select the object, OverIf there is an intersection in the movement locus of the pointer in the last fixed time, and if at least a part of the object is included in the loop of the movement locus having the intersection as a start point and an end point, the movement locus becomes the predetermined value. Is determined to match the patternMay be.
[0019]
In general, when pointing to something, the action of drawing a circle so as to surround the object or drawing a ring so as to partially cover the object is improved.
According to this configuration, the selection of the object is determined by the movement of the pointer based on the general operation.
[0020]
Claims2The method for confirming the selection of an object described inOneIn the configuration, the movement trajectory of the pointer in the past fixed timeRead from storage meansIt is characterized by displaying.
[0021]
By displaying past movements (movement trajectories) of the pointer in a series of object selection and confirmation operations, object selection by the pointer can be easily confirmed.
[0022]
Claims3The storage medium storing the object selection determination program according to claim 1.OrClaim2And a computer-readable storage medium storing a program for executing the object selection determination method.
[0023]
Here, the object selection determination program stored in the storage medium is as follows.
(1) Using a pointing device capable of pointing a pointer to an arbitrary object from one or more objects displayed on a plane or a space, executing a method for the information processing apparatus to select and confirm the object pointed to by the pointer. The computer as the information processing apparatus inputs the movement trajectory of the pointer as coordinates on a two-dimensional plane, and determines the selection of the designated object when the movement trajectory pattern matches a predetermined pattern. Is a program that functions as follows.
[0026]
(1 ') In addition, the object selection and determination program includes the (1))ofIn the configuration, a small area for input determination is displayed at a specific position based on the entry position of the pointer with respect to the object, and when a movement trajectory when the pointer leaves the object passes through the small area, It is designed to function so as to determine that the movement trajectory matches the predetermined pattern.
[0028]
(2Further, the object selection and determination program causes the computer to display the movement locus of the pointer at a certain time in the past.And (1 ')Is added to the configuration of FIG.
[0029]
Note that the object selection determination program is distributed in a form stored in a storage medium such as a CD-ROM, and is also distributed in a form downloaded from a WWW (World Wide Web) server on the Internet.
[0030]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Note that the embodiments of the present invention will be described separately from the first embodiment to the fourth embodiment below, due to the difference in the mode of the object selection determination method.
[0031]
<< 1st Embodiment >>
A method for determining the selection of an object according to the first embodiment will be described.
In the first embodiment, the selection of an object is determined based on the departure angle between the approach direction and the departure direction of the pointer with respect to the object.
Here, FIG. 1 is a block diagram of an information processing apparatus in which the object selection and determination method according to the first embodiment is performed. FIG. 2 is a schematic diagram of pointing an object on a screen of a display device with a pointer.
[0032]
An information processing apparatus M in which the object selection and determination method of the first embodiment is performed will be described.
As shown in FIG. 1, the information processing apparatus M has a configuration in which a main controller 1, a storage device 2, a display device 3, a pointing device 4, and the like are connected to a bus (Bass) 5 via a controller (not shown) or the like. Having. This configuration is the same as a general personal computer.
[0033]
The main control device 1 includes an MPU (Micro Processing Unit), a RAM (Random Access Memory), and the like, and controls the information management device M as a whole. For this reason, in the main control device 1, a position information input unit, a position information storage unit, a movement trajectory pattern extraction unit, a pattern matching unit, and an OS (Operating System) as an object selection determination program have been activated. The main controller 1 also operates application programs such as word processing software. The OS has a function of monitoring the position information of the pointer and the position information of the object.
[0034]
The storage device 2 includes a hard disk device and the like. The storage device 2 stores an object selection determination program, an OS, an application program, and the like. The display device 3 includes a liquid crystal monitor or the like, and displays a processing result in the main control device 1 including a movement of the pointer.
[0035]
The pointing device 4 is an input device for selecting (instructing) an object displayed on the display device 3 with a pointer. As the pointing device 4, a device for directly inputting coordinates, such as a mouse or a tablet, can be used. In addition, in the case where the output of the sensor responding to the movement is appropriately converted and displayed on the screen, or in the case where an image input device such as a CCD camera is used, a specific color region is extracted from a continuous image input. A device whose center coordinates are sequentially obtained and displayed on a screen can be used as the pointing device 4.
[0036]
The pointer and the object will be described.
As shown in FIG. 2, on the screen of the display device 3 (see FIG. 1), as objects O, an icon object Oi, a menu object Om, and a button object Ob are arranged. The menu object Om and the button object Ob are displayed in the application program window W.
A pointer P is displayed on the screen of the display device 3. The pointer P moves freely on the screen in conjunction with the movement of the pointing device 4 (see FIG. 1), and indicates the object O (Oi, Om, Ob).
As described above, the positions of the object O and the pointer P are monitored by the OS. Further, the position information input means of the object selection determination program acquires (inputs) the position monitored by the OS as position information.
[0037]
Here, the function of the object selection confirmation program will be described (see FIGS. 1 and 2).
The position information input means inputs (acquires) the position information (pointer position information and object position information) of the pointer P and the object O monitored by the OS from the OS at predetermined time intervals.
[0038]
The position information storage means stores pointer position information corresponding to the movement of the pointer P out of the input position information in a storage area of a RAM constituting the main controller 1 by a predetermined data capacity. The stored pointer position information corresponds to the position history. The object position information is also stored in the storage area of the RAM.
[0039]
When the pointer P passes through the object O, the movement trajectory pattern extracting means obtains a departure angle θ between the approach direction and the departure direction of the pointer P with respect to the object O by calculation (formulas used in the calculation will be described later). That is, the movement trajectory pattern extraction means extracts the feature of the pattern of the movement trajectory of the pointer P as the separation angle θ.
Further, the movement trajectory pattern extraction means monitors whether or not the pointer P has entered the object O, and turns the entry flag from OFF to ON when the pointer P has entered the object O. On the other hand, when the pointer P has left the object O, the entry flag is changed from ON to OFF. This entry flag is also a flag indicating whether or not the pointer P exists on the object O.
[0040]
The pattern matching means determines whether the calculated departure angle θ is smaller than a predetermined specific angle (angle threshold φ), that is, whether the departure angle θ takes a value in a specific range. . The comparison between the separation angle θ and the angle threshold φ is performed as an absolute value. Then, when the detachment angle θ ≦ the angle threshold φ, it is determined that the selection of the object O is determined. When determining that the selection has been confirmed, the pattern matching means outputs the selection confirmation information to the OS or an application program as required.
[0041]
Here, the approach direction, the leaving direction, and the leaving angle θ of the pointer P with respect to the object O will be described. FIGS. 3A and 3B are schematic diagrams for explaining an object selection deciding operation. FIG. 3A shows a movement locus of a pointer with respect to an object, and FIG. 3B shows a principle of calculating a detachment angle θ.
[0042]
The pointer P moves with time according to the operation of the operator of the pointing device 4 and draws a movement locus. The pointer P shown in FIG. 3A draws a moving trajectory that enters the object O from the left side, changes its direction substantially at the center of the object O, and then leaves the left side of the object O (the moving trajectory is a position). The history is stored in the storage area of the RAM). In this figure, P (t) is pointer position information immediately after the pointer P enters the object O. P (t−1) is pointer position information input immediately before P (t), that is, pointer position information immediately before the pointer P enters the object O. Q (s) is pointer position information immediately after the pointer P leaves the object O. Q (s-1) is pointer position information input immediately before Q (s), that is, pointer position information immediately before the pointer P leaves the object O.
[0043]
The detachment angle θ is an angle between the approach direction of the pointer P with respect to the object O and the detachment direction. That is, in FIG. 3B, the angle between the approach direction vector and the release direction vector is the release angle θ. The departure angle θ represents the coordinates in the pointer position information P (t), P (t-1), Q (S), and Q (s-1) as (x1, y1), (x2, y2), Assuming that (x3, y3) and (x4, y4), it can be obtained by the following formula.
[0044]
θ = cos^-1A ... (1)
However,
a1 = (x2-x1), a2 = (x4-x3)
b1 = (y2-y1), b2 = (y4-y3)
A = (a1 · a2 + b1 · b2) / [(a1²+ B1²) (A2²+ B2²)]^1/2
[0045]
Next, the operation in the object selection and determination method of the first embodiment will be described with reference to the processing flowchart of FIG. 4 (see FIGS. 1 to 3 as appropriate).
FIG. 4 is a processing flowchart in the object selection and determination method according to the first embodiment.
Note that the position of the object O does not change. The processing flowchart of FIG. 4 is continuously executed at a constant period while the pointer P is on the screen of the display device 3.
[0046]
First, as an initial setting, an entry flag indicating that the pointer P has entered (or reached) the object O is set to OFF (S101). Next, the position information input means inputs pointer position information from the OS (S102). If the pointer position information is not input, the object selection determination program ends the processing on the assumption that the pointer P does not exist on the screen of the display device 3 (S103). On the other hand, when the pointer position information is input, the position information storage means stores the pointer position information (S104).
[0047]
Next, branching is performed according to the sign of the entry flag (S105). If the entry flag is OFF, the process proceeds to step S106, and if the entry flag is ON, the process proceeds to step S110.
[0048]
Hereinafter, a process when the entry flag is OFF in step S105 will be described (process on the entry side).
The movement trajectory pattern extracting means determines whether or not the object O exists at the position where the pointer P is currently located, in other words, whether or not the pointer P has entered the object O from the input pointer position information (S106). It should be noted that the object position information is known in advance by the movement trajectory pattern extraction means.
[0049]
If it is determined in step S106 that the pointer P has not entered the object O (NO), the entry flag remains OFF and the process returns to step S102. That is, while the entry flag is OFF, the operator has not yet pointed the object O on the screen with the pointer P. During this time, the loop from step S102 to step S106 is repeatedly executed. Note that the operator operates the pointer P by operating the pointing device 4.
[0050]
On the other hand, if it is determined in step S106 that the object O exists (YES), the entry flag is set to ON since the pointer P has entered the object O (S107). The sign of the entry flag is switched by the movement trajectory pattern extracting means.
[0051]
The movement trajectory pattern extraction means stores the time t when the pointer P enters the object O (S108). Further, the movement trajectory pattern extraction means stores the pointer position information at that time t as P (t), and stores the pointer position information immediately before that as P (t-1) (S109).
After that, the process returns to step S102. That is, the operation waits until the pointer P separates from the object O by the operation of the pointing device 4.
[0052]
The above is the processing when the entry flag is OFF in step S105 (processing when the pointer P enters the object O).
Subsequently, the processing when the entry flag is ON in step S105 will be described below (processing when the pointer P leaves the object O).
[0053]
The movement trajectory pattern extraction means determines whether or not the object O exists at the position where the pointer P is currently located, in other words, whether or not the pointer P has left the object O from the input pointer position information (S110). If it is determined that the object O exists, the process returns to step S102 to wait for the pointer P to leave the object O.
[0054]
If it is determined in step S110 that the object O does not exist at the position where the pointer P is currently located, it is determined that the pointer P has left the object O, and the entry flag is set to OFF (S111).
The movement trajectory pattern extraction means stores the time s when the pointer P has left the object O (S112).
[0055]
The moving trajectory pattern extracting means compares the difference (s−t) between the time s when the pointer P leaves the object and the time t when the pointer P enters the object O with a preset time threshold T. (S113).
[0056]
If T ≦ s−t is not satisfied in step S113, that is, if the difference (s−t) is shorter than the time threshold T, it is considered that the pointer P has simply passed over the object O. It returns to the process of S101. The fact that the pointer P passes through the object O in a short time is because the operator of the pointing device 4 can be regarded as having no intention to select the object O involved in the passing.
[0057]
On the other hand, if T ≦ s−t in step S113, that is, if the difference (s−t) is longer than the time threshold T, the pointer P does not simply pass over the object O. , The pointer position information at the time s at which the pointer departed is stored as Q (s), and the pointer position information immediately before the time s is stored as Q (s-1) (S114).
[0058]
The pattern matching means includes pointer position information P (t), P (t-1) before and after the pointer P enters the object O, and coordinates Q (s), Q (s) before and after the pointer P leaves the object O. -1), the departure angle θ (θ ≦ π) formed by the vector in the approach direction and the vector in the departure direction of the pointer is calculated by the above equation (1) (S115).
[0059]
The pattern matching means compares the separation angle θ with a predefined angle threshold φ (S116). The value of the angle threshold φ may be set to a small value (for example, 90 °> φ) in order to avoid erroneous recognition as a mere passing. When θ> φ, it is assumed that the pointer P simply passes over the object O. Considering and returning to step S101. On the other hand, when θ ≦ φ, the selection of the object O is determined, and the identification information of the object O indicated by the pointer P is output to the information processing device (S117). Note that the identification information of the object O includes information about which object on the screen has been selected. When the selection is confirmed, for example, a home page or the like linked to the object O is displayed.
Then, the process returns from step S117 to step S101 to continue the process.
[0060]
Incidentally, the movement trajectory of the pointer P indicated by the imaginary line in FIG. 3 is a movement trajectory in the case of simple passing. In this case, the departure angle θ between the approach direction and departure direction of the pointer P with respect to the object O is larger than the angle threshold φ of 90 °.
[0061]
Note that the angle threshold value φ is determined based on experiments and logical calculations. However, when the angle threshold value φ is reduced, there are many situations in which even if there is an intention to select the object O, it is regarded as a mere passage. On the other hand, when the angle threshold φ is increased, the situation in which the selection of the object O is determined for the operation of simply passing the pointer P increases.
Here, if the angle threshold φ is 90 °, the selection of the object O can be appropriately determined. That is, the angle threshold φ of 90 ° means that the pointer P has turned on the object O. Normally, it is rare that the pointer P changes direction on the object O just passing through the object O. Therefore, if the angle threshold φ is set to 90 °, it is possible to reliably determine whether or not the pointer P has changed direction, and whether or not the operator has an intention to select and confirm the object O.
[0062]
The time threshold value T is also determined based on experiments and logical calculations. However, if the time threshold value T is increased, there are many situations in which the object O is determined to be a mere passage even if there is an intention to select the object O. On the other hand, when the time threshold T is reduced, the situation in which the selection of the object O is determined in response to the operation of simply passing the pointer P increases (this point is the same in the following second embodiment). .
[0063]
Therefore, according to the object selection and determination method of the first embodiment, the operator's intention is reliably transmitted to the computer (information management device) only by a simple operation of turning the pointer on the object. be able to. In this case, a switch such as an external switch for confirming the selection is unnecessary.
[0064]
<< 2nd Embodiment >>
Next, a method for determining the selection of an object according to the second embodiment will be described.
In the second embodiment, an input determination small area is displayed at a specific position based on the entry position of the pointer, and whether the movement trajectory when the pointer leaves the object is displayed through the input determination small area is determined. Whether or not the object is selected is determined based on whether or not the object is selected.
In addition, about the structure which has the same as 1st Embodiment, the same code | symbol is attached | subjected with reference to the drawing of 1st Embodiment, and the description is abbreviate | omitted.
[0065]
The hardware configuration of the information processing apparatus M (see FIG. 1) in which the object selection and determination method of the second embodiment is performed is the same as that of the first embodiment. Therefore, the description is omitted. However, as a software configuration, the object selection determination program has an input determination small area display unit.
Hereinafter, the points different from the first embodiment will be mainly described.
[0066]
The input confirmation small area display means will be described with reference to FIGS.
FIG. 5 is a schematic diagram for explaining the selection operation of the object and the display position of the input confirmation small area. FIG. 6 is a schematic diagram for explaining the selection and confirmation operation of an object different from that in FIG.
[0067]
The trajectory of the pointer (see FIG. 2) shown in FIG. 5 enters the object O from the left (the trajectory is stored in the storage area of the RAM as a position history). In this figure, P (t) is pointer position information immediately after the pointer P enters the object O. P (t−1) is pointer position information input immediately before P (t), that is, pointer position information immediately before the pointer P enters the object O. Reference symbol C is the center point of the object O. Symbol B (B1, B2) is an input confirmation small area.
[0068]
The display position of the input confirmation small area B is determined as follows.
The position of the intersection Q at which the line segment (moving locus) formed by the pointer position information P (t) and P (t-1) intersects the outer periphery of the object O is obtained. Next, a virtual line segment QC connecting the intersection Q and the center point C is assumed. A virtual straight line extending from the center point C at a predetermined display angle α (positive value) with respect to the virtual line segment QC is assumed (a straight line extending to the upper left in FIG. 5). An intersection Q1 between the virtual straight line and the contour of the object O is determined. Then, an input determination small area B (B1) having a predetermined size is displayed adjacent to the outside of the object O based on the intersection Q1. Further, a virtual straight line extending from the center point C at a predetermined display angle α (negative value) with respect to the virtual line segment QC is assumed (a straight line extending to the lower left in FIG. 5). An intersection Q2 between the virtual straight line and the contour of the object O is obtained. Then, an input determination small area B (B2) having a predetermined size is displayed adjacent to the outside of the object O on the basis of the intersection Q2.
[0069]
Incidentally, when the display angle α is set to 180 °, the input determination small area B is displayed in a direction substantially straight with respect to the approach direction of the pointer P to the object O. In this case, if the pointer P is made to pass straight through the object O and leave, the object O is selected (the operability is good). However, it is assumed that it is difficult to determine whether the pointer P has simply passed through the object O or whether the pointer P has passed in order to select and confirm the object O. Therefore, when the display angle α is set to a value close to a right angle (a value equal to or less than the right angle) as in the second embodiment, it is determined whether the pointer P has simply passed the object O or passed to attempt to select and confirm the object O Becomes easier. That is, it is possible to reliably capture the intention of the operator.
[0070]
Note that the display position of the input confirmation small area B may be determined as follows (see FIG. 6).
In this modification, eight display position candidates of the input confirmation small area are set in advance adjacent to the outside of the object O. In FIG. 6, two of the selected areas are selected to be input determination small areas B (B1, B2). Then, the remaining six remain unselected candidates B '. In this modification, when the intersections Q1 and Q2 are obtained, the candidate B 'closest to the intersections is selected and the input confirmation small area is displayed. The method of obtaining the intersections Q, Q1, and Q2 is the same as the method described with reference to FIG. 5, and a description thereof will be omitted.
[0071]
Incidentally, the size of the input confirmation small area B is appropriately set in consideration of operability and the like. Also, the number of the input confirmation small areas B is appropriately set in consideration of the operability and the like (at least one exists). Further, the input confirmation small area B may be displayed separately from the object O. Alternatively, the input confirmation small area B may be displayed inside the object O.
[0072]
Next, the operation of the object selection and determination method according to the second embodiment will be described with reference to the processing flowchart of FIG. 7 (see FIGS. 1, 2 and 5 as appropriate).
FIG. 7 is a processing flowchart in the object selection and determination method according to the second embodiment.
Note that the position of the object O does not change. The process flowchart of FIG. 7 is continuously executed at a constant period while the pointer P is on the screen of the display device 3.
[0073]
First, as an initial setting, an entry flag indicating that the pointer P has entered (or reached) the object O is set to OFF (S201). Next, the position information input means inputs pointer position information from the OS (S202). When the pointer position information is not input, the object selection determination program ends the processing on the assumption that the pointer P does not exist on the screen of the display device 3 (S203). On the other hand, when the pointer position information is input, the position information storage means stores the pointer position information (S204).
[0074]
Next, branching is performed according to the sign of the entry flag (S205). If the entry flag is OFF, the process proceeds to step S206, and if the entry flag is ON, the process proceeds to step S210.
[0075]
In the following, a description will be given of a process when the entry flag is OFF in step S205.
The movement trajectory pattern extracting means determines whether or not the object O exists at the position where the pointer P is currently located, in other words, whether or not the pointer P has entered the object O, from the input pointer position information (S206). It should be noted that the object position information is known in advance by the movement trajectory pattern extraction means.
[0076]
If it is determined in step S206 that the pointer P has not entered the object O (NO), the entry flag remains OFF and the process returns to step S202. That is, while the entry flag is OFF, the operator has not yet pointed the object O on the screen with the pointer P. During this time, the loop from step S202 to step S206 is repeatedly executed. Note that the operator operates the pointer P by operating the pointing device 4.
[0077]
On the other hand, if it is determined in step S206 that the object O exists (YES), the entry flag is set to ON because the pointer P has entered the object O (S207). The sign of the entry flag is switched by the movement trajectory pattern extracting means.
[0078]
The movement trajectory pattern extraction means stores the time t when the pointer P enters the object O (S208). Further, the movement trajectory pattern extraction means stores the pointer position information at that time t as P (t), and stores the pointer position information immediately before that as P (t-1) (S209).
Then, the process returns to step S202.
[0079]
The above is the processing when the entry flag is OFF in step S205 (processing when the pointer P enters the object O).
Subsequently, the processing when the entry flag is ON in step S205 will be described below (processing when the pointer P leaves the object O).
[0080]
The input determination small area display means obtains intersections Q1 and Q2 so that the input determination small area B is displayed adjacent to the object O. This method is as described with reference to FIG. Then, based on the intersections Q1 and Q2, the input confirmation small area display means displays the candidate B 'closest to the intersection Q1 as the input confirmation small area B1 adjacent to the outside of the object O. Further, the candidate B 'closest to the intersection Q2 is displayed adjacent to the outside of the object O as the input determination small area B2.
[0081]
In the next step S212, the movement trajectory pattern extraction means determines whether or not the pointer P has departed from the object O (determines whether or not the object O exists at the position where the pointer P is currently located). If the pointer P has not departed from the object O, the process returns to step S202 while keeping the departure flag ON to wait for the pointer P to depart from the object O. That is, the information processing apparatus M waits for the pointer P to be separated from the object O by the operation of the pointing device 4. A flag that is turned ON when the input confirmation small area B is displayed may be provided, and when this flag is ON, steps S210 and S211 may be skipped.
By the way, if the operator of the pointing device 4 has an intention to select and confirm the object O indicated by the pointer P, the operator operates the pointer P so as to pass through the input confirmation small area B (B1 or B2). .
[0082]
On the other hand, if it is determined in step S212 that the pointer P has departed from the object O, the movement trajectory pattern extraction unit turns off the entry flag (S213). Then, the time s at which the pointer P has left the object O is stored (S214).
[0083]
Next, in step S215, the movement trajectory pattern extraction unit determines whether or not the pointer P has passed the input determination small area B (B1 or B2). As specific determination means, for example, pointer position information (coordinates) before and after the pointer P departs from the object O is held, and a part of a line connecting the pointer position information is a small area for input determination. It is determined whether or not it has the coordinates of B.
[0084]
If it is not determined in step S215 that the pointer P (movement trajectory) has passed through the input confirmation small area B (NO), the process proceeds to step S218 to display the displayed input confirmation small area B (B1 and B2). to erase. This is because the continuous display of the input confirmation small area B causes confusion for the operator of the pointing device 4. If a flag that turns on when the input confirmation small area B is displayed is provided, this flag is turned off. Then, the process returns to step S201.
Incidentally, the movement trajectory of the pointer P indicated by the imaginary line in FIGS. 5 and 6 does not pass through the input confirmation small area B (B1 or B2), but merely passes through the object O of the pointer P (merely Pass).
[0085]
On the other hand, when it is determined in step S215 that the pointer P has passed the small area B for input determination (YES), the following processing is performed.
That is, in step S216, the pattern matching unit compares the difference (s−t) between the time S when the pointer P departs from the object O and the time t when the pointer P enters the object O with a preset time threshold T. . If T ≦ s−t is not satisfied (NO), it is considered that the pointer P has simply passed through the object O, and the process proceeds to step S218 to delete the input confirmation small area B (B1 and B2). Then, the process returns to step S201. The fact that the pointer P passes through the object O in a short time is because the operator of the pointing device 4 can be regarded as having no intention to select the object O involved in the passing.
[0086]
Conversely, if T ≦ s−t (YES), the selection of the object O is determined, and the identification information of the object O indicated by the pointer P is output to the information processing device (S217). Note that the identification information of the object O includes information indicating which object O on the screen has been selected. When the selection is confirmed, for example, a home page or the like linked to the object O is displayed.
Then, the process returns from step S217 to step S201 via step S218, and continues the processing.
[0087]
Therefore, according to the object selection and determination method of the second embodiment, navigation is performed to the input determination small area, so that the intention of the operator can be more reliably transmitted to the computer (information management device). In this case, a switch such as an external switch for confirming the selection is unnecessary.
[0088]
<< 3rd Embodiment >>
Next, a method for determining the selection of an object according to the third embodiment will be described.
In the third embodiment, the determination of the selection of an object is determined based on whether or not the object exists in the ring drawn by the movement locus of the pointer.
In addition, about the structure which has the same as 1st Embodiment, the same code | symbol is attached | subjected with reference to the drawing of 1st Embodiment, and the description is abbreviate | omitted.
[0089]
The hardware configuration of the information processing apparatus M (see FIG. 1) in which the object selection and determination method of the third embodiment is performed is the same as that of the first embodiment. Therefore, the description is omitted.
Hereinafter, the points different from the first embodiment will be mainly described (see FIGS. 1 and 2 as appropriate).
[0090]
In the object selection determination program of the third embodiment, the position information storage means configures the main control device 1 with the pointer position information corresponding to the movement of the pointer P among the position information input by the position information input means. The data is stored in a FIFO (First In Firsts Out) constructed in a storage area of the RAM. The object position information is also stored in the storage area of the RAM.
[0091]
The FIFO is a first-in first-out buffer. It is assumed that N pieces of pointer position information are stored in the FIFO. When new pointer position information P0 is stored in the FIFO, the oldest pointer position information PN is deleted, and the other pointer position information uses its index as its index. Incremented by one. In other words, the pointer position information that was previously P0 is changed to P1, the one that was P1, the index is changed to P2, and the one that was Pn-1 is changed to Pn, and the pointer position information over a certain period of time in the past is sequentially updated. (0 ≦ n <N).
[0092]
Since the holding period of the pointer position information depends on the size N of the FIFO and the input period of the pointer position information, a specific value of N can be set according to the exercise ability and preference of the operator. If the value of N is set to be large or the input period of the pointer position information is set to be long, that is, if the holding period of the pointer position information is set to be long, the selection of the object O is determined even if the operation of moving the pointer P is slow. Becomes possible.
[0093]
The moving trajectory pattern extracting means and the pattern matching means will be described with reference to FIG.
FIG. 8A is a schematic diagram for explaining an object selection deciding operation, and FIG. 8B is a diagram for explaining an operation of giving a predetermined label to a pixel.
[0094]
The moving trajectory pattern extracting means determines whether or not the moving trajectories of the pointer P intersect (whether or not the moving trajectory draws a ring). For this reason, the movement trajectory pattern extraction means extracts continuous pointer position information Pn and Pn-1 from the FIFO and sets them as line segments Tn (0 ≦ n <N). Based on the latest line segment T0 of the line segments Tn, it is determined whether or not the line segment T0 intersects with the line segment Tk in the past movement locus (1 ≦ k <N).
Incidentally, in the case of intersection, the movement locus of the pointer P draws a ring as shown in FIG. In FIG. 8, the pointer position information P0, P1 corresponds to the line segment T0, the pointer position information P1, P2 corresponds to the line segment T1, the pointer position information P2, P3 corresponds to the line segment T2,... The pointer position information P7, P8. Corresponds to the line segment T8 (the smaller the index number, the newer the data).
[0095]
When it is determined that the movement trajectory pattern extraction means intersects, the pattern matching means determines whether or not any object O exists in the wheel formed by the movement trajectory of the pointer P. For this reason, the pattern matching means assigns a predetermined label to all the pixels (pixels) in the region included in the ring (see FIG. 8B). On the other hand, the pattern matching means determines whether all of the objects O, O,... Displayed on the screen have at least a part of the area of the object O to which the label is provided. Is detected. When the object O having at least a part of the label is detected, it is determined that the object O exists in the ring.
[0096]
It should be noted that in order to make a more accurate determination, it is sufficient to detect whether or not the above-described label has been given to the center of the object O. Alternatively, it may be detected whether or not the label is given to the entire area (or 50% or more [threshold]) of one object O.
[0097]
Next, the operation of the object selection and determination method according to the third embodiment will be described with reference to the processing flowchart of FIG. 9 (see FIGS. 1, 2 and 8 as appropriate).
FIG. 9 is a processing flowchart in the object selection and determination method according to the third embodiment.
Note that the position of the object O does not change. The processing flowchart of FIG. 9 is continuously executed at a constant period while the pointer P is on the screen of the display device 3.
[0098]
First, the position information input means inputs pointer position information from the OS (S301). The latest pointer position information is P0, which indicates the current position of the pointer P. When the pointer position information is not input, the object selection determination program terminates the process on the assumption that the pointer P does not exist on the screen of the display device 3 (S302).
[0099]
On the other hand, when the pointer position information is input, the position information storage means stores the input latest pointer position information P0 in the FIFO (S303). When the FIFO is already full (when N pieces of data are stored), the oldest stored data is deleted and new data is stored. At this time, the index is rewritten to P1 for the pointer position information that was P0 before, and to P2 for P1 that was previously P1. Further, even when the pointer is not full, the index of the pointer position information is sequentially rewritten so that first-in first-out can be performed.
[0100]
The position information storage unit determines whether or not N pieces of pointer position information are stored in the FIFO (S304). If the N pieces of pointer position information are not stored (NO), the process returns to step S301.
[0101]
In the following, a description will be given of a process in a case where N pieces of pointer position information are stored in the FIFO in step S304 (YES).
[0102]
The moving trajectory pattern extracting unit sequentially extracts sets Pn and Pn−1 (1 ≦ n ≦ N) of the adjacent pointer position information from the FIFO and sets them as line segments Tn (0 ≦ n <n) (S305). This is a linear approximation of the movement trajectory of the pointer P composed of the pointer position information from the current time to the past N times at a certain time. The line segment Tn is stored in a line segment storing FIFO.
[0103]
As a preparation for performing the next process, the counter k is initialized to 1 (S306).
[0104]
The moving trajectory pattern extracting means extracts one line segment Tk (1 ≦ k <N) from the FIFO (S307). Based on the line segment T0 corresponding to the latest pointer position information P0, it is determined whether there is an intersection between the line segment T0 and the line segment Tk (1 ≦ k <N) (S308). Specifically, an intersection between a straight line including the line segment T0 and a straight line including the line segment Tk (1 ≦ k <N) is determined, and it is determined whether the intersection exists on the line segment T0. If there is no intersection, it is determined that the line segment T0 does not intersect with the line segment Tk, and the process proceeds to step S309. If there is an intersection, the process proceeds to step S311 assuming that the line segment T0 and the line segment Tk intersect.
[0105]
If it does not intersect with the line segment T0, the counter k is incremented by 1 (S309). The value of the counter k is checked to determine whether or not the value has become N (S310). If k = N, there is no line segment Tk (1 ≦ k <N) that intersects with the line segment T0, so the process returns to step S301. On the other hand, when k is not equal to N, that is, when k is less than N, the processing of steps S307 to S309 is continuously performed for all the line segments Tk (1 ≦ k <N). That is, it is determined whether or not the line segment Tk crosses the reference line segment T0 sequentially going back in the past.
[0106]
Hereinafter, a case where it is determined in step S308 that the line segments T0 and Tk intersect will be described.
The moving trajectory pattern extracting means records a line segment Tk (k <N) having an intersection with the line segment T0. (S311).
[0107]
The pattern matching means determines whether any object O exists in a polygonal region (in a ring) surrounded by a plurality of line segments starting from the line segment T0 and ending at Tk obtained in step S311. Is determined (S312). The determination method is as described above (see FIG. 8).
[0108]
As a result of the determination, if the object O exists in the area (in the ring) surrounded by the line segments T0 to Tk, the selection of the object is determined, and the identification information of the object pointed by the pointer P is sent to the information processing apparatus. It outputs (S313). Note that the identification information of the object O includes information indicating which object O on the screen has been selected. When the selection is confirmed, for example, a home page or the like linked to the object O is displayed.
Then, the process returns to step S301.
[0109]
If the result of determination in step S312 is that there is no object in the area surrounded by the line segments T0 to Tk, the flow directly returns to step S301.
[0110]
Therefore, according to the method for determining and selecting an object according to the third embodiment, by generally operating the pointing device based on a familiar operation when pointing to an object, the intention of the operator can be reliably determined by the computer (information management device). ). In this case, a switch such as an external switch for confirming the selection is unnecessary.
[0111]
If there are a plurality of objects in the ring, it may be determined that only the object having a larger area included in the ring is selected and determined. Alternatively, it may be determined that all have been selected and determined. This is because there is a case where it is desired to simultaneously select a plurality of objects and activate them. Also, there is a case where a plurality of objects are to be selected and erased at the same time.
[0112]
<< 4th Embodiment >>
Next, a method for determining the selection of an object according to the fourth embodiment will be described.
In the fourth embodiment, the moving trajectory of the pointer is displayed on the screen (drawing) according to the third embodiment in which the determination of the selection of the object is determined based on whether or not the object is present in the ring drawn by the moving trajectory of the pointer. The configuration shown in FIG.
In addition, about the structure which has the same as 1st Embodiment and 3rd Embodiment, it refers to the drawing of 1st Embodiment and 3rd Embodiment, attaches | subjects the same code | symbol, and abbreviate | omits the description.
[0113]
Hereinafter, the points different from the third embodiment will be mainly described (see FIGS. 1, 2 and 8 as appropriate).
The moving trajectory pattern extracting means of the fourth embodiment has a function of drawing a line segment on the screen of the display device 3 of the information processing device M, in addition to the moving trajectory pattern extracting means of the third embodiment. For example, a LINE command statement or the like is included in a routine of a program constituting the movement trajectory pattern extraction means, and a line segment specified by pointer position information is drawn.
Note that the line segment to be drawn is stored in the line-segment FIFO. When the movement trajectory of the pointer P is drawn, the movement trajectory of the pointer shown in FIG. 8 becomes visible to the operator of the pointing device 4. In addition, it has a function of deleting drawn line segments.
[0114]
Next, the operation of the object selection and determination method according to the fourth embodiment will be described with reference to the processing flowchart of FIG.
FIG. 10 is a processing flowchart in the object selection deciding method of the fourth embodiment.
[0115]
The processing from step S401 to step S405 in the processing flowchart in FIG. 10 is the same as the processing from step S301 to step S305 in the processing flowchart (see FIG. 9) in the third embodiment, and a description thereof will be omitted.
[0116]
In step S406, the line segment Tn (0 ≦ n <N) is extracted from the line segment storage FIFO, and the line segment is drawn on the screen of the display device 3 by a LINE command or the like. This allows the operator of the pointing device 4 to see the line segment.
[0117]
The processing from step S407 to step 414 is the same as that from step S306 to step S313 in the processing flowchart (see FIG. 9) in the third embodiment, and a description thereof will be omitted. It should be noted that there is a step of deleting the drawn line segment at an appropriate place in this processing flowchart.
[0118]
According to the object selection and determination method of the fourth embodiment, similarly to the third embodiment, the intention of the operator can be reliably determined by operating the pointing device based on the familiar operation when pointing to an object. Can be transmitted to a computer (information management device). In addition, the operator can correct the movement of the pointer (feedback the movement of the pointer) according to the movement locus of the pointer drawn and displayed on the screen. In addition, it can be used as a material for determination of the intention of the operator of the pointing device to cancel an operation of selecting an object. Further, when deciding the selection of an object, a switch for deciding the selection of an external switch or the like is not required.
[0119]
The present invention can be widely modified without being limited to the above-described embodiments.
For example, the pointing device is limited to a specific one as long as it can operate the pointer on the screen and can grasp the position information of the pointer, such as one that captures the operator with a CCD camera. Everything can be applied without. Further, the present invention can be applied to an input device of a type in which an instruction is given to a computer by touching a screen with a pen tip or a finger tip. Furthermore, the present invention can be applied to an input device that gives an instruction by pointing to a TV screen or the like with a laser pointer or an input device that gives an instruction by measuring the direction pointed by a fingertip. Further, the program and hardware for executing the object selection and determination method are not limited to specific ones. For example, the program may be written in C language or may be written in Visual Basic (trademark). The hardware may be a personal computer, a TV having a personal computer function, or various information-related devices. The objects to be operated are not only those displayed on the desktop, but also displayed on windows such as word processors and spreadsheet software, and those displayed on windows such as Internet browser software. There.
[0120]
In the first embodiment and the second embodiment, a plurality of angle thresholds (separation angles) and input determination small areas are defined, and the opening / closing, movement, erasure, and the like of the object are associated with each other. A specific action on the object can be assigned at the same time as the selection is confirmed. Further, in the fourth embodiment, if the drawing of the line segment is displayed only when the pointer approaches a certain distance from the object, operability can be improved.
[0121]
Although the time threshold is set in the first embodiment and the second embodiment, the time threshold is not an essential requirement of the present invention. Therefore, a configuration that does not use the time threshold may be adopted. Of course, even without using the time threshold, the selection of the object can be surely determined according to the operator's will.
[0122]
【The invention's effect】
The present invention described above has the following remarkable effects. According to the first aspect of the present invention, the selection of the object is determined only by moving the pointer in a predetermined pattern. Therefore, when the selection of the object is determined, it is not necessary to attach or operate the external switch, so that the burden on the operator (user) can be greatly reduced. Further, since the selection of the object is determined by the operator intentionally operating and moving the pointer, the selection of the object is not unconsciously determined. Therefore, the affinity with the operator is extremely high. Further, the present invention can be used regardless of the type of the pointing device as long as the pointing device can acquire the position information of the pointer. For this reason, for example, it can be used as a method for selecting and confirming an object in various pointing devices for a disabled person having a different degree of movement disorder or a different part of the disorder.. MaClaims1According to the invention described in (1), navigation is performed to the input confirmation small area, and selection of the object can be confirmed more reliably. Therefore, the intention of the operator can be reliably and easily transmitted to the computer.. MaClaims2According to the invention described in (1), the movement trajectory of the pointer in the past predetermined time is displayed (drawn), so that the selection of the object can be more reliably determined. And claims3According to the invention described in (1), claim 1OrClaim2A storage medium storing the invention can be obtained.
[Brief description of the drawings]
FIG. 1 is a block diagram of an information processing apparatus in which an object selection and determination method according to a first embodiment of the present invention is performed.
FIG. 2 is a schematic diagram of pointing an object on a screen of the display device of FIG. 1 with a pointer.
FIGS. 3A and 3B are schematic diagrams illustrating an object selection determination operation according to the first embodiment of the present invention, wherein FIG. 3A illustrates a movement locus of a pointer with respect to the object, and FIG. 3B illustrates a principle of calculating a detachment angle θ; Is shown.
FIG. 4 is a processing flowchart in an object selection and determination method according to the first embodiment of the present invention.
FIG. 5 is a schematic diagram for explaining an object selection deciding operation and a display position of an input deciding small area according to a second embodiment of the present invention.
FIG. 6 is a schematic diagram for explaining an operation for selecting and confirming an object different from FIG. 5 and a display position of an input confirming small area.
FIG. 7 is a processing flowchart in an object selection confirmation method according to a second embodiment of the present invention.
FIGS. 8A and 8B are schematic diagrams illustrating an operation of confirming selection of an object according to a third embodiment of the present invention, and FIGS. 8B and 8B are diagrams illustrating an operation of assigning a predetermined label to a pixel. .
FIG. 9 is a processing flowchart in an object selection and determination method according to a third embodiment of the present invention.
FIG. 10 is a processing flowchart in an object selection and determination method according to a fourth embodiment of the present invention.
[Explanation of symbols]
M ... Information processing device
1 Main controller
3 Display device
4 Pointing device
P ... pointer
θ: separation angle (first embodiment)
φ… Angle threshold
α: display angle (second embodiment)

Claims (3)

Used Pointe I ing device capable indicated by the pointer any object from one or more objects displayed on the plane or space, in the method of selecting the determination of object information processing apparatus is designated by the pointer,
An information processing device that receives an input of a movement locus of the pointer as position information on a two-dimensional plane, stores the input in a storage unit,
A small area for input confirmation is displayed at a specific position based on the entry position of the pointer with respect to the object,
When the movement trajectories of said pointer from said object is detached passes through the small region, and determine child for the selection of said object
A method for confirming the selection of an object characterized by the following. The information processing apparatus reads the moving trajectory of the pointer in the past for a predetermined time from the storage unit, and the view by
2. The method according to claim 1, wherein the object is selected. A storage medium storing an object selection determination program, wherein a program for causing a computer to execute the procedure in the object selection determination method according to claim 1 or 2 is stored in a storage medium readable by the computer. .

Images