JP4445582B2 - Virtual sense presentation device - Google Patents

Description

  An object of the present invention is to provide a virtual sensory presentation device that makes it possible to feel as if an actual phenomenon has occurred based on visual information.

A. Lecuyer, J. Burkhardt, and L. Etenne, “Feeling bumps and holes without a haptic interface: the perception of pseudo-haptics textures”, SIGCHI, 2004. There has been proposed a tactile sensation presentation method that allows the user to feel a sense of unevenness by changing the moving speed of the pointer that is moved by. A. Lecuyer, S. Coquillart, A. Kheddar, P. Richard and P. Coiffet, "Pseudo-Haptic Feedback: Can Isometric Input Devices Simulate Force Feedback?", IEEE International Conference on Virtual Reality (IEEE VR), pages 83-90, 2000. (Non-Patent Document 2) is a technique that makes the user feel a sense of force (heavy, light feeling) by changing the rate of change of the image that changes according to the user's operation. Proposed. However, these known techniques can be applied only when the controller is used, such as when the movement of the pointer image (cursor) is controlled by an operation controller such as a mouse. Techniques for slowing down the cursor movement speed relative to a specific position on the image are also described in Japanese Patent Application Laid-Open No. 2000-148350 (Patent Document 1), Japanese Patent Application Laid-Open No. 2006-268862 (Patent Document 2), and the like. As described above, this technique is also employed for the purpose of controlling the amount of movement of the cursor according to the intention of the operator.
“Feeling bumps and holes without a haptic interface: the perception of pseudo-haptics textures”, SIGCHI, 2004. "Pseudo-Haptic Feedback: Can Isometric Input Devices Simulate Force Feedback?", IEEE International Conference on Virtual Reality (IEEE VR), pages 83-90, 2000. JP 2000-148350 A JP 2006-268862 A

  Conventionally, however, a person who uses a controller for operation based on visual information cannot reliably give a virtual skin sensation / force sensation, and a person who does not use the controller (operator, observer, etc.) There is no virtual sensation presentation device that can reliably present a virtual equilibrium sensation.

  An object of the present invention is to provide a virtual sensation presentation apparatus that can reliably present a virtual skin sensation / force sensation or a virtual equilibrium sensation.

Humans recognize the environment by deeply connecting visual and tactile sensations. As a result of taking action on the environment, visual and tactile feedback is performed consistently to recognize objects existing in the environment. At that time, visual and tactile sensations influence each other. For example, when drawing a line with a pen, the appearance and tactile reaction force are different between drawing on paper and drawing on another material. Thus, visual changes and tactile changes usually occur simultaneously and are related. That is, in a state in which visual tactile associated, if any visual change occurs, and hung it, but also occurred tactile change and humans illusion. Therefore, by presenting appropriate visual information, it is possible to present a tactile sensation with only a visual change even if there is no tactile sense presenting device. The present invention makes it possible for the user to generate a pseudo (virtual) tactile sensation (skin sensation + force sensation) or a pseudo virtual equilibrium sensation against this background. In other words, the present invention utilizes the relationship between visual perception and tactile sensation / equilibrium sensation in human perception. According to the present invention, a human sense of illusion is used instead of using a large-scale device, so that a tactile sensation can be presented anywhere at low cost.

  Specifically, first, a virtual sensation presentation device of the first type of the present invention includes an image display device, scroll image image data providing means, pointer image image data providing means, relationship duration detecting means, Scroll change command generation means. The image display device displays an image on the screen based on the image data and the input command. For example, when a personal computer is used as hardware, a display device and a drive circuit for displaying an image on the screen of the display device correspond to the image display device.

  The scroll image image data providing means provides the image display device with the image data for the scroll image necessary for displaying the scroll image to be scrolled on the screen as the image data. The scroll image data providing means is realized by a program if a personal computer is used as hardware.

  The pointer image image data providing means includes a movement of a controller operated by an operator viewing the scroll image, a movement of a stylus moved by the operator on the operation surface, or a finger moved by the operator on the operation surface. The image data for pointer image necessary for displaying on the scroll image a pointer image that can be controlled on the scroll image according to the movement of the image is provided as image data to the image display device. Here, the “scroll image” is a moving background image, and the moving direction of the scroll image may be any of the left-right direction, the up-down direction, and the diagonal direction. When a personal computer is used as hardware, the controller to be operated is typically a mouse. The stylus is a pen-type instrument that is used when a contact-type (touch panel-type) input device is used for input by bringing the tip into contact with the operation surface. The movement of the finger moving on the operation screen is detected by a contact-type input device that uses a human finger instead of the stylus. In this type of contact-type input device, a device that can detect a pressing force when the operation surface is pressed with a stylus or a finger may be used. The “pointer image” is an image displayed on the scroll image, such as a cursor (which may be a specific image such as a character image) or an image showing a locus of contact points of a stylus or a finger. It is. The pointer image image data providing means is a means realized by a program if a personal computer is used as hardware.

  Further, the relationship continuation period detecting means detects the specific area after the pointer image has a predetermined positional relationship with respect to the specific area image in the scroll image based on the scroll image image data and the pointer image image data. A relationship continuation period is detected until the pointer image is not in a predetermined positional relationship with respect to the image. Here, the specific area image is an area arbitrarily determined in the scroll image, and its shape is not particularly limited. For example, it may be a belt-like image or a character or building image. Further, the positional relationship in which the pointer image is predetermined with respect to the specific region image is typically a relationship in which the pointer image is included in the specific region image. However, it may be determined that this positional relationship is established immediately before the pointer image enters the specific area image, and it is further determined that this positional relationship is still established immediately after the pointer image is output from the specific area image. May be. The relationship duration detection means is a means realized by a program if a personal computer is used as hardware.

  The scroll change command generating means generates a scroll change command for changing the scroll speed and / or the scroll direction of the scroll image when the relation duration detecting means detects the relation duration. That is, when changing only the scroll speed, changing the scroll direction, or changing both the scroll speed and the scroll direction, any of the three types of changes can be executed. This means is also realized by a program if a personal computer is used as hardware.

  In the case of using the above devices and means, the image display device changes the scroll speed and / or the scroll direction of the scroll image according to the scroll change command when the scroll change command is input as the input command. The changing speed of the pointer image is determined by the operator. Regardless of the direction of movement of the pointer image, if the scroll speed is reduced when a scroll change command is input, the user feels that the speed of the pointer image has been reduced. If the stylus is used, this change in feeling gives the operator the feeling that the stylus has become heavy or the material of the operation surface on which the stylus is moved has changed. This means that when the scrolling speed is constant and sufficiently high relative to the speed of the pointer image, the movement of the pointer image is relative to the scroll image, that is, the scrolling for the operator who is watching the pointer image. It is perceived to move at the same speed as the scroll speed on the opposite side of the image movement. In this state, if the scroll speed is slow, it seems that the moving speed of the pointer is slow. Conversely, when the scrolling speed is increased, the user feels that the moving speed of the pointer has increased. If the stylus is used, this change in feeling gives the operator a feeling that the stylus has become lighter or that the material of the operation surface on which the stylus is moved has changed. Feeling heavy changes feeling like writing on rubber, feels like being touched, etc. Feeling lighter changes resistance depending on the person, such as no resistance or moving on a smooth surface.

  Regardless of the movement direction of the pointer image, when the scroll direction changes when a scroll change command is input (for example, when the scroll direction changes to a direction orthogonal to the previous scroll direction), the changed scroll direction is reversed. It feels like the pointer image moves in the direction of. If the stylus is used, this change in feeling gives the operator a feeling that the stylus has slipped on the operation surface.

  As described above, according to the present invention, even when the moving speed of the controller, the stylus, or the finger is not changed, by changing the scrolling speed and / or the scrolling direction, both the pointer image and the specific area image can be obtained. From visual information, it is possible to virtually present (feel) changes in the sense that the controller, stylus, and fingers become heavier, lighter, and slippery.

  The virtual sensation presentation apparatus of the second type of the present invention has nothing to operate, and presents a virtual sensation change to an observer of the image while the image display device displays the image. Therefore, the second type virtual sensation presentation device includes an image display device, scroll image data providing means, positional relationship detection means, relationship duration detection means, and scroll change command generation means. . The positional relationship detection means detects the positional relationship between the observer who is looking at the scroll image and the specific area image in the scroll image. The positional relationship detection means detects a change in the position of the observer from image data obtained using a plurality of cameras fixed on the image display device side, and based on the result, the observer and the specific area image are detected. The positional relationship may be detected. In any case, the configuration of the positional relationship detection means is arbitrary. The relationship continuation period detecting unit detects a relationship continuation period until the predetermined positional relationship disappears after the positional relationship detected by the positional relationship detecting unit becomes the predetermined positional relationship. Other devices and means have the same functions as the devices and means used in the first type virtual sensation presentation device.

  In the second type of virtual sensation presentation apparatus, the observer is made to feel a change in sensation by changing the relationship between the movement speed and movement direction of the observer with respect to the specific area image and the scroll direction and scroll speed of the scroll image. be able to. For example, if the scroll speed is reduced when a scroll change command is input, the observer feels that his / her own movement speed has also become slower regardless of the actual movement speed and movement direction of the observer. This change in sensation based on the visual information gives the observer a feeling that the body has become heavier. This is because if the scroll speed is a constant speed and faster than the movement speed of the observer, the observer mainly perceives relative movement with respect to the scroll image. From this state, when the scrolling speed becomes slow, it feels that its own moving speed has become slow. In this situation, if the scrolling speed is further increased, the observer feels that his / her moving speed has increased, and as a result, he / she feels that his / her body has become lighter. Note that the greater the difference between the scroll speed and the movement speed of the observer, the greater the change in the sensation experienced by the observer.

  In the second type of virtual sensation presentation apparatus, the observer is made to feel a change in sensation by changing the relationship between the movement speed and movement direction of the observer with respect to the specific area image and the scroll direction and scroll speed of the scroll image. be able to. For example, when the moving direction of the observer and the scrolling direction of the scroll image match (when the scrolling direction is the left-right direction and the observer is moving while looking at the image in the same left-right direction as the scrolling direction), scrolling is performed. If the scroll speed is slowed down when the change command is input, it feels that the speed of the specific area image has slowed regardless of the actual moving speed of the observer. As a result, the observer feels that his / her moving speed has also slowed down. This change in sensation based on the visual information gives the observer a sense of becoming heavier. In this situation, if the scrolling speed is increased, the observer feels that the speed of the specific area image is increased from the visual information about the specific area image. This change in sensation based on the visual information gives the observer a feeling that his / her moving speed has increased, and as a result, the sensation that the body has become lighter.

  Furthermore, when the scroll image is projected on the screen in front of the viewer and the scroll direction is, for example, the left-right direction, the viewer moves in the left-right direction while viewing the image. At this time, when the scroll direction is changed in the vertical direction, the observer receives a sense of moving obliquely upward or obliquely downward. Further, when the scroll direction of the scroll image is, for example, the vertical direction, it is assumed that the observer is moving toward the image. At this time, if the scroll direction changes in the left-right direction, the observer receives a sense that he is being pulled in the lateral direction.

  In the third type virtual sensation presentation device, the pointer image image data providing means is a movement of the stylus and a pressing force that the operator who is looking at the scroll image moves on the operation surface or the operator is on the operation surface. The operation detecting means for detecting the movement of the finger to be moved and the pressing force is included, and based on the output of the operation detecting means, the image data for pointer image is provided as image data to the image display device. In this case, the scroll change command generating means may be configured as follows. That is, the scroll change command generation means generates a scroll change command for changing the scroll speed of the scroll image according to the pressing force detected by the operation detection means when the relation continuation period detection means detects the relation continuation period. Configure as follows. The image display device is configured to change the scroll speed of the scroll image according to the scroll change command when the scroll change command is input as the input command. Regardless of the movement direction of the pointer image, when a scroll change command indicating that the pressing force has increased during the relationship continuation period is input, if the scroll speed is slowed, both the pointer image and the specific area image From the visual information, the user feels that the speed of the pointer image has slowed down. This change in sensation is that when the stylus is used, the pointer moves slowly despite the increase in pressing force, that is, the stylus is heavy or the resistance of the operating surface that moves the stylus increases. Receive a feeling. Conversely, when a scroll change command indicating that the pressing force has increased during the relationship continuation period is input, if the scrolling speed is increased, the pointer information and the visual information on both the specific area images are obtained from the pointer information. Feels like the image speed has increased. If the stylus is used, the change in the sensation feels that the stylus has become lighter due to an increase in the pressing force, or that the resistance of the operation surface for moving the stylus is small and smooth.

  In any of the first to third types of virtual sensation presentation devices, a function for generating an image vibration command for vibrating the specific area image or the entire scroll image together with the scroll change command is added to the scroll change command generating means. May be. The image display device is configured to vibrate the specific area image or the entire scroll image when an image vibration command is input as the input command. In this way, the operator feels as if the moving controller or the like is moving on the uneven surface. In addition, the observer feels as if the moving floor is uneven.

  Further, a function for generating a visual feature change command for changing the visual feature of the specific area image together with the scroll change command may be added to the scroll change command generating means. The image display device is configured to change the visual feature of the specific area image when the visual feature change command is input as the input command. For example, when a specific area image is vibrated, a visual feature change command is generated.

  Needless to say, the change in the scroll speed by the scroll change command may include the specific region image being stopped for a predetermined time (occurrence of a temporary stop state). In this way, it is possible to give the operator and the observer the feeling that he / she has entered a region that is clearly different from the previous one. If the predetermined time is 300 to 400 ms, it is possible to give not only the feeling of stopping but the feeling of entering a different area. If this time is too short, this sensation cannot be obtained, and if this time is too long, a sense of stopping will appear strongly.

  The image display device may further include a sound generation device that outputs a sound whose tempo changes according to the scroll speed when a scroll image is displayed on the screen. When such a sound generator is provided, for example, if the tempo of the sound is increased when the controller or the like feels heavy, it will feel that the weight is also heavy from the auditory sense, and a virtual sensation is presented. The effect can be increased. Similarly, the image display device may further include a sound generation device that outputs sound whose sound intensity (amplitude magnitude) changes according to the scroll speed when the scroll image is displayed on the screen. . In this case, for example, when the sound intensity is weak when feeling light, the sense of weight is reduced from the sense of hearing, and the presentation effect of the virtual sensation is increased. it can.

It is a block diagram which shows the structure of an example of embodiment of the 1st type virtual sensation presentation apparatus of this invention. (A) thru | or (D) is a figure for demonstrating an example of the change of the image displayed on the screen of a display apparatus at the time of utilizing a mouse | mouth as an input device. It is a figure used in order to explain the situation at the time of changing scroll speed. It is a figure used in order to explain the situation at the time of inputting using a stylus. It is a flowchart which shows the algorithm of an example of the program used when implement | achieving the main part of embodiment of FIG. 1 using a computer. It is a figure which shows notionally the relationship between a scroll image and a pointer image (trajectory of the tip of a stylus). It is a figure which shows an example of the state in the case of moving a finger | toe on the screen which serves as an operation surface, and detecting a finger | toe movement. It is a block diagram which shows the structure of an example of embodiment of the 2nd type virtual sense presentation apparatus of this invention. It is a figure used in order to explain the situation of an example at the time of putting the 2nd type virtual feeling presentation device into practical use. 10 is a flowchart showing an example of a program algorithm used when the main part of the configuration of FIG. 9 is realized using a computer. It is a block diagram which shows the structure of the 3rd type virtual sense presentation apparatus which developed the 1st type virtual sense presentation apparatus. 13 is a flowchart illustrating an example of a program algorithm used when the main part of the configuration of FIG. 12 is realized using a computer.

  Hereinafter, an embodiment of a virtual sensation presentation device of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing the configuration of an example of an embodiment of a first type virtual sensation presentation device of the present invention. The virtual sensation presentation device 1 includes an image display device 3, scroll image data providing means 5, pointer image image data providing means 7, relationship duration detecting means 9, scroll change command generating means 11, And a sound generator 13. The image display device 3 includes a display device 31 and a drive circuit 32 for displaying an image on the screen of the display device 31. Further, the scroll image image data providing means 5 provides the image data for scroll image necessary for displaying the scroll image to be scrolled on the screen of the display device 31 to the drive circuit 32 of the image display device 3 as image data.

  The pointer image image data providing means 7 displays on the scroll image a pointer image that moves in a controllable manner on the scroll image in accordance with the movement of the input device 8 operated by the operator viewing the scroll image. Necessary pointer image image data is provided as image data to the drive circuit 32 of the image display device 3. Here, the input device 8 is a contact type input device including a controller such as a mouse or a contact type operation surface that detects the movement of the stylus or the operator's finger itself. Therefore, the movement of the controller, the movement of the stylus that the operator moves on the operation surface, or the movement of the finger that the operator moves on the operation surface is input to the pointer image image data providing means 7. . In the present specification, the “scroll image” is a moving background image, and the moving direction of the scroll image may be any of the left-right direction, the up-down direction, and the diagonal direction.

  2A to 2D are diagrams for explaining an example of a change in an image displayed on the screen DS of the display device 31 when a mouse is used as the input device 8. As can be seen from FIG. 2, in this example, the scroll image scrolls from the right to the left of the screen (the background image moves).

  When a personal computer is used as hardware, the controller to be operated is typically a mouse. The stylus is a pen-type instrument that is used when a contact-type (touch panel-type) input device is used for input by bringing the tip into contact with the operation surface. Many contact-type input devices are configured by combining a display monitor and a matrix switch. In many cases, the display monitor is also used as the display device 31 (image display unit) of the image display device 3 described above. In this example, the arrow image shown in FIG.

  Further, the relationship continuation period detecting means 9 determines the pointer image P in advance for the specific area image S (see FIG. 2) in the scroll image based on the scroll image image data D1 and the pointer image image data D2. After the positional relationship is reached, a relationship continuation period T is detected until the pointer image P no longer has a predetermined positional relationship with respect to the specific region image S. Here, the specific area image S is an area arbitrarily determined in the scroll image, and its shape is not particularly limited. In the example of FIG. 2, the specific area image S is shown as a band-shaped image. Here, the predetermined positional relationship of the pointer image P with respect to the specific region image S is typically a relationship in which the pointer image P is included in the specific region image S. Note that it may be determined that this positional relationship is established immediately before the pointer image P enters the specific area image S, and this positional relationship is still established immediately after the pointer image P comes out of the specific area image S. Of course, it may be determined that it is present.

  The scroll change command generating unit 11 generates a scroll change command C for changing the scroll speed and / or scroll direction of the scroll image when the relationship duration detecting unit 9 detects the relationship duration T. In the present embodiment, the scroll change command generation means 11 is configured to input the scroll change command C to the sound generator. In this example, any one of the three types of changes can be executed by changing only the scrolling speed, changing the scrolling direction, and changing both the scrolling speed and the scrolling direction, depending on the program setting described later. .

  The sound generator 13 outputs a sound whose tempo or sound intensity (amplitude magnitude) changes in proportion to the scroll speed of the scroll image in synchronization with the scroll image.

  In the apparatus of FIG. 1, the image display device 3 changes the scroll speed and / or the scroll direction of the scroll image according to the scroll change command C when the scroll change command C is input as an input command. In the example of FIG. 2, only the scroll speed is changed. The changing speed of the pointer image is determined by the operator. In the example of FIG. 2, the movement direction of the pointer image P and the scroll direction of the scroll image are the same. In this example, during the period in which the pointer image P is in the specific area image S, the scroll change command C is input to the image display device 3, and the scroll speed is reduced. In FIG. 2, the length of the arrow (pointer image P) displayed on the screen D has a size proportional to the speed of scroll display. During the period in which the pointer image P is in the specific area image S, the operator feels that the speed of the pointer image P has slowed down from the visual information about both the pointer image P and the specific area image S. receive. This change in feeling gives the operator the feeling that the mouse has become heavier if the mouse is used as a controller. In this situation, conversely, if the scroll speed is increased, the visual information about both the pointer image P and the specific area image S feels as if the speed of the pointer image is increased. This change in feeling gives the operator the feeling that the mouse has become lighter if the mouse is used. In the example of FIG. 2, the specific area image S is vibrated left and right during the relationship continuation period T. Therefore, the operator can feel as if the mouse is vibrating. This relationship can be organized as shown in FIG.

  Further, unlike the example of FIG. 2, even when the moving direction of the pointer image P and the scroll direction of the scroll image are opposite to each other, if the scroll speed is reduced when the scroll change command C is input, the pointer image P And from the visual information about both images of the specific area image S, the user feels that the speed of the pointer image P has been reduced. If the stylus ST is used as shown in FIG. 4, this change in feeling gives the operator the feeling that the stylus ST has become heavy or the material of the operation surface that moves the stylus ST has changed. In this situation, when the scroll speed is increased, the visual information about both the pointer image P and the specific area image S feels as if the speed of the pointer image P is increased. If the stylus ST is used, this change in feeling gives the operator a feeling that the stylus ST has become lighter or that the material of the operation surface on which the stylus ST is moved has changed.

  Further, when the movement direction of the pointer image P and the scroll direction of the scroll image coincide with each other, if the scroll direction is changed to a direction orthogonal to the previous scroll direction when the scroll change command C is input, the pointer image P From the visual information about both images of the specific area image S, the pointer image P feels like moving in the changed scroll direction. If the stylus ST is used, this change in feeling gives the operator the feeling that the stylus ST has slipped on the operation surface (the stylus has moved up and down, left and right, or diagonally).

  As described above, according to the apparatus of the present embodiment, even when the moving speed of the controller, the stylus, and the finger is not changed, the pointer image and the specific area image can be changed by changing the scroll speed and / or the scroll direction. From the visual information about both images, it is possible to virtually present to the operator changes in the sense that the controller, stylus, and fingers become heavier, lighter, and slippery.

  In addition, when the scroll image is displayed on the screen as in the present embodiment, if the sound that changes the tempo according to the scroll speed is output from the sound generation device 13, the virtual sensation presentation effect is increased. It becomes possible to make it. For example, if the tempo of the sound is increased when the controller or the like feels heavy, it is felt that the weight is increased from the sense of hearing, and the effect of presenting a virtual sensation can be increased. Also, if the controller feels light for a virtual sensation, if the sound intensity becomes weak, it will make you feel that the weight is lighter from the sense of hearing, and increase the presentation effect of the virtual sensation Can do.

FIG. 5 is a flowchart showing an algorithm of an example of a program used when the main part excluding the image display device 3, the input device 8, and the sound generator 13 of the embodiment of FIG. 1 is realized using a computer. . Note that when the program is installed and operated in the computer, each unit shown in FIG. 1 is realized in the computer. Naturally, various data are stored in a memory attached to the computer. In step ST1, scroll image data D1 used for displaying a scroll image is input. In step ST2, pointer image image data D2 used for displaying a pointer image is input. In step ST3, sound data reproduced by the sound generator 13 is input. When these data are input, in step ST4, display of a scroll image is started on the image display device 3, and display of a pointer image is started when the input device 8 is operated. Furthermore, sound is output in synchronization with the display of the scroll image. In step ST5, it is determined whether or not the specific area image S and the pointer image P have a specific positional relationship. In step ST5, if it is determined that the specific area image S and the pointer image P have a specific positional relationship (established), the process proceeds to step ST6, and it is determined that the specific positional relationship is not satisfied. In this case, the process returns to step ST4, and image display and sound output are performed. In step ST6, the scroll speed and / or the scroll direction are changed, the specific area image is vibrated, and the acoustic tempo or sound intensity is changed. In step ST7, it is determined whether or not the specific area image and the pointer image have deviated from the specific positional relationship (the relationship has been canceled). If it is determined in step ST7 that the specific positional relationship has been resolved, the process returns to step ST4 and the same operation is repeated. If it is determined in step ST7 that the specific positional relationship has not been resolved, step ST7 again. Operation is performed.

  FIG. 6 conceptually shows the relationship between the scroll image and the pointer image (trajectory of the tip of the stylus) when this program is executed. Such an environment is the same as a drawing environment in which a picture is drawn while rotating an annular canvas CC. Since the operator draws a picture while rotating the annular canvas CC, the drawn line is affected by the canvas CC, and an unexpected line may be obtained. When this is realized using a personal computer, the screen D becomes a canvas CC and the stylus ST becomes a pen. Then, an image (specific area image) that is a visual clue is displayed on the screen. This image may be an image with a shadow that expresses unevenness in a plane. In this case, the convex surface is brightened and the concave surface is represented as a shadow. If the convex surface is touched, the brightness of the line drawn by the stylus ST is reduced, and the scroll speed of the scroll image (background image) is reduced. At the end of the convex surface, the brightness of the line is increased and the scroll speed is increased (returned to the original speed). In this way, uneven tactile elements can be obtained even in a drawing environment.

  FIG. 7 shows an example of a state in which the finger F is moved on the screen D that also serves as the operation surface and the movement of the finger F is detected. As shown in FIG. 7, it goes without saying that a plurality of specific area images S1 and S2 having different shapes may be displayed in the scroll image. Also, the moving direction of the finger F is arbitrary, and the scroll direction of the scroll image is also arbitrary.

  In the above embodiment, the change in the scroll speed by the scroll change command C may include stopping the specific area image S for a predetermined time (occurrence of a temporary stop state). In this way, it can be surely felt that a different area has been entered from the previous area. If this predetermined time is set to 300 to 400 ms, this feeling can be provided with certainty.

  FIG. 8 is a block diagram showing a configuration of an example of an embodiment of the second type virtual sensation presentation device of the present invention. In FIG. 2, the same blocks as those used in the block diagram of FIG. 1 are denoted by reference numerals obtained by adding 100 to the number of reference numerals shown in FIG. In this virtual sensation presentation apparatus 101, there is nothing to operate, and a change in virtual sensation is presented to an observer M who observes the image while the image display apparatus 103 displays the image. Therefore, in this second type of virtual sensation presentation device embodiment, the image display device 103, the scroll image image data providing means 105, the positional relationship detecting means 108, the relationship duration detecting means 109, and the scroll change. A command generator 111 and a sound generator 113 are provided. In this embodiment, as shown in FIG. 9, the display device 131 of the image display device 103 is installed on the floor surface on which the observer M walks. The positional relationship detection means 108 detects the positional relationship between the observer M who is viewing the scroll image displayed on the screen D of the display device 131 and the specific area image S in the scroll image. As shown in FIG. 9, the positional relationship detection means 108 detects a change in the position of the observer from the image data obtained by using the two cameras CM1 and CM2 fixed on the image display device 103 side. Based on the above, the positional relationship between the observer M and the specific area image S is detected. The configuration of the detection structure of the positional relationship detection means 108 is arbitrary, and the positional relationship may be detected using a plurality of infrared sensors. The relationship continuation period detection unit 109 detects the relationship continuation period T until the predetermined positional relationship disappears after the positional relationship detected by the positional relationship detection unit 108 becomes the predetermined positional relationship. The other devices 103 and 113 and the respective means 105 and 111 have the same functions as the corresponding devices and means used in the first type virtual sensation presentation device 1 shown in FIG.

  Note that in the second type of virtual sensation presentation apparatus 101, the sensation is given to the observer M by changing the relationship between the movement speed and movement direction of the observer M with respect to the specific region image S and the scroll direction and scroll speed of the scroll image. You can feel the change. For example, as shown in FIG. 9, when the moving direction of the observer M and the scroll direction of the scroll image coincide with each other, if the scroll speed is increased when the scroll change command C is input, the specific area image S is displayed. From the visual information, the user feels that the speed of the specific area image S has increased. This change in sensation based on the visual information gives the observer M the feeling that his / her own speed has increased and the body has become lighter. Conversely, if the scrolling speed is slowed down, the observer's speed becomes a sense of slowing down, and a virtual feeling that the observer's body becomes heavy can be given to the observer M.

  Even when the moving direction of the observer M and the scroll direction of the scroll image are opposite to each other, if the scroll speed is reduced when the scroll change command C is input, the speed of the specific area image S is determined from the visual information. I feel like I'm late. This change in sensation gives the observer M a feeling that his / her moving speed has slowed down or a feeling that his body has become heavy. Even in this state, if the scroll speed is increased when the scroll change command C is input, the visual information feels that the speed of the specific area image S has increased. This change in sensation gives the observer M a feeling that his / her movement speed has increased or a feeling that his body has become lighter.

  FIG. 10 is a flowchart showing an example of a program algorithm used when the main part of the configuration of FIG. 8 is realized using a computer. Of steps ST21 to ST27, steps ST25 and ST27 are substantially the same as the steps of the flowchart shown in FIG. 5 except that the positional relationship between the specific area image S and the observer M is determined. Since there is no change, description of each step is omitted.

  Note that when the scroll image is displayed on a long wall in front of the observer M and the scroll direction is, for example, the left-right direction, the observer M is moving in the left-right direction while viewing the image. . At this time, when the scroll direction changes in the vertical direction, the observer M receives a sense of moving diagonally upward or moving diagonally downward. Further, it is assumed that the observer M is moving toward the image D when the scroll direction of the scroll image is, for example, the vertical direction. At this time, when the scroll direction changes in the left-right direction, the observer M feels as if he is being pulled sideways.

  FIG. 11 is a block diagram illustrating a configuration of a third type virtual sensation presentation apparatus obtained by developing the first type virtual sensation presentation apparatus. In FIG. 11, the same blocks as the components used in the block diagram of FIG. 1 are denoted by reference numerals obtained by adding 200 to the number of reference numerals shown in FIG. In this embodiment, the pointer image image data providing means 207 moves and presses the stylus ST that the operator viewing the scroll image moves on the operation surface, or the finger that the operator moves on the operation surface. 1 is different from the configuration of FIG. 1 in that it includes an operation detection means 207a that detects the movement of F and the pressing force. As the operation detection means 207a that can detect the pressing force, for example, a device that detects the pressing force from the output of the force sensor by arranging force sensors at the four corners of the operation surface can be used. Further, the scroll change command generating means 211 is a scroll change that changes the scroll speed of the scroll image in accordance with the pressing force detected by the operation detecting means 207a when the relation duration detecting means 209 detects the relation duration T. Command C is generated. The image display device 203 is configured to change the scroll speed of the scroll image according to the scroll change command C when the scroll change command C is input as an input command. If it does in this way, pressing force will change according to change of an operator's intention.

  In this example, regardless of the movement direction of the pointer image P, when the scroll change command C indicating that the pressing force has increased during the relationship continuation period T is input, if the scroll speed is decreased, the pointer image P and From the visual information on both images of the specific area image S, the user feels that the speed of the pointer image P has been reduced. If the stylus ST is used, the change in the sense is that the pointer movement becomes slow despite the increase in the pressing force, that is, the stylus ST is heavy or the resistance of the operation surface that moves the stylus ST is reduced. Receive a feeling of growing. Conversely, when a scroll change command C indicating that the pressing force has increased during the relationship continuation period T is input, if the scroll speed is increased, the visual information about both the pointer image P and the specific area image S is displayed. Is felt as if the speed of the pointer image P has increased. If the stylus ST is used, this sensation change is felt as the stylus ST becomes lighter or the operation surface for moving the stylus ST becomes smaller and smoother as the pressing force increases. .

  FIG. 12 is a flowchart showing an example of a program algorithm used when the main part of the configuration of FIG. 11 is realized using a computer. Of steps ST31 to 38, the steps are substantially the same as the steps of the flowchart shown in FIG. 5 except that the pressing force is detected at step ST36 and the scroll speed is changed according to the pressing force at step ST37. Since there is no place, explanation of each step is omitted.

  In both the second and third types of virtual sensation presentation devices, the scroll change command generation means 111 and 211 have an image vibration command (vibrating the specific area image S or the entire scroll image together with the scroll change command C ( A function for generating a visual feature change command) may be added. The image display devices 103 and 203 are configured to vibrate the specific area image S or the entire scroll image when an image vibration command (visual feature change command) is input as the input command. If it does in this way, the operator M will receive the feeling as if the controller etc. which are moving are moving on the unevenness | corrugation. Further, the observer M receives a feeling as if the moving floor is uneven.

  Note that the change of the visual feature of the specific area image by the visual feature change command includes not only the above-described vibration but also various changes such as color change and blinking.

  According to the virtual sensation presentation device of the present invention, both the pointer image and the specific area image can be obtained by changing the scroll speed and / or the scroll direction even when the moving speed of the controller, the stylus, or the finger is not changed. From the visual information about, it is possible to virtually present (feel) changes in the sense that the controller, stylus, and fingers become heavier, lighter, and slippery.

  According to another virtual sensation presentation apparatus of the present invention, a change in sensation to the observer is achieved by changing the relationship between the movement speed and movement direction of the observer with respect to the specific area image and the scroll direction and scroll speed of the scroll image. Can make you feel.

Claims (13)

An image display device that displays an image on a screen based on image data and an input command;
Scroll image data providing means for providing image data for scroll image necessary for displaying a scroll image to be scrolled on the screen as the image data;
The scroll according to the movement of the controller operated by the operator who is viewing the scroll image, the movement of the stylus that the operator moves on the operation surface, or the movement of the finger that the operator moves on the operation surface Pointer image image data providing means for providing, as the image data, pointer image image data necessary for displaying a pointer image that is controllably moved on the image on the scroll image;
Based on the scroll image image data and the pointer image image data, after the pointer image has a predetermined positional relationship with respect to the specific area image in the scroll image, the specific area image A relationship continuation period detecting means for detecting a relationship continuation period until the pointer image is not in the predetermined positional relationship;
Scroll change command generating means for generating a scroll change command for changing the scroll speed and / or scroll direction of the scroll image when the relationship duration detecting means detects the relationship duration;
When the scroll change command is input as the input command, the image display device changes the scroll speed and / or the scroll direction of the scroll image according to the scroll change command ,
The scroll change command generating means has a function of generating an image vibration command for vibrating the specific area image or the entire scroll image together with the scroll change command,
The virtual display device is characterized in that the image display device vibrates the specific area image or the entire scroll image when the image vibration command is input as the input command . An image display device that displays an image on a screen based on image data and an input command;
Scroll image data providing means for providing image data for scroll image necessary for displaying a scroll image to be scrolled on the screen as the image data;
The scroll according to the movement of the controller operated by the operator who is viewing the scroll image, the movement of the stylus that the operator moves on the operation surface, or the movement of the finger that the operator moves on the operation surface Pointer image image data providing means for providing, as the image data, pointer image image data necessary for displaying a pointer image that is controllably moved on the image on the scroll image;
Based on the scroll image image data and the pointer image image data, after the pointer image has a predetermined positional relationship with respect to the specific area image in the scroll image, the specific area image A relationship continuation period detecting means for detecting a relationship continuation period until the pointer image is not in the predetermined positional relationship;
Scroll change command generating means for generating a scroll change command for changing the scroll speed and / or scroll direction of the scroll image when the relationship duration detecting means detects the relationship duration;
When the scroll change command is input as the input command, the image display device changes the scroll speed and / or the scroll direction of the scroll image according to the scroll change command,
The scroll change command generating means has a function of generating a visual feature change command for changing a visual feature of the specific area image together with the scroll change command,
The image display apparatus, when the visual characteristic change command is input as the input command, the specific region wherein the visual characteristic virtual sensation presentation apparatus you and changing the image. The change of the scroll speed by scrolling change command, the virtual sensation presentation apparatus according to claim 1 or 2 comprising to predetermined time stopping the specific area image. The virtual sensation presentation apparatus according to claim 3 , wherein the predetermined time is 300 to 400 ms. 3. The virtual device according to claim 1, further comprising: a sound generation device that outputs a sound whose tempo changes according to the scroll speed when the image display device displays the scroll image on the screen. Sensory presentation device. The image display device, wherein when the scroll image is displayed on the screen, according to claim 1 or 2, further comprising a sound generating device for outputting an acoustic sound little changes in response to the scrolling speed Virtual sense presentation device. An image display device that displays an image on a screen based on image data and an input command;
Scroll image data providing means for providing image data for scroll image necessary for displaying a scroll image to be scrolled on the screen as the image data;
A positional relationship detecting means for detecting a positional relationship between an observer watching the scroll image and a specific area image in the scroll image;
A relation continuation period detecting means for detecting a relation continuation period until the predetermined positional relation disappears after the positional relation detected by the positional relation detection means becomes a predetermined positional relation;
Scroll change command generating means for generating a scroll change command for changing the scroll speed and / or scroll direction of the scroll image when the relationship duration detecting means detects the relationship duration;
The image display device is configured to change the scroll speed and / or scroll direction of the scroll image according to the scroll change command when the scroll change command is input as the input command. Sensory presentation device. An image display device that displays an image on a screen based on image data and an input command;
Scroll image data providing means for providing image data for scroll image necessary for displaying a scroll image to be scrolled on the screen as the image data;
It comprises an operation detection means for detecting the movement and pressing force of the stylus that the operator viewing the scroll image moves on the operation surface or the movement and pressing force of the finger that the operator moves on the operation surface. Then, based on the output of the operation detection means, the pointer image image data necessary for displaying on the scroll image a pointer image that can be controlled to move on the scroll image is displayed as the image data. Image data providing means for pointer image provided to the apparatus;
Based on the scroll image image data and the pointer image image data, after the pointer image has a predetermined positional relationship with respect to the specific area image in the scroll image, the specific area image A relationship continuation period detecting means for detecting a relationship continuation period until the pointer image is not in the predetermined positional relationship;
A scroll change command for generating a scroll change command for changing a scroll speed of the scroll image in accordance with the pressing force detected by the operation detection means when the relation duration detection means detects the relation duration. Generating means,
The virtual display device according to claim 1, wherein when the scroll change command is input as the input command, the image display device changes the scroll speed of the scroll image according to the scroll change command. The scroll change command generating means has a function of generating an image vibration command for vibrating the specific area image or the entire scroll image together with the scroll change command,
The image display apparatus, when the image vibration command is input as the input instruction, the virtual sensation presentation apparatus according to claim 7 or 8, characterized in that vibrating the specific area image or entire scroll image . The scroll change command generating means has a function of generating a visual feature change command for changing a visual feature of the specific area image together with the scroll change command,
The virtual sensation according to claim 7 or 8 , wherein the image display device changes the visual feature of the specific region image when the visual feature change command is input as the input command. Presentation device. The virtual sensation presentation device according to claim 7 or 8 , wherein the change of the scroll speed by the scroll change command includes stopping the specific area image for a predetermined time. The image display device, wherein when the scroll image is displayed on the screen, the virtual described sound generating apparatus according to claim 7 or 8, further comprising an outputting a sound tempo is changed in response to the scrolling speed Sensory presentation device. The image display apparatus, when displaying the scroll image on the screen, according to claim 7 or 8, further comprising a sound generating device for outputting an acoustic sound little changes in response to the scrolling speed Virtual sense presentation device.

Images