JP7651874B2 - Non-contact input support device, non-contact input support method, and non-contact input support program - Google Patents

Description

The present invention relates to technology that supports the operation of an aerial display.

Various methods of assistance for operating the operation screen have been devised. For example, Patent Document 1 describes a method of using a marker when operating the operation screen (air display) in a non-contact manner.

The shape of this marker changes when the user (operator) operates the operation screen with their hand. Specifically, when the user's hand is away from the operation screen, the marker is initially displayed in a divided circular shape. When the user's hand approaches the operation screen, the marker forms a ring.

JP 2014-219938 A

In the configuration of Patent Document 1, the shape of the marker changes depending on the position of the user's hand relative to the operation screen, but for users who cannot grasp the state that the marker represents, they may not understand the intention behind the marker's change, which may lead to erroneous operation of the operation screen. In addition, when used by a wide range of users, from the elderly to children, instructions on how to operate the device are required, which is not very convenient.

Therefore, the object of the present invention is to provide a non-contact input support device that allows users to intuitively operate the operation screen without the need for detailed explanations.

According to one example of the present disclosure, the non-contact input support device includes a projection unit that displays in mid-air an operation screen that accepts user operations, a measurement unit that measures the distance between the operation screen and the user's hand that overlaps the operation screen, a shadow generation unit that generates a shadow of the hand from the results of measurement by the measurement unit, and a display control unit that displays the shadow at a position on the operation screen that overlaps with the hand.

In this configuration, the shadow of the user's hand is displayed on the operation screen according to the relative positions of the hand and the operation screen. Therefore, even though the user is operating the operation screen, the user feels as if he or she is operating an actual operation panel or buttons, improving convenience.

According to one example of the present disclosure, the measurement unit of this non-contact input support device may measure the hand in three dimensions.

With this configuration, the user's hand can be measured in three dimensions, and the distance between the operation screen and the user's hand can be grasped in three dimensions.

According to one example of the present disclosure, the shadow generation unit of the non-contact input support device includes a positional relationship calculation unit, a shadow image generation unit, and a position control unit. The positional relationship calculation unit may calculate the three-dimensional position of the hand and the operation screen using the measurement result of the measurement unit. The shadow image generation unit may generate a shadow similar to the shape of the hand using the measurement result and the three-dimensional position. The position control unit may control the position at which the shadow is displayed on the operation screen.

With this configuration, a shadow can be generated according to the shape and position of the user's hand. This allows the user to feel as if they are operating an actual control panel or buttons, improving convenience.

According to one example of the present disclosure, the shadow generating unit of the non-contact input support device may determine the shading of the shadow based on the ratio between the distance between the operation screen and the user's hand and the distance between the projection unit and the operation screen.

With this configuration, the shadow becomes darker when the user's hand approaches the operation screen, and becomes lighter when the hand moves away. This makes the shadow more similar to the actual shadow movement, making it easier for the user to see.

According to one example of the present disclosure, the shadow generating unit of the non-contact input support device may determine the size of the shadow based on the ratio of the distance between the operation screen and the user's hand to the distance between the projection unit and the operation screen.

With this configuration, the shadow can be configured to become smaller when the user's hand approaches the operation screen and become larger when the hand moves away. This makes the shadow more similar to the actual shadow movement, making it easier for the user to see.

This invention provides an input support device that allows users to intuitively operate an operation screen without the need for detailed explanations.

FIG. 1 is a schematic block diagram of a non-contact input support device according to a first embodiment of the present invention. FIG. 2 is a block diagram of the non-contact input support device according to the first embodiment of the present invention. 3A, 3B, and 3C are diagrams showing the movement of the shadow of the non-contact input support device according to the first embodiment of the present invention. FIG. 4 is a flowchart showing a flow of processing by the measurement unit of the non-contact input support device according to the first embodiment of the present invention. FIG. 5 is a flowchart showing a flow of processing by the shadow generating unit and the display control unit of the non-contact input support device according to the first embodiment of the present invention. 6A, 6B, and 6C are diagrams showing the movement of a shadow of a non-contact input support device according to a second embodiment of the present invention. FIG. 7 is a diagram showing the movement of a shadow of the non-contact input support device according to the third embodiment of the present invention.

Below, we will explain how to implement the present invention with reference to several figures.

Application examples
First, an example to which the present invention is applied will be described with reference to Fig. 1. Fig. 1 is a schematic diagram of a non-contact input support device according to a first embodiment of the present invention. The non-contact input support device 10 includes a display unit 110, a measurement unit 120, a shadow generation unit 130, and a display control unit 140. The display unit 110 includes a projection unit 111 and an operation screen 112. The operation screen 112 is, for example, an aerial display or an aerial monitor. Note that the distance between the measurement unit 120 and the operation screen 112 is previously defined as a distance d0.

The projection unit 111 displays information from, for example, a liquid crystal display of a ticket vending machine or the like (hereinafter, the ticket vending machine screen) on the operation screen 112. More specifically, the ticket vending machine screen is reflected using a reflective material or the like and displayed on the operation screen 112. The operation screen 112 displays the ticket vending machine screen.

The user brings their hand close to the operation screen 112. More specifically, the user brings their hand close to a button in order to press the button. The measurement unit 120 measures the shape of the user's hand, the orientation of the hand, and the distance between the operation screen 112 and the hand. The measurement unit 120 outputs the measurement results (hereinafter, the measurement results) to the shadow generation unit 130.

The shadow generation unit 130 generates a shadow (e.g., a two-dimensional image of the hand) that resembles the shape of the user's hand based on the measurement results. The shadow generation unit 130 outputs the shadow, the position of the shadow, the size of the shadow, etc. to the display control unit 140.

The display control unit 140 outputs the shadow of the user's hand to the projection unit 111. The projection unit 111 displays the shadow of the hand on the operation screen 112 so that the shadow of the hand overlaps with the measurement unit 120, the user's hand, and the shadow of the user's hand (are lined up in one direction).

With this configuration, the user can see the shadow of their hand on the operation screen 112, and can intuitively operate how to move their hand to the button.

Configuration example 1
Fig. 1 is a schematic block diagram of a non-contact input support device according to a first embodiment of the present invention. Fig. 2 is a block diagram of the non-contact input support device according to the first embodiment of the present invention. Figs. 3(A), 3(B), and 3(C) are diagrams showing the movement of a shadow in the non-contact input support device according to the first embodiment of the present invention. Fig. 4 is a flowchart showing the processing flow of a measurement unit of the non-contact input support device according to the first embodiment of the present invention. Fig. 5 is a flowchart showing the processing flow of a shadow generation unit and a display control unit of the non-contact input support device according to the first embodiment of the present invention.

A more specific configuration example will be described using the configuration of the non-contact input support device 10 in FIG. 1 and the block diagram of the non-contact input support device in FIG. 2 described above. First, the shadow will be defined as follows. The shadow generated by the shadow generation unit 130 is, for example, a two-dimensional image generated to resemble the shadow (dark part) of a hand formed when light is blocked by the hand. In other words, it is different from the shadow that is actually visible from the user's hand and light.

The non-contact input support device 10 includes a display unit 110, a measurement unit 120, a shadow generation unit 130, and a display control unit 140. The display unit 110, the measurement unit 120, the shadow generation unit 130, and the display control unit 140 are connected so as to be able to send and receive data signals. These may be connected by a wired method such as a cable, or by a wireless method such as Bluetooth (registered trademark). The non-contact input support device 10 and the ticket vending machine 20 are also connected so that the projection unit 111 can display the ticket vending machine screen on the operation screen 112, and further so that the results of the user's operation on the operation screen 112 can be output to the ticket vending machine 20.

First, we will explain the configuration of the non-contact input support device 10.

(Configuration of display unit 110)
The display unit 110 includes a projection unit 111 and an operation screen 112. The operation screen 112 is, for example, an aerial display or an aerial monitor.

The projection unit 111 reflects light onto the ticket machine screen of the ticket machine 20 using a reflective material or the like. The operation screen 112 is formed by forming an image of this reflected light in the air. The ticket machine screen of the ticket machine 20 is displayed on this operation screen 112. When a user operates the operation screen 112, a motion sensor or area sensor (not shown) detects this operation. By detecting this operation, the user can operate the ticket machine 20 by operating the operation screen 112.

(Configuration of the measurement unit 120)
The measurement unit 120 is, for example, a 3D camera. The 3D camera may be any of a stereo camera, a Time of Flight (TOF) camera, and a Light Detection and Ranging (LiDAR). The 3D camera to be used may be determined based on the location where the measurement unit 120 is installed and the processing performance of a computing device that calculates the results of the measurement unit 120.

The measurement unit 120 measures the shape and orientation of the user's hand, as well as the distance between the operation screen 112 and the user's hand (hereinafter referred to as hand distance). Note that because the measurement unit 120 is a 3D camera, it is possible to obtain three-dimensional information with depth, rather than a flat image, of the user's hand shape and orientation.

(Configuration of the shadow generation unit 130)
The shadow generation unit 130 includes an information acquisition unit 131 , a positional relationship calculation unit 132 , a shadow image generation unit 133 , and a position control unit 134 .

The information acquisition unit 131 acquires the shape, hand orientation, and hand distance of the user (hereinafter, measurement results) using the measurement unit 120. The positional relationship calculation unit 132 uses the measurement results to calculate the three-dimensional positional relationship between the user's hand and the operation screen 112 (hereinafter, three-dimensional position). When determining the three-dimensional position, the positional relationship calculation unit 132 may use the coordinates of an arbitrary point, or may use the coordinates that form the outline of the user's hand.

The three-dimensional position is a coordinate that indicates the position of the user's hand in the three-dimensional space formed by the measurement unit 120 and the operation screen 112. In other words, the three-dimensional position indicates where the user's hand is located in this three-dimensional space.

The shadow image generation unit 133 generates a shadow similar to the shape of the user's hand and fingers measured by the measurement unit 120. The position control unit 134 determines the position where the shadow is output on the operation screen 112 (hereinafter, the shadow output position). Note that the shadow output position is the coordinate where the shadow is output when the operation screen 112 is represented on the XY plane.

(Configuration of the display control unit 140)
The display control unit 140 outputs the shadow formed by the shadow image generation unit 133 and the shadow output position to the projection unit 111. The projection unit 111 outputs the shadow and the shadow output position to the operation screen 112.

(Processing flow of the non-contact input support device 10)
Based on the above-mentioned configuration, a specific process flow of the non-contact input support device 10 will be described. A user purchases a ticket. At this time, the user holds his/her hand over the operation screen 112. The measurement unit 120 measures three-dimensional information. Note that the measurement unit 120 continues to perform measurements until the user removes his/her hand from the operation screen 112 (ends screen operation) or completes the purchase of the ticket at the ticket vending machine 20.

The measurement unit 120 transmits the three-dimensional measurement results measured by the measurement unit 120 to the information acquisition unit 131. The information acquisition unit 131 acquires the measurement results. The information acquisition unit 131 outputs the measurement results to the positional relationship calculation unit 132.

The positional relationship calculation unit 132 calculates the three-dimensional position from the measurement results. The positional relationship calculation unit 132 outputs the measurement results and the three-dimensional position to the shadow image generation unit 133.

The shadow image generating unit 133 generates a shadow (two-dimensional image) of the user's hand using the measurement results and the three-dimensional position. The shadow image generating unit 133 also determines the size, shape, and shading of the shadow. A specific method for the shadow image generating unit 133 to determine the size and shading of the shadow will be described later. The shadow image generating unit 133 outputs the size, shape, shading, and three-dimensional position of the shadow to the position control unit 134.

The position control unit 134 uses the three-dimensional position to determine the shadow output position. In other words, the position control unit 134 determines the coordinates at which the shadow is output on the operation screen 112, which is an XY plane. The position control unit 134 outputs the shadow and the shadow output position to the display control unit 140.

The display control unit 140 determines where on the operation screen 112 the shadow should be displayed. The display control unit 140 outputs the size, shape, and shade of the shadow, as well as the shadow output position, to the projection unit 111. The projection unit 111 outputs the shadow to the operation screen 112.

The user moves their hand closer to a button displayed on the operation screen 112 while watching the movement of the shadow of their own hand displayed on the operation screen 112. The user operates (presses) the button displayed on the operation screen 112 to purchase a ticket.

This configuration allows the user to see the shadow of their hand on the operation screen 112. In other words, even when operating the operation screen 112, which has no physical form, the user can intuitively understand how to move their hand to the button by referring to the shadow of their hand.

Next, the specific process performed by the shadow image generating unit 133 when generating a shadow will be described with reference to Figures 3(A), 3(B), and 3(C).

Figure 3 (A) shows a state in which the user's hand 201 is separated from the operation screen 112. Figure 3 (B) shows a state in which the user's hand is approaching the operation screen 112. Figure 3 (C) shows a state immediately before the user touches a button on the operation screen 112.

First, the structure of the shadow 202A will be described with reference to Fig. 3(A). As shown in Fig. 3(A), the user's hand 201 is separated from the operation screen 112 by a distance d1. The shadow image generating unit 133 generates the shadow 202A of the user's hand 201. In order to determine the size and shade of the shadow, the shadow image generating unit 133 calculates the ratio of the distance d1 between the user's hand and the operation screen 112 to the distance d0 between the measuring unit 120 and the operation screen 112.

The structure of the shadow 202B will be described with reference to Fig. 3(B). As shown in Fig. 3(B), the user's hand 201 is separated from the operation screen 112 by a distance d2. The shadow image generating unit 133 generates a shadow 202B of the user's hand 201. The shadow image generating unit 133 calculates the ratio of the distance d2 between the user's hand and the operation screen 112 to the distance d0 between the measuring unit 120 and the operation screen 112, similar to Fig. 3(A).

Next, the structure of the shadow 202C will be described with reference to Fig. 3(C). As shown in Fig. 3(C), the user's hand 201 is in a state of almost contacting the operation screen 112 (distance d3 ≈ 0). The shadow image generating unit 133 generates a shadow 202C of the user's hand 201. As in Fig. 3(A), the shadow image generating unit 133 calculates the ratio of the distance d3 between the user's hand and the operation screen 112 to the distance d0 between the measuring unit 120 and the operation screen 112.

First, compare the states in FIG. 3(A) and FIG. 3(B). Comparing distance d1 and distance d2, there is a relationship of d1>d2. At this time, the shadow image generating unit 133 generates shadow 202B so that it is smaller than shadow 202A according to the above-mentioned ratio. Furthermore, the shadow image generating unit 133 generates shadow 202B so that the color density is darker than shadow 202A according to the ratio.

Next, the states in FIG. 3(B) and FIG. 3(C) are compared. Comparing distance d2 and distance d3, there is a relationship of d2>d3. At this time, the shadow image generating unit 133 generates shadow 202C so that it is smaller than shadow 202B according to the above-mentioned ratio. Also, the shadow image generating unit 133 generates shadow 202C so that it is approximately the same size as the user's hand 201 according to the ratio. Furthermore, the shadow image generating unit 133 generates the color intensity of shadow 202C so that it is darker than shadow 202B according to the ratio.

Comparing the sizes of shadow 202A, shadow 202B, and shadow 202C, shadow 202A > shadow 202B > shadow 202C. Also, comparing the color density (shade) of shadow 202A, shadow 202B, and shadow 202C, shadow 202A is the lightest, and it gets gradually darker from shadow 202B to shadow 202C. In this way, the shadow image generating unit 133 generates a smaller shadow with a darker color when the user's hand approaches the operation screen 112. Also, the shadow image generating unit 133 generates a larger shadow with a lighter color when the user's hand moves away from the projection unit 111. In other words, the shadow image generating unit 133 generates a shadow to have a shape similar to that of a shadow in nature.

As a result, when a user operates the non-contact input support device 10, the user can operate it in a manner that is almost the same as when the user actually uses the ticket vending machine 20. This reduces the sense of discomfort felt by the user when operating the non-contact input support device 10.

The process flow of the non-contact input support device 10 will be explained using the flowcharts in Figures 4 and 5. First, the process flow of the measurement unit 120 will be explained using Figure 4.

The measurement unit 120 detects whether the user's hand is held over the operation screen 112 (S101). If the measurement unit 120 detects the user's hand (S102: Yes), it continues measurement (S103).

The measurement unit 120 outputs the measurement result to the information acquisition unit 131 (S104). The measurement unit 120 returns to step S101 to continue measuring the operation screen 112. The measurement unit 120 repeats steps S101 to S104.

If the measurement unit 120 cannot detect the user's hand (S102: No), it stops the measurement (S105). In this case, the measurement unit 120 outputs the measurement result in which the shadow is not displayed to the information acquisition unit 131 (S106).

Next, the processing flow of the shadow generation unit 130 and the display control unit 140 will be explained using the flowchart in FIG. 5.

The information acquisition unit 131 acquires the measurement results from the measurement unit 120 (S111). If the measurement results include three-dimensional information (S112: Yes), the information acquisition unit 131 outputs the measurement results to the positional relationship calculation unit 132 (S113).

The positional relationship calculation unit 132 calculates the three-dimensional position from the measurement result. The positional relationship calculation unit 132 outputs the measurement result and the three-dimensional position to the shadow image generation unit 133 (S114).

The shadow image generating unit 133 generates a shadow of the user's hand using the measurement result and the three-dimensional position (S115). The shadow image generating unit 133 determines the size and shading of the shadow according to the three-dimensional position and ratio. The shadow image generating unit 133 outputs the shadow and the three-dimensional position to the position control unit 134.

The position control unit 134 uses the three-dimensional position to determine the shadow output position (S116). The position control unit 134 outputs the shadow and the shadow output position to the display control unit 140.

The display control unit 140 determines at what position on the operation screen 112 the shadow should be displayed based on the shadow and the shadow output position (S117). The display control unit 140 outputs the shadow and the shadow output position to the projection unit 111.

The projection unit 111 outputs a shadow on the operation screen 112 using the shadow and the shadow output position obtained from the display control unit 140 (S118).

If the measurement result does not include three-dimensional information (S112: No), in other words, if the measurement unit 120 cannot detect the user's hand, the information acquisition unit 131 outputs to the display control unit 140 to hide the shadow (S118). The display control unit 140 hides the shadow of the projection unit 111.

With this configuration, it is possible to provide a non-contact input support device that allows the user to intuitively operate the operation screen without needing detailed explanations. In addition, the size and shade of the shadow are changed according to the ratio of the distance between the user's hand and the operation screen 112 and the distance between the measurement unit 120 and the operation screen 112, so that the user can perform operations that are more in line with the real environment.

In the above configuration, an example of applying the non-contact input support device to a ticket vending machine has been described. However, the application is not limited to ticket vending machines, and can be applied to devices that use a display displayed in space, such as an aerial display (operation screen).

In the above configuration, an example has been described in which a shade is set for the shadow. However, the configuration may be such that the color of the shadow is changed instead of only the shade. For example, when the user touches a button on the operation screen 112, the color of the shadow may be changed to red.

In the above configuration, an example has been shown in which the measurement unit 120 measures the shape and orientation of the user's hand, and the distance between the operation screen 112 and the user's hand. However, the measurement unit 120 may be configured to detect the movement (motion) of the user's hand.

Specifically, as shown in FIG. 3(C), a configuration is shown in which the user operates a button displayed on the operation screen 112 when the user's hand 201 is almost in contact with the operation screen 112 (distance d3 ≒ 0). However, the measurement unit 120 may be configured to detect the action of bending the finger or pressing the button after bringing the hand 201 close to the button. In other words, even if the user's finger (hand 201) has not reached the operation screen 112 (there is a certain distance between the finger and the operation screen), the measurement unit 120 may be configured to determine that the user has operated the button.

Configuration example 2
Next, a specific configuration of the character input device will be described with reference to Fig. 6(A), Fig. 6(B), and Fig. 6(C). Fig. 6(A), Fig. 6(B), and Fig. 6(C) are diagrams showing the movement of the shadow of the non-contact input support device according to the second embodiment of the present invention.

The second embodiment differs from the first embodiment in the output position of the shadow by the shadow image generating unit 133 and the position control unit 134. Other points are the same as those in the first embodiment, and a description of similar points will be omitted.

In Fig. 6(A), Fig. 6(B), and Fig. 6(C), the user's hand 201 is almost in contact with the operation screen 112, similar to Fig. 3(C) in the first embodiment. In this case, the closer the user's hand 201 is to the operation screen 112, the more likely it is that the user's line of sight, the user's hand 201, and the shadow will be aligned in a straight line (aligned in one direction). In this case, the shadow image generation unit 133 and the position control unit 134 adjust the size of the shadow and the shadow output position.

A more specific method will be described with reference to Figs. 6(A), 6(B), and 6(C). The shadow image generating unit 133 outputs the shadow, which is two-dimensional image information, and the three-dimensional position to the position control unit 134. The position control unit 134 determines the shadow output position on the operation screen 112.

In FIG. 6(A), when the user's hand 201 is almost in contact with the operation screen 112, the position control unit 134 moves the shadow 203A along the X-axis. The distance moved along the X-axis should be within the range on the X-axis where the button to be operated is displayed, and should be any position where the user does not feel uncomfortable. In this way, the shadow image generation unit 133 can form a portion where the user's hand 201 and the shadow 203A do not overlap at the user's line of sight.

Similarly, in FIG. 6(B), when the user's hand 201 is almost in contact with the operation screen 112, the position control unit 134 moves the shadow 203B along the Y axis. The distance moved along the Y axis should be within the range on the Y axis where the button to be operated is displayed, and should be any position where the user does not feel uncomfortable. This allows the shadow image generation unit 133 to form a portion where the user's hand 201 and shadow 203B do not overlap at the user's line of sight.

Next, in FIG. 6(C), when the user's hand 201 is almost in contact with the operation screen 112, the shadow image generating unit 133 forms a shadow 203C larger than the hand 201. The size of the shadow 203C is enlarged by an arbitrary magnification from the user's hand 201. This magnification may be any value that does not cause the user to feel uncomfortable. This allows the shadow image generating unit 133 to form a portion where the user's hand 201 and the shadow 203C do not overlap at the user's line of sight.

This configuration also allows the user to operate the non-contact input support device without needing detailed instructions. In addition, the shadow is prevented from being completely obscured by the user's hand due to the user's line of sight. In other words, the user can easily recognize the position of their hand using the shadow.

Configuration example 3
Next, a specific configuration of the character input device will be described with reference to Fig. 7. Fig. 7 is a diagram showing the shadow of a non-contact input support device according to a third embodiment of the present invention.

The third embodiment differs from the first embodiment in the method of outputting a shadow by the shadow image generating unit 133 and the position control unit 134. Other points are the same as those in the first embodiment, and a description of similar points will be omitted.

The user operates a button on the operation screen 112, for example, using the index finger of the hand 201. The positional relationship calculation unit 132 calculates the three-dimensional position from the measurement result. The positional relationship calculation unit 132 outputs the measurement result and the positional relationship to the shadow image generation unit 133.

The shadow image generating unit 133 generates a shadow 204 using the measurement results and the three-dimensional position. At this time, the shadow image generating unit 133 displays the part corresponding to the user's finger (finger 205 in the shadow 204) in a darker color. In other words, the part of the shadow 204 where the user's hand is closest to the operation screen 112 (the index finger in this case) is displayed in a darker color. This allows the user to recognize which finger is about to perform the operation.

This configuration also makes it possible to provide a non-contact input support device that allows the user to intuitively operate the operation screen without the need for detailed explanations. In addition, because the finger operating the button is displayed in dark color, the user can recognize which part is closest to the button on the operation screen 112, thereby reducing operational errors.

Note that this invention is not limited to the above-described embodiment as it is, and in the implementation stage, the components can be modified to be more specific without departing from the gist of the invention. In particular, the order of the operational processing procedures in each device is not limited to the above-described order, and the order may be changed as necessary. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components from different embodiments may be appropriately combined.

Furthermore, the correspondence between the configuration according to the present invention and the configuration according to the above-described embodiment can be described as follows.
<Additional Notes>
A projection unit (111) that displays an operation screen (112) in the air that accepts user operations, a measurement unit (120) that measures the distance between the operation screen (112) and the user's hand that overlaps the operation screen (112),
a shadow generating unit (130) that generates a shadow of the hand from the result of measurement by the measuring unit (120);
a display control unit (140) that displays the shadow at a position overlapping with the hand on the operation screen (112);
A non-contact input support device (10) comprising:

d0, d1, d2, d3...distance 10...non-contact input support device 20...ticket machine 110...display unit 111...projection unit 112...operation screen 120...measurement unit 130...shadow generation unit 131...information acquisition unit 132...positional relationship calculation unit 133...shadow image generation unit 134...position control unit 140...display control unit 201...hands 202A, 202B, 202C, 203A, 203B, 203C, 204...shadow 205...finger

Claims (7)

a projection unit that displays in mid-air an operation screen that accepts operations by a user;
a measuring unit that measures a distance between the operation screen and the user's hand overlapping the operation screen;
a shadow generating unit that generates a shadow of the hand from the result of the measurement by the measuring unit and determines a position at which the generated shadow of the hand is to be displayed on the operation screen;
a display control unit that controls display of the shadow of the hand on the operation screen;
Equipped with
the shadow generating unit changes the color of the shadow of the hand depending on whether the distance between the user's hand and the operation screen measured by the measuring unit is longer or shorter than a set distance, and moves a position of the shadow of the hand displayed on the operation screen in a set direction when the user's hand is in contact with the operation screen.
Non-contact input support device. The non-contact input support device according to claim 1 , wherein the measurement unit measures the hand in three dimensions. The shadow generating unit includes:
a positional relationship calculation unit that calculates a three-dimensional position between the hand and the operation screen using a result of the measurement by the measurement unit;
a shadow image generating unit that generates a shadow of the hand similar to the shape of the hand by using the measurement result and the three-dimensional position;
a position control unit that determines a position at which the shadow of the hand is displayed on the operation screen,
The non-contact input support device according to claim 1 or 2 . The non-contact input support device according to claim 3 , wherein the shadow generating unit determines a shade of the shadow of the hand based on a ratio between a distance between the operation screen and the user's hand and a distance between the measuring unit and the operation screen. The non-contact input support device according to claim 3 , wherein the shadow generating unit determines a size of the shadow of the hand based on a ratio between a distance between the operation screen and the user's hand and a distance between the measuring unit and the operation screen. An information processing device,
a reception process of receiving operations from a user on an operation screen displayed in the air;
a measuring step of measuring a position of the user's hand overlapping the operation screen;
a generating step of generating a shadow of the hand from the measurement result and determining a position at which the generated shadow of the hand is to be displayed on the operation screen;
a display step of controlling display of the shadow of the hand on the operation screen;
Run
the generating step is a step of changing a color of the shadow depending on whether the distance between the user's hand and the operation screen measured in the measuring step is longer or shorter than a set distance, and moving a position of the shadow of the hand displayed on the operation screen in a set direction when the user's hand is in contact with the operation screen .
A method for supporting non-contact input. In the information processing device,
a reception process of receiving operations from a user on an operation screen displayed in the air;
a measuring step of measuring a position of the user's hand overlapping the operation screen;
a generating step of generating a shadow of the hand from the measurement result and determining a position at which the generated shadow of the hand is to be displayed on the operation screen;
a display step of controlling display of the shadow of the hand on the operation screen;
Run the command,
the generating step is a step of changing a color of the shadow depending on whether the distance between the user's hand and the operation screen measured in the measuring step is longer or shorter than a set distance, and moving a position of the shadow of the hand displayed on the operation screen in a set direction when the user's hand is in contact with the operation screen .
Non-contact input support program.

Images