JP7755201B2 - Head-mounted display device, authentication method, and authentication program - Google Patents

Description

The present invention relates to a head-mounted display device, an authentication method, and an authentication program.

User authentication may be required when starting up an information processing device or accessing a computer network. Only if this user authentication is successful will it be possible to use the information processing device or access the computer network.

User authentication typically requires entering a password via a keyboard. However, more secure user authentication methods have been developed to prevent password leaks caused by using a keyboard.

As a highly secure user authentication method, Patent Document 1 discloses a display system that detects the line of sight of a user gazing at an image displayed on a screen, detects the image the user is gazing at based on the line of sight, and performs user authentication using the detected image as authentication information.

Japanese Patent Application Laid-Open No. 2005-149326

In recent years, head mounted displays (HMDs), which are devices worn on the user's head, have become popular. While head mounted display devices are expected to use user authentication that detects the user's line of sight, as described in Patent Document 1, there is a demand for authentication with higher security.

The present invention was made in light of these circumstances, and aims to provide a head-mounted display device, authentication method, and authentication program that can perform authentication with higher security.

To solve the above problems, the head-mounted display device, authentication method, and authentication program of the present invention employ the following means.

To solve the above problems, one aspect of the present invention, a "head-mounted display device," is a head-mounted display device in which a display is mounted on the user's head so that it is visible to the user, and includes: a face direction detection means for detecting the direction of the user's face; a gaze detection means for detecting the user's gaze; an image display control means for changing an object displayed in an image display area of the display in accordance with the detection result by the face direction detection means; and an authentication object used for authentication is displayed in the image display area in accordance with the user's actions, and authentication means for performing authentication when the gaze detection means detects that the user has focused their gaze on the authentication object.

To solve the above problem, one aspect of the present invention is an "authentication method" for a head-mounted display device in which a display is worn on the user's head so that it is visible to the user, and which is equipped with a face direction detection means for detecting the direction of the user's face and a gaze detection means for detecting the user's gaze, and which includes a first step of changing an object displayed in an image display area of the display in accordance with the detection result of the face direction detection means, and a second step of displaying an authentication object used for authentication in the image display area in accordance with the user's movements, and performing authentication when the gaze detection means detects that the user has focused their gaze on the authentication object.

To solve the above problem, one aspect of the present invention, an "authentication program," includes a head-mounted display device mounted on a user's head so that the display is visible to the user, and includes face direction detection means for detecting the direction of the user's face and gaze detection means for detecting the user's gaze. The head-mounted display device functions as: image display control means for changing an object displayed in an image display area of the display in accordance with the detection results of the face direction detection means; and authentication means for displaying an authentication object used for authentication in the image display area in accordance with the user's movements and for performing authentication when the gaze detection means detects that the user has focused their gaze on the authentication object.

Various technical limitations may be imposed on the above-mentioned "head-mounted display device," as exemplified below. Furthermore, technical limitations of the same nature may be imposed on the processing steps executed by the "authentication method" and the functions of the "authentication program."

The authentication object is not displayed in the image display area when authentication begins, but is displayed in the image display area depending on the user's actions.

An authentication display area is set in the image display area, and the authentication means performs authentication when the authentication object is displayed in the authentication display area in response to the user's actions and the gaze detection means detects that the user has focused their gaze on the authentication object.

The user's action may be the user changing the direction of their face or the user moving the authentication object.

The authentication means performs authentication based on the detection results of the gaze detection means and the face direction detection means.

When a real, pre-registered identifying object is displayed on the display, the authentication object is displayed on the display.

The authentication object is a real-world identifier displayed on the display.

The authentication means determines that the user has started the authentication operation when it detects that the user has directed their gaze toward the authentication object for a predetermined period of time or longer, and that the user has ended the authentication operation when it detects that the user has averted their gaze from the authentication object for a predetermined period of time or longer.

The authentication means starts or ends the authentication operation when it detects the user blinking a predetermined number of times or more within a predetermined time period.

The authentication means starts or ends the authentication operation when it detects that the user has made a predetermined head movement.

The present invention has the advantage of enabling more secure authentication using a head-mounted display.

1 is a configuration diagram of a display system according to a first embodiment of the present invention. FIG. 2 is a schematic diagram showing an image display area of the HMD according to the first embodiment of the present invention. FIG. 2 is a functional block diagram showing the electrical configuration of the HMD according to the first embodiment of the present invention. FIG. 2 is a schematic diagram showing user authentication using an HMD according to the first embodiment of the present invention. FIG. 2 is a schematic diagram showing an authentication display area according to the first embodiment of the present invention. 4 is a flowchart showing the flow of user authentication processing according to the first embodiment of the present invention. FIG. 10 is a schematic diagram showing user authentication using an HMD according to a second embodiment of the present invention. FIG. 11 is a schematic diagram showing user authentication using an HMD according to a third embodiment of the present invention. FIG. 10 is a functional block diagram showing the electrical configuration of an HMD according to a third embodiment of the present invention. 11 is a flowchart showing the flow of user authentication processing according to the third embodiment of the present invention.

Below, an embodiment of a head-mounted display device, authentication method, and authentication program according to the present invention will be described with reference to the drawings.

1. First embodiment
A first embodiment of the present invention will be described below.

[1-1. Display System Configuration]
FIG. 1 is a diagram showing the configuration of a display system 10 according to this embodiment.

The display system 10 includes a head mounted display device (HMD) 12 and a server 14.

The HMD 12 is equipped with a display 16, such as an LCD (Liquid Crystal Display), mounted on the user's head so that it can be viewed by the user. The display 16 is fixed to the user's head by, for example, an elastic band 18.

The external shape of the HMD 12 shown in Figure 1 is an example, with the display 16 positioned in front of both eyes, but this is not limiting and the HMD 12 may be of a type that projects an image onto the lenses of glasses, like glasses. Furthermore, the image may not be viewed with both eyes, but rather with one eye. Furthermore, the HMD 12 may be either a transparent type that allows the user to view the outside world even when worn by the user, or a non-transparent type that does not allow the user to view the outside world. Furthermore, the image that the HMD 12 displays on the display 16 may be either virtual reality (VR) or augmented reality (AR). In the HMD 12 according to this embodiment, VR is displayed on the display 16.

The HMD 12 and server 14 are capable of sending and receiving information via a communication line 20. The HMD 12 may have a communication function for directly accessing the communication line 20, or it may access the communication line 20 via another information processing device, such as a mobile phone terminal.

The server 14 stores various information such as programs executed by the HMD 12, and sends and receives information to and from the HMD 12 as needed.

As shown in FIG. 2, the HMD 12 has an image display area 22 on the display 16 that covers the user's field of vision. Various images (also called objects) are displayed in the image display area 22. Note that the image display area 22 in this embodiment roughly corresponds to the field of vision of the user wearing the HMD 12.

The HMD 12 according to this embodiment performs user authentication when the user activates the HMD 12, and allows the user to operate the device if the user authentication is successful. An authentication object 24, which is an object used for user authentication, is displayed in the image display area 22. User authentication is then performed when a line-of-sight detection unit 32 (see Figure 3) provided in the HMD 12 detects that the user has focused their line of sight on the authentication object 24.

As an example, the authentication object 24 according to this embodiment is formed of multiple elements 26 (hereinafter referred to as "authentication elements"), at least one of which must be visually recognized to result in successful authentication. Furthermore, user authentication according to this embodiment is successful when the multiple authentication elements 26 are visually recognized in a predetermined order, but this is not limited to this. The order in which the multiple authentication elements 26 are visually recognized may not be predetermined, and the condition for successful authentication may be the length of time the authentication elements 26 are continuously visually recognized. Furthermore, the authentication object 24 may be a single image, rather than being formed of multiple authentication elements 26.

[1-2. Functional block diagram of HMD]
FIG. 3 is a functional block diagram showing the electrical configuration of the HMD 12 according to this embodiment.

In addition to the display 16, the HMD 12 includes a face direction detection unit 30, a gaze detection unit 32, a main control unit 34, a memory unit 36, and a communication unit 38.

The face direction detection unit 30 includes, for example, an acceleration sensor or a gyro sensor, and detects the direction of the face of the user wearing the HMD 12.

The gaze detection unit 32 is equipped with, for example, a camera that captures an image of the user's eyes, and detects the user's gaze based on the results of capturing an image of the user's eyes.

The main control unit 34 is, for example, a CPU (Central Processing Unit), microprocessor, DSP (Digital Signal Processor), etc., and controls the overall operation of the HMD 12.

The storage unit 36 is composed of, for example, RAM (Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory, and stores a work area when the main control unit 34 executes processes based on various programs, various data such as images, and programs used in processing by the main control unit 34.

The communication unit 38 has the function of connecting to the communication line 20, and is, for example, a NIC (Network Interface Controller), which connects to a communication network such as a mobile phone network. Note that instead of or in addition to the NIC, the communication unit 38 may also have the function of connecting to a wireless LAN (Local Area Network), a function of connecting to a wireless WAN (Wide Area Network), or a function that enables short-range wireless communication such as Bluetooth (registered trademark), infrared communication, etc.

The display 16, face direction detection unit 30, line of sight detection unit 32, main control unit 34, memory unit 36, and communication unit 38 are electrically connected to one another via a system bus. Therefore, the main control unit 34 can display images on the display 16, perform various processes using the detection results of the face direction detection unit 30 and line of sight detection unit 32, access the memory unit 36, and access various communication networks and other terminals via the communication unit 38. In addition to the above configuration, the HMD 12 is also equipped with a speaker and earphone jack for outputting sound, a USB (Universal Serial Bus) connector for sending and receiving data to and from other information processing devices, etc.

The HMD 12 can also be connected to a controller via wired or wireless connection, allowing the user to operate it using the controller.

The main control unit 34 includes an image display control unit 40 and an authentication processing unit 42. Each unit of the main control unit 34 is realized by a program stored in the memory unit 36.

The image display control unit 40 outputs data indicating the image (object) to be displayed in the image display area 22 to the display 16. The image display control unit 40 also changes the object displayed in the image display area 22 according to the detection result by the face direction detection unit 30.

The authentication processing unit 42 performs processing related to user authentication.

[1-3. User authentication according to the first embodiment]
User authentication according to this embodiment will be described.

In this embodiment, user authentication is performed when an authentication object 24 is displayed in the image display area 22 in response to the user's actions, and the gaze detection unit 32 detects that the user has focused their gaze on the authentication object 24.

Figure 4 is a schematic diagram showing user authentication according to this embodiment, and shows a schematic representation of the image space 44 displayed by the HMD 12. Since the HMD 12 according to this embodiment displays virtual reality (VR), the image space 44 shown in Figure 4 is a VR space. The area surrounded by a dashed line in Figure 4 is the entire image space 44 that can be viewed by a user wearing the HMD 12 by moving their head. On the other hand, the area surrounded by a solid line is the image space 44 displayed in the image display area 22, and is the range that is actually viewed by a user wearing the HMD 12.

As shown in Figure 4(A), the authentication object 24 exists in the image space 44, but is not displayed in the image display area 22 when user authentication begins. Therefore, the user can turn their face to cause the authentication object 24, which exists in the image space 44, to be displayed in the image display area 22, as shown in Figure 4(B).

Furthermore, in user authentication according to this embodiment, an authentication display area 22A is set in the image display area 22. As shown in FIG. 5, the image display area 22 is virtually divided into an authentication display area 22A and a non-authentication display area 22B. User authentication is then performed when an authentication object 24 is displayed in the authentication display area 22A.

When the authentication object 24 is displayed in the image display area 22, a display may be displayed to prompt the user to display the authentication object 24 in the authentication display area 22A. For example, the HMD 12 may flash the authentication display area 22A, display letters or arrows, or make an audio announcement to prompt the user to place the authentication object 24 into the authentication display area 22A.

Furthermore, the division of the authentication display area 22A shown in FIG. 5 is one example, and other divisions are also possible. Furthermore, the image display area 22 may not be provided with a non-authentication display area 22B, and the authentication display area 22A may be the entire image display area 22.

The HMD 12 then determines whether authentication has been successful based on whether the results of gaze detection by the gaze detection unit 32 match the registered authentication gaze information. The authentication gaze information is generated when the user views the authentication object 24 in order to register it, and is pre-stored in the memory unit 36.

[1-4. Determining the Start or End of Authentication Operation]
Next, the start or end of the authentication operation by the user will be described.

The authentication processing unit 42 determines that the authentication operation has started when the user performs a predetermined operation on the controller connected to the HMD 12, and that the authentication operation has ended when the user performs another predetermined operation on the controller. An example of a predetermined operation on the controller is clicking a predetermined button.

The gaze detection unit 32 then begins detecting the user's gaze after the authentication processing unit 42 receives the user's start of the authentication operation. The gaze detection unit 32 then ends the user's gaze detection after the authentication processing unit 42 receives the user's end of the authentication operation. The authentication processing unit 42 then determines whether user authentication has been successful based on the gaze detection results of the gaze detection unit 32.

Note that the start or end of the authentication operation by the user is not limited to a predetermined operation on the controller, and may also be determined based on the detection results of the line of sight detection unit 32 or face direction detection unit 30.

For example, the authentication processing unit 42 determines that the user's authentication operation has begun when the gaze detection unit 32 detects that the user has directed their gaze toward the authentication object 24 for a predetermined period of time or longer. The authentication processing unit 42 then determines that the user's authentication operation has ended when the gaze detection unit 32 detects that the user has averted their gaze from the authentication object 24 for a predetermined period of time or longer. Note that the predetermined period of time is not the period during which the user unconsciously directs or averts their gaze, but a period of time during which it can be determined that the user did so intentionally, such as a few seconds.

Furthermore, the authentication processing unit 42 starts or ends the authentication operation when the gaze detection unit 32 detects that the user has blinked a predetermined number of times or more within a predetermined time. A predetermined number of times or more within a predetermined time is a number that can be determined to be intentional blinks by the user, rather than blinks that the user performs unconsciously, such as three or more blinks within one second.

The authentication processing unit 42 also starts or ends the authentication operation when the face direction detection unit 30 detects that the user has made a predetermined head movement. The predetermined movement is, for example, a predetermined number of times in a predetermined direction within a predetermined time, which is not an unconscious movement made by the user, but a number of times that can be determined to be an intentional movement made by the user, such as tilting the head to the right twice in one second.

Furthermore, the authentication processing unit 42 may determine that the user's authentication operation has begun when the gaze detection unit 32 detects that the user has directed their gaze at a predetermined object displayed on the display 16 for a predetermined period of time or longer, and may determine that the user's authentication operation has ended when the gaze detection unit 32 again detects that the user has directed their gaze at the predetermined object for a predetermined period of time or longer.

The start or end of any authentication operation is determined when the authentication object 24 is displayed in the authentication display area 22A. Furthermore, when the HMD 12 receives an action by the user indicating the start or end of the authentication operation, it issues a notification to that effect (hereinafter referred to as an "authentication start notification" or "authentication end notification"). The authentication start notification or authentication end notification may be, for example, the display of a specified image on the display 16 or the output of a specified sound.

When the authentication start notification is given, the user knows that they can now perform the authentication operation. Furthermore, when the authentication completion notification is given, the user knows that the HMD 12 has accepted the authentication operation they performed.

In this way, the HMD 12 according to this embodiment accepts the start or end of the authentication operation when the user performs a predetermined action. This allows the user to perform user authentication at their own timing, and prevents erroneous detection of the line of sight during user authentication.

[1-5. User authentication process according to this embodiment]
6 is a flowchart showing the flow of user authentication processing executed by the main control unit 34. A program for executing the user authentication processing is stored in advance in a predetermined area of the storage unit 36. Note that the user authentication processing according to this embodiment is executed, for example, when the user starts up the HMD 12.

First, in step 100, it is determined whether the authentication object 24 is displayed in the authentication display area 22A, and if the determination is affirmative, the process proceeds to step 102. Note that when the HMD 12 is started up, the authentication object 24 is not displayed in the authentication display area 22A. Therefore, the user moves their head, for example, to cause the authentication object 24 to be displayed in the authentication display area 22A.

In step 102, the authentication processing unit 42 determines whether the user has performed a predetermined action indicating the start of the authentication operation, and if the determination is affirmative, proceeds to step 104. If the determination is affirmative, the authentication processing unit 42 issues a notification that authentication has started.

In step 104, the gaze detection unit 32 performs gaze detection for user authentication.

In the next step 106, the authentication processing unit 42 determines whether the user has performed a predetermined action indicating that the authentication operation is to be terminated. If the determination is affirmative, the process proceeds to step 108; if the determination is negative, the process returns to step 104. If the determination is affirmative, the authentication processing unit 42 notifies the user that authentication has been terminated.

In step 108, the authentication processing unit 42 performs authentication based on the results of the gaze detection, and then terminates the user authentication process. Specifically, the success or failure of authentication is determined based on whether the results of gaze detection on the authentication object 24 match the registered authentication gaze information. If authentication is successful, the user is able to operate the HMD 12. On the other hand, if user authentication fails, the process returns to step 100 and the user authentication process is performed again.

As described above, the HMD 12 according to this embodiment includes a face direction detection unit 30 that detects the direction of the user's face, and a gaze detection unit 32 that detects the user's line of sight, and changes the object displayed in the image display area 22 of the display 16 according to the detection results of the face direction detection unit 30. The HMD 12 then displays an authentication object 24 in the image display area according to the user's movements, and performs user authentication when the gaze detection unit 32 detects that the user has focused their gaze on the authentication object 24.

In this way, the HMD 12 according to this embodiment locates the authentication object 24 when the user moves their head, and displays the authentication object 24 on the display 16. The HMD 12 then performs user authentication by detecting the user's line of sight aligned with the displayed authentication object 24, thereby enabling authentication with a higher level of security.

The authentication object 24 may also be made virtually movable by the user. In this case, for example, when the authentication object 24 is displayed in the non-authentication display area 22B, the user virtually grasps the authentication object 24 with their hand and moves the authentication object 24 to the authentication display area 22A. The operation of grasping the authentication object 24 may be performed via a controller connected to the HMD 12.

Furthermore, multiple authentication objects 24 may exist in the image space 44. In this case, only one of the multiple authentication objects 24 is pre-registered as the object to be used for user authentication. The user moves their head to find one of the registered authentication objects 24 and have it displayed on the display 16, thereby performing user authentication.

The authentication object 24 may also move. In this case, for example, multiple authentication objects 24 are displayed on the display 16 while moving. The user moves their head to find one of the registered authentication objects 24 and have it displayed on the display 16, thereby performing user authentication.

[2. Second embodiment]
A second embodiment of the present invention will now be described.

[2-1. Configuration of the HMD according to the second embodiment]

The configuration of the display system 10 and HMD 12 according to this second embodiment is similar to the configuration of the display system 10 and HMD 12 according to the first embodiment shown in Figures 1 to 3, and therefore will not be described further.

[2-2. User authentication according to the second embodiment]
The authentication processing unit 42 according to this embodiment performs user authentication based on the result of gaze detection by the gaze detection unit 32 and the result of face direction detection by the face direction detection unit 30. In other words, the condition for successful user authentication is that the user looks at the authentication object 24 and that the direction of the face is a pre-registered direction.

Pre-registered directions are, for example, up, down, right, and left. In the example of Figure 7, authentication is successful when the face direction detected by the face direction detection unit 30 and the gaze direction detected by the gaze detection unit 32 are opposite to each other.

This enables the HMD 12 of this embodiment to provide more secure user authentication.

3. Third embodiment
A third embodiment of the present invention will now be described.

[3-1. Configuration of the HMD according to the third embodiment]

The configurations of the display system 10 and HMD 12 according to this embodiment are similar to the configurations of the display system 10 and HMD 12 according to the first embodiment shown in FIGS. 1 and 2, and therefore a description thereof will be omitted.

The image displayed on the display 16 by the HMD 12 according to this embodiment is AR. For this reason, a camera 56, which will be described in detail later, is provided on the front of the HMD 12 and captures an image (hereinafter referred to as a "real space image") of the surroundings of the user wearing the HMD 12 (hereinafter referred to as the "real space"). AR is realized by displaying objects on the display 16 along with the real space image.

[3-2. User authentication according to the third embodiment]
8 , when a pre-registered identifying object (hereinafter referred to as a “registered identifying object”) 50 is displayed on the display 16, the HMD 12 according to this embodiment displays an authentication object 24 on the display 16. The registered identifying object 50 is, for example, an actual object, and characteristic parts such as shape and color for identifying the registered identifying object 50 are stored in the storage unit 36. The characteristic parts of the registered identifying object 50 are extracted by performing image analysis on a real-space image acquired when registering the registered identifying object 50.

In the example of FIG. 8 , the registered identification object 50 is a monitor 54 placed on a desk 52, and when this monitor 54 is displayed on the display 16, the authentication object 24 is displayed on the display 16. In other words, the initial position of the authentication object 24 is predetermined in real space. That is, the authentication object 24 does not exist in the image space until the registered identification object 50 is displayed in the authentication display area 22A. Therefore, if the registered identification object 50 is not displayed on the display 16, the user can move their head to cause the registered identification object 50 to be displayed on the display 16.

Note that the registered identification object 50 shown in Figure 8 is just one example, and it may be another object such as a clock or window, but an object whose position does not change in real space is preferable. Also, in the example of Figure 8, the authentication object 24 is displayed superimposed on the registered identification object 50, but it does not necessarily have to be superimposed.

When the authentication object 24 is displayed on the display 16, the user moves their head so that the authentication object 24 is displayed in the authentication display area 22A. After the authentication object 24 is displayed in the authentication display area 22A, the user performs user authentication by aligning their line of sight with the authentication object 24.

Note that if the user moves their head after the authentication object 24 is displayed on the display 16, causing the registered identification object 50 to disappear from the display 16, the HMD 12 may either hide the authentication object 24 that was displayed, or continue to display the authentication object 24, thereby enabling user authentication.

Figure 9 is a functional block diagram showing the electrical configuration of the HMD 12 according to this embodiment. Note that components in Figure 9 that are the same as those in Figure 3 are given the same reference numerals as in Figure 3, and their description will be omitted.

The camera 56 is mounted on the front of the HMD 12 and captures real-space images of the user's surroundings.

The image display control unit 40 displays the real-space image acquired by the camera 56 on the display 16 in real time, and also displays a non-existent image (object) superimposed on the real-space image on the display 16.

The camera image analysis unit 58 identifies the object captured by the camera 56 by performing image analysis on the real-space image. In other words, the real-space image is subjected to image analysis while being displayed on the display 16.

The authentication processing unit 42 determines whether the object identified by the camera image analysis unit 58 matches the registered identification object 50 by image matching or the like. That is, if the real-space image contains an object that matches the registered identification object 50, the authentication processing unit 42 determines that the registered identification object 50 is being displayed on the display 16, and causes the authentication object 24 to be displayed on the display 16.

Figure 10 is a flowchart showing the flow of user authentication processing according to this embodiment. Note that steps in Figure 10 that are the same as those in Figure 6 are given the same reference numerals as in Figure 6, and their explanations will be omitted in part or in whole.

First, in step 200, the authentication processing unit 42 determines whether the registered identification object 50 is displayed on the display 16, in other words, whether the registered identification object has been imaged by the camera 56, and if the determination is affirmative, the process proceeds to step 202. Note that if the HMD 12 is a see-through type and the determination is affirmative in step 200, the actual registered identification object 50 has been imaged in a predetermined area (e.g., the central area) of the camera 56's imaging range, and the user is directly viewing the registered identification object 50 that has passed through the display 16. In other words, if the HMD 12 is a see-through type, the image captured by the camera 56 does not need to be displayed on the display 16.

In step 202, the authentication processing unit 42 displays the authentication object 24 on the display 16 and proceeds to step 100.

Steps 100 to 108 are the same as the user authentication process in the first embodiment. If user authentication fails, the process returns to step 200 and the user authentication process is performed again.

As described above, the HMD 12 according to this embodiment realizes AR by displaying objects along with a real-space image on the display 16, and when a registered identification object 50 is displayed on the display 16, the authentication object 24 is displayed on the display 16.

This enables the HMD 12 of this embodiment to provide more secure user authentication.

Note that multiple registered identification devices 50 may be registered in advance, and the registered identification device 50 used for user authentication may be changed for each user authentication or every specified number of days. When the registered identification device 50 used for user authentication changes, information about the changed registered identification device 50 is notified to the user's pre-registered email address, etc.

Furthermore, the registered identification object 50 is not limited to an object such as the monitor 54, but may also be, for example, an identification code such as a barcode or QR code (registered trademark), or a predetermined image. These identification codes and images may be printed on paper or the like, or may be displayed on a computer monitor, a mobile information terminal, or the like.

Furthermore, the registered identification object 50 may not be a real object, but a virtual object displayed on the display 16. In this case, the user moves to the registered identification object 50 within the VR space and displays it on the display 16, or holds the registered identification object 50 within the VR space and displays it on the display 16.

Furthermore, the real or virtual registered identification object 50 itself may be the authentication object 24. In this case, user authentication is performed when the registered identification object 50 is displayed in the authentication display area 22A. Furthermore, if the HMD 12 is a see-through type, user authentication may be performed when an image of the real registered identification object 50 is captured in a predetermined area (e.g., the central area) of the imaging range of the camera 56.

4. Other Embodiments
Although the present invention has been described above using the above-mentioned embodiments, the technical scope of the present invention is not limited to the scope described in the above-mentioned embodiments. Various modifications and improvements can be made to the above-mentioned embodiments without departing from the gist of the invention, and modifications and improvements are also included in the technical scope of the present invention. Furthermore, the above-mentioned embodiments may be combined as appropriate.

For example, in each of the above embodiments, a configuration in which the HMD 12 includes the authentication processing unit 42 has been described, but the present invention is not limited to this, and a configuration in which the server 14 includes the authentication processing unit 42 may also be adopted. In this configuration, the detection results detected by the HMD 12 for user authentication are transmitted to the server 14, and the server 14 performs user authentication based on the received detection results and transmits the authentication results to the HMD 12.

In addition, in the above embodiments, the authentication according to the present invention has been described as being applied to user authentication performed when the HMD 12 is started up, but the present invention is not limited to this and may be applied, for example, to user authentication performed when software executed on the HMD 12 is started up. It may also be applied to authentication within a game executed on the HMD 12. In a form applied to game authentication, the HMD 12 may perform authentication when it detects that the line of sight has been aligned with the authentication object 24, and a predetermined event may occur if the authentication is successful.

Furthermore, in the first and second embodiments, a form in which the HMD 12 displays VR has been described, but the present invention is not limited to this, and the HMD 12 may also display AR.

Furthermore, the user authentication processing flow described in each of the above embodiments is merely an example, and unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the present invention.

12 HMD (Head Mounted Display Device)
16 Display 22 Image display area 22A Authentication display area 24 Authentication object 30 Face direction detection unit (face direction detection means)
32 Line of sight detection unit (line of sight detection means)
40 Image display control unit (image display control means)
42 Authentication processing unit (authentication means)

Claims (4)

a camera, a display, and a processor;
the processor causes the display to superimpose an object associated with a position in the real space onto an image of the real space captured by the camera;
the processor, before user authentication, places an authentication object used for user authentication outside a display area of the display;
the processor enables a user authentication operation when the authentication object and a registered identification object present in the real space are displayed in a predetermined area in a display area of the display;
when the authentication object is displayed in the display area of the display, the processor executes an announcement to prompt a user to display the authentication object within the predetermined area.
Device. A device comprising a camera, a display, and a processor,
superimposing an object associated with a position in the real space on an image captured by the camera and displaying the image on the display;
Before user authentication, an authentication object used for user authentication is placed outside a display area of the display;
enabling a user authentication operation when the authentication object and a registered identification object existing in real space are displayed in a predetermined area in a display area of the display;
When the authentication object is displayed in the display area of the display, an announcement is made to prompt the user to display the authentication object within the predetermined area.
A program for executing a process. 1. An apparatus comprising a camera, a display, and a processor,
superimposing an object associated with a position in the real space on an image captured by the camera and displaying the image on the display;
Before user authentication, an authentication object used for user authentication is placed outside a display area of the display;
enabling a user authentication operation when the authentication object and a registered identification object existing in real space are displayed in a predetermined area in a display area of the display;
When the authentication object is displayed in the display area of the display, an announcement is made to prompt the user to display the authentication object within the predetermined area.
Information processing methods. A device including a camera and a display, and a server,
The server
superimposing an object associated with a position in the real space on an image captured by the camera and displaying the image on the display;
Before user authentication, an authentication object used for user authentication is placed outside a display area of the display;
enabling a user authentication operation when the authentication object and a registered identification object existing in real space are displayed in a predetermined area in a display area of the display;
The device comprises:
When the authentication object is displayed in the display area of the display, an announcement is made to prompt the user to display the authentication object within the predetermined area.
system.