JP7691290B2 - Information processing system, information processing method, and program - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act On April 12, 2021, TIS Inc. published an information processing system and information processing method (XR Pay, an example of how to use XR Pay) on its website at the following URL: https://www.tis.co.jp/news/2021/tis_info/20210412_1.html.

The present invention relates to an information processing system, an information processing method, and a program.

A system has been disclosed that allows users to complete the purchasing process for products within a virtual space (Patent Document 1).

JP 2019-79439 A

The system described in Patent Document 1 is configured to allow a user to perform actions in a virtual space, such as adding an object to a virtual cart (cart-in) or expressing a favorable impression of a product (like). The system also uses a specific gesture, such as bending the user's index finger twice in a timely manner, as a trigger to initiate an action such as cart-in or "like," allowing the purchase of a product. However, in order to ensure a deeper immersion of the user in the virtual space, there is room for improvement in the action that triggers the user to select a product and the process of detecting that action.

Therefore, in order to solve the above problems, the present invention aims to provide a system that allows a user to complete product selection operations more intuitively in an augmented reality space or a virtual reality space.

An information processing system according to one aspect of the present invention includes a display processing unit that executes processing for displaying, on a display unit of the display device, an instruction object for indicating a product object displayed in three dimensions in the augmented reality space or the virtual reality space, which moves in response to hand movements of a user wearing a display device that displays an augmented reality space or a virtual three-dimensional space, and, if there is a designated product object that is a product object that overlaps or intersects with the instruction object, executes processing for displaying the designated product object closer to the user by a pinch operation by the user; and a payment unit that completes an operation related to payment for a selected product, which is a product corresponding to the designated product object selected by a selection operation by the user, in response to a selection operation by the user on a user interface related to a payment means for making payment for the selected product.

In an information processing method according to one aspect of the present invention, a computer executes a process for displaying, on a display unit of the display device, an instruction object for indicating a product object displayed in three dimensions in the augmented reality space or the virtual reality space, which moves in response to a hand movement of a user wearing a display device that displays an augmented reality space or a virtual three-dimensional space, and, if there is a designated product object that is a product object that overlaps or intersects with the instruction object, executes a process for displaying the designated product object closer to the user by a pinch operation by the user, and completes an operation related to payment for a selected product that is a product corresponding to the designated product object selected by a selection operation by the user, in response to a selection operation by the user on a user interface related to a payment means for making payment for the selected product.

A program according to one aspect of the present invention causes a computer to execute a process for displaying, on a display unit of the display device, an instruction object for indicating a product object displayed in three dimensions in the augmented reality space or the virtual reality space, the instruction object moving in response to hand movements of a user wearing a display device that displays an augmented reality space or a virtual three-dimensional space, and, if there is a designated product object that is a product object that overlaps or intersects with the instruction object, execute a process for displaying the designated product object closer to the user by a pinch operation by the user, and complete an operation related to payment for a selected product that is a product corresponding to the designated product object selected by a selection operation by the user, in response to a selection operation by the user on a user interface related to a payment means for making payment for the selected product.

The present invention enables product selection in a virtual space using gestures that are more similar to product selection actions in the real world, providing a more intuitive product purchasing experience in a virtual space and offering a deeper sense of immersion to the user.

FIG. 1 is a diagram showing an example of the configuration of a virtual space payment system 10. A diagram showing an example of an overview of the processing of the virtual space payment system 10. FIG. 2 is a diagram showing an example of the configuration of a wallet providing server 100. FIG. 11 is a diagram showing the configuration of user information D111. FIG. 13 is a diagram showing the configuration of retail store information D112. FIG. 11 is a diagram showing the configuration of UI information D113. FIG. 13 is a diagram showing an example of a store identification UI in a virtual space. FIG. 13 is a diagram showing an example of a product confirmation UI in a virtual space. FIG. 13 is a diagram illustrating an example of a product selection UI in a virtual space. FIG. 13 is a diagram showing another example of a product selection UI in a virtual space. FIG. 11 is a diagram showing an example of a menu UI in a virtual space. A figure showing an example of a first wallet UI in a virtual space. A figure showing an example of a second wallet UI in a virtual space. A figure showing an example of a third wallet UI in a virtual space. A figure showing an example of a fourth wallet UI in a virtual space. A figure showing another example of the fifth wallet UI in the virtual space. FIG. 13 is a diagram showing an example of a purchase product confirmation UI in a virtual space. FIG. 2 is a diagram showing an example of the configuration of a virtual space management server 200. FIG. 2 is a diagram showing an example of store space information D211. FIG. 13 is a diagram showing an example of product placement information D212. FIG. 2 is a diagram showing an example of product information D213. FIG. 13 is a diagram showing an example of action information D214. FIG. 2 illustrates an example of the configuration of a user terminal 300. FIG. 2 is a flow diagram showing an example of processing of the virtual space payment system 10. FIG. 2 illustrates an example of a hardware configuration of a computer.

The virtual space payment system 10 according to one embodiment of the present invention will be described in detail below with reference to the drawings. However, the embodiment described below is merely an example, and is not intended to exclude various modifications or application of techniques not specifically described below. In other words, the present invention can be implemented by modifying it in various ways or combining each embodiment without departing from the spirit of the invention. In addition, in the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.
====Overview of Virtual Space Payment System 10===

The virtual space payment system 10 is a system that allows a user to complete the purchase procedure for a product or the like in an augmented real space (hereinafter referred to as "augmented reality space") or a virtual three-dimensional space (hereinafter referred to as "virtual reality space") projected onto a terminal device such as goggles worn by the user, without losing the sense of immersion in the space. Hereinafter, for convenience, augmented reality space and virtual reality space may be collectively referred to as "virtual space." In other words, the virtual space payment system 10 allows a user to complete the purchase procedure for a 3D avatar without having the user leave the virtual space (for example, without removing a terminal device such as goggles). In this way, the virtual space payment system can prevent the user from losing the sense of immersion in the virtual space.

The augmented reality space is, for example, a space that includes images that are displayed as if projected onto real space by a user wearing a terminal device that is a transparent head-mounted display. The virtual reality space is, for example, a virtual space that is configured with images displayed on a terminal device that is a non-transparent head-mounted display worn by the user. Note that the virtual reality space includes so-called mixed reality space (a space in which real-world information is reflected in a digital space). The image displayed on the terminal device is, for example, an image that the user can view as a three-dimensional image. Hereinafter, among the images displayed on the terminal device, an object that the user can view in the virtual space as if it were an actual object may be referred to as a "virtual object".

First, referring to FIG. 1, an overview of the configuration of the virtual space payment system 10 will be described. FIG. 1 is a diagram showing an example of the configuration of the virtual space payment system 10. As shown in FIG. 1, the virtual space payment system 10 may include, for example, a wallet providing server 100, a virtual space management server 200, a user terminal 300, and a retail store terminal 400.

The wallet providing server 100 is a server that processes information for enabling a user to carry out a purchasing procedure in a virtual space. The wallet providing server 100 processes information for a retailer to open a store in a virtual space (e.g., information about virtual objects of the store, information about virtual objects of products, etc.), and processes information about a user interface for enabling a user to carry out a purchasing procedure in a virtual space (e.g., information about virtual objects related to payment). The wallet providing server 100 may provide the information for opening a store and the information about the user interface to the virtual space management server 200, which will be described later, in a software development kit (hereinafter referred to as "SDK").

As a result, the virtual space payment system 10 can easily enable retailers to open stores and users to complete purchase procedures in the virtual space. Furthermore, businesses that operate virtual spaces can easily use the SDK to enable retailers to open stores and users to complete purchase procedures, thereby attracting retailers and encouraging consumer participation. As a result, the virtual space payment system 10 can provide the business with the economic benefit of being able to expand its business area.

The virtual space management server 200 is, for example, a server for providing a virtual space to a user. The user terminal 300 provides a virtual space to a user based on information provided by the virtual space management server 200. The virtual space management server 200 causes the user terminal 300 to display a predetermined user interface based on information related to the user interface provided by the wallet providing server 100, for example.

The user terminal 300 is, for example, a non-transparent head-mounted display or a transparent head-mounted display, and is a device that displays virtual objects in a virtual space so that the user can see them. The user terminal 300 has a function of recognizing the user's actions.

The retail store terminal 400 is, for example, a device that transmits information about a store opened in a virtual space and the products to be displayed. The retail store terminal 400 generates various information by accepting operational input from, for example, the manager of the retail store.

In the above description, the wallet providing server 100 and the virtual space management server 200 are described as being provided separately, but this is not limited to the above. The wallet providing server 100 and the virtual space management server 200 may be configured as a single server or may be a cloud server. Furthermore, the wallet providing server 100 may be configured as multiple servers divided according to function, for example.

Next, referring to FIG. 2, an overview of the processing of the virtual space payment system 10 will be described. FIG. 2 is a diagram showing an example of the overview of the processing of the virtual space payment system 10. First, in step S1, the retail store terminal 400 accepts, for example, a user's operation input to a web page provided by the wallet providing server 100, and transmits information about the store (hereinafter referred to as "store information") and information about the product (hereinafter referred to as "product information") to the wallet providing server 100. Although not shown here, the wallet providing server 100 may, for example, provide the store information and product information to a business that creates three-dimensional data, and thereby obtain three-dimensional data of the store and the product from the business. Note that the wallet providing server 100 may, or may not, automatically obtain three-dimensional data of the store and the product from the business by providing the store information and product information to the business.

Next, in step S2, the wallet providing server 100 transmits, for example, store information and product information to the electronic commerce server 500. This enables the wallet providing server 100 to transfer funds to a retail store when the user performs a procedure in the virtual space to purchase a product provided by the retail store.

Next, in step S3, the wallet providing server 100 transmits, for example, store information, product information, and three-dimensional data of the store and products to the virtual space management server 200. This allows retailers to open stores in virtual space and display products with simple operational input. This eliminates the barrier that retailers have to prepare three-dimensional data of the store and products when opening stores in virtual space.

Next, in step S4, the user terminal 300 installs, for example, application software (hereinafter referred to as "virtual space APP") for displaying the virtual space on the display unit 1007 of the user terminal 300. Note that the user terminal 300 may display the virtual space on the display unit 1007 through a web browser, without using the virtual space APP.

Next, in step S5, the user terminal 300 installs application software (hereinafter referred to as "wallet APP") that allows the user to use a payment method to purchase products in the virtual space, for example.

Next, in step S6, the user terminal 300 transmits information about the user (hereinafter referred to as "user information") acquired based on user operation input via the wallet APP to the wallet providing server 100. The user information may include, for example, information about the payment method used by the user (hereinafter referred to as "payment method information").

Next, in step S7, the wallet providing server 100 identifies a user interface (hereinafter referred to as a "wallet UI") corresponding to the payment method used by the user, for example, based on the user information. The wallet providing server 100 transmits the user information and information regarding the wallet UI to the virtual space management server 200. This allows the user to purchase a product in the virtual space using the desired payment method.

Next, in step S8, the user terminal 300 accepts user input, for example, to add a product from a store in the virtual space to the cart. The user terminal 300 accepts user input, for example, to select a product in the cart, and executes the purchase procedure. For convenience, the operations performed by the user from selecting a product to executing the purchase procedure will be referred to as a "purchase operation" below, and information for executing payment through the purchase operation will be referred to as "payment execution information." The user interface provided to the user through the user terminal 300 during the purchase operation will be described later.

Next, in step S9, the virtual space management server 200 transmits information about the purchased product (hereinafter referred to as "purchase information") to the wallet providing server 100. The purchase information includes, for example, information about the product that is the subject of the user's purchase operation. Next, in step S10, the wallet providing server 100 transmits the purchase information to the electronic commerce server 500. This allows the user to complete the purchase procedure without removing the user terminal 300.

Next, in step S11, the electronic commerce server 500 executes a transfer process for the sales proceeds of the product for which the user has completed the purchase procedure to the retail store. This allows the retail store to obtain the sales proceeds in the virtual space.

In this way, in the virtual space payment system 10, even if there are a plurality of virtual space management servers 200, each virtual space management server 200 can transmit purchase information to an appropriate electronic commerce server 500 via the wallet providing server 100. In other words, the user can complete a transaction, for example, within a VR (Virtual Reality) SNS (Social Networking Service) without removing the user terminal 300, thereby eliminating the problem of the user losing their sense of immersion in the virtual space.
===Configuration of Wallet Providing Server 100===

Next, the configuration of the wallet providing server 100 will be described with reference to FIG. 3. FIG. 3 is a diagram showing an example of the configuration of the wallet providing server 100. As shown in FIG. 3, the wallet providing server 100 includes functional units, for example, a storage unit 110, an acquisition unit 120, a UI information providing unit 130, a display processing unit 140, a payment unit 150, and a product classification unit 160. Each functional unit is a function realized, for example, by the processor 1001 reading out a program stored in the memory 1002.

The memory unit 110 stores user information D111, retail store information D112, and UI information D113.

First, the user information D111 will be described with reference to FIG. 4. FIG. 4 is a diagram showing the configuration of the user information D111. As shown in FIG. 4, the user information D111 may include, for example, the following items: [User ID], [Name], [Email address], [Address], [Telephone number], [Payment method], [Purchased product], and [UI ID]. [User ID] stores a user ID that can uniquely identify a user. [Name] stores the user's name. [Email address] stores the email address used by the user. [Address] stores the user's current address. [Telephone number] stores the user's telephone number. [Payment method] stores the payment method used by the user in the virtual space. Here, the user with user ID "U0001" has registered credit card A, bank account B, virtual currency C, and electronic money D as payment methods. The wallet providing server 100 generates information about the wallet UI associated with the user with user ID "U0001". [Purchased product] stores data about products that the user has previously purchased in the virtual space. [UIID] stores a UIID that can uniquely identify each UI.

Next, the retail store information D112 will be described with reference to FIG. 5. FIG. 5 is a diagram showing the configuration of the retail store information D112. As shown in FIG. 5, the retail store information D112 may include items such as [Retail Store ID], [Retail Store Name], [Email Address], [Address], [Telephone Number], [Store 3D Data], [Product 3D Data], and [Sales History]. [Retail Store ID] stores a retail store ID that can uniquely identify a retail store. [Retail Store Name] stores the name of the retail store. [Email Address] stores the email address used by the retail store. [Address] stores the current location of the retail store. [Telephone Number] stores the telephone number of the retail store. [Store 3D Data] stores three-dimensional data related to the retail store. [Product 3D Data] stores three-dimensional data related to products linked to the retail store. [Sales History] stores the sales history of products of the retail store in the virtual space.

Next, the UI information D113 will be described with reference to FIG. 6. FIG. 6 is a diagram showing the configuration of the UI information D113. As shown in FIG. 6, the UI information D113 may include items such as [UI ID], [UI name], [UI type], [UI data], and [Virtual space ID]. [UI ID] stores a UI ID that can uniquely identify various UIs. [UI name] stores the name of various UIs. [UI type] stores the type of UI, such as a UI for identifying a store, a UI for checking products, a UI for executing a payment method, a UI for checking purchased products, a UI related to a menu, etc. [UI data] stores data for displaying the UI in a virtual space. [Virtual space ID] stores a virtual space ID that can uniquely identify the virtual space to which the UI corresponds.

The acquisition unit 120 acquires information from other devices, such as the virtual space management server 200, the user terminal 300, the retail store terminal 400, and the e-commerce server 500.

The UI information providing unit 130 generates information about the UI related to the purchase operation by the user (hereinafter referred to as "UI information"). The UI related to the purchase operation generates, for example, a UI for identifying a store in the virtual space (hereinafter referred to as "store identification UI"), a UI for confirming a product (hereinafter referred to as "product confirmation UI"), a UI for selecting products in a cart (hereinafter referred to as "product selection UI"), a wallet UI for executing a payment method, a UI for confirming purchased products (hereinafter referred to as "purchased product confirmation UI"), and a UI related to a menu (hereinafter referred to as "menu UI"). The information about the UI generated by the UI information providing unit 130 is transmitted to the virtual space management server 200. As a result, the virtual space management server 200 can provide the user terminal 300 with a UI corresponding to the user in the virtual space. Hereinafter, for convenience, when "user's hand" is mentioned, it means the hand of an avatar corresponding to the user in the virtual reality space, and the real hand of the user in the augmented reality space.

First, an example of a store identification UI will be described with reference to FIG. 7. FIG. 7 is a diagram showing an example of a store identification UI in a virtual space. As shown in FIG. 7, the store identification UI includes an identification object U1 that is displayed in the virtual space for a store where payment can be made by a payment means. The identification object U1 is displayed, for example, when the user approaches the store within a predetermined distance (here, "L"). The identification object U1 is, for example, an object that indicates a logo, a type of payment means, or the like. This allows the virtual space payment system 10 to easily find stores where the user's own payment means can be used, thereby improving user convenience.

Next, an example of a product confirmation UI will be described with reference to FIG. 8. FIG. 8 is a diagram showing an example of a product confirmation UI in a virtual space. As shown in FIG. 8, the product confirmation UI may include an instruction object U2 that is displayed in the virtual space in response to the movement of the user's hand recognized by the user terminal 300. Here, the instruction object U2 is, for example, an object that operates in response to the movement of at least a part of the user's hand. Specifically, the instruction object U2 may be, for example, an object that extends linearly from the index finger of the user. The instruction object U2 may also be, for example, an object that extends linearly from the part where the index finger and thumb of one hand of the user overlap. The linearly extending object may also be displayed so as to penetrate the product. The instruction object U2 may also be, for example, a point-like object that indicates the intersection between the virtual extension line pointed to by the index finger of one hand of the user and the product. The instruction object U2 may also be, for example, an object that indicates the intersection between the virtual extension line pointed to by the index finger of one hand of the user and the product with a mark such as an arrow. As a result, the virtual space payment system 10 can improve the convenience of the user since the user can select a product with an intuitive action without approaching the product. In addition, when the instruction object U2 overlaps or intersects with the product and the user performs an action of pinching an item by overlapping the index finger and thumb (hereinafter referred to as a "pinch action"), the product confirmation UI brings the product closer to the user and displays the product so that it can be rotated or transformed according to the movement of the user's hand. As a result, the virtual space payment system 10 can improve the convenience of the user since the user can check the product at hand from various angles without approaching the product. In addition, the product confirmation UI may include a price display object U3 that displays the price, specifications, etc. of the product when the product is specified by the instruction object U2. As a result, the virtual space payment system 10 allows the user to select a product in the virtual space with a gesture that is more similar to the product selection action in the real world, and thus can provide the user with a deeper sense of immersion by realizing a product purchase experience with a more intuitive operation in the virtual space. Furthermore, the virtual space payment system 10 improves user convenience by allowing users to easily check product prices, specifications, etc.

As shown in FIG. 8, the product confirmation UI may include a display object U4 that displays, in a predetermined area, a number of products related to the product specified by the user. For example, if the product specified by the user is "bag A," related products include bags of the same brand, bags of a similar shape, bags of a similar color, bags of a similar use, and the like, in relation to bag A. The display object U4 is a spherical object that displays three-dimensional image data of the product in the inner area of the spherical shape. Each of the multiple display objects U4 may be displayed in a line in a direction away from the user so that at least a portion of them overlap. This can improve the user's visibility of the product in the virtual space. The product confirmation UI may also be displayed to move in response to the user's hand movement over the multiple display objects U4. This allows the user to view the product through an intuitive operation.

Next, an example of a product selection UI will be described with reference to FIG. 9. FIG. 9 is a diagram showing an example of a product selection UI in a virtual space. As shown in FIG. 9, the product selection UI includes, for example, a cart object U5 and a product display object U6 that displays a selected product. The cart object U5 is an object displayed in three dimensions for storing selected products. The cart object U5 is displayed so as to cover one of the user's palms, and moves in accordance with the movement of the palm. The product display object U6 is an object that displays a product in a reduced size within a predetermined area when the product is moved to the cart object U5. The product display object U6 is an object that displays three-dimensional image data of the product in an inner area of a sphere in response to the user's pinching action of the product with the other hand. The product display object U6 is displayed so as to be movable by the user while accepting the user's pinching action. This allows the user to select products by intuitive operation.

Also, with reference to FIG. 10, another example of the product selection UI will be described. FIG. 10 is a diagram showing another example of the product selection UI in the virtual space. As shown in FIG. 10, the product selection UI includes, for example, a product display object U7 that displays a plurality of products added to the cart by the user. The product display object U7 is, for example, an area in which each product moved to the cart object U5 shown in FIG. 9 is reduced and displayed within a predetermined area. The product selection UI may display, for example, a plurality of product display objects U7 in a line so as to surround the upper body (torso) of the user's avatar. This makes it easier for the user to view the selected product, thereby improving user convenience. In addition, the product selection UI cancels the selection state of the product in response to a hand movement of the user that removes the product display object U7 from the line. This allows the user to cancel the selection state of the product by an intuitive operation. In addition, the product selection UI moves the product along the line in response to a hand movement that moves the product display object U7 in a line along the line. This allows users to view selected products using intuitive operations.

Next, an example of the menu UI will be described with reference to FIG. 11. FIG. 11 is a diagram showing an example of the menu UI in a virtual space. As shown in FIG. 11, the menu UI includes, for example, an icon U8 arranged so as to surround the hand of a user with the thumb pointed forward. Surrounding the hand includes, for example, arranging the icon U8 in a circle or an arc around the wrist. The icon U8 is an item of the menu (for example, an item that transitions to checkout, cart, or inventory). This allows the user to easily grasp the items included in the menu, thereby improving user convenience. The menu UI may also include an instruction object U9 that indicates one of the multiple icons U8. The instruction object U9 is displayed so as to indicate, for example, the direction indicated by the tip of the user's thumb. The menu UI may change the icon U8 indicated by the instruction object U9 in response to, for example, the user's wrist twisting in one direction. This allows the user to select an item of the menu by an intuitive operation.

Next, an example of the first wallet UI will be described with reference to FIG. 12. FIG. 12 is a diagram showing an example of the first wallet UI in the virtual space. As shown in FIG. 12, the first wallet UI includes, in the virtual space, a card-shaped object U10 corresponding to a payment method in the real space. In the first wallet UI, a plurality of card-shaped objects U10 are displayed in one direction so that some of them overlap. This allows the user to select a payment method by an intuitive operation. In addition, the display position of the first wallet UI may be changed according to the frequency of use by the user. Specifically, the first wallet UI may display the card-shaped objects U10 in order of frequency of use (for example, in FIG. 12, "credit card A" is used most frequently) in an overlapping manner on the upper side. This allows the user to easily perform a selection operation for a frequently used payment method. In addition, the first wallet UI may display a limit display object U11 including the usage limit of each payment method. This allows the user to easily grasp the available amount of each payment method, thereby improving the user's convenience.

Next, an example of the second wallet UI will be described with reference to FIG. 13. FIG. 13 is a diagram showing an example of the second wallet UI in a virtual space. As shown in FIG. 13, the second wallet UI includes an object U12 in a virtual space having a shape corresponding to the type of asset related to the payment method. The second wallet UI may include, for example, a wallet-shaped object corresponding to an everyday account, an object shaped like a piggy bank corresponding to a savings account, and a three-dimensional polygonal object corresponding to crypto assets. This allows the user to select a payment method through an intuitive operation.

Next, an example of the third wallet UI will be described with reference to FIG. 14. FIG. 14 is a diagram showing an example of the third wallet UI in a virtual space. As shown in FIG. 14, the third wallet UI includes an input object U13 for allowing a user to input secret information in a virtual space. The input object U13 is, for example, a combination lock. The input object U13 is an object that accepts input of secret information in response to a user's action of pinching and rotating the combination lock. When the validity of the secret information is confirmed, the third wallet UI causes the wallet providing server 100 to transmit payment execution information for executing a payment. This allows the user to execute a payment by an intuitive operation. The third wallet UI may be an input object representing a pin pad, a signature pad, or a keyboard.

Next, an example of the fourth wallet UI will be described with reference to FIG. 15. FIG. 15 is a diagram showing an example of the fourth wallet UI in a virtual space. As shown in FIG. 15, the fourth wallet UI includes a slide object U14 that slides in response to the movement of the user's hand. The slide object U14 may be provided on a plate-shaped object U15. The fourth wallet UI may also display the slide object U14 on one of the user's palms. The fourth wallet UI transmits payment execution information in response to the movement of the user's hand sliding the slide object U14. This allows the user to execute a payment by an intuitive operation.

Furthermore, with reference to FIG. 16, another example of the fifth wallet UI will be described. FIG. 16 is a diagram showing another example of the fifth wallet UI in a virtual space. As shown in FIG. 16, the fifth wallet UI may include a card-shaped slide object U16 and a gate object U17 that allows the slide object U16 to pass through. In this case, the fifth wallet UI transmits payment execution information in response to the movement of the user's hand sliding the slide object U16. This allows the user to execute a payment by an intuitive operation.

Next, an example of a purchase product confirmation UI will be described with reference to FIG. 17. FIG. 17 is a diagram showing an example of a purchase product confirmation UI in a virtual space. As shown in FIG. 17, the purchase product confirmation UI includes an icon U18 (e.g., in a folder format, etc.) corresponding to a category of products (e.g., clothing, games, etc.) classified by a product classification unit described later. The purchase product confirmation UI may display products associated with the icon U18 selected by the user in a predetermined format, for example. This can improve the operability of the user. The user can easily find three-dimensional image data of the purchased product. In addition, the predetermined format may include a format in which three-dimensional image data of the purchased clothing is hung and displayed in the wardrobe U19 in the case of an icon U18 corresponding to a clothing category, for example. This allows the purchased product to be viewed as if it were confirmed in real space, thereby improving the convenience of the user. In addition, the purchase product confirmation UI may display, for example, a combination of multiple purchased products. Specifically, as shown in FIG. 17, the purchase product confirmation UI may display clothing that is hung in a wardrobe U19 and that has been selected by the user, as a human-shaped object U20. This allows the user to shop while checking how the clothing will be coordinated with the clothes they currently own, improving user convenience. In addition, for large, highly customizable products with complex mechanisms such as cars, motorbikes, and bicycles, the user can consider combinations at the time of purchase and preview functions, improving user convenience.

The display processing unit 140 executes a process of transmitting information regarding the various UIs to the virtual space management server 200 so that the virtual space management server 200 displays the various UIs generated by the UI information providing unit 130 on the display unit 1007 of the user terminal 300. The display processing unit 140 of the wallet providing server 100 may transmit information regarding the various UIs directly to the user terminal 300 so that the various UIs are displayed on the display unit 1007 of the user terminal 300. The information regarding the various UIs may include a program for displaying the various UIs on the display unit 1007 of the user terminal 300.

The payment unit 150 completes operations related to payment for products selected by the user in the virtual space, for example, in response to a selection operation by the user on a user interface (e.g., the first to fourth wallet UIs) related to the payment method. Specifically, the user terminal 300 transmits payment execution information to the virtual space management server 200 based on a selection operation by the user on, for example, the first or second wallet UI, or on, for example, the third, fourth, or fifth wallet UI, related to the payment method. The payment unit 150 executes payment processing based on purchase information transmitted from the virtual space management server 200 based on the payment execution information. The payment processing is, for example, a process of transmitting purchase information including information on the product purchased by the user to the electronic commerce server 500. The payment unit 150 executes the payment processing, thereby completing payment by the user in the virtual space.

The product classification unit 160 classifies information about purchased products, which are, for example, products paid for by the payment unit 150, into one of a plurality of categories. The product classification unit 160 stores the classified information about the purchased products in the user information D111.
===Configuration of Virtual Space Management Server 200===

Next, the configuration of the virtual space management server 200 will be described with reference to FIG. 18. FIG. 18 is a diagram showing an example of the configuration of the virtual space management server 200. As shown in FIG. 18, the virtual space management server 200 includes functional units, for example, a storage unit 210, an acquisition unit 220, a display processing unit 230, and a purchase information transmission unit 240. Each functional unit is a function that is realized, for example, when the processor 1001 reads out a program stored in the memory 1002.

The storage unit 110 stores, for example, store space information D211, product placement information D212, product information D213, action information D214, UI information D215, and user information D216. Note that UI information D215 and user information D216 are similar to UI information D113 and user information D111, and therefore a description thereof will be omitted.

First, the store space information D211 will be described with reference to FIG. 19. FIG. 19 is a diagram showing an example of the store space information D211. The store space information D211 is a database that stores three-dimensional data of a store in a virtual space. The three-dimensional data of a store includes data on the three-dimensional shape of a room consisting of walls, floors, ceilings, product display shelves, etc. As shown in FIG. 19, the store space information D211 may include items such as [Store Space ID], [Store Space Name], and [Update History]. [Store Space ID] stores a store space ID that can uniquely identify a store in the virtual space. [Store Space Name] stores the name of the store space. The name of the point p space may be, for example, the store name. [Update History] stores the history of updates to the store space.

Next, the product placement information D212 will be described with reference to FIG. 20. FIG. 20 is a diagram showing an example of the product placement information D212. The product placement information D212 stores the placement of objects. The objects are objects in the virtual space that the user focuses on. As shown in FIG. 20, the product placement information D212 may include items such as [Store Space ID], [Object ID], [3D Coordinates], [Scale], and [Update Date and Time]. [Store Space ID] stores the store space ID. [Object ID] stores the object ID of the object placed in the virtual space. [3D Coordinates] stores the coordinates (x, y, z) indicating the position of the object placed in the store in the virtual space. [Scale] stores the scale when the object is displayed in the virtual space. [Update Date and Time] stores the update date and time of each record.

Next, product information D213 will be described with reference to FIG. 21. FIG. 21 is a diagram showing an example of product information D213. Product information D213 stores attributes of objects, etc. As shown in FIG. 21, product information D213 may include items such as [Object ID], [Product name], [Price], and [Update date and time]. [Object ID] stores the object ID of the corresponding object. [Product name] stores the product name. [Price] stores the price of the product. [Update date and time] stores the update date and time of each record.

Next, the action information D214 will be described with reference to FIG. 22. FIG. 22 is a diagram showing an example of the action information D214. The action information D214 stores an action corresponding to a user's action. As shown in FIG. 22, the action information D214 may include items such as [action ID], [action name], [recognition action], [UI ID], and [execution date and time]. [Action ID] stores an action ID that can uniquely identify the type of action. [Action name] stores the name of the action. [Recognition action] stores information corresponding to the user's action recognized by the user terminal 300. Specifically, the recognized user's action includes, for example, the user's hand movements shown in FIG. 8, FIG. 9, FIG. 10, FIG. 14, FIG. 15, FIG. 16, and FIG. 11. [UI ID] stores a UI ID that can uniquely identify various UIs. [Execution date and time] stores the date and time when the action was executed by the user.

The acquisition unit 220 acquires information from other devices, such as the wallet providing server 100 and the user terminal 300. The various information acquired by the acquisition unit 220 is stored in the storage unit 210.

The display processing unit 230 executes a process for displaying, for example, three-dimensional image data relating to the virtual space stored in the storage unit 210 on the display unit 1007 of the user terminal 300. Specifically, the display processing unit 230 causes the display unit 1007 of the user terminal 300 to display a virtual space including a UI as shown in Figures 7 to 17.

The purchase information transmitting unit 240 identifies the product that is the target of the purchase operation when, for example, payment execution information is obtained from the user terminal 300. The purchase information transmitting unit 240 transmits purchase information related to the product that is the target of the purchase operation to the wallet providing server 100.
===User Terminal 300===

Next, the configuration of the user terminal 300 will be described with reference to FIG. 23. FIG. 23 is a diagram showing an example of the configuration of the user terminal 300. As shown in FIG. 23, the user terminal 300 includes functional units, for example, a storage unit 310, an acquisition unit 320, an action recognition unit 330, and a display processing unit 340. Each functional unit is a function that is realized, for example, when the processor 1001 reads out a program stored in the memory 1002.

The storage unit 310 stores, for example, user information D311 and action information D312. The user information D311 is similar to the user information D111, and therefore its description will be omitted. The action information D312 is similar to the action information D214, and therefore its description will be omitted. The storage unit 310 may store information similar to the UI information D215. The acquisition unit 320 acquires information from other devices, such as the wallet providing server 100 and the virtual space management server 200. Various information acquired by the acquisition unit 320 is stored in the storage unit 310. The action recognition unit 330 recognizes the user's action, for example, based on information acquired by the sensor group 1008 included in the user terminal 300. Specifically, the action recognition unit 330 generates information related to the action (hereinafter referred to as "action information") by referring to the action information D312. The user terminal 300 transmits the action information to the virtual space management server 200. This allows a UI corresponding to the user's operation to be displayed on the display unit 1007. The display processing unit 340 causes the display unit 1007 to display, for example, three-dimensional image data related to the virtual space acquired from the virtual space management server 200 and three-dimensional image data related to various UIs.
Processing of Virtual Space Payment System 10

An example of the processing of the virtual space payment system 10 will be described with reference to Figure 24. Figure 24 is a flow diagram showing an example of the processing of the virtual space payment system 10. Below, an example of the processing flow will be described, from when a retail store opens a store in a virtual space, to when a user finds the retail store in the virtual space, purchases clothes at the retail store, and completes the payment processing. The following description will also refer to Figures 7 to 17 as appropriate.

In step S100, the retail store terminal 400 of the retail store transmits store information and product information to the wallet providing server 100. The wallet providing server 100 may create three-dimensional image data of the store and the product based on the store information and product information. The wallet providing server 100 associates the store information, product information, and the three-dimensional image data of the store and the product with a store ID and stores them in the storage unit 110.

Next, in step S101, the wallet providing server 100 transmits store information, product information, and three-dimensional image data of the store and products to the virtual space management server 200. At this time, the wallet providing server 100 may transmit the store information, product information, and three-dimensional image data of the store and products together with a program for applying the store information, product information, and three-dimensional image data of the store and products to the virtual space management server 200 provided by the virtual space management server 200. In other words, the wallet providing server 100 may transmit the store information, product information, and three-dimensional image data of the store and products to the virtual space management server 200 by means of an SDK. This allows retailers to easily open stores in the virtual space.

Next, in step S102, the virtual space management server 200 stores store information, product information, and three-dimensional image data of the stores and products in the storage unit 210. The virtual space management server 200 reflects the stores and products in the virtual space.

Next, in step S200, the wallet providing server 100 transmits a wallet APP for enabling the use of various UIs in the virtual space in response to a request from the user terminal 300. Also, the virtual space management server 200 transmits a virtual space APP for displaying the virtual space in response to a request from the user terminal 300.

Next, in step S201, the user terminal 300 accepts user input and transmits user information and payment method information to the wallet providing server 100. The wallet providing server 100 stores the user information and payment method information in the storage unit 110.

Next, in step S202, the wallet providing server 100 associates UI information corresponding to the payment method information with the user information and transmits it to the virtual space management server 200. This allows the virtual space management server 200 to provide a wallet UI to the user. In step S203, the virtual space management server 200 associates the wallet UI with the user information and stores it in the storage unit 210.

Next, in step S300, the user terminal 300 transmits authentication information to the virtual space management server 200. In step S301, the virtual space management server 200 transmits information about the virtual space to the user terminal 300. In step S302, the user terminal 300 displays the virtual space on the display unit 1007. The user terminal 300 transmits operation information to the virtual space management server 200 by recognizing the user's operation with reference to the action information D312. In step S303, the virtual space management server 200 transmits UI information about various UIs corresponding to the user's operation based on the operation information to the user terminal 300. Next, in step S304, the user terminal 300 displays various UIs on the display unit 1007.

The following describes the process in which the user terminal 300 displays various UIs. Note that, in the following, it is described that the virtual space management server 200 refers to the UI information 215 and causes the display unit 1007 of the user terminal 300 to display various UIs. However, for example, if the UI information is stored in the user terminal 300, the user terminal 300 may refer to the UI information and cause the display unit 1007 to display various UIs.

As shown in FIG. 7, when the virtual space management server 200 determines that the distance between the avatar and the store in the virtual space has reached a predetermined distance, it refers to the UI information D215 and causes the display unit 1007 to display a store identification UI. This allows the user to easily recognize stores that accept the registered payment method, thereby improving user convenience.

As shown in FIG. 8, when a user checks a product in a store, the virtual space management server 200 refers to the UI information D215 and displays a product confirmation UI (here, an instruction object U2) on the display unit 1007 of the user terminal 300 to pull the product closer to the user. The virtual space management server 200 acquires operation information related to the user's hand movement recognized by referring to the action information D312 on the user terminal 300, and can rotate, shrink, or enlarge the product by pulling it closer to the user's hand. The virtual space management server 200 also refers to the product information D213 to identify products related to the product specified by the instruction object U2. The virtual space management server 200 displays each of the three-dimensional image data of the identified multiple products within the area of a spherical display object U4. The virtual space management server 200 moves the spherical display object U4 in accordance with the hand movement based on the operation information related to the user's hand movement recognized by the user terminal 300. This allows the user to check related products in a list, and the virtual space payment system 10 allows the user to view the products by intuitive operations.

9, the virtual space management server 200 displays a product selection UI (here, a cart object U5 and a product display object U6) for the user to add products in the store to the cart on the display unit 1007 of the user terminal 300. The virtual space management server 200 acquires motion information related to the motion of the user sticking out the palm of the left hand, which is recognized by referring to the action information D312 on the user terminal 300. The virtual space management server 200 displays the cart object U5 on the display unit 1007 of the user terminal 300 based on the UI information D215. This allows the virtual space payment system 10 to display the cart by an intuitive motion, thereby improving the user's convenience. The virtual space management server 200 acquires motion information related to the motion of the user pinching the product, which is recognized by referring to the action information D312 on the user terminal 300. The virtual space management server 200 displays the product display object U6 including the three-dimensional image data of the product on the display unit 1007 of the user terminal 300 by referring to the UI information D215. When the virtual space management server 200 acquires operation information regarding the operation of moving the product display object U6 to the cart object U5, the virtual space management server 200 adds the product to a purchase list (hereinafter referred to as a "purchase list"). The purchase list is, for example, a list stored in the memory unit 210 of the virtual space management server 200. This allows the virtual space payment system 10 to add products to the cart through intuitive operations, improving user convenience.

Also, as shown in FIG. 10, the virtual space management server 200 displays a product selection UI (here, a product display object U7) for viewing products stored in the purchase list on the display unit 1007 of the user terminal 300. When the virtual space management server 200 acquires operation information from the user terminal 300, for example, related to an operation of selecting an icon U8 indicating a cart in the menu UI, the virtual space management server 200 displays the product display object U7 on the display unit 1007. When the virtual space management server 200 acquires information related to an operation of removing the product display object U7 from the row, the virtual space management server 200 removes the product displayed in the product display object U7 from the purchase list. This allows the virtual space payment system 10 to easily view the products in the cart and to remove products from the cart by an intuitive operation, thereby improving user convenience.

As shown in FIG. 11, the virtual space management server 200 displays a menu UI for selecting an item on the display unit 1007 of the user terminal 300. The virtual space management server 200 acquires motion information related to the motion of sticking out one hand with the thumb facing forward, which is recognized by the user terminal 300 with reference to the action information D312. The virtual space management server 200 displays an icon U8 included in the menu UI on the display unit 1007 of the user terminal 300 so as to surround the user's stuck-out hand. This allows the user to easily check the menu items, improving user convenience. When the virtual space management server 200 acquires motion information related to the operation of selecting a specific icon U8 of the menu UI from the user terminal 300, it displays an object corresponding to the icon U8 on the display unit 1007 of the user terminal 300.

As shown in Figures 12 and 13, the virtual space management server 200 displays a wallet UI (here, the first wallet UI or the second wallet UI) on the display unit 1007 of the user terminal 300 to allow the user to select a payment method when purchasing a product. For example, when the virtual space management server 200 acquires operation information related to an operation to pay for a product in a purchase list recognized by referring to action information D312 on the user terminal 300, the virtual space management server 200 displays the first wallet UI or the second wallet UI on the display unit 1007. The virtual space management server 200 generates purchase information based on a purchase operation for a payment method (e.g., a credit card U10 or a wallet object U12) selected by the user from the first wallet UI or the second wallet UI.

As shown in Figs. 14, 15, and 16, the virtual space management server 200 displays a wallet UI (here, the third wallet UI, the fourth wallet UI, or the fifth wallet UI) on the display unit 1007 of the user terminal 300, which accepts an action to confirm a purchase when making a payment with a specified payment method. For example, when the virtual space management server 200 acquires operation information related to an operation to select a payment method shown in Figs. 12 and 13, the virtual space management server 200 displays the third wallet UI, the fourth wallet UI, or the fifth wallet UI on the display unit 1007 of the user terminal 300. When the user terminal 300 recognizes an action to execute a user's payment for the third wallet UI, the fourth wallet UI, or the fifth wallet UI by referring to the action information D312, it transmits payment execution information to execute the payment to the virtual space management server 200.

Next, in step S305, purchase information is generated based on the payment execution information, and the purchase information is transmitted to the wallet providing server 100. Next, in step S306, the wallet providing server 100 transmits the purchase information to the electronic commerce server 500 to settle the product that is the subject of the payment. This allows the user to complete payment-related operations in the virtual space by operating the user terminal 300, without removing the user terminal 300.

Next, in step S306, the wallet providing server 100 identifies the e-commerce server 500 corresponding to the purchased item based on the purchase information, and transmits the purchase information. This enables the e-commerce server 500 to execute the payment process to the retailer linked to the purchase information. This enables the user to complete the transaction, for example, within a VR (Virtual Reality) SNS (Social Networking Service).

Next, in step S307, the user terminal 300 receives a payment feasibility notification from the wallet providing server 100 regarding whether or not payment is possible at the electronic commerce server 500. This allows the user to know whether or not payment can be made in the virtual space, thereby improving user convenience.

Next, in step S308, the virtual space management server 200 stores the purchased product information in the user information D216. As shown in Fig. 17, the virtual space management server 200 may refer to the user information D216 and cause the display unit 1007 of the user terminal 300 to display a purchased product confirmation UI that displays three-dimensional image data related to the product for which the purchase procedure has been completed as a three-dimensional image.
====Modifications====

The virtual space payment system 10 may further include a configuration for issuing a token number that is encrypted (anonymized) based on confidential information (e.g., card number, etc.) of a payment means, such as a credit card. Specifically, for example, after step S6 in FIG. 2, the user terminal 300 transmits information about the card number to the service provider that issues the token number through user input. The user terminal 300 obtains the token number corresponding to the card number. The user terminal 300 stores the token number in cooperation with the virtual space APP. When the user terminal 300 receives an action to purchase a product from the user in the virtual space, it transmits payment execution information to the virtual space management server 200. The virtual space management server 200 transmits purchase information including the token number to the wallet provision server 100 (step S9 in FIG. 2). This allows the user to be provided with a secure purchase process.

Furthermore, the virtual space payment system 10 may manage three-dimensional data related to purchased goods on a blockchain platform. Specifically, a purchase certificate for the purchased goods is issued to the user, and the three-dimensional data related to the goods is stored in distributed storage. In addition, the transaction history of the goods is stored on the blockchain platform. This allows the user to prove that he or she is a legitimate purchaser of the goods, and allows the data to be used even if the content distribution company's service is suspended.
===Hardware Configuration===

The wallet providing server 100 and the virtual space management server 200 that realize the functions related to the above-described embodiments may be, for example, a server computer, a personal computer (e.g., desktop, laptop, tablet, etc.), a media computer platform (e.g., cable, satellite set-top box, digital video recorder), a handheld computer device (e.g., PDA, email client, etc.), or other types of computers or communication platforms.

The user terminal 300 and retail store terminal 400 that realize the functions related to the above-mentioned embodiments may be, for example, a smartphone, a mobile phone (feature phone), a personal computer (e.g., desktop, laptop, tablet, etc.), a media computer platform (e.g., cable, satellite set-top box, digital video recorder), a handheld computer device (e.g., PDA (Personal Digital Assistant), email client, etc.), a wearable device (glasses-type device, watch-type device, etc.), another type of computer, or a communication platform. It is preferable that the user terminal 300 is a transparent head-mounted display or a non-transparent head-mounted display.

With reference to FIG. 25, an example of a hardware configuration in which the wallet providing server 100, virtual space management server 200, user terminal 300, and retail store terminal 400 are realized by a computer 1000 will be described. Note that the various functions of the wallet providing server 100, virtual space management server 200, user terminal 300, and retail store terminal 400 can be realized by dividing them into multiple devices.

Figure 25 is a diagram showing an example of the hardware configuration of a computer. As shown in Figure 25, the computer 1000 includes, for example, a processor 101, a memory 1002, a storage device 1003, an input I/F unit 1004, a data I/F unit 1005, a communication I/F unit 1006, a display unit 1007, and a sensor group 1008. Note that the sensor group 1008 is hardware provided in the user terminal 300, and does not necessarily have to be provided in the wallet providing server 100, the virtual space management server 200, the user terminal 300, and the retail store terminal 400.

The processor 1001 is a control unit that controls various processes in the computer 1000 by executing programs stored in the memory 1002.

The memory 1002 is a storage medium such as a RAM (Random Access Memory). The memory 1002 temporarily stores the program code of the program executed by the processor 1001 and data required when the program is executed.

The storage device 1003 is a non-volatile storage medium such as a hard disk drive (HDD) or flash memory. The storage device 1003 stores an operating system and various programs for implementing the above configurations.

The input I/F unit 1004 is a device for receiving input from a user. Specific examples of the input I/F unit 1004 include a keyboard, a mouse, a touch panel, various sensors, a wearable device, and the like. The input I/F unit 1004 may be connected to the computer 1000 via an interface such as a USB (Universal Serial Bus).

The data I/F unit 1005 is a device for inputting data from outside the computer 1000. A specific example of the data I/F unit 1005 is a drive device for reading data stored in various storage media. The data I/F unit 1005 may be provided outside the computer 1000. In that case, the data I/F unit 1005 is connected to the computer 1000 via an interface such as a USB.

The communication I/F unit 1006 is a device for performing data communication via the Internet N, either wired or wirelessly, with devices external to the computer 1000. The communication I/F unit 1006 may be provided external to the computer 1000. In that case, the communication I/F unit 1006 is connected to the computer 1000 via an interface such as a USB.

The display unit 1007 is a device for displaying various types of information. Specific examples of the display unit 1007 include a liquid crystal display, an organic EL (Electro-Luminescence) display, a display of a wearable device, and the like. The display unit 1007 may be provided outside the computer 1000. In that case, the display unit 1007 is connected to the computer 1000 via, for example, a display cable. In addition, when a touch panel is adopted as the input I/F unit 1004, the display unit 1007 can be configured as an integral part of the input I/F unit 1004.

Note that in the above, when the display unit 1007 of the user terminal 300 is mentioned, the display unit 1007 includes a liquid crystal display panel. If the user terminal 300 is a transparent head mounted display, the real space is not displayed on the display unit 1007. If the user terminal 300 is a non-transparent head mounted display or a smartphone, an image in which a captured image of the real space and an image of the virtual reality space are superimposed may be generated and displayed on the display unit 1007. If the user terminal 300 is a transparent head mounted display or a non-transparent head mounted display, the display unit 1007 may include two display panels and display a stereo image.

The sensor group 1008 includes various sensors. The sensor group 1008 includes a sensor for grasping the space around the user, a sensor for grasping the position, posture, and hand movement of the user, a sensor for detecting the user's viewpoint and line of sight, and the like. The sensor for grasping the space around the user is an RGB image sensor, a depth sensor, and the like. The sensor for detecting the position, posture, hand movement, and line of sight of the user is a gyro sensor, a digital compass, an acceleration sensor, a proximity sensor, a motion tracking sensor, and the like.
====Summary====

The virtual space payment system 10 in this embodiment includes a display processing unit 140 that executes a process for displaying, on the display unit 1007 of the user terminal 300 (display device), an instruction object U2 (e.g., an object extending linearly) for indicating a product object displayed in three dimensions in the augmented reality space or the virtual reality space, which moves in response to the hand movement of a user wearing a user terminal 300 (display device) that displays a virtual reality space that is an augmented reality space or a virtual three-dimensional space, and executes a process for displaying the designated product object closer to the user by a pinch operation by the user when there is a designated product object that is a product (product object) that overlaps or intersects with the instruction object U2, and a payment unit 150 that completes an operation related to the payment of the selected product in response to a selection operation by the user on a user interface related to a payment means for making a payment for a selected product that is a product corresponding to the product (designated product object) selected by a selection operation by the user. This makes it possible to select a product in the virtual space by a gesture that is more similar to a product selection action in the real world, and thus provides the user with a deeper sense of immersion by realizing a product purchase experience by a more intuitive operation in the virtual space.

In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing to display the instruction object U2 as an object extending linearly from at least a part of the user's hand. This allows the user to select a product through an intuitive operation.

In addition, in the virtual space payment system 10 of this embodiment, the payment unit 150 completes the operation related to the payment of the product selected by the user in the augmented reality space or virtual reality space based on a token number (identification information) that conceals the identification code that identifies the payment method. This makes it possible to provide the user with a secure purchase process.

In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing to display an identification object U1 (a specified identification sign) in the augmented reality space or virtual reality space for stores that accept payments by a payment means. This allows users to easily find stores that accept their own payment means, improving user convenience.

In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 displays a display object U4 that displays multiple related products related to the product of the designated product object in a specified display area in the augmented reality space or virtual reality space, and displays the multiple display objects U4 in a row in a direction away from the user so that at least a portion of each of the multiple display objects U4 overlaps, and executes processing to move the multiple display objects U4 in response to the user's hand movement relative to the multiple display objects U4. This can improve the user's visibility of the products in the virtual space. It also allows the user to view the products through intuitive operations.

In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing for reducing and displaying the selected product object within a specified area (here, product display object U6) in response to a user's pinching action of the selected product object, as recognized by the user terminal 300 (display device) in the augmented reality space or virtual reality space. This allows the user to select products through intuitive operations.

In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 displays, in the augmented reality space or virtual reality space, multiple selected products that have been set as candidates for purchase by the user's selection operation on the selected product object, on the product display object U6, and each of the multiple product display objects U7 is displayed in a line surrounding the torso of the user or the user's avatar, and executes processing to move the multiple product display objects U7 in response to the user's hand movements on the multiple product display objects U7. This makes it easier for the user to view the selected products, thereby improving user convenience.

In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing to hide the product display object U7 on which the selected product is displayed, so that the selected product is removed from the selection list of purchase candidates, in response to an action by the user to move the selected product displayed on the product display object U7, which is recognized on the user terminal 300 (display device), out of a predetermined range in the augmented reality space or virtual reality space. This allows the user to deselect the product through an intuitive operation.

In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing for displaying, in the augmented reality space or virtual reality space, a plurality of icons U8 constituting a menu UI (menu object) that is displayed so as to surround at least one hand of the user as recognized by the user terminal 300 (display device). This allows the user to easily grasp the items included in the menu, thereby improving user convenience.

In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 displays, in the augmented reality space or virtual reality space, an instruction object U9 that indicates one of the icons U8 included in the menu UI (menu object), and executes processing to change the icon U8 indicated by the instruction object U9 in response to the user's wrist twisting action in one direction recognized by the user terminal 300 (display device). This allows the user to select a menu item by an intuitive operation.

In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing to display, in the augmented reality space or virtual reality space, a user interface relating to a payment method in the shape of a card corresponding to that payment method in the real space, and to display multiple card-shaped objects U10 (user interfaces) in a row in one direction so that they partially overlap. This allows the user to select a payment method through an intuitive operation.

In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing to display a user interface related to a payment method in an augmented reality space or virtual reality space in a shape (here, object U12) according to the type of asset. This allows the user to select a payment method through an intuitive operation.

In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing for displaying an input object U13 representing a combination lock for allowing the user to input secret information in the augmented reality space or virtual reality space, and the settlement unit 150 executes the user's payment based on information input to the input object in response to the user's hand movements recognized on the display device. This allows the user to execute payment through intuitive operations.

In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing for displaying a slide object U16 that slides in the augmented reality space or virtual reality space in response to the user's hand movements recognized on the user terminal 300 (display device), and the settlement unit 150 executes the user's payment based on information input to the slide object U16 in response to the user's hand movements recognized on the user terminal 300 (display device). This allows the user to execute payment through intuitive operations.

In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing for displaying a slide object U14 on one palm of the user as recognized by the user terminal 300 (display device) in the augmented reality space or virtual reality space. This allows the user to perform payment by intuitive operations.

The virtual space payment system 10 in this embodiment further includes a product classification unit 160 that classifies information about purchased items, which are items paid for by the settlement unit 150, into one of a plurality of categories, and the display processing unit 140 displays a plurality of folders corresponding to the plurality of categories in the augmented reality space or virtual reality space, and in response to a user's selection operation on one of the plurality of folders, executes processing to display a purchased item object (here, three-dimensional image data of clothing), which is a purchased item classified into the category corresponding to the folder that received the selection operation. This can improve user operability.

In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing for combining and displaying multiple purchased item objects (here, three-dimensional image data of clothing) according to item categories in the augmented reality space or virtual reality space, based on information about the purchased item objects (here, three-dimensional image data of clothing) stored in the item classification unit 160. This allows the purchased items to be viewed as if they were being viewed in real space, improving user convenience.

In addition, in the virtual space payment system 10 of this embodiment, the product classification unit 160 stores information about the user's clothing, which is the purchased product, and the display processing unit 140 executes processing to display clothing objects, which are clothing, hung in the wardrobe U19 so that they can be viewed by the user in the augmented reality space or virtual reality space. This allows the user to shop while checking how the clothes will be coordinated with the clothes they currently own, improving user convenience.

10... Virtual space payment system, 100... Wallet providing server, 200... Virtual space management server, 300... User terminal, 400... Retailer terminal.

Claims (7)

a display processing unit that executes processing for displaying, on a display unit of a display device, an instruction object for indicating a product object that is three-dimensionally displayed in an augmented reality space or a virtual reality space, which is a virtual three-dimensional space , and that moves in response to a user's hand movement in the augmented reality space or a virtual reality space;
a payment unit that accepts a selection operation by the user on a user interface related to a payment means for making a payment for a selected product , which is a product corresponding to the designated product object, and completes an operation related to the payment for the selected product in response to the selection operation ;
Equipped with
The display processing unit is
In the augmented reality space or the virtual reality space, a plurality of display objects are displayed in a predetermined display area, each of the display objects representing a plurality of related products that are related to the product of the designated product object;
displaying the plurality of display objects in a line in a direction away from the user so that at least a portion of each of the plurality of display objects overlaps with each other, and executing a process for moving the plurality of display objects in response to a movement of the user's hand relative to the plurality of display objects;
Information processing system. the display processing unit executes a process for displaying the instruction object as an object extending linearly from at least a part of the user's hand.
The information processing system according to claim 1 . the display processing unit executes a process for reducing and displaying the selected product object within a predetermined area in response to a pinching action of the user on the display device in the augmented reality space or the virtual reality space, the selected product object indicating the selected product.
3. The information processing system according to claim 1 or 2 . the display processing unit causes a product display object to display, in the augmented reality space or the virtual reality space, a plurality of selected products set as candidates for purchase by a selection operation by the user on a selected product object indicating the selected product, wherein each of the plurality of product display objects is displayed in a connected manner so as to surround a torso of the user or an avatar of the user;
executing a process for moving the plurality of product display objects in response to a hand movement of the user relative to the plurality of product display objects;
The information processing system according to any one of claims 1 to 3 . a display processing unit that executes processing for displaying, on a display unit of a display device, an instruction object for indicating a product object that is three-dimensionally displayed in an augmented reality space or a virtual reality space, which is a virtual three-dimensional space, and that moves in response to a user's hand movement in the augmented reality space or a virtual reality space;
a payment unit that accepts a selection operation by the user on a user interface related to a payment means for making a payment for a selected product, which is a product corresponding to the designated product object, and completes an operation related to the payment for the selected product in response to the selection operation;
Equipped with
the display processing unit causes a product display object to display, in the augmented reality space or the virtual reality space, a plurality of selected products set as candidates for purchase by a selection operation by the user on a selected product object indicating the selected product, wherein each of the plurality of product display objects is displayed in a connected manner so as to surround a torso of the user or an avatar of the user;
execute a process for moving the plurality of product display objects in response to a hand movement of the user relative to the plurality of product display objects;
the display processing unit, in response to an action of the user moving the selected product displayed on the product display object, which is recognized on the display device, outside a predetermined range in the augmented reality space or the virtual reality space, executes a process for hiding the product display object on which the selected product is displayed, so that the selected product is removed from a setting of a candidate for purchase.
Information processing system. The computer
executing a process for displaying, on a display unit of a display device, an instruction object for indicating a product object displayed in three-dimensional form in an augmented reality space or a virtual reality space, which is a virtual three-dimensional space , the instruction object moving in response to a user's hand movement in the augmented reality space or a virtual reality space;
accepting a selection operation by the user on a user interface related to a payment means for making a payment for a selected product, which is a designated product object that is a product object indicated by the instruction object and corresponds to the designated product object , and completing an operation related to the payment for the selected product in response to the selection operation;
Displaying a plurality of display objects in a predetermined display area in the augmented reality space or the virtual reality space, each of the display objects representing a plurality of related products that are related to the product of the designated product object;
displaying the plurality of display objects in a line in a direction away from the user so that at least a portion of each of the plurality of display objects overlaps with each other, and moving the plurality of display objects in accordance with a movement of the user's hand relative to the plurality of display objects;
An information processing method for performing the above. On the computer,
executing a process for displaying, on a display unit of a display device, an instruction object for indicating a product object displayed in three-dimensional form in an augmented reality space or a virtual reality space, which is a virtual three-dimensional space , the instruction object moving in response to a user's hand movement in the augmented reality space or a virtual reality space;
accepting a selection operation by the user on a user interface related to a payment means for making a payment for a selected product, which is a designated product object that is a product object indicated by the instruction object and corresponds to the designated product object , and completing an operation related to the payment for the selected product in response to the selection operation;
Displaying a plurality of display objects in a predetermined display area in the augmented reality space or the virtual reality space, each of the display objects representing a plurality of related products that are related to the product of the designated product object;
displaying the plurality of display objects in a line in a direction away from the user so that at least a portion of each of the plurality of display objects overlaps with each other, and moving the plurality of display objects in accordance with a movement of the user's hand relative to the plurality of display objects;
A program that executes the following.

Images