JP7848860B2 - Information processing device, information processing system, information processing method, and program - Google Patents

Description

This disclosure relates to an information processing device, an information processing system, an information processing method, and a non-temporary computer-readable medium.

In restaurants, ordering systems that accept orders via apps using information terminals are known. For example, Patent Document 1 discloses a method in which individual menu information of a restaurant, generated based on the order history of visiting users, is displayed on a matching app, and food and beverage orders are accepted through the matching app. The server that receives the order information displays the order information on the restaurant terminal and notifies the restaurant staff to start cooking.

International Publication No. 2021/162062

However, the method described in Patent Document 1 above is based on user operation upon arrival at the store, and therefore is insufficient in terms of reducing the effort required for ordering upon arrival.

The purpose of this disclosure is, in view of the above-mentioned problems, to provide an information processing device, an information processing system, an information processing method, and a non-temporary computer-readable medium that can suitably reduce the effort required for ordering when customers visit a store.

An information processing device according to one aspect of the present disclosure includes: registration means for registering provisional order information, which includes at least item information for identifying items related to a provisional order, in association with the user's biometric information, when received from a user terminal used by the user; authentication control means for controlling biometric authentication when received from a user who has visited a store; and transmission control means for transmitting the provisional order information associated with the user's biometric information to a terminal at the store visited by the user, if the biometric authentication is successful.

An information processing system according to one aspect of this disclosure comprises a biometric authentication device that performs biometric authentication and a server. The server includes registration means for registering provisional order information, which includes at least item information for identifying items related to a provisional order, from a user terminal used by a user, in association with the user's biometric information; authentication control means for receiving biometric information of a user who has visited a store, transmitting the biometric information to the biometric authentication device, and receiving the result of the biometric authentication from the biometric authentication device; and transmission control means for transmitting the provisional order information associated with the user's biometric information to the store's terminal if the biometric authentication is successful.

An information processing method according to one aspect of this disclosure, upon receiving provisional order information from a user terminal used by a user, which includes at least item information for identifying items related to a provisional order, registers the provisional order information in association with the user's biometric information, and upon receiving biometric information of a user who has visited a store, controls biometric authentication, and if the biometric authentication is successful, transmits the provisional order information associated with the user's biometric information to the store's terminal.

A non-temporary computer-readable medium according to one aspect of this disclosure stores a program for a computer to implement the following functions: a registration function that registers provisional order information, which includes at least item information for identifying items related to a provisional order, associated with the user's biometric information when provisional order information is received from a user terminal used by the user; an authentication control function that controls biometric authentication when biometric information of a user who has visited a store is received; and a transmission control function that transmits the provisional order information associated with the user's biometric information to the store's terminal when biometric authentication is successful.

This disclosure provides an information processing device, an information processing system, an information processing method, and a non-temporary computer-readable medium that can effectively reduce the effort required for ordering when customers visit a store.

This is a block diagram showing the configuration of the information processing device according to Embodiment 1. This is a flowchart showing the flow of the information processing method according to Embodiment 1. This is a block diagram showing the overall configuration of the information processing system according to Embodiment 2. This is a block diagram showing the configuration of a facial recognition device according to Embodiment 2. This flowchart shows the flow of the facial information registration process according to Embodiment 2. This flowchart shows the flow of the facial recognition process according to Embodiment 2. This is a block diagram showing the configuration of the user terminal according to Embodiment 2. This is a block diagram showing the configuration of a store terminal according to Embodiment 2. This is a block diagram showing the server configuration according to Embodiment 2. This is a sequence diagram showing the flow of the member registration process according to Embodiment 2. This is a sequence diagram showing the flow of provisional order processing according to Embodiment 2. This figure shows an example of the display on a user terminal according to Embodiment 2. This is a sequence diagram showing the order processing flow according to Embodiment 2. This figure shows an example of the display on a store terminal according to Embodiment 2. This figure shows an example of the display on a store terminal according to Embodiment 2. This is a sequence diagram showing the flow of the promotional information provision process according to Embodiment 2. This figure shows an example of the display on a user terminal according to Embodiment 2.

The embodiments of this disclosure will be described in detail below with reference to the drawings. In each drawing, the same or corresponding elements are denoted by the same reference numerals, and redundant explanations will be omitted where necessary for clarity.

<Embodiment 1>
First, Embodiment 1 of this disclosure will be described. Figure 1 is a block diagram showing the configuration of the information processing device 10 according to Embodiment 1. The information processing device 10 consists of one or more computers that enable a user to place a provisional order for items in a store before visiting the store. The store is an eatery such as a cafe or restaurant, an apparel shop, a general store, a beauty salon, or a gym.

Here, the information processing device 10 is connected to a network (not shown). The network may be wired or wireless. The network may also be connected to a user terminal (not shown) used by the user and a store terminal (not shown) installed in a store visited by the user.

The information processing device 10 comprises a registration unit 11, an authentication control unit 12, and a transmission control unit 15.

The registration unit 11 is also referred to as the registration means. When the information processing device 10 receives provisional order information and the user's biometric information from the user terminal, the registration unit 11 associates the provisional order information with the user's biometric information and registers it. Provisional order information is information relating to a provisional order. Provisional order information includes at least item information. Item information is information for identifying the item relating to the provisional order. Items include meal menus, training menus, products, or services. Item information may specify not only the type and quantity of the item but also the details of the customization. Customization may include specifying cooking preferences or adding or changing ingredients.

Here, biometric information refers to the information used in biometric authentication, which will be described later. Biometric authentication includes facial recognition, fingerprint recognition, iris recognition, vein recognition, gait recognition, or other biometric authentication methods. And biometric information is facial information, iris information, vein information, gait information, or other biometric information.

Furthermore, the information processing device 10 may receive a user ID associated with the user's biometric information from the user terminal instead of the user's biometric information. In this case, the registration unit 11 may associate the provisional order information with the user's biometric information by registering the provisional order information and the user ID in association.

The authentication control unit 12 is also referred to as the authentication control means. When the authentication control unit 12 receives biometric information of a user who has visited the store from the store terminal, it controls biometric authentication based on the user's biometric information. When the information processing device 10 controls biometric authentication, it may mean that the information processing device 10 performs biometric authentication using the user's biometric information, or it may mean that the information processing device 10 transmits the biometric information to another device, such as a biometric authentication device, to perform biometric authentication.

The transmission control unit 15 is also referred to as the transmission control means. If biometric authentication is successful, the transmission control unit 15 transmits the provisional order information associated with the user's biometric information to the store terminal.

Figure 2 is a flowchart showing the flow of the information processing method according to Embodiment 1. First, the information processing device 10 receives provisional order information from the user terminal (S10). At this time, the information processing device 10 may have already obtained the user's biometric information from the user terminal. Next, the registration unit 11 of the information processing device 10 registers the provisional order information in association with the user's biometric information (S11).

Subsequently, when a user visits a store, the information processing device 10 receives the user's biometric information from the store's terminal (S12). Next, the authentication control unit 12 controls biometric authentication based on the user's biometric information (S13). If biometric authentication is successful (Yes in S14), the transmission control unit 15 transmits the provisional order information associated with the user's biometric authentication to the store terminal (S15). Then the information processing device 10 terminates processing. On the other hand, if biometric authentication fails (No in S14), the transmission control unit 15 terminates processing.

In this way, the information processing device 10 receives a provisional order from the user in advance, and once the user's identity is verified by biometric authentication upon arrival at the store, it transmits the information related to the provisional order to the store. By allowing users to place provisional orders in advance, the hassle of ordering upon arrival at the store can be significantly reduced. This is particularly effective in cafes and other establishments where detailed customization is possible, as users can place their desired order without having to explain the details of their customization to the staff upon arrival.

<Embodiment 2>
Next, Embodiment 2 of the present disclosure will be described. Embodiment 2 is a specific example of Embodiment 1. Figure 3 is a block diagram showing the overall configuration of the information processing system 1000 according to Embodiment 2. The information processing system 1000 is a system consisting of one or more computers that enables a user to place a provisional order for store items before visiting the store. In Embodiment 2, the store is a cafe and the items are food menus, but it is not limited to these.

The information processing system 1000 comprises a facial recognition device 100, an information processing device (hereinafter referred to as a server) 200, a user terminal 300, and one or more store terminals 400. Each device and terminal is connected to the others via a network N. The network N is a wired or wireless communication line.

The user terminal 300 is a personal computer, smartphone, tablet, or other information terminal used by the user. The user uses the user terminal 300 to register as a member and place a provisional order.

Each store terminal 400 is an information terminal installed in each store. For example, a store terminal 400 is a cash register in a store.

The store terminal 400 accepts facial recognition requests from users when they visit the store. In this second embodiment, the store terminal 400 captures at least the face of the user to be authenticated and transmits the facial image (hereinafter sometimes referred to as the captured image) or facial feature information extracted from the facial image to the server 200. Facial recognition is performed by the facial recognition device 100 via the store terminal 400 and the server 200.

If the user's facial recognition is successful, the store terminal 400 receives and displays the user's provisional order information. The store employee who views the provisional order information confirms with the user whether they wish to accept the items listed in the provisional order as a final order, and enters the order intention into the store terminal 400. Alternatively, the user who views the display may enter their order intention into the store terminal 400. The store terminal 400 accepts the final order based on the provisional order information and the order intention. This final order is a legitimate order. Upon receiving the final order, the store terminal 400 prompts the store employee to provide the items listed in the final order.

The facial recognition device 100 is an example of a biometric authentication device and is a facial recognition device that stores facial feature information of multiple people. The facial recognition device 100 registers the user's facial feature information when a member is registered, as described later. At this time, the facial recognition device 100 issues a user ID to the user. The user ID is information that identifies the user, for example, a membership number. The facial recognition device 100 also has a facial recognition function that, in response to a facial recognition request received from an external source, compares the facial image or facial feature information included in the request with the facial feature information of each user. In this embodiment 2, the facial recognition device 100 obtains a facial image of a visiting user from the store terminal 400 via the server 200 and performs facial recognition using the facial image. The facial recognition device 100 then returns the comparison result (facial recognition result) to the server 200.

Server 200 is an example of the information processing device 10 of Embodiment 1.
First, the server 200 registers the user in response to a registration request from the user terminal 300.

When the server 200 receives a provisional order request from a user terminal 300 of a registered member, it requests the facial recognition device 100 to perform facial recognition on the user's facial image or facial feature information. The server 200 then identifies the user ID from the facial recognition result received from the facial recognition device 100. When the server 200 receives provisional order information from a user terminal of a registered member, it registers the provisional order information in association with the user ID.

Furthermore, when the server 200 receives a request from the store terminal 400 for output of provisional order information from a user who has visited the store, it requests the facial recognition device 100 to perform facial recognition on the user's facial image or facial feature information. The server 200 then identifies the user ID from the facial recognition result received from the facial recognition device 100. The server 200 then transmits the provisional order information registered in association with the identified user ID to the store terminal 400.

Figure 4 is a block diagram showing the configuration of the face recognition device 100 according to Embodiment 2. The face recognition device 100 comprises a face information DB (Database) 110, a face detection unit 120, a feature point extraction unit 130, a registration unit 140, and an authentication unit 150. The face information DB 110 stores user IDs 111 and the user's face feature information 112 in association with each other. The face feature information 112 is a set of feature points extracted from a face image. The face recognition device 100 may delete the face feature information 112 in the face feature DB 110 in response to a request from the registered user. Alternatively, the face recognition device 100 may delete the face feature information 112 after a certain period of time has elapsed since its registration.

The face detection unit 120 detects face regions included in the registration image for registering face feature information and supplies them to the feature point extraction unit 130. The feature point extraction unit 130 extracts feature points from the face regions detected by the face detection unit 120 and supplies the face feature information to the registration unit 140. The feature point extraction unit 130 also extracts feature points included in the face image received from the server 200 and supplies the face feature information to the authentication unit 150.

The registration unit 140 issues a new user ID 111 when registering facial feature information. The registration unit 140 associates the issued user ID 111 with the facial feature information 112 extracted from the registered image and registers it in the facial information DB 110. The authentication unit 150 performs facial authentication using the facial feature information 112. Specifically, the authentication unit 150 compares the facial feature information extracted from the facial image with the facial feature information 112 in the facial information DB 110. The authentication unit 150 returns to the server 200 whether or not the facial feature information matches. Whether or not the facial feature information matches corresponds to the success or failure of authentication. Note that "match" of facial feature information means that the degree of matching is equal to or greater than a predetermined value.

Figure 5 is a flowchart showing the flow of the face information registration process according to Embodiment 2. First, the face recognition device 100 acquires the user's registration image included in the face registration request (S21). For example, the face recognition device 100 receives the face registration request from the server 200 that received the member registration request from the user terminal 300 via the network N. However, the face recognition device 100 is not limited to this and may also receive the face registration request directly from the user terminal 300. Next, the face detection unit 120 detects the face region included in the registration image (S22). Next, the feature point extraction unit 130 extracts feature points from the face region detected in step S22 and supplies the face feature information to the registration unit 140 (S23). Finally, the registration unit 140 issues a user ID 111 and registers the user ID 111 and the face feature information 112 in the face information DB 110 (S24). Alternatively, the facial recognition device 100 may receive facial feature information 112 from the facial registration requester and register it in the facial information DB 110 in association with the user ID 111.

Figure 6 is a flowchart showing the flow of the face recognition process according to Embodiment 2. First, the feature point extraction unit 130 acquires face feature information for authentication (S31). For example, the face recognition device 100 receives a face recognition request from the server 200 via the network N and extracts face feature information from the face image included in the face recognition request as in steps S21 to S23. Alternatively, the face recognition device 100 may receive face feature information from the server 200. Next, the authentication unit 150 compares the acquired face feature information with the face feature information 112 in the face information DB 110 (S32). If the face feature information matches, that is, if the degree of matching of the face feature information is greater than or equal to a predetermined value (Yes in S33), the authentication unit 150 identifies the user ID 111 of the user whose face feature information matched (S34). The authentication unit 150 then returns to the server 200 as a face recognition result, indicating that face recognition was successful and the identified user ID 111 (S35). If no matching facial feature information exists (No in S33), the authentication unit 150 returns a message to the server 200 indicating that facial authentication failed as the facial authentication result (S36).

Figure 7 is a block diagram showing the configuration of a user terminal 300 according to Embodiment 2. The user terminal 300 includes a camera 310, a storage unit 320, a communication unit 330, a display unit 340, an input unit 350, and a control unit 360.

The camera 310 is a camera that takes pictures according to the control of the control unit 360. The memory unit 320 is a storage device that stores programs for realizing the various functions of the user terminal 300. The communication unit 330 is a communication interface with the network N. The display unit 340 is a display device. The input unit 350 is an input device that receives input from the user. The display unit 340 and the input unit 350 may be configured as an integrated unit, such as a touch panel. The control unit 360 controls the hardware of the user terminal 300.

Figure 8 is a block diagram showing the configuration of a store terminal 400 according to Embodiment 2. The store terminal 400 includes a camera 410, a storage unit 420, a communication unit 430, a display unit 440, an input unit 450, and a control unit 460.

The camera 410 is a camera that takes pictures according to the control of the control unit 460. The memory unit 420 is a storage device that stores programs for realizing each function of the store terminal 400. The communication unit 430 is a communication interface with the network N. The display unit 440 is a display device. The input unit 450 is an input device that accepts input. The display unit 440 and the input unit 450 may be configured as an integrated unit, such as a touch panel. The control unit 460 controls the hardware of the store terminal 400.

Figure 9 is a block diagram showing the configuration of the server 200 according to Embodiment 2. The server 200 comprises a storage unit 210, a memory 220, a communication unit 230, and a control unit 240.

The storage unit 210 is a storage device such as a hard disk or flash memory. The storage unit 210 stores the program 211, the user DB 212, the provisional order DB 213, the store DB 214, and the final order DB 215. The program 211 is a computer program on which the processing of the information processing method according to this embodiment 2 is implemented.

User DB 212 is a database that stores user information of member users. Specifically, User DB 212 stores user information 2122 associated with user ID 2121. For example, user information 2122 includes attribute information, account information for payment, and contact information such as telephone number and email address.

The provisional order database 213 is a database that stores provisional order information. Specifically, the provisional order database 213 stores provisional order information 2132 associated with user ID 2131. For example, provisional order information 2132 includes a store ID for identifying the store the user plans to visit, and item information related to the provisional order. The item information related to the provisional order specifies the type of item, quantity, whether or not it is customized, and if so, the details of the customization. Furthermore, provisional order information 2132 may also include the time the provisional order was received. Since the user ID is associated with facial feature information in the facial recognition device 100, it can be said that the provisional order information is linked to the facial feature information via the user ID.

The store database 214 is a database that stores the items that each store can offer. Specifically, the store database 214 includes a store ID 2141 and an item list 2142. The item list 2142 lists the items that the store can offer and the customization menus that the store can accommodate. In this embodiment 2, since the store is a cafe, the item list 2142 is also called a menu.

The main order database 215 stores the order information. Specifically, the main order database 215 stores the order information 2152 associated with the user ID 2151. For example, the order information 2152 includes a store ID to identify the store where the user placed the order, and item information related to the order. The item information related to the order includes information that identifies the type, quantity, whether or not customization is performed, and, if customization is performed, the details thereof. The order information 2152 may also include the time the order was received. Here again, since the user ID is associated with facial information in the facial recognition device 100, it can be said that the order information is associated with facial information via the user ID.

Memory 220 is a volatile storage device such as RAM (Random Access Memory), and is a storage area for temporarily holding information when the control unit 240 is operating. The communication unit 230 is a communication interface with the network N.

The control unit 240 is a processor, or control device, that controls each component of the server 200. The control unit 240 loads the program 211 from the storage unit 210 into the memory 220 and executes the program 211. In this way, the control unit 240 realizes the functions of the registration unit 241, the authentication control unit 242, the determination unit 243, the additional identification unit 244, the transmission control unit 245, and the generation unit 246.

The registration unit 241 is an example of the registration unit 11 in Embodiment 1.
When the communication unit 230 receives a member registration request including a registration image from the user terminal 300, the registration unit 241 transmits a face registration request to the face recognition device 100. The registration unit 241 then receives the issued user ID from the face recognition device 100, which has registered the face feature information. The registration unit 241 also receives user information from the user terminal 300 and registers the issued user ID and user information in the user database 212.

Furthermore, when the registration unit 241 receives provisional order information from the user terminal 300 of a user who has successfully undergone facial recognition, it associates the provisional order information with the user ID and registers it in the provisional order DB 213. At this time, the user terminal 300 may send provisional order information for each store, and the registration unit 241 may associate the provisional order information with the user ID and register it in the provisional order DB 213 for each store.

Furthermore, when the registration unit 241 receives order information, including the user's intention to order at the time of their visit, from the store terminal 400, it associates this order information with the user ID and registers it in the order database 215. This allows for a record of the user's actual orders.

The authentication control unit 242 is an example of the authentication control unit 12 of Embodiment 1. The authentication control unit 242 identifies the user's user ID by controlling facial recognition based on a facial image. Specifically, when the authentication control unit 242 receives a facial image from a user terminal 300 or a store terminal 400, it transmits a facial recognition request including the facial image or facial feature information extracted from the facial image to the facial recognition device 100 via the network N. The authentication control unit 242 then obtains the facial recognition result from the facial recognition device 100. If facial recognition is successful, the authentication control unit 242 extracts the user ID from the facial recognition result.

The determination unit 243 is also referred to as the determination means. If facial recognition is successful, the determination unit 243 determines whether the store visited by the user is the store specified by the user when placing a provisional order. Specifically, if facial recognition is successful, the determination unit 243 determines whether the store visited by the user is the store identified by the store ID included in the provisional order information of the provisional order DB 213. The determination unit 243 supplies the determination result to the transmission control unit 245.

The additional identification unit 244 is also referred to as the additional identification means. If facial recognition is successful, the additional identification unit 244 reads the item information of the provisional order from the provisional order DB 213 and identifies other items that are recommended to be ordered in combination with the item information indicated by the item. These other items are also referred to as additional items. Additional items are, for example, side dishes that are compatible with the menu items in the provisional order. For example, the server 200 maintains a table that associates items that are generally defined as compatible combinations with each other, and the additional identification unit 244 may use this table to identify additional items when facial recognition is successful. Alternatively, the additional identification unit 244 may identify additional items based on the user's past order history. Specifically, if facial recognition is successful, the additional identification unit 244 reads the user's order history from the order DB 215 and analyzes combinations of items that the user prefers to order based on the order history. The additional identification unit 244 then identifies additional items based on the analysis results.

The transmission control unit 245 is an example of the transmission control unit 15 in Embodiment 1. First, the transmission control unit 245 transmits an item list to the user terminal 300 that made the provisional order request.

The transmission control unit 245 then transmits the provisional order information when the user visits the store specified during the provisional order process. Specifically, if the determination unit 243 determines that the store visited by the user is the store identified by the store ID included in the provisional order information 2132 of the provisional order DB 213, the transmission control unit 245 transmits the provisional order information to the store terminal 400 at that store. Upon receiving the provisional order information, the store terminal 400 displays the provisional order information on the display unit 440, and the store staff who view it prompt the user to place a final order.

On the other hand, if the determination unit 243 determines that the store visited by the user is not the store identified by the store ID included in the provisional order information, the transmission control unit 245 will not transmit the provisional order information to the store's terminal 400. This prevents provisional order information from being transmitted to a store unintended by the user.

Furthermore, when the transmission control unit 245 transmits the provisional order information to the store terminal 400, it also transmits the item information of the additional items identified by the additional item identification unit 244 to the store terminal 400. The item information of the additional items is also referred to as additional item information. The transmission of the additional item information may be simultaneous with the transmission of the provisional order information, or it may be performed before or after the transmission of the provisional order information. Upon receiving the additional item information, the store terminal 400 displays the additional item information on the display unit 440, and the store clerk who views it recommends the additional items to the user.

The generation unit 246 generates promotional information for each user based on the history of the order information stored in the order database 215. The promotional information includes at least one of the following: information about items recommended to the user and discount information for those items. By providing benefits to users in this way, active use of the service can be encouraged.

Figure 10 is a sequence diagram showing the flow of the member registration process according to Embodiment 2. First, the user terminal 300 takes a photograph of user U (S500), includes the registration image generated by the photograph in the member registration request, and sends it to the server 200 (S501). Then, the registration unit 241 of the server 200 includes the registration image included in the received member registration request in the face registration request and sends it to the face recognition device 100 (S502). Then, the face recognition device 100 registers the face information (face feature information) of user U based on the registration image included in the received face registration request (S503). Then, the face recognition device 100 notifies the server 200 of the issued user ID (S504). The user terminal 300 also sends user information to the server 200 (S505). At this time, the user information may include attribute information, account information, and contact information. The registration unit 241 of the server 200 registers the notified user ID and user information in the user DB 212, associating them with each other (S506).

Figure 11 is a sequence diagram showing the flow of the provisional order processing according to Embodiment 2. The provisional order processing is performed after the member registration processing. First, the user terminal 300 takes a picture of user U (S510). Then the user specifies the store for which the provisional order will be placed. The user terminal 300 includes the face image generated by the picture and the store ID of the specified store in the provisional order request and sends it to the server 200 (S511).

The authentication control unit 242 of the server 200 sends a face authentication request for the face region of user U in the received face image to the face authentication device 100 (S512). The face authentication device 100 then performs face authentication on the face region of user U in the face image included in the received face authentication request (S513). Here, it is assumed that there was a user ID that was successfully authenticated. The face authentication device 100 sends the face authentication result, including the fact that face authentication was successful and the user ID, to the server 200 (S514). At this time, the server 200 may notify the user terminal 300 whether or not face authentication was successful.

Upon receiving the facial recognition result, the transmission control unit 245 of the server 200 reads the item list associated with the store ID included in the provisional order request from the store DB 214 and transmits it to the user terminal 300 (S515).

The user terminal 300, having received the item list, displays the item list on the display unit 340 (S516). The user terminal 300 then accepts the user's selection of the type, quantity, and whether or not customization is required for the provisional order, and if customization is required, the details of the customization (S517).

Figure 12 shows an example of the display of the user terminal 300 according to Embodiment 2. The display unit 340 shows the menu of store A, which the user specified when making a provisional order request. On this display screen, the user can select main menus and side menus as item types. Furthermore, on this display screen, the user can specify cooking preferences and preferred toppings for the selected menu. In this example, the cooking preference is the preferred roast level of the coffee beans. The user terminal 300 receives the selected information as item information.

In this way, users can review and specify their menu and customizations in detail before visiting the store, saving them the trouble of having to explain their order in detail upon arrival. Furthermore, the time required for ordering upon arrival is reduced, alleviating congestion at the checkout counter.

Returning to Figure 11, the explanation continues. The user terminal 300, having received the selection operation, includes the item information corresponding to the selection operation and the store ID of that store in the provisional order information, and sends the provisional order information to the server 200 (S518).

The registration unit 241 of the server 200, upon receiving the provisional order information, registers the provisional order information in the provisional order DB 213, associating it with the user ID identified from the facial recognition result (S519). At this point, the provisional order information does not need to be transmitted to the store terminal 400.

Furthermore, if the registration unit 241 receives a request from the user terminal 300 to cancel the provisional order information, it may delete the provisional order information from the provisional order DB 213. This allows the user to easily cancel the provisional order from the user terminal 300 even if they are unable to pick up the items related to the provisional order.

Figure 13 is a sequence diagram showing the flow of the order processing according to Embodiment 2. First, the store terminal 400 takes a photograph of the user who has come to the store's cash register (S520). Then, the store terminal 400 sends the face image generated by the photograph and the store ID of that store to the server 200 in an output request (S521). The information processing system 1000 executes S522 to S523 in the same way as S512 to S513. Here, it is assumed that there was a user ID that was successfully authenticated. The face authentication device 100 sends the face authentication result, including the fact that face authentication was successful and the user ID, to the server 200 (S524). At this time, the server 200 may notify the store terminal 400 whether or not face authentication was successful. The authentication control unit 242 of the server 200 extracts the user ID included in the face authentication result.

Next, the determination unit 243 of the server 200 uses the extracted user ID to identify the provisional order information associated with that user ID in the provisional order DB 213 (S525). Then, the determination unit 243 compares the store ID of the store terminal 400 that requested the output with the store ID included in the provisional order information and determines whether they match (S526). Here, it is assumed that they match.

Next, the additional identification unit 244 identifies additional items corresponding to the items in the item information included in the provisional order information (S527).

The transmission control unit 245 transmits the item information of the provisional order information and the item information of the additional items to the store terminal 400 according to the result of the matching determination by the determination unit 243 (S528).

The store terminal 400, having received the item information, displays the received item information on the display unit 440 (S529). The store terminal 400 then accepts input from the store employee who confirmed the user's intention to order, or from the user themselves (S530).

Figure 14 shows an example of the display of the store terminal 400 according to Embodiment 2. The display unit 440 displays the items and customizations for the provisional order, and a message asking whether the user wants to place a final order for the items in the provisional order. The user confirms the contents of the provisional order and responds with their intention to place an order for the items and customizations in the provisional order.

The store terminal 400, having received the response, then displays the display screen shown in Figure 15. Figure 15 is a diagram showing an example of the display of the store terminal 400 according to Embodiment 2. The display unit 440 displays the contents of the additional items. The user confirms the contents of the additional items and responds with their intention to order the additional items.

The store terminal 400 then generates the final order information based on the order intentions for the items and customizations related to the provisional order, as well as the order intentions for additional items.

The store terminal 400 prompts store staff to provide the ordered items by displaying the order information on the display unit 440 or on other terminals within the store. This allows users to easily order the items they want while minimizing the effort required to order upon arrival. In cases where the items are food or beverage products or meal menus, as in this example, the user can receive freshly prepared food after arriving by making a provisional order upon arrival.

Furthermore, if the items in the provisional order are not to be made into a final order, the store terminal 400 accepts requests from the store clerk or the user themselves to select the type, quantity, whether or not customization is required, and, if so, the details of any customization. At this time, the store terminal 400 may display a screen on the display unit 440 similar to the display screen shown in Figure 12. The store terminal 400 then accepts the selected information as the final order information. This allows the user to order items that match their mood at the time of their visit.

Returning to Figure 13, let's continue the explanation. The store terminal 400 that generated the order information includes the store's store ID in the order information and sends it to the server 200 (S531).
The registration unit 241 of the server 200 registers the order information in the order database 215, associating it with the user ID included in the facial recognition result (S532).

Furthermore, at this time, the server 200 may perform the payment using the account information associated with user ID 2121 in the user database 212. This eliminates the hassle of payment at the time of the customer's visit.

Thus, according to Embodiment 2, the ordering process for users upon arrival at the store is conveniently streamlined, while enabling flexible ordering that matches the user's prior preferences or mood at the time. Furthermore, the server 200 can contribute to increasing store sales by recommending additional items that match the provisional order.

Figure 16 is a sequence diagram showing the flow of the promotional information provision process according to Embodiment 2. The promotional information provision process may be triggered by the actual order, or it may be performed periodically, for example, regardless of the timing of the actual order. Furthermore, the promotional information provision process is performed for each user.

First, the generation unit 246 of the server 200 reads the history of the order information associated with the user's user ID in the order DB 215 (S535). Then, the generation unit 246 generates promotional information based on the history of the order information (S536). For example, the generation unit 246 may analyze the user's preferences from the history of the order information for each user and identify the type of item or store they prefer based on the analysis results. Then, the generation unit 246 may generate promotional information related to the identified store or item type as promotional information for the user.

The analysis of preferences may also be based on the frequency of store visits and item purchases calculated from the order history. Furthermore, the analysis of preferences may also be based on user attribute information in addition to the order history.

For example, the generation unit 246 may analyze the time of day each user uses the store and generate promotional information for items suitable for the time of day when the user is most likely to use the store, as promotional information for the user.

The generation unit 246 then transmits the promotional information to the user's user terminal 300 (S537). The user terminal 300, upon receiving the promotional information, notifies the user by displaying the promotional information on the display unit 340, etc. (S538).

Figure 17 shows an example of the display of the user terminal 300 according to Embodiment 2. The display unit 340 displays promotional information recommended for user X. As an example, the display unit 340 displays discount information from store A, which user X frequents, and recommendation information for items from other stores related to items that user X frequently purchases.

In this way, server 200 can make users feel that they are getting a benefit by providing them with personalized promotional information. As a result, it can encourage users to use the service.

Furthermore, this disclosure can be implemented by having a processor execute a computer program to perform any desired process.

In the examples described above, the program includes a set of instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more of the functions described in the embodiments. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrically, optically, acoustically, or otherwise propagating signals.

Furthermore, this disclosure is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention. For example, in Embodiment 2 described above, the facial recognition function was provided by the facial recognition device 100, but the server 200 may have the facial recognition function instead of or in addition to the facial recognition device 100.

Furthermore, in the above embodiment 2, the store terminal 400 confirmed the user's intention to order at the time of the order, generated the order information, and then prompted the store clerk to provide the items for the order. However, the store terminal 400 may generate the provisional order information as the order information without waiting for confirmation of the intention to order, and then prompt the store clerk to provide the items for the order.

Furthermore, the user terminal 300 may receive multiple provisional order entries from the user for the same store and register these multiple provisional order entries in the provisional order DB 213 of the server 200. For example, the user may input the details of the customization when ordering a blended coffee from store A as the first provisional order entry, and the user may input the details of the customization when ordering a cafe latte from store A as the second provisional order entry.

The registration unit 241 of the server 200 may associate the multiple provisional order information with the user ID and register it in the provisional order DB 213. When a user visits the store, the transmission control unit 245 may send all of the multiple provisional order information associated with the user ID of the user whose facial recognition was successful to the store terminal 400, or it may select at least one and send it to the store terminal 400. If the transmission control unit 245 sends multiple provisional order information to the store terminal 400, the store terminal 400 may display the multiple provisional order information so that the user can select the provisional order information to be used as the final order information. This allows the user to easily place an order according to their mood at the time.

Furthermore, the provisional order information in the provisional order DB 213 may include a group ID that identifies a group of stores belonging to the same chain, in place of or in addition to the store ID. If they belong to the same group, the items they handle are likely to be the same or similar. Therefore, the transmission control unit 245 may send a common item list to the user terminal 300 when a provisional order request is made, provided the group ID is the same. The registration unit 241 may then register the provisional order information and user ID in the provisional order DB 213 for each group. The determination unit 243 may then determine, upon a user's visit, whether the group to which the visited store belongs is identified by the group ID included in the provisional order information in the provisional order DB 213. The transmission control unit 245 transmits the provisional order information to the store terminal 400 if the group to which the visited store belongs is identified by the group ID included in the provisional order information; otherwise, it does not need to transmit it. This allows users to appropriately provisionally order their desired items regardless of the branch, as long as the store belongs to the same group.

Furthermore, if the provisional order information includes a group ID, the additional identification unit 244 may select additional items from items common to the entire group. The generation unit 246 may also generate promotional information for items that can be used commonly within the same group.

Here, even with the same group ID, the basic menu and limited menu may differ from store to store. Therefore, the transmission control unit 245 may send different item lists to the user terminal 300 when a provisional order is requested if the stores are different, even if they have the same group ID. Also, the additional item selection unit 244 may select additional items for each store if the stores are different, even if they have the same group ID. For example, the additional item selection unit 244 may prioritize selecting items that are exclusive to the visited store or items recommended by the visited store. When the generation unit 246 generates promotional information, it may also generate different promotional information for different items for different stores, even if they belong to the same group.

Furthermore, if a predetermined amount of time has elapsed since the provisional order information was registered in the provisional order DB 213, the server 200 may invalidate the provisional order information. In this case, the registration unit 241 may delete the provisional order information from the provisional order DB 213, or the transmission control unit 245 may choose not to transmit the provisional order information and the item information of the additional items to the store terminal 400. This makes it possible to reflect the pre-registered content in the actual order according to the user's current mood, while allowing for cancellation without any operation if the user's mood changes or they are unable to receive the order.

Furthermore, in the above embodiment 2, the additional item identification unit 244 of the server 200 identifies additional items that are compatible with the items in the provisional order, and the transmission control unit 245 recommends the additional items via the store terminal 400. In this case, the additional item identification unit 244 may recommend different additional items depending on how the user uses the store. For example, the additional item identification unit 244 may use a first table to identify additional items if the user is using the eat-in service, and use a second table different from the first table to identify additional items if the user is using the take-out service. For example, the first table is a table that associates compatible items with each other, and the second table is a table that associates compatible items from the take-out menu with each other. As an example, if the user has provisionally ordered a blended coffee, the server 200 may recommend a cake as an additional item if the user wishes to use the eat-in service, and a sandwich if the user wishes to use the take-out service. This makes it possible to provide recommendations that match the actual usage.

Furthermore, the server 200 may recommend additional items only if the user wishes to use the store in a predetermined manner. For example, the additional item identification unit 244 may identify additional items if the user wishes to eat in, but does not need to identify additional items if the user wishes to take out. In this case, the first table described above may be used to identify the additional items.

Furthermore, in the above embodiment 2, the output request for provisional order information including the user's facial image was received from the store terminal 400, but it may also be received from the user terminal 300 instead of the store terminal 400. In this case, the user terminal 300 may identify the store ID from its own location information and include it in the output request, or it may include its own location information in the output request instead of the store ID. Also in this case, the transmission control unit 245 of the server 200 may send the provisional order information and the item information of the additional items to the user terminal 300, or it may send them to the store terminal 400 of the store visited by the user.

Some or all of the above embodiments may also be described as follows, but are not limited to the following:
(Note 1)
When provisional order information, which includes at least item information for identifying the items in the provisional order, is received from a user terminal used by the user, a registration means registers the provisional order information in association with the user's biometric information.
When biometric information of a user who has visited the store is received, an authentication control means for controlling biometric authentication is provided,
An information processing apparatus comprising: transmission control means for transmitting the provisional order information associated with the user's biometric information to a terminal at the store visited by the user if the biometric authentication is successful.
(Note 2)
The aforementioned provisional order information further includes a store ID that identifies the store,
The information processing device further includes a determination means for determining whether the store visited by the user is a store identified by the store ID included in the provisional order information associated with the user's biometric information, if the biometric authentication is successful.
The information processing device according to Appendix 1, wherein the transmission control means does not transmit the provisional order information to the terminal of the store visited by the user if the store visited by the user is not the store identified by the store ID.
(Note 3)
If the biometric authentication is successful, the system further provides additional identification means for identifying additional items that are recommended to be ordered in combination with the items in the provisional order.
The information processing device according to Appendix 1 or 2, wherein the transmission control means transmits information of the identified additional items together with the provisional order information to the store's terminal.
(Note 4)
The information processing device described in Appendix 3, wherein the registration means receives order information, including the user's intention to order at the time of the visit, from a terminal at the store visited by the user, and registers the order information in association with the user's biometric information.
(Note 5)
The system further comprises a generation means for generating promotional information for the user based on the history of the order information associated with the user's biometric information,
The transmission control means transmits the promotional information to the user terminal as described in Appendix 4.
(Note 6)
The aforementioned additional identification means is,
If the biometric authentication is successful, read the order history associated with the user's biometric information.
Based on the order history mentioned above, additional items will be identified,
The information processing device according to Appendix 4 or 5, wherein the transmission control means transmits the identified additional items together with the provisional order information to the store's terminal.
(Note 7)
The aforementioned item is a meal menu,
The information processing device according to any one of the appendices 3 to 5, wherein the additional identification means identifies the additional item using a first table that associates compatible eat-in menus when the user is using the eat-in service.
(Note 8)
The information processing device described in Appendix 7, wherein the additional identification means identifies the additional item using a second table that associates compatible takeout menus when the user is using the takeout service.
(Note 9)
A biometric authentication device that performs biometric authentication,
Equipped with a server,
The aforementioned server,
When provisional order information, which includes at least item information for identifying the items in the provisional order, is received from a user terminal used by the user, a registration means registers the provisional order information in association with the user's biometric information.
Authentication control means that, upon receiving biometric information of a user who has visited the store, transmits the biometric information to the biometric authentication device and receives the result of the biometric authentication from the biometric authentication device,
An information processing system having a transmission control means that, if the biometric authentication is successful, transmits the provisional order information associated with the user's biometric information to the store's terminal.
(Note 10)
The information processing system described in Appendix 9, further comprising the terminals of the aforementioned store.
(Note 11)
When provisional order information, which includes at least item information for identifying the items in the provisional order, is received from the user terminal used by the user, the provisional order information is registered in association with the user's biometric information.
When biometric information of a user who has visited the store is received, biometric authentication is controlled.
An information processing method that, if the biometric authentication is successful, transmits the provisional order information associated with the user's biometric information to the store's terminal.
(Note 12)
When provisional order information, which includes at least item information for identifying the items in the provisional order, is received from the user terminal used by the user, a registration function is provided to register the provisional order information in association with the user's biometric information.
When biometric information of a user who has visited the store is received, an authentication control function is activated to control biometric authentication,
A non-temporary computer-readable medium containing a program for a computer to implement a transmission control function that, if the biometric authentication is successful, transmits the provisional order information associated with the user's biometric information to the store's terminal.

10 Information processing device 11 Registration unit 12 Authentication control unit 15 Transmission control unit 100 Facial recognition device 110 Facial information database
111 User ID
112 Facial feature information 120 Face detection unit 130 Feature point extraction unit 140 Registration unit 150 Authentication unit 200 Information processing device (server)
210 Storage Unit 211 Program 212 User DB
2121 User ID
2122 User Information 213 Provisional Order Database
2131 User ID
2132 Provisional Order Information 214 Store Database
2141 Store ID
2142 Item List 215 Main Order Database
2151 User ID
2152 Order Information 220 Memory 230 Communication Unit 240 Control Unit 241 Registration Unit 242 Authentication Control Unit 243 Judgment Unit 244 Additional Identification Unit 245 Transmission Control Unit 246 Generation Unit 300 User Terminal 310 Camera 320 Storage Unit 330 Communication Unit 340 Display Unit 350 Input Unit 360 Control Unit 400 Store Terminal 410 Camera 420 Storage Unit 430 Communication Unit 440 Display Unit 450 Input Unit 460 Control Unit 1000 Information Processing System

Claims (10)

When provisional order information is received from a user terminal used by a user, which includes at least item information to identify the items in the provisional order and a store ID to identify the store , the registration means registers the provisional order information in association with the user's biometric information.
When biometric information of a user who has visited the store is received, an authentication control means for controlling biometric authentication is provided,
If the biometric authentication is successful, a transmission control means transmits the provisional order information associated with the user's biometric information to a terminal at the store visited by the user .
If the biometric authentication is successful, the system includes a determination means for determining whether the store visited by the user is the store identified by the store ID included in the provisional order information associated with the user's biometric information.
The transmission control means does not transmit the provisional order information to the terminal of the store visited by the user if the store visited by the user is not the store identified by the store ID.
Information processing device. If the biometric authentication is successful, the system further provides additional identification means for identifying additional items that are recommended to be ordered in combination with the items in the provisional order.
The information processing device according to claim 1 , wherein the transmission control means transmits information of the identified additional items together with the provisional order information to the store's terminal. The information processing device according to claim 2, wherein the registration means receives actual order information, including the user's intention to order at the time of the visit, from a terminal at the store visited by the user, and registers the actual order information in association with the user's biometric information. The system further comprises a generation means for generating promotional information for the user based on the history of the order information associated with the user's biometric information,
The information processing apparatus according to claim 3 , wherein the transmission control means transmits the promotional information to the user terminal. The aforementioned additional identification means is,
If the biometric authentication is successful, read the order history associated with the user's biometric information.
Based on the order history mentioned above, additional items will be identified,
The information processing apparatus according to claim 3 or 4 , wherein the transmission control means transmits the identified additional items together with the provisional order information to the store's terminal. The aforementioned item is a meal menu,
The information processing device according to any one of claims 2 to 4 , wherein the additional identification means identifies the additional item using a first table that associates compatible eat-in menus when the user is using the eat-in service. The aforementioned additional identification means identifies the additional item using a second table that associates compatible takeout menus when the user is using takeout.
The information processing apparatus according to claim 6. A biometric authentication device that performs biometric authentication,
Equipped with a server,
The aforementioned server,
When provisional order information is received from a user terminal used by a user, which includes at least item information to identify the items in the provisional order and a store ID to identify the store , the registration means registers the provisional order information in association with the user's biometric information.
Authentication control means that, upon receiving biometric information of a user who has visited the store, transmits the biometric information to the biometric authentication device and receives the result of the biometric authentication from the biometric authentication device,
If the biometric authentication is successful, a transmission control means transmits the provisional order information associated with the user's biometric information to the store's terminal .
If the biometric authentication is successful, the system has a determination means for determining whether the store visited by the user is the store identified by the store ID included in the provisional order information associated with the user's biometric information.
The transmission control means does not transmit the provisional order information to the terminal of the store visited by the user if the store visited by the user is not the store identified by the store ID.
Information processing system. When provisional order information is received from a user terminal used by a user, which includes at least item information to identify the items in the provisional order and a store ID to identify the store , the provisional order information is registered on the server in association with the user's biometric information.
When biometric information of a user who has visited the store is received, biometric authentication is controlled.
If the biometric authentication is successful, the server transmits the provisional order information associated with the user's biometric information to the store's terminal .
If the biometric authentication is successful, the server determines whether the store visited by the user is the store identified by the store ID included in the provisional order information associated with the user's biometric information.
The server will not send the provisional order information to the terminal of the store visited by the user if the store visited by the user is not the store identified by the store ID.
Information processing methods. When provisional order information is received from a user terminal used by a user, which includes at least item information to identify the items in the provisional order and a store ID to identify the store , the system has a registration function that registers the provisional order information in association with the user's biometric information.
When biometric information of a user who has visited the store is received, an authentication control function is activated to control biometric authentication,
If the biometric authentication is successful, a transmission control function transmits the provisional order information associated with the user's biometric information to the store's terminal .
If the biometric authentication is successful, a determination function is provided to determine whether the store visited by the user is the store identified by the store ID included in the provisional order information associated with the user's biometric information.
To make this a reality on a computer ,
The transmission control function will not transmit the provisional order information to the terminal of the store visited by the user if the store visited by the user is not the store identified by the store ID.
program.