JP7600626B2 - Information processing system, information processing device, method, and program - Google Patents

Description

The present invention relates to an information processing system, etc.

There is technology for registering biometric information at stores or online sites. In the technology disclosed in Patent Document 1, if a newly acquired face image matches an already registered face image, the new face image is additionally registered. In addition, Patent Document 2 discloses that face images are registered in an in-store customer list, and are further associated with purchase history.

International Publication No. 2017/146160 JP 2002-41770 A

However, if identification information such as a member ID (identifier) and biometric information have not been registered, there is a problem in that behavioral history such as purchase history and store visit history cannot be registered.

One example of the objective of the present invention is to provide an information processing system that can ascertain a customer's behavioral history before member ID information and biometric information are associated.

The information processing system according to one aspect of the present invention includes a first acquisition means for acquiring biometric information of a customer, an identification information registration means for registering the acquired biometric information and the customer's identification information in association with each other, a behavioral information registration means for registering the customer's behavioral information in association with the identification information, and an update means for updating the identification information associated with the behavioral information and biometric information of the customer to the acquired member identification information when the member identification information of the customer is acquired.

In one aspect of the present invention, an information processing device includes a first acquisition means for acquiring biometric information of a customer, an identification information registration means for registering the acquired biometric information and the customer's identification information in association with each other, a behavioral information registration means for registering the customer's behavioral information in association with the identification information, and an update means for updating the identification information associated with the behavioral information and biometric information of the customer to the acquired member identification information when the member identification information of the customer is acquired.

A method according to one aspect of the present invention acquires biometric information of a customer, associates the acquired biometric information with identification information of the customer and registers the customer's behavioral information in association with the identification information, and when member identification information of the customer is acquired, updates the identification information associated with the behavioral information and biometric information of the customer to the acquired member identification information.

A program in one aspect of the present invention causes a computer to execute a process of acquiring biometric information of a customer, registering the acquired biometric information in association with the customer's identification information, registering behavioral information of the customer in association with the identification information, and, when member identification information of the customer is acquired, updating the identification information associated with the behavioral information and biometric information of the customer to the acquired member identification information.

According to the present invention, it is possible to ascertain a customer's behavioral history before member ID information and biometric information are associated.

FIG. 1 is a block diagram illustrating a configuration example of an information processing system according to a first embodiment. FIG. 2 is a flowchart illustrating an operation example of the information processing system according to the first embodiment. FIG. 3 is a block diagram of a configuration example of an information processing system according to the second embodiment. As shown in FIG. FIG. 4 is an explanatory diagram showing an example of the member DB. FIG. 5 is an explanatory diagram illustrating an example of a face DB. FIG. 6 is an explanatory diagram showing an example in which facial feature amounts are registered in a face DB, and behavior information is registered in the face DB. FIG. 7 is an explanatory diagram showing an example in which facial feature amounts have already been registered in a face DB, and behavioral information is registered in the face DB. FIG. 8 is an explanatory diagram showing an example of updating. FIG. 9 is a flowchart illustrating an operation example of the information processing system according to the second embodiment. FIG. 10 is a block diagram of a configuration example of an information processing system according to the third embodiment. As shown in FIG. FIG. 11 is an explanatory diagram showing an example of updating after confirming a face image. FIG. 12 is a flowchart illustrating an operation example of the information processing system according to the third embodiment. FIG. 13 is a block diagram illustrating a configuration example of an information processing system according to the fourth embodiment. As illustrated in FIG. FIG. 14 is an explanatory diagram showing an example of accepting a request for registration or non-registration. FIG. 15 is a flowchart illustrating an operation example of the information processing system according to the fourth embodiment. FIG. 16 is an explanatory diagram illustrating an example of a hardware configuration of an information processing system.

Below, embodiments of an information processing system, an information processing device, an information processing method, and a program according to the present invention will be described in detail with reference to the drawings. The disclosed technology is not limited to these embodiments.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration example of an information processing system according to a first embodiment. The information processing system 1 manages the member identification information of a customer and the behavior history of the customer who has come to a predetermined location. Here, it is assumed that the customer has registered as a member in advance and has a member card or a member code, but the biometric information has not yet been linked to the member ID information. The information processing system 1 has a first acquisition unit 101, an identification information registration unit 102, a behavior information registration unit 103, and an update unit 104. Each unit may be realized by the same device (e.g., an information processing device). Alternatively, each unit may be realized by a plurality of different devices. For example, the first acquisition unit 101, the identification information registration unit 102, and the behavior information registration unit 103 may be the same device, and the update unit 104 may be a different device.

The first acquisition unit 101 acquires biometric information of a customer. More specifically, the first acquisition unit 101 acquires biometric information of a customer who has come to a predetermined location, for example. The predetermined location is not particularly limited. The predetermined location is a place where people gather, such as a store, an event space, or a station. The store may be a single store, or a commercial facility where multiple stores are gathered. In the case of a store, the biometric information may be acquired near the entrance when a customer visits the store. For example, in the case of an event space, the biometric information may be acquired when the customer enters the event venue. For example, in the case of a station, the biometric information may be acquired when the customer leaves the train onto the station platform or arrives at the station building.

The biometric information is not particularly limited. Examples of biometric information include facial feature information, facial image data, fingerprint or palm print information, iris information, hand or finger vein information, ear information, voice information, or a combination of these. The facial feature amount is hereinafter referred to as a facial feature amount. Here, the facial feature amount information will be described as an example. The format of the facial feature amount information is not particularly limited. In the case of facial feature amount information, unlike facial image data, there is an advantage that it may not correspond to personal information. The first acquisition unit 101 acquires facial feature amounts obtained from an image of a customer. The facial feature amount is obtained, for example, by extracting facial feature points and the like from an image of a face. Note that the technique for extracting the facial feature amount may be an existing technique. Therefore, the technique for extracting the facial feature amount is not particularly limited. The first acquisition unit 101 may have a function for extracting facial feature amounts from an image of a face. In addition, when another device has a function for extracting facial feature amounts from an image of a face, the first acquisition unit 101 may acquire facial feature amount information from the other device via a communication network or the like.

The identification information registration unit 102 registers the acquired biometric information in association with the customer's identification information. The customer's identification information here is, for example, customer ID information. For example, the customer ID information at the time of registration is different from each member's identification information (member ID information) and is information that is assigned to all customers whose biometric information has been acquired. For example, if the member ID is a number starting with A (e.g., "A00001"), the customer ID should be a number starting with B (e.g., "B00001"); it is better to separate the member ID from the newly assigned customer ID. Specifically, the identification information registration unit 102 registers the customer's behavior information in the store in association with the customer ID information associated with the biometric information.

The behavior information registration unit 103 registers the behavior information of the customer in association with the information of the customer ID associated with the biometric information. Examples of customer behavior in a specific location include entering and leaving a store, movement within the store, and purchase history. Specifically, the behavior information of the customer may be, for example, information on the date and time when the customer came to the specific location. For example, if the movement path of the customer in a specific location can be tracked by an imaging device, the behavior information of the customer may be information obtained from the specific movement path of the customer. For example, in the case of a store, the behavior information may be information on which product the customer picked up. Also, the behavior information may be information on which product the customer's gaze was directed at. The technology for tracking the movement path of the customer may be any existing technology. Therefore, the technology for tracking the movement path of the customer is not particularly limited. For example, a system for tracking the movement path of the customer may be linked to the information processing system 1. For example, the movement path of the customer is identified by constantly capturing images of the customer in a specific location by an imaging device.

When the update unit 104 acquires the member ID information of the customer, it updates the customer ID information associated with the behavioral information and the biometric information to the acquired member ID information. The member ID information is identification information associated with the member information. The member information is information including at least one of the customer's name, gender, age, and contact information. For example, in the case of a store, the member ID information may be acquired at the time of payment. In addition, in the case of a station, the member ID information may be acquired when entering or exiting the ticket gate. In the case of an event space, the member ID information may be acquired when receiving a specified member service. Specifically, when tracking the customer's movement line, the update unit 104 updates the customer ID information associated with the characteristic amount of the customer to the acquired member ID information. For example, when tracking the customer's movement line, the update unit 104 updates the customer ID information associated with the movement line having the same time and position at which the member ID information was acquired to the member ID information. Here, the position being the same includes being approximately the same. For example, the position of the movement line of a person who is in the vicinity (within a specified range) of a device that reads member ID information is assumed to be the same as the position where the member ID information was acquired. If the customer movement line is not tracked, the first acquisition unit 101 may acquire the customer's features again when the customer's member ID information is acquired. Then, the update unit 104 updates the customer ID information associated with the same features as the newly acquired customer features to the acquired member ID information.

Figure 2 is a flowchart showing an example of an operation of the information processing system 1 according to the first embodiment. Here, for example, the processing results of each step by the information processing system 1 in Figure 2 are stored in the storage unit. First, the first acquisition unit 101 acquires biometric information (step S101). Next, the identification information registration unit 102 associates the biometric information with customer ID information and registers them (step S102). Then, the behavioral information registration unit 103 associates the behavioral information with customer ID information and registers them (step S103).

Next, the update unit 104 determines whether or not the member ID information has been acquired (step S104). In step S104, the update unit 104 determines whether or not the member ID information has been acquired by a reading device or the like. If the member ID information has been acquired (step S104: Yes), the update unit 104 updates the customer ID information associated with the biometric information to the member ID information (step S105). After step S105, the information processing system 1 ends the operation of the flow. Also, if the member ID information has not been acquired (step S104: No), the information processing system 1 ends the operation of the flow. For example, the update unit 104 may determine that the member ID information has not been acquired when it is detected that the customer has left a specified location. Alternatively, if the answer is No in step S104, and the member ID information has not been acquired even after a certain period of time has elapsed, the information processing system 1 may end the operation of the flow.

In this way, if the answer is No in step S104, steps S101 to S103 are repeated. Specifically, for example, a certain customer visited the store on May 10th. However, the customer did not purchase any products at the store. In such a case, the information processing system 1 ends the operation of the flow as No in step S104. Meanwhile, the same customer visited the store again on May 11th. And the customer presented the membership code when purchasing a product at the store on that day. In such a case, the information processing system 1 starts the process again from step S101 on May 11th, and the answer is Yes in step S104, so in step S105, the information of the customer ID is updated to the information of the membership ID. Specifically, on May 10th, the facial feature amount of the customer and the information of the customer ID (for example, "B00001") are associated with the behavioral information that the customer visited the store on May 10th. Then, on May 11th, the customer's facial features and customer ID information (e.g., "B00002") are associated with behavioral information indicating that the customer visited the store on May 11th. Then, at the time of payment on May 11th, the information processing system 1 acquires the member ID information (e.g., "A00001"). Then, the information processing system 1 updates the customer ID information ("B00001" and "B00002") associated with the customer's facial features to the member ID information ("A00001").

Next, the effects of the first embodiment will be described. For example, the information processing system 1 registers behavioral information in association with customer ID information and biometric information. Then, when member ID information is acquired, the information processing system 1 updates the customer ID information to the member ID information. This enables the information processing system 1 to link, for example, the behavioral history of a customer at a specified location before linking the biometric information to the member ID information to the member ID information. Ultimately, the information processing system 1 can assist in collecting more behavioral history of customers.

For example, a customer may take a picture of his/her face. Then, the customer may upload face image information as online member information. In such a case, even if there are many customers, it is estimated that the number of customers who actually upload face images is small. For example, if only a few customers upload image data for 100 customers, it is difficult to analyze the behavior of multiple customers. On the other hand, in the information processing system 1, the history of the customer's behavior in a specified place before the biometric information is linked to the member ID information can also be linked to the member ID information. Therefore, the information processing system 1 can support the analysis of the behavior of more customers. In addition, the information processing system 1 can support the analysis of customer behavior with greater accuracy. For example, the information processing system 1 can improve the accuracy of analysis when performing an analysis similar to the conversion rate analyzed on an electronic commerce (EC (Electronic Commerce)) site on customers who visit a store. The conversion rate is, for example, the ratio of the number of customers who actually purchased to the number of customers who visited the website. On the other hand, in the case of customers who actually visited a location, for example, an analysis can be made of the ratio of the number of customers who actually performed a specified action to the number of customers who visited a specified location. Alternatively, an analysis can be made on a customer-by-customer basis of the ratio of the number of times a customer actually made a purchase to the number of times the customer visited a store. In this way, the information processing system 1 can also aggregate the behavioral history of customers before linking the member ID information, thereby improving the accuracy of customer behavior analysis.

Furthermore, as described above, facial features have the advantage that, unlike facial image data, they may not be personal information. Compared to facial image data, facial features are less likely to identify an individual if leaked. Therefore, the information processing system 1 uses facial features as biometric information. This allows the information processing system 1 to reduce the risk of a customer being identified if biometric information is leaked.

(Embodiment 2)
Next, the second embodiment will be described in detail with reference to the drawings. In the second embodiment, for example, an example will be described in which it is determined whether newly acquired biometric information has been registered. In the second embodiment, if newly acquired biometric information has not been registered, the customer ID information and the biometric information are newly associated and registered. Below, as long as the explanation of the second embodiment is not unclear, the explanation of the contents that overlap with the above explanation will be omitted. In addition, in the second embodiment, an example will be described in which face features are used as biometric information.

Fig. 3 is a block diagram showing an example of the configuration of an information processing system according to the second embodiment. The information processing system 2 has an information processing device 20, an imaging device 22, and a reading device 23. In the second embodiment, the information processing system 2 is newly provided with the information processing device 20, the imaging device 22, and the reading device 23. In addition, the information processing device 20 is newly provided with each functional unit.

Each device may be the same device. For example, the information processing device 20 and the reading device 23 may be the same device. For example, the imaging device 22 and the reading device 23 may be the same device. There is no particular limitation as to which combinations of devices are the same device.

The imaging device 22 is a device that captures images or videos of customers. The imaging device 22 may capture images of customers at all times, for example, to track the customer's movement line. As shown in FIG. 3 as imaging device 22-1 to imaging device 22-n, the information processing system 2 may have multiple imaging devices 22. Note that n is an integer equal to or greater than 1. There are no particular limitations on the position at which the imaging device 22 is attached.

In FIG. 3, the information processing device 20 has, for example, a first acquisition unit 201, an identification information registration unit 202, a behavior information registration unit 203, an update unit 204, a determination unit 205, and a storage unit 206. Note that each functional unit may be realized by a different device. In the second embodiment, the determination unit 205 and the storage unit 206 are added to the first embodiment. The first acquisition unit 201 has the function of the first acquisition unit 101 according to the first embodiment. The identification information registration unit 202 has the function of the identification information registration unit 102 according to the first embodiment. The behavior information registration unit 203 has the function of the behavior information registration unit 103 according to the first embodiment. The update unit 204 has the function of the update unit 104 according to the first embodiment.

First, the storage unit 206 will be described. The storage unit 206 stores the processing results of each function of the information processing device 20. The storage unit 206 also stores data used in the processing of each unit. The information processing device 20 loads a program including various commands for implementing each functional unit into the storage unit 206 and executes the program, which will be described in detail in a hardware configuration example in FIG. 16 described later. Examples of the storage unit 206 include a ROM (Read Only Memory), a RAM (Random Access Memory), a semiconductor memory, a HDD (Hard Disk Drive), and a SSD (Solid State Drive). The storage unit 206 may also be a combination of these.

In FIG. 3, the memory unit 206 stores, for example, a member DB 2011 and a face DB 2012 as memory contents. The memory contents of the memory unit 206 in FIG. 3 are one example. For example, these various pieces of information may be stored in a memory unit of another device (for example, a storage device) connected to the information processing device 20 via a communication network. The member DB 2011 and the face DB 2012 may be the same as those used on the EC site, for example. For this reason, the member DB 2011 and the face DB 2012 may be stored in a storage device that is available to both the EC site and the information processing system 2. This allows the information processing system 2 to aggregate the behavioral history of customers on both the EC site and the actual store.

Figure 4 is an explanatory diagram showing an example of member DB 2011. In member DB 2011, member ID information, which is member identification information, is associated with member information for each member. In member DB 2011 in Figure 4, row-wise records include member ID information and member information for each member. In Figure 4, member information includes name information, gender information, age information, and contact information. Member information is not limited to these, and may be changed in various ways. Here, for example, member IDs are represented by numbers starting with "A."

FIG. 5 is an explanatory diagram showing an example of the face DB 2012. In the face DB 2012, customer ID information, which is customer identification information, facial feature information, which is biometric information, and behavior history information, which is behavior information, are associated with each other. In the member DB 2011 in FIG. 5, the row-direction records include customer ID information, facial feature information, and behavior history information. In FIG. 5, when the specified location is a store, various examples of behavior history include store visits, store exits, the number of store visits by month, products picked up, products that were looked at for a specified period of time, and positions where the user stood for a specified period of time. In the case of an event space, various examples of behavior history include the time of entry, the time of exit, and stopping by or participating in each event area that the user was interested in. In the case of a station, various examples of behavior history include the time of entry through the ticket gate, the time of exit through the ticket gate, the time of arrival at the station, the time of getting off at the station platform, and whether or not a product was purchased at a store in the station. In the example of Figure 5, when the customer ID information is "B0002", the facial feature value is registered as Y and the behavioral history is registered as "Visited the store on August 1st."

Returning to the explanation of FIG. 3, each functional unit will be explained. The first acquisition unit 201 acquires, for example, biometric information of a customer who has come to a predetermined location. As described above, in the second embodiment, the biometric information is information on the facial features of the customer. For this reason, the first acquisition unit 201 acquires information on the facial features obtained from an image of a customer who has come to a predetermined location. As described in the first embodiment, the predetermined location is not particularly limited. For example, the predetermined location is a location where membership information can be managed for each member by a membership card and a membership code. For example, the image here is captured by the imaging device 22. For example, in the case of a store, the imaging device 22 installed near the entrance captures the face of a customer who has come to the store. The process of extracting facial features from the facial image is not particularly limited. For example, multiple facial feature points may be extracted from the facial image. As described in the first embodiment, the first acquisition unit 201 performs a process of extracting facial features from the facial image. As a result, the first acquisition unit 201 may acquire information on facial features by extraction processing. Alternatively, the first acquisition unit 201 may receive information on facial features from an existing device that extracts facial features.

The determination unit 205 determines whether there is customer ID information associated with the acquired facial feature information. Specifically, the determination unit 205 searches the face DB 2012 for facial feature information that matches the acquired facial feature information.

If the customer ID information associated with the acquired facial feature information is not registered in the face DB 2012, the identification information registration unit 202 registers the acquired facial feature information and the customer ID information in association with each other. The customer ID here is, for example, an ID different from the member ID and the customer ID already registered in the face DB 2012.

In addition, when there is customer ID information associated with the acquired facial feature information, the identification information registration unit 202 does not newly register the facial feature information. In this case, the identification information registration unit 202 may update the registered facial feature information to the newly acquired facial feature information. In addition, even when the biometric information is information other than facial feature information, the identification information registration unit 202 associates the acquired facial feature information with customer ID information and registers it according to the judgment result of the judgment unit 205.

Then, the behavior information registration unit 203 registers the behavior information of the customer at a predetermined location in association with the information of the customer ID. As described in the first embodiment, for example, if the imaging device 22 is attached to the entrance of the store, the behavior information of the customer may be, for example, information indicating whether the customer passed through the entrance gate. The behavior information of the customer may be, for example, information indicating what date and time the customer came, or information indicating what time the customer came and left. For example, the behavior information registration unit 203 may register the behavior information at the timing when the occurrence of the customer's behavior is detected. Specifically, the behavior information registration unit 203 may register the behavior information at the timing when the facial feature amount of the customer is acquired. Also, for example, if the customer's movement line can be tracked by constantly photographing the customer with the imaging device 22, the behavior information of the customer may be information obtained from the specific movement line of the customer. The behavior information is information registered in the behavior history as described above in the face DB 2012 in FIG. 5.

When the member ID information of the customer is acquired, the update unit 204 updates the customer ID information associated with the biometric information to the acquired member ID information. For example, in the case of a store, the member ID information may be acquired at the time of payment. In the case of a station, the member ID information may be acquired when entering or leaving the ticket gate. In the case of an event space, the member ID information may be acquired when receiving a specified member service. Specifically, when the customer's movement path is being tracked, the update unit 204 updates the customer ID information associated with the customer's characteristic amount to the acquired member ID information. When the customer's movement path is not being tracked, the first acquisition unit 201 may acquire the customer's characteristic amount again when the member ID information of the customer is acquired. Then, the update unit 204 updates the customer ID information associated with the same characteristic amount as the newly acquired customer's characteristic amount to the acquired member ID information.

More specifically, when the first acquisition unit 201 acquires the customer's member ID information, the first acquisition unit 201 may acquire the customer's facial features again. Then, the update unit 204 searches the face DB 2012 for facial feature information that matches the acquired facial feature information. The update unit 204 updates the customer ID information associated with the detected facial feature information to the member ID information.

FIG. 6 is an explanatory diagram showing an example in which facial feature amounts are registered in the face DB 2012 and behavioral information is registered in the face DB 2012. In FIG. 6, the first acquisition unit 201 acquires, for example, information on facial feature amount X obtained from a face image. The determination unit 205 determines whether or not the information on facial feature amount X is registered in the face DB 2012. In FIG. 6, since the information on facial feature amount X is not registered in the face DB 2012, the identification information registration unit 202 registers the customer ID information "B00001" and the information on facial feature amount X in the face DB 2012. The customer ID information "B00001" and the information on facial feature amount X are associated with each other. That is, in the face DB 2012, the same record includes the customer ID information "B00001" and the information on facial feature amount X. In FIG. 6, the behavioral history information is empty in the face DB 2012. Then, the behavioral information registration unit 203 registers the behavioral information of the customer at the specified location in association with the customer ID information "B00001." In FIG. 6, new information "Visited on October 1, 2020" has been registered as behavioral history information.

FIG. 7 is an explanatory diagram showing an example in which facial feature information has already been registered in the face DB 2012 and behavioral information is registered in the face DB 2012. In FIG. 7, the first acquisition unit 201 acquires, for example, facial feature X information obtained from the customer's face image 2013. The determination unit 205 determines whether the facial feature X information is registered in the face DB 2012. In FIG. 7, the facial feature X information is registered in the face DB 2012. For this reason, the identification information registration unit 202 does not newly register the facial feature X information in the face DB 2012. In the face DB 2012, information "Visit the store on October 1, 2020" is registered as behavioral history information. In other words, this customer has already visited the store on October 1, 2020. Then, the behavioral information registration unit 203 registers the behavioral information of the customer at a specified location in association with the customer ID information "B00001". In FIG. 6, new information such as "Visited the store on October 5, 2020" and "Picked up product A on October 5, 2020" is registered as behavioral history information. As a result, according to the face DB 2012 after the behavioral information registration in FIG. 7, the customer visited the store on October 1, 2020 and October 5, 2020. Furthermore, according to the face DB 2012, it can be seen that the customer became interested in product A when he visited the store on October 5, 2020.

Returning to the explanation of FIG. 3, when the update unit 204 acquires the customer's member ID information, it updates the customer ID information associated with the facial feature information to the acquired member ID information. For example, the reading device 23 reads the member card or member code presented by the customer. This acquires the customer's member ID information. If the specified location is a store or event space, the reading device 23 reads the member card or member code. For example, the timing of reading can be when making a payment or when receiving a specified service. In this way, the customer's member ID information corresponding to the member card or member code may be acquired. Also, if the specified location is a station, the customer holds a non-contact IC (Integrated Circuit) card over the reading device 23. For example, the timing of holding the IC card over the reading device 23 can be when the customer enters the ticket gate, when exiting the ticket gate, or when purchasing a product from a ticket machine, vending machine, or the like. Then, the reading device 23 reads the non-contact IC card. In this way, the member ID information registered in the IC card may be acquired. In FIG. 3, multiple reading devices 23 may be provided.

Figure 8 is an explanatory diagram showing an example of an update. For example, member ID information "A00001" is acquired. Member ID information "A00001" is registered in member DB 2011. Therefore, the update unit 204 updates the customer ID information "B00001" associated with the facial feature amount X information to member ID information "A00001". As a result, member DB 2011 and face DB 2012 are associated. That is, facial feature amount X and member information are associated via the member ID. In Figure 8, for example, it is possible to identify that the name of a customer having facial feature amount X is "●●". As a result, according to the updated face DB 2012 and member DB 2011 in Figure 8, a 30-year-old female customer named "●●" visited the store on October 1, 2020 and October 5, 2020. Furthermore, according to face DB 2012 and member DB 2011, it is known that this female customer was interested in product A when she visited the store on October 5, 2020. In this way, it becomes possible to know the customer's behavior before the member ID information "A00001" was registered in face DB 2012.

FIG. 9 is a flowchart showing an example of an operation of the information processing system 2 according to the second embodiment. Here, for example, the processing results of each step by the information processing system 2 in FIG. 9 are stored in the memory unit 206 or a storage device connected via a communication network. The first acquisition unit 201 acquires facial feature values obtained from a captured face image (step S201). Then, the determination unit 205 determines whether the facial feature values have been registered in the face DB 2012 (step S202). If they have been registered (step S202: Yes), the determination unit 205 proceeds to step S204. On the other hand, if they have not been registered (step S202: No), the identification information registration unit 202 registers the customer ID information and the facial feature value information in association with each other (step S203).

Next, after step S203 or if the answer is Yes in step S202, the behavioral information registration unit 203 registers the behavioral information of the customer in association with the customer ID information (step S204). Next, the update unit 204 determines whether or not the member ID information has been acquired (step S205). For example, the reading device 23 reads a member card or a member code. As a result, the update unit 204 determines that the member ID information has been acquired (step S205: Yes). If the member ID information has been acquired (step S205: Yes), the update unit 204 updates the customer ID information to the member ID information (step S206). After step S206, the information processing system 2 ends the operation of the flow. Also, if the member ID information has not been acquired (step S205: No), the information processing system 2 ends the operation of the flow.

As explained in the first embodiment, if the answer is No in step S205, steps S201 to S204 are repeated. Specifically, for example, if the customer leaves the store without obtaining member ID information, the flow ends. If the customer returns to the store, steps S201 to S204 are performed again. Also, for example, step S204 may be repeated when tracking the customer's movement path. For example, if it is possible to track the customer's movement path within the store, not just when the customer visits the store, new customer behavior information is registered each time the customer takes some action within the store.

Embodiment 2 is not limited to the above example, and various modifications are possible. We will now explain some modified examples of embodiment 2.

<First Modification of Second Embodiment>
When there is facial feature information that matches the acquired facial feature information, the determination unit 205 may further determine whether or not member ID information is registered as the customer ID information. For example, when member ID information is not registered as the customer ID information, the update unit 204 performs an update process when member ID information is acquired. On the other hand, when member ID information is registered as the customer ID information, the behavior information registration unit 203 registers the behavior information of the customer in association with the member ID information, and the update unit 204 does not need to perform an update process. This allows the information processing system 2 to prevent multiple updates with the same update content.

Here, the method of determining whether or not member ID information is registered as customer ID information is not particularly limited. For example, the determination unit 205 may search for member ID information that matches the customer ID information associated with the facial feature amount information from the member DB 2011. When member ID information that matches the customer ID information is present in the member DB 2011, the determination unit 205 determines that the member ID information has already been registered as the customer ID information. On the other hand, when member ID information that matches the customer ID information is not present in the member DB 2011, the determination unit 205 determines that the member ID information has already been registered as the customer ID information. Also, for example, if a number starting with "A" is the member ID, the determination unit 205 determines that the member ID information has already been registered as the customer ID information in the face DB 2012 when a number starting with "A" is set as the customer ID information. For example, if a number starting with "B" is set as the customer ID information, the determination unit 205 determines that member ID information has not already been registered as the customer ID information in the face DB 2012. Then, the update unit 204 may not perform an update process even if member ID information is acquired.

<Modification 2 of Second Embodiment>
A customer may have multiple membership cards or multiple membership codes. In such a case, the acquired member ID information differs from the member ID information registered in the face DB 2012. In this case, the update unit 204 may additionally register the acquired member ID information as face ID information when the member ID information registered as face ID information in the face DB 2012 differs from the acquired member ID information. For example, assume that the member ID information "A00001" is acquired, but the member ID information "A00002" is associated with the facial feature amount information in the face DB 2012. In such a case, the update unit 204 associates the member ID information "A00002" and the member ID information "A00001" with the facial feature amount information. As a result, in the face DB 2012, the facial feature amount information is linked to the multiple member ID information as customer ID information. Therefore, the information processing system 2 can link all behavior history information with all membership information even if a customer has multiple membership cards or membership codes.

<Third Modification of Second Embodiment>
In the case where a customer has multiple member cards or member codes as in the second modification, in the third modification, the customer may be asked whether he/she has multiple member cards or member codes. Although not shown, for example, when the member ID information registered in the face DB 2012 as face ID information is different from the acquired member ID information, the information processing system 2 may ask the customer whether he/she has multiple member cards, member codes, etc., using an output device or the like. Furthermore, when the member ID information registered in the face DB 2012 as face ID information is different from the acquired member ID information, the information processing system 2 may ask the customer which member ID (member card or member code) to use. Then, the update unit 204 may register the member ID information currently used by the customer, out of the registered member ID information and the acquired member ID information, in the face DB 2012 as face ID information. The member ID information currently used by the customer may be input, for example, via a reception unit (third reception unit) not shown in the second embodiment.

This concludes the description of each variant of embodiment 2. Next, the effects of embodiment 2 will be described. For example, when newly acquired biometric information has not been registered, the information processing system 2 registers new customer ID information and biometric information in association with each other. On the other hand, for example, when newly acquired biometric information has been registered, the information processing system 2 does not register new customer ID information and biometric information in association with each other. This allows the information processing system 2 to aggregate behavioral information of the same customer. Therefore, the information processing system 2 can more easily grasp the behavioral history of a customer before it was linked to membership information.

(Embodiment 3)
Next, a third embodiment will be described in detail with reference to the drawings. In the third embodiment, an example will be described in which, when member ID information is acquired, a face image is displayed to confirm whether the face image is a customer, and then the customer ID information is updated to the member ID information. Below, as long as the explanation of the third embodiment is not unclear, explanations of contents that overlap with the above explanation will be omitted. Also, in the third embodiment, an example will be described in which face features are used as biometric information.

Fig. 10 is a block diagram showing an example of the configuration of an information processing system according to the third embodiment. Information processing system 3 includes the components of information processing systems 1 and 2 according to the first and second embodiments as basic components. Information processing system 3 includes an information processing device 30, a terminal device 31, an imaging device 32, and a reading device 33. Terminal device 31 is added to information processing system 3 from the second embodiment.

Each device may be the same device. For example, the information processing device 30 and the terminal device 31 may be the same device. For example, the terminal device 31 and the reading device 33 may be the same device. For example, the imaging device 32 and the reading device 33 may be the same device. There is no particular limitation as to which combinations of devices are the same device.

The information processing device 30 has the functions of the information processing device 20 according to the second embodiment. The imaging device 32 has the functions of the imaging device 22 according to the second embodiment. The reading device 33 has the functions of the reading device 23 according to the second embodiment.

The information processing device 30 has a first acquisition unit 301, a determination unit 305, an identification information registration unit 302, a behavior information registration unit 303, an update unit 304, a second acquisition unit 307, a display control unit 308, and a first reception unit 309. The information processing device 30 has the second acquisition unit 307, the display control unit 308, and the first reception unit 309 added to the information processing device 30 according to the second embodiment. The first acquisition unit 301 has the functions of the first acquisition units 201 and 201 according to the first and second embodiments. The determination unit 305 has the functions of the determination unit 305 according to the second embodiment. The identification information registration unit 302 has the functions of the identification information registration units 102 and 202 according to the first and second embodiments. The behavior information registration unit 303 has the functions of the behavior information registration units 103 and 203 according to the first and second embodiments. The update unit 304 has the functions of the update units 104 and 204 according to the second embodiment. The memory unit 306 has the functions of the memory unit 206 according to the second embodiment. The member DB 3011 and face DB 3012, which are the stored contents of the memory unit 306, may be the same as the member DB 2011 and face DB 2012 according to the second embodiment. Note that the information processing device 30 according to the third embodiment may not have, for example, the determination unit 305.

The terminal device 31 has, for example, an output device. Examples of the output device include a display device (display unit) that displays images and a printer that prints images. The terminal device 31 also has, for example, an input device that accepts input of operations by a user. Examples of the input device include a keyboard, a mouse, and a touch panel display. Note that the information processing device 30 may have both or either of the output device and the input device. Also, the information processing device 30 may have both or either of the output device and the input device in a device or the like different from the information processing device 30 and the terminal device 31.

The first acquisition unit 301, the determination unit 305, the identification information registration unit 302, and the behavior information registration unit 303 are unchanged from the second embodiment. Detailed description of these will be omitted.

When the member ID information is acquired, the second acquisition unit 307 acquires facial image information of the customer. There is no particular limitation on the data format of the facial image information. Here, there is no particular limitation on the timing at which the facial image is captured by the imaging device 32. For example, the facial image may be captured by the imaging device 32 after the member ID information is acquired. Or, for example, the facial image may be captured by the imaging device 32 in order to obtain facial feature amounts. In other words, the second acquisition unit 307 may acquire a facial image used to extract facial feature amounts.

The display control unit 308, for example, causes the customer's facial image to be displayed on the display unit. The display unit of the terminal device 31 is taken as an example of the display unit. As described above, the display unit may be a display unit provided on the information processing device 30 or another device. Furthermore, the display control unit 308 is not limited to a display unit, and may be an output control unit that outputs the customer's facial image to another output device such as a printer. Specifically, the display control unit 308, for example, causes the facial image to be displayed on the display unit of the terminal device 31.

Next, the first reception unit 309 receives, for example, an input as to whether the displayed image shows the customer's face. The first reception unit 309 receives, for example, an input as to whether the image shows the customer's face via an input device of the terminal device 31. Specifically, the first reception unit 309 receives an input by the customer as to whether the facial image shows the customer's face. For example, the customer checks whether the image displayed on the display unit of the terminal device 31 shows his or her own face. Then, the customer inputs to the first reception unit 309 whether the image shows the customer's face via the input device of the terminal device 31. In this way, the first reception unit 309 receives an input by the customer as to whether the facial image shows the customer's face via the input device of the terminal device 31. The input device may be provided in the information processing device 30 or another device.

Alternatively, the first reception unit 309 receives input from a person in charge at a specified location as to whether or not the facial image shows the face of a customer. Here, in the case of a store, the person in charge may be a store clerk. In the case of an event space, the person in charge may be an event staff member. In the case of a station, the person in charge may be a station staff member. The person in charge checks whether or not the image displayed on the terminal device 31 shows the face of a customer. The person in charge then inputs whether or not the image shows the face of a customer using the input device of the terminal device 31. In this way, the first reception unit 309 receives input from the person in charge as to whether or not the facial image shows the face of a customer via the input device of the terminal device 31.

If it is accepted that the facial image shows the face of a customer, the update unit 304 updates the customer ID information to the acquired member ID information. Also, for example, if it is accepted that the facial image does not show the face of a customer, the update unit 304 does not update the customer ID information to the acquired member ID information.

11 is an explanatory diagram showing an example of updating after confirming a face image. When the member ID information is acquired, the second acquisition unit 307 acquires the customer's face image 3013 information. In FIG. 11, the display control unit 308 displays, for example, the customer's name information "●●" corresponding to the member ID information "A00001" and the face image 3013 on the touch panel display of the terminal device 31. Also, in FIG. 11, the display control unit 308 displays buttons for accepting inputs of "Confirm" and "No" on the touch panel display of the terminal device 31. The terminal device 31 can accept inputs of whether or not to show the customer's face using these buttons. In FIG. 11, for example, if "Confirm" is pressed, it is input that the face image 3013 shows the customer's face. If "No" is pressed, it is input that the face image 3013 does not show the customer's face.

The terminal device 31 accepts pressing of the "Confirm" or "No" button by the customer or store clerk. The first reception unit 309 then accepts the input contents to the touch panel display (input device) of the terminal device 31. In FIG. 11, because "Confirm" was pressed, the update unit 304 updates the customer ID information associated with the facial feature amount X to the acquired member ID information "A00001". Although not shown, if "No" was pressed, the update unit 304 does not update the customer ID information associated with the facial feature amount X to the acquired member ID information "A00001".

Figure 12 is a flowchart showing an example of an operation of the information processing system 3 according to the third embodiment. In Figure 12, steps S301 to S305 are the same as steps S201 to S205 shown in Figure 9. For this reason, explanations of steps S301 to S305 are omitted. In addition, for example, the processing results of each step by the information processing system 3 in Figure 12 are stored in the storage unit 306 or a storage device connected via a communication network.

If step S305 is Yes, the second acquisition unit 307 acquires an image of the customer's face (step S306). Step S306 may be performed at the same timing as step S301, for example. Next, the display control unit 308 displays the face image on the display unit (step S307). Then, the first reception unit 309 receives an input of whether or not the face image shows the customer's face (step S308). If it is input that the face image does not show the customer's face (step S308: No), the information processing system 3 ends the operation of the flow. On the other hand, if it is input that the face image shows the customer's face (step S308: Yes), the update unit 304 updates the customer ID information to the member ID information (step S309). Then, after step S309, the information processing system 3 ends the series of operations.

Next, the effects of the third embodiment will be described. For example, when member ID information is acquired, the information processing system 3 displays a facial image. Then, the information processing system 3 checks whether the facial image is a customer. If the facial image shows the face of a customer, the information processing system 3 updates the customer ID information to member ID information. This allows the information processing system 3 to link correct facial feature information to member ID information.

Facial images may also be checked by the customer. This allows the information processing system 3 to link more accurate facial feature information to member ID information. Facial images may also be checked by a store clerk or other staff member. This allows the information processing system 3 to prevent customer impersonation. This allows the information processing system 3 to link more accurate facial feature information to member ID information.

(Embodiment 4)
Next, a fourth embodiment will be described in detail with reference to the drawings. In the fourth embodiment, an example will be described in which the customer ID information is updated to the member ID information after confirming with the customer whether or not to register the biometric information. Below, the description of the contents that overlap with the above description will be omitted to the extent that the description of the fourth embodiment is not unclear. In addition, in the fourth embodiment, an example will be described in which face features are used as biometric information.

Fig. 13 is a block diagram showing an example of the configuration of an information processing system according to the fourth embodiment. The information processing system 4 includes the basic components of the information processing systems 1, 2, and 3 described in the first, second, and third embodiments. The information processing system 4 includes an information processing device 40, a terminal device 41, an imaging device 42, and a reading device 43. There are no devices or the like that have been newly added to the information processing system 4 from the third embodiment. The information processing device 40 has the functions of the information processing devices 20 and 30 according to the second and third embodiments. The imaging device 42 has the functions of the imaging devices 22 and 32 according to the second and third embodiments. The reading device 43 has the functions of the reading devices 23 and 33 according to the second and third embodiments. The terminal device 41 has the functions of the terminal device 31 according to the third embodiment.

Each device may be the same device. For example, the information processing device 40 and the terminal device 41 may be the same device. For example, the terminal device 41 and the reading device 43 may be the same device. For example, the imaging device 42 and the reading device 43 may be the same device. There is no particular limitation as to which combinations of devices are the same device.

The information processing device 40 has a first acquisition unit 401, a judgment unit 405, an identification information registration unit 402, a behavior information registration unit 403, an update unit 404, a second acquisition unit 407, a display control unit 408, a first reception unit 409, a second reception unit 410, and a memory unit 406. The information processing device 40 has the second reception unit 410 added to the information processing device 40 according to the third embodiment. The first acquisition unit 401 has the functions of the first acquisition units 101, 201, and 301 according to the first, second, and third embodiments. The judgment unit 405 has the functions of the judgment units 205 and 305 according to the second and third embodiments. The identification information registration unit 402 has the functions of the identification information registration units 102, 202, and 302 according to the first, second, and third embodiments. The behavior information registration unit 403 has the functions of the behavior information registration units 103, 203, and 303 according to the first, second, and third embodiments. The update unit 404 has the functions of the update units 104, 204, and 304 according to the first, second, and third embodiments. The first reception unit 409 has the functions of the first reception unit 309 according to the third embodiment. The memory unit 406 has the functions of the memory units 206 and 306 according to the second and third embodiments. The member DB 4011 and face DB 4012, which are the contents stored in the memory unit 406, may be the same as the member DB 2011 and 3011 and the face DB 2012 and 3012 according to the second and third embodiments.

In addition, the information processing device 40 according to the fourth embodiment may not have, for example, the determination unit 405. In addition, the information processing device 40 according to the fourth embodiment may not have the second acquisition unit 407 and the display control unit 408.

The first acquisition unit 401, the determination unit 405, the identification information registration unit 402, the behavior information registration unit 403, the second acquisition unit 407, the display control unit 408, and the first reception unit 409 are unchanged from the third embodiment. Detailed description of these components will be omitted.

When the member ID information is acquired, the second reception unit 410 receives an input from the customer as to whether or not to register the biometric information. The second reception unit 410 receives an input from the customer as to whether or not to register via, for example, an input device of the terminal device 41. More specifically, for example, the second reception unit 410 receives an input from the customer. For example, biometric information may correspond to personal information, such as face image data. Therefore, when the biometric information corresponds to personal information, the information processing device 40 can improve the customer's trust by having the customer decide whether or not to link the member ID information with the biometric information. This allows the information processing system 4 to leave the decision on whether or not to register to the customer's discretion. Furthermore, even when the information does not correspond to personal information, such as face features, the information processing system 4 can leave the decision on whether or not to link the member ID information with the biometric information to the customer's discretion. Therefore, the information processing system 4 can improve the customer's trust.

Next, if it is accepted that the biometric information will be registered, the update unit 404 updates the customer ID information associated with the biometric information to the acquired member ID information. On the other hand, if it is accepted that the biometric information will not be registered, the update unit 404 does not update the customer ID information associated with the biometric information to the acquired member ID information.

Furthermore, if it is accepted that the biometric information will not be registered, the update unit 404 may delete the association between the biometric information, the customer ID information, and the behavioral history information from the face DB 4012. Note that there is no particular limitation as to whether or not to perform this deletion.

14 is an explanatory diagram showing an example of accepting whether or not to register. The display control unit 408 may, for example, display a button on the terminal device 41 that allows the user to input whether or not to register the facial feature amount X. The display unit of the terminal device 41 displays, for example, a facial image 4013, the button, and text on the screen. The text includes the name of the customer acquired from the member DB 4011. Specifically, the text is a text for confirming whether or not to register the facial feature amount. In addition, in FIG. 14, a "Register" button and a "Do not register" button are displayed on the screen of the terminal device 41 together with the facial image 4013. When the customer presses the "Register" button, the second reception unit 410 accepts the registration of the facial feature amount X. On the other hand, when the customer presses the "Do not register" button, the second reception unit 410 accepts the non-registration of the facial feature amount.

In FIG. 14, because the "Register" button has been pressed, the second reception unit 410 accepts that the facial feature amount X is to be registered. The update unit 404 then updates the customer ID information "B00001" associated with the facial feature amount X to the acquired member ID information "A00001". Although not shown, if the "Do not register" button has been pressed, the second reception unit 410 accepts that the facial feature amount X is not to be registered. The update unit 404 then does not update the customer ID information "B00001" associated with the facial feature amount X to the acquired member ID information "A00001".

Figure 15 is a flowchart showing an example of an operation of the information processing system 4 according to the fourth embodiment. In Figure 15, a description will be given taking facial features as an example of biometric information. In Figure 15, steps S401 to S405 are the same as steps S201 to S205 shown in Figure 9 and steps S301 to S305 shown in Figure 12. For this reason, a description of steps S401 to S405 will be omitted. Also, for example, the processing results of each step by the information processing system 4 in Figure 15 are stored in the storage unit 406 or a storage device connected via a communication network.

If the answer is Yes in step S405, the second reception unit 410 receives an input as to whether or not to register facial features (step S406). If it is received that facial features will not be registered (step S406: No), the information processing system 4 ends the operation of the flow. If it is received that facial features will be registered (step S406: Yes), the update unit 404 updates the customer ID information to member ID information (step S407). After step S407, the information processing system 4 ends the operation of the flow.

Although not shown, embodiment 3 and embodiment 4 may be combined. For example, the information processing system 4 may perform step S406 after the Yes result of step S308 shown in FIG. 12. Alternatively, for example, the information processing system 4 may perform step S406 immediately before step S306 or immediately before step S308 shown in FIG. 12. For example, if it is accepted that registration will be performed (step S406: Yes) and it is accepted that the facial image shows the face of the customer (step S308: Yes), the update unit 404 updates the customer ID information to the member ID information. On the other hand, if it is accepted that registration will not be performed (step S406: No) or if it is accepted that the facial image does not show the face of the customer (step S308: No), the update unit 404 does not update the customer ID information to the member ID information.

<Modification of the Fourth Embodiment>
In the fourth embodiment, as a modified example, a mechanism may be provided in which a customer can receive a benefit when he or she selects to register biometric information. The benefit is not particularly limited to, for example, points given to members, issuance of discount coupons, invitation to events, and the like. For example, in the fourth embodiment, when accepting whether or not to register, a message indicating that a benefit will be given may be displayed on the display unit. Also, for example, in the fourth embodiment, when accepting whether or not to register, the display control unit 408 may display the company's rules on the handling of biometric information on the display unit. In order to realize such a function, for example, the fourth embodiment may have a granting unit that grants a benefit, which is not shown in the figure. Also, although not shown in the figure, in the member DB 4011, for example, point information may be associated with member information.

Next, the effects of the fourth embodiment will be described. For example, when the information processing system 4 acquires member ID information, it confirms with the customer whether or not to register the biometric information. This allows the information processing system 4 to leave it up to the customer to decide whether or not to register the biometric information. Therefore, the information processing system 4 can improve the customer's trust in the handling of biometric information.

This completes the explanation of each embodiment. Next, the hardware configuration of the information processing system will be explained. FIG. 16 is an explanatory diagram showing an example of the hardware configuration of an information processing system. Information processing system 5 is an example in which information processing systems 1, 2, 3, and 4 according to embodiments 1, 2, 3, and 4 are realized by a computer. Information processing system 5 has, for example, an information processing device 50, a terminal device, an imaging device, and a reading device.

First, the hardware configuration of the information processing device 50 will be described. The information processing device 50 is, for example, an example in which the information processing devices 20, 30, and 40 according to the second, third, and fourth embodiments are realized by a computer.

The information processing device 50 has a CPU (Central Processing Unit) 501, a ROM 502, a RAM 503, a storage device 504, and a communication interface 505. Each component is connected to the other via a bus 506.

The CPU 501 controls the entire information processing device 50. For example, the CPU 501 may operate an OS (operating system) to control the entire information processing device 50. The CPU 501 may have multiple cores.

In FIG. 16, the information processing device 50 has, for example, a ROM 502, a RAM 503, and a storage device 504 as storage units. Examples of the storage device 504 include a semiconductor memory such as a flash memory, an HDD, and an SSD. For example, the storage device 504 stores various programs such as an OS program, an application program, and the programs according to the first, second, third, and fourth embodiments. Alternatively, the ROM 502 stores an application program. The ROM 502 may also store the programs according to the first, second, third, and fourth embodiments. The RAM 503 is used as a work area for the CPU 501.

The CPU 501 also loads a program stored in the storage device 504 or the ROM 502. The CPU 501 then executes each process coded in the program. The CPU 501 may also download various programs via the communication network 510. The CPU 501 also functions as a part or all of the information processing device 50. The CPU 501 may then execute the processes or instructions in the illustrated flowchart based on the program.

The communication interface 505 is connected to a communication network 510, such as a LAN (Local Area Network) or a WAN (Wide Area Network), via a wireless or wired communication line. This allows the information processing device 50 to be connected to an external device and an external computer via the communication network 510. The communication interface 505 serves as an interface between the communication network 510 and the inside of the information processing device 50. The communication interface 505 also controls the input and output of data from the external device and the external computer.

However, the hardware configuration of the information processing device 50 shown in FIG. 16 is one example. Components other than those shown in FIG. 16 may be added, or some components may not be included. For example, the information processing device 50 may have a drive device. The CPU 501 may read a program or data from a recording medium attached to the drive device to the RAM 503. Examples of recording media include optical disks, flexible disks, magneto-optical disks, and USB (Universal Serial Bus) memories. In addition, for example, the information processing device 50 may have input devices such as a keyboard and a mouse. The information processing device 50 may have output devices such as a printing device and a display.

Next, the hardware configuration of the terminal device 51 will be described. The terminal device 51 is an example in which the terminal devices 31 and 41 described in the third and fourth embodiments are realized by a computer. The terminal device 51 has, for example, a CPU 511, a ROM 512, a RAM 513, a storage device 514, a communication interface 515, and an input/output device 517. Each component is connected to each other via a bus 516.

The CPU 511 controls the entire terminal device 51. The terminal device 51 has a ROM 512, a RAM 513, and a storage device 514 as a storage unit. Examples of the storage device 514 include a semiconductor memory such as a flash memory, a HDD, and an SSD. For example, the storage device 514 stores an OS program and an application program. Alternatively, the ROM 512 stores an application program. And the RAM 513 is used as a work area for the CPU 511.

The CPU 511 also loads a program stored in the storage device 514 or the ROM 512. The CPU 511 then executes each process coded in the program. The CPU 511 may also download various programs via the communication network 510. The CPU 511 also functions as a part or all of the terminal device 51. The CPU 511 may then execute the processes or instructions in the illustrated flowchart based on the program.

The communication interface 515 is connected to a communication network 510, such as a LAN or WAN, via a wireless or wired communication line. As a result, the terminal device 51 is connected to an external device or an external computer via the communication network 510. The communication interface 515 serves as an interface between the communication network 510 and the inside of the terminal device 51. The communication interface 515 controls the input and output of data from the external device or the external computer. The input/output device 517 also accepts inputs from customer operations or staff operations. The input/output device 517 also outputs data. The input/output device 517 may display images. The input/output device 517 is, for example, a touch panel display.

The hardware configuration of the terminal device 51 shown in FIG. 16 is an example. Components other than those shown in FIG. 16 may be added, or some components may not be included. For example, the terminal device 51 may have a drive device. The CPU 511 may read programs and data from a recording medium attached to the drive device and store them in the RAM 513. Examples of recording media include optical disks, flexible disks, magneto-optical disks, and USB (Universal Serial Bus) memories. For example, the terminal device 51 may also have input devices such as a keyboard and a mouse. The terminal device 51 may also have an output device such as a printer.

Next, an example of the hardware configuration of the imaging device 52 will be described. The imaging device 52 corresponds to the imaging devices 22, 32, and 42 described in the second, third, and fourth embodiments. The imaging device 52 has a camera 521 and a communication interface 522. The components are connected by a bus 523. The communication interface 522 is connected to a communication network 510 such as a LAN or a WAN, for example, through a wireless or wired communication line. The camera 521 has a function capable of capturing images. The camera 521 may also be capable of capturing moving images. As described above, a plurality of imaging devices 52 may be provided. The hardware configuration of the imaging device 52 is an example. Components other than those shown in FIG. 16 may be added.

Next, an example of the hardware configuration of the reading device 53 will be described. The reading device 53 corresponds to the reading devices 23, 33, and 43 described in the second, third, and fourth embodiments. The reading device 53 has, for example, a reader 531 and a communication interface 532. Each component is connected by a bus 533. The communication interface 532 is connected to a communication network 510 such as a LAN or a WAN, for example, through a wireless or wired communication line. The reader 531 reads the member card or the member code. For example, if the member code is a barcode, the reader 531 is a barcode reader. If the member code is a two-dimensional code, the reader 531 may be a camera. For example, if the member card is a contactless IC card, the reader 531 is a reader for the IC card. The hardware configuration of the reading device 53 is an example. Components other than those shown in FIG. 16 may be added.

This completes the description of the hardware configuration of information processing system 5. There are various variations in the methods of realizing information processing systems 1, 2, 3, and 4 described in embodiments 1, 2, 3, and 4. For example, information processing system 5 may be realized by any combination of different computers and programs for each component. Furthermore, multiple components provided in each device may be realized by any combination of a single computer and program.

In addition, some or all of the components of the information processing systems 1, 2, 3, and 4 may be realized by circuits for specific applications. In addition, some or all of the information processing device 50 may be realized by a general-purpose circuit including a processor such as an FPGA (Field Programmable Gate Array). In addition, some or all of the information processing systems 1, 2, 3, and 4 may be realized by a combination of circuits for specific applications, a combination of general-purpose circuits, or a combination of circuits for specific applications and general-purpose circuits. In addition, these circuits may be a single integrated circuit. Or, these circuits may be divided into multiple integrated circuits. And the multiple integrated circuits may be configured by being connected via a bus.

In addition, when some or all of the components of each device are realized by multiple computers and circuits, the multiple computers and circuits may be centralized or distributed.

The information processing method described in each embodiment is realized by an apparatus such as the information processing device 50 executing the information processing method. The information processing method is also realized by a computer such as the information processing device 50 executing a program prepared in advance. The program described in each embodiment is recorded on a computer-readable recording medium such as an HDD, SSD, flexible disk, optical disk, flexible disk, magneto-optical disk, or USB memory. The program is then executed by the computer reading it from the recording medium. The program may also be distributed via the communication network 510.

The components of the information processing systems 1, 2, 3, and 4 in each embodiment described above may be realized in hardware, as in the computer information processing system 5 shown in FIG. 16. Alternatively, each component may be realized by a computer device or firmware based on program control.

Although the present invention has been described above with reference to each embodiment, the present invention is not limited to the above-mentioned embodiment. The configuration and details of each invention may include embodiments to which various modifications that a person skilled in the art can grasp within the scope of the present invention are applied. The present invention may include embodiments in which the matters described in this specification are appropriately combined or replaced as necessary. For example, matters described using a specific embodiment may also be applied to other embodiments to the extent that no contradiction occurs. For example, although multiple operations are described in order in the form of a flowchart, the order of description does not limit the order in which the multiple operations are performed. Therefore, when implementing each embodiment, the order of the multiple operations may be changed to the extent that the content is not affected.

1, 2, 3, 4, 5 Information processing system 20, 30, 40, 50 Information processing device 22, 32, 42, 52 Imaging device 23, 33, 43, 53 Reading device 31, 41, 51 Terminal device 101, 201, 301, 401 First acquisition unit 102, 202, 302, 402 Identification information registration unit 103, 203, 303, 403 Behavior information registration unit 104, 204, 304, 404 Update unit 205, 305, 405 Determination unit 206, 306, 406 Storage unit 307, 407 Second acquisition unit 308, 408 Display control unit 309, 409 First reception unit 410 Second reception unit 2011, 3011, 4011 Member DB
2012, 3012, 4012 Face DB

Claims (9)

A first acquisition means for acquiring biometric information of a customer;
an identification information registration means for registering the acquired biometric information and the customer's identification information in association with each other;
a behavior information registration means for registering behavior information of the customer in association with the identification information;
a second receiving means for receiving an input by the customer as to whether or not to register the biometric information when the member identification information of the customer is acquired;
an update means for updating the identification information associated with the behavioral information and the biometric information of the customer to the acquired member identification information when it is accepted that the biometric information will be registered , and deleting the association between the behavioral information, the biometric information, and the identification information of the customer when it is accepted that the biometric information will not be registered ;
An information processing system comprising: a determination means for determining whether there is identification information associated with the acquired biometric information,
the identification information registration means, when there is no identification information associated with the biometric information, registers the biometric information and the identification information in association with each other;
The information processing system according to claim 1 . The biometric information includes information on facial features obtained from an image of the customer's face.
3. The information processing system according to claim 1 or 2. a second acquisition means for acquiring the image when the member identification information is acquired;
A display control means for displaying the acquired image;
a first receiving means for receiving an input as to whether the displayed image shows the face of the customer;
Equipped with
the updating means updates the identification information associated with the biometric information to the acquired member identification information when it is accepted that the biometric information shows the face of the customer.
The information processing system according to claim 3 . The first reception means receives an input by a person in charge or the customer as to whether or not to show the face of the customer.
5. The information processing system according to claim 4. the biometric information of the customer is obtained from an image of the customer captured by an imaging device installed near an entrance of the store;
the first acquisition means, when acquiring the member identification information of the customer at the time of payment, further acquires biometric information of the customer from an image of the customer captured by an imaging device capable of capturing an image of the customer making the payment;
the updating means updates the identification information associated with the biometric information that matches the biometric information of the customer making the payment to the acquired member identification information;
6. An information processing system according to claim 1. A first acquisition means for acquiring biometric information of a customer;
an identification information registration means for registering the acquired biometric information and the customer's identification information in association with each other;
a behavior information registration means for registering behavior information of the customer in association with the identification information;
a second receiving means for receiving an input by the customer as to whether or not to register the biometric information when the member identification information of the customer is acquired;
an update means for updating the identification information associated with the behavioral information and the biometric information of the customer to the acquired member identification information when it is accepted that the biometric information will be registered , and deleting the association between the behavioral information, the biometric information, and the identification information of the customer when it is accepted that the biometric information will not be registered ;
An information processing device comprising: The computer
Acquire the customer's biometric information,
registering the acquired biometric information and the customer's identification information in association with each other;
registering the behavioral information of the customer in association with the identification information;
When the member identification information of the customer is acquired, an input as to whether or not the customer wishes to register the biometric information is accepted;
when it is accepted that the biometric information will be registered, the identification information associated with the behavioral information and the biometric information of the customer is updated to the acquired member identification information , and when it is accepted that the biometric information will not be registered, the association between the behavioral information, the biometric information, and the identification information of the customer is deleted.
How to perform the action . On the computer,
Acquire the customer's biometric information,
registering the acquired biometric information and the customer's identification information in association with each other;
registering the behavioral information of the customer in association with the identification information;
When the member identification information of the customer is acquired, an input as to whether or not the customer wishes to register the biometric information is accepted;
when it is accepted that the biometric information will be registered, the identification information associated with the behavioral information and the biometric information of the customer is updated to the acquired member identification information , and when it is accepted that the biometric information will not be registered, the association between the behavioral information, the biometric information, and the identification information of the customer is deleted.
A program that executes a process .

Images