JP7523069B2 - Facial recognition system, facial recognition method, information processing terminal and control method thereof - Google Patents

Description

This disclosure relates to a face recognition system, a face recognition method, an information processing terminal, and a control method thereof.

Conventionally, a method using facial recognition has been known as a biometric authentication technology. By using facial recognition, the user does not need to have a specific device or tool in their possession when authenticating, and can easily be authenticated by simply having the authentication device recognize their face.

Patent Document 1 discloses a system in which multiple users are registered as a group at a hotel front desk or the like, rights are associated with biometric authentication information such as the face of each user for that group, authentication is performed using the biometric authentication information, and when authentication is successful, permission to use the hotel facilities is granted based on the rights set for the group.

JP 2018-190274 A

Consider providing various services using face recognition. For example, Patent Document 1 discloses allocating a range of rights to a group and setting rights that each user belonging to the group can use within that range of rights. However, in Patent Document 1, there is no rights manager within the group, so managing rights within the group is cumbersome. In particular, there is no sufficient consideration given to the ability of the manager within the group to properly grasp the usage status of rights by other users or to change the settings according to the situation.

The present disclosure has been devised in consideration of the above-mentioned conventional circumstances, and aims to provide a face authentication system, a face authentication method, an information processing terminal, and a control method thereof that can further improve the convenience when users belonging to a group use various services through face authentication.

The present disclosure provides a face authentication system having a first registration unit that registers rights information of a parent user in association with a group, a second registration unit that registers face images of one or more child users in association with the group, an identification unit that, upon receiving a face image, identifies a group to which a user corresponding to the face image belongs by performing face authentication based on the information registered in the second registration unit, a first determination unit that determines whether a desired process requested by a child user corresponding to the face image is permitted based on the rights information associated with the group identified by the determination unit, a processing unit that executes the desired process based on the determination result by the first determination unit, and a notification unit that notifies the parent user of at least one of the determination result by the first determination unit and the processing result by the processing unit.

The present disclosure also provides a face authentication method including a first registration step of registering rights information of a parent user in a storage unit in association with a group; a second registration step of registering face images of one or more child users in the storage unit in association with the group; a specification step of identifying a group to which a user corresponding to the face image belongs by performing face authentication based on the information registered in the storage unit when a face image is received; a determination step of determining whether or not a desired process requested by a child user corresponding to the face image is permitted based on the rights information associated with the group identified in the determination step; a processing step of executing the desired process based on the determination result of the determination step; and a notification step of notifying the parent user of at least one of the determination result of the determination step and the processing result of the processing step.

The present disclosure also provides an information processing terminal having a request unit that requests a face authentication system to register rights information of a parent user who is a user of the information processing terminal in association with a group, and a receiving unit that receives notification of at least one of the results of the determination and the results of the desired processing when a child user belonging to the group determines whether a desired processing is permitted based on face authentication using the child user's face and the rights information registered in association with the group.

The present disclosure also provides a control method for an information processing terminal, the control method including a request step of requesting a face authentication system to register rights information of a parent user who is a user of the information processing terminal in association with a group, and a receiving step of receiving a notification of at least one of the results of the determination and the results of the desired processing when a child user belonging to the group determines whether a desired processing is permitted based on face authentication using the face of the child user and the rights information registered in association with the group.

In addition, any combination of the above components, and conversions of the expressions of this disclosure between methods, devices, systems, storage media, computer programs, etc., are also valid aspects of this disclosure.

According to the present disclosure, it is possible to further improve the convenience of users belonging to a group when using various services through facial recognition.

FIG. 1 is a diagram showing an example of an overall configuration of a system according to a first embodiment; FIG. 1 is a diagram showing an example of a functional configuration of each device according to the first embodiment; FIG. 1 is a diagram showing an example of a hardware configuration of a mobile terminal according to a first embodiment; FIG. 1 is a diagram showing an example of a hardware configuration of an information processing device according to a first embodiment; FIG. 1 is a diagram showing an example of a database configuration according to the first embodiment; Processing sequence of an example of a registration processing procedure between each device according to the first embodiment FIG. 13 is a diagram showing an example of a UI screen during registration processing according to the first embodiment; FIG. 13 is a diagram showing an example of a UI screen during registration processing according to the first embodiment; Processing sequence of an example of a matching process procedure between devices according to the first embodiment FIG. 13 is a diagram showing an example of a UI screen during a matching process according to the first embodiment; Processing sequence of another example 1 of the collation processing procedure between each device FIG. 13 is a diagram showing an example of a UI screen in another example 1 of the matching process; Processing sequence of another example 2 of the collation processing procedure between each device FIG. 13 is a diagram showing an example of a UI screen in another example 2 of the matching process; Processing sequence of another example 3 of the collation processing procedure between each device FIG. 13 is a diagram showing an example of a UI screen in another example 3 of the matching process; Processing sequence of another example 4 of the matching processing procedure between each device FIG. 13 is a diagram showing an example of a UI screen in another example 4 of the matching process; Processing sequence of another example 5 of the collation processing procedure between each device FIG. 13 is a diagram showing an example of a UI screen in another example 5 of the matching process;

Below, with reference to the attached drawings as appropriate, embodiments that specifically disclose a face recognition system, a face recognition method, an information processing terminal, and a control method thereof according to the present disclosure will be described in detail. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of already well-known matters or duplicate explanations of substantially identical configurations may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the attached drawings and the following explanation are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

<First embodiment>
[System configuration]
FIG. 1 shows an example of the overall configuration of a system according to the first embodiment. In this embodiment, the system includes a parent mobile terminal 10, a child mobile terminal 20, a face authentication system 30, a face matching terminal 40, and a service providing system 50, and the various devices are connected to each other via a network 60 so as to be able to communicate with each other. In FIG. 1, each of the various devices is shown as one device, but a plurality of devices may be included. In addition, each of the various devices is shown as one device, but the plurality of devices in FIG. 1 may be realized by a single device, such as by having the functions of the face authentication system 30 and the face matching terminal 40 in the same device. In this specification, "parent" and "child" indicate the relative relationship between users in setting the rights described below. Therefore, one user can be both a parent user and a child user, and one user can be a child user of multiple parent users. In the following description, when it is necessary to clearly indicate the role of each user in the parent-child relationship, they will be described as "user (parent)" and "user (child)". Details will be described later.

The parent mobile terminal 10 and the child mobile terminal 20 are mobile terminals owned by each user, and for convenience, the mobile terminal owned by a user with the role of "parent" will be described as the parent mobile terminal 10, and the mobile terminal owned by a user with the role of "child" will be described as the child mobile terminal 20.

The facial authentication system 30 performs the facial authentication process according to this embodiment, and stores and manages various facial information. The service providing system 50 is a system for providing various services in cooperation with the facial authentication system 30. Examples of services that can be provided by the service providing system 50 will be described later. The facial authentication system 30 and the service providing system 50 may be configured as on-premise systems provided within the facilities of the administrators of the respective services, or may be configured as off-premise systems using cloud computing or the like.

The face matching terminal 40 is installed in facilities that provide various services, and is used to acquire a face image of a user and verify the user's rights. The network 60 is configured to connect various devices so that they can communicate with each other, and there are no particular limitations on whether the network is wired or wireless or on the communication standard.

[Functional configuration]
FIG. 2 is a diagram showing an example of the functional configuration of various devices according to the present embodiment. Each part of the various devices may be realized by a control unit (for example, a processor) of each device reading out a program stored in a storage unit, or by providing dedicated hardware. Here, the processor is configured using, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array). Note that only parts corresponding to the functions according to the present embodiment are shown here, and each device may further include parts for realizing other functions. In addition, the configuration shown in FIG. 2 is an example, and the parts shown in FIG. 2 may be configured together, or may be further divided into more details. In addition, the links and arrows of each part shown in FIG. 2 show an example of data transmission and reception, but are not limited to this connection configuration, and other cooperation and connection may be performed.

The parent mobile terminal 10 includes a usage status display unit 11, a parent right information registration unit 12, a child right information registration unit 13, a parent face information acquisition unit 14, a child face information acquisition unit 15, an image capture unit 16, and a child registration link transmission unit 17. The usage status display unit 11 displays the usage status of the rights held by the user (parent) of the parent mobile terminal 10 based on the notification contents from the face authentication system 30. The parent right information registration unit 12 performs a process of registering the rights held by the user (parent) (hereinafter referred to as "parent right information") in the face authentication system 30. The child right information registration unit 13 performs a process of registering the rights granted to another user (child) (hereinafter referred to as "child right information") within the scope of the parent right information held by the user (parent) of the parent mobile terminal 10 (in other words, on the premise of satisfying the parent right information) in the face authentication system 30. That is, the child right information is conceptually or substantially included in the parent right information.

The parent face information acquisition unit 14 acquires the face information (face image) of the user (parent) via the imaging unit 16 and performs a process of registering the face information in the face authentication system 30. The child face information acquisition unit 15 acquires the face information (face image) of the user (child) via the imaging unit 16 and performs a process of registering the face information in the face authentication system 30. The imaging unit 16 includes a camera for acquiring face images, and passes the face image obtained by photographing the face based on the user's operation to the parent face information acquisition unit 14 and the child face information acquisition unit 15. When the child mobile terminal 20 performs a process of photographing a face image when registering the face information (face image) of the user (child), the child registration link transmission unit 17 performs a process of transmitting the request to the child mobile terminal 20. The request here includes access information (hereinafter referred to as the "child registration link") to the face authentication system 30 used when registering the face image.

The child mobile terminal 20 includes a child registration link receiving unit 21, an imaging unit 22, and a child face information acquiring unit 23. The child registration link receiving unit 21 receives a request sent from the parent mobile terminal 10 and notifies the user of the request. The imaging unit 22 includes a camera for acquiring a face image, and passes the face image obtained by photographing the face based on the user's operation to the child face information acquiring unit 23. The child face information acquiring unit 23 acquires face information (face image) of the user (child) via the imaging unit 22. The child face information acquiring unit 23 then performs a process of registering the face information of the user (child) in the face authentication system 30 based on the child registration link included in the request sent from the parent mobile terminal 10.

The face authentication system 30 includes a parent-child relationship determination unit 31, a parent face information management unit 32, a child face information management unit 33, a usage right determination unit 34, a parent right management unit 35, and a child right management unit 36. The face authentication system 30 further includes an external system linkage unit 37, a usage history management unit 38, a usage result notification unit 39, and a result notification unit 45. The parent-child relationship determination unit 31 performs face authentication based on the registered face image to identify the user of the face image included in the matching request from the face matching terminal 40. Furthermore, the parent-child relationship determination unit 31 determines the parent-child relationship set for the identified user. A publicly known method can be used as a face authentication method, and is not particularly limited. For example, a method such as template matching, feature point extraction, or three-dimensional face authentication may be used. The parent face information management unit 32 registers and manages the face image transmitted from the parent mobile terminal 10 as the face image of the user (parent). The child face information management unit 33 registers and manages the face images sent from the parent mobile terminal 10 and the child mobile terminal 20 as face images of the user (child). The parent face information management unit 32 and the child face information management unit 33 set the parent-child relationship based on instructions from the user (parent).

The usage right determination unit 34 determines whether the usage right specified in the matching request from the face matching terminal 40 is permitted to the user identified based on the face information. When making its determination, the usage right determination unit 34 refers to information managed by the parent right management unit 35 and the child right management unit 36. The parent right management unit 35 holds and manages the parent right information specified by the parent mobile terminal 10. The parent right information indicates the content of the rights to use various services provided by the service providing system 50 in addition to the user authentication service. Certain rights are granted by applying for or purchasing the use of various services. The child right management unit 36 holds and manages the child right information specified by the parent mobile terminal 10.

The external system linking unit 37 links with an external system (e.g., one of the various systems included in the service providing system 50) to perform processing according to the rights of each user, based on a matching request from the face matching terminal 40, etc. The content of the linking may vary depending on the requested content, and may be, for example, processing such as referencing or updating a database (hereinafter referred to as DB) held by the external system. Alternatively, the external system may be configured to execute part of the processing according to the requested content. Furthermore, when processing that does not require linking with an external system is requested, the external system linking unit 37 may perform the processing using various information held inside the face authentication system 30.

The usage history management unit 38 holds and manages the processing executed in response to a request from the face matching terminal 40, that is, the usage history of the rights. The usage history may be managed including information on the processing performed in the service providing system 50 as well as the processing within the face authentication system 30. The usage result notification unit 39 notifies the parent mobile terminal 10 of the usage history managed by the usage history management unit 38 at a predetermined timing. The parent mobile terminal 10 that is the notification destination here is a mobile terminal owned by the user (parent) if the user corresponding to the face image shown in the request from the face matching terminal 40 is a parent. On the other hand, if the user corresponding to the face image is a child, the notification destination is a mobile terminal owned by a user (parent) who has a parent-child relationship with the user (child). The result notification unit 45 notifies the requesting face matching terminal 40 of the result of face authentication and the processing content executed as a result in response to the processing requested from the face matching terminal 40.

The face matching terminal 40 includes an imaging unit 41, a face information acquisition unit 42, a usage right setting unit 43, and a matching result display unit 44. The imaging unit 41 includes a camera for acquiring a face image, and passes the face image obtained by photographing the face based on the user's operation to the face information acquisition unit 42. The face information acquisition unit 42 acquires face information (face image) of the user who is the subject of face recognition via the imaging unit 41, and passes it to the usage right setting unit 43. The usage right setting unit 43 sets the rights that the user who is the subject of face recognition wishes to use, and transmits them to the face recognition system 30 together with the face image as a matching request. The rights to be used here differ depending on the content of the service to be provided.

The service providing system 50 may be configured with various systems depending on the content of the service to be provided. As an example, it may include a payment system 51 that provides payment services, an accommodation management system 52 that manages accommodation at accommodation facilities, a facility management system 53 that manages the use of facilities such as amusement facilities, or a lost child determination system 54 that manages lost users. Each of the various systems listed here may be configured with separate servers or devices. In this embodiment, several examples of scenes to which the present disclosure can be applied will be specifically described.

[Device configuration]
(Mobile device)
3 is a diagram showing an example of a hardware configuration of an information processing terminal applicable as the parent mobile terminal 10 and the child mobile terminal 20 according to the present embodiment. In the present embodiment, the parent mobile terminal 10 and the child mobile terminal 20 are described as having the same hardware configuration, but may have different configurations. The mobile terminal 300, which is an information processing terminal, may be, for example, a smartphone, a tablet terminal, a mobile PC (Personal Computer), a wearable terminal, or the like. Here, a smartphone will be described as an example.

The mobile terminal 300 includes a CPU 301, a communication unit 302, a touch panel display 303, a microphone 304, a speaker 305, a memory 306, a storage 307, and a camera 309. The CPU 301 provides various functions by reading and executing various programs (e.g., an application 308) stored in the storage 307.

The communication unit 302 communicates with the outside world via a mobile phone communication network, the Internet, short-range wireless communication, etc., and transmits and receives various data and signals. The communication method used by the communication unit 302 may be, for example, a 4G/5G communication method using a wide area communication line such as a telephone line, or a short-range wireless communication method such as the Wi-Fi (registered trademark) standard for wireless LAN (Local Area Network) or the Bluetooth (registered trademark) standard.

The touch panel display 303 functions as a display means for displaying various information to the user, and also accepts touch operations from the user. The microphone 304 accepts the user's voice and surrounding audio information as input. The speaker 305 outputs audio information. The memory 306 is a volatile storage area, and is used, for example, as a work area when the CPU 301 performs various processes. The storage 307 is a non-volatile storage area, and stores, for example, applications 308 for various processes according to this embodiment. The applications 308 provide various functions by being read and executed by the CPU 301. The camera 309 photographs an object based on a user operation and acquires it as an image. In this embodiment, the user's face is photographed and acquired as a facial image.

(Information processing device)
4 is a diagram showing an example of the hardware configuration of an information processing device applicable as the face authentication system 30 according to the present embodiment and the face matching terminal 40. Here, a PC is taken as an example of the information processing device, but for example, the face matching terminal 40 may be configured as a dedicated terminal for face matching.

The information processing device 400 includes a CPU 401, a communication unit 402, a display unit 403, an external I/F (interface) 404, a memory 405, and a storage device 406. The face matching terminal 40 is further equipped with a camera (not shown). The CPU 401 realizes various functions by reading out various programs and data stored in the memory 405 and the storage device 406 and executing processing. The communication unit 402 communicates with the outside via the Internet, short-range wireless communication, etc., and transmits and receives various data and signals. The communication method used by the communication unit 302 may be, for example, the Wi-Fi (registered trademark) standard of wireless LAN, which is a short-range wireless communication, or the Bluetooth (registered trademark) standard.

The display unit 403 displays various information based on instructions from the CPU 401. The external I/F 404 is an interface for transmitting and receiving data to and from an external device, and accepts data input via an operating device (not shown) such as a mouse or keyboard. The memory 405 is a storage area for storing and holding various information, and is composed of, for example, a ROM (Read Only Memory) which is a non-volatile storage area, and a RAM (Random Access Memory) which is a volatile storage area. The storage device 406 is a storage area for storing and holding various information, and is composed of, for example, a HDD (Hard Disk Drive). The various DBs according to this embodiment may be stored in the storage device 406.

[Database configuration]
Fig. 5 is a diagram showing an example of the DB configuration according to this embodiment. The DB configuration shown here is an example, and is not limited to this configuration. Therefore, any of the DBs shown in Fig. 5 may be configured together as one DB, or one DB may be divided into multiple DBs. In addition, other items may be further added.

Figure 5 (a) shows an example of the configuration of a face information DB according to this embodiment. The face information DB is managed, for example, by the parent face information management unit 32 and the child face information management unit 33 of the face authentication system 30. The face information DB includes columns of Person ID, face image, name, telephone number, and SNS. The Person ID column indicates a Person ID, which is identification information for uniquely identifying a user. The Person ID is assigned, for example, by the face authentication system 30 when the user is registered. Note that the specifications of the IDs (number of digits, characters used, etc.) included in each DB shown below are examples and are not limited to those shown in Figure 5. The face image column indicates a face image corresponding to the face of the registered user. The face image may be stored as registered image data as is, or may be converted into a specific data format according to the face authentication algorithm, etc. before being stored. The face image column may indicate the storage destination of the face image.

The name column indicates the name information of the user. The phone number column indicates the user's phone number. The SNS column indicates the information used on the user's SNS (Social Network Service). For example, email addresses and account information are set in the SNS column. Note that there are no particular limitations on the services that can be used as SNS, and examples include communication tools such as email. Note that this name information is used as the display name of each user (child) on the parent mobile terminal 10.

5B shows an example of the configuration of the right information DB according to the present embodiment. The right information DB is managed, for example, by the parent right management unit 35 of the face authentication system 30. The right information DB includes columns of right ID, Person ID, and right information details. The right ID column indicates a right ID, which is identification information for uniquely identifying a right. The right ID is assigned, for example, when a user (parent) acquires a right. The Person ID column indicates the Person ID of the user who has the right (i.e., the user (parent)), and corresponds to the Person ID shown in FIG. 5A. The right information details column indicates detailed information about the right. The content of the detailed information here changes depending on the content of various services provided by the service providing system 50. For example, when the service is an accommodation management service, information such as the accommodation facility, the accommodation date and time, the number of guests, and the accommodation plan is managed as detailed information about the right.

Figure 5 (c) shows an example of the configuration of a group DB according to this embodiment. The group DB is managed, for example, by the parent face information management unit 32 of the face authentication system 30. The group DB includes columns of group ID, rights ID, parent member ID, and child member ID. The group ID column indicates the group ID, which is information for uniquely identifying a group. The group ID is assigned, for example, when a user (parent) sets up a group. The rights ID column indicates the rights ID of the rights used in the group, and corresponds to the rights ID shown in Figure 5 (b). The parent member ID column indicates the Person ID of the user (parent) who is the administrator of the group, and corresponds to the Person ID shown in Figure 5 (b). The child member ID column indicates the Person ID of a user (child) who belongs to the group, and corresponds to the Person ID in Figure 5 (a).

Figure 5(d) shows another example of the configuration of the rights information DB according to this embodiment. Details of Figure 5(d) will be described later together with application examples of this disclosure.

[Operation sequence]
Next, the processing sequence between the devices of the system according to this embodiment will be described. The processing according to this embodiment is divided into two steps: a right registration process and a right usage (check) process. In the registration process, a user (parent) registers the rights he or she owns and one or more users (children) who can use those rights. In the usage process, the availability of the registered rights is determined based on the user's face image, and the rights are used according to the result. The flow of each process will be described below with reference to the drawings.

(registration process)
6 is a diagram showing a processing sequence of the registration process according to the present embodiment. In each device, which is a processing subject in the processing sequence, a control unit such as a CPU reads out and executes a program corresponding to each function shown in FIG. 2, thereby realizing each function.

The parent mobile terminal 10 receives a registration instruction from the user (step S601). The registration instruction here may be given, for example, via a screen (not shown) that is displayed when accessing a system corresponding to the service to be used. The registration instruction may also be configured to allow the user to select whether or not to register a right when the right is acquired in the subsequent step S602.

The parent mobile terminal 10 acquires the desired rights via a screen displayed by accessing a system corresponding to the service to be used (step S602). For example, assume that a reservation for use of a facility is made using the facility management system 53. In this case, a reservation for use of the facility is made via a reservation screen 700 as shown in FIG. 7(a). The reservation screen 700 includes setting items for reservation contents 701 and reservation person information 704 (customer information). The reservation contents 701 include a facility name 702 indicating the facility to be used and a date and time 703 indicating the date and time of use. The reservation person information 704 includes a name 705 indicating the name of the person who made the reservation and a contact 706 indicating the contact information of the person who made the reservation. The user makes a reservation by inputting each of the displayed information and then pressing the reservation button 707. That is, in the case of the example of FIG. 7(a), the user acquires the right to use the facility by making a reservation. Note that the reservation screen 700 shown in FIG. 7(a) is an example, and the displayed and input items may differ depending on the content of the service to be used. After acquiring the rights, the parent mobile terminal 10 notifies the face authentication system 30 of information related to the rights.

Based on the data received from the parent mobile terminal 10, the face authentication system 30 stores the rights acquired by the user as parent right information (step S603). Specifically, the parent right information is registered in the right information DB shown in FIG. 5(b).

The parent mobile terminal 10 photographs the face of the parent user and acquires it as a facial image (step S604). For example, the parent mobile terminal 10 displays a photographing screen 710 for registering the facial image of the user (parent) as shown in FIG. 7(b) and encourages the user to take a photograph. The photographing screen 710 may display an icon 711 or a message to acquire an appropriate facial image. After acquiring the facial image, the parent mobile terminal 10 registers it as the facial image of the user (parent) in the facial authentication system 30. Note that the facial image may be configured to allow the user to select from images that have already been photographed.

The facial authentication system 30 registers the facial image of the user (parent) acquired from the parent mobile terminal 10 (step S605). Specifically, the facial image of the user (parent) is registered in the facial information DB shown in FIG. 5(a). At this time, the facial authentication system 30 may perform a process of converting the facial image into arbitrary data according to the facial authentication algorithm, etc.

The parent mobile terminal 10 sets up a group including one or more users (children) who can use the acquired rights (step S606). For example, the parent mobile terminal 10 displays a registration screen as shown in FIG. 7(c) and (d) to allow the user (parent) to set up a group. FIG. 7(c) shows an example of the configuration of a group registration screen 720. In the registration screen 720, a list 721 of users (children) included in the group is displayed. In the example of FIG. 7(c), as shown in the icon 722, two users are specified as children, "child 1" and "child 2". A user (child) can be added by pressing the add button 723. When the add button 723 is pressed, a registration screen 730 shown in FIG. 7(d) is displayed. The registration screen 730 is a screen for setting information about the user (child), and a setting item 731 showing basic information about the user (child) is displayed. In the example of FIG. 7(d), the basic information includes the user's (child's) name 732, face information 733, phone number 735, and SNS 736. The registration screen 730 further includes a registration button 734 and a send button 737. When the registration button 734 is pressed, the parent mobile terminal 10 takes a face image of the child user. On the other hand, when the send button 737 is pressed, the child mobile terminal 20 carried by the child user takes a face image of the user. The parent mobile terminal 10 transmits various setting information of the group to the face authentication system 30, and registers the group.

The face authentication system 30 registers the group using the various setting information of the group received from the parent mobile terminal 10 (step S607). Specifically, the group information is registered in the group DB shown in FIG. 5(c).

The parent mobile terminal 10 determines whether the facial image of the child user is to be acquired from the parent mobile terminal 10 or the child mobile terminal 20 (step S608). As described above, when the registration button 734 is pressed on the registration screen 730 in FIG. 7(d), the facial image of the child user is assumed to be captured by the parent mobile terminal 10 (step S608, YES), and the processing of the parent mobile terminal 10 proceeds to step S609. On the other hand, when the send button 737 is pressed on the registration screen 730 in FIG. 7(d), the facial image of the child user is assumed to be captured by the child mobile terminal 20 (step S608, NO), and the processing of the parent mobile terminal 10 proceeds to step S610.

The parent mobile terminal 10 photographs the face of the user who will become the child and acquires it as a facial image (step S609). For example, the parent mobile terminal 10 displays a photographing screen 800 for registering the facial image of the user (child) as shown in FIG. 8(a) and encourages the user to take a photograph. The photographing screen 800 may display an icon 801 or a message to acquire an appropriate facial image. After acquiring the facial image, the parent mobile terminal 10 registers it as the facial image of the user (child) in the facial authentication system 30. Note that the facial image may be configured to allow the user to select from images that have already been photographed.

The parent mobile terminal 10 transmits a request to capture the face information (face image) of the child user to the child mobile terminal 20 (step S610). The request includes access information (child registration link) to the face authentication system 30 used when registering the face image. The request may be transmitted by email, for example, and the destination may be the information set in the SNS 736 of the registration screen 730 in FIG. 7(d). When the request is transmitted by email, for example, email 810 with the contents as shown in FIG. 8(b) may be transmitted. In the email 810, a subject 811 indicating an outline of the request is set. The text 812 includes the details of the request and a registration button 813 for executing face registration. When the registration button 813 is pressed, the face image registration process is performed.

Based on the registration request (e.g., email 810) received from the parent mobile terminal 10, the child mobile terminal 20 photographs the face of the user who will become the child and acquires it as a facial image (step S611). For example, when the registration button 813 shown in the email 810 is pressed, the child mobile terminal 20 displays a shooting screen 820 for registering the facial image of the user (child) as shown in FIG. 8 (c) and prompts the user to take a photograph. The shooting screen 820 may display an icon 821 or a message to acquire an appropriate facial image. After acquiring the facial image, the child mobile terminal 20 registers it as the facial image of the user (child) in the facial authentication system 30. This process may be performed using a specific SNS app in response to the registration request received from the parent mobile terminal 10, or may be performed via a web browser (not shown).

The facial authentication system 30 registers the facial image of the user (child) received from the parent mobile terminal 10 or the child mobile terminal 20 (step S612). Specifically, the facial image of the user (child) is registered in the facial information DB shown in FIG. 5(a). At this time, the facial authentication system 30 may perform a process of converting the facial image into arbitrary data according to the facial authentication algorithm, etc.

The face authentication system 30 sets up a parent-child relationship based on the registered user (parent) and user (child) information (step S613). Specifically, the contents of the group DB shown in FIG. 5(c) are updated based on the specified information to associate each user. In this embodiment, the registrant of the group is set as the user (parent), and other users belonging to the group are set as users (children).

The parent mobile terminal 10 accepts the right setting for the user (child) (step S614). For example, the right setting for the user (child) is performed by displaying a setting screen 830 as shown in FIG. 8(d). The setting screen 830 shows basic information 831 of the user (child) that has already been set, and right information 832 that is permitted for the user (child). Here, the items of the right information 832 are assumed to be payment services, and include whether or not payment can be used 833, the upper limit of payment amount 834, and whether or not coupons can be used 835. When the setting button 836 is pressed, the contents of the entered right information 832 are passed to the face authentication system 30, and registration is instructed. The setting screen 830 shown in FIG. 8(d) is an example, and the displayed and entered items may differ depending on the contents of the service to be used.

The face authentication system 30 determines whether the user (child) rights information received from the parent mobile terminal 10 is within the scope of the user's (parent's) rights (step S615). At this time, if the information exceeds the user's (parent's) rights, the parent mobile terminal 10 may be notified of this and prompted to re-input. Alternatively, the configuration may be controlled so that settings that exceed the user's (parent's) rights cannot be specified on the setting screen 830 shown in FIG. 8(d).

Based on the result of the verification in step S615, the face authentication system 30 stores the rights information for the user (child) (step S616). Then, this processing sequence in FIG. 6 ends.

In the above example, the child rights information is set (step S614) after each face image is registered, but the process is not limited to this. For example, the rights information of the user (child) may be set in conjunction with the group setting process (step S606). If the face image of the child user is not registered, the rights cannot be used even if rights information has been set for the child.

Addition and deletion of users (children) to a group may be performed at any time and is not limited to the timing when the group is set up. Furthermore, the parent-child relationship is not limited to that set when the group is registered. For example, the configuration may be such that the user (parent) can be changed at any time.

Addition and deletion of rights for the user (parent) may be performed at any time. The configuration may be such that the user (parent) can update an already created group by adding or deleting new rights. In this case, instead of the processing of steps S604 to S607, the configuration may be such that a group to which newly acquired rights are to be associated can be selected from groups already created in association with the user (parent).

When setting up a group, the system may be configured so that it is possible to create a group by copying one that has already been created by the user (parent). This can reduce the effort required for group setting.

(Matching process: basic pattern)
Next, the matching process according to the present embodiment will be described. FIG. 9 is a diagram showing a processing sequence of the matching process according to the present embodiment. In each device, which is a processing subject in the processing sequence, a control unit such as a CPU reads out and executes a program corresponding to each function shown in FIG. 2, thereby realizing each function. Note that, before the matching process described below, various information is registered by the registration process described using FIG. 6. In addition, the basic flow of the process does not change whether the user performing face matching is a parent or a child, but here, in order to more clearly explain the features of the present disclosure, the target user (hereinafter referred to as the "target user") performing face authentication will be described as a user (child).

First, the face matching terminal 40, face authentication system 30, and service providing system 50 work together to perform common face matching processing (step S900). The common face matching processing is made up of steps S901 to S907. This processing is also performed in common in other cases described below. Note that although there may be changes to the screens displayed and the contents of data exchanged between various devices depending on the case being applied, the general flow is the same.

The face matching terminal 40 receives a face matching instruction (step S901). The instruction here may be given, for example, by an operator of the facility in which the face matching terminal 40 is installed performing an instruction operation when performing face matching.

The face matching terminal 40 photographs the face of the target user using a camera (not shown) and acquires it as a face image (step S902). For example, the face matching terminal 40 displays a photographing screen 1000 for face matching as shown in FIG. 9(a) to prompt the target user to take a photograph. The photographing screen 1000 may display an icon 1001, a message, or the like, in order to acquire an appropriate face image.

The face matching terminal 40 sets the rights to be used by the target user (step S903). The settings here may be predefined on the face matching terminal 40 side depending on the content of the service to be used, or may be made by an operator of the facility where the face matching terminal 40 is installed selecting from several options. For example, if the service to be used is admission to a facility, "admission" is set as the right to be used.

The face matching terminal 40 transmits an authentication request (matching request) including the face image acquired in step S902 and the rights set in step S903 to the face authentication system 30 (step S904).

Based on the authentication request received from the face matching terminal 40, the face authentication system 30 checks the registration status of the face information (step S905). Here, the face authentication system 30 performs face authentication processing by referring to the face information DB shown in FIG. 5(a). As described above, any known face authentication method can be used, and there is no particular limitation. For example, methods such as template matching, feature point extraction, and three-dimensional face authentication may be used. This makes it possible to identify the target user indicated by the face image.

The facial authentication system 30 identifies the parent-child relationship of the target user (here, user (child)) and identifies the user (parent) and its rights information (step S906). For example, the facial authentication system 30 performs processing by referring to the rights information DB and group DB shown in Figures 5 (b) and (c). In addition, as necessary, the facial authentication system 30 makes a request to any system included in the service providing system 50, which is an external system, to confirm the rights held by the user. Whether or not this confirmation request is sent or received depends on the configuration of the various DBs and the content of the service being used.

When the service providing system 50 receives a request to confirm rights from the face authentication system 30, it performs a process to confirm the rights of each user in response to the request and returns the result to the face authentication system 30 (step S907). As described above, the confirmation process on the service providing system 50 side may be omitted depending on the DB configuration, system configuration, etc. This is the common face matching process included in step S900.

The face authentication system 30 notifies the face matching terminal 40 of the result of the common face matching process (step S908). Here, as an example, it is assumed that the user's face is set to be authenticated in step S903. In this case, the face authentication system 30 notifies the face matching terminal 40 of the authentication result based on the user's face image.

The face matching terminal 40 displays the processing result received from the face authentication system 30 (step S909). For example, the face matching terminal 40 displays an authentication result screen 1010 as shown in FIG. 10(b). The authentication result screen 1010 includes an icon 1011 indicating the authentication result, a face image 1012 used for face authentication, user information 1013 of the target user, and a message 1014 regarding the authentication result.

The face authentication system 30 saves the processing result of the common face matching process in step S900 as history information (step S910). For example, when face authentication is performed, information to the effect that face authentication has been performed and the authentication result are saved. In other words, the contents of the rights that the user has used or attempted to use are saved.

The face authentication system 30 identifies the parent mobile terminal 10 owned by the user (parent) based on the parent-child relationship identified in step S906, and notifies the parent mobile terminal 10 of the history information saved in step S910 (step S911). The notification may be sent by email, for example.

The parent mobile terminal 10 displays the notification received from the face authentication system 30. If the notification is made by email, an email 1020 with the content as shown in FIG. 10(c) may be sent, for example. In the email 1020, a subject 1021 indicating an overview of the history information is set. The body 1022 also includes details of the history information (such as the content of the rights used). Then, this processing sequence in FIG. 9 ends.

In the above processing sequence, an example is shown in which the content of use of rights is notified for each use. However, this is not limited to this, and for example, a configuration may be used in which the usage history for a specified period is notified in chronological order or as a cumulative result. Furthermore, the timing of notification to the user (parent) is not limited to the timing of the use of rights, and may be configured so that the user (parent) can set it to a desired timing. Such notification settings may be configured, for example, to be made in the group setting process (step S606) in FIG. 6, and notifications may be made according to the settings.

(Matching process: facility use case)
The basic processing sequence of the collation processing according to the present embodiment has been described with reference to Fig. 9 and Fig. 10. Next, a case where this basic processing sequence is applied to a facility usage case will be described.

In this case, it is assumed that the right to use the facility multiple times (or by multiple people) (e.g., admission rights) is purchased and registered in advance by the user (parent) based on the registration process shown in FIG. 6. In this case, if the user belongs to a group, the right associated with the group may be used without identifying the right, or the right assigned to each user (child) may be specified and used. In the former case, it can be implemented, for example, with a right information DB configuration as shown in FIG. 5(b). In the latter case, it can be implemented, for example, with a right information DB configuration as shown in FIG. 5(d). In the right information DB configuration shown in FIG. 5(d), when a user (parent) registers multiple usage rights (here, tickets for facility use) in association with a group, each of the multiple usage rights is assigned to a user (child) who can use it. In this configuration, the user (child) uses the assigned usage right among the multiple usage rights associated with the group.

Figure 11 is a diagram showing a processing sequence when the matching process according to this embodiment is applied to facility use. Each device, which is the processing subject in the processing sequence, realizes each function by, for example, a control unit such as a CPU reading and executing a program corresponding to each function shown in Figure 2.

The face matching terminal 40, face authentication system 30, and service providing system 50 work together to perform a common face matching process (step S900). This process is the common face matching process described with reference to FIG. 9. In this process, for example, the face matching terminal 40 displays a photographing screen 1200 for face matching as shown in FIG. 12(a) to prompt the target user to take a photograph. The photographing screen 1200 may display an icon 1201 or a message to obtain an appropriate face image.

The face authentication system 30 checks the rights information available to the target user (here, user (child)) and updates the rights information to be used according to the matching result of the common face matching process (step S1101). For example, the face authentication system 30 performs the checking process by referring to the rights information DB and group DB shown in Figures 5 (b) and (c). In addition, as necessary, the face authentication system 30 makes an update request associated with the use of the rights held by the user (parent) to any system included in the service providing system 50, which is an external system. Whether or not this update request is sent or received depends on the configuration of the various DBs and the content of the service to be used. For example, in the case of the right to enter a facility, the DB is updated to indicate that the right has been used (lost) upon entry.

When the service providing system 50 receives an update request for use of rights from the face authentication system 30, it performs an update process for the rights of each user in response to the request and returns the result to the face authentication system 30 (step S1102). As described above, the update process on the service providing system 50 side may be omitted depending on the DB configuration, system configuration, etc.

Based on the processing contents up to this point, the face authentication system 30 notifies the face matching terminal 40 of the availability information indicating whether the target user can use the right (step S1103). Specifically, the face authentication of the user (child) is successful, and a notification is given as to whether or not the facility can be used.

The face matching terminal 40 displays the availability information received from the face authentication system 30 (step S1104). For example, the face matching terminal 40 displays an authentication result screen 1210 as shown in FIG. 12(b). The authentication result screen 1210 includes an icon 1211 indicating availability, a face image 1212 used for face authentication, user information 1213 of the target user, and a message 1214 regarding the authentication result.

The face authentication system 30 stores the usage result received from the service providing system 50 as history information (step S1105). Specifically, when using a facility, information such as the success or failure of face authentication, whether the facility can be used, and the date and time of use is stored. In other words, the contents of the rights that the target user has used or attempted to use are stored.

The face authentication system 30 identifies the parent mobile terminal 10 owned by the user (parent) based on the parent-child relationship identified in the common face matching process in step S900, and notifies the parent mobile terminal 10 of the history information saved in step S1105 (step S1106). The notification here may be sent by email, for example.

The parent mobile terminal 10 displays the notification received from the face authentication system 30. If the notification is made by email, an email 1220 with the content as shown in FIG. 12(c) may be sent, for example. In the email 1220, a subject 1221 indicating an overview of the history information is set. The body 1222 also includes details of the usage information (date and time of usage, user, facility name, etc.). Then, this processing sequence in FIG. 11 ends.

(Matching process: Case of payment use)
Next, a case where the basic processing sequence explained in FIG. 9 and FIG. 10 is applied to the use of a payment service will be explained.

In this case, it is assumed that the user (parent) has registered account information that can be used for payments in advance as a right to use the payment service based on the registration process shown in FIG. 6 in cooperation with the payment system 51, which is one of the service providing systems 50. More specifically, the availability of payment 833 and the upper limit of payment amount 834 are set on the setting screen 830 shown in FIG. 8(d).

Figure 13 is a diagram showing a processing sequence when the matching process according to this embodiment is applied to the use of a payment service. Each device, which is the processing subject in the processing sequence, realizes each function by, for example, a control unit such as a CPU reading and executing a program corresponding to each function shown in Figure 2.

The face matching terminal 40, the face authentication system 30, and the service providing system 50 cooperate to perform the face matching common process (step S900). This process is the face matching common process described with reference to FIG. 9. In this process, for example, the face matching terminal 40 displays a payment screen 1400 as shown in FIG. 14(b) and accepts the setting of the right to use (step S903). The payment screen 1400 includes payment information 1401 indicating the amount of payment, a cancel button 1402 for canceling the execution of the payment, and a decision button 1403 for instructing the execution of the payment. Furthermore, in the face matching common process, the face matching terminal 40 displays a photographing screen 1410 for face matching as shown in FIG. 14(b) to prompt the target user to take a photograph. The photographing screen 1410 may display an icon 1411 or a message in order to obtain an appropriate face image.

The facial authentication system 30 checks the rights information available to the target user (here, user (child)) and confirms the rights information to be used according to the matching result of the common facial matching process (step S1301). For example, the facial authentication system 30 performs the confirmation process by referring to the rights information DB and group DB shown in Figures 5 (b) and (c). Furthermore, the facial authentication system 30 identifies the service providing system 50 (here, the payment system 51) that provides the service to be used based on the rights.

The face authentication system 30 requests the service provider system 50 (here, the payment system 51) that provides the service used by the user to use the service based on the identified rights (step S1302). This request includes the usage amount, user information, etc.

The service providing system 50 provides the service based on the request from the face authentication system 30 (step S1303). Specifically, the payment system 51 performs payment processing based on the specified amount. Then, the service providing system 50 returns the result of the use of the service based on the right to the face authentication system 30.

The face authentication system 30 notifies the face matching terminal 40 of the usage results received from the service providing system 50 (step S1304).

The face matching terminal 40 displays the usage result received from the face authentication system 30 (step S1305). For example, the face matching terminal 40 displays a usage result screen 1420 as shown in FIG. 14(c). The usage result screen 1420 includes an icon 1421 indicating the result of the payment, a payment amount 1422, and an end button 1423 for closing the usage result screen 1420.

The facial authentication system 30 stores the usage results received from the service providing system 50 as history information (step S1306). Specifically, when a payment service is used, information such as the success or failure of facial authentication, the payment amount, and the date and time of use is stored. In other words, the contents of the rights that the user has used or attempted to use are stored.

The face authentication system 30 identifies the parent mobile terminal 10 owned by the user (parent) based on the parent-child relationship identified in the common face matching process in step S900, and notifies the parent mobile terminal 10 of the history information saved in step S1306 (step S1307). The notification may be sent by email, for example.

The parent mobile terminal 10 displays the notification received from the face authentication system 30. If the notification is made by email, email 1430 with the content as shown in FIG. 14(d) may be sent, for example. In email 1430, a subject 1431 indicating an overview of the history information is set. Furthermore, the body 1432 includes details of the usage information (such as the date and time of usage, the user, and the payment amount). Then, this processing sequence in FIG. 13 ends.

(Matching Process: Variation 1 of Payment Use)
13 and 14 have described a case where the system is applied to a payment service. Here, a modified example of the case where the system is applied to a payment service will be described. More specifically, a case where the rights setting of a user (child) is updated based on the processing sequence of FIG. 13 will be described. Note that the same reference numbers are used for processes that overlap with those shown in FIG. 13, and descriptions thereof will be omitted.

The parent mobile terminal 10 accepts the right setting for the user (child) (step S1501). For example, the right setting for the user (child) is performed by displaying a setting screen 1600 as shown in FIG. 16 (a). The setting screen 1600 shows basic information 1601 of the user (child) that has already been set, and right information 1602 to be granted to the user (child). Items of the right information 1602 include here whether or not payment can be used 1603, the upper limit of payment amount 1604, whether or not notification to the user (parent) is required 1605, and conditions 1606 for when notification is performed. The upper limit of payment amount 1604 is set to the cumulative amount that can be used. As the condition 1606, for example, a case where the remaining available amount becomes a predetermined amount (for example, less than 1000 yen) as a result of using the amount set in the upper limit of payment amount 1604 may be specified. When the setting button 1607 is pressed, the parent mobile terminal 10 passes the contents of the entered rights information 1602 to the face authentication system 30 and instructs it to be registered.

The face authentication system 30 determines whether the user (child) rights information received from the parent mobile terminal 10 is within the scope of the user's (parent's) rights (step S1502). At this time, if the information exceeds the user's (parent's) rights, the parent mobile terminal 10 may be notified of this and prompted to re-input. Alternatively, the setting screen 1600 shown in FIG. 16(a) may be configured so that settings that exceed the user's (parent's) rights cannot be specified.

Based on the confirmation result of step S1502, the face authentication system 30 stores the rights information for the user (child) (step S1503).

Next, the face matching terminal 40, the face authentication system 30, and the service providing system 50 cooperate to perform the face matching common process (step S900). This process is the face matching common process described with reference to FIG. 9. In this process, for example, the face matching terminal 40 displays a payment screen 1610 as shown in FIG. 16(b) and accepts the setting of the right to use (step S903). The payment screen 1610 includes payment information 1611 indicating the amount of payment, a cancel button 1612 for canceling the execution of the payment, and a decision button 1613 for instructing the execution of the payment. Furthermore, in the face matching common process, the face matching terminal 40 displays a photographing screen 1620 for face matching as shown in FIG. 16(c) to prompt the user to take a photograph of the target user. The photographing screen 1620 may display an icon 1621 or a message to obtain an appropriate face image.

The face authentication system 30 performs steps S1301 to S1306 in the same manner as the processing sequence shown in FIG. 13. In this series of steps, the face matching terminal 40 displays a usage result screen 1630 as shown in FIG. 16(d) as the result of using the payment service. The usage result screen 1630 includes an icon 1631 indicating the result of the payment, a payment amount 1632, and an end button 1633 for closing the usage result screen 1630.

After the processing of step S1306, the face authentication system 30 determines whether the notification condition set on the setting screen 1600 is satisfied as a result of using the service (step S1504). In this example, it is assumed that the upper limit of payment 1604 is set to "10,000 yen", notification necessity 1605 is set to "ON", and notification condition 1606 is set to "less than 1,000 yen" on the setting screen 1600. In this case, as shown in FIG. 16(d), if the payment amount is "9,980 yen", the condition 1606 is satisfied. If the notification condition for resetting is satisfied (step S1504, YES), the processing of the face authentication system 30 proceeds to step S1505. On the other hand, if the notification condition for resetting is not satisfied (step S1504, NO), the processing of the face authentication system 30 proceeds to step S1307.

The face authentication system 30 identifies the parent mobile terminal 10 owned by the user (parent) based on the parent-child relationship identified in the common face matching process in step S900, and notifies the parent mobile terminal 10 of the history information saved in step S1306 (step S1505). The notification may be sent by email, for example. The notification includes a message urging the user (child) to reset their rights.

The parent mobile terminal 10 displays the notification received from the face authentication system 30 (step S1308). If the notification is made by email and includes a message prompting the user to reset the settings in step S1505, an email 1640 with the contents as shown in FIG. 16(e) may be sent. In the email 1640, a subject 1641 indicating an overview of the history information is set. The text 1642 includes details of the usage information (such as the date and time of usage, the user, and the payment amount) and a message prompting the user (child) to reset the rights. The email 1640 further includes a change button 1643 for changing the settings of the user (child) rights. If the processing in step S1307 is performed, the parent mobile terminal 10 displays an email as shown in FIG. 14(d).

The parent mobile terminal 10 judges whether or not a reset of the rights of the user (child) has been instructed (step S1506). For example, if the change button 1643 of the email 1640 shown in FIG. 16(e) is pressed, it is assumed that a reset instruction has been given (step S1506, YES), and the processing of the parent mobile terminal 10 returns to step S1506, and the user (child) is reset. At this time, the parent mobile terminal 10 causes the right setting for the user (child) to be performed by displaying, for example, a setting screen 1650 as shown in FIG. 16(f). The configuration of the setting screen 1650 is basically the same as that of the setting screen 1600 shown in FIG. 16(a). Here, as shown in the upper limit of payment 1651, it may be configured to display the values before and after the change. If a reset instruction has not been given (step S1506, NO), the processing returns to step S900, and the processing is repeated as appropriate.

(Matching Process: Variation 2 of Payment Use)
In Figures 13 and 14, a case where the application is made to a payment service has been described. Here, a second modified example of the case where the application is made to a payment service will be described. More specifically, a case where the rights setting of a user (child) is updated based on the processing sequence of Figure 13 will be described. Note that the same reference numbers are used for processes that overlap with those shown in Figure 13, and descriptions thereof will be omitted. Here, an example will be described assuming an insufficient balance, i.e., an attempt to make a payment that exceeds the upper limit of the available amount.

The face matching terminal 40, the face authentication system 30, and the service providing system 50 cooperate to perform the face matching common process (step S900). This process is the face matching common process described with reference to FIG. 9. In this process, for example, the face matching terminal 40 displays a payment screen 1800 as shown in FIG. 18(a) and accepts the setting of the right to use (step S903). The payment screen 1800 includes payment information 1801 indicating the amount of payment, a cancel button 1802 for canceling the execution of the payment, and a decision button 1803 for instructing the execution of the payment. Furthermore, in the face matching common process, the face matching terminal 40 displays a photographing screen 1810 for face matching as shown in FIG. 18(b) to prompt the target user to take a photograph. The photographing screen 1810 may display an icon 1811 or a message in order to obtain an appropriate face image.

The face authentication system 30 performs steps S1301 to S1306 in the same manner as the processing sequence shown in FIG. 13. In this series of steps, the face matching terminal 40 displays a usage result screen 1820 as shown in FIG. 18(c) as the result of using the payment service. The usage result screen 1820 includes an icon 1821 indicating the result of the payment, a payment amount 1822, and an end button 1823 for closing the usage result screen 1820. Here, a case is shown where the payment has failed due to insufficient balance.

After processing in step S1306, the face authentication system 30 determines whether or not the payment has failed due to insufficient balance as a result of using the service (step S1701). If the payment has failed due to insufficient balance (step S1701, YES), the processing of the face authentication system 30 proceeds to step S1702. On the other hand, in other cases (step S1701, NO), the processing of the face authentication system 30 proceeds to step S1307.

The face authentication system 30 identifies the parent mobile terminal 10 owned by the parent user based on the parent-child relationship identified in the common face matching process in step S900, and notifies the parent mobile terminal 10 of the history information saved in step S1306 (step S1702). The notification may be sent by email, for example. The notification includes a message urging the user (child) to reset their rights.

The parent mobile terminal 10 displays the notification received from the face authentication system 30 (step S1308). If the notification is made by email and includes a message prompting the user to reset the settings in step S1505, an email 1830 with the contents as shown in FIG. 18(d) may be sent. In the email 1830, a subject 1831 indicating an overview of the history information is set. The text 1832 includes details of the usage information (such as the date and time of usage, the user, and the payment amount) and a message prompting the user (child) to reset the rights. The email 1830 further includes a change button 1833 for changing the settings of the user (child) rights. If the processing in step S1307 is performed, the parent mobile terminal 10 displays an email 1430 as shown in FIG. 14(d).

The parent mobile terminal 10 determines whether or not a reset of the user's (child's) rights has been instructed (step S1703). For example, if the change button 1833 of the email 183 shown in FIG. 18(d) is pressed, it is assumed that a reset instruction has been given (step S1703, YES), and the processing of the parent mobile terminal 10 proceeds to step S1704. If a reset instruction has not been given (step S1703, NO), the processing returns to step S900, and the processing is repeated as appropriate.

The parent mobile terminal 10 accepts the right setting for the user (child) (step S1704). For example, the right setting for the user (child) is performed by displaying a setting screen 1840 as shown in FIG. 18 (e). The setting screen 1840 shows basic information 1841 of the user (child) that has already been set, and right information 1842 that is permitted for the user (child). Items of the right information 1842 here include payment availability 1843 and a payment upper limit 1844. The payment upper limit 1844 is set as the cumulative amount that can be used. As shown in the payment upper limit 1844, the configuration may be such that the values before and after the change are displayed. When the setting button 1845 is pressed, the parent mobile terminal 10 passes the contents of the entered right information 1842 to the face authentication system 30 and instructs it to be registered.

The face authentication system 30 determines whether the user (child) rights information received from the parent mobile terminal 10 is within the scope of the user's (parent's) rights (step S1705). At this time, if the information exceeds the user's (parent's) rights, the parent mobile terminal 10 may be notified of this and prompted to re-input. Alternatively, the setting screen 1840 shown in FIG. 18(e) may be configured so that settings that exceed the user's (parent's) rights cannot be specified.

Based on the result of the verification in step S1705, the face authentication system 30 stores the rights information for the user (child) (step S1706). Then, this processing sequence in FIG. 17 ends.

In the above example, the available amount and remaining amount are described as being common to the entire group, but it is also possible to set an upper limit or remaining amount for each user (child). In this case, a setting screen may be provided that displays the setting information for each user (child) in an identifiable manner. For example, even if the remaining amount is 10,000 yen, a certain user (child) may be restricted to using only up to 500 yen unless the rights are reset from the parent mobile terminal 10. In this way, it is possible to prevent a user (child) from excessively using the rights of the entire group.

(Matching process: case of using accommodation management service)
Next, a case where the basic processing sequence described in Fig. 9 and Fig. 10 is applied to the use of an accommodation management service will be described. The accommodation management service here is a service provided in cooperation with the accommodation management system 52 of the service providing system 50, and will be described as a service that presents user information related to accommodation through face recognition.

Figure 19 is a diagram showing a processing sequence when the matching process according to this embodiment is applied to facility use. Each device, which is the processing subject in the processing sequence, realizes each function by, for example, a control unit such as a CPU reading and executing a program corresponding to each function shown in Figure 2.

The face matching terminal 40, face authentication system 30, and service providing system 50 work together to perform a common face matching process (step S900). This process is the common face matching process described with reference to FIG. 9. In this process, for example, the face matching terminal 40 displays a photographing screen 2000 for face matching as shown in FIG. 20(a) to prompt the target user to take a photograph. The photographing screen 2000 may display an icon 2001 or a message in order to obtain an appropriate face image.

The face authentication system 30 requests information registered for the identified user from the service providing system 50 (here, the accommodation management system 52) that provides the service used by the user (step S1901). This request includes user information (such as parent-child relationships) managed by the face authentication system 30.

The service providing system 50 provides the service based on the request from the face authentication system 30 (step S1902). Specifically, the accommodation management system 52 identifies the registration information based on the specified user information. Then, the service providing system 50 returns the identified registration information to the face authentication system 30.

The face authentication system 30 notifies the face matching terminal 40 of the registration information received from the service providing system 50 (step S1903).

The face matching terminal 40 displays the registration information received from the face authentication system 30 (step S904). For example, the face matching terminal 40 displays an information provision screen 2010 as shown in FIG. 20(b). The information provision screen 2010 includes an icon 2011 indicating the face authentication result, a face image 2012 used in the face authentication, and information 2013 registered in correspondence with the user (child). In this example, the room number of the accommodation facility is displayed as the information 2013.

The face authentication system 30 stores the service usage results, including the registration information received from the service providing system 50, as history information (step S1905). Specifically, when the accommodation management service is used, information such as the success or failure of face authentication, the provided registration information, and the date and time of service usage is stored.

The face authentication system 30 identifies the parent mobile terminal 10 owned by the user (parent) based on the parent-child relationship identified in the common face matching process in step S900, and notifies the parent mobile terminal 10 of the history information saved in step S1905 (step S1906). The notification here may be sent by email, for example.

The parent mobile terminal 10 displays the notification received from the face authentication system 30. If the notification is made by email, an email 2020 with the contents as shown in FIG. 20(c) may be sent, for example. In the email 2020, a subject 2021 indicating an overview of the history information is set. The body 2022 also includes details of the history information (such as the date and time of use, the user, and the registered information referenced). Then, this processing sequence in FIG. 19 ends.

(Specific application examples)
The cooperation with the payment service described with reference to FIGS. 11 to 14 can be applied to a situation where a person moves around empty-handed in an accommodation facility (such as a hotel or inn) or an amusement facility. For example, when a child (i.e., a user (child)) belonging to a group moves around the facility empty-handed, it is possible to purchase items, use and manage admission to paid facilities (such as lounges, hot spring facilities, pools, gyms, etc.), and board transportation (such as shuttle buses and vehicles for guests only) using only face authentication. In addition, the user (parent) can manage the behavior of the user (child) by restricting use, and can also check information at the time of use.

The technology can also be applied to services such as rental cars, car sharing, and bus tours. For example, in a rental car service, facial recognition can be used to allow the service to be used in cases where the person who makes the reservation and the person who receives the vehicle are different. In addition, in services such as bus tours, facial recognition can be used to allow the service to be used in cases where the boarding location of the bus is different for each participant.

In addition, even if the user (child) is not with the user (parent), the user (child) can act freely within the scope of pre-defined rights without needing any specific tools. For example, the user (child) can check into an accommodation booked by the user (parent) using only facial recognition, and enter a booked room without a key (room key). In addition, users who are not registered in the group are restricted from checking in, which is expected to have a positive effect on security.

The lost child determination service provided by the lost child determination system 54 can be implemented in the same manner as the lodging management service shown in FIG. 19. For example, by photographing and authenticating a child's face image, it is possible to obtain information on which group (e.g., family) the child belongs to. This makes it possible to determine whether the child is lost or not, and to notify the relevant parties (i.e., the user (parent)). In addition, if a user (child) belonging to a group experiences some kind of trouble (such as injury or illness), it is possible to notify the user (parent) by authenticating the user's (child's) face.

The lost child determination service may take a picture of the child's face using a terminal used to utilize the rights. In an embodiment, terminals used to utilize the rights are placed in various locations within the facility, so lost child notifications can be made accurately without the need for special terminals.

The lost child determination service may also notify information about the device on which face authentication was last performed, in response to an instruction from a user (parent) or a user (child) other than the lost child. In this embodiment, face authentication is performed when a child (an example of a user (child)) uses a right, so information about the device on which the child (an example of a user (child)) last used or attempted to use a right is recorded in the system. Therefore, by adopting such a configuration, even if the child (an example of a user (child)) does not understand how to operate the terminal, the child's (an example of a user (child)) approximate current location can be known through operations by a parent (an example of a user (parent)) or a family member.

In addition, by authenticating the user's face, the user can easily check the registered information, which can prevent the user from taking actions based on incorrect information (such as the room number of the accommodation). For example, by installing face recognition terminals in elevators, the elevators can be controlled to automatically stop at the floor where the accommodation room is located, and terminals installed in the elevator halls on each floor can display information (such as maps and arrows) guiding the user to the accommodation.

In addition, by registering coupons, preferential treatments, points, etc. issued by facilities as rights to be registered to a group, these can be used by users (children) belonging to the group in addition to the user (parent), improving the convenience of using the service.

It is also possible for other users in a group to take over and use a service that has been used by a certain user. Specifically, a situation can be envisaged in which a certain user uses a luggage storage service and then receives luggage that had been left by another user in the same group. It is also possible for rights granted to a user (parent) to be shared. For example, it can be envisaged that points granted to a user (parent) can be shared and used by other users (children) in the group.

As described above, the face authentication system 30 according to the first embodiment includes a parent face information management unit 32 and a parent right management unit 35 that register the right information of a parent user in association with a group, a child face information management unit 33 that registers face images of one or more child users in association with the group, a parent-child relationship determination unit 31 that, when a face image is received, performs face authentication based on the information registered in the child face information management unit 33 to identify the group to which the user corresponding to the face image belongs, a usage right determination unit 34 that determines whether or not the desired processing requested by the child user corresponding to the face image is permitted based on the right information associated with the group identified by the parent-child relationship determination unit 31, an external system linkage unit 37 that executes the desired processing based on the determination result by the usage right determination unit 34, and a usage result notification unit 39 that notifies the parent user of at least one of the determination result by the usage right determination unit 34 and the processing result by the external system linkage unit 37.

As a result, the facial authentication system 30 can further improve the convenience for users belonging to a group when using various services through facial authentication.

The child rights management unit 36 also accepts and registers the settings of rights granted to each child user belonging to the group within the scope indicated by the parent user's rights information.

This allows the face authentication system 30 to grant any rights within the scope of the parent user's rights to each child user belonging to the group.

In addition, the parent rights management unit 35 updates the rights information of the parent user associated with the group in response to a request from the parent user.

This allows the parent user to set appropriate sharing rights for the group.

Facial authentication system 30 is communicatively connected to facial matching terminal 40, which has a facial information acquisition unit 42 that acquires a facial image of the user, and a usage right setting unit 43 that sets information on the processing desired by the user and transmits a request to facial authentication system 30 including the user's facial image acquired by facial information acquisition unit 42 and the set information on the desired processing. Facial authentication system 30 further has a result notification unit 45 that transmits the determination result by usage right determination unit 34 and the processing result by external system linkage unit 37 to the facial matching terminal 40 as a response to a request from the facial matching terminal 40.

This allows the face authentication system 30 to easily perform any processing by acquiring a face image using the face matching terminal 40.

Facial authentication system 30 is communicatively connected to parent mobile terminal 10, and parent mobile terminal 10 has a parent facial information acquisition unit 14 that acquires a facial image and requests facial authentication system 30 to register rights information of the parent user who is the user of parent mobile terminal 10 in association with the group, and a child registration link transmission unit 17 that determines whether or not to acquire a facial image of a child user belonging to the group at parent mobile terminal 10, and transmits a request including information for registering the facial image of the child user in facial authentication system 30 to child mobile terminal 20 that acquires the facial image of the child user when it is determined that the facial image of the child user is not to be acquired at parent mobile terminal 10.

This allows the face authentication system 30 to easily register the face image of the child user on a terminal other than the parent mobile terminal 10.

In addition, if it is determined that the facial image of the child user is to be acquired by the parent mobile terminal 10, the child facial information acquisition unit 15 acquires the facial image of the child user and then requests the facial authentication system 30 to register the facial image of the child user in association with the group.

This allows the face authentication system 30 to easily register the face image of the child user on the parent mobile terminal 10 as well.

Furthermore, the face authentication system 30 is communicatively connected to a service providing system 50 that executes at least a portion of the desired processing, and the external system linking unit 37 executes the desired processing in cooperation with the service providing system 50 based on the determination result by the usage right determination unit 34.

This allows the facial authentication system 30 to perform processing in cooperation with systems that provide various services external to the facial authentication system 30.

The desired process is one of a payment service, an accommodation management service, an admission management service to a facility, or a registration information provision service.

This allows the facial recognition system 30 to apply facial recognition processing to payment services, accommodation management services, facility admission management services, or registration information provision services.

<Other embodiments>
In the above-described embodiments, the notification of the use of the right is made to the parent mobile terminal 10, but may be made to the child mobile terminal 20. For example, the user (parent) can specify the child mobile terminal 20 owned by the user (child) to be notified, thereby making it possible to make the notification. This allows the user (child) to check the result of the use of the right through the child mobile terminal 20. The notification of the use of the right may also be made to the child mobile terminal 20 owned by the user (child) who used the right. In this way, the user (child) can check the information on the right and service used by the user (child) through the child mobile terminal 20 later. In particular, when the user (child) uses the right away from the user (parent), it is difficult for the user (child) to know whether the use of the right was really successful, which may cause anxiety. Therefore, if the notification is made to the child mobile terminal 20 owned by the user (child) who used the right, the user (child) can check the success or failure of the use of the right from the user (child) himself, thereby reducing such anxiety. In addition, as long as the use of the right by the user (child) is successful, the notification of the use of the right may not be made to the parent mobile terminal 10. This makes it unnecessary to check the notification on the parent mobile terminal 10 when no particular problem has occurred, and therefore makes it possible to prevent excessive notifications to the user (parent). In addition, the parent mobile terminal 10 or the child mobile terminal 20 may make a request to the face authentication system 30 or the like to check the history of use of rights.

In addition, in each of the above-mentioned embodiments, the face matching terminal 40 has been described as being installed in a facility that provides various services, but it may also be installed inside a mobile body. One example is a shuttle bus that runs between a station and a hotel. In this case, the desired processing is a transportation service that uses the mobile body, or a service provided inside the mobile body (in-car sales, tourist guides, etc.). Furthermore, the face matching terminal 40 does not need to be associated with a specific facility, etc. For example, the face matching terminal 40 may be in a form in which software for providing various services is installed in a mobile terminal (specifically, a parent mobile terminal 10 or a child mobile terminal 20) owned by a user, etc. In this case, the software provides the service on the condition that face authentication is successful and the right is used.

In addition, in each of the above-mentioned embodiments, the facial information of the user (parent) is registered in the facial authentication system, but registration of the user (parent)'s face may be omitted. The user (parent) is likely to carry the parent mobile terminal 10 to understand the usage status of the user (child)'s rights, and in this case, the parent mobile terminal 10 can be used for the user (parent)'s own use of the rights. Similarly, if the user (parent) does not plan to use the rights himself and only manages the rights for the user (child), registration of the user (parent)'s facial information may be omitted. In these cases, instead of the facial information of the user (parent), other biometric information of the user (parent) (such as a fingerprint) or an ID uniquely assigned to the user (parent) may be used as information to identify the representative of the group.

In addition, in each of the above-described embodiments, each piece of user information is registered by the parent mobile terminal 10 or the child mobile terminal 20, but this is not limited to this. For example, registration may be performed by a device installed in the facility, such as a terminal installed at the front desk of a hotel.

In addition, in each of the above-mentioned embodiments, it has been described that there is only one user (parent) in a group, but it may be possible to set multiple users (parents). In this case, notifications to the parent mobile terminal 10 may be sent to terminals owned by each of the multiple users (parents). In addition, the notifications received and the rights that can be changed may be different between users (parents). For example, it is conceivable that only one user (parent) can change rights related to money, but other users (parents) can change rights that are not related to money (such as the right to use facilities free of charge).

In addition, in each of the above-mentioned embodiments, the information notified to the parent mobile terminal 10 may be only the result of the face authentication of the user (child) or only the result of the service usage. In particular, if the face authentication of the user (child) fails, the result of the determination as to whether the service can be used is not possible, so it is not necessary to notify the result of the usage.

In addition, programs and applications for implementing the functions of one or more of the above-described embodiments can be supplied to a system or device via a network or storage medium, etc., and one or more processors in the computer of the system or device can read and execute the programs.

It may also be realized by a circuit that realizes one or more functions (e.g., an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array)).

Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to the examples described. It is clear to a person skilled in the art that various modifications, revisions, substitutions, additions, deletions, and equivalents may be made within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure. Furthermore, the components in the various embodiments described above may be combined in any manner as long as they do not deviate from the spirit of the invention.

The present disclosure is useful as a face recognition system, a face recognition method, an information processing terminal, and a control method thereof that can improve the convenience of actions according to rights for multiple users.

10 Parent mobile terminal 20 Child mobile terminal 30 Face authentication system 40 Face matching terminal 50 Service providing system 60 Network 300 Mobile terminal 400 Information processing device

Claims (14)

a first registration unit that registers rights information of a parent user in association with a group;
a second registration unit that registers face images of one or more child users in association with the group;
an identification unit that, when receiving a face image, identifies a group to which a user corresponding to the face image belongs by performing face authentication based on the information registered by the second registration unit;
a first determination unit that determines whether or not a desired process requested by a child user corresponding to the face image is permitted based on rights information associated with the group identified by the identification unit;
a processing unit that executes the desired process based on a result of the determination by the first determination unit;
a notification unit that notifies the parent user of information including at least one of a result of the determination by the first determination unit and a result of the processing by the processing unit , and a change button for transitioning to a screen for resetting the rights information ;
Facial recognition system. the second registration unit accepts and registers settings of rights granted to each child user belonging to the group within a range indicated by the rights information of the parent user;
The face authentication system according to claim 1 . the first registration unit updates the right information of the parent user associated with the group in response to a request from the parent user.
The face authentication system according to claim 1 or 2. The face authentication system is communicatively connected to a face matching terminal,
The face matching terminal includes:
A first acquisition unit that acquires a face image of a user;
a first setting unit for setting information of the desired processing;
a first request unit configured to transmit a request to the face authentication system, the request including the face image of the user acquired by the first acquisition unit and information on a desired process set by the first setting unit;
having
The face recognition system further comprises:
a first transmission unit that transmits a determination result by the first determination unit and a processing result by the processing unit to the face matching terminal as a response to the request;
A face authentication system according to any one of claims 1 to 3. The face authentication system is communicatively connected to an information processing terminal,
The information processing terminal includes:
A second acquisition unit that acquires a face image;
a second request unit that requests the face authentication system to register rights information of a parent user who is a user of the information processing terminal in association with a group;
a second determination unit that determines whether or not to acquire a face image of a child user belonging to the group at the information processing terminal;
a second transmission unit that transmits a request including information for registering the face image of the child user in the face authentication system to another information processing terminal that acquires the face image of the child user when it is determined that the face image of the child user is not to be acquired by the information processing terminal;
having
A face authentication system according to any one of claims 1 to 4. When it is determined that a facial image of the child user is to be acquired by the information processing terminal, after the second acquisition unit acquires the facial image of the child user, the second request unit requests the face authentication system to register the facial image of the child user in association with a group.
The face authentication system according to claim 5 . the face authentication system is communicatively connected to a service providing system that executes at least a part of the desired processing;
The processing unit executes the desired process in cooperation with the service providing system based on a result of the determination by the first determination unit.
A face authentication system according to any one of claims 1 to 6. The desired process is any one of a payment service, an accommodation management service, an admission management service to a facility, and a registration information provision service.
A face authentication system according to any one of claims 1 to 7. The desired process is a payment service,
The rights information includes an upper limit of the amount that the child user can use in the payment service,
The resetting of the right information is a resetting of the upper limit price.
A face authentication system according to any one of claims 1 to 7. The notification unit is
notifying the user of information including a change button for transitioning to a screen for resetting the upper limit amount together with the determination result or the processing result at least one of a case where the difference between the usage amount used by the payment service used by the child user and the upper limit amount becomes a predetermined amount or a case where the usage amount exceeds the upper limit amount;
The face authentication system according to claim 9. a first registration step of registering rights information of a parent user in a storage unit in association with the group;
a second registration step of registering face images of one or more child users in the storage unit in association with the group;
a step of identifying a group to which a user corresponding to a face image belongs by performing face authentication based on information registered in the storage unit when a face image is received;
a determination step of determining whether or not a desired process requested by a child user corresponding to the face image is permitted based on rights information associated with the group identified in the identification step;
a processing step of executing the desired processing based on a result of the determination step;
a notification step of notifying the parent user of information including at least one of the determination result in the determination step and the processing result in the processing step, and a change button for transitioning to a screen for resetting the rights information;
having
Face recognition method. An information processing terminal,
a request unit that requests the face authentication system to register rights information of a parent user who is a user of the information processing terminal in association with a group;
a receiving unit that, when a child user belonging to the group determines whether a desired process is permitted based on face authentication using the face of the child user and rights information registered in association with the group, receives information including a notification of at least one of the result of the determination and the result of the desired process, and a change button for transitioning to a screen for resetting the rights information ;
having
Information processing terminal. The information processing terminal further comprises:
An acquisition unit for acquiring a face image;
a determination unit that determines whether or not to acquire a face image of a child user belonging to the group at the information processing terminal;
a transmission unit that transmits a request including information for registering the face image of the child user to another information processing terminal that acquires the face image of the child user when it is determined that the face image of the child user is not to be acquired by the information processing terminal;
having
13. An information processing terminal according to claim 12 . A method for controlling an information processing terminal, comprising:
a request step of requesting the face authentication system to register rights information of a parent user who is a user of the information processing terminal in association with a group;
a receiving step of receiving information including a notification of at least one of a result of the determination and a result of the desired processing, and a change button for transitioning to a screen for resetting the right information, when a child user belonging to the group determines whether a desired processing is permitted based on face authentication using the face of the child user and right information registered in association with the group ;
having
A method for controlling an information processing terminal.

Images