JP7624600B2 - Personal identification terminal, program, personal identification method, and personal identification system - Google Patents

Description

This disclosure relates to an identity verification terminal, a program, an identity verification method, and an identity verification system.

Non-Patent Document 1 describes a method of age verification using digital technology, in which age is verified using an official identification document such as a driver's license or My Number card. The age verification methods disclosed using digital technology include a method in which the official identification document itself is inserted into some device and the age is verified based on the date of birth, and a method in which the identity is verified in advance by confirming the official identification document in person or by mail, and then the identity is authenticated using a mobile phone or biometric authentication, etc.

Japan Franchise Association, "Guidelines for verifying age of alcohol and tobacco using digital technology", [online], searched on February 24, 2023, Internet <URL: https://www. digital. go. jp/assets/contents/node/basic_page/field_ref_resources/0196dc94-0101-4063-8d6e-a8af88589d2c/5db45cc1/20221130_meeting_administrative_research_working_group_outline_02.pdf>

In recent years, convenience stores and other outlets have increasingly been introducing unmanned registers (hereafter referred to as "self-registers") that allow customers to process their purchases themselves as a means of reducing labor through technology. However, products that require age verification, such as alcohol and tobacco (hereafter referred to as "licensed products"), require employees or other personnel to verify the customer's age in person, posing a challenge in terms of reducing labor.

Non-Patent Document 1 discloses that an official identification card is read by reading the card face information, an Integrated Circuit (hereinafter referred to as "IC") reader reading, and a Personal Identification Number (hereinafter referred to as "PIN code") code input to confirm the date of birth and age. By using an official identification card to verify age in advance, it is believed that licensed products can be purchased at self-checkout registers without the involvement of an employee. However, there is a risk that a minor may pose as an adult and purchase licensed products by verifying age using another person's official identification card. Note that "adult" here refers to the age at which licensed products can be purchased.

This disclosure was devised in light of the conventional situation described above, and aims to provide highly reliable identity verification.

The present disclosure provides an identity verification terminal that includes a camera that captures an image of a user's face, a reader that acquires a registration image that is an image of the face from an identity verification document, and a processor that acquires the image of the user's face captured by the camera, compares the registration image with the captured image, and completes identity verification of the user if facial authentication between the face of a person appearing in the registration image and the face of the user is successful, and registers the registration image or the captured image as an image for identity verification.

The present disclosure also provides a program that causes a computer to execute the following processes: a process of having a camera capture a picture of a user's face; a process of acquiring a registration image from an identification document; a process of acquiring a captured image of the user's face; a process of matching the registration image with the captured image; and a process of completing identity verification of the user when facial authentication between the face of a person appearing in the registration image and the face of the user is successful, and registering the registration image or the captured image as an image for identity verification.

The present disclosure also provides a method of identity verification that includes photographing a user's face, acquiring a registration image from an identity verification document, acquiring a photographed image of the user's face, comparing the registration image with the photographed image, and completing identity verification of the user if facial authentication between the face of a person appearing in the registration image and the face of the user is successful, and registering the registration image or the photographed image as an image for identity verification.

The present disclosure also provides an identity verification system including a self-checkout register and an identity verification terminal including a first camera that photographs the face of the user, a reader that obtains a registration image from an identity verification document, and a processor that obtains a first photographed image of the user's face photographed by the first camera, compares the registration image with the first photographed image, and completes identity verification of the user if facial authentication between the face of a person appearing in the registration image and the face of the user is successful, and registers the registration image or the first photographed image as an identity verification image; a second camera that photographs the face of the user, and a processor that obtains the identity verification image from the identity verification terminal, compares the second photographed image of the user's face photographed by the second camera with the identity verification image, and permits the purchase of a licensed item if facial authentication between the face in the second photographed image and the face in the identity verification image is successful.

These comprehensive or specific aspects may be realized as a system, device, method, integrated circuit, computer program, or recording medium, or as any combination of a system, device, method, integrated circuit, computer program, and recording medium.

This disclosure makes it possible to perform highly reliable identity verification.

A flowchart showing an example of a conventional method for purchasing licensed products. A system configuration diagram of an identity verification system according to the present embodiment. A diagram showing an application running on a terminal device. A flowchart showing an example of a member registration process in a member application. A flowchart showing an example of a process for identity verification using a My Number card Flowchart of image registration process for identity verification using My Number card A flowchart showing an example of a process for identity verification using a driver's license. Flowchart of image registration process for personal identification using driver's license A diagram showing an example of a screen showing items that have been registered in a member app Flowchart of the self-checkout licensed product purchase authorization process A diagram to explain the use case of using the hotel member app A diagram to explain the use case of using a rental car member app A diagram to explain the use case of using a gas station member app A diagram to explain the use case of using the member app of government services A diagram explaining the use case of applying for financial services using an identity verification app A diagram to explain the use case when registering a credit card payment app A diagram to explain the use cases when using a credit card payment app Block diagram showing a modified example of the personal identification system A flowchart showing an example of a process for registering an image for personal identification in a personal identification server and reusing the image. A flowchart showing an example of the registered process shown in FIG. 17 .

Below, with reference to the drawings as appropriate, a detailed description will be given of the embodiments specifically disclosing the identity verification terminal, program, identity verification method, and identity verification system according to the present disclosure. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and duplicate descriptions of substantially identical configurations may be omitted. This is to avoid the following description becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the attached drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

An example of a conventional method for purchasing licensed items will be described with reference to FIG. 1. FIG. 1 is a flowchart showing an example of a conventional method for purchasing licensed items.

In a store such as a convenience store or supermarket, in order for a user (e.g., a person shopping at the store) to be able to purchase licensed products at a self-checkout register, the company that manages the store may require the user to register in advance for an application. Hereinafter, an application provided by a company that registers users as members will be referred to as a "member application." Note that stores are not limited to being managed by companies, but may also be managed by corporations, public organizations, associations, or individuals. There is a method in the member application that allows a user to purchase licensed products at a self-checkout register by verifying their age in advance using an identification card. An example of a method for purchasing licensed products using a member application will be described below.

The user registers as a member by entering personal information such as name and date of birth into the member app. The user follows the instructions of the member app to verify their age using an identification card (step St100). The identification card is, for example, a driver's license or a My Number card. For example, the member app verifies the user's age by reading the information in the IC chip of an identification card such as a driver's license or My Number card with a terminal device such as a smartphone owned by the user. The identification card can be interpreted as an identity verification document.

The user enters the store (step St101), picks up the products, including the licensed products, to be purchased (step St102), and carries the products to the self-checkout register. The user operates the self-checkout register and selects whether or not to purchase a bag to put the products in (hereinafter referred to as a "plastic bag") (step St103). The user has the self-checkout register read the licensed products (step St104).

The user has the self-checkout register read the registration information of the member app (step St105). For example, the user has the self-checkout register read a barcode that includes information indicating that the user has passed age verification and is an adult as registration information of the member app.

The self-checkout register completes age verification based on the registration information read in step St105 (step St106) and permits the purchase of licensed items.

The user makes the payment at the self-checkout (step St107) and leaves the store (step St108).

This allows users to purchase licensed products at the self-checkout using the member app registration information they have registered in advance.

However, when a user registers as a member in the member app in step St100 and uses an identification card to verify their age, there is a problem in that even if the user is a minor, the user may be able to falsify their age by using the identification card of another adult.

In addition, when the self-checkout register uses the registration information of the member app to verify the user's age in step St106 and allows the user to purchase a licensed product, there is a risk that the user may use a terminal device that contains a member app for another adult for which age verification has already been completed, causing the self-checkout register to read the registration information of another person. This poses the problem that a user may be able to purchase a licensed product at the self-checkout register even if they are a minor.

In addition, when the user makes a payment in step St107, the self-checkout register must read the points information (e.g., a points card), the payment code (e.g., a Quick Response code (hereinafter, referred to as a "QR code (registered trademark)")), and/or the credit card, etc., which is cumbersome for the user.

Although an example of a method for purchasing licensed products at a store such as a convenience store or supermarket has been described, the method is not limited to the above-mentioned stores, and unmanned sales of licensed products at tourist spots, theme parks, concert venues, etc. may also be envisioned. For example, a user can complete age verification in advance using a member app provided by the tourist spot, theme park, or concert venue, and by using the registration information of the member app, the user can purchase licensed products without the help of a store clerk at stores and vending machines located within the tourist spot, theme park, or concert venue. For ease of explanation, the following explanation uses a use case at a store such as a convenience store or supermarket.

Below, we will explain an identity verification system 1 (see Figure 2) that has been devised in consideration of the above-mentioned situation and is capable of performing highly reliable identity verification. In identity verification system 1, a member app is linked to an application (hereinafter referred to as an "identity verification app") that confirms in advance whether a user registered in the member app is the user himself/herself (hereinafter referred to as "identity verification"), and identity verification is performed in advance. Completion of identity verification indicates that the name and date of birth information registered in the member app is highly reliable information, and is synonymous with the user's age being confirmed. Note that the member app and identity verification app may be configured as a single application.

Next, referring to FIG. 2, a system configuration diagram of the identity verification system 1 according to the present embodiment will be described. FIG. 2 is a system configuration diagram of the identity verification system 1 according to the present embodiment.

The identity verification system 1 includes a terminal device 10, an identity verification authentication business server 20, an identity verification server 30, a member company server 40, a payment company server 50, a point company server 60, and a self-checkout register 70. The identity verification authentication business server 20 may be omitted from the identity verification system 1. The identity verification authentication business server 20 may also be read as an administrative server.

The terminal device 10 is a device that performs advance identity verification of a user. The terminal device 10 is, for example, a smartphone or a tablet. The terminal device 10 has a processor 11, a reader 12, a camera 13, a communication I/F 14, a display 15, and a memory 16.

The processor 11 is, for example, a Central Processing Unit (hereinafter referred to as "CPU"), a Digital Signal Processor (hereinafter referred to as "DSP"), a Graphical Processing Unit (hereinafter referred to as "GPU"), or a Field Programmable Gate Array (hereinafter referred to as "FPGA"). The processor 11 functions as a controller that manages the overall operation of the terminal device 10. The processor 11 performs control processing for managing the operation of each part of the terminal device 10, data input/output processing between each part of the terminal device 10, data arithmetic processing, and data storage processing. The processor 11 operates according to a program stored in the memory 16. The processor 11 uses the memory 16 during operation, and temporarily stores data generated or acquired by the processor 11 in the memory 16. The processor 11 executes programs for the member application and the identity verification application. The identity verification application will be described in FIG. 4.

The reader 12 is a device that acquires information contained in an IC chip of an identification card such as a driver's license or a My Number card. The reader 12 is, for example, a communication I/F, and acquires information in the IC chip using a short-range wireless communication technology such as Near Field Communication (hereinafter referred to as "NFC"). I/F stands for interface. The reader 12 may be omitted from the terminal device 10 and may be an external device such as an IC card reader. The reader 12 outputs personal information acquired from the identification card to the processor 11.

The camera 13 is a device that photographs the identification card and the user's face. The camera 13 is configured to have at least a lens (not shown) and an image sensor (not shown) as optical elements. The lens receives light reflected by an object within the angle of view of the area photographed by the camera 13 and forms an optical image of the object on the light receiving surface of the image sensor (in other words, the photographing surface). The image sensor is a solid-state photographing element such as a Charged Coupled Device (hereinafter referred to as "CCD") or a Complementary Metal Oxide Semiconductor (hereinafter referred to as "CMOS"). The image sensor converts the optical image formed on the photographing surface through the lens into an electrical signal every predetermined time (for example, 1/30 (second)). For example, when the predetermined time is 1/30 (second), the frame rate of the camera 13 is 30 fps. Additionally, the camera 13 may generate image data (video data) by performing a predetermined signal processing on the electrical signal at each predetermined time. Note that the image data is a still image, and the video data is a video. The camera 13 outputs the image data (video data) to the processor 11. Hereinafter, the image data and the video data are referred to as captured images.

The communication I/F 14 is an interface circuit that performs wireless or wired communication between the terminal device 10 and the identity verification authentication business server 20, between the terminal device 10 and the identity verification server 30, between the terminal device 10 and the member company server 40, between the terminal device 10 and the payment company server 50, between the terminal device 10 and the point company server 60, and between the terminal device 10 and the self-register 70. Communication between the terminal device 10 and the identity verification authentication business server 20, between the terminal device 10 and the identity verification server 30, between the terminal device 10 and the member company server 40, between the terminal device 10 and the payment company server 50, between the terminal device 10 and the point company server 60, and between the terminal device 10 and the self-register 70 may be via a network. The communication method by the communication I/F 14 is, for example, a wide area network (hereinafter referred to as "WAN"), a local area network (hereinafter referred to as "LAN"), a long term evolution (hereinafter referred to as "LTE"), mobile communication such as 4G and 5G, power line communication, short-range wireless communication (e.g. Bluetooth (registered trademark) communication), or communication for mobile phones. Note that when the reader 12 is a communication I/F, the reader 12 and the communication I/F 14 may be configured as a single communication I/F.

The display 15 is a display device that displays information such as registration information for the member app, input screens for the member app and the identity verification app, and the verification results for the identity verification app. The display 15 also accepts input operations from the user. The display 15 is, for example, a touch panel display.

The memory 16 is configured, for example, using Random Access Memory (hereinafter referred to as "RAM") and Read Only Memory (hereinafter referred to as "ROM"), and temporarily stores programs necessary for the operation of the terminal device 10, programs for the member app and the identity verification app, and further data generated during operation. The RAM is, for example, a work memory used during the operation of the terminal device 10. The ROM, for example, stores and holds in advance programs for controlling the terminal device 10.

The "identity verification authentication business operator" operates a service that performs identity verification etc. using the electronic certificate of the My Number card. The identity verification authentication business operator server 20 is a server device that provides a public personal authentication service. The identity verification authentication business operator server 20 accepts the electronic certificate of the My Number card in response to a request for a service that requires identity verification from the identity verification server 30, and checks the validity of the electronic certificate. If the identity verification authentication business operator server 20 is able to check the validity of the electronic certificate, it returns the result to the identity verification server 30. This completes identity verification when using the service.

The identity verification server 30 is a server device of a business that provides an identity verification app that verifies the identity of a user. The identity verification server 30 temporarily stores information on the user's identification card in memory (not shown). The identity verification server 30 also stores personal information such as the user's name, age, and identity verification image acquired by the identity verification app in memory. The identity verification image will be described later. Note that if the member app and identity verification app are configured as a single application, the identity verification server 30 may be omitted, and the above-mentioned personal information of the user may be stored in the memory (not shown) of the member company server 40.

The member company server 40 is a server device of a company that manages the store and provides a member app that registers users who use the store as members. The member company server 40 stores information such as personal information entered and registered by users in the member app. In addition, the member company server 40 may grant company-specific points when a user purchases an item at a store and link them to the registration information of the member app. The member company server 40 and the terminal device 10, the member company server 40 and the payment company server 50, the member company server 40 and the point company server 60, and the member company server 40 and the self-checkout 70 are connected so as to be able to communicate with each other.

The payment company server 50 is a server device of a business that provides payment services. The payment company server 50 provides payment services such as credit cards, QR codes (registered trademark), or electronic money. The payment company server 50 allows payment information to be linked to the registration information of a member app. When a product is purchased using the registration information of a member app to which the payment information is linked, the payment company server 50 communicates with the member company server 40 and performs payment processing. The payment company server 50 and the terminal device 10, the payment company server 50 and the member company server 40, and the payment company server 50 and the self-checkout 70 are connected so as to be able to communicate with each other.

The point company server 60 is, for example, a server device of an external point-granting company that is affiliated with the member company server 40. The point company server 60 allows its company's points to be linked to the registration information of the member app. When a product purchase is completed using the registration information of the member app linked to the point information, the point company server 60 communicates with the member company server 40 and grants points. The point company server 60 and the terminal device 10, the point company server 60 and the member company server 40, and the point company server 60 and the self-checkout 70 are connected so that they can communicate with each other.

The self-checkout register 70 is an unmanned register installed in a store. A user shopping at the store can operate the self-checkout register 70 and complete the product purchase procedure himself/herself. The self-checkout register 70 has a processor 71, a reader 72, a camera 73, a communication I/F 74, a display 75, and a memory 76.

Processor 71 is, for example, a CPU, DSP, GPU or FPGA. Processor 71 functions as a controller that manages the overall operation of self-checkout 70. Processor 71 performs control processing to integrate the operation of each part of self-checkout 70, data input/output processing between each part of self-checkout 70, data calculation processing, and data storage processing. Processor 71 operates according to a program stored in memory 76. Processor 71 uses memory 76 during operation, and temporarily stores data generated or acquired by processor 71 in memory 76.

The reader 72 is a device that acquires information contained in an IC chip of an identification card such as a driver's license or a My Number card. The reader 72 is, for example, an IC card reader. The reader 72 is also a communication I/F and may acquire information in the IC chip using a short-range wireless communication technology such as NFC. The reader 72 outputs personal information acquired from the identification card to the processor 71. The reader 72 also reads registration information (e.g., a barcode) of a member app displayed on the display 15 of the terminal device 10, and outputs the registration information to the processor 71.

The camera 73 is a device that photographs the face of a user who purchases a licensed product at the self-checkout register 70. The camera 73 is configured to have at least a lens (not shown) and an image sensor (not shown) as optical elements. The lens receives light reflected by an object within the angle of view of the area photographed by the camera 73 and forms an optical image of the object on the light receiving surface of the image sensor (in other words, the photographing surface). The image sensor is, for example, a solid-state photographing element such as a CCD or a CMOS. The image sensor converts the optical image formed on the photographing surface through the lens into an electrical signal every predetermined time (for example, 1/30 (second)). For example, when the predetermined time is 1/30 (second), the frame rate of the camera 73 is 30 fps. The camera 73 may also generate a photographed image by performing predetermined signal processing on the electrical signal every predetermined time described above. The camera 73 outputs the photographed image to the processor 71.

The communication I/F 74 is an interface circuit that communicates wirelessly or wired between the terminal device 10 and the self-register 70, the identity verification server 30 and the self-register 70, the member company server 40 and the self-register 70, the settlement company server 50 and the self-register 70, and the point company server 60 and the self-register 70. The communication between the terminal device 10 and the self-register 70, the identity verification server 30 and the self-register 70, the member company server 40 and the self-register 70, the settlement company server 50 and the self-register 70, and the point company server 60 and the self-register 70 may be via a network. The communication method by the communication I/F 74 is, for example, mobile communication such as WAN, LAN, LTE, 4G, 5G, power line communication, short-range wireless communication (for example, Bluetooth (registered trademark) communication), or communication for mobile phones. In addition, when the reader 72 is a communication I/F, the reader 72 and the communication I/F 74 may be configured as one communication I/F. The communication I/F 74 receives member app registration information (personal information such as an image for personal identification, name, and age) from the communication I/F 14 using a short-range wireless communication technology such as NFC.

Display 75 is a display that displays a screen for purchasing products, a guide for age verification, the results of age verification, and the like. Display 75 may be a touch panel display or the like, and may accept input from the user. When display 75 accepts input from the user, the input may be accepted using a mouse, a touch pen, a keyboard, or the like. Display 75 may also be a non-contact type aerial display in which a display is displayed in the air. In this case, display 75 accepts input from the user using an infrared sensor or the like.

The memory 76 is configured, for example, using RAM and ROM, and temporarily stores programs necessary for the operation of the self-register 70, personal information of the user received from the terminal device 10, and data generated during operation. The RAM is, for example, a work memory used during the operation of the self-register 70. The ROM, for example, stores and holds in advance programs for controlling the self-register 70.

Next, applications that run on a terminal device will be described with reference to FIG. 3. FIG. 3 is a diagram showing applications that run on a terminal device.

The terminal device 10 has a member app MA and an identity verification app HA. The member app MA and the identity verification app HA are stored in the memory 16.

The member app MA is an application provided by the member company server 40. The member app MA is installed on the terminal device 10 based on user operation.

The identity verification application HA is an application provided by the identity verification server 30. The identity verification application HA is installed on the terminal device 10 based on user operation.

The member app MA and the identity verification app HA operate in cooperation with the processor 11. The member app MA and the identity verification app HA may be configured as a single application. When the member app MA and the identity verification app HA are configured as a single application, the identity verification server 30 or the member company server 40 provides the application.

Next, an example of a member registration process in the member app MA will be described with reference to FIG. 4. FIG. 4 is a flowchart showing an example of a member registration process in the member app MA.

The processor 11 installs the member app MA in response to an input operation by the user to install the member app MA on the terminal device 10 (step St200).

The member app MA starts the process of registering the user as a new member (step St201).

The member app MA displays a screen on the display 15 requesting the user to input an email address. The member app MA receives information about the email address through an input operation by the user (step St202). The member app MA transmits the email address received in the processing of step St202 to the member company server 40. The member company server 40 transmits an authentication code to the received email address.

The user inputs the received authentication code into the member app MA. The member app MA receives the authentication code through the user's input operation (step St203).

The member app MA displays a screen on the display 15 requesting the user to enter a password and member information. The member app MA receives the password and member information through the user's input operation (step St204). The member information is, for example, information such as name, date of birth, gender, or address.

The member app MA displays a screen on the display 15 requesting the user to agree to the membership terms and conditions. The member app MA receives information indicating that the user has agreed to the membership terms and conditions through an input operation by the user, and completes the user's membership registration (step St205).

Note that the member registration method of the member app shown in FIG. 3 is merely an example, and is not limited to the example shown in FIG. 3. For example, the member app MA may request a phone number from the user instead of an email address in the processing of step St202, or may register the user as a member in cooperation with another application (e.g., a communication app, etc.).

Next, an example of identity verification processing using a My Number card will be described with reference to FIG. 5. FIG. 5 is a flowchart showing an example of identity verification processing using a My Number card. The processing of the flowchart in FIG. 5 is executed by an identity verification app HA.

The identity verification application HA determines whether or not to use a My Number card (step St300). For example, the identity verification application HA displays a screen on the display 15 asking the user whether or not to use a My Number card as identification for identity verification. The identity verification application HA performs the process of step St300 based on an input operation from the user. In addition, the type of identification used for identity verification of the user may be predetermined by the identity verification server 30, in which case the process of step St300 is omitted.

If the identity verification application HA determines based on the user's input operation that the My Number card will not be used (step St300, NO), it executes the process shown in FIG. 7.

When the identity verification app HA determines that a My Number card will be used based on the user's input operation (step St300, YES), it displays instructions to the user on the display 15 to read the information on the IC chip of the My Number card. For example, the identity verification app HA displays instructions to the user on the display 15 to hold the terminal device 10 over the My Number card.

When the user sets the My Number card in the terminal device 10 in a state in which the information on the IC chip can be read, the identity verification app HA reads the information on the IC chip of the My Number card (step St301). The identity verification app HA obtains personal information such as the user's name and date of birth as information on the IC chip.

The identity verification application HA starts an authentication process to verify the validity of the electronic certificate of the My Number card of the user (for example, displays a screen on the display 15 requesting the user to enter a password), and shares the authentication information of the My Number card with the identity verification authentication business operator server 20 (step St302).

The identity verification application HA receives the result of the verification of the validity of the electronic certificate from the identity verification authentication business operator server 20 (step St303). If the identity verification application HA can verify the validity of the electronic certificate, it determines that it has been able to prove that the logged-in user is the correct user.

The identity verification application HA determines whether or not to register payment information (step St304). The identity verification application HA displays a screen on the display 15 asking the user whether or not to register payment information. If the identity verification application HA determines that payment information is to be registered based on the user's input operation (step St304, YES), it acquires the payment information based on the user's input operation (step St305).

The identity verification app HA transmits to the payment company server 50 the payment information acquired in the processing of step St305 and a request for permission to link the payment information with the registration information of the member app MA. The payment company server 50 checks whether the received payment information is correct against information stored in the memory (not shown) of the payment company server 50. The payment company server 50 receives, for example, a credit card number, expiration date, and security code as payment information, and checks whether there are any errors by comparing the information with the information stored in the payment company server 50. If the payment company server 50 determines that the received payment information is correct, it transmits information permitting linking (i.e., linking) with the registration information of the member app MA to the identity verification app HA. The identity verification app HA links the payment information to the registration information of the member app MA (step St306). Then, the processor 11 advances the processing to step St307.

If the identity verification app HA determines based on the user's input operation that payment information should not be registered (step St304, NO), the identity verification app HA determines whether or not to register point information (step St307).

The identity verification app HA displays a screen on the display 15 asking the user whether or not to register point information. If the identity verification app HA determines that point information is to be registered based on the user's input operation (step St307, YES), it acquires the point information based on the user's input operation (step St308). The identity verification app HA acquires the point information, for example, by receiving input information of a point card number from the user or by linking with an application provided by the point company server 60.

The identity verification app HA transmits to the point company server 60 a request for permission to link the point information acquired in the processing of step St308 with the registration information of the member app MA. The point company server 60 checks whether the received point information is correct, and if it is correct, transmits information to the identity verification app HA that permits linking (i.e., linking) with the registration information of the member app MA. The identity verification app HA links the point information with the registration information of the member app MA (step St309). Note that the identity verification app HA may link the point information with the registration information of the member app MA by linking with an application provided by the point company server 60. After processing of step St309, the identity verification app HA executes processing of step St310.

When the identity verification app HA determines not to register point information based on the user's input operation (step St307, NO), it executes a process for registering an image of the user's face using the My Number card (hereinafter referred to as the "identification image registration process") (step St310). The identity verification image registration process using the My Number card is described in detail in FIG. 6. Hereinafter, the face image to be registered in the member app MA determined in the identity verification image registration process is referred to as the identity verification image. Note that the identity verification image may be any of the face image on the face of the My Number card, the face image in the IC chip of the My Number card, and a face image of the user's face.

After the process of step St310, the identity verification app HA executes a process when identity verification is completed (hereinafter referred to as "identity verification completion process") (step St311). When the identity verification app HA is able to register the identity verification image in the process of step St310, the identity verification app HA links the registration information of the member app MA with the identity verification image and registers it in the identity verification server 30 as identity verification completion process. In other words, the identity verification app HA links personal information such as the name and date of birth that the user has registered in advance in the member app MA with the identity verification image and registers it in the identity verification server 30. However, the personal information registered in advance by the user may also be linked with the identity verification image and registered in the identity verification app HA or the member app MA. The identity verification app HA may also display a screen on the display 15 indicating that identity verification has been completed. If the identity verification application HA is unable to register an identity verification image in the process of step St310, the identity verification application HA may display a screen indicating that identity verification is incomplete on the display 15 as identity verification completion processing. Note that the identity verification application HA may notify the user that identity verification is complete or incomplete by outputting a sound from a speaker (not shown).

The identity verification application HA may link the personal information acquired from the My Number card with the identity verification image. This allows the identity verification application HA to generate registration information in which the personal information and the identity verification image are linked, without the user having to input personal information in advance.

Furthermore, as part of the identity verification completion process, the identity verification app HA creates a code that holds information linking the payment information, point information, and identity verification image information. The code is, for example, a one-dimensional code such as a barcode or a two-dimensional code such as a QR code (registered trademark). The code may also be a multi-dimensional code having two or more dimensions. The identity verification app HA registers the created code in the member app MA. The member app MA further links the registration information to the code of the identity verification app HA.

The code may also be created by the member app MA. In this case, the identity verification app HA may obtain a code linked to the registration information of the member app MA from the member app MA, and link the payment information, point information, and identity verification image information to the code. The member app MA may also obtain the payment information, point information, and identity verification image from the identity verification app, and create a code linked to the payment information, point information, identity verification image, and registration information.

The order of the processing of steps St304, St307, and St310 is not limited to the order shown in FIG. 4 and may be any order.

The identity verification app HA may also use personal information acquired from the My Number card to determine whether the name and date of birth registered in the member app MA match. In other words, the identity verification app HA acquires name and date of birth information from the My Number card, determines whether the name and date of birth information entered by the user in the member app MA match the information on the My Number card, and if they match, registers the identity verification image by linking it to the information in the member app MA.

Next, the process of registering an image for personal identification using a My Number card will be described with reference to FIG. 6. FIG. 6 is a flowchart of the process of registering an image for personal identification using a My Number card. Each process in the flowchart of FIG. 6 is executed by the personal identification app HA.

The identity verification application HA judges whether to use the face image stored in the IC chip of the My Number card (hereinafter, the face image stored in the IC chip of the My Number card and the driver's license will be referred to as the "registered image") as the identity verification image to be registered in the member application MA (step St400). The identity verification application HA displays a screen on the display 15 that allows the user to select whether to use the registered image of the My Number card as the identity verification image to be registered in the member application MA. If the identity verification application HA judges that the registered image of the My Number card will be used as the identity verification image to be registered in the member application MA based on the user's selection operation (step St400, YES), it displays a screen on the display 15 that asks the user to input the verification number B of the My Number card. The verification number B is a 14-digit number consisting of a 6-digit date of birth, a 4-digit expiration date, and a 4-digit security code printed on the face of the My Number card. The identity verification application HA receives the verification number B input from the user (step St401). If the face of the My Number card can be read by OCR, the identity verification application HA may automatically obtain the verification number B from the results of OCR reading of the face of the My Number card in step St401.

The identity verification application HA obtains a registration image from the IC chip of the My Number card using the verification number B obtained in the processing of step St401 as a key (step St402).

The identity verification app HA causes the camera 13 to capture an image of the user's face (step St403). By executing the process of step St403 and performing face authentication, it is possible to verify that the operator of the identity verification app HA is the actual person. Note that the process of step St403 may be omitted if the image on the My Number card is permitted to be used as the identity verification image simply by possessing the My Number card. In this case, the processes of steps St403 to St409 are omitted, and the identity verification app HA executes the process of step St410 after the process of step St402.

The identity verification application HA compares the registered image obtained from the My Number card with the photographed image of the user's face obtained in the processing of step St403 using known technology such as face matching technology (step St404).

The identity verification application HA compares the registered image with the captured image to determine whether facial authentication between the face of the person in the registered image and the face of the person in the captured image is successful (step St405).

If the identity verification application HA determines that face authentication has failed (step St405, NO), it displays a result screen indicating that face authentication has failed on the display 15 (step St406).

The identity verification application HA determines whether face authentication between the registered image and the captured image has been performed less than n times (step St407). n is an integer equal to or greater than 1.

If the identity verification app HA determines that face authentication between the registered image and the captured image has been performed less than the nth time (step St407, NO), it executes the process of step St403 again.

When the identity verification app HA determines that the facial authentication between the registered image and the captured image has not been performed less than n times, i.e., that it has been performed n times or more (step St407, YES), the identity verification app HA displays a screen on the display 15 to prompt the user to end the procedure of using the image on the My Number card as the identity verification image (step St408). For example, the identity verification app HA displays a screen on the display 15 presenting an alternative method to prompt the user to end the procedure. An alternative method could be a procedure by mail or a prompt to inquire with the administrator of the identity verification app HA.

The identity verification application HA displays a screen on the display 15 asking the user whether to retake a photograph or to end the procedure. The identity verification application HA determines whether to retake a photograph based on the user's input operation (step St409). If the identity verification application HA determines to retake a photograph based on the user's input operation (step St409, YES), it executes the process of step St403 again.

When the identity verification app HA ends the process, it may link information indicating that the user's identity could not be verified to the registration information of the member app MA, or it may do nothing to the registration information of the member app MA (i.e., the identity verification of the member app MA remains incomplete). When the identity verification app HA ends the process, it may display a screen on the display 15 indicating that the user's identity could not be verified.

If the identity verification app HA determines that face authentication has been successful (step St405, YES), it links the registered image of the My Number card to the registration information of the member app MA as an identity verification image and registers it in the identity verification server 30 (step St410). In addition, the identity verification app HA links information indicating that identity verification has been completed to the registration information of the member app MA and registers it in the identity verification server 30.

When the identity verification application HA determines based on the user's selection operation that the registered image of the My Number card will not be used as the identity verification image to be registered in the member application MA (step St400, NO), it activates the camera 13 and causes the camera 13 to capture the user's face (step St411). For example, the identity verification application HA displays a notification on the display 15 that the face will be captured, and then activates the camera 13 and causes the camera 13 to capture the user's face. Note that since the registered image of the My Number card is a black and white image, there is an advantage in that a new face capture and the captured image, which is a color image, can be used as the identity verification image for face authentication at the self-register 70, as this allows for more accurate face authentication. Note that face authentication at the self-register 70 will be described in FIG. 8.

The identity verification application HA displays a screen on the display 15 requesting the user to input the verification number B of the My Number card. The user inputs the verification number B into the identity verification application HA (step St412).

The identity verification application HA obtains the registration image from the IC chip of the My Number card using the obtained verification number B as a key (step St413).

The identity verification application HA compares the registration image acquired from the My Number card in the processing of step St413 with the photographed image of the user's face acquired in the processing of step St411 using publicly known technology such as face matching technology (step St414).

The identity verification application HA compares the registered image with the captured image to determine whether facial authentication between the face of the person in the registered image and the face of the person in the captured image is successful (step St415).

If the identity verification application HA determines that face authentication has failed (step St415, NO), it displays a result screen indicating that face authentication has failed on the display 15 (step St416).

The identity verification application HA determines whether face authentication between the registered image and the captured image has been performed less than n times (step St417).

If the identity verification app HA determines that face authentication between the registered image and the captured image has been performed less than the nth time (step St417, NO), it executes the process of step St411 again.

If the identity verification app HA determines that the facial authentication between the registered image and the captured image has not been performed less than n times, i.e., that it has been performed n times or more (step St417, YES), it displays a screen on the display 15 prompting the user to end the procedure of using the captured image as an identity verification image (step St418).

The identity verification application HA displays a screen on the display 15 asking the user whether to retake a photograph or end the procedure. The identity verification application HA determines whether to retake a photograph based on the user's input operation (step St419). If the identity verification application HA determines to retake a photograph based on the user's input operation (step St419, YES), it executes the process of step St411 again.

When the identity verification app HA determines not to re-take a photograph based on the user's input operation (step St419, NO), it ends the process. When ending the process, the identity verification app HA may link information indicating that the user's identity could not be verified to the registration information of the member app MA, or may do nothing to the registration information of the member app MA (i.e., the identity verification of the member app MA remains incomplete). When ending the process, the identity verification app HA may display a screen on the display 15 indicating that the user's identity could not be verified.

If the identity verification app HA determines that face authentication has been successful (step St415, YES), it links the captured image to the registration information of the member app MA as an identity verification image and registers it in the identity verification server 30 (step St420). In addition, the identity verification app HA links information indicating that identity verification has been completed to the registration information of the member app MA and registers it in the identity verification server 30.

By comparing the registered image stored in the IC chip of the My Number card with the captured image by the identity verification app HA, for example, if a minor borrows an adult's My Number card and attempts to register it, facial authentication will fail. In this way, the identity verification system 1 prevents a minor user from masquerading as an adult for identity verification. By comparing the registered image with the captured image, the identity verification app HA can link the registered image or captured image of the user to the registration information of the member app MA as an identity verification image, and prevent the linking of an image of the face of another person other than the user to the registration information. In this way, the identity verification app HA can link a highly reliable identity verification result and an identity verification image to the registration information of the member app MA. Furthermore, by performing identity verification, the identity verification app HA can register highly reliable age information in the member app MA. In other words, completion of identity verification is synonymous with completion of age verification.

When the member app MA and the identity verification app HA are separate applications, by linking the member app MA and identity verification app HA provided by each company, the identity verification system 1 can perform highly accurate identity verification of the user without incorporating a new identity verification program into the member app MA.

The member app MA and the identity verification app HA may be a single application. In this case, the user can register as a member and verify his/her identity using a single application, without using two applications. In other words, the identity verification system 1 can reduce the complexity of the user's identity verification process.

In addition, if the terminal device 10 is loaded with My Number card information, the terminal device 10 can verify the identity of the user by linking the My Number card information loaded on the terminal device 10 with the identity verification app HA. However, there is a possibility that a minor may use the terminal device 10 of another person who is an adult (e.g., a parent or the owner of a terminal device obtained illegally) to verify their identity and register the member app MA. Even if the terminal device 10 is loaded with My Number card information, the terminal device 10 can prevent a minor from impersonating another person and registering the member app MA by photographing the user's face when verifying their identity.

Next, an example of the process of identity verification using a driver's license will be described with reference to FIG. 7. FIG. 7 is a flowchart showing an example of the process of identity verification using a driver's license. The process of the flowchart in FIG. 7 is executed by the identity verification app HA. Note that the same reference numerals are used for processes similar to those in FIG. 5, and the description will be omitted.

If the identity verification application HA determines in the process of step St300 in the flowchart of FIG. 5 that the My Number card will not be used (step St300, NO), it displays instructions to the user to read the information on the IC chip of the driver's license on the display 15. Note that the identity verification application HA may start from the process of step St500 if it has been decided in advance that the driver's license will be used as identification.

The identity verification app HA displays a screen on the display 15 requesting the user to input the driver's license PIN 1. The identity verification app HA acquires the PIN 1 in response to the user's input operation. The identity verification app HA displays a screen on the display 15 requesting the user to set the driver's license in a state where it can be read by the terminal device 10. When the user sets the driver's license in a state where the terminal device 10 can read the information on the IC chip, the identity verification app HA reads the information on the IC chip of the driver's license using the PIN 1 as a key (step St500).

If step St307 is NO or after step St309, the identity verification application HA executes identity verification image registration processing using the driver's license (step St505). Identity verification image registration processing using the driver's license is described in FIG. 8.

Next, the process of registering an image for personal identification using a driver's license will be described with reference to FIG. 8. FIG. 8 is a flowchart of the process of registering an image for personal identification using a driver's license. Each process in the flowchart of FIG. 8 is executed by the personal identification application HA.

The identity verification app HA determines whether to use the registered image stored in the IC chip of the driver's license as the identity verification image to be registered in the member app MA (step St600). The identity verification app HA displays on the display 15 a screen that allows the user to select whether to use the registered image of the driver's license as the identity verification image to be registered in the member app MA. If the identity verification app HA determines based on the user's selection operation that the registered image of the driver's license will be used as the identity verification image to be registered in the member app MA (step St600, YES), it displays on the display 15 a screen that requests the user to enter the PIN numbers 1 and 2 of the driver's license. The identity verification app HA acquires the PIN numbers 1 and 2 based on the user's input operation (step St601). The identity verification app HA may display on the display 15 a screen that requests the user to enter the PIN number 2 and acquire only the PIN number 2.

The identity verification application HA determines whether or not the registration image has been obtained from the IC chip of the driver's license using the PIN numbers 1 and 2 as keys (step St602). The identity verification application HA determines whether or not the processing of step St602 is performed based on whether or not the registration image has been obtained from the IC chip of the driver's license.

If the identity verification application HA is unable to obtain the registration image from the IC chip of the driver's license using PIN numbers 1 and 2 as keys (i.e., PIN number 1 and/or PIN number 2 are incorrect) (step St602, NO), it executes the process of step St600 again.

If the identity verification application HA is able to obtain a registration image from the IC chip of the driver's license using the PIN numbers 1 and 2 as keys (step St602, YES), it causes the camera 13 to capture a picture of the user's face (step St603).

The identity verification application HA compares the registered image obtained from the driver's license with the photographed image of the user's face obtained in the processing of step St603 using publicly known technology such as face matching technology (step St604).

The identity verification application HA compares the registered image with the captured image to determine whether facial authentication between the face of the person in the registered image and the face of the person in the captured image is successful (step St605).

If the identity verification application HA determines that face authentication has failed (step St605, NO), it determines whether face authentication between the registered image and the captured image has been performed less than n times (step St606).

If the identity verification app HA determines that face authentication between the registered image and the captured image has been performed less than n times (step St606, NO), it executes the process of step St600 again.

If the identity verification app HA determines that the face authentication between the registration image and the captured image has not been performed less than the nth time, i.e., that it has been performed at least the nth time (step St606, YES), it does not allow the registration of the registration image (step St607). If the identity verification app HA determines that it is not possible to register the registration image, it may link information indicating that the user's identity could not be verified to the registration information of the member app MA, or it may do nothing to the registration information of the member app MA (i.e., the identity verification of the member app MA remains incomplete).

If the identity verification application HA determines that face authentication has been successful (step St605, YES), it registers the registered image of the driver's license as an identity verification image by linking it to the registration information of the member application MA (step St608). In addition, the identity verification application HA links information indicating that identity verification has been completed to the registration information of the member application MA.

The processing of steps St603 to St607 may be omitted. In other words, when using an image of a driver's license as an image for personal identification, the image of the driver's license may be used as the image for personal identification without performing face recognition. In this case, the personal identification application HA executes the processing of step St608 after the processing of step St602, YES.

In addition, if an image of the face of the driver's license is used instead of an image of the IC chip, when face authentication is successful in the processes of steps St603 to St605, the image captured in the process of step St603 or the image of the face of the license may be used as the image for identity verification.

If the identity verification application HA determines based on the user's selection operation that the registered image of the driver's license will not be used as the identity verification image to be registered in the member application MA (step St600, NO), it activates the camera 13 and causes the camera 13 to capture the user's face (step St609). For example, the identity verification application HA displays a notification on the display 15 that the face will be captured, and then activates the camera 13 and causes the camera 13 to capture the user's face.

The identity verification application HA displays a screen on the display 15 requesting the user to input the driver's license PIN numbers 1 and 2. The identity verification application HA acquires the PIN numbers 1 and 2 based on the user's input operation (step St610). The identity verification application HA may display a screen on the display 15 requesting the user to input the PIN number 2 and acquire only the PIN number 2.

The identity verification application HA determines whether or not the registration image has been obtained from the IC chip of the driver's license using the PIN numbers 1 and 2 as keys (step St611). The identity verification application HA determines whether or not the processing of step St611 is performed based on whether or not the registration image has been obtained from the IC chip of the driver's license.

If the identity verification application HA is unable to obtain the registered image from the IC chip of the driver's license using the PINs 1 and 2 as keys (i.e., the PINs 1 and 2 are incorrect) (step St611, NO), it executes the process of step St600 again.

If the identity verification application HA is able to obtain the registered image from the IC chip of the driver's license using the PIN numbers 1 and 2 as keys (step St611, YES), it compares the captured image obtained in the processing of step St609 with the registered image obtained in the processing of step St611 using publicly known technology such as face matching technology (step St612).

The identity verification application HA compares the registered image with the captured image to determine whether facial authentication between the face of the person in the registered image and the face of the person in the captured image is successful (step St613).

If the identity verification app HA determines that face authentication has failed (step St613, NO), it determines whether face authentication between the registered image and the captured image has been performed less than n times (step St614).

If the identity verification app HA determines that face authentication between the registered image and the captured image has been performed less than n times (step St614, NO), it executes the process of step St600 again.

If the identity verification app HA determines that the facial authentication between the registered image and the captured image has not been performed less than the nth time, i.e., that it has been performed at least the nth time (step St614, YES), it does not allow the captured image to be registered (step St615). If it is not possible to register the captured image, the identity verification app HA may link information indicating that the user's identity could not be verified to the registration information of the member app MA, or may not do anything to the registration information of the member app MA (i.e., it leaves the identity verification of the member app MA incomplete).

If the identity verification application HA determines that face authentication has been successful (step St613, YES), it registers the captured image as an identity verification image by linking it to the registration information of the member application MA (step St616). In addition, the identity verification application HA links information indicating that identity verification has been completed to the registration information of the member application MA.

Next, referring to FIG. 9, an example screen for an item for which registration has been completed in the member app MA will be described. FIG. 9 is a diagram showing an example screen for an item for which registration has been completed in the member app MA.

Screen MN is an example of a screen displayed on the display 15 of the terminal device 10. Screen MN shows the registration information items of the member app MA.

Screen MN includes the items "Register image for identity verification", "Register points", "My Number card", "Driver's license", "Electronic money linkage", and "Credit card linkage". Note that the items of registration information for the member app MA displayed on the display 15 are not limited to the above-mentioned examples. "Government (official identification card)" indicates the identification card used to register the user's facial image (i.e., to verify the user's identity).

The screen MN has marks CH1, CH2, CH3, CH4, CH5, and CH6 next to each item to indicate whether or not registration has been completed. In the example shown in FIG. 8, for items for which registration has been completed, marks with a circle located on the right, such as marks CH1, CH2, and CH3, are displayed, and for items for which registration has not been completed, marks with a circle located on the left, such as marks CH4, CH5, and CH6, are displayed. Note that the design of the mark is only an example, and a mark of any design, such as a check mark, may be displayed for items for which registration has been completed. In addition, whether or not registration has been completed may be indicated by emphasizing the text of each item (making the text bold, making the text darker, or filling in the background of the text, etc.) instead of a mark. In the example shown in FIG. 8, it is indicated that the registration of the identification image and the registration of points have been completed. It is also indicated that the My Number card was used to register the identification image. In addition, it is also indicated that neither electronic money nor credit card is linked with regard to payment information.

In addition, marks CH1, CH2, CH3, CH4, CH5, and CH6 may be buttons that accept touch operations by the user, and when the user touches a mark, the registration process for the corresponding item may be started. For example, when the user touches mark CH1, the registration process for the identity verification image is started.

Next, an example of the process of permitting the purchase of licensed items by the self-checkout register 70 will be described with reference to FIG. 10. FIG. 10 is a flowchart of the process of permitting the purchase of licensed items by the self-checkout register 70. Each process in the flowchart of FIG. 10 is executed by the self-checkout register 70.

The self-checkout register 70 receives information about pre-purchase processing based on input operations from a user who is a customer purchasing an item at the store (step St700). Information about pre-purchase processing is, for example, information about whether or not to purchase a shopping bag. Note that the processing of step St700 may be omitted.

The self-checkout register 70 displays a screen on the display 75 that allows the user to select whether or not they have registered with the member app MA. The self-checkout register 70 determines whether or not the user has registered with the member app MA based on the user's input operation (step St701).

If the self-checkout register 70 determines that the user is not registered in the member app MA based on the user's input operation (step St701, NO), the user has the self-checkout register 70 read the licensed item to receive information about the licensed item (step St703).

The self-register 70 displays instructions for reading the user's identification on the display 75. An example of an instruction for reading the identification is "Please hold your identification over the reader." The self-register 70 reads the information on the IC chip of the My Number card or driver's license held over the reader 72 by the user, and acquires the personal information such as the date of birth and the registered image stored in the IC chip (step St704). At this time, when acquiring the information on the IC chip of the driver's license, it is necessary to input the set PIN. Therefore, for convenience, the self-register 70 may acquire the personal information such as the date of birth and the registered image from the image of the face of the My Number card or driver's license instead of reading the information on the IC chip.

The self-checkout register 70 determines whether the date of birth obtained in the processing of step St704 is a date of birth indicating an adult (step St705).

If the self-checkout register 70 determines that the date of birth is not a date of birth that indicates an adult (i.e., a date of birth that indicates the age of a minor) (step St705, NO), it disallows the purchase of the licensed product as a NG process (step St706). As a NG process, the self-checkout register 70 displays, for example, a screen on the display 75 indicating that the purchase of the licensed product is not permitted.

If the self-checkout register 70 determines that the date of birth indicates that the user is an adult (step St705, YES), it activates the camera 73 and captures an image of the user's face (step St707). The self-checkout register 70 compares the image of the user's face captured in the process of step St707 with the registered image acquired in the process of step St704, and performs face authentication using known technology such as face matching technology (step St708).

The self-checkout register 70 verifies the identity of the user by determining whether or not facial authentication between the captured image and the registered image is successful in the processing of step St708, and determines whether or not the user is eligible to purchase licensed products (i.e., whether or not the user is an adult) (step St709).

If the self-checkout register 70 determines that face authentication between the captured image and the registered image has failed and determines that the user is not eligible to purchase the licensed product (step St709, NO), it disallows the purchase of the licensed product as a NG process (step St723). As a NG process, the self-checkout register 70 displays, for example, a screen on the display 75 indicating that the purchase of the licensed product is not permitted.

If the self-register 70 determines that facial authentication between the captured image and the registered image has been successful and determines that the user is eligible to purchase the licensed product (step St709, YES), it displays a screen on the display 75 that allows the user to select a payment method. The self-register 70 makes the payment using the payment method selected by the user (step St710). Note that the self-register 70 may request the user to input information about the points they wish to link (e.g., registration information about the points) before making the payment.

If the self-register 70 receives input regarding the points information to be linked from the user before processing in step St710, the self-register 70 executes the points granting process (step St711). For example, as part of the points granting process, the self-register 70 sends the points information and purchase information (e.g., purchase date and time and purchase amount) to the point company server 60. The point company server 60 grants points to the corresponding registered information based on the points information and purchase information received from the self-register 70. Note that if the self-register 70 does not receive input regarding the points information to be linked from the user, the self-register 70 omits the process in step St711.

When the self-checkout register 70 determines that the user has registered in the member app based on the user's input operation (step St701, YES), it displays a screen on the display 75 that includes instructions to have the reader 72 read a code linked to the registration information in the member app MA. For example, based on the instructions displayed on the display 75, the user holds the code displayed on the display 15 of the terminal device 10 over the reader 72. The self-checkout register 70 reads the code using the reader 72 (step St7021).

The self-register 70 determines whether an identity verification image has been registered in the member app MA (step St7022). Specifically, the self-register 70 determines whether an identity verification image is linked to the registration information of the member app MA from the code read in the processing of step St7021. As a result, the self-register 70 determines whether identity verification of the user (i.e., age verification) has been completed.

If the self-register 70 determines that the identity verification image has not been registered in the member app MA (i.e., unregistered) (step St7022, NO), the user has the self-register 70 scan the licensed item to receive information about the licensed item (step St712).

Because the user has not registered an image for identity verification in the member app MA and identity verification has not been completed in advance, the self-register 70 must obtain information from the user's identification card and verify the user's age. The self-register 70 displays instructions for reading the user's identification card on the display 75. An example of an instruction for reading the identification card is an instruction such as "Please hold your identification card over the reader." The self-register 70 reads the information on the IC chip of the My Number card or driver's license held over the reader 72 by the user and obtains the personal information such as date of birth and the registered image stored in the IC chip (step St713). At this time, when obtaining the information on the IC chip of the driver's license, it is necessary to input the set PIN. Therefore, for convenience, the self-register 70 may obtain the personal information such as date of birth and the registered information from the card face image of the My Number card or driver's license instead of reading the information on the IC chip.

The self-checkout register 70 determines whether the date of birth obtained in the processing of step St713 is a date of birth indicating an adult (step St714).

If the self-checkout register 70 determines that the date of birth is not a date of birth that indicates an adult (i.e., a date of birth that indicates the age of a minor) (step St714, NO), it disallows the purchase of the licensed product as a NG process (step St706). As a NG process, the self-checkout register 70 displays, for example, a screen on the display 75 indicating that the purchase of the licensed product is not permitted.

If the self-checkout register 70 determines that the date of birth indicates an adult (step St714, YES), it activates the camera 73 and captures an image of the user's face (step St715). The self-checkout register 70 compares the image of the user's face captured in the process of step St715 with the registered image acquired in the process of step St716, and performs face authentication using known technology such as face matching technology (step St716).

The self-checkout register 70 verifies the user's identity by determining whether facial authentication between the captured image and the registered image is successful, and determines whether the user is eligible to purchase licensed products (i.e., whether the user is an adult) (step St717).

If the self-checkout register 70 determines that face authentication between the captured image and the registered image has failed and determines that the user is not eligible to purchase the licensed product (step St717, NO), it disallows the purchase of the licensed product as a NG process (step St723). As a NG process, the self-checkout register 70 displays, for example, a screen on the display 75 indicating that the purchase of the licensed product is not permitted.

If the self-checkout register 70 determines that facial authentication between the captured image and the registered image is successful and determines that the user is eligible to purchase the licensed product (step St717, YES), it determines whether the payment information has been registered in the member app (step St718).

If the self-checkout 70 determines that the payment information has not been registered in the member app MA by referencing the information linked to the member app MA (step St718, NO), it displays a screen on the display 75 that allows the user to select a payment method. The self-checkout 70 makes the payment using the payment method selected by the user (step St710).

The self-register 70 references the information linked to the member app MA to determine whether point information has been registered in the member app MA, and if it determines that it has been registered, executes a point granting process (step St711). As part of the point granting process, the self-register 70 sends purchase information (e.g., information on the purchase date and time and the purchase amount) to the terminal device 10. The terminal device 10 sends the received purchase information and the point information linked to the member app MA to the point company server 60. The point company server 60 grants points to the corresponding registered information based on the point information and purchase information received from the terminal device 10.

If the self-checkout register 70 determines that the payment information has been registered in the member app MA by referencing the information linked to the member app MA (step St718, YES), the self-checkout register 70 performs payment based on the payment information linked to the registration information of the member app MA (step St719). After processing step St719, the self-checkout register 70 executes processing step St711.

If the self-checkout register 70 determines that an identity verification image has been registered in the member app MA (step St7022, YES), it obtains the identity verification image from the member app MA.

The self-checkout register 70 receives information about the licensed item when the user has the self-checkout register 70 scan the licensed item (step St720).

Because the user has already registered an identity verification image in the member app MA and identity verification has been completed, the self-register 70 does not need to verify the user's age using an ID card. However, there is a possibility that a minor may purchase licensed goods at the self-register 70 using another person's terminal device 10 that has the member app MA installed and for which identity verification has been completed. Thus, to prevent minors from purchasing licensed goods at the self-register 70 using another person's terminal device 10, the self-register 70 photographs the face of the user who is the purchaser, performs facial authentication, and determines whether the image matches the identity verification image in the member app MA.

The self-checkout register 70 activates the camera 73 and captures a picture of the user's face (step St721). The self-checkout register 70 compares the captured image of the user's face with the identity verification image registered in the member app MA, and performs face authentication using known technology such as face matching technology (step St722).

The self-checkout register 70 verifies the identity of the user by determining whether facial authentication between the captured image and the identity verification image is successful, and determines whether the user is eligible to purchase licensed products (i.e., whether the user is an adult) (step St717).

If the self-checkout register 70 determines that the facial authentication between the captured image acquired in the process of step St721 and the image for personal identification has failed and determines that the user is not eligible to purchase a licensed product (step St717, NO), it denies the purchase of the licensed product as a NG process (step St723). As a NG process, the self-checkout register 70 displays, for example, a screen on the display 75 indicating that the purchase of the licensed product is not permitted.

If the self-checkout register 70 determines that the facial authentication between the captured image and the identity verification image acquired in the processing of step St721 was successful and determines that the user is eligible to purchase a licensed product (step St717, YES), it determines whether the payment information has been registered in the member app (step St718). The processing after step St718, that is, steps St710, St711, and St719, has been explained once, so explanation is omitted here.

If the self-checkout register 70 does not receive information about a licensed product (i.e., if the user does not purchase a licensed product), it executes processing of step St718 after processing of step St701.

When the self-checkout register 70 receives information about a licensed item, it may also determine whether the user has registered with the member app MA (i.e., process step St701).

The identity verification system 1 can register a user's personal information, an identity verification image, payment information, and point information by linking them to one code. The identity verification system 1 can verify the user's age, award points, and make payments all with that one code. This allows the identity verification system 1 to eliminate the need for the user to present various pieces of information and input payment information when purchasing licensed products, thereby increasing convenience.

As described above, the identity verification system 1 photographs the face of a user who purchases licensed products at the self-checkout register 70 and performs facial authentication by comparing the photographed image with the registered image on the identification card, thereby preventing minors from masquerading as adults to purchase licensed products.

Even when the self-register 70 completes identity verification and uses the member app MA linked to the identity verification image to determine whether to permit the purchase of a licensed product, the self-register 70 compares the captured image with the identity verification image and performs facial authentication. This allows the identity verification system 1 to prevent a minor from using an adult's terminal device 10 to pose as an adult and purchase a licensed product at the self-register 70. For example, if a minor attempts to purchase a licensed product at the self-register 70 using an adult's terminal device 10, facial authentication between the captured image and the identity verification image will fail, and the minor will not be able to purchase the licensed product.

<Use cases for hotels, rental cars, and gas stations>
11A is a diagram for explaining a use case in which a hotel member application MA is used. In the following description, the hotel member application MA is referred to as a hotel application.

The user operates the terminal device 10 and registers the user's personal information in the hotel app using the method shown in FIG. 4 (St801). The personal information registered in the hotel app is referred to as registration information.

The user operates the terminal device 10 and uses the identity verification application HA to verify the identity of the user using the method shown in Figures 5 and 6 (or the method shown in Figures 7 and 8) (Step 802).

If the identity verification is successful in step St802, the terminal device 10 links the hotel app and the identity verification app HA and registers them in the identity verification server 30 (St803). As a result, the registration information of the hotel app and the identity verification image of the identity verification app HA are linked and registered in the identity verification server 30.

The user makes a hotel reservation through the hotel app. The hotel app generates a code (e.g., a barcode or two-dimensional code) corresponding to the registration information and information related to the reservation (Step 804).

When the user checks in to a hotel, the code (e.g., a barcode or two-dimensional code) generated by the hotel app is displayed on the terminal device 10 and read by the hotel's self-checkout register 70 (St805). Reading this code corresponds to step St7021 in FIG. 10. Note that the user may display the code on the identity verification app HA instead of the hotel app and have the self-checkout register 70 read it.

The self-checkout 70 inquires of the identity verification server 30 as to whether the registration information corresponding to the read code is linked to the identity verification image, and if so, obtains the registration information and the identity verification image from the identity verification server 30 and performs identity verification (St806). To verify the identity of the user, the self-checkout 70 may photograph the user's face, compare the photographed image with the identity verification image, and determine that the user is the person in question if the comparison is successful. The comparison between the photographed image and the identity verification image may be performed by the identity verification server 30.

If the identity verification in step St806 is successful, the self-register 70 performs check-in processing. This allows the self-register 70 to perform reservation confirmation, identity verification, membership authentication, bonus allocation, payment processing, point allocation, etc., for the user associated with the code, all at once.

The user stays at the hotel (Step 807). Then, when checking out of the hotel, the user has the hotel app display the code and has the self-checkout register 70 read it (Step 808). This allows the self-checkout register 70 to perform identity verification, membership authentication, rewards, payment processing, points allocation, etc. of the user associated with this code all at once.

Figure 11B is a diagram for explaining a use case in which the rental car member app MA is used. In the following description, the rental car member app MA is referred to as the rental car app.

The user operates the terminal device 10 to register the user's personal information in the rental car app using the method shown in FIG. 4 (St811). The personal information registered in the rental car app is referred to as registration information.

The user operates the terminal device 10 and uses the identity verification application HA to verify the identity of the user using the driver's license in the manner shown in Figures 7 and 8 (Step 812).

If the identity verification is successful in step St812, the terminal device 10 links the rental car app and the identity verification app HA and registers them in the identity verification server 30 (St813). As a result, the registration information of the rental car app and the identity verification image of the identity verification app HA are linked and registered in the identity verification server 30.

The user makes a reservation for a rental car through the rental car app. The rental car app generates a code corresponding to the registration information and the information related to the rental car reservation (Step 814).

When the user rents a car, the rental car app displays the code generated above on the terminal device 10 and the self-checkout register 70 at the rental car store reads it (St815). Reading this code corresponds to step St7021 in FIG. 10. Note that the user may display the code on the identity verification app HA instead of the rental car app and have the self-checkout register 70 read it.

The self-checkout 70 inquires of the identity verification server 30 as to whether the registration information corresponding to the read code is linked to the identity verification image, and if so, obtains the registration information and the identity verification image from the identity verification server 30 and performs identity verification (St816). To verify the identity of the user, the self-checkout 70 may photograph the user's face, compare the photographed image with the identity verification image, and determine that the user is the person in question if the comparison is successful. The comparison between the photographed image and the identity verification image may be performed by the identity verification server 30.

If the self-register 70 is successful in the identity verification in step St815, it performs the rental car rental process. This allows the self-register 70 to perform reservation confirmation, identity verification, membership authentication, bonus allocation, payment processing, point allocation, etc. for the user associated with the code all at once. The code read by the self-register 70 in step St815 may also be used as an electronic key for the rental car.

The user rents a car (Step 817). Then, when returning the rental car, the user displays the code on the rental car app and has the self-registered cash register 70 read it (Step 818). This allows the self-registered cash register 70 to perform identity verification, membership authentication, rewards, payment processing, points allocation, etc. of the user associated with this code all at once.

Figure 11C is a diagram for explaining a use case in which the gas station member app MA is used. In the following description, the gas station member app MA is referred to as the gas app.

The user operates the terminal device 10 to register the user's personal information in the gasoline app using the method shown in FIG. 4 (St821). The personal information registered in the gasoline app is referred to as registration information.

The user operates the terminal device 10 and uses the identity verification application HA to verify the identity of the user using the driver's license or My Number card in the manner shown in Figures 7 and 8 (or Figures 5 and 8) (St822).

If the identity verification is successful in step St822, the terminal device 10 links the gasoline app and the identity verification app HA and registers them in the identity verification server 30 (St823). As a result, the registration information of the gasoline app and the identity verification image of the identity verification app HA are linked and registered in the identity verification server 30.

When the user uses a service at a gas station (e.g., refueling or washing the car), the user causes the terminal device 10 to display a code corresponding to the registration information of the gasoline app, and causes the self-checkout register 70 at the gas station to read it (St824). Reading this code corresponds to step St7021 in FIG. 10. Note that the user may also display the code on the display of the identity verification app HA instead of the gasoline app, and cause the self-checkout register 70 to read it.

The self-checkout 70 inquires of the identity verification server 30 as to whether the registration information corresponding to the read code is linked to the identity verification image, and if so, obtains the registration information and the identity verification image from the identity verification server 30 and performs identity verification (St825). To verify the identity of the user, the self-checkout 70 may photograph the user's face, compare the photographed image with the identity verification image, and determine that the user is the person in question if the comparison is successful. The comparison between the photographed image and the identity verification image may be performed by the identity verification server 30.

If the self-checkout 70 is successful in the identity verification in step St825, it provides the service (e.g., starts refueling, starts the car wash, etc.). This allows the self-checkout 70 to select the type and amount of refueling for the user associated with this code, select the car wash items, verify the user's identity, authenticate the member, grant benefits, process payments, grant points, etc., all at once.

The user uses the gas station's services (Step 826).

Note that an identity verification image once registered in the identity verification server 30 from a member app MA of a certain service (e.g., a hotel app) may be usable for identity verification in a member app MA of another service (e.g., a rental car app or a gasoline app) with the user's consent. This allows a user to omit the registration of an identity verification image from the next time onwards, once an identity verification image has been registered.

<Use cases for government services>
12 is a diagram for explaining a use case in which a member application MA of a government service is used. Hereinafter, the member application MA of a government service is referred to as a citizen application.

The user operates the terminal device 10 and registers the user's personal information in the citizen app using the method shown in FIG. 4 (Step 901). The personal information registered in the citizen app is referred to as registration information.

The user operates the terminal device 10 and uses the identity verification app HA to verify his/her identity using the method shown in Figures 5 and 6 (or the method shown in Figures 7 and 8) (St902). Here, identity verification includes not only verifying whether the user is the person in question, but also verifying whether the user is a resident of the administrative district (e.g., city, ward, town, or village) in which the citizen app is provided.

If the identity verification is successful in step St902, the terminal device 10 links the citizen app and the identity verification app HA and registers them in the identity verification server 30. As a result, the registration information of the citizen app and the identity verification image of the identity verification app HA are linked and registered in the identity verification server 30 (St903).

When entering a facility managed by the government (e.g., a museum, science museum, zoo, etc.), the user displays a code corresponding to the registration information of the citizen app and has the self-checkout register 70 at the facility read it (St904). Reading this code corresponds to step St7021 in FIG. 10. Note that the user may display the code on the identity verification app HA instead of the citizen app and have the self-checkout register 70 read it.

The self-checkout 70 inquires of the identity verification server 30 as to whether the registration information corresponding to the read code is linked to the identity verification image, and if so, obtains the registration information and the identity verification image from the identity verification server 30 and performs identity verification (St906). To verify the identity of the user, the self-checkout 70 photographs the user's face, compares the photographed image with the identity verification image, and if the comparison is successful, determines that the user is the real person. The comparison of the photographed image with the identity verification image may be performed by the identity verification server 30.

If the self-register 70 is successful in the identity verification in step St906 (for example, if it is verified that the user is a resident of the administrative district of the facility), it presents free admission to the terminal device 10 (St907). The user enters the facility free of charge (St909). Note that free admission is just one example, and it may be possible to offer a discount on the admission fee or to provide a special benefit. Furthermore, the self-register 70 may also verify the user's age, etc., when verifying the user's identity, and may offer a discount or a special benefit according to the user's age.

If the self-register 70 fails to verify the identity of the user in step St906, the self-register 70 presents the entrance fee to the terminal device 10, and the terminal device 10 pays the entrance fee to the self-register 70 (St908). The self-register 70 then displays permission to enter. At this time, the self-register 70 may open the entrance gate to allow the user to enter (St909). This allows the user to enter the facility.

By using the above process, users can easily use the citizen app to receive services from the administrative district in which they live. In addition, the above process allows the administrative authorities providing the services to perform strict and speedy identity verification and easily collect user information, reducing the workload of employees.

<Financial services use cases>
FIG. 13 is a diagram for explaining a use case in which an application for a financial service is made using an identity verification application.

The terminal device 10 acquires the identity verification app provided by the financial server (Steps 921 and 922).

The identity verification app HA of the terminal device 10 authenticates the user's My Number card. For example, the identity verification app HA sends the electronic certificate of the IC chip of the My Number card to the financial server (St923). The financial server queries the identity verification authentication business operator server 20 for the electronic certificate of the My Number card (St924), and if the query is successful, links the query result to the identity verification app HA (St925). This enables the financial server to verify the validity of the electronic certificate of the My Number card, and to trust that information exchanged with the terminal device 10 thereafter belongs to the user.

The identity verification application HA of the terminal device 10 transmits the identity verification image of the user to the financial server (St926). The financial server registers the received identity verification image (St927). The identity verification image may be an image of the user's face, a facial photograph image printed on the face of the identity verification document, or facial image data stored in the IC chip of the My Number card.

The user operates the terminal device 10 to apply for financial services (St928), and the financial server accepts the application (St929).

The terminal device 10 captures a picture of the user's face, generates a captured image, and transmits it to the financial server (Step 930).

The financial server receives the captured image in step St930, compares the captured image with the identity verification image, and performs identity verification (St931). Note that in this process, the financial server performs identity verification, but the financial server may request identity verification from the above-mentioned identity verification server 30, and the identity verification server 30 may perform identity verification.

If identity verification is successful in step St931, the financial server executes the application process (St932).

The financial server transmits the application result to the terminal device 10 (St933). If the identity verification and application processing are successful, the financial server transmits information indicating the application success and the application contents as the application result to the terminal device 10. If the identity verification fails or the application processing fails, the financial server transmits information indicating the application failure as the application result to the terminal device 10.

The terminal device 10 receives the application result from the financial server and displays the application result (success or failure, etc.) on the display 15 (St934).

By performing the above process, the user can apply for financial services using the terminal device 10 without visiting the financial company's counter. In addition, the financial company can verify that the user is the person in question, making it possible to apply for financial services using the terminal device 10.

<Use case of credit card payment app>
14 is a diagram for explaining a use case when registering a credit card payment application. The credit card payment application may correspond to the above-mentioned member application MA. In the following, the credit card payment application will be described as including the function of the above-mentioned identity verification application HA, but the credit card payment application and the identity verification application HA may be separate applications.

The terminal device 10 obtains the credit card payment app provided by the credit card company from the payment company server 50 and installs it (Steps 941 and 942).

The user starts the credit card payment app of the terminal device 10 (St943). Then, the user inputs his/her credit card number and expiration date into the credit card payment app (St944). Note that this input may be manual input by the user, or may be automatic input by capturing an image of the credit card with the camera 13 of the terminal device 10.

The user inputs the PIN of the My Number card into the credit card payment app of the terminal device 10 (Step 945). Alternatively, the credit card payment app captures an image of the My Number card using the camera 13 of the terminal device 10 and performs OCR to obtain the My Number card verification number B.

The credit card payment application of the terminal device 10 uses the PIN or verification number B of step St945 to obtain personal information, facial image data, etc. from the IC chip of the My Number card (St946).

The credit card payment application of the terminal device 10 captures a picture of the user's face and generates a captured image (Step 947).

The credit card payment application of the terminal device 10 performs a one-to-one comparison between the facial image data acquired in step St946 and the image of the user's face captured in step St947, and performs identity verification (St948).

The credit card payment application of the terminal device 10 transmits the identity verification result of step St948 to the payment company server 50 (St949). The following describes the case where identity verification is successful. Note that if identity verification fails, the subsequent processes do not need to be executed.

The credit card payment application of the terminal device 10 requests the identity verification server 30 to register an identity verification image (St950). The identity verification image may be face image data obtained from the IC chip of the My Number card in step S946, a face image printed on the face of the My Number card, or a photographed image of the user's face taken in step S947.

The identity verification server 30 links the credit card payment application with the identity verification image and registers it in a specified database (Step 951).

The credit card payment application of the terminal device 10 requests the payment company server 50 to register the user's personal information and confidential information (St952). Here, the personal information may be information acquired in step St946. The confidential information may be, for example, confidential information known only to the user and input by the user.

The payment company server 50 links the personal information and confidential information from step St952 with the credit card number and expiration date, and stores them in a specified database (St953).

The payment company server 50 issues a new user ID to the user and sends the user ID to the credit card payment application of the terminal device 10 (Step 954).

The credit card payment application on the terminal device 10 displays the user ID and registration success in step St954 (St955) and ends this process.

By performing the above process, the credit card company can securely link the user's credit card to the credit card payment app using the user's My Number card. In addition, the user can easily obtain a credit card payment app linked to their credit card by using their My Number card without having to manually input personal information, etc.

Figure 15 is a diagram to explain a use case when using a credit card payment app.

The user selects a product, for example, in a web browser or app on the terminal device 10, and presses the purchase button on the purchase screen (St961). The user then presses the payment button on the web browser or app (St962), and selects a payment method (St963). Here, the user selects credit card payment as the payment method (St964).

The terminal device 10 launches the credit card payment application (Step 965).

The credit card payment application on the terminal device 10 automatically inputs payment information (e.g., payment amount, credit card number, expiration date, user name, etc.) (Step 966).

The credit card payment application of the terminal device 10 captures a picture of the user's face with the camera 13 and generates a captured image (Step 967).

The credit card payment application of the terminal device 10 sends a facial recognition request to the identity verification server 30 (St968). The facial recognition request includes the captured image generated in step St967.

The identity verification server 30 receives the face authentication request in step St968 and performs one-to-N face matching processing (St969). For example, as one-to-N face matching processing, the identity verification server 30 searches the database for an identity verification image having a face matching score with the photographed image that is equal to or higher than a predetermined threshold. The face matching score is a score indicating the likelihood that the two face images are of the same person. If the search fails, that is, if the identity verification image presumed to be the same person as the photographed image does not exist in the database, the identity verification server 30 transmits a face authentication result including face authentication failure and a matching score to the terminal device 10. If the search is successful, that is, if the identity verification image presumed to be the same person as the photographed image exists in the database, the identity verification server 30 transmits a face authentication result including face authentication success, a matching score, and the date and time when the identity verification image was registered (hereinafter referred to as the registration date and time) to the terminal device 10.

The credit card payment app of the terminal device 10 determines whether or not the face authentication was successful based on the face authentication result of step St969 (St970). If it is determined that the face authentication was successful, the credit card payment app of the terminal device 10 requests payment from the payment company server 50 (St971). Note that if it is determined that the face authentication was unsuccessful, the credit card payment app of the terminal device 10 may display a message to that effect and end the process.

The payment company server 50 receives the payment request in step St971 and performs payment processing using the credit card number linked to the user (St972). The payment company server 50 transmits a completion of the payment processing to the terminal device 10.

The credit card payment app of the terminal device 10 displays the completion of the payment process of step St972 (St973). At this time, the credit card payment app may also display information regarding point allocation, bonus allocation, etc., as necessary. Then, this process ends.

By performing the above process, the credit card company can prevent fraudulent use of credit cards by performing facial authentication of the user through the credit card payment app and performing payment processing if facial authentication is successful. In addition, by making payments through the credit card payment app, the user can easily make payments without having to enter credit card numbers, etc.

<Identity verification via in-store terminal>
FIG. 16 is a block diagram showing a modified example of the personal identification system 1. In FIG.

As shown in FIG. 16, the identity verification system 1 according to the modified example includes a store terminal 80, an identity verification server 30, a face matching DB 31, a payment company server 50, and an identity verification authentication business operator server 20. The identity verification system 1 may also include a member company server 40 and/or a point company server 60 shown in FIG. 2.

The face matching DB31 manages face images used for face matching of store users as a database. The identity verification server 30 uses the face matching DB31 to perform face matching of users.

The store terminal 80 includes a processor 81, a memory 82, a communication I/F 83, an IC reader 84, a code reader 85, a camera 86, and a display 87. The processor 81, the memory 82, the communication I/F 83, and the display 87 may be similar to the processor 71, the memory 76, the communication I/F 74, and the display 75 of the self-checkout register 70 described above. The IC reader 84 is a device that reads information stored in the IC chip of the identification document. The code reader 85 is a device that reads bar codes and two-dimensional codes. The camera 86 captures a picture of the user's face and generates a captured image.

The store terminal 80 cooperates with the identity verification server 30 to verify the identity and age of the user based on a photographed image of the user's face and their identity verification document. The store terminal 80 also cooperates with the payment company server 50 to perform payment processing for licensed products and the like purchased by the user. Details of the processing performed by the store terminal 80 will be explained later as appropriate.

Figure 17 is a flowchart showing an example process for registering an identity verification image in the identity verification server 30 and reusing it.

The store terminal 80 uses the code reader 85 to read the barcode (or two-dimensional code) of the licensed item (Step 1001).

The store terminal 80 displays the payment amount for the licensed item read in step St1001 on the display 87 (St1002).

The store terminal 80 determines whether the identity verification image of the user who purchases the licensed product has been registered in the identity verification server 30 (St1003). For example, the store terminal 80 queries the user as to whether the identity verification image has been registered in the past, and makes the determination based on the user's response.

When the store terminal 80 determines that the user's identity verification image has already been registered in the identity verification server 30 (for example, when the user responds to the inquiry that the image has already been registered) (St1003, YES), it performs a registration process (St1004). Details of the registration process will be described later (see FIG. 18).

If the store terminal 80 determines that the user's identity verification image has not been registered with the identity verification server 30 (for example, if the user responds to the inquiry that it has not been registered) (St1003, NO), it proceeds to the next step, St1005.

The store terminal 80 requests the user to submit an identity verification document (Step 1005). In response, the user provides the store terminal 80 with an identity verification document that the user possesses.

The store terminal 80 uses the IC reader 84 to acquire face image data, age information, etc. from the IC chip of the identification document provided by the user, and temporarily stores the data in the memory 82 (St1006). The store terminal 80 may also photograph the face of the identification document with a designated camera and acquire the face photo printed on the face of the document as face image data. The store terminal 80 may also photograph the face of the identification document with a designated camera, perform OCR, and determine age information from the date of birth printed on the face of the document.

The store terminal 80 determines from the age information acquired in step St1006 whether the user is old enough to purchase the licensed product read in step St1001 (St1007).

If the store terminal 80 determines that the user is not old enough to purchase the licensed product (Step 1007, NO), it displays to the user a message indicating that the licensed product cannot be purchased (Step 1008), and ends this process. In this case, the store terminal 80 may call a store clerk.

If the store terminal 80 determines that the user is old enough to purchase a licensed product (Step 1007, YES), it uses the camera 86 to capture a picture of the user's face (Step 1009).

The store terminal 80 performs face matching between the image of the user's face captured in step St1009 and the face image data acquired from the IC chip of the identification document that was temporarily stored in memory 82 in step St1006. In other words, the store terminal 80 performs identity verification. The store terminal 80 determines whether or not face matching is successful (St1010), and if face matching is unsuccessful (St1010, NO), returns the process to step St1009. Note that if face matching has failed a predetermined number of times or more, the store terminal 80 may end this process and call a store clerk.

If face matching is successful (St1010, YES), the store terminal 80 sends a payment request for the payment amount displayed in step St1002 to the payment company server 50 (St1011).

The payment company server 50 receives the payment request in step St1011, executes the payment process (St1012), and sends a payment completion notification to the store terminal 80.

The store terminal 80 receives a payment completion notification from the payment company server 50 and displays the completion of payment on the display 87 (Step 1013).

The store terminal 80 inquires of the user whether or not it is OK to register the face image used for face matching (St1014). At this time, the store terminal 80 may also display the advantage that by registering the face image, the user will not need to present personal identification documents the next time he or she purchases a licensed product.

If the user responds that they do not approve the registration of the face image used for face matching (Step 1014, NO), the store terminal 80 ends this process.

If the user responds that they agree to register the face image used for face matching (St1014, YES), the store terminal 80 sends a request to register the face image to the identity verification server 30. Here, the face image sent to the identity verification server 30 may be face image data acquired from the IC chip of the identity verification document, face image data acquired by photographing the face of the identity verification document, or an image of the user captured in step St1009.

The identity verification server 30 receives the facial image registration request sent in step St1016, registers the facial image in the facial matching DB 31 (St1017), and transmits a registration completion notification to the store terminal 80. The facial image registered in the identity verification server 30 is referred to as an identity verification image.

The store terminal 80 receives a registration completion notification from the identity verification server 30, displays the completion of facial image registration on the display 87 (St1018), and ends this process.

By the above process, the identity verification image that has been successfully matched is registered in the identity verification server 30. Next, a detailed description will be given of the registration process (St1004) that is executed when it is determined in step St1003 that the identity verification image has been registered in the identity verification server 30 (St1003, YES).

Figure 18 is a flowchart showing an example of the registered process of Figure 17. In other words, Figure 18 corresponds to details of step St1004 of Figure 17.

The store terminal 80 uses the camera 86 to capture a photo of the user's face (St1101).

The store terminal 80 sends a face authentication request including the image of the user's face captured in step St1101 to the identity verification server 30 (St1102).

The identity verification server 30 receives the face authentication request from the store terminal 80 and executes one-to-N face matching processing (St1103). For example, as one-to-N face matching processing, the identity verification server 30 searches the face matching DB 31 for an identity verification image whose face matching score with the photographed image is equal to or higher than a predetermined threshold. The face matching score is a score indicating the likelihood that two face images are the same person. If the search fails, that is, if the identity verification image estimated to be the same person as the photographed image does not exist in the face matching DB 31, the identity verification server 30 transmits a face authentication result including face authentication failure and a matching score to the store terminal 80. If the search is successful, that is, if the identity verification image estimated to be the same person as the photographed image exists in the face matching DB 31, the identity verification server 30 transmits a face authentication result including face authentication success, a matching score, and the date and time when the identity verification image was registered (hereinafter referred to as the registration date and time) to the store terminal 80.

The store terminal 80 determines whether the facial authentication result received from the identity verification server 30 is successful (Step 1104).

If the store terminal 80 determines that face authentication has failed (St1104, NO), the store terminal 80 proceeds to step St1005 in FIG. 17. Note that, if face authentication has failed, the store terminal 80 may return to step St1101 several times to repeat face photography and face authentication, rather than immediately proceeding to step St1005.

When the store terminal 80 determines that the facial authentication was successful (St1104, YES), it performs a validity determination of the facial authentication result (St1105). For example, the store terminal 80 performs the validity determination of the facial authentication result based on at least one of the following conditions (A1) to (A4). Note that the store terminal 80 may adopt one of the following conditions (A1) to (A4), or may adopt two or more of them. Furthermore, when two or more conditions are adopted, the store terminal 80 may determine that the result is valid when all of the conditions are satisfied, or may determine that the result is valid when at least one of the conditions is satisfied.

(A1) The store terminal 80 determines that the facial matching score is invalid when it is less than a predetermined reference value, and determines that the facial matching score is valid when it is equal to or greater than the reference value. The reference value is a value greater than the threshold value for facial matching in the identity verification server 30.
(A2) If the time that has elapsed since the registration date and time of the personal identification image is greater than a predetermined reference value, the store terminal 80 determines that the image is invalid. If the time that has elapsed is equal to or less than the reference value, the store terminal 80 determines that the image is valid.
(A3) The store terminal 80 returns to step St1101 and, if the number of times that face photography and face authentication are repeated is equal to or greater than a predetermined threshold, determines that the process is invalid; if the number of times that the process is repeated is less than the threshold, determines that the process is valid.
(A4) If the captured image has a local change in characteristics (for example, a change in hairstyle or the presence or absence of glasses) compared to the image for identity verification, the store terminal 80 determines that the image is invalid; if there is no change in characteristics, the store terminal 80 determines that the image is valid.

In this way, by performing the validity determination in step St1105, it is possible to perform more rigorous identity verification. Note that the validity determination in step St1105 may be omitted. Alternatively, whether or not to perform the validity determination in step St1105 may be determined depending on the type of licensed product.

If the store terminal 80 determines that the image is not valid (St1105, NO), the process proceeds to step St1005 in FIG. 17. This causes the personal identification image to be updated.

When the store terminal 80 determines that the image is valid (Step 1105, YES), it displays the expiration date of the current identification image on the display 87 (Step 1106). The expiration date of the identification image may be a predetermined period from the registration date and time. This allows the user to know that if the expiration date is short, the user will need to bring an identification document the next time. Note that the identification server 30 may delete an identification image whose expiration date has passed from the face matching DB 31.

The store terminal 80 sends a payment request to the payment company server 50 for the payment amount displayed in step St1002 (St1107). In other words, if an identity verification image of the same person as the photographed image exists in the face matching DB 31 of the identity verification server 30, the store terminal 80 permits the user of the photographed image to purchase the licensed product. This is because the existence of an identity verification image of the same person as the photographed image in the face matching DB 31 of the identity verification server 30 is synonymous with the user being a person who has previously been permitted to purchase a licensed product.

The payment company server 50 receives the payment request in step St1107, executes the payment process (St1108), and sends a payment completion notification to the store terminal 80.

The store terminal 80 receives a payment completion notification from the payment company server 50 and displays the completion of payment on the display 87 (Step 1109). Then, this process ends.

By performing the above process, a user whose identity verification image has been registered in the identity verification server 30 can purchase licensed products without presenting an identity verification document.

(Modification)
The above-mentioned identity verification process of comparing the face information of the person (e.g., a photographed image of the face) with registered face information (e.g., an image for identity verification), the judgment process based on information acquired from an IC chip in an identity verification document (e.g., in the case of judging whether or not a licensed product can be sold, the judgment process based on information on the date of birth acquired from the IC chip), and the payment process may all be realized by cooperation between a payment terminal linked to a POS register installed in a store and the user's terminal device 10. For example, the user may start a payment application of the terminal device 10 (e.g., a smartphone), and the payment application may perform the above-mentioned process with the payment terminal. Alternatively, the user may hold the terminal device 10 (e.g., a smartphone) over the payment terminal, and the terminal device 10 may perform the above-mentioned process with the payment terminal through short-range wireless communication represented by NFC communication. Alternatively, the user may approach the payment terminal (e.g., pass through a specified gate) while keeping the terminal device 10 (e.g., a smartphone) in a pocket of a bag or clothing, and the terminal device 10 may perform the above-mentioned process with the payment terminal through specified wireless communication.

(Summary of this embodiment)
<Technology 1>
The identity verification terminal (e.g., terminal device 10) of this embodiment includes a camera (e.g., camera 13) that photographs the user's face, a reader (e.g., reader 12) that acquires a registration image, which is an image of the face, from an identity verification document, and a processor (e.g., processor 11) that acquires the image of the user's face captured by the camera, compares the registered image with the captured image, and completes identity verification of the user if facial authentication between the face of the person appearing in the registered image and the user's face is successful, and registers the registration image or the captured image as an identity verification image.

As a result, when the identity verification terminal according to this embodiment performs identity verification of a user using an identity verification document, it photographs the user's face and compares the photographed image with the registered image on the identity verification document, thereby preventing the user from registering as another person and performing highly reliable identity verification.

<Technology 2>
In the personal identification terminal described in Technology 1, the processor registers personal information, such as the user's name and date of birth, which the user has registered in advance, in association with an image for personal identification.

As a result, the identity verification terminal according to this embodiment can create registration information that links a highly reliable identity verification image that has been verified with the user's identity information. By using the registration information, the identity verification terminal can determine with a high degree of accuracy whether the user is a minor or an adult.

<Technology 3>
In the personal identification terminal according to the first or second aspect of the present invention, a processor creates a code that holds information linking personal information and a personal identification image.

As a result, when a user purchases a licensed product at a self-checkout (e.g., self-checkout 70), the user can skip age verification at the self-checkout by having the self-checkout read the code. As a result, the identity verification terminal according to this embodiment can improve user convenience.

<Technique 4>
In the personal identification terminal according to any one of the first to third aspects, the code is a one-dimensional code or a two-dimensional code.

As a result, the identity verification terminal according to this embodiment can eliminate age verification at the self-checkout register and improve user convenience by having the self-checkout register read a one-dimensional or two-dimensional code linked to the user's registration information.

<Technology 5>
In the personal identification terminal according to any one of the first to fourth techniques, the processor further links information stored in the code with information on a service linked by the user.

As a result, the identity verification terminal according to this embodiment can further link other services to the code linked to the user's registration information, allowing the user to use a variety of services all at once, thereby improving user convenience.

<Technique 6>
In the personal identification terminal according to any one of the first to fifth aspects, the service information is payment information or point information.

As a result, the identity verification terminal according to this embodiment further links payment information and point information to a code linked to the user's registration information, eliminating the need for the user to input payment information and point information when purchasing an item, and reducing the complexity of the user's purchasing process at a self-checkout register. The identity verification terminal also shortens the time required for the user's purchasing process, preventing the self-checkout register from becoming overcrowded.

<Technique 7>
In an identity verification terminal described in any one of Technologies 1 to 6, if a processor determines that facial authentication between the face of a person appearing in a registered image and the user's face has failed, and if the number of times the registered image has been matched with the captured image is equal to or greater than a threshold, a screen is displayed on a display device indicating that the user's identity could not be verified and a screen prompting the user to end the procedure.

As a result, the identity verification terminal according to this embodiment can prevent the registration of an image of a person's face that differs from the registered image on the identity verification document, and can verify the identity of the user with high accuracy.

<Technique 8>
In the personal identification terminal according to any one of techniques 1 to 7, when the personal identification document is a My Number card, the processor obtains an electronic certificate for personal information held on the My Number card.

As a result, the identity verification terminal according to this embodiment can prevent a user from verifying their identity using another person's identity document by obtaining an electronic certificate.

<Technique 9>
In the identity verification terminal according to any one of techniques 1 to 8, the processor displays on the display device a screen indicating that registration of the user's identity verification has been completed.

This allows the identity verification terminal of this embodiment to notify the user whether identity verification was successful or not.

<Technique 10>
In the personal identification terminal according to any one of the first to ninth aspects, the personal identification document is a My Number card or a driver's license.

As a result, the identity verification terminal according to this embodiment can perform highly reliable identity verification of a user by using a highly reliable official identification document as the identity verification document.

<Technology 11>
In an identity verification terminal described in any one of Technologies 1 to 10, a processor obtains identity information of the user, such as the user's name and date of birth, from the identity verification document, determines whether the input information of the name and date of birth entered by the user matches the identity information, and if the input information matches the identity information, links the input information to an identity verification image and registers it.

As a result, the identity verification terminal according to this embodiment can verify the identity of the user based on the information that the user has preregistered in the application, and can register the identity verification image in association with the information.

Although the embodiments have been described above with reference to the attached drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, corrections, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also fall within the technical scope of the present disclosure. Furthermore, the components in the above-described embodiments may be combined in any manner as long as it does not deviate from the spirit of the invention.

The technology disclosed herein is useful as an identity verification terminal, program, identity verification method, and identity verification system that perform highly reliable identity verification.

REFERENCE SIGNS LIST 1 identity verification system 10 terminal device 20 identity verification authentication business server 30 identity verification server 31 face matching DB
40 Member company server 50 Settlement company server 60 Point company server 70 Self-checkout 80 Store terminal 11, 71, 81 Processor 12, 72 Reader 13, 73, 86 Camera 14, 74, 83 Communication I/F
15, 75, 87 Display 16, 76, 82 Memory 84 IC reader 85 Code reader MN Screen CH1, CH2, CH3, CH4, CH5, CH6 Mark

Claims (16)

A camera that captures the user's face;
A reader that acquires a registration image, which is a face image, from an identification document;
a processor that acquires a photographed image of the face of the user captured by the camera, compares the photographed image with the registered image, and completes identity verification of the user when face authentication between the face of a person appearing in the registered image and the face of the user is successful, and registers the registered image or the photographed image as an identity verification image ;
The processor,
registering the user's personal information, including the user's name and date of birth, which the user has registered in advance, in association with the user's personal identification image;
creating a code for storing information linking the identity information and the identity verification image;
Further linking the payment information or point information linked by the user to the information held by the code;
Identity verification terminal. The code is a one-dimensional code or a two-dimensional code.
2. The personal identification terminal according to claim 1 . the processor determines that face authentication between the face of the person appearing in the registration image and the face of the user has failed, and when the number of times the registration image and the captured image have been matched is equal to or greater than a threshold, displays on a display device a screen indicating that the identity of the user could not be verified and a screen prompting the user to end the procedure.
2. The personal identification terminal according to claim 1. When the identity verification document is a My Number Card, the processor obtains an electronic certificate for personal information held on the My Number Card.
2. The personal identification terminal according to claim 1. The processor displays on a display device a screen indicating that registration of the user's identity verification has been completed.
2. The personal identification terminal according to claim 1. The identity document is a My Number card or a driver's license.
2. The personal identification terminal according to claim 1. the processor acquires personal information of the user, such as the name and date of birth, from the identification document, determines whether the input information of the name and date of birth entered by the user matches the personal information, and if the input information matches the personal information, associates the input information with the identification image and registers them;
2. The personal identification terminal according to claim 1. A process of taking a picture of a user's face with a camera;
obtaining an enrollment image from an identity document;
A process of acquiring a photographed image of the face of the user;
A process of matching the registered image with the captured image;
a process of completing identity verification of the user when face authentication between a face of a person appearing in the registration image and the face of the user is successful, and registering the registration image or the captured image as an identity verification image;
a process of registering personal information, including the user's name and date of birth, which the user has registered in advance, in association with the personal identification image;
A process of generating a code that holds information linking the personal information and the personal verification image;
and further linking the information stored in the code with payment information or point information associated with the user .
program. The code is a one-dimensional code or a two-dimensional code.
The program according to claim 8 . and further causing the computer to execute a process of determining that face authentication between the face of the person appearing in the registration image and the face of the user has failed, and if the number of times that the registration image and the captured image have been matched is equal to or greater than a threshold, displaying on a display device a screen indicating that the identity of the user could not be verified and a screen prompting the user to end the procedure.
The program according to claim 8 . and further causing the computer to execute a process of obtaining a digital certificate for the personal information held in the identity verification document.
The program according to claim 8 . and further causing the computer to execute a process of displaying on a display device a screen indicating that registration of the user's identity verification has been completed.
The program according to claim 8 . Identification documents are My Number Card or driver's license.
The program according to claim 8 . and further causing the computer to execute a process of acquiring personal information of the user, such as the name and date of birth, from the personal identification document, a process of determining whether or not the input information of the name and date of birth input by the user matches the personal information, and a process of linking the input information with the personal identification image and registering the input information when the input information matches the personal information.
The program according to claim 8 . The computer
Acquire a photographed image of the user's face taken by the camera ;
A registration image , which is a facial image from an identification document obtained from a reader , is compared with the captured image;
When face authentication between the face of the person in the registration image and the face of the user is successful, identity verification of the user is completed, and the registration image or the captured image is registered as an identity verification image;
registering the user's personal information, including the user's name and date of birth, which the user has registered in advance, in association with the user's personal identification image;
creating a code for storing information linking the identity information and the identity verification image;
Further linking the payment information or point information linked by the user to the information held by the code;
How you verify your identity. an identity verification terminal comprising: a first camera that photographs a user's face; a reader that acquires a registration image from an identity verification document; and a processor that acquires a first photographed image of the user's face photographed by the first camera, compares the registration image with the first photographed image, and completes identity verification of the user when facial authentication between the face of a person appearing in the registration image and the face of the user is successful, and registers the registration image or the first photographed image as an identity verification image;
a processor that acquires the identity verification image from the identity verification terminal, compares a second captured image of the user's face captured by the second camera with the identity verification image, and permits the purchase of a licensed product if facial authentication between the face in the second captured image and the face in the identity verification image is successful ;
The identity verification terminal includes:
registering the user's personal information, including the user's name and date of birth, which the user has registered in advance, in association with the user's personal identification image;
creating a code for storing information linking the identity information and the identity verification image;
Further linking the payment information or point information linked by the user to the information held by the code;
Identity verification system.

Images