JP7409019B2 - Mobile terminals, identity verification systems and programs - Google Patents

Description

The present invention relates to a mobile terminal, an identity verification system, and a program used for identity verification processing.

2. Description of the Related Art Conventionally, banks and the like have provided so-called Internet banking services to customers via communication networks such as the Internet. Internet banking services are considered to utilize, for example, biometric information in order to confirm the customer's identity using a terminal such as a mobile terminal owned by the customer. As an example, a method has been disclosed in which an individual is verified by face authentication using a face image stored in an IC card such as a driver's license and a face image of a customer who is an operator of a terminal (for example, Patent Document 1).

JP2015-88080A

Since the device described in Patent Document 1 uses facial image data stored in the chip of an IC card, the information terminal had to be equipped with a reader for reading the IC card.
In order to use a mobile terminal when applying for the start of transactions for not only services such as banks but also services of various financial institutions, it is necessary to be able to confirm the identity with electronic data, and it is necessary to read the IC card. There is a need for a system that can be used in other ways.

Therefore, an object of the present invention is to provide a mobile terminal, an identity verification server, an identity verification system, and a program for verifying identity using electronic data obtained by a method other than reading an IC card.

The present invention solves the above problems by the following solving means. Note that, in order to facilitate understanding, the description will be given with reference numerals corresponding to the embodiments of the present invention, but the present invention is not limited thereto. Further, the configurations described with reference numerals may be improved as appropriate, and at least a portion thereof may be replaced with other configurations.

A first invention is a mobile terminal (1) having a photographing unit (23), which acquires data necessary for user identification at the time of starting a service using only the mobile terminal owned by the user. an identity verification image acquisition means (13) that acquires an identity verification image, which is an image of an identity verification document (3) on which a face photo is posted, from the photographing unit; A stored image acquisition means (15) that acquires a stored image that is an image that includes both the face of the holder who is in possession of the document from the photographing section, and a verification image that is an image that includes the face of the holder and the A verification image acquisition means (16) acquired from a photographing unit, the identity verification image acquired by the identity verification image acquisition means, and the verification image acquired by the verification image acquisition means are used to verify the identity using the images. This is a mobile terminal equipped with a verification image transmitting means (16) for transmitting a verification image to an identity verification server (4).
A second invention is a mobile terminal (1) according to the first invention, which includes a display section (25) that displays an image of a photographing range by the photographing section (23), the identification document (3) and the holder. The portable terminal is characterized by comprising: guide image output means (14) for outputting a guide image indicating the arrangement range of the face of the user to the display section.
A third invention is that in the mobile terminal (1) of the second invention, the guide image output means (14) sets the arrangement range of the identification document (3) to a random position that does not overlap with the position of the face. The mobile terminal is characterized in that the guide image shown in a range of a predetermined size or more is output to the display section (25).
A fourth invention is a mobile terminal (1) according to the second invention or the third invention, in which an image of the identification document (3) included in the stored image acquired by the stored image acquisition means (15) and , a mobile terminal comprising a saved image confirmation means (15) for confirming the position of the owner's face image based on the guide image outputted by the guide image output means (14). .
A fifth invention is a mobile terminal (1) according to any one of the first invention to the fourth invention, wherein the stored image acquisition means (16) is configured to store a plurality of identification documents (3) in different positions. The mobile terminal is characterized in that the stored image is acquired from the photographing unit (23).
A sixth invention is a mobile terminal (1) according to any one of the first to fifth inventions, in which a face photo image is extracted from the identification image acquired by the identification image acquisition means (13). The verification image transmission means (16) includes a face photograph image extraction means (13), and the verification image transmission means (16) sends the face photograph image extracted by the face photograph image extraction means and the verification image to the identity verification server (4). This mobile terminal is characterized by the ability to transmit.
A seventh invention is an identity verification server (4) that is communicably connected to any of the mobile terminals (1) of the first to sixth inventions and performs identity verification processing, face matching means (43) for matching the matching image received from the mobile terminal with a face photo image included in the personal identification image; and result transmitting means for transmitting matching result information by the face matching means to the mobile terminal. (44).
An eighth invention is an identity verification system (100) comprising the mobile terminal (1) according to any one of the first to sixth inventions and the identity verification server (4) according to the seventh invention. In this case, when the identity has not been verified, the mobile terminal transmits the identity verification image acquisition means (13), the stored image acquisition means (15), the verification image acquisition means (16), and the verification image transmission. The process by the means (16) is executed, and based on the collation result information, a service is provided between the mobile terminal and the service providing server (7), which is communicatively connected to the identity verification server and performs service processing. This identity verification system is characterized by comprising a service start determination means (18) for determining whether or not to start processing.
The ninth invention has a photographing unit (23), and uses a computer (1) owned by the user, which is used to verify the user's identity at the time of starting a service, to collect the identity verification document (3) on which the face photograph is posted. an identity verification image acquisition unit that acquires an identity verification image as an image from the photographing unit; and a stored image that is an image including both the identity verification document and the face of the holder of the identity verification document; A stored image acquisition means that acquires from the photographing section, a verification image acquisition means that acquires a verification image that is an image including the face of the owner from the photographing section, and the identity verification that is acquired by the identity verification image acquisition means. A program ( 21b ).

According to the present invention, it is possible to provide a mobile terminal, an identity verification server, an identity verification system, and a program for verifying identity using electronic data obtained by a method other than reading an IC card.

FIG. 1 is a diagram showing the overall configuration of an identity verification system according to a first embodiment. FIG. 2 is a functional block diagram of the mobile terminal according to the first embodiment. FIG. 2 is a functional block diagram of the identity verification server according to the first embodiment. 2 is a flowchart illustrating service start processing in the mobile terminal according to the first embodiment. 2 is a flowchart illustrating identity verification processing in the mobile terminal according to the first embodiment. FIG. 2 is a flowchart showing identity verification image processing in the identity verification system according to the first embodiment. FIG. 2 is a flowchart showing stored image processing in the identity verification system according to the first embodiment. FIG. 3 is a diagram showing an example of display on the mobile terminal according to the first embodiment. FIG. 3 is a diagram showing an example of display on the mobile terminal according to the first embodiment. 2 is a flowchart showing verification image processing in the identity verification system according to the first embodiment. FIG. 2 is an overall configuration diagram of an identity verification system and a functional block diagram of a mobile terminal and an identity verification server according to a second embodiment; FIG. 12 is a flowchart showing identity verification image processing in the identity verification system according to the second embodiment. FIG. 3 is a functional block diagram of a mobile terminal in an identity verification system according to a third embodiment. 12 is a flowchart showing identity verification image processing in the identity verification system according to the third embodiment. FIG. 7 is a diagram illustrating a display example on a mobile terminal according to a third embodiment. FIG. 7 is a diagram showing an example of a document position guide image displayed on a mobile terminal according to a third embodiment. 12 is a flowchart showing stored image processing in the identity verification system according to the third embodiment. 12 is a flowchart showing verification image processing in the identity verification system according to the third embodiment. 13 is a flowchart illustrating a matching image acquisition process in a mobile terminal according to a third embodiment. FIG. 7 is a diagram illustrating a display example on a mobile terminal according to a third embodiment.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. Note that this is just an example, and the technical scope of the present invention is not limited to this.

(First embodiment)
FIG. 1 is a diagram showing the overall configuration of an identity verification system 100 according to the first embodiment.
FIG. 2 is a functional block diagram of the mobile terminal 1 according to the first embodiment.
FIG. 3 is a functional block diagram of the identity verification server 4 according to the first embodiment.
The personal identification system 100 shown in FIG. 1 is a system for performing personal identification using only the mobile terminal 1 when starting various services for the first time. Further, the identity verification system 100 is a system that performs identity authentication required at the time of starting a second or subsequent service using only the mobile terminal 1.

Here, various services include, for example, services related to opening an account at a bank, services related to various banking operations, car sharing services, real estate rental, etc., and refer to various services that require identity verification. Currently, the method of verifying identity when opening a bank account, etc. is by sending mail that does not require forwarding to the home of the person applying for opening an account. Recently, financial institutions are considering ways to complete online identity verification. Identity verification in the present invention refers to a verification process that enables online procedures using a mobile terminal or the like when starting a transaction with a financial institution such as opening an account. On the other hand, personal authentication refers to a state in which the user's identity has already been verified and is used to prevent impersonation, etc. when starting a service for the second time or later.
In the following description, an example will be described in which the mobile terminal 1 is used to start a service that requires identity verification or identity authentication.

The identity verification system 100 includes a mobile terminal 1 and an identity verification server 4. The mobile terminal 1 can be connected to a communication network N via a base station R for wireless communication, for example. Further, the identity verification server 4 is communicably connected via a communication network N.
The identity verification system 100 is connected to the service providing server 7 via the communication network N. The service providing server 7 is a server for performing service processing with the mobile terminal 1.

The mobile terminal 1 is, for example, a terminal owned by a person who wishes to receive services provided by the service providing server 7. Hereinafter, a person who seeks to receive a service will be referred to as a user. The mobile terminal 1 is, for example, a portable device such as a smartphone or a tablet that also has the functions of a computer.
As shown in FIG. 2, the mobile terminal 1 includes a control section 10, a storage section 20, a camera 23 (photographing section), a touch panel display 25, and a communication interface section 29.

The control unit 10 is a CPU (central processing unit) that controls the entire mobile terminal 1. The control unit 10 cooperates with the above-mentioned hardware to execute various functions by appropriately reading and executing the operating system (OS) and various application programs stored in the storage unit 20.
The control unit 10 includes an application starting unit 11, an identity verification unit 12, an identity authentication unit 17, and a service start determination unit 18 (service start determination means).

When the transaction application 21a is executed, the application activation unit 11 receives a request for personal authentication from the service providing server 7 that performs processing corresponding to the transaction application 21a, and activates the personal identification application 21b (program).
The identity verification unit 12 is a control unit that performs identity verification processing. The identity verification unit 12 includes an identity verification image processing unit 13 (identity verification image acquisition means, facial photo image extraction means), a guide image output unit 14 (guide image output means, second guide image output means), and a saved image. It includes a processing section 15 (stored image acquisition means, image confirmation means, second image confirmation means) and a verification image processing section 16 (verification image acquisition means, verification image transmission means).

The identity verification image processing unit 13 acquires an identity verification image, which is an image of the identity verification document, when the user operates the camera 23 to photograph his or her own identity verification document. Here, the identity verification document is an officially recognized identification card with a photograph of the user's face on it. A driver's license 3 is shown in FIG. 1 as an example of an identification document. Note that the identity verification document may also be a My Number card or the like on which the user's face photo is posted.
Then, the identity verification image processing unit 13 extracts a face photo image from the acquired identity verification image. Thereafter, the identity verification image processing unit 13 stores the extracted facial photograph image in the storage unit 20.

The guide image output unit 14 displays a guide image (second guide image) on the touch panel display 25 for acquiring an image that includes both the identity verification document and the face of the user (holder) who possesses the identity verification document. Output to. The guide image is an auxiliary image that shows the arrangement range, and serves as a guide for obtaining an image in which the identity verification document and the user's face are arranged at the specified positions.

The stored image processing unit 15 acquires a stored image, which is an image including both the identification document and the face of the user holding the identification document, by the user operating the camera 23 . At that time, the user operates his or her mobile terminal 1 so that both the identification document and the face of the user carrying the identification document are placed in the guide position indicated by the guide image.
The stored image processing unit 15 confirms that the image of the identification document and the user's face image included in the acquired stored image are within the range of the guide image. Then, when the saved image has been confirmed, the saved image processing unit 15 stores the saved image in the storage unit 20.

The matching image processing unit 16 acquires a matching image, which is the user's face image, and is used for matching, when the user operates the camera 23 to photograph his or her own face.
Then, the verification image processing section 16 transmits the acquired verification image and the facial photograph image stored in the storage section 20 to the identity verification server 4 .

The person authentication section 17 is a control section that performs person authentication. The personal authentication unit 17 acquires a matching image, which is the user's face image and is used for matching, when the user operates the camera 23 to take a picture of his or her own face, and stores the obtained matching image and the storage unit 20. The stored face photo image is sent to the identity verification server 4.
The service start determination unit 18 receives the verification result (verification result information) from the identity verification server 4, and determines whether or not the service can be started based on the received verification result.

The storage unit 20 is a storage area such as a semiconductor memory device for storing programs, data, etc. necessary for the control unit 10 to execute various processes.
The storage unit 20 includes a program storage unit 21 . The program storage unit 21 is a storage area that stores various application programs (hereinafter, application programs are referred to as applications, applications, programs, etc.). The program storage unit 21 stores a transaction application 21a and an identity verification application 21b. In addition, although this embodiment uses the transaction application 21a and the personal identification application 21b below, it is also possible to use an application that combines the functions of the two applications.

The transaction application 21a and the identity verification application 21b are installed in the mobile terminal 1 in advance, or are downloaded to the mobile terminal 1 by communicating with an application distribution server (not shown) via the communication network N.
The transaction application 21a is a program for performing service processing with the service providing server 7.
The identity verification application 21b is a program for performing various functions of identity verification and identity authentication executed by the control unit 10 of the mobile terminal 1.

Camera 23 is a photographing device. The camera 23 has an in-camera 23a and an out-camera 23b. The in-camera 23a is a camera provided on the touch panel display 25 side of the mobile terminal 1. The outside camera 23b is a camera provided on the back side of the mobile terminal 1.
The touch panel display 25 has a function as a display section configured with a liquid crystal panel or the like, and an input section for performing various operation inputs using a user's finger.
The communication interface section 29 is an interface for communicating with various servers via the communication network N, and serves as a transmitting section and a receiving section.

Returning to FIG. 1, the driver's license 3 is in the form of a card, and as described above, is an official identification card. The user's personal information including the user's address, name, date of birth, and a photograph of the user's face is written on the surface of the driver's license 3. A user of the mobile terminal 1 uses the mobile terminal 1 to photograph his or her driver's license 3 or the like in the identity verification system 100 .

The identity verification server 4 is a server that performs identity verification with the mobile terminal 1 through online procedures, and performs identity verification through face authentication. The identity verification server 4 is owned by a company different from the company providing the service. Note that the identity verification server 4 may be owned by a company that provides services.
As shown in FIG. 3, the identity verification server 4 includes a control section 40, a storage section 50, and a communication interface section 59.

The control unit 40 is a CPU that controls the entire identity verification server 4. The control unit 40 cooperates with the above-mentioned hardware and executes various functions by appropriately reading and executing the OS and application programs stored in the storage unit 50.
The control section 40 includes a face matching section 43 (face matching means) and a matching result transmitting section 44 (result transmitting means).

The face matching unit 43 receives the matching image and the facial photo image transmitted by the mobile terminal 1. Then, the face matching unit 43 matches the received matching image and the facial photograph image. Here, the face matching unit 43 may indicate the matching result between the matching image and the facial photograph image by a score. For example, the face matching unit 43 performs image matching processing and calculates the degree of matching of the face photo images as a score. Here, the score is expressed, for example, by a numerical value from 0 to 100, and the higher the degree of matching, the higher the numerical value.
The verification result transmitter 44 transmits the verification result to the mobile terminal 1.

The storage unit 50 is a storage area such as a hard disk or a semiconductor memory device for storing programs, data, etc. necessary for the control unit 40 to execute various processes.
The storage unit 50 includes a program storage unit 51. The program storage unit 51 is a storage area that stores various programs. The program storage unit 51 stores an identity verification program 51a.
The identity verification program 51a is a program for performing various functions executed by the control unit 40 of the identity verification server 4.
The communication interface unit 59 is an interface for communicating with various servers such as the service providing server 7 and the mobile terminal 1 via the communication network N.

Here, the computer refers to an information processing device equipped with a control section, a storage device, etc., and the mobile terminal 1 and the identity verification server 4 are information processing devices each equipped with a control section, a storage section, etc. included in the concept.

The service providing server 7 shown in FIG. 1 is a server for performing service processing. In FIG. 1, the identity verification system 100 shows only one service providing server 7, but it may include a plurality of servers for each service.
Although not shown, the service providing server 7 includes a control section, a storage section, a communication interface section, and the like.

The base station R shown in FIG. 1 is a base station for wireless communication, and performs relay for the mobile terminal 1 to communicate with various servers. The base station R is, for example, a base station for a wireless LAN (Local Area Network) or a base station for a mobile terminal communication network of a communication carrier.
The communication network N is a network between various servers or between various servers and the base station R, and is an Internet line, a mobile terminal communication network, or the like.

Next, the processing of the identity verification system 100 will be explained.
FIG. 4 is a flowchart showing service start processing in the mobile terminal 1 according to the first embodiment.
First, a user who wants to start a service performs a touch operation such as a tap to select an icon (not shown) of the transaction application 21a displayed on the touch panel display 25 of the user's mobile terminal 1, for example. By doing so, the control unit 10 of the mobile terminal 1 executes the transaction application 21a.

When the transaction application 21a is executed, the main screen (not shown) of the transaction application 21a is displayed on the touch panel display 25 of the mobile terminal 1. Therefore, when the user performs an operation to start a service from the main screen (for example, an operation to select a menu), in step S10 (hereinafter simply referred to as "S") of FIG. accepts service start operations.
Next, in S11, the control unit 10 (app activation unit 11) activates the identity verification application 21b.

In S12, the control unit 10 determines whether the identity has been verified. The case where the user's identity has been verified means that the transaction application 21a has already been used for service, and therefore, the user's identity verification process has already been performed. Therefore, if the user's identity has been verified, this corresponds to the case where the second or subsequent service start has been accepted. Further, the case where the identity has not been verified corresponds to the case where the start of service using the transaction application 21a is accepted for the first time.
Note that the control unit 10 may store whether or not the identity has been verified by associating the identity verification application 21b with the transaction application 21a for which the identity has been verified. good. Furthermore, if the identity has been verified, the face photo image is stored in the storage unit 20, so the control unit 10 may determine whether or not the face photo image is stored in the storage unit 20. .
If the identity has been verified (S12: YES), the control unit 10 moves the process to S14. On the other hand, if the identity has not been verified (S12: NO), the control unit 10 moves the process to S13.
In S13, the control unit 10 (identity verification unit 12) performs identity verification processing. The identity verification process will be described later. After that, the control unit 10 moves the process to S15.

On the other hand, in S14, the control unit 10 (identity authentication unit 17) performs identity authentication processing. The control unit 10 first activates the in-camera 23a and causes the user to photograph his or her own face, thereby acquiring a matching image that is the user's face image. Then, the control unit 10 transmits the acquired verification image and the facial photograph image obtained from the identification document stored in the storage unit 20 to the identification server 4 . Note that in the identity verification server 4, the control unit 40 (face matching unit 43, matching result transmitting unit 44) matches the matching image with the facial photograph image, and transmits the matching result to the mobile terminal 1.

In S15, the control unit 10 (service start determination unit 18) determines whether or not authentication is successful based on the verification result. For example, when the score obtained as a comparison result is equal to or greater than a threshold value, the control unit 10 determines that authentication has been successful. If it is determined that the authentication has been successful (S15: YES), the control unit 10 moves the process to S16. On the other hand, if it is not determined that the authentication has been successful (S15: NO), the control unit 10 moves the process to S18.
In S16, the control unit 10 ends execution of the identity verification application 21b.
In S17, the control unit 10 performs service processing with the service providing server 7 corresponding to the transaction application 21a. After that, the control unit 10 ends this process.

On the other hand, in S18, the control unit 10 outputs an authentication error message to the touch panel display 25.
In S19, the control unit 10 ends execution of the identity verification application 21b. After that, the control unit 10 ends this process.

Next, the identity verification process performed by the mobile terminal 1 will be explained.
FIG. 5 is a flowchart showing the identity verification process in the mobile terminal 1 according to the first embodiment.
FIG. 6 is a flowchart showing the identity verification image processing in the identity verification system 100 according to the first embodiment.
FIG. 7 is a flowchart showing stored image processing in the identity verification system 100 according to the first embodiment.
8 and 9 are diagrams showing display examples on the mobile terminal 1 according to the first embodiment.
FIG. 10 is a flowchart showing verification image processing in the identity verification system 100 according to the first embodiment.

In S30 of FIG. 5, the control unit 10 (identity verification image processing unit 13) of the mobile terminal 1 performs identity verification image processing.
Here, the identity verification image processing will be explained based on FIG. 6.
In S40 of FIG. 6, the control unit 10 of the mobile terminal 1 activates the outside camera 23b, displays a through image (image within the shooting range) of the outside camera 23b on the touch panel display 25, and makes it possible to photograph the identity verification document. do. Then, when the user taps a photographing button (not shown) with the identification document reflected in the through image, the control unit 10 acquires the identification image.

At that time, the control unit 10 first displays an identification document selection screen (not shown) for selecting what to photograph as an identification document (driver's license 3 or My Number card), and prompts the user to confirm the identity of the user. The type of confirmation document may be selected. Then, the control unit 10 may change the guidance for having the user take a photograph depending on the type of identification document. For example, in the case of driver's license 3, it is necessary to photograph the front and back sides. On the other hand, if you have a My Number card, you only need to take a photo of the front side.
Furthermore, in order to confirm that the identity verification document is genuine, the control unit 10 includes an image taken from a direction that shows the thickness of the card, which is the side of the identity verification document, as part of the identity verification image. You may obtain it.
Furthermore, the control unit 10 may obtain a video recording of a series of actions instead of a plurality of still images.

In S41, the control unit 10 extracts a face photo image from the identity verification image. For example, the control unit 10 analyzes the identity verification image and identifies the identity verification document. Then, the control unit 10 specifies the image position of the face photograph from the identification document and extracts the image at the specified image position.
In S42, the control unit 10 causes the storage unit 20 to store the extracted facial photograph image. After that, the control unit 10 moves the process to S31 in FIG.

In S31 of FIG. 5, the control unit 10 performs storage image processing.
Here, storage image processing will be explained based on FIG. 7.
In S50 of FIG. 7, the control unit 10 causes the touch panel display 25 to display an explanation screen for photographing a saved image.
FIG. 8(A) is an example of the explanation screen 80. The explanation screen 80 is a screen that explains that the user's face and identification document are to be photographed together.

In S51 of FIG. 7, the control unit 10 (guide image output unit 14) specifies a guide image and displays the specified guide image on the touch panel display 25.
FIG. 8(B) is an example of the guide image 81. The guide image 81 is an image including a face position area 81a where the user's face is placed and a document position area 81b where the user's identification document is placed.

Here, the control unit 10 may randomly designate one guide image from a plurality of previously prepared guide images with different document position areas and display it on the touch panel display 25. Here, the document position area is provided at a position that does not overlap the face position area. Further, the document location area is an area having a size larger than a predetermined size that allows the identification document to be visually recognized.
Note that the guide image is not limited to a still image, and may be a moving image. Furthermore, guide images with different document position areas include those in which the position of the document position area differs in the vertical and horizontal directions relative to the photographing position, as well as those in which the position of the document position area differs in the depth direction with respect to the photographing position.

In S52 of FIG. 7, the control unit 10 (saved image processing unit 15) activates the in-camera 23a and causes the touch panel display 25 to display the through image of the in-camera 23a together with the guide image.
As shown in FIG. 9, the user positions and adjusts the face so that his or her face is reflected in the face position area 81a of the guide image 81, and the identification document is reflected in the document position area 81b of the guide image 81. Adjust the position of your identity verification documents. Then, when the user taps the photographing button 81c, the control unit 10 acquires a saved image including the user's face and the identification document photographed by the in-camera 23a.

Here, the control unit 10 may sequentially display a plurality of guide images having different positions in the document position area so that the user can obtain a plurality of saved images. Further, the control unit 10 may cause the user to obtain a saved image of the video according to a guide image of the video. In other words, the control unit 10 outputs a different guide image each time the user is verified. Therefore, in order to obtain a saved image corresponding to the output guide image, the user associates the position of the face and the position of the identification document with the guide image.

In S53 of FIG. 7, the control unit 10 (saved image processing unit 15) confirms the acquired stored image. Specifically, the control unit 10 confirms that the image of the identification document included in the acquired saved image and the user's face image roughly match the range of the guide image.
In S54, the control unit 10 (archived image processing unit 15) stores the archived image in the storage unit 20 on the condition that the archived image has been confirmed. In addition, if the saved image cannot be confirmed, that is, if the image of the identity verification document included in the saved image and the user's face image do not fit within the range of the guide image, an error will occur. , the subsequent processing may not be performed, or the processing from S51 may be performed again. After that, the control unit 10 moves the process to S32 in FIG.

In S32 of FIG. 5, the control unit 10 (matching image processing unit 16) performs matching image processing.
Here, matching image processing will be explained based on FIG. 10.
In S60 of FIG. 10, the control unit 10 of the mobile terminal 1 activates the in-camera 23a and causes the touch panel display 25 to display a through image of the in-camera 23a. Then, when the user taps a photographing button (not shown) with his or her own face reflected in the through-the-lens image, the control unit 10 acquires a matching image.

In S61, the control unit 10 transmits the obtained verification image together with the facial photograph image stored in the storage unit 20 to the identity verification server 4. Here, the facial photograph image stored in the storage unit 20 is one that is acquired and stored by the mobile terminal 1 in the identity verification image processing.
In S62, the control unit 40 (face matching unit 43) of the identity verification server 4 receives the matching image and the face photo image from the mobile terminal 1.

In S63, the control unit 40 (face matching unit 43) performs face matching processing to match the matching image and the facial photograph image. The control unit 40 performs image matching processing, and calculates, for example, the degree of matching between the matching image and the facial photograph image as a score.
In S64, the control unit 40 (verification result transmitting unit 44) transmits the verification result to the mobile terminal 1. After that, the control unit 40 ends this process.
In S65, the control unit 10 (verification image processing unit 16) of the mobile terminal 1 receives the verification result. Thereafter, the control unit 10 moves the process to S15 in FIG. 4, and performs a determination process to determine whether or not authentication has been achieved based on the verification result, as described above.
Note that, if it is determined in the determination process that authentication cannot be performed (NO in S15 in FIG. 4), it is desirable that the control unit 10 delete the face photograph image and the saved image stored in the storage unit 20.

As described above, according to the first embodiment, the identity verification system 100 has the following effects.
(1) When providing a service using the mobile terminal 1, if the mobile terminal 1 has the camera 23, it is possible to perform identity verification using only the mobile terminal 1. Therefore, even if the mobile terminal 1 does not have a reading function such as a reader, it is convenient to be able to verify the identity using the camera 23 that a general mobile terminal 1 would have. . Furthermore, the user does not have to go to a counter or the like and can be provided with an environment where he or she can verify his/her identity anytime and anywhere. Additionally, since the identity verification process uses biometric information, security is high. Furthermore, there is no need to register biometric information in advance such as fingerprints, which is highly convenient.

(2) Identity verification by comparing the face photo image obtained from the identity verification document with the user's own face image taken by the user, and identity verification by acquiring a stored image containing both the identity verification document and the user's face. This is convenient because it can be done in one flow.
(3) Only images that follow the instructions of the guide image are allowed to be saved. Since the guide image shows the arrangement range of the identity verification document and the arrangement range of the user's face, it can be shown in an easy-to-understand manner to the user. Furthermore, guide images with different placement positions of the identification images are randomly output. Therefore, the system becomes such that images taken in advance cannot be used, making it difficult for fraudsters to commit fraud.

(Second embodiment)
In the second embodiment, a case will be described in which a part of the processing using an identity verification document is performed by an identity verification server. In the following description, parts that perform the same functions as those in the first embodiment described above will be given the same reference numerals or the same suffixes, and redundant explanation will be omitted as appropriate.

FIG. 11 is an overall configuration diagram of the identity verification system 200 and a functional block diagram of the mobile terminal 201 and the identity verification server 204 according to the second embodiment.
The identity verification system 200 includes a mobile terminal 201 and an identity verification server 204, which are communicably connected via a communication network N and a base station R (not shown). Although not shown, the identity verification system 200 is connected to the service providing server 7 via a communication network N.
The mobile terminal 201 includes a control section 210, a storage section 220, a camera 23, a touch panel display 25, and a communication interface section 29.

The control unit 210 includes an application starting unit 11 , an identity verification unit 212 , an identity authentication unit 17 , and a service start determination unit 18 .
The identity verification section 212 includes an identity verification image processing section 213 , a guide image output section 14 , a stored image processing section 15 , and a verification image processing section 16 .
The identity verification image processing unit 213 acquires an identity verification image by the user operating the camera 23 . Then, the identity verification image processing unit 213 extracts a facial photograph image from the acquired identity verification image, and causes the storage unit 220 to store the facial photograph image. Further, the identity verification image processing unit 213 transmits the acquired identity verification image to the identity verification server 204.

The storage unit 220 includes a program storage unit 221 . The program storage unit 221 stores a transaction application 21a and an identity verification application 221b. The identity verification application 221b is a program for performing various functions of identity verification and identity authentication executed by the control unit 210 of the mobile terminal 201.

The identity verification server 204 includes a control section 240, a storage section 250, and a communication interface section 59.
The control unit 240 includes an identity verification image analysis unit 241, a face verification unit 43, and a verification result transmission unit 44.
The identity verification image analysis unit 241 receives an identity verification image obtained by photographing the driver's license 3 from the mobile terminal 201 . Then, the identity verification image analysis unit 241 converts the characters included in the identity verification document into text using OCR (optical character recognition). Thereafter, the identity verification image analysis unit 241 transmits the text string to the mobile terminal 201.
The storage unit 250 includes a program storage unit 251. The program storage unit 251 stores an identity verification program 251a, which is a program for performing various functions executed by the control unit 240 of the identity verification server 204.

Next, the processing of the identity verification system 200 will be explained.
The service start process and the identity verification process are the same as those in the first embodiment (FIGS. 4 and 5).
The identity verification image processing performed in the identity verification process will be described based on FIG. 12.
FIG. 12 is a flowchart showing identity verification image processing in the identity verification system 200 according to the second embodiment.
The processing from S240 to S242 is similar to the processing from S40 to S42 in the first embodiment (FIG. 6).
In S243, the control unit 210 of the mobile terminal 201 transmits the acquired identity verification image to the identity verification server 204.
Note that the processing in S241 and S242 and the processing in S243 do not occur before or after the processing in S243, and the processing in S241 and S243 may be performed in parallel, or the processing in S243 may be performed before the processing in S241. good.

In S244, the control unit 240 of the identity verification server 204 receives the identity verification image.
In S245, the control unit 240 (identity verification image analysis unit 241) obtains text information obtained by converting the identity verification document included in the received identity verification image into text.
In S246, the control unit 240 transmits the acquired character information to the mobile terminal 201. After that, the control unit 240 ends this process.

In S247, the control unit 210 (identification confirmation unit 212) of the mobile terminal 201 receives the text information and causes the touch panel display 25 to display the received text information. Note that the control unit 210 of the mobile terminal 201 may have a function of correcting displayed character information. Then, the acquired text information can be transmitted to the service providing server 7, for example, when the user's identity has been verified. After that, the control unit 210 moves the process to the saved image process (processing corresponding to S31 in FIG. 5) of the identity verification process.
The storage image processing and the matching image processing are the same as those in the first embodiment (FIGS. 7 and 10).

As described above, according to the second embodiment, the identity verification system 200 has the following effects.
(1) The mobile terminal 201 transmits the acquired identity verification image to the identity verification server 204, and the identity verification server 204 converts text information included in the identity verification image into text and transmits it to the mobile terminal 201. Therefore, when a user uses a service, it is sufficient to use text information such as a name, etc., which saves the user the trouble of inputting it.
(2) By making it possible to modify the text information received by the mobile terminal 201, even if there is a misconversion or the name or address written on the identity verification document has changed, only the changed part can be edited. You can change it to make it more convenient.

(Third embodiment)
In the third embodiment, various photographing modes performed by a user using a mobile terminal will be explained in a manner that improves security.

FIG. 13 is a functional block diagram of the mobile terminal 301 in the identity verification system 300 according to the third embodiment.
The identity verification system 300 includes a mobile terminal 301. Although not shown, in the identity verification system 300, a mobile terminal 301 is connected to the identity verification server 4 and the service providing server 7 via a communication network N and a base station R (not shown) (see FIG. 1). .
The mobile terminal 301 includes a control section 310, a storage section 320, a camera 23, a touch panel display 25, and a communication interface section 29.
The control unit 310 includes an application starting unit 11 , an identity verification unit 312 , an identity authentication unit 17 , and a service start determination unit 18 .

The identity verification unit 312 includes an identity verification image processing unit 313 (first guide image output means, identity verification image acquisition means, first repetition means, first image confirmation means, face photo image extraction means), and a stored image. Processing section 315 (second guide image output means, stored image acquisition means, second repetition means, second image confirmation means, stored image transmission means), matching image processing section 316 (instruction output means, matching image acquisition means) (a verification image transmission means, a third repetition means, a third image confirmation means).

The identity verification image processing unit 313 causes the touch panel display 25 to output a document position guide image (first guide image). The document position guide image is an image that specifies the photographing direction in which the face photo can be confirmed and the placement position of the identity verification document, including the size of the identity verification image. The user operates his or her mobile terminal 301 to copy his or her own identification document according to the guide position indicated by the document position guide image. Then, when the user operates the camera 23 to take a picture, the identity verification image processing unit 313 acquires an identity verification image that is an image of the identity verification document.

The identity verification image processing unit 313 then outputs a document position guide image with a different photographing direction and arrangement position from the previous one, and performs a process of causing the user to obtain an identity verification image according to the document position guide image. The identity verification image processing unit 313 outputs this document position guide image and repeats the process of having the user photograph the identity verification document multiple times. The identity verification image processing unit 313 then stores the acquired identity verification image in the storage unit 320. Here, one of the document position guide images output by the identity verification image processing unit 313 is for identity verification from the front of the identity verification document, which is a photographing direction perpendicular to the surface containing the face photo of the identity verification document. This is for taking pictures of documents.

The identity verification image processing unit 313 extracts a face photo image from one of the stored identity verification images. In this case, it is preferable that the identity verification image processing unit 313 extracts the face photo image using the identity verification image obtained by photographing the identity verification document from the front. By doing so, the control unit 310 does not need to perform additional processing such as correcting distortion of the acquired facial photograph image, for example. Thereafter, the identity verification image processing unit 313 causes the storage unit 320 to store the extracted facial photograph image.

Further, the identity verification image processing unit 313 verifies the acquired identity verification image based on the document position guide image used at the time of acquisition. The identity verification image processing unit 313 analyzes the identity verification image and confirms, for example, that the identity verification document with the face photo posted from the designated photographing direction has been photographed within the frame indicated by the document position guide image. do.

The saved image processing unit 315 acquires a saved image that is an image including both the identity verification document and the face of the user who is in possession of the identity verification document by the user operating the camera 23 . At this time, the saved image processing unit 315 outputs a multi-position guide image (second guide image) to the touch panel display 25. The user operates his or her mobile terminal 301 so that both the identity verification document and the face of the user holding the identity verification document fit into the guide position indicated by the multiple position guide image. The stored image processing unit 315 may randomly acquire and output multiple position guide images from the storage unit 320 that stores a plurality of multiple position guide images with different guide positions (arrangement positions) of the identity verification document.

The stored image processing unit 315 outputs multiple position guide images with different guide positions of the identity verification document, and repeats the process of acquiring stored images multiple times in accordance with the multiple position guide images. Then, the archived image processing unit 315 stores the acquired archived image in the storage unit 320.
The saved image processing unit 315 confirms that the image of the identification document and the user's face image included in the acquired saved image are within the guide range of the multi-position guide image. Then, when the saved image is confirmed, the saved image processing unit 315 stores the saved image in the storage unit 320.
Further, the saved image processing unit 315 may transmit the acquired saved image to the identity verification server 4.

The matching image processing unit 316 obtains a matching image to be used for matching, which is the user's face image, by photographing the user's own face.
First, the verification image processing unit 316 causes the touch panel display 25 to output a face photographing guide image (instruction information) for photographing the user's own front face. The user operates his or her mobile terminal 301 to activate the in-camera 23a to orient his or her face to the front so that the user fits in the guide position indicated by the face photographing guide image. Then, when the user operates the camera 23 to take a picture, the matching image processing unit 316 acquires a matching image.

Next, the matching image processing unit 316 causes the touch panel display 25 to output an instruction to change the orientation of the user's face, and also outputs the instruction from, for example, a speaker (not shown) of the mobile terminal 301. If the user changes the direction of his or her face, the user cannot visually recognize the touch panel display 25, so the voice instruction is effective. Then, the matching image processing unit 316 outputs a countdown as an audio signal, controls the camera 23 when the shooting timing comes, and acquires a matching image.
Note that the matching image processing unit 316 may perform the process of photographing the matching image multiple times by changing the orientation of the user's face, for example, in the up, down, left, and right directions.
Then, the matching image processing unit 316 stores the acquired matching image in the storage unit 320.

Further, the verification image processing unit 316 checks whether the acquired verification image satisfies the photographing conditions, that is, whether it complies with the previously output instruction content.
Furthermore, the verification image processing unit 316 transmits the plurality of acquired verification images and the facial photograph image stored in the storage unit 320 to the identity verification server 4 .
Note that, among the plurality of comparison images received from the mobile terminal 301, the identity verification server 4 compares, for example, a comparison image obtained by photographing the user's front face with a facial photograph image.

The storage unit 320 includes a program storage unit 321. The program storage unit 321 stores a transaction application 21a and an identity verification application 321b. The identity verification application 321b is a program for performing various functions of identity verification and identity authentication executed by the control unit 310 of the mobile terminal 301.

Next, the processing of the identity verification system 300 will be explained.
The service start process and the identity verification process are the same as those in the first embodiment (FIGS. 4 and 5).
The identity verification image processing performed in the identity verification process will be explained based on FIG. 14.
FIG. 14 is a flowchart showing identity verification image processing in the identity verification system 300 according to the third embodiment.
FIG. 15 is a diagram showing a display example on the mobile terminal 301 according to the third embodiment.
FIG. 16 is a diagram showing an example of a document position guide image displayed on the mobile terminal 301 according to the third embodiment.

In S340 of FIG. 14, the control unit 310 (identity verification image processing unit 313) of the mobile terminal 301 activates the outside camera 23b and causes the touch panel display 25 to display a through image of the outside camera 23b. Further, the control unit 310 outputs, for example, a document position guide image 350 such as shown in FIG. 15(A) with the front side of the identification document in the photographing direction to the touch panel display 25.
The document position guide image 350 shown in FIG. 15(A) is an image that includes a document position area 350a in which an identification document is placed and a photographing button 350b. Further, the document position guide image 350 is a transparent image. This is to output together with the through image of the out camera 23b, and at least the document position area 350a needs to be transparent.

When the user places the identification document so that it is reflected in the document position area 350a and taps the shooting button 350b, the control unit 310 (identity identification image processing unit 313) performs identification in S341 of FIG. Get the image. Then, the control unit 310 causes the storage unit 320 to store the acquired identity verification image.
In S342, the control unit 310 (identity verification image processing unit 313) activates the out camera 23b again and causes the touch panel display 25 to display the through image of the out camera 23b again. Then, the control unit 310 outputs to the touch panel display 25, for example, a document position guide image 351 as shown in FIG. 15(B), which instructs to take an identification image including the side surface of the identification document.

The document position guide image 351 shown in FIG. 15(B) is an image including a document position area 351a in which an identification document is placed and a photographing button 351b. Further, like the document position guide image 350 described above, the document position guide image 351 is a transparent image. The document position area 351a specifies the photographing direction from an oblique direction so that the face photograph of the identity verification document can be confirmed and the thickness of the long side of the identity verification document can be confirmed. Therefore, the document position area 351a is shown as a trapezoid with a long base.
Note that this is just an example; for example, the document position area 351a may not be indicated by a frame line, but may be indicated by lines to indicate the positions of the two long sides of the identification document. Further, the document position area 351a may be used to confirm the thickness of the short side of the identification document.

When the user places the identification document so that it is reflected in the document position area 351a and taps the shooting button 351b, the control unit 310 (identity identification image processing unit 313) performs identification in S343 of FIG. Get the image. Then, the control unit 310 causes the storage unit 320 to store the acquired identity verification image.

In S344, the control unit 310 (identity verification image processing unit 313) confirms the acquired identity verification image based on the document position guide image.
For example, the control unit 310 determines that the identity verification document is within the document position area 350a and that the identity verification image has been acquired in a predetermined size, or that the acquired identity verification image is an identity verification document. Confirm that it is present (for example, that it is the front of driver's license 3). The control unit 310 performs this confirmation on all acquired personal identification images. The control unit 310 also checks whether the resolution of the acquired identity verification image satisfies a predetermined condition. For example, if the amount of light is insufficient due to camera shake or the like, the resolution of the identity verification image will no longer meet the predetermined conditions.

Note that if the identity verification image cannot be verified, that is, if the conditions for the identity verification image are not satisfied, the control unit 310 performs the process from S340 again until a predetermined number of retries. Good too. Furthermore, for example, if the number of retries exceeds a predetermined number, the control unit 310 may determine that it is an error and not perform subsequent processing.

In S345, the control unit 310 (identity verification image processing unit 313) extracts a face photo image from the identity verification image. For example, the control unit 310 extracts a face photo image from the identity verification image obtained by photographing the identity verification document obtained in S341 from the front.
In S346, the control unit 310 (identity verification image processing unit 313) stores the extracted facial photograph image in the storage unit 320. After that, the control unit 310 performs storage image processing (see S31 in FIG. 5), which will be described next.

In this example, a document position guide image 350 with the front side of the identification document in the photographing direction is first output and photographed, and then a document position guide image 351 with a different photographing direction and photographing position is output and photographed. has been explained using an example, but the invention is not limited to this. The order is not limited as long as one of the multiple shots of the identification document is taken with the front of the identification document facing the direction of the photograph. Further, the number of times of photographing is not limited to two, but may be three or more times.

Further, in the identity verification system 300, document position guide images in various photographing directions and photographing positions are stored in the storage unit 320, and the control unit 310 randomly selects a document position guide image from among the document position guide images. , may be output. Document position guide images in various photographing directions and photographing positions are, for example, a document position guide image 361 shown in FIG. 16(A) to a document position guide image 364 shown in FIG. 16(D).

The document position guide image 361 shown in FIG. 16(A) is a photographing direction in which the thickness of the identity verification document can be photographed, and a document position area 361a for placing the identity verification document at the top of the screen at a close distance and photographing it. include.
The document position guide image 362 shown in FIG. 16(B) is a photographing direction in which the thickness of the identity verification document can be photographed, and a document position area 362a for placing and photographing the identity verification document at the bottom of the screen at a close distance. include.
On the other hand, the document position guide image 363 shown in FIG. 16C and the document position guide image 364 shown in FIG. includes document position areas 363a and 364a for placing and photographing an identification document.
Note that the storage unit 320 may also store a document position guide image that is photographed so that the document position area is placed in the center of the screen.

Further, in this example, the identity verification image is confirmed after photographing the identity verification document from different photographing directions multiple times, but the present invention is not limited to this. The control unit 310 may check the identity verification image each time it acquires the identity verification image. Further, after acquiring the identity verification image, the control unit 310 causes the user to confirm the identity verification image, and when the user inputs an instruction to save the identity verification image, the controller 310 stores the acquired identity verification image and confirms the identity of the user. It may also be possible to retake it by

Next, the saved image processing performed in the identity verification processing will be explained based on FIG. 17.
FIG. 17 is a flowchart showing stored image processing in the identity verification system 300 according to the third embodiment.
S350 is similar to the process of S50 in the first embodiment (FIG. 7).
In S351, the control unit 310 (stored image processing unit 315), for example, specifies one document position guide image from among the plurality of document position guide images stored in the storage unit 320, and stores the specified document position guide image. , is displayed on the touch panel display 25. Here, the plurality of document position guide images stored in the storage unit 320 are those in which the guide positions of the identity verification documents are different, and the guide positions of the identity verification documents do not overlap with the guide positions of the face images. It is something that exists.
S352 is similar to the process of S50 in the first embodiment (FIG. 7).

In S353, the control unit 310 (stored image processing unit 315) removes the previously identified one document position guide image from among the plurality of document position guide images stored in the storage unit 320, and stores the document position guide images. One document position guide image is identified and the identified document position guide image is displayed on the touch panel display 25.
S354 is similar to the process of S352.
In S355, the control unit 310 (stored image processing unit 315) checks the acquired multiple stored images. Specifically, the control unit 310 confirms that the image of the identification document included in the acquired saved image and the user's face image roughly match the guide range of each document position guide image.

In S356, the control unit 310 (saved image processing unit 315) stores the stored image in the storage unit 320 on the condition that the stored image has been confirmed. In addition, if the saved image cannot be confirmed, that is, if the image of the identity verification document included in the saved image and the user's face image do not fit within the guide range of the guide image, an error will occur. Then, the subsequent processing may not be performed, or the processing from S351 and S353 may be performed again. At this time, the control unit 310 may output a document position guide image different from the document position guide images outputted so far, or may output the document position guide image that has caused an error again.

In S357, the control unit 310 (saved image processing unit 315) transmits the stored image stored in the storage unit 320 to the identity verification server 4. By doing so, the identity verification server 4 may also perform verification processing on the saved image. After that, the control unit 310 performs matching image processing (see S32 in FIG. 5), which will be described next.

Next, the verification image processing performed in the identity verification processing will be explained based on FIG. 18.
FIG. 18 is a flowchart showing verification image processing in the identity verification system 300 according to the third embodiment.
FIG. 19 is a flowchart showing the matching image acquisition process in the mobile terminal 301 according to the third embodiment.
FIG. 20 is a diagram showing a display example on the mobile terminal 301 according to the third embodiment.
In S350 of FIG. 18, the control unit 310 (matching image processing unit 316) of the mobile terminal 301 performs matching image acquisition processing.

Here, the matching image acquisition process will be explained based on FIG. 19.
In S370 of FIG. 19, the control unit 310 (verification image processing unit 316) activates the in-camera 23a and causes the touch panel display 25 to display a through image of the in-camera 23a. Then, the control unit 310 outputs, for example, a face photographing guide image 370 as shown in FIG. 20(A) to the touch panel display 25.
The face photographing guide image 370 shown in FIG. 20(A) is for obtaining a face image of the user from the front. Further, the face photographing guide image 370 is an image including a face position area 370a that specifies the position of the user's own face.

In S371 of FIG. 19, the control unit 310 (matching image processing unit 316) acquires a matching image and stores it in the storage unit 320. To obtain the matching image, the user moves the position of the mobile terminal 301 so that his or her own face is reflected in the face position area 370a, and taps a photographing button (not shown), so that the control unit 310 takes the photograph. The user's face image may be acquired by controlling the image forming apparatus. Further, the control unit 310 may acquire the user's face image by automatically capturing the image after a predetermined period of time (for example, 5 seconds, etc.) has elapsed since the face capture guide image 370 was output. . Furthermore, the control unit 310 may have a function of detecting the face and the blinking of the eyes, and the control unit 310 may perform control to capture the image after detecting the blinking, thereby obtaining the user's face image.

In S372, the control unit 310 (matching image processing unit 316) outputs an instruction to the user to change the direction of the face. Here, the instruction from the control unit 310 may be displayed as a face photographing guide image 371 shown in FIG. 20(B), for example, or may be output as a sound from a speaker (not shown). Although the face photographing guide image 371 shown in FIG. 20(B) directs the user to face to the right, it may direct the user's face to any direction, for example, up, down, left, or right.

In S373, the control unit 310 (matching image processing unit 316) acquires the matching image and stores it in the storage unit 320. In this shooting, since the user's face is not facing the touch panel display 25 of the mobile terminal 301, it is difficult for the user to tap a shooting button (not shown). Therefore, the control unit 310 outputs the face photographing guide image 371 and counts a predetermined period of time (for example, 5 seconds, etc.) by voice, and then performs control to automatically perform photographing to acquire the user's face image. You may. Further, the control unit 310 automatically controls the camera in response to detecting that a face is reflected in the face position area 371a within a predetermined time (for example, 5 seconds, etc.) after outputting the face photography guide image 371. The face image of the user may be obtained by controlling the camera to capture the image.

In S374, the control unit 310 (verification image processing unit 316) checks whether the acquired verification image complies with the corresponding instruction content. If the control unit 310 can confirm the confirmation, the control unit 310 moves the process to S361 in FIG. 18 . Note that if the confirmation cannot be made, the control unit 310 may not perform the subsequent processing.
The processing from S361 to S365 in FIG. 18 is similar to the processing from S61 to S65 in the first embodiment (FIG. 10).

As described above, according to the third embodiment, the identity verification system 300 has the following effects.
(1) The mobile terminal 301 obtains a plurality of identification images by photographing an identification document from a plurality of different photographing directions in which a face photograph can be confirmed. At least one of the plurality of identification images is configured to allow confirmation of the thickness of the identification document. Therefore, it becomes possible to determine whether the document is a true identity verification document based on the identity verification image, and security can be improved.
(2) The mobile terminal 301 is configured to output a document position guide image for obtaining an identity verification image. The document position guide image is an image that specifies the shooting direction in which the face photo can be confirmed and the placement position of the identity verification document, including the size of the identity verification image, so the user can confirm the identity according to the document guide image. Documents can be photographed. Furthermore, since the photographing direction and placement position are specified, it is no longer necessary to prepare identification documents and photograph on the spot, thereby preventing fraudulent acts such as using images prepared in advance.
(3) The mobile terminal 301 includes one that changes the direction of the user's face based on instruction information and photographs the user's face image. Therefore, it is necessary to take pictures according to the instruction information, and it is possible to prevent fraudulent acts such as using images prepared in advance.
Additionally, when the user changes the direction of his or her face, a voice instruction or an automatic photo shoot will be provided. Therefore, even when it is difficult for the user to perform an operation for photographing, it can be handled and the system can be highly convenient.

Although the embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above. Furthermore, the effects described in the embodiments are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments. In addition, although the embodiment mentioned above and the modification mentioned later can be used in combination suitably, detailed description is abbreviate|omitted.

(Deformed form)
(1) In each of the embodiments, the identity verification server performs the image matching process, and in the second embodiment, the identity verification server performs the character recognition process, but the present invention is not limited to this. For example, an image matching server that is communicably connected to an identity verification server and an OCR server is provided, and the image matching server is used to match facial images, and the OCR server is used to perform character recognition processing. There may be.
(2) In each embodiment, the processing using the identity verification server and the mobile terminal has been described, but the content of the processing performed by the identity verification server and the mobile terminal is not limited thereto. The identity verification server may perform processes other than image acquisition performed by the mobile terminal, or the processes performed by the identity verification server may be performed by the mobile terminal. For example, the verification process for each acquired image may be performed by an identity verification server. Furthermore, the verification process for each image may be performed by both the identity verification server and the mobile terminal.

(3) In each of the embodiments, an example has been described in which the identity verification process involves processing the identity verification image and then processing the stored image, but the order of processing may be reversed.
(4) In each embodiment, an example has been described in which the transaction application is executed and then the identity verification application is started, but the invention is not limited to this. The identity verification application may be executed independently.
(5) In each embodiment, no special processing is performed on the identity verification documents included in the identity verification image and the identity verification documents included in the saved image, but they may be checked to see if they are the same. . Furthermore, when a plurality of images are acquired in each process, it may be confirmed that the subjects are the same.

(6) In the third embodiment, a case has been described in which a plurality of identity verification images are acquired and a facial photograph image is extracted from the identity verification images photographed from the front, but the present invention is not limited to this. A face photograph image may be extracted from an identification image obtained by photographing from a direction in which the thickness of the identification document can be confirmed. In that case, for example, a known image correction technique may be used to correct distortion of an image that looks like it was taken from the front, and the image may be used for face matching processing. Further, this correction technique may be used to confirm that a plurality of identity verification images are images of the same identity verification document.

(7) In the third embodiment, a guide image is output during photographing. At that time, although there is no special mention of the time from when the guide image is output to when it is photographed, the control unit will treat it as an error and prevent further processing if the guide image is not photographed within a predetermined time. You can also do this.

(8) A variety of services are applicable for identity verification and identity authentication in each embodiment. For example, in the case of banking, it can be used for account opening and account linking processes, cash withdrawals using ATMs, and the like. When using an ATM, you authenticate your identity with a mobile terminal, and if you are authenticated, display a QR code (registered trademark) that is read by the ATM, thereby using the mobile terminal as a substitute for a cash card. I can do it. In addition, when an agent performs work at a counter on behalf of the principal, the agent can receive information proving that the principal has authenticated the person's identity, thereby eliminating the need to prepare a power of attorney in advance. It's over.

1,201,301 Mobile terminal 3 Driver's license 4,204 Identity verification server 7 Service providing server 10, 40, 210, 240, 310 Control unit 12,212,312 Identity verification unit 13,213,313 Identity verification image processing unit 14 Guide image output unit 15,315 Saved image processing unit 16,316 Verification image processing unit 17 Identity authentication unit 18 Service start determination unit 20, 50, 220, 250, 320 Storage unit 21a Transaction application 21b, 221b, 321b Identity verification application 23 Camera 25 Touch panel display 43 Face verification unit 44 Verification result transmission unit 100, 200, 300 Identity verification system 241 Identity verification image analysis unit

Claims (9)

A mobile terminal having a photographing section,
The data necessary for verifying the user's identity at the time of starting the service is acquired using only the mobile terminal owned by the user,
an identity verification image acquisition means for acquiring an identity verification image, which is an image of an identity verification document on which a face photograph is posted, from the photographing unit;
a stored image acquisition means for acquiring a stored image, which is an image including both the identification document and the face of the holder of the identification document, from the photographing unit;
stored image processing means for storing the stored image acquired by the stored image acquisition means in a storage unit;
a verification image acquisition means for acquiring a verification image that is an image including the face of the owner from the photographing unit;
On the condition that the stored image is stored in the storage unit, the identity verification image acquired by the identity verification image acquisition means and the verification image acquired by the verification image acquisition means are stored using images. Verification image transmission means for transmitting to an identity verification server that performs identity verification;
A mobile terminal equipped with The mobile terminal according to claim 1,
means for receiving verification result information using the identity verification image and the verification image from the identity verification server, and determining whether or not authentication is successful based on the verification result information;
means for deleting the stored image stored in the storage unit when it is determined that the authentication cannot be performed;
A mobile terminal equipped with In the mobile terminal according to claim 1 or claim 2 ,
a display unit that displays an image of the photographing range by the photographing unit;
A guide image indicating a placement range of the identity verification document and the face of the holder , one guide image identified from a storage unit that stores a plurality of guide images having different positions in the placement range of the identity verification document. , a guide image output means for outputting to the display section,
A mobile terminal equipped with. The mobile terminal according to claim 3 ,
Each of the plurality of guide images is one in which the arrangement range of the identification document does not overlap with the position of the face, and is different in the vertical and horizontal directions and the depth direction with respect to the photographing position, and A mobile terminal in which an identity verification document is shown in a visible size range larger than a predetermined size. The mobile terminal according to claim 3 or 4 ,
Confirming the position of the image of the identity verification document included in the stored image acquired by the stored image acquisition means and the face image of the holder based on the guide image outputted by the guide image output means. Equipped with image confirmation means ,
The stored image processing means stores the stored image in the storage unit when the stored image confirmation means can confirm the stored image . The mobile terminal according to any one of claims 1 to 5 ,
In the mobile terminal, the stored image acquisition means acquires a plurality of stored images in which the identification documents are arranged in different positions from the photographing unit. The mobile terminal according to any one of claims 1 to 6 ,
comprising a facial photograph image extraction means for extracting a facial photograph image from the personal identification image acquired by the personal identification image acquisition means;
In the mobile terminal, the verification image transmission means transmits the face photograph image extracted by the face photograph image extraction means and the verification image to the identity verification server. A mobile terminal according to any one of claims 1 to 7 ,
an identity verification server that is communicably connected to the mobile terminal and performs identity verification processing ;
An identity verification system equipped with
The identity verification server is
face matching means for matching the matching image received from the mobile terminal with a face photo image included in the personal identification image;
result transmitting means for transmitting matching result information by the face matching means to the mobile terminal;
Equipped with
The mobile terminal is
If the identity has not been verified, executing processing by the identity verification image acquisition means, the stored image acquisition means, the stored image processing means, the verification image acquisition means, and the verification image transmission means;
a service start determination that determines whether to start service processing with a service providing server that is communicably connected to the mobile terminal and the identity verification server and that performs service processing, based on the verification result information; An identity verification system with means. A computer owned by the user, which has a photographing unit and is used to confirm the user's identity at the time of starting the service,
an identity verification image acquisition means for acquiring an identity verification image, which is an image of an identity verification document on which a face photograph is posted, from the photographing unit;
a stored image acquisition means for acquiring a stored image, which is an image including both the identification document and the face of the holder of the identification document, from the photographing unit;
stored image processing means for storing the stored image acquired by the stored image acquisition means in a storage unit;
a verification image acquisition means for acquiring a verification image that is an image including the face of the owner from the photographing unit;
On the condition that the stored image is stored in the storage unit, the identity verification image acquired by the identity verification image acquisition means and the verification image acquired by the verification image acquisition means are stored using images. Verification image transmission means for transmitting to an identity verification server that performs identity verification;
A program to make it work.

Images