JP4803604B2 - Face recognition method and program for operating on web browser - Google Patents

Description

  The present invention relates to a face recognition method that operates on a standard Web browser using the Internet and a program thereof.

As the prior art, Patent Document 1 and the like describe that an image processing service is provided by linking software running on the user side and a server, and Patent Document 2 and the like recognize based on images. A device that performs device control via a computer is described.
In addition, some technologies that can use the face recognition function without installing software on the user side are already known, but they transmit raw video information to the face recognition software that operates on the server side. Is realized.
JP 2006-287893 A Japanese translation of PCT publication No. 2004-511870

  However, the techniques described in Patent Document 1 and Patent Document 2 require a procedure for installing software on a user's computer, and the labor involved in acquiring many users is a major issue. . In addition, as described above, when unprocessed video information is sent to the face recognition software that runs on the server side, the server becomes overloaded when many users use it at the same time. There is a problem that it is difficult to obtain a face recognition result in real time due to a problem that the communication capacity with a computer on the user side is limited.

  In view of the above problems, an object of the present invention is to automatically download face recognition software running on a user side computer through the Internet, and to download the face recognition software on an AVM (Action Script Virtual Machine: action) on the browser of the user side computer. By using an execution environment called a script virtual machine) and combining this face recognition software with the software that sequentially transfers the face recognition software to the AVM and the server, the user can install no new software on the user side computer. A face recognition method that operates on a Web browser that operates in real time and a program thereof.

The present invention employs the following means in order to solve the above problems.
The first means operates on the web browser, wherein the face recognition software is provided on the AVM of the web browser from a software server storing the face recognition software in response to a request from the web browser This is a face recognition method.
The second means obtains information described in HTML stored in the Web server in response to a request from the Web browser, and transmits the information from the Web browser to the software server storing the face recognition software. A face recognition method operating on a web browser, wherein the face recognition software is automatically downloaded from the software server onto the AVM of the web browser.
According to a third means, in the first means or the second means, the face recognition software provided on the AVM operates on a Web browser characterized by operating under an execution environment of the AVM. This is a face recognition method.
The fourth means is any one of the first means to the third means, wherein the software server generates a random number, a function that stores the generated random number in a storage area on the software server, The face recognition software provided on the AVM from the software server in response to a request from the web browser is a face recognition software in which the random number is embedded. It is face recognition software characterized by being.
A fifth means is any one of the first means to the fourth means, wherein the face recognition software captures a video from a video input device connected to a user computer and uses it by a face detection function. This is a face recognition method that operates on a Web browser, which has a function of detecting a person's face position and extracting a specified face feature from a range near the face position.
A sixth means is the face recognition method operating on a Web browser, characterized in that, in the fifth means, the face recognition software has a function of transmitting the extracted face features to the software server. It is.
Seventh means is the software server according to the fifth means, wherein the face recognition software includes the same random number as the random number embedded when the face recognition software is provided to the AVM together with the extracted facial features. It is a face recognition method which operates on a Web browser characterized by having a function of transmitting to
According to an eighth means, in the fifth means, the face recognition software encrypts the extracted facial features with the same random number as the random number embedded when the face recognition software is provided to the AVM. A face recognition method operating on a Web browser, which has a function of transmitting to the software server.
A ninth means is a face recognition method operating on a Web browser, characterized in that, in the sixth means, the software server has a function of collating using the received extracted facial features. is there.
According to a tenth means, in the seventh means, the software server receives the received facial feature and the random number, and collates the received random number with a random number stored in the storage area of the software server. This is a face recognition method that operates on a Web browser, characterized in that it performs matching of face features.
According to an eleventh means, in the seventh means or the eighth means, the software server decrypts the received encrypted facial feature with a random number stored in the storage area of the software server. This is a face recognition method that operates on a Web browser characterized by performing collation.
A twelfth means is the Web according to any one of the ninth means to the eleventh means, wherein the software server has a function of executing different operations depending on a result of collation of the facial features. This is a face recognition method that operates on a browser.
A thirteenth means is a face recognition method operating on a web browser, characterized in that, in the fifth means, the face recognition software has a function of performing collation using the extracted face features. .
A fourteenth means is a face recognition method operating on a web browser, characterized in that, in the thirteenth means, the face recognition software has a function of executing different operations depending on the result of collation of the face features. .
The fifteenth means is a computer program that describes a control procedure for causing a computer to execute the face recognition method that operates on the Web browser according to any one of the first to fourteenth means.
A sixteenth means is a recording medium storing the computer program according to the fifteenth means.

  Many Web browsers currently in common use have an AVM execution environment in advance, and users can use face recognition software that operates in real time without the hassle of installing a standard Web browser. be able to. Service providers can significantly reduce the cost of distributing facial recognition software. Both user and provider costs can be reduced to almost zero, and the spread of face recognition devices that operate in real time can be greatly spurred.

A first embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a diagram showing a network configuration to which a face recognition method operating on a Web browser and a program thereof according to the present embodiment and a second embodiment described later are applied.
In the figure, 1 is the Internet, 2 is a Web server connected to the Internet 1, 3 is a software server connected to the Internet 1, 4 is a user side computer connected to the Internet 1, and 5 is a user side computer 4. A camera 6 is a user who uses the user side computer 4 and the camera 5.

2 is a functional block diagram showing the detailed configuration of the Web server 2, software server 3, and user computer 4 shown in FIG. 1, and FIG. 3 shows the relationship between the Web server 2, software server 3, and user computer 4. It is a figure which shows the process sequence in.
In these figures, the input from the user 6 is understood as an HTTP request by the user input interpretation function 411 of the Web browser 41 of the user side computer 4. Here, the HTTP request is a URL that is information for identifying the location of each server on the Internet 1 and information for transmitting / receiving information on the Internet 1 including instructions such as whether to transmit or receive information. This is a standard communication procedure. The HTTP request generated here is notified to the Web server 2 through the Internet 1 from the HTTP request generation function 412 of the Web browser 41, and the request content is interpreted by the HTTP request interpretation function 21 of the Web server 2. In the Web server 2, the HTML data indicated by the request is retrieved from the HTML data storage function 23 by the HTML data retrieval function 22, and is transmitted to the user side computer 4 by the HTML data transmission function 24.

  The content of the HTML data received by the Web browser 41 of the user side computer 4 is interpreted by the HTML data interpretation function 413, and an HTTP request related to software download is generated from the software information described in the HTML. Generated at function 414. The HTTP request generated by the Web browser 41 is notified to the software server 3 through the Internet 1.

  The HTTP request notified to the software server 3 is interpreted by the HTTP request interpretation function 31. Further, the software data retrieval function 32 extracts the software data indicated by the request from the software data storage function 33. In the subsequent procedure, communication between the transmitted software data and the software server 3 is performed, but different software data having a communication function equivalent to the transmitted software data is impersonated as the software data and Since there is a risk of communication, in the present invention, it is also possible to embed random numbers in order to prevent communication by impersonation. Therefore, the random number generated by the random number generation function 34 is stored on the software server 3 by the random number storage function 35 and embedded in the software data extracted from the software data storage function 33 by the random number embedding function 36. The data is transmitted from the software data transmission function 37 to the user side computer 4 together with the software data. Note that the above random number may be used for encryption of communication contents between the user side computer 4 and the software server 3.

  The software data received by the user computer 4 is transferred to the software data storage area 421 of the AVM 42 inside the browser by the data transfer function 415 of the Web browser 41. When the transfer of the software data is completed, the execution control function 422 of the AVM 42 automatically starts executing the stored software.

  Under the AVM execution environment 423 of the AVM 42, the transmitted software acquires an image from the camera 5 by using an interface function with the camera 5 provided as a standard in the AVM 42, and the face detection function 424 uses the image to detect the image in the image. Detect face parts. In the software, a feature extraction function 425 including a procedure for extracting some features useful for collation from a face image is programmed in advance, and facial features are extracted from the face area detected by the feature extraction function 425. The extracted feature data is embedded by the random number embedding function 426 with the same random number as the random number embedded in the software data at the time of reception, and transmitted to the software server 3.

  The feature data embedded with the random number received by the software server 3 is subjected to interpretation of communication contents by the HTTP communication interpretation function 38. The random number embedded in the feature data is collated with the random number stored in the random number storage function 35 in the random number collation function 39, and when both random numbers match, the feature is discriminated by the pre-programmed feature collation function 40. . Based on the result of the feature discrimination, the control function 50 executes different operations.

FIG. 4 is a flowchart showing a processing procedure in the AVM 42 in the software server 3 and the Web browser 41 of the user side computer.
First, in step S1, an image is acquired from the camera 5 using an interface function with the camera 5 provided as a standard in the AVM 42 under the AVM execution environment 423 of the AVM 42. Next, in step S2, face detection is performed by the face detection function 424. Next, in step S3, it is determined whether face detection has been performed normally. If it has been performed normally, facial features are extracted by the feature extraction function 425 in step S4. In step S 5, the extracted facial feature data is transmitted to the software server 3 through the Internet 1. Next, in step S <b> 6, the random number embedded in the feature data received by the software server 3 is compared with the random number stored in the random number storage function 35 of the software server 3. Next, if the two random numbers match in step S7, facial features are discriminated in step S8 by the pre-programmed feature matching function 40. If it is determined in step S9 that the facial features are normal, a predetermined operation 1 is performed in step S10. If the face detection is not normally performed in step S3, operation 4 is performed in step S11, and the process of step S1 is performed again. If the two random numbers do not match in step S7, operation 3 is performed and the process is terminated. In step S9, if the facial feature is not properly determined, a predetermined operation 2 is performed in step S13, and the process of step S1 is performed.
The present invention can be used to create various services. For example, when used for a service for authenticating a user, the user is led to an authenticated page by operation 1. In operation 2, a page explaining that the user is not permitted to use any more services is displayed on the Web browser. In operation 3, it is assumed that unauthorized access has been made to the service, a warning is displayed on the Web browser, and information about the access source is recorded. In operation 4, the service remains in a standby state.

Next, a second embodiment of the present invention will be described with reference to FIG. 1 and FIGS.
FIG. 5 is a functional block diagram showing detailed configurations of the Web server 2, the software server 3 and the user computer 4 shown in FIG. 1, and FIG. 6 is a diagram between the Web server 2, the software server 3 and the user computer 4. It is a figure which shows the process sequence in.

  In these drawings, the input from the user 6 is understood as an HTTP request by the user input interpretation function 411 of the Web browser 41 of the user side computer 4. The HTTP request generated here is notified to the Web server 2 through the Internet 1 from the HTTP request generation function 412 of the Web browser 41, and the request content is interpreted by the HTTP request interpretation function 21 of the Web server 2. In the Web server 2, the HTML data indicated by the request is retrieved from the HTML data storage function 23 by the HTML data retrieval function 22, and is transmitted to the user side computer 4 by the HTML data transmission function 24.

  The content of the HTML data received by the Web browser 41 of the user side computer 4 is interpreted by the HTML data interpretation function 413, and the software information described in the HTML is sent to the display generation function 417. An HTTP request related to software download is generated in the HTTP request generation function 414 from the software information. The HTTP request generated by the Web browser 41 is transmitted to the software server 3 through the Internet 1.

  The HTTP request received by the software server 3 is interpreted by the HTTP request interpretation function 31. In the software server 3, the software data retrieval function 32 extracts the software data indicated by the request from the software data storage function 33. The software data extracted from the software data storage function 33 is transmitted from the software data transmission function 37 to the user computer 4.

  The software data received by the user computer 4 is transferred to the software data storage area 421 of the AVM 42 inside the browser by the data transfer function 415 of the Web browser 41. When the transfer of the software data is completed, the execution control function 422 of the AVM 42 automatically starts executing the stored software.

  Under the AVM execution environment 423 of the AVM 42, an image is acquired from the camera 5 by using the interface function with the camera 5 that the AVM 42 has as a standard by the transmitted software, and the face in the image is detected from the image by the face detection function 424. Detect the part. In the software, a feature extraction function 425 including a procedure for extracting some features useful for collation from a face image is programmed in advance, and facial features are extracted from the face area detected by the feature extraction function 425. The extracted features are distinguished by a pre-programmed feature matching function 426. Based on the result of the feature determination, the control function 50 controls the display generation function 417 of the browser. As a result, the user 6 can obtain different display results depending on the face matching result.

FIG. 7 is a flowchart showing a processing procedure in the AVM 42 in the software server 3 and the Web browser 41 of the user side computer.
First, in step S11, an image is acquired from the camera 5 using an interface function with the camera 5 provided as a standard in the AVM 42 under the AVM execution environment 423 of the AVM 42. Next, in step S12, face detection is performed by the face detection function 424. Next, in step S13, it is determined whether face detection has been performed normally. If it has been performed normally, the facial feature is extracted by the feature extraction function 425 in step S14. Next, in step S15, facial feature discrimination is performed by a pre-programmed feature matching function 426. If it is determined in step S16 that the facial features are normal, a predetermined operation 1 is performed in step S17. If the face detection is not normally performed in step S13, operation 3 is performed in step S18, and the process of step S11 is performed again. In step S16, if the facial feature is not normally determined, a predetermined operation 2 is performed in step S19, and the process in step S11 is performed.
The present invention can be used to create various services. For example, when used for a service for authenticating a user, the user is led to an authenticated page by operation 1. In operation 2, a page explaining that the user is not permitted to use any more services is displayed on the Web browser. In operation 3, the service remains in a standby state. When the present invention is used for a service for presenting advertisements to a user, an operation 1 displays an advertisement directed to a user within a specific range on the Web browser. In operation 2, a general advertisement is displayed. In operation 3, an advertisement for attracting a user is displayed.

  In each of the above embodiments, the functions of HTTP interpretation, data search, and data transmission in the Web server and software server are implemented using software called Apache, and are written in the Ruby language executed on Apache. Data is managed by the script. The embedding of random numbers in the first embodiment is realized by using the function of flex2 which is an actionscript compiler, and the algorithm of RSA encryption is used for encrypting data using random numbers. For the face detector, a detection method using a cascade discriminator using Haar features was used, and the Haar feature extraction method was used for feature extraction. For the feature matching, an SVM algorithm was used.

It is a figure which shows the network structure to which the distribution method of the face recognition software which operate | moves on the web browser which concerns on invention of 1st Embodiment and 2nd Embodiment, and its program are applied. FIG. 4 is a functional block diagram illustrating detailed configurations of a Web server 2, a software server 3, and a user computer 4 according to the invention of the first embodiment. 6 is a flowchart illustrating a processing procedure among the Web server 2, the software server 3, and the user computer 4 according to the invention of the first embodiment. It is a figure which shows the process sequence in AVM42 in the web browser 41 of the software server 3 and a user side computer according to invention of 1st Embodiment. FIG. 10 is a functional block diagram illustrating detailed configurations of a Web server 2, a software server 3, and a user computer 4 according to the invention of the second embodiment. It is a figure which shows the process sequence between the web server 2, the software server 3, and the user side computer 4 in connection with invention of 2nd Embodiment. It is a flowchart which shows the process sequence in AVM42 in the web browser 41 of the software server 3 and a user side computer based on invention of 2nd Embodiment.

Explanation of symbols

1 Internet 2 Web server 21 HTTP request interpretation function 22 HTML data search function 23 HTML data storage function 24 HTML data transmission function 3 Software server 31 HTTP request interpretation function 32 Software data search function 33 Software data storage function 34 Random number generation function 35 Random number storage Function 36 Random number embedding function 37 Software data transmission function 38 HTTP communication interpretation function 39 Random number collation function 40 Feature collation function 50 Control function 4 User computer 41 Web browser 411 User input interpretation function 412 HTTP request generation function 413 HTML data interpretation function 414 HTTP request generation function 415 Data transfer function 416 Display generation function 42 AVM
421 Software data storage area 422 Execution control function 423 AVM execution environment 424 Face detection function 425 Feature extraction function 426 Random number embedding function 5 Camera 6 User

Claims (11)

In response to a request from a web browser, the face recognition software is provided on the AVM of the web browser from a software server storing the face recognition software, and the face recognition software is connected to a user computer. A video is captured from the device, a face position is detected from the video by a face detection function, a specified facial feature is extracted from a range near the facial position, the extracted facial feature is transmitted to the software server, and the software server Is a face recognition method operating on a Web browser, characterized in that matching is performed using the received extracted facial features . In response to a request from the Web browser, information described in HTML stored in the Web server is obtained, and the information is transmitted from the Web browser to the software server storing the face recognition software. The face recognition software is automatically downloaded onto the AVM of the web browser , the face recognition software captures a video from a video input device connected to a user computer, detects a face position by a face detection function, Extracting a specified facial feature from a range near the face position, transmitting the extracted facial feature to the software server, and the software server performing matching using the received extracted facial feature. A face recognition method that operates on a characteristic Web browser. 3. The face recognition method that operates on a web browser according to claim 1, wherein the face recognition software provided on the AVM operates under an execution environment of the AVM. The software server has a function of generating a random number, a function of storing the generated random number in a storage area on the software server, and a function of embedding the same random number in the face recognition software, according to a request from the Web browser 4. The face recognition software provided on the AVM from the software server is face recognition software in which the random number is embedded, according to any one of claims 1 to 3. A face recognition method that operates on a web browser. The face recognition software has a function of transmitting, to the software server, the same random number as the random number embedded when the face recognition software is provided to the AVM together with the extracted face feature. Item 5. A face recognition method that operates on a Web browser according to any one of Items 1 to 4 . The face recognition software has a function of encrypting the extracted face feature with the same random number embedded in the AVM when the face recognition software is provided and transmitting the encrypted feature to the software server. A face recognition method that operates on a Web browser according to claim 1 . The software server receives the received facial feature and random number, collates the received random number with a random number stored in the storage area of the software server, and performs facial feature matching A face recognition method that operates on a Web browser according to claim 5 . 7. The software server according to claim 5 or 6 , wherein the software server performs verification after decrypting the received encrypted face feature with a random number stored in a storage area of the software server. Face recognition method that operates on the web browser of The face operating on a Web browser according to claim 7 or 8 , wherein the software server has a function of executing different operations depending on a result of collation of the face feature. Recognition method. A computer program describing a control procedure for causing a computer to execute the face recognition method operating on the Web browser according to any one of claims 1 to 9 . A recording medium storing the computer program according to claim 10 .

Images