JP6747112B2 - Information processing system, image processing device, information processing device, and program - Google Patents

Description

The present application relates to an information processing system, an image processing device, an information processing device, and a program.

As an authentication method for authenticating a user, an image authentication technique such as face authentication, which does not require an operation such as a password input and can prevent spoofing due to loss or theft of an IC card, is widely used.

In addition, by extracting the areas such as eyes and mouth of the face image captured by the camera and determining whether the areas are changing in the time direction, it is a photograph whether the captured face is an actual face. There is known a biometric determination technique for determining whether or not it is (see, for example, Patent Document 1).

For example, in an information processing device that performs face authentication, by using biometric determination as disclosed in Patent Document 1, it is possible to reduce spoofing due to a facial photograph or the like.

However, for example, in the conventional biometric determination technique as disclosed in Patent Document 1, when determining the movements of various people, it is possible to suppress a decrease in the determination accuracy and reduce the time required for the determination. Was accompanied by difficulties.

The embodiment of the present invention is made in view of the above problems, in an information processing system for performing a biometric determination to determine whether the person photographed in the image is a biometric, the accuracy of the biometric determination It is an object of the present invention to reduce the time required for biometric determination while suppressing the decrease of

In order to solve the above problems, an information processing system according to an embodiment of the present invention uses information on a living body identification space extracted in advance from an image of a living body to determine whether the person photographed in the image is a living body. An information processing system for determining a living body, the image capturing unit capturing an image, a feature information extracting unit extracting feature information of a person captured in the image, and the feature information extracting unit extracting the feature information. A first storage unit that stores information of unnecessary elements that are not used for the biometric determination among the elements of the first projection vector that projects the characteristic information onto the discrimination space that includes the identification space of the biological body; A living body for determining whether or not the person photographed in the image is a living body by using a second projection vector obtained by removing the unnecessary element from the projection vector and the characteristic information extracted by the characteristic information extracting unit. And a determination unit.

According to the embodiment of the present invention, in the information processing system that performs the biometric determination to determine whether the person imaged in the image is a biometric, it is necessary for the biometric determination while suppressing a decrease in accuracy of the biometric determination. The time can be shortened.

It is a figure which shows the structural example of the information processing system which concerns on one Embodiment. FIG. 3 is a diagram showing an example of a hardware configuration of an image forming apparatus according to an embodiment. It is a figure which shows the hardware structural example of the information processing apparatus which concerns on one Embodiment. FIG. 3 is a diagram showing an example of a functional configuration of the image forming apparatus according to the first embodiment. 6 is another example of the functional configuration of the image forming apparatus according to the first embodiment. It is a figure showing an example of functional composition of an information processor concerning a 1st embodiment. It is a figure which shows the example of the face image which concerns on 1st Embodiment. It is a figure for demonstrating the HALC feature-value which concerns on 1st Embodiment. It is a figure for demonstrating the CHLAC feature-value which concerns on 1st Embodiment. It is a figure for explaining an example of discriminant analysis concerning a 1st embodiment. It is a figure for demonstrating another example of the discriminant analysis which concerns on 1st Embodiment. It is a figure which shows the example of the projection vector which concerns on 1st Embodiment. It is a figure which shows the example of the unnecessary element of the projection vector which concerns on 1st Embodiment. 6 is a flowchart showing an example of unnecessary element extraction processing according to the first embodiment. 6 is a flowchart showing an example of biometric determination processing of the image forming apparatus according to the first embodiment. It is a figure for demonstrating the subspace method which concerns on 2nd Embodiment. It is a figure which shows the image of the partial space of biometric information and non-biometric information which concern on 2nd Embodiment. It is a figure which shows the image of the subspace obtained by the principal component analysis which concerns on 2nd Embodiment. It is a figure which shows the image of the subspace on the complementary space which concerns on 2nd Embodiment. It is a figure which shows the example of the projection vector which concerns on 2nd Embodiment. It is a figure which shows the example of the unnecessary element of the projection vector which concerns on 2nd Embodiment. 9 is a flowchart showing an example of unnecessary element extraction processing according to the second embodiment. It is a flow chart which shows example (1) of living body judging processing concerning other embodiments. It is a figure which shows the example of the setting screen of the biometric determination mode which concerns on other embodiment. It is a flow chart which shows example (2) of living body judgment processing concerning other embodiments.

Embodiments of the present invention will be described below with reference to the accompanying drawings.

<System configuration>
FIG. 1 is a diagram illustrating a configuration example of an information processing system according to an embodiment. The information processing system 100 captures an image of an image forming apparatus 101 such as an MFP (Multifunction Peripheral/Product) having a plurality of functions such as printing, copying, scanning, and facsimile transmission/reception, and a user 103 of the image forming apparatus 101, as an example. And a camera 102.

An image forming apparatus (an example of an image processing apparatus) 101 uses a camera 102 to capture a 103 image of a user who uses the image forming apparatus 101, and extracts a face image of the user 103. Further, the image forming apparatus 101 performs face authentication of the user 103 using the extracted face image, for example.

In addition, the image forming apparatus 101 prevents the person (for example, the user 103) whose image is captured by the camera 102 from being a living body in order to prevent authentication breakthrough by spoofing due to, for example, holding the photograph of another person over the camera 102. Biometric determination is performed to determine whether or not

For example, the image forming apparatus 101 has biometric information that is information on the identification space of the living body extracted in advance from the image of the living body, and information on the identification space of the non-living body extracted from the image of the non-living body (such as a photograph). The non-biological information is stored in the storage unit in advance. The image forming apparatus 101 also extracts the characteristic information of the user 103 from the image captured by the camera 102, and calculates the distance and the similarity between the extracted characteristic information and the biometric information and the non-biometric information stored in the storage unit. Based on this, it is determined whether or not the person imaged in the image is a living body.

Further, the image forming apparatus 101 according to the present embodiment prestores information of an unnecessary space (hereinafter referred to as an unnecessary element) that is not used for the biometric determination among the identification spaces used for the biometric determination, and Except for this, extraction of feature information and calculation of similarity are executed.

As the information of the unnecessary element stored in the image forming apparatus 101, for example, the information of the unnecessary element previously created by another information processing apparatus can be stored in the storage unit of the image forming apparatus 101. Alternatively, the image forming apparatus 101 may have a function of generating information about unnecessary elements.

The image forming apparatus 101 according to the present embodiment selects unnecessary elements from the first projection vector that projects the characteristic information extracted from the captured image onto the discrimination space including the living body identification space and the non-living body identification space. The biometric determination of the feature information is performed using the removed second projection vector.

Preferably, the image forming apparatus 101 extracts the feature information excluding unnecessary elements when extracting the feature information extracted from the captured image.

Accordingly, the image forming apparatus 101 can reduce the process related to the unnecessary element in the feature information extraction process and the biometric determination process, and thus can speed up the biometric determination process. Further, as an unnecessary element that is not used for the biometric determination, an element that does not affect (or has little influence on) the determination result of the biometric determination is stored in advance, so that the accuracy of the biometric determination can be prevented from decreasing.

As described above, according to the present embodiment, in the information processing system 100 that performs the biometric determination that determines whether or not the person imaged in the image is the biometric, the biometric determination is performed while suppressing a decrease in accuracy of the biometric determination. The time required for can be shortened.

The configuration of the information processing system 100 shown in FIG. 1 is an example. For example, the image forming apparatus 101 may be another image processing apparatus that performs face authentication, such as a PC (Personal Computer), a tablet terminal, a smartphone, a video conference apparatus, a time and attendance management apparatus, an ATM (Automatic teller machine), or a navigation apparatus. It may be an image processing device.

In addition, the information processing system 100 may include another information processing device that creates the unnecessary space information described above.

<Hardware configuration>
(Hardware configuration of image forming apparatus)
FIG. 2 is a diagram illustrating an example of a hardware configuration of the image forming apparatus 101 according to the embodiment. The image forming apparatus 101 includes, for example, a main body 210 that realizes various image forming functions such as a copy function, a scanner function, a fax function, and a printer function, and an operation unit 220 that receives user operations. Note that accepting a user's operation is a concept including accepting information (including a signal indicating a coordinate value of a screen) input according to the user's operation. The main body 210 and the operation unit 220 are communicably connected to each other via a dedicated communication path 240. As the communication path 240, for example, a USB (Universal Serial Bus) standard can be used, but any standard may be used regardless of whether it is wired or wireless.

The main body 210 can perform an operation according to the operation received by the operation unit 220. Further, the main body 210 can communicate with an external device such as a client PC (personal computer), and can also perform an operation according to an instruction received from the external device.

First, the hardware configuration of the main body 210 will be described. As shown in FIG. 2, the main body 210 includes a CPU (Central Processing Unit) 211, a ROM (Read Only Memory) 212, a RAM (Random Access Memory) 213, a storage unit 214, a communication I/F (Interface) 215, and a connection I. /F216, the engine part 217, the human body detection sensor 218, and the system bus 219.

The CPU 211 controls the overall operation of the main body 210 by executing a program stored in the ROM 212, the storage unit 214, or the like, using the RAM 213 as a work area (work area). For example, the CPU 211 uses the engine unit 217 to realize various functions such as the above-described copy function, scanner function, fax function, and printer function.

The ROM 212 is, for example, a non-volatile memory that stores a BIOS (Basic Input/Output System) executed when the main body 210 is activated, various settings, and the like. The RAM 213 is a volatile memory used as a work area or the like of the CPU 211. The storage unit 214 is, for example, a non-volatile storage device that stores an OS (Operating System), application programs, various data, and the like, and is configured by, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like. It

The communication I/F 215 is a network interface for connecting the main body 210 to the network 250 and performing communication with an external device connected to the network 250. The connection I/F 216 is an interface for communicating with the operation unit 220 via the communication path 240.

The engine unit 217 is hardware for performing general-purpose information processing and processing other than communication for realizing functions such as a copy function, a scanner function, a fax function, and a printer function. The engine unit 217 includes, for example, a scanner (image reading unit) that scans and reads an image of a document, a plotter (image forming unit) that prints on a sheet material such as paper, and a fax unit that performs fax communication. .. Further, the engine unit 217 may include a specific option such as a finisher for sorting printed sheet materials and an ADF (automatic document feeder) for automatically feeding a document.

The human body detection sensor 218 is a sensor that detects a human body within a detection range around the image forming apparatus 101, and, for example, a pyroelectric sensor or the like is used.

The system bus 219 is connected to each of the above components and transmits an address signal, a data signal, various control signals, and the like.

Next, the hardware configuration of the operation unit 220 will be described. As shown in FIG. 2, the operation unit 220 includes a CPU 221, a ROM 222, a RAM 223, a flash memory 224, a communication I/F 225, an operation panel 226, a connection I/F 227, an internal camera 228, an external connection I/F 229, and a system bus 230. And so on.

The CPU 221 controls the overall operation of the operation unit 220 by executing a program stored in the ROM 222, the flash memory 224, or the like using the RAM 223 as a work area (work area).

The ROM 222 is, for example, a non-volatile memory that stores a BIOS executed when the operation unit 220 is activated, various settings, and the like. The RAM 223 is a volatile memory used as a work area of the CPU 221 or the like. The flash memory 224 is, for example, a non-volatile storage device that stores an OS, application programs, various data, and the like.

The communication I/F 225 is a network interface for connecting the operation unit 220 to the network 250 and performing communication with an external device connected to the network 250.

The operation panel 226 receives various inputs according to the user's operation and displays various information (for example, information according to the received operation, information indicating the operation status of the image forming apparatus 101, setting status, etc.). The operation panel 226 is composed of, for example, a liquid crystal display (LCD) having a touch panel function, but is not limited to this. The operation panel 226 may be composed of, for example, an organic EL (Electro Luminescence) display device having a touch panel function. Further, in addition to or instead of this, the operation panel 226 can be provided with an operation unit such as a hardware key and a display unit such as a lamp.

The connection I/F 227 is an interface for communicating with the main body 210 via the communication path 240.

The internal camera 228 is a camera that captures an image around the image forming apparatus 101, and is an example of the camera 102 in FIG. 1.

The external connection I/F 229 is an interface such as a USB for connecting an external device. The external device connected to the external connection I/F 229 includes, for example, the external camera 231. The external camera 231 is a camera that captures images around the image forming apparatus 101, and is another example of the camera 102 in FIG. 1. The image forming apparatus 101 has either the internal camera 228 or the external camera 231, or both.

The system bus 230 is connected to each component described above and transmits an address signal, a data signal, various control signals, and the like.

(Hardware configuration of information processing device)
FIG. 3 is a diagram illustrating a hardware configuration example of the information processing device according to the embodiment. The image processing apparatus according to this embodiment may be, for example, an information processing apparatus as shown in FIG. Further, another information processing apparatus for creating unnecessary space information described above with reference to FIG. 1 has a hardware configuration as shown in FIG.

The information processing device 300 has a general computer configuration, and includes, for example, a CPU 301, a RAM 302, a ROM 303, a storage device 304, a network I/F 305, an input device 306, a display device 307, an external I/F 308, and a bus. 309 etc.

The CPU 301 is an arithmetic device that realizes each function of the information processing device 300 by reading a program or data stored in the ROM 303, the storage device 304, or the like onto the RAM 302 and executing a process. The RAM 302 is a volatile memory used as a work area or the like of the CPU 301. The ROM 303 is a non-volatile memory that can retain programs and data even when the power is turned off.

The storage device 304 is, for example, a large-capacity storage device such as an HDD or an SSD, and stores an OS, application programs, various data, and the like. The network I/F 305 is a communication interface for connecting the information processing device 300 to the network 250 and the like.

The input device 306 is, for example, a pointing device such as a mouse or an input device such as a keyboard, and is used to input each operation signal to the information processing device 300. The display device 307 is a display device such as a display, and displays the processing result and the like by the information processing device 300.

The external I/F 308 is an interface with an external device. The external device includes, for example, an external camera 310 that captures an image around the information processing device 300, a recording medium, and the like.

The bus 309 is connected to each of the above components and transmits an address signal, a data signal, various control signals, and the like.

[First Embodiment]
<Functional configuration>
FIG. 4 is a diagram illustrating an example of a functional configuration of the image forming apparatus according to the first embodiment. The image forming apparatus 101 realizes the functional configuration shown in FIG. 4 by executing a predetermined program by at least one CPU of the CPU 211 and the CPU 221 of FIG.

Note that the functional configuration shown in FIG. 4 is an example. For example, at least some of the components included in the operation unit 220 may be included in the main body 210, and at least some of the components included in the main body 210 may be included in the operation unit 220. May be included.

(Functional structure of the main unit)
The main body 210 of the image forming apparatus 101 includes a human body detection unit 421, a state control unit 422, an image forming unit 423, a communication unit 424, and the like.

The human body detection unit 421 is means for detecting a human body around the image forming apparatus 101 by using, for example, the human body detection sensor 218 shown in FIG. 2, and is executed by, for example, the CPU 211 (or CPU 221) in FIG. It is realized by the program. For example, when a human body is detected within the detection range of the human body detection sensor 218, the human body detection unit 421 notifies the state control unit 422 and the like that the human body has been detected.

The state control unit 422 is a unit that controls the power state of the operation unit 220 and the main body 210, and is realized by a program executed by the CPU 211 (or CPU 221) in FIG. 2, for example. For example, when the image forming apparatus 101 is not used for more than a preset time, the state control unit 422 consumes less power than the normal state in which the image forming process can be performed in the image forming apparatus 101. Shift to the state. In the power saving state, the image forming apparatus 101 can reduce power consumption by, for example, stopping the functions of the operation unit 220, the engine unit 217 of the main body 210, and the storage unit 214.

Further, when the image forming apparatus 101 receives a notification indicating that a human body is detected, for example, from the human body detection unit 421 while the image forming apparatus 101 is in the power saving state, the state control unit 422 causes the operation unit 220 to be in the normal power saving state. Return to the normal state where the operation of is possible. For example, the state control unit 422 returns the operation unit 220 from the power saving state to the normal state by notifying the operation unit 220 of the return instruction via the communication unit 424.

The image forming unit 423 is a unit that executes various image forming functions (for example, printing, copying, scanning, and faxing) of the image forming apparatus 101. For example, the engine unit 217 of FIG. 2 and the CPU 211 of FIG. It is realized by a program executed in.

The communication unit 424 is means for communicating with the operation unit 220, and is realized by, for example, the connection I/F 216 in FIG. 2 and a program executed by the CPU 211 in FIG.

(Functional configuration of the operation unit)
The operation unit 220 of the image forming apparatus 101 includes, for example, a photographing unit 401, a face image detection unit 402, a characteristic information extraction unit 403, a living body determination unit 404, a noise determination unit 405, an unnecessary space extraction unit 406, an image authentication unit 407, and a display. It has an input unit 408, a biometric information storage unit 409, a non-biological information storage unit 410, an identification space information storage unit 411, an unnecessary space information storage unit 412, a communication unit 413, and the like.

The image capturing unit (acquisition unit) 401 is a unit that captures an image of the periphery of the image forming apparatus 101 using the internal camera 228 in FIG. 2 and the camera 102 such as the external camera 231. For example, the CPU 221 (in FIG. 2). Alternatively, it is realized by a program executed by the CPU 211). The image capturing unit 401 acquires, for example, images of users around (for example, in front of) the image forming apparatus 101 at predetermined time intervals.

The face image detection unit 402 is a unit that detects a face area from an image captured by the image capturing unit 401, and is realized by, for example, a program executed by the CPU 221 (or CPU 211) in FIG. FIG. 7 shows an example of the face image detected by the face image detection unit 402.

In FIG. 7A, an image 710 captured by the image capturing unit 401 is captured of the user 103 in front of the image forming apparatus 101. As described above, it is assumed that the camera 102 of the image forming apparatus 101 is installed so that the user 103 who uses the image forming apparatus 101 is included in the image 710 captured by the image capturing unit 401.

The face image detection unit 402 detects, for example, a face image 720 of the user 103 as shown in FIG. 8B from the image 710 captured by the image capturing unit 401 using, for example, a known pattern matching technique. Then, the detected face image 720 is extracted.

Any method may be used as the method for detecting the face image by the face image detection unit 402, but here, as an example, the outline of the face detection by the Haar-like feature will be described.

In the face detection based on the Haar-like feature, the Haar-like feature amount that is a value obtained by subtracting the sum of the pixel values of the white region from the sum of the pixel values of the black region in the rectangle to be calculated in the search region. To use. Since the above-described rectangular layout location may be located anywhere in the search area, there is a possibility of tens of thousands of layout locations in the search area. For each of these, the weak classifiers in the search area are weighted by pre-learning by Boosting, and dozens of weak classifiers having the highest importance are selected to create a strong classifier. With this strong discriminator (a black-and-white rectangular pattern indicating the human face-likeness), it is possible to determine whether or not the region is a human face region.

The feature information extraction unit 403 is a unit that extracts the feature information (feature amount) of a person included in the image captured by the imaging unit 401, and is realized by, for example, a program executed by the CPU 221 (or CPU 211) in FIG. It

The characteristic information extraction unit 403 extracts characteristic information indicating an action from the time-series face images extracted by the face image detection unit 402 from the images captured by the imaging unit 401 at predetermined time intervals. As an example of the feature information extracted by the feature information extraction unit 403, a stereoscopic higher-order autocorrelation feature amount (CHLAC: Cubic Higher-order Local Auto Correction) can be used.

Before describing the three-dimensional higher-order autocorrelation feature amount (hereinafter, referred to as CHLAC feature amount), a higher-order local autocorrelation feature amount (HLAC: Higher-order Local Auto Correction), which is a predecessor method of the CHLAC feature amount, is described. ) Will be described.

(HLAC feature quantity)
In the high-order local autocorrelation feature amount (hereinafter, referred to as HLAC feature amount), the high-order autocorrelation at a certain point r on the binarized image is calculated.

Since it is expected that the target pixel r (reference point) and the neighboring region have a high correlation, the HLAC feature amount is calculated by calculating the correlation from the local region instead of calculating the correlation of the entire region. Is required. An Nth-order (N is a natural number) autocorrelation function is represented by the following (Equation 1).

ここで、ｆは時系列の画像であり、ｒは参照点、ａｉ（ｉ＝１，...，Ｎ）はＮ個の変位である。次元を２次、変位方向を３×３に限定した場合、２値の動画像のＨＡＬＣ特徴量における変位ベクトルの組合せは、例えば、図８に示すように、０次が１個、１次が４個、２次が２０個の計２５次元のベクトルとして表される。

Here, f is a time-series image, r is a reference point, and ai (i=1,..., N) is N displacements. When the dimension is secondary and the displacement direction is limited to 3×3, the combination of displacement vectors in the HALC feature amount of the binary moving image is, for example, as shown in FIG. It is represented as a total of 25-dimensional vectors, including 4 and 20 degrees.

This HLAC feature quantity has two properties, i.e., invariance and additivity.

The position invariance is a property that the feature amount does not change regardless of the position of the object in the image. In addition, the additivity is, for example, when the feature amount of the motion of the person A is fA and the feature amount of the motion of the person B is fB, and when the person A and the person B perform the motion in the same image, It is a property that the amount becomes fA+fB.

(CHLAC feature quantity)
Next, the CHLAC feature amount used as the feature information in this embodiment will be described.

The HLAC feature amount is for obtaining the correlation of the two-dimensional local region, whereas the CHLAC feature amount is for obtaining the correlation of the local region with time added. The CHLAC feature amount can be obtained by integrating the Nth-order autocorrelation function shown in (Equation 2) below with respect to the time of a time-series image.

時間方向の積分範囲は、どの程度の時間方向の相関を取るかによって決定する任意のパラメータである。

The integration range in the time direction is an arbitrary parameter determined by how much the correlation in the time direction is taken.

The number of combinations of the displacement vectors of the CHLAC feature amount based on the binary moving image is 0 for the 1st order, 13 for the 1st order, and 237 for the 2nd order, and is represented as a total of 251 dimensional vectors. Like the HLAC feature amount, the CHLAC feature amount has two properties of position invariance and additivity.

Here, an image of a method of calculating the CHLAC feature amount will be described with reference to FIG.

In FIG. 9, for example, the image capturing unit 401 or the like captures time-series images 901 to 904 at predetermined time intervals.

Next, for example, the feature information extraction unit 403 calculates the difference between the time-series images 901 to 904 and binarizes the obtained difference image. In the example of FIG. 9, the image 911 shows a binary difference image between the image 901 and the image 902. An image 912 shows a binary difference image between the image 902 and the image 903. Further, the image 913 shows a binary difference image between the image 903 and the image 904.

Subsequently, the feature information extraction unit 403 calculates up to the second-order correlation between the HLAC feature amount of its own and the preceding and following images 911 and 913 for the binarized image 912.

Further, the feature information extraction unit 403 calculates and integrates CHLAC for all pixels in the image 912.

Thereby, for example, one feature vector (CHLAC feature amount) corresponding to the three binary images shown in the frame 920 of FIG. 9 is generated. It should be noted that in the case of identifying a minute motion such as a face as in the present embodiment, it is desirable to normalize the norm of the generated feature vector.

As described above, the CHLAC feature amount can be used as an example of the feature information extracted by the feature information extraction unit 403.

Further, as another example, the feature information extraction unit 403 uses a time-series image to calculate by block matching where a small area in the image has moved in the image of the next frame, and the movement of each area. It may be one that uses the feature amount represented by a vector as the feature information. Here, the following description will be made assuming that the characteristic information extraction unit 403 extracts the CHLAC characteristic amount.

Preferably, the characteristic information extraction unit 403 extracts the characteristic amount excluding unnecessary elements (unnecessary dimensions) stored in the unnecessary space information storage unit 412 from the 251-dimensional vector of CHLAC characteristic amounts. Thereby, for example, when 10 unnecessary elements are stored in the unnecessary space information storage unit 412, the characteristic information extraction unit 403 can calculate 241 dimensions of the 251 dimension characteristic information. The processing can be speeded up.

Returning to FIG. 4, the description of the functional configuration of the operation unit 220 will be continued.

The living body determination unit 404 includes information on the identification space of the living body that is extracted in advance from the image of the living body (hereinafter referred to as biometric information), and information on the identification space of the non-living body that is extracted in advance from the non-living body (such as a photograph) Hereinafter, the biometric determination is performed using (non-biometric information). The biometric determination unit 404 is realized by, for example, a program executed by the CPU 221 (or CPU 211) of FIG.

The biometric determination unit 404 compares the feature information extracted by the feature information extraction unit 403 with biometric information stored in advance in the biometric information storage unit 409 or non-biometric information stored in advance in the non-biometric information storage unit 410, It is determined whether or not the person photographed in the image is a living body (living body determination).

It should be noted that examples of methods for determining the living body include discriminant analysis and subspace method. In the first embodiment, as an example, the following description will be made assuming that the biometric determination unit 404 performs the biometric determination by discriminant analysis.

(Discriminant analysis)
Here, the outline of the discriminant analysis will be described with reference to FIGS.

For example, as shown in FIG. 10A, it is assumed that there is a discrimination space generated from three pieces of biometric information (biological information 1 to 3) and five pieces of non-biological information (non-biological information 1 to 5). The biometric information 1 to 3 is an example of a biometric identification space (biometric information) stored in the biometric information storage unit 409 and extracted in advance from a biometric image. Further, the non-living body information 1 to 5 is an example of the non-living body identification space (non-living body information) stored in the non-living body information storage unit 410 and extracted in advance from the non-living body image.

As an example, when the biometric determination unit 404 projects the feature information extracted by the feature information extraction unit 403 onto the discriminant space and the closest identification space is biometric information, the feature information extracted by the feature information extraction unit 403. Is the biological feature information. On the other hand, the biometric determination unit 404 extracts the feature information extracted by the feature information extraction unit 403 if the closest identification space is non-biometric information when the feature information extracted by the feature information extraction unit 403 is projected onto the determination space. Is determined to be non-living body feature information.

For example, in the example of FIG. 10A, the characteristic information 1001 projected on the discrimination space is the closest to the “living body 1”, so the living body determination unit 404 determines that the characteristic information 1001 is the characteristic information of the living body. To do. On the other hand, in the example of FIG. 10B, the feature information 1002 projected on the discriminant space is the closest to the “non-biometric information 2 ”, so the biometric determination unit 404 determines that the feature information 1002 is the non-biometric feature information. Judge that there is.

The distance between the feature information projected on the discriminant space and each identification space (biological information 1 to 3, non-biological information 1 to 5) is, for example, the distance to the center (center of gravity) of each identification space. To be judged. As the distance, for example, Euclidean distance or Mahalanobis distance is used.

As another example, the biometric determination unit 404 does not cause the feature information extracted by the feature information extraction unit 403 to belong to the closest identification space when projected onto the determination space, but a fixed range from the identification space of the biological body. You may look at the distance of.

For example, the feature information projected on the discriminant space in FIG. 11 has a predetermined distance range 1101 from the biometric information 1, a predetermined distance range 1102 from the biometric information 2, or a predetermined distance range 1103 from the biometric information 3. If it is inside, the biometric determination unit 404 determines that the biometric characteristic information.

In this method, since the information necessary for biometric determination is the distance between the center of the identification space of the biometric information 1 to 3 and the feature information, the information between the center of the identification space of the non-biometric information 1 to 5 and the feature information is obtained. The distance does not have to be calculated. In this method, the shorter the distance from the center of the identification space of the biometric information 1 to 3, the higher the similarity, and the longer the distance, the lower the similarity.

In the present embodiment, the biometric determination unit 404 will be described below as performing biometric determination by discriminant analysis.

Returning to FIG. 4, the description of the functional configuration of the operation unit 220 will be continued.

The noise determination unit 405 is a unit that determines whether or not there is an image in which noise that influences the biological determination, such as flicker of a fluorescent lamp, is present among the time-series images captured by the imaging unit 401. The noise determination unit 405 is realized by, for example, a program executed by the CPU 221 (or CPU 211) of FIG.

Any method may be used as the noise determination method by the noise determination unit 405, but as an example, the noise determination unit 405 obtains the difference between the luminance values of the pixels in the two images arranged in time series, and calculates the sum thereof. To do. In addition, the noise determination unit 405 can determine that noise is present when the calculated sum total value exceeds the threshold value.

The unnecessary space extraction unit 406 does not use the feature information extracted by the feature information extraction unit 403 in the biometric determination, out of the elements of the projection vector (first projection vector) that projects the feature information in the determination space as illustrated in FIG. The element information is extracted and stored in the unnecessary space information storage unit 412.

Further, the unnecessary space extraction unit 406 extracts the projection vector (second projection vector) excluding the unnecessary elements stored in the unnecessary space information storage unit 412 from the elements of the first projection vector as the identification space information storage unit 411. Remember.

Here, the discriminant space and the projection vector will be described.

Discriminant analysis is a method of obtaining a space that minimizes the within-class variance of its own class (identification space) and maximizes the inter-class variance with other classes. When there are multiple classes, the space that minimizes the variance within each class and maximizes the inter-class variance between each class is obtained.

The intra-class dispersion W is represented by the following (Formula 3).

また、クラス間分散Ｂは、次の（式４）で表される。

The interclass dispersion B is represented by the following (formula 4).

上記の（式３）、（式４）において、データはｄ次元のベクトルｘで表される。また、クラス数はｃ≧２、総データ数はｎ 、データ集合はＸ＝｛ｘ｝、データ全体の平均値は_aveｘであるものとする。また、クラスｉのデータ集合をｘ_i、データ集合ｘ_iのデータ数をｎ_i、データ数ｎ_iのデータの平均を_aveｘ_iとする。Ｔは転置行列を示す。

In the above (Formula 3) and (Formula 4), the data is represented by a d-dimensional vector x. Further, it is assumed that the number of classes is c≧2, the total number of data is n 2, the data set is X={x}, and the average value of all the data is _ave x. Further, the data set of class i is x _i , the number of data of the data set x _i is n _i , and the average of the data of the number of data n _i is _ave x _i . T represents a transposed matrix.

Here, the eigenvectors corresponding to _C−1 large eigenvalues λ ₁ ,..., λ _C-1 of W ⁻¹ B are A ₁ ,..., A _C-1 . Using this, the d×(C−1) matrix A=[A ₁ , A ₂ ,..., A _C-1 ] is determined. The dimension reduction of the data x in the C-1 dimensional space by ^AT x using this matrix is called projection to the discriminant space.

Here, the biometric information, which is the identification space of the living body stored in the biometric information storage unit 409, and the non-living body information, which is the non-living body identification space stored in the non-biometric information storage unit 410, are 251 dimensional data x. The following description will be given. However, the data x may be data other than the 251 dimensional data.

FIG. 12 is a diagram showing an example of the projection vector according to the first embodiment. The projection vector shown in FIG. 12 is an example of the projection vector when performing discriminant analysis of 8 classes (8 types of identification spaces). When 8-class discriminant analysis is performed, the discriminant space is 7-dimensional, and there are 7 eigenvectors projected onto the discriminant space: A ₁ , A ₂ , A ₃ , A ₄ , A ₅ , A ₆ , and A _7. become.

In FIG. 12, the element number indicates the dimension of 251 dimensions. Also, A ₁ , A ₂ , A ₃ , A ₄ , A ₅ , A ₆ , and A ₇ represent the values of the eigenvectors.

The eigenvectors A ₁ , A ₂ , A ₃ , A ₄ , A ₅ , A ₆ , A ₇ projected onto the discriminant space include one or more biometric information and one or more non-biometric information. Be done.

The unnecessary space extraction unit 406 extracts an unnecessary element that is an unnecessary element (dimension) from the projection vector as shown in FIG. 12, for example. For example, in FIG. 12, the unnecessary space extraction unit 406 determines that an element corresponding to an element number whose absolute value of all values corresponding to each element number is less than a threshold value is an unnecessary element, and the unnecessary space information storage unit 412. For example, FIG. 13 shows an example in which the threshold value is set to “0.010” in FIG.

FIG. 13 is a diagram showing an example of unnecessary elements of the projection vector according to the first embodiment. In the example of FIG. 13, for element numbers ₃ , ₄ , _7, and 246, all values corresponding to the eigenvectors A ₁ , A ₂ , A ₃ , A ₄ , A ₅ , A ₆ , and A ₇ are threshold values “0”. Since it is less than 0.010”, the unnecessary space extraction unit 406 determines that it is an unnecessary element and stores it in the unnecessary space information storage unit 412.

Preferably, the unnecessary space extraction unit 406 removes the projection vector (second projection vector) from the projection vector (first projection vector) shown in FIG. 12 by removing unnecessary elements stored in the unnecessary space information storage unit 412, It is stored in the identification space information storage unit 411.

The threshold value “0.010” is an example, and it is assumed that the threshold value is set in advance so that an element that has a small influence on the discrimination result of the discriminant analysis is extracted as an unnecessary element. Alternatively, the threshold may be set by an administrator, a user, or the like using the operation panel 226 or the like.

The living body determination unit 404 according to the present embodiment uses the second projection vector obtained by removing the unnecessary elements stored in the unnecessary space information storage unit 412 from the projection vector (first projection vector) as shown in FIG. Then, the discriminant analysis of the characteristic information extracted by the characteristic information extraction unit 403 is performed. For example, the biometric determination unit 404 uses the second projection vector stored in the identification space information storage unit 411 by the unnecessary space extraction unit 406 to determine whether the person imaged by the imaging unit 401 is a living body. Determine whether.

Preferably, the feature information extraction unit 403 excludes unnecessary elements (unnecessary dimensions) stored in the unnecessary space information storage unit 412 from the 251-dimensional feature information when extracting the feature information, and extracts the feature information. Extract.

Returning to FIG. 4, the description of the functional configuration of the operation unit 220 will be continued.

The image authentication unit 407 is means for executing conventional image authentication such as face authentication, and is realized by, for example, a program executed by the CPU 221 (or CPU 211) in FIG. For example, the image authentication unit 407 permits the use of the image forming apparatus 101 when the image authentication is successful and the biometric determination unit 404 determines that the person photographed in the image is a biometric.

The display input unit 408 is means for displaying a display screen such as an operation screen and a setting screen on the operation panel 226 of FIG. 2 and accepting an input operation to the operation panel 226. For example, the CPU 221 (or the CPU 211 of FIG. 2). ) Is implemented by the program executed.

The biometric information storage unit 409 is a unit that stores biometric information that is information of the identification space of the biological body extracted in advance from the image of the biological body, and is executed by, for example, the flash memory 224, the RAM 223 of FIG. 2, and the CPU 221 of FIG. It is realized by the program etc.

The non-living body information storage unit 410 is means for storing non-living body information which is information of the non-living body identification space extracted in advance from the non-living body image, and is, for example, the flash memory 224, the RAM 223 of FIG. It is realized by a program or the like executed by the CPU 221.

The identification space information storage unit (second storage unit) 411 removes unnecessary elements from the first projection vector that projects the feature information extracted by the feature information extraction unit 403 onto the biometric information and the non-biometric information. This is a means for storing the projection vector of 2. The identification space information storage unit 411 is realized by, for example, the flash memory 224 of FIG. 2, the RAM 223, the program executed by the CPU 221 of FIG.

The unnecessary space information storage unit (first storage unit) 412 does not use the feature information extracted by the feature information extraction unit 403 for biometric determination out of the first projection vectors projected onto the biometric information and the non-biometric information. It is a means for storing information of unnecessary elements. The unnecessary space information storage unit 412 is realized by, for example, the flash memory 224 of FIG. 2, the RAM 223, the program executed by the CPU 221 of FIG.

With the above-described configuration, the image forming apparatus 101 can omit unnecessary element calculation in, for example, the feature information extraction process by the feature information extraction unit 403, the biometric determination process in the biometric determination unit 404, and thus the biometric determination can be performed at high speed. Can be converted.

Further, since the image forming apparatus 101 extracts, as an unnecessary element, an element that has a small influence on the determination result of the biometric determination, out of the elements included in the first projection vector, it is possible to suppress a decrease in the precision of the biometric determination. ..

Therefore, according to the image forming apparatus 101 according to the present embodiment, in the image forming apparatus 101 that performs the biometric determination to determine whether or not the person imaged in the image is a biometric, the deterioration of the accuracy of the biometric determination is suppressed. At the same time, it becomes possible to shorten the time required for biometric determination.

The functional configuration of the image forming apparatus 101 illustrated in FIG. 5 is an example. For example, the image forming apparatus 101 may not include the unnecessary space extraction unit 406.

FIG. 5 is another example of the functional configuration of the image forming apparatus according to the first embodiment. The image forming apparatus 101 shown in FIG. 5 has a configuration in which the unnecessary space extracting unit 406 is removed from the image forming apparatus 101 shown in FIG.

In this case, the image forming apparatus 101 stores the unnecessary element acquired by another information processing apparatus in the unnecessary space information storage unit 412 in advance. Further, the image forming apparatus 101 stores in advance in the identification space information storage unit 411 a second projection vector obtained by another information processing apparatus and having the unnecessary elements removed from the first projection vector.

As a result, even in the image forming apparatus 101 shown in FIG. 5, in the image forming apparatus 101 that makes a living body determination for determining whether or not the person imaged in the image is a living body, it is possible to prevent the accuracy of the living body determination from decreasing. Therefore, the time required for the biometric determination can be shortened.

FIG. 6 is a diagram illustrating an example of a functional configuration of the information processing device according to the first embodiment. The information processing apparatus 600 shown in FIG. 6 includes the photographing unit 401, the characteristic information extraction unit 403, the noise determination unit 405, the unnecessary space extraction unit 406, and the biometric information storage unit 409 in the functional configuration of the image forming apparatus 101 shown in FIG. , A non-biological information storage unit 410, an identification space information storage unit 411, and an unnecessary space information storage unit 412.

With the above configuration, the information processing apparatus 600 acquires information of unnecessary elements that are not used for biometric determination in the first projection vector from the image captured by the imaging unit 401, and stores the information in the unnecessary space information storage unit 412. To do. Further, the information processing device 600 stores, in the identification space information storage unit 411, a second projection vector obtained by removing unnecessary elements from the first projection vector. The information of the unnecessary elements stored in the unnecessary space information storage unit 412 by the information processing apparatus 600 and the information of the second projection vector stored in the identification space information storage unit 411 are, for example, the image forming apparatus 101 shown in FIG. It is stored in and used.

<Process flow>
Subsequently, a flow of processing of the living body determination method according to the first embodiment will be described.

(Unnecessary element extraction process)
FIG. 14 is a flowchart showing an example of unnecessary element extraction processing according to the first embodiment. This process shows an example of a process of extracting an unnecessary element by the unnecessary space extracting unit 406 of the image forming apparatus 101 or the information processing device 600, and a process of generating a second projection vector.

In step S1401, the characteristic information extraction unit 403 reads the images captured by the image capturing unit 401 at predetermined time intervals.

In step S1402, the feature information extraction unit 403 extracts feature information (for example, CHLAC feature amount) from the image read in step S1401.

Subsequently, the unnecessary space extraction unit 406 executes the unnecessary space extraction processing 1400 shown in steps S1403 to S1409.

In step S1403, the unnecessary space extraction unit 406 creates the above-described discriminant space.

In the processes of steps S1404 to S1407, the unnecessary space extraction unit 406 determines whether or not each element is an unnecessary element from the projection vector element number 1 to 251 (Nmax) as shown in FIG. 12, for example. To do.

For example, in step S1401, the unnecessary space extraction unit 406 initializes the element number N to 1.

In step S1405, the unnecessary space extraction unit 406 determines whether the absolute value of each value of the element number N is less than a threshold value (for example, "0.010").

If the absolute value of each value of the element number N is not less than the threshold value (for example, “0.010”), the unnecessary space extraction unit 406 moves the process to step S1407. On the other hand, when each value of the element number N is less than the threshold value (for example, “0.010”), the unnecessary space extracting unit 406 shifts the processing to step S1406.

When the process proceeds to step S1406, the unnecessary space extraction unit 406 stores the element number N for which the absolute value of each value is less than the threshold value (determined as an unnecessary element) in the unnecessary space information storage unit 412 as an unnecessary element. Remember.

Upon proceeding to step S1407, the unnecessary space extraction unit 406 determines whether the element number N is greater than or equal to Nmax, which is the maximum value (for example, 251) of the element number N.

If the element number N is not greater than or equal to Nmax, the unnecessary space extraction unit 406 moves the process to step S1408. On the other hand, when the element number N is Nmax or more, the unnecessary space extracting unit 406 shifts the processing to step S1409.

When the process proceeds to step S1408, the unnecessary space extraction unit 406 adds 1 to the element number N, returns the process to step S1405, and repeats the same process.

When the process proceeds to step S1409, the unnecessary space extraction unit 406 removes the unnecessary elements stored in the unnecessary space information storage unit 412 from the projection vector (first projection vector) as shown in FIG. A second projection vector) is created. The unnecessary space extraction unit 406 also stores the created second projection vector in the identification space information storage unit 411.

By the above processing, for example, the information of the element numbers 3, 4, 7, 246 and the like of the projection vector shown in FIG. 13 is stored in the unnecessary space information storage unit 412. Further, for example, among the projection vectors shown in FIG. 13, the 247-dimensional projection vector excluding the element numbers 3, 4, 7, and 246 is stored in the identification space information storage unit 411.

(Biometric determination process)
FIG. 15 is a flowchart showing an example of the biometric determination process of the image forming apparatus according to the first embodiment.

In step S1501, the image capturing unit 401 captures a plurality of time-series images at predetermined time intervals.

In step S1502, the face image detecting unit 402 performs a process of detecting a face image from the image captured by the capturing unit 401, and determines whether the face image is detected.

If no face image is detected, the face image detection unit 402 returns the process to step S1501 and repeats the same process. On the other hand, if a face image is detected, the face image detection unit 402 moves the process to step S1503.

In step S1503, the characteristic information extraction unit 403 reads out unnecessary element information (for example, element numbers 3, 4, 7, 246, etc.) stored in the unnecessary space information storage unit 412.

In step S1504, the feature information extraction unit 403 extracts feature information (for example, CHLAC feature amount) from the face image detected by the face image detection unit 402.

Preferably, at this time, the characteristic information extraction unit 403 extracts the characteristic information of the element numbers (dimensions) excluding the unnecessary elements (for example, the element numbers 3, 4, 7, 246, etc.) read from the unnecessary space information storage unit 412. To do. This is because the feature information of the unnecessary element is not used in the biometric determination process by the biometric determination unit 404. As a result, the load of the characteristic information extraction processing by the characteristic information extraction unit 403 is reduced, and the processing speed can be increased.

In step 1505, the biometric determination unit 404 reads the second projection vector stored in the identification space information storage unit 411.

In step S1506, the biometric determination unit 404 performs biometric determination of the face image detected by the face image detection unit 402 using the read second projection vector and the feature information extracted by the feature information extraction unit 403. Accordingly, the image capturing unit 401 determines whether the person imaged in the image is a living body or a non-living body (for example, a photograph).

In step S1507, the biometric determination unit 404 branches the process according to, for example, the result of the biometric determination.

When it is not determined that the face image is a biometric face image, the biometric determination unit 404 returns the process to step S1501 and repeats the same process. On the other hand, if it is determined that the image is a face image of a living body, the process proceeds to the next step (S1508).

How to use the determination result of the biological determination in the next step is arbitrary.

As an example, in step S1508, the image authentication unit 407 executes the face authentication processing of the detected face image, and when the face authentication is successful, permits the use of the image forming apparatus 101. On the other hand, the image authentication unit 407 does not permit the use of the image forming apparatus 101 unless the face authentication is successful.

As described above, according to the present embodiment, the characteristic information extraction unit 403 and the biometric determination unit 404 execute the processing by excluding the unnecessary elements stored in the unnecessary space information storage unit 412. To be done. Further, since the unnecessary element stored in the unnecessary space information storage unit 412 is an element (dimension) that is determined not to affect the biometric determination, it is possible to suppress a decrease in accuracy of the biometric determination.

Therefore, according to the present embodiment, in the information processing system 100 that performs the biometric determination that determines whether or not the person captured in the image is the biometric, it is necessary for the biometric determination while suppressing the decrease in the accuracy of the biometric determination. The time can be shortened.

[Second Embodiment]
In the first embodiment, an example of using discriminant analysis for biometric determination has been described. However, the discriminant analysis is an example of the biometric determination method, and the biometric determination method may be another method.

In the second embodiment, an example of using the subspace method for biometric determination will be described.

<Functional configuration>
The functional configuration of the information processing system 100 according to the second embodiment is the same as the functional configuration of the information processing system 100 shown in FIGS.

However, the biometric determination unit 404 according to the second embodiment performs the biometric determination using the subspace method.

In addition, the unnecessary space extraction unit 406 according to the second embodiment does not need to exist commonly in the biometric information stored in the biometric information storage unit 409 and the non-biometric information stored in the non-biometric information storage unit 410. Elements are extracted by principal component analysis and stored in the unnecessary space information storage unit 412.

(Biometric determination by subspace method)
Here, a living body determination method based on the subspace method will be described.

FIG. 16 is a diagram for explaining the subspace method according to the second embodiment. When the subspace method is used for biometric determination, the angle of the class (identification space) can be used as the degree of similarity (similarity indicating the biometricity).

For example, in the 251-dimensional discriminant space 1600 shown in FIG. 16, the “biological information 1” prepared in advance is an m-dimensional (m is a natural number) partial space P1601, and the characteristic information for biometric determination is n-dimensional (n is a natural number). Subspace Q1602 of At this time, between the subspaces P1601 and Q1602, n canonical angles θi that are angles between the subspaces can be defined (m≧n).

The n canonical angles θi are represented by the following (Equation 5).

このとき、類似度は次の（式６）で定義される。

At this time, the similarity is defined by the following (formula 6).

この類似度は、２つの部分空間が一致するとき１．０となり、２つの部分空間が直交するとき０．０となる。

This degree of similarity is 1.0 when the two subspaces match and is 0.0 when the two subspaces are orthogonal.

As an example, the biometric determination unit 404 can calculate the similarity between the feature information for performing biometric determination and each class (identification space), and set the class with the highest similarity as the determination result of the biometric determination.

For example, it is assumed that there are eight classes (identification spaces) of biometric information 1 to 3 and non-biometric information 1 to 5. Further, the similarity between the feature information and the biometric information 1 is “0.75”, the similarity between the feature information and the biometric information 2 is “0.96”, and the similarity between the feature information and the biometric information 3 is “0. 23”, and the similarity between the feature information and the non-biological information 1 is “0.52”. Further, the similarity between the feature information and the non-biometric information 2 is “0.33”, the similarity between the feature information and the non-biometric information 3 is “0.45”, and the similarity between the feature information and the non-biometric information 4 is It is assumed that “0.68” and the degree of similarity between the feature information and the non-living body information 5 is “0.11”.

In this case, the biometric determination unit 404 can classify the biometric information 2 that has a higher degree of similarity with the feature information for which the biometric determination is performed than other classes, and determine that the biometric information is the biometric feature information.

As another example, the biometric determination unit 404 may use biometric information as a target and set a class equal to or larger than a threshold value as the determination result of the biometric determination.

For example, the similarity between the feature information and the biometric information 1 is “0.66”, the similarity between the feature information and the biometric information 2 is “0.86”, and the similarity between the feature information and the biometric information 3 is “ 0.23", and the similarity between the feature information and the non-biological information 1 is "0.52". Further, the similarity between the feature information and the non-biometric information 2 is “0.33”, the similarity between the feature information and the non-biometric information 3 is “0.45”, and the similarity between the feature information and the non-biometric information 4 is It is assumed that “0.88” and the similarity between the feature information and the non-living body information 5 is “0.11”.

Further, the target threshold value is set to "0.80". In this case, the biometric determination unit 404 is biometric feature information because there is biometric information 2 whose degree of similarity to the feature information for performing biometric determination is a threshold value or more among the target biometric information 1 to 3. Can be determined.

In this method, the biometric determination unit 404 does not have to calculate the degree of similarity with the non-biometric information 1 to 5. Further, as another method, the biometric determination unit 404 may add to the determination condition that the degree of similarity with the non-biometric information 1 to 5 is equal to or less than another threshold value.

(Extraction of unnecessary elements by principal component analysis)
Next, a method of extracting unnecessary elements by principal component analysis will be described.

In the second embodiment, the unnecessary space extraction unit 406 has a contribution rate of the top X% (X hash) when the information common to each class (identification space) is subjected to the principal component analysis for all the classes. The element of the principal component vector of the threshold value) is stored in the unnecessary space information storage unit 412 as an unnecessary element.

FIG. 17 is a diagram showing an image of a partial space of biometric information and non-biometric information according to the second embodiment. In FIG. 17, since it is difficult to draw a three-dimensional space or more at the same origin, the 251-dimensional discriminant space 1700 is represented by a three-dimensional space.

In the example of FIG. 17, three partial spaces of “biological information 1” 1701, “biological information 2” 1702, and “non-biological information 1” 1703 are shown in the discrimination space 1700.

The unnecessary space extraction unit 406 performs a principal component analysis and contributes in order to extract unnecessary elements that commonly exist in each class ("biological information 1", "biological information 2", "non-biological information 1"). The principal component vector with the highest X% is extracted.

The unnecessary space extraction unit 406, for example, performs a principal component analysis according to the following procedure, and extracts a principal component vector whose contribution rate is the top X%.

First, the unnecessary space extraction unit 406 obtains the covariance matrix S from the feature amounts of biometric information and non-biometric information. The covariance matrix S is obtained by the following (Formula 7).

この共分散行列Ｓの各要素(Ｓxx, Ｓxy, Ｓyy)については、下記のように求めることができる。

Each element (Sxx, Sxy, Syy) of this covariance matrix S can be obtained as follows.

The vector X shown in the following (Equation 8) is introduced.

同様にして、ベクトルＹを導入する。ベクトルＹは、（式８）のＸをＹに置き換えることにより得られる。

Similarly, the vector Y is introduced. The vector Y is obtained by replacing X in (Equation 8) with Y.

The variance using the vector X and the vector Y can be expressed in the form of the product of the vertical vector and the horizontal vector as shown in (Equation 9).

（式９）において、「'」は、転置を表す。

In (Formula 9), "'" represents transposition.

The number of elements in the covariance matrix S changes depending on the number of classes of biometric information and non-biometric information. The covariance matrix S shown in (Equation 7) shows an example when the number of classes is two. When the number of classes is 3, the covariance matrix is [3×3], and when the number of classes is 4, the covariance matrix is [4×4].

Next, the unnecessary space extraction unit 406 obtains the eigenvalue λ and the eigenvector μ of the covariance matrix S by the relational expression shown in (Expression 10).

また、不要空間抽出部４０６は、求めた固有値λの中で、値が大きい順にλ1…nを決定し、決定したλ1…nの各々について、（式１０）を用いて、固有ベクトルμ1…nの値を求める。

In addition, the unnecessary space extraction unit 406 determines λ1...n in descending order of the eigenvalues λ obtained, and uses (Equation 10) for each of the determined λ1... Find the value.

Next, the unnecessary space extraction unit 406 sets the eigenvector μ1 corresponding to the eigenvalue λ1 having the largest value among the obtained eigenvalues λ as the first principal component vector. Here, the eigenvalue λ1 corresponds to the dispersion in that direction.

The number of principal component vectors that the unnecessary space extraction unit 406 obtains is determined from the eigenvalue λi based on the cumulative contribution rate obtained by the following (Equation 11).

なお、累積寄与率は、寄与率を所定の順序で順次に加えたものである。例えば、第１主成分から、第ｋ主成分（ｋは２以上の整数）までの寄与率の和を、第ｋ主成分までの累積寄与率と呼ぶ。第ｋ主成分までの累積寄与率は、ｋ個の主成分が、全体の情報量の何パーセントの情報を保有しているかを示している。

The cumulative contribution rate is the contribution rate sequentially added in a predetermined order. For example, the sum of the contribution rates from the first principal component to the kth principal component (k is an integer of 2 or more) is called the cumulative contribution rate up to the kth principal component. The cumulative contribution rate up to the k-th principal component indicates what percentage of the total amount of information the k principal components hold.

In this embodiment, a principal component vector required for biometric determination is extracted by setting a threshold value (X%) indicating what percentage of the total amount of information the principal component has.

For example, from three subspaces of “biological information 1” 1701, “biological information 2” 1702, and “non-biological information 1” 1703 shown in FIG. 17, the contribution ratio of the principal component analysis is 95% (threshold of X% When a principal component vector satisfying one example) is extracted, a subspace 1801 obtained by the principal component analysis as shown in FIG. 18 is obtained.

Since the partial space 1801 obtained by this principal component analysis is a space (vector) common to the three partial spaces of “biological information 1” 1701, “biological information 2” 1702, and “non-biological information 1” 1703, It can be determined that it is not effective for biometric determination.

Subsequently, the unnecessary space extraction unit 406 projects a partial space 1801 onto a complementary space to create a partial space excluding the partial space 1801.

For example, the projected subspace is Pc, the input subspace is P, its projection matrix is P=[p1, p2,..., pn], the constrained subspace (complement space) is C, and its projection matrix is C=[c1. , c2,..., Cn], the number of bases used in P is K, and the number of bases used in projection is C.

At this time, the basis vector pci of the subspace Pc obtained by projecting P onto C is represented by the following (Equation 12).

例えば、図１７に示す「生体情報１」１７０１、「生体情報２」１７０２、「非生体情報３」１７０３を、部分空間１８０１の補空間に射影したときのイメージを図１９に示す。

For example, FIG. 19 shows an image when the “biological information 1” 1701, the “biological information 2” 1702, and the “non-biological information 3” 1703 shown in FIG. 17 are projected onto the complementary space of the partial space 1801.

For example, when the subspace 1801 obtained by the principal component analysis is 30-dimensional in the 251-dimensional discriminant space 1700, the number of elements of the projection vector is 251. Further, the complementary space 1900 of the subspace 1801 obtained by the principal component analysis becomes a 221-dimensional space. In the example of FIG. 19, images of a partial space 1901 of “biological information 1” 1701, a partial space 1902 of “biological information 2” 1702, and a partial space 1903 of “non-biometric information 1” on a complementary space 1900 are shown. ing.

FIG. 20 is a diagram showing an example of a projection vector according to the second embodiment. The projection vector shown in FIG. 20 shows an example of the projection vector for projecting the feature information to be the target of the biometric determination onto the complementary space 1900 (221 dimensions) of the subspace 1801 obtained by the principal component analysis.

The unnecessary space extraction unit 406 extracts unnecessary elements that are unnecessary elements (dimensions) from the projection vector as shown in FIG. 20, for example. For example, in FIG. 20, the unnecessary space extraction unit 406 determines that the element corresponding to the element number whose absolute value of all values corresponding to each element number is less than the threshold is an unnecessary element, and the unnecessary space information storage unit 412. For example, FIG. 21 shows an example in which the threshold value is set to “0.010” in FIG.

FIG. 21 is a diagram showing an example of unnecessary elements of the projection vector according to the second embodiment. In the example of FIG. 21, the element numbers 3, 4, 7, and 246 have the absolute values of all values corresponding to the eigenvectors μ1 to μ221 less than the threshold value “0.010”. , And is stored in the unnecessary space information storage unit 412.

Preferably, the unnecessary space extraction unit 406 removes the projection vector (second projection vector) from the projection vector (first projection vector) shown in FIG. 20 by removing unnecessary elements stored in the unnecessary space information storage unit 412. It is stored in the identification space information storage unit 411.

Also in the present embodiment, the biometric determination unit 404 uses the second projection vector stored in the identification space information storage unit 411, as in the first embodiment, of the feature information extracted by the feature information extraction unit 403. Perform discriminant analysis.

Preferably, the feature information extraction unit 403 also excludes unnecessary elements (unnecessary dimensions) stored in the unnecessary space information storage unit 412 from the 251-dimensional feature information when extracting the feature information, and extracts the feature information. Extract.

The principal component vector with the highest contribution rate is information common to each class (identification space). Therefore, in the present embodiment, since unnecessary elements common to each class are removed from the second projection vector used for biometric determination, each class is differentiated and the determination accuracy of biometric determination is improved. Can be expected.

<Process flow>
FIG. 22 is a flowchart showing an example of unnecessary space extraction processing according to the second embodiment.

In step S2201, the characteristic information extraction unit 403 reads the images captured by the image capturing unit 401 at predetermined time intervals.

In step S2202, the feature information extraction unit 403 extracts feature information (for example, CHLAC feature amount) from the image read in step S2201.

Subsequently, the unnecessary space extraction unit 406 executes the unnecessary space extraction processing 2200 shown in steps S2203 to S2212.

In step S2203, the unnecessary space extracting unit 406 extracts unnecessary elements that are commonly present in each class (for example, “biometric information 1”, “biometric information 2”, and “non-biometric information 1” in FIG. 17). Then, the above-mentioned principal component analysis is performed.

In step S2204, the unnecessary space extraction unit 406 extracts the principal component vector whose cumulative contribution ratio is X%.

In step S2205, the unnecessary space extraction unit 406 creates a subspace 1801 (common subspace) obtained by the principal component analysis as shown in FIG. 18, for example.

In step S2206, the unnecessary space extraction unit 406 creates a complementary space 1900 of the subspace 1801 obtained by the principal component analysis as shown in FIG. 19, for example.

In the processing of steps S2207 to S2210, the unnecessary space extraction unit 406 determines whether or not each element is an unnecessary element from the projection vector element number 1 to 251 (Nmax) as shown in FIG. 20, for example. To do.

For example, in step S2207, the unnecessary space extraction unit 406 initializes the element number N to 1.

In step S2208, the unnecessary space extraction unit 406 determines whether the absolute value of each value of the element number N is less than a threshold value (for example, "0.010").

When the absolute value of each value of the element number N is not less than the threshold value (for example, “0.010”), the unnecessary space extraction unit 406 moves the process to step S2210. On the other hand, when the absolute value of each value of the element number N is less than the threshold value (for example, “0.010”), the unnecessary space extracting unit 406 shifts the processing to step S2209.

When the process proceeds to step S2209, the unnecessary space extraction unit 406 stores, in the unnecessary space information storage unit 412, the element number N for which the absolute value of each value is determined to be less than the threshold value (determined as an unnecessary element) as an unnecessary element. Remember.

In step S2210, the unnecessary space extraction unit 406 determines whether the element number N is greater than or equal to Nmax which is the maximum value (for example, 251) of the element number N.

If the element number N is not greater than or equal to Nmax, the unnecessary space extraction unit 406 moves the process to step S2211. On the other hand, when the element number N is Nmax or more, the unnecessary space extracting unit 406 shifts the processing to step S2212.

Upon proceeding to step S2211, the unnecessary space extraction unit 406 adds 1 to the element number N, returns the processing to step S2208, and repeats similar processing.

When proceeding to step S2212, the unnecessary space extraction unit 406 removes the unnecessary elements stored in the unnecessary space information storage unit 412 from the projection vector (first projection vector) as shown in FIG. A second projection vector) is created. The unnecessary space extraction unit 406 also stores the created second projection vector in the identification space information storage unit 411.

By the above processing, for example, information such as the element numbers 3, 4, 7, and 246 among the projection vectors shown in FIG. 21 is stored in the unnecessary space information storage unit 412. Further, for example, among the projection vectors shown in FIG. 21, the 247-dimensional projection vector excluding the element numbers 3, 4, 7, and 246 is stored in the identification space information storage unit 411.

The flow of the biometric determination process according to the second embodiment is the same as the biometric determination process of the first embodiment shown in FIG. However, the biometric determination unit 404 according to the second embodiment uses the second projection vector stored in the identification space information storage unit 411 to perform the biometric determination using the subspace method described above.

As described above, according to the present embodiment, the characteristic information extraction unit 403 and the biometric determination unit 404 execute the processing by excluding the unnecessary elements stored in the unnecessary space information storage unit 412. To be done.

Further, according to the present embodiment, since unnecessary elements common to each class are removed from the second projection vector used for biometric determination, each class is differentiated and the determination accuracy of biometric determination is improved. You can expect an effect.

[Other Embodiments]
The processing shown in the first embodiment and the second embodiment described above is an example, and various modifications and applications are possible.

As an example, when the image captured by the image capturing unit 401 has noise, the information processing system 100 can discard the image and use the next image to improve the accuracy of the biometric determination.

FIG. 23 is a flowchart showing an example (1) of the biometric determination process according to another embodiment. Note that the processing in steps S1501 and S1502 to 1508 in FIG. 23 is the same as the living body determination processing according to the first embodiment shown in FIG. 15, so differences will be mainly described here.

In step S2301, the noise determination unit 405 determines whether the amount of noise included in the image captured by the image capturing unit 401 is less than a predetermined value.

When the amount of noise included in the image captured by the image capturing unit 401 is equal to or larger than the predetermined value, the noise determining unit 405 discards the image captured by the image capturing unit 401 and returns the process to step S1501.

On the other hand, when the amount of noise included in the image captured by the imaging unit 401 is equal to or greater than the predetermined value, the noise determination unit 405 shifts the processing to step S1503 and starts the biometric determination processing.

Through the above processing, when the image captured by the image capturing unit 401 has noise, the information processing system 100 discards the image and uses the next image to improve the accuracy of the biometric determination. ..

As another example, the image forming apparatus 101 performs high-speed biometric determination using the unnecessary element information stored in the unnecessary space information storage unit 412 and the second projection vector stored in the identification space information storage unit 411. Whether or not can be settable.

For example, the image forming apparatus 101 can display a biometric determination mode setting screen 2400 as shown in FIG. 24 on the operation panel 226 and accept settings made by an administrator, a user, or the like. In the example of FIG. 24, on the biometric determination mode setting screen 2400, a “high-speed processing mode” button 2401 for selecting unnecessary element information and high-speed biometric determination using the second projection vector, and a normal biometric determination A “normal processing mode” button 2402 for performing the above is displayed.

The administrator or the user selects the “high-speed processing mode” button 2401 from the biometric determination mode setting screen 2400 to select unnecessary element information and high-speed biometric determination using the second projection vector. be able to. In addition, the administrator or the user selects the “normal processing mode” button 2402 from the biometric determination mode setting screen 2400 to perform high-speed biometric determination using unnecessary element information and the second projection vector. It is possible to select a normal biometric determination that is not performed.

FIG. 25 is a flowchart showing an example (2) of the biometric determination process according to another embodiment. Note that the processes of steps S1501, S1502, and S1504 to S1508 of FIG. 25 are the same as the biometric determination process according to the first embodiment shown in FIG. 15, so here, the differences from the first embodiment will be mainly described. I will explain.

In step S2501, the biometric determination unit 404 determines whether the image forming apparatus 101 is set to the high-speed processing mode in which the biometric determination is performed at high speed using the unnecessary element information and the second projection vector.

When the high-speed processing mode is set, the biometric determination unit 404 shifts the processing to step S1503 and executes the biometric determination processing of the first embodiment shown in FIG.

On the other hand, when the high-speed processing mode is not set, the biometric determination unit 404 executes the normal biometric determination process using the biometric feature information, the non-biometric feature information, and the like.

By the above-described processing, for example, in the image forming apparatus 101 shown in FIG. 5, even when the necessary information is not stored in the identification space information storage unit 411 and the unnecessary space information storage unit 412, the image forming apparatus 101 determines the biological body. Will be able to do.

100 Information Processing System 101 Image Forming Apparatus (Example of Image Processing Apparatus)
401 Imaging unit 403 Feature information extraction unit 404 Biometric determination unit 406 Unnecessary space extraction unit 411 Identification space information storage unit (second storage unit)
412 unnecessary space information storage unit (first storage unit)

JP, 2006-330936, A

Claims (11)

Using the information of the identification space of the living body previously extracted from the image of the living body, an information processing system for performing a living body determination to determine whether the person photographed in the image is a living body,
A photographing unit for photographing the image,
A characteristic information extraction unit that extracts characteristic information of the person photographed in the image,
A first stored information of an unnecessary element that is not used for the biometric determination among the elements of the first projection vector that projects the characteristic information extracted by the characteristic information extraction unit onto the discrimination space including the identification space of the biological body. Storage unit of
Whether the person imaged in the image is a living body by using the second projection vector obtained by removing the unnecessary element from the first projection vector and the feature information extracted by the feature information extraction unit. A biological determination unit for determining
Information processing system having. The feature information extraction unit,
The information processing system according to claim 1, wherein the characteristic information excluding the unnecessary elements is extracted from the image captured by the imaging unit. The feature information is a vector of multiple dimensions,
The information processing system according to claim 1, wherein the unnecessary element information includes information that specifies a dimension that is not used for the biometric determination, among the plurality of dimension vectors. The information processing system according to claim 1, further comprising a second storage unit that stores the second projection vector obtained by removing the unnecessary element from the first projection vector. Among the elements of the first projection vector, an element whose absolute value of a value corresponding to a plurality of identification spaces including the identification space of the living body is less than a threshold value is determined to be the unnecessary element, and the first storage The information processing system according to any one of claims 1 to 4, further comprising: an unnecessary space extraction unit that is stored in the unit. Among the elements of the first projection vector, an element common to a plurality of identification spaces including the identification space of the living body and the identification space of the non-living body is determined to be the unnecessary element by the principal component analysis, The information processing system according to any one of claims 1 to 5, further comprising: an unnecessary space extraction unit that is stored in one storage unit. An information processing apparatus included in the information processing system according to claim 1,
An acquisition unit for acquiring the image,
A characteristic information extraction unit that extracts characteristic information of the person photographed in the image,
From the characteristic information extracted by the characteristic information extracting unit, information of unnecessary elements not used for biometric determination is extracted and stored in the first storage unit, and a projection vector is created by removing the unnecessary elements from the characteristic information. And an unnecessary space extracting section which is stored in the second storage section,
An information processing apparatus having. Using the information of the identification space of the living body extracted in advance from the image of the living body, an image processing device for performing a living body determination to determine whether the person photographed in the image is a living body,
A photographing unit for photographing the image,
A characteristic information extraction unit that extracts characteristic information of the person photographed in the image,
A first stored information of an unnecessary element that is not used for the biometric determination among the elements of the first projection vector that projects the characteristic information extracted by the characteristic information extraction unit onto the discrimination space including the identification space of the biological body. Storage unit of
Whether the person imaged in the image is a living body by using the second projection vector obtained by removing the unnecessary element from the first projection vector and the feature information extracted by the feature information extraction unit. A biological determination unit for determining
An image processing apparatus having. The image processing apparatus according to claim 8, wherein
A main body having at least an engine section,
An operation unit that receives an operation instruction to the main body,
Have
An image processing apparatus comprising the feature information extraction unit, the first storage unit, and a living body determination unit in the operation unit. Using the information of the identification space of the living body previously extracted from the image of the living body, a computer for performing a living body determination for determining whether the person photographed in the image is a living body,
A photographing unit for photographing the image,
A characteristic information extraction unit that extracts characteristic information of the person photographed in the image,
A first stored information of an unnecessary element that is not used for the biometric determination among the elements of the first projection vector that projects the characteristic information extracted by the characteristic information extraction unit onto the discrimination space including the identification space of the biological body. Storage unit of
Whether the person imaged in the image is a living body by using the second projection vector obtained by removing the unnecessary element from the first projection vector and the feature information extracted by the feature information extraction unit. A biological determination unit for determining
Program to function as. The step of acquiring the captured image,
Extracting characteristic information of the person photographed in the image,
A step of extracting information of unnecessary elements not used for biometric determination from the extracted characteristic information and storing the extracted information in the first storage unit;
Creating a projection vector excluding the unnecessary elements from the characteristic information and storing the projection vector in a second storage unit;
A program that causes a computer to execute.

Images