JP4953292B2 - Image input device, personal authentication device, and electronic device - Google Patents

Description

  The present invention relates to an image input device suitable for the purpose of inputting an image of a subject inside a living body (for example, a vein of a human finger or a fingerprint of a dermis under the skin), and a personal authentication device using such an image of the subject. .

  Patent Documents 1, 2, and 3 describe personal authentication devices that image a vein pattern inside a finger while illuminating a human finger with infrared light or near infrared light, and perform personal authentication from the vein pattern. .

  Patent Documents 4 and 5 and Non-Patent Document 1 describe thin image input devices using a compound eye optical system. Non-Patent Document 1 also describes an application example to fingerprint input for the purpose of application to a fingerprint authentication system.

  Further, for example, a contact type image sensor having a rod lens array in which refractive index distribution type rod lenses are arranged in a line and a line imaging element in which the rod lens and the light receiving element of the rod lens array correspond one-to-one is disclosed in, for example, a patent. Reference 6 describes.

JP 2004-27281 A JP 2005-92375 A JP 7-21373 A Japanese Patent No. 3705766 JP 2001-61109 A JP 2002-218160 A Igenji et al., “Development of Thin Image Input Device Using Compound Eye Optical System”, Journal of the Institute of Image Information and Television Engineers Vol. 57, No. 9, pp. 1135-1141 (2003)

  The personal authentication devices described in Patent Documents 1, 2, and 3 use a monocular optical system, so that the subject distance and the imaging distance are limited, and there is a limit to thinning. In order to mount the personal authentication device on an electronic device such as a mobile phone, a small information terminal (such as a PDA), or a laptop computer, it is required to make the personal authentication device thinner.

  In order to reduce the thickness of the personal authentication device, it is most important to reduce the thickness of an image input device for inputting an image of a subject inside a living body such as a finger vein or a dermal fingerprint. In order to reduce the thickness of the image input device, it is generally advantageous to apply a compound eye optical system as described in Patent Documents 4 and 5 and Non-Patent Document 1. However, an image input device for personal authentication does not have to be simply reduced in thickness, but must be capable of inputting a subject image inside a living body for use in personal authentication as a high-quality image.

  Note that the contact type image sensor using the rod lens array and the line imaging element as described in Patent Document 6 has a limit in resolving power due to the relationship of imaging with each rod lens as one pixel, and high definition. This is not suitable for inputting a subject image. In addition, the gradient index rod lens requires an optical path length of about several millimeters or more in order to obtain an image forming action, and thus there is a limit to thinning the optical system.

  Therefore, a main object of the present invention is to provide a thin image input device suitable for use as a high-definition image for inputting an image of a subject inside a living body such as a finger vein or a dermal fingerprint, and a personal authentication device using the same. There is to do.

The invention according to claim 1 is an image input device for inputting an image of a subject existing inside a stationary living body .
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Moving means for moving at least the image sensor of the imaging optical system in a plane perpendicular to the optical axis of the lens;
Stitching processing means for creating a single compound eye image by performing a process of stitching a plurality of compound eye images picked up by the image sensor when the image sensor moves to a plurality of different positions in the plane;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for each single-eye image for the compound eye image generated by the splicing processing means;
Reconstructing processing means for generating a single reconstructed image by performing processing for reconstructing a single image from the compound eye image corrected by the correcting processing means using the parallax between the single-eye images. ,
An image input device for inputting a reconstructed image generated by the reconstructing processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. It is characterized by that.
The invention according to claim 2 is an image input device for inputting an image of a subject existing inside a stationary living body .
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Moving means for moving at least the image sensor of the imaging optical system in a plane perpendicular to the optical axis of the lens;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for each single-eye image for the compound eye image captured by the imaging device;
When the image sensor is moved to a plurality of different positions in the plane, the image is picked up by the image sensor, and a plurality of compound images corrected by the correction process are connected to generate one compound eye image. , Stitching processing means,
Reconstructing processing means for generating a single reconstructed image by performing processing for reconstructing a single image from the compound eye images generated by the stitching processing means using the parallax between the single-eye images. And
An image input device for inputting a reconstructed image generated by the reconstructing processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. It is characterized by that.
The invention according to claim 3 is an image input device for inputting an image of a subject existing inside a stationary living body .
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Moving means for moving at least the image sensor of the imaging optical system in a plane perpendicular to the optical axis of the lens;
Stitching processing means for creating a single compound eye image by performing a process of stitching a plurality of compound eye images picked up by the image sensor when the image sensor moves to a plurality of different positions in the plane;
Reconstructing processing means for generating a single reconstructed image by performing processing for reconstructing a single image from a compound eye image generated by the splicing processing means using parallax between the single-eye images;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for the reconstructed image generated by the reconstruction processing means,
An image input device for inputting the reconstructed image corrected by the correction processing means as the image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. It is characterized by that.

The invention described in claim 4 is an image input device for inputting an image of a subject existing inside a stationary living body .
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Moving means for moving at least the image sensor of the imaging optical system in a plane perpendicular to the optical axis of the lens;
Processing for reconstructing a single image from the plurality of compound eye images captured by the image sensor when the image sensor moves to a plurality of different positions in the plane, using the parallax between the single-eye images Reconstructing processing means for generating a plurality of reconstructed images by performing
A stitching processing means for performing a process of stitching a plurality of reconstructed images generated by the reconstruction processing means to generate a single image;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for a single image generated by the stitching processing means,
An image input device for inputting a single image corrected by the correction processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. It is characterized by that.
The invention according to claim 5 is an image input device for inputting an image of a subject existing inside a stationary living body .
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Moving means for moving at least the image sensor of the imaging optical system in a plane perpendicular to the optical axis of the lens;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for each single-eye image for the compound eye image captured by the imaging device;
When the image sensor moves to a plurality of different positions in the plane, the parallax between the single-eye images is used from each of a plurality of compound-eye images captured by the image sensor and corrected by the correction processing unit. Reconstructing processing means for reconstructing a single image and generating a plurality of reconstructed images;
Stitching processing means for performing a process of stitching together the plurality of reconstructed images generated by the reconstruction processing means to generate a single image;
An image input device for inputting a single image generated by the stitching processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. It is characterized by that.
The invention described in claim 6 is an image input device for inputting an image of a subject existing inside a stationary living body .
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Moving means for moving at least the image sensor of the imaging optical system in a plane perpendicular to the optical axis of the lens;
Processing for reconstructing a single image from the plurality of compound eye images captured by the image sensor when the image sensor moves to a plurality of different positions in the plane, using the parallax between the single-eye images Reconstructing processing means for generating a plurality of reconstructed images by performing
Correction processing means for performing image deterioration correction processing by the lens based on optical transfer function data relating to the lens prepared in advance for a plurality of reconstructed images generated by the reconstruction processing means;
Stitching processing means for generating a single image by stitching together the plurality of reconstructed images corrected by the correction processing means;
An image input device for inputting a single image generated by the stitching processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. It is characterized by that.

The invention according to claim 7 is an image input device for inputting an image of a subject existing inside a living living body .
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
A plurality of reconstructed images are obtained by performing a process of reconstructing a single image using parallax between the single-eye images from each of a plurality of compound-eye images continuously captured by the image sensor at predetermined time intervals. Reconfiguration processing means to generate,
Reconstructed image parallax detection means for detecting parallax between a plurality of reconstructed images generated by the reconstruction processing means;
A stitching processing means for generating a single image by performing a process of stitching together a plurality of reconstructed images generated by the reconstruction processing means using the parallax detected by the parallax detection means between the reconstructed images When,
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for a single image generated by the stitching processing means,
An image input device for inputting a single image corrected by the correction processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. It is characterized by that.
The invention according to claim 8 is an image input device for inputting an image of a subject existing inside a living living body .
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for each single-eye image for the compound eye image captured by the imaging device;
A process of reconstructing a single image using parallax between the single-eye images from each of a plurality of compound-eye images continuously captured at a predetermined time interval and corrected by the correction unit. Reconstructing processing means for generating a plurality of reconstructed images, and
Reconstructed image parallax detection means for detecting parallax between a plurality of reconstructed images generated by the reconstruction processing means;
A stitching processing means for generating a single image by performing a process of stitching together a plurality of reconstructed images generated by the reconstruction processing means using the parallax detected by the parallax detection means between the reconstructed images And
An image input device for inputting a single image generated by the stitching processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. It is characterized by that.
The invention according to claim 9 is an image input device for inputting an image of a subject existing inside a living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
A plurality of reconstructed images are obtained by performing a process of reconstructing a single image using parallax between the single-eye images from each of a plurality of compound-eye images continuously captured by the image sensor at predetermined time intervals. Reconfiguration processing means to generate,
Correction processing means for performing image deterioration correction processing by the lens based on optical transfer function data relating to the lens prepared in advance for a plurality of reconstructed images generated by the reconstruction processing means;
Reconstructed image parallax detecting means for detecting parallax between a plurality of reconstructed images corrected by the correction processing means;
A splicing processing unit that generates a single image by performing a process of splicing a plurality of reconstructed images corrected by the correction processing unit using the parallax detected by the reconstructed inter-parallax detection unit; Have
An image input device for inputting a single image generated by the stitching processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. It is characterized by that.

According to a tenth aspect of the present invention, in the image input device for inputting an image of a subject existing inside a moving living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
A compound eye inter-image parallax detecting means for detecting a parallax between a plurality of compound eye images continuously captured by the imaging device at a predetermined time interval;
A stitching processing means for generating a single compound eye image by performing a process of stitching a plurality of compound eye images captured by the image sensor using the parallax detected by the compound eye inter-image parallax detection means;
Reconstructing processing means for generating a single reconstructed image by performing processing for reconstructing a single image from the compound eye images generated by the stitching processing means using the parallax between the single-eye images. And
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for each single-eye image for the compound eye image generated by the splicing processing means;
An image input device for inputting the reconstructed image corrected by the correction processing means as the image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. It is characterized by that.
The invention according to claim 11 is an image input device for inputting an image of a subject existing inside a living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for each single-eye image for the compound eye image captured by the imaging device;
A compound-eye image parallax detection unit that detects parallax between a plurality of compound-eye images that are continuously captured at predetermined time intervals by the imaging device and corrected by the correction unit;
A stitching processing means for generating a single compound eye image by performing a process of stitching a plurality of compound eye images captured by the image sensor using the parallax detected by the compound eye inter-image parallax detection means;
Reconstructing processing means for generating a single reconstructed image by performing processing for reconstructing a single image from the compound eye images generated by the stitching processing means using the parallax between the single-eye images. And
An image input device for inputting a reconstructed image generated by the reconstructing processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. It is characterized by that.
According to a twelfth aspect of the present invention, in an image input device for inputting an image of a subject existing inside a moving living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
A compound eye inter-image parallax detecting means for detecting a parallax between a plurality of compound eye images continuously captured by the imaging device at a predetermined time interval;
A stitching processing means for generating a single compound eye image by performing a process of stitching a plurality of compound eye images captured by the image sensor using the parallax detected by the compound eye inter-image parallax detection means;
Reconstructing processing means for generating a single reconstructed image by performing processing for reconstructing a single image from a compound eye image generated by the splicing processing means using parallax between the single-eye images;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for the reconstructed image generated by the reconstruction processing means,
An image input device for inputting the reconstructed image corrected by the correction processing means as the image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. It is characterized by that.

Invention of claim 13, the image input apparatus according to the invention of any one of claims 1 to 12, means for performing a control for intermittently emitting the light source, the imaging in the light emitting state of the light source By taking the difference between the compound eye image captured by the element and the compound eye image captured by the image sensor in the extinction state of the light source, a bias component due to light other than near-infrared light emitted by the light source is removed. And a bias component removal processing unit that generates the compound eye image, and the compound eye image after the bias component removal generated by the bias component removal processing unit is treated as a compound eye image captured by the imaging device. And

The invention of claim 14, wherein, in the image input apparatus according to the invention of any one of claims 1 to 12, with respect to the light source, the intensity of the near infrared light irradiated on the living body sinusoidally Means for performing control, and a plurality of compound-eye images respectively captured by the imaging element at a plurality of different phase points in a sinusoidal change period of the intensity of near-infrared light irradiated on the living body by the light source. Bias component removal processing means for generating a compound eye image in which a bias component due to light other than near-infrared light irradiated by the light source is removed by arithmetic processing, and the bias generated by the bias component removal processing means The compound eye image after component removal is treated as a compound eye image imaged by the image sensor.

According to a fifteenth aspect of the present invention, there is provided a personal authentication device based on the image input device according to any one of the first to fourteenth aspects and a subject image input by the image input device. And authentication processing means for performing the above.

According to a sixteenth aspect of the present invention, there is provided an electronic apparatus comprising the personal authentication device according to the fifteenth aspect of the present invention, wherein the operation is controlled in accordance with a result of the personal authentication processing by the personal authentication device.

According to the present invention, the following effects can be obtained.
(1) Since the imaging optical system has a structure that can be easily thinned, a thin image input device can be realized as a whole.
(2) Although near-infrared light is absorbed by a light source in a portion where a vein or the like does not exist, but absorbs near-infrared light with a light source, a clear image such as a vein is formed on the image sensor. be able to. Further, in a compound eye optical system, it is normal that the surface having the positive power of the lens in the lens array (for example, the convex surface of the plano-convex lens) is directed to the subject side. Since a lens having a surface having zero or negative power and a surface having positive power on the image surface side (for example, a plano-convex lens having a convex surface facing the image surface side) is used, even under a condition where the subject distance is small The change in MTF due to the difference in the expected angle of the subject (the incident angle of the light beam to the lens) is reduced, and in-plane errors such as distortion and curvature are also suppressed. Furthermore, the subject distance varies depending on individual differences in skin thickness, etc., but reconstructing a single image from a compound eye image using parallax between single-eye images can easily cope with variations in the subject distance, reducing resolution, etc. Can be compensated. As described above, it is possible to input a high-quality, high-definition image of a subject inside a living body such as a vein.
(3) Since the image degradation correction process by the lens is performed based on the optical transfer function data of the lens, it is possible to input a higher quality image corrected by the lens image degradation. Then, as a lens of the lens array, by using a lens having a surface having zero or negative power on the object side and a surface having positive power on the image surface side, the expected angle of the object (to the lens) Since the change of the MTF due to the difference in the incident angle of the light beam is suppressed, the image degradation correction process is also easy.
(4) By selecting the optical transfer function data used for the image degradation correction process according to the subject distance, high-accuracy image degradation correction can be performed even if the subject distance fluctuates. The influence can be reduced and the quality of the input image can be further improved.
(5) Since at least the image sensor of the imaging optical system is moved or the subject is moved and the subject is moved, a standard area image sensor or line image sensor that is smaller than the size of the subject to be imaged can be used. It is. For this reason, the cost of the image input apparatus can be reduced as compared with the case where a dedicated image sensor that matches the size of the subject is used.
(6) Since an image input device that can be easily reduced in thickness is used, a thin personal authentication device suitable for use in an electronic device can be realized.
(7) Since an image of an in-vivo subject such as a vein can be input as a high-quality image, highly accurate personal authentication can be performed.
(8) Since it is not easily affected by individual differences in skin thickness or external light, stable and highly accurate personal authentication is possible.
(9) Since the personal authentication device can be reduced in thickness, it is possible to avoid an increase in the size of the electronic device due to the provision of the personal authentication device.
(10) The security for the electronic device can be improved by performing login control or the like in the electronic device of the small information terminal or the notebook personal computer according to the result of the personal authentication processing by the personal authentication device.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Here, a human finger is assumed as a living body, and a vein inside the finger is assumed as a subject. A case is assumed in which a vein image is input and personal authentication is performed using features of the vein pattern. In addition, in the drawings referred to in the following description, the same reference numerals are used for the same or equivalent elements in order to reduce duplication of description.

[First Embodiment]
FIG. 1 is an explanatory diagram of an image input apparatus and a personal authentication apparatus according to the first embodiment of the present invention. In FIG. 1, an imaging optical system 100, a stitching processing unit 101, a preprocessing unit 102, a reconstruction calculation unit 103, and a postprocessing unit 104 constitute an image input device. The authentication calculation unit 105 and the registration data memory 106 constitute an authentication processing unit that performs a personal authentication process using a vein pattern. Such an authentication processing means and an image input device constitute a personal authentication device.

  In FIG. 1, 1 schematically shows a finger (living body). The imaging optical system 100 inputs the image by dividing and imaging the vein 2 inside the finger 1 in a stationary state as a subject. The imaging optical system 100 includes a light source 6, a lens array 3, a light shielding member (light shielding means) 4, an imaging element 5, and an optical bandpass filter 7.

  The lens array 3 is for forming a subject image, and a plurality of lenses 3a are two-dimensionally arrayed in a plane substantially orthogonal to the lens optical axis.

  According to the present invention, each lens 3a of the lens array 3 has a surface having 0 or negative power on the subject side and a surface having positive power on the image surface side (lower surface side in the figure). It is said. In the present embodiment (and each embodiment described later), a plano-convex lens having a convex surface facing the image surface is used as the lens 3a as shown in the figure. The convex surface may be a spherical surface or an aspherical surface, and if it is an aspherical surface, the degree of freedom in design for improving the optical characteristics of the lens 3a is increased.

  The light shielding member 4 is for preventing crosstalk on the image plane of the light beam that has passed through each lens 3a of the lens array 3 and suppressing noise light such as ghost and flare. The light shielding member 4 of the present embodiment has a height from each lens 3a of the lens array 3 to the image plane, and has a structure in which openings (through holes) corresponding to the lenses 3a are arranged in a two-dimensional array. The portion has a square cross-sectional shape. The light shielding member 4 may be a pinhole array having an opening corresponding to each lens 3a of the lens array 3, or may be provided on the upper surface or the lower surface or both upper and lower surfaces of a transparent parallel plate in which the opening corresponding to each lens 3a is formed. What formed the opaque film | membrane by vapor deposition etc. may be used.

  The image pickup device 5 is an image pickup means for picking up a compound eye image that is a set of images (single-eye images) formed by the respective lenses 3a of the lens array 3, and an area in which the light receiving elements 5a are arranged in a two-dimensional array. It is an image sensor. As such an image sensor 5, a standard CCD image sensor or a CMOS image sensor can be used. Here, the imaging element 5 has a circuit that adjusts the gain of a photoelectric conversion signal by the light receiving element 5a or converts an analog signal into a digital signal, and outputs a captured image as digital image data. . Note that the imaging element 5 captures each single-eye image as a two-dimensional image composed of a plurality of pixels.

  The light source 6 irradiates the finger (living body) 1 with near infrared light having a low absorption rate by the living body, and is a light emitting diode (LED), for example. Near-infrared light irradiated on the finger 1 by the light source 6 is absorbed by reduced hemoglobin in the vein (subject) 2 inside the finger 1, but is hardly absorbed in portions other than the vein 2. The vein pattern can be seen through. The vein pattern is formed as a compound eye image on the imaging surface of the imaging device 5 by each lens 3a of the lens array 3.

  The optical bandpass filter 7 transmits only light in a predetermined wavelength range including near infrared light emitted from the light source 6. The optical bandpass filter 7 is provided to remove the influence of noise light from a light source other than the light source 6. When it is not necessary to consider the incidence of noise light, or when the influence of noise light is removed by arithmetic processing as in the eighth embodiment described later, the optical bandpass filter 7 may be omitted. It is also possible to arrange the optical bandpass filter 7 on the image plane side of the lens array 3, for example, on the imaging plane of the imaging device 5.

  Although only one light source 6 is shown in FIG. 1, a plurality of light sources 6 may be installed so as to illuminate the area of the subject. A laser diode (LD) may be used as the light source 6. In FIG. 1, the light source 6 is arranged so as to illuminate the finger 1 from the surface (the upper surface in the drawing) that does not face the lens array 3. However, since the near-infrared light irradiated from the light source 6 is diffused in all directions inside the finger 1, the light source 6 may be arranged so as to illuminate the finger 1 from the side surface or the lower surface. Even vein images can be taken without problems. Further, a light guide that guides near-infrared light emitted from the light source 6 toward the finger 1 may be added.

  The imaging element 5 captures a compound eye image of a vein pattern (subject image) formed by the lens array 3 and outputs it as digital image data. This digital image data is input to a joining processing unit (joining processing means) 101. In the present embodiment, the imaging optical system 100 as a whole captures subject images at a plurality of different positions while being moved in the direction indicated by the arrow 15 in a plane orthogonal to the optical axis of the lens 3a. The plurality of captured compound eye images are input to the stitching processing unit 101. The stitching processing unit 101 performs a process of stitching a plurality of inputted compound eye images to generate one compound eye image.

  FIG. 2 shows an example of a drive mechanism that moves the imaging optical system 100. In FIG. 2, 8 is a stage to which the image sensor 5 is fixed, 10 is a ball screw engaged with the stage 8, and 9 is a stepping motor coupled to one end of the ball screw 10. As the pulse generated by the pulse generator 20 is supplied to the stepping motor 9 via the motor driver 19, the stepping motor 9 rotates and rotationally drives the ball screw 10, so that the stage 8 is indicated by an arrow. Move forward and backward. Reference numeral 21 denotes a counter that counts pulses supplied from the pulse generator 20, and the position of the stage 8, that is, the position or movement amount of the imaging optical system 100 can be detected from the value of the counter 21. The value of the counter 21 is supplied to the stitching processing unit 101 and the image sensor driving unit 22 as position information or movement amount information.

  The imaging optical system 100 may be driven using a DC motor, an ultrasonic motor, a piezo element, or the like instead of a stepping motor. Further, the position or movement amount may be detected using a linear scale, a displacement meter, or the like.

  An operation for imaging the vein pattern of the entire finger will be described. It is assumed that the size of the subject, that is, the vein 2 is 60 × 10 mm, and the area where the imaging optical system 100 can image once is 10 × 10 mm. The imaging optical system 100 is moved to a position where the tip (nail side end) of the finger 1 can be observed. With this position as the initial position, the image sensor drive unit 22 causes the image sensor 5 to perform the first imaging. Thereafter, the image pickup optical system 100 is moved in the direction of the arrow 15 and the image pickup device driving unit 22 causes the image pickup device 5 to pick up an image every time the image pickup optical system 100 is moved by 10 mm. In this manner, a total of six imaging operations are performed, so that six compound eye images covering the entire subject are captured and input to the stitching processing unit 101. The stitching processing unit 101 stitches the input six compound images in the moving direction (sub-scanning direction) of the imaging optical system 100 based on the position or movement amount information given from the counter 21, thereby One compound eye image covering the whole is generated. Note that if the image is taken at predetermined fixed intervals (fixed movement amounts), it is not always necessary to supply position information or movement amount information (count value of the counter 21) to the stitching processing unit 101.

  Moreover, since the near-infrared light irradiated by the light source 6 diffuses inside the finger 1, it is not always necessary to move the light source 6 as long as the near-infrared light reaches the entire finger.

  In addition, the lens array 3, the light shielding member 4, and the optical bandpass filter 7 of the imaging optical system 100 are sized so as to cover the entire subject, and imaging is performed while moving only the imaging device 5 in the direction of the arrow 15. Such an aspect may be included in the present embodiment. Even if the elements 3, 4, and 7 other than the image pickup device 5 are sized to cover the entire subject, the cost does not increase so much, and the driving load is reduced as compared with the case where the image pickup optical system 100 is moved as a whole.

  The compound eye image data generated by the stitching processing unit 101 is preprocessed by the preprocessing unit 102 and then transferred to the reconstruction calculation unit 103. In the preprocessing by the preprocessing unit 102, for example, a region of each single-eye image excluding the shadow portion by the light shielding member 4 in the compound eye image is extracted, and noise components such as isolated points are obtained by smoothing or averaging the single-eye image. Is removed, a single-eye image including a vein pattern is extracted, and further, an enhancement process for sharpening the vein pattern is performed on the single-eye image as necessary. The reconstruction calculation unit 103 reconstructs a single image from a pre-processed compound eye image by a reconstruction calculation using parallax between single-eye images described later. The single image data is subjected to post-processing such as removal of reconstruction noise by the post-processing unit 104 as necessary, and then input to the authentication calculation unit 104 as vein (subject) image data. Up to here is the operation as the image input apparatus. Note that the pre-processing and post-processing described above are pre-processing and post-processing of the reconstruction process. That is, the pre-processing unit 102 and the post-processing unit 104 together with the reconstruction calculation unit 103 constitute “reconstruction processing means”.

  The authentication calculation unit 105 extracts the feature amount of the vein pattern from the input vein image, and compares the feature amount with the feature amount of the registrant's vein pattern stored in the registration data memory 106. Perform personal authentication. That is, if the distance between the extracted feature quantity and the registrant's feature quantity is less than or equal to a predetermined value, the person to be authenticated (the owner of the finger 1) is authenticated as the registrant himself, but otherwise the authentication is rejected. Is done. Since the personal authentication technique itself using such a vein pattern is known, further details are omitted.

  The lens array 3 is made of a transparent resin or glass material. For example, the lens array 3 is formed by a processing method such as a reflow method, an area gradation mask method, or a polishing method, or a molding process using a mold manufactured by these processing methods. Etc. The light shielding member 4 can be similarly processed by using a material such as resin, glass, or metal, but an opaque material or a coating on a transparent material is used to suppress transmission and reflection of light.

  In FIG. 1, the opening (through hole) of the light shielding member 4 has a uniform cross-sectional area from the lens 3 a to the imaging surface of the imaging element 5 in a plane substantially orthogonal to the lens optical axis. However, the opening may have a tapered shape that decreases as the cross-sectional area approaches the imaging surface. Such a tapered opening makes it difficult for light rays obliquely incident on the opening to be reflected from the inner surface of the opening to the image pickup surface of the image pickup device 5, thereby effectively preventing flare and ghost due to internal reflection of the opening. Can be prevented. Moreover, if the required height of the light shielding wall member 4 becomes larger than the size of the opening, it becomes difficult to process the light shielding member 4. In this case, the light shielding member 4 having a height that can be easily processed can be formed, and the structure can be formed by laminating and bonding the light shielding member 4 in the height direction.

  FIG. 3 shows a simulation example of a compound eye image captured by the imaging optical system 100. FIG. 3A is an original image for generating a compound eye image, and FIG. 3B is a compound eye image. A black portion existing between single-eye images in the compound-eye image is a shadow by the light shielding wall member 4. The single-eye image is an image formed by each lens 3a of the lens array 3, and different portions of the subject are imaged according to the lens position. In FIG. 1, 2a represents a field of view by one lens 3a and corresponds to a region observed as a single eye image. Reference numeral 2b denotes a portion where the field of view overlaps between adjacent lenses 3a, and this represents an overlapping region common to adjacent single-eye images in FIG.

  Since there are individual differences in the distance from the finger skin surface to the vein, the distance from the lens array 3 to the vein 2 in FIG. 1 varies depending on the person to be authenticated. In contrast to the state shown in FIG. 1, when the vein 2 approaches the lens array 3 as shown in FIG. 4A or the height of the light shielding member 4 increases, there is no overlap area between adjacent single-eye images. On the contrary, when the vein 2 moves away from the lens array 3 as shown in FIG. 4B or the height of the light shielding member 4 becomes small as shown in FIG. 4B, the overlapping region 2b between the adjacent single-eye images. Becomes larger.

  When there is no overlapping area between adjacent single-eye images, individual single-eye images in the compound-eye image are extracted, and the single-eye images are simply joined together after reversing the up / down / left / right inversion by the lens 3a. A single image can be reconstructed. However, if there is an overlapping area between adjacent single-eye images, one of the overlapping areas becomes invalid, so reconstructing a single image by simply stitching the single-eye images makes the image smaller Since the number of pixels constituting the pixel number decreases, the resolution decreases. Also, as shown in FIG. 4B, when the vein moves away from the lens array, the overlapping area increases and the number of invalid pixels increases. At the same time, the optical magnification of the imaging optical system decreases and the vein pattern image becomes smaller. descend.

  In the present embodiment, in order to compensate for such a decrease in resolving power due to an increase in invalid pixels and a decrease in optical magnification, the reconstruction calculation unit 103 uses a reconfiguration using parallax between individual images as described below. Perform configuration processing.

  Since there is a parallax due to the positional relationship between the lens 3a and the vein 2, that is, the subject, between the single-eye images, each single-eye image is an image that is slightly shifted with the parallax. In the present specification, “parallax” between single-eye images represents a shift amount (unit: length) of each single-eye image with respect to the single-eye image used as a reference in a compound-eye image. When the parallax of a single-eye image is used, the structure of the subject buried in one pixel in the single-eye image can be reproduced. For the detection of parallax between single-eye images, for example, the sum of squares of luminance deviation between single-eye images obtained by the following equation (1) can be used.

(1) In the equation, I _B is the reference ommatidium images set arbitrarily in the compound-eye image, as a reference of parallax ommatidium images. I _m represents the ommatidium images, m has a value of N (the number of lenses 3a N are of the lens array 3) from 1 to the number of ommatidia image. P _x and P _y represent parallaxes in the x and y directions of the single-eye image with respect to the reference single-eye image. The luminance deviation between the single-eye image and the reference single-eye image is calculated for all the pixels constituting the single-eye image, and the square sum E is obtained. P _x, calculates the E while gradually changing the P _y, P _x when E is _minimized, the P _y, respectively at x, the parallax with respect to the reference ommatidium image in the y-direction. FIG. 5 shows a bird's-eye view of changes in E with changes in P _x and P _y when P _x is taken on the _x axis, P _y is taken on the _y axis, and E is taken on the z axis.

As described above, P _x and P _y when E is the minimum in FIG. 5 are parallaxes of the single-eye image with respect to the reference single-eye image in the x and y directions. When the size of the parallax is considered to be smaller than the pixel size of the image sensor 5, the individual image is enlarged so that the size of the parallax is the pixel size or an integer multiple of the pixel size, that is, The number of constituent pixels is increased, and the parallax may be obtained from the least square sum of the luminance deviation between the enlarged single-eye images. In enlarging a single-eye image in that case, it is necessary to use an interpolation calculation that determines the luminance of each pixel while referring to adjacent pixels. Since the enlargement ratio can be estimated from the optical magnification, the lens pitch of the lens array 3, and the pixel size of the image sensor 5, if the enlargement ratio is determined so that the estimated disparity has a length corresponding to the pixel size. Good. If the processing accuracy of the lens pitch of the lens array 3 is sufficiently high, the parallax between the individual images can be calculated geometrically if the distance of the subject from the lens array 3 is known. Therefore, the parallax between a pair of single-eye images may be detected, and the parallax between the single-eye images may be calculated based on the following equation (2). In equation (2), δ is the parallax of an arbitrary single-eye image, Δ is the parallax of the actually detected single-eye image, and N is the center of the reference single-eye image in the x or y direction (horizontal or vertical direction) in the image. The distance from the center of the single-eye image from which the parallax was detected to n, and n represents the distance from the center of the reference single-eye image to the center of any single-eye image.

  When the parallax between single-eye images is large, such as when the subject distance is short, the reference single-eye image is not fixed to one single-eye image, and parallax detection may be performed between adjacent single-eye images. In that case, one adjacent single-eye image is a reference single-eye image, and the other is a single-eye image for detecting parallax. Since there are single-eye images in which the vein pattern is not observed in the single-eye image, as described above, the single-eye image in which the vein pattern is observed in the preprocessing is extracted, and the parallax is detected for the single-eye image. The parallax of a single-eye image in which no image is observed may be calculated by the above equation (2) using the parallax detected for other single-eye images. Note that, instead of the method of minimizing the sum of squares of luminance deviations, a method based on cross-correlation calculation between individual images may be used.

  FIG. 6 is an explanatory diagram of a method for reconstructing a single image. In FIG. 6, pixel brightness is extracted from each single-eye image 30a in the compound-eye image 30, and the extracted pixel brightness is arranged at a position determined according to the position and parallax of the single-eye image in the reconstructed image 31 in the virtual space. The reconstructed image 31 can be obtained by repeating the same pixel luminance arrangement for all pixels of each single-eye image.

  In addition, when a pixel with lost luminance is generated in the reconstructed image 31 due to the influence of the size of parallax or the shadow of the light shielding member, interpolation is performed with reference to the luminance of the adjacent pixel. When the parallax is smaller than the pixel size, enlarge the reconstructed image so that the size of the parallax is the pixel size or an integer multiple of the pixel size. Can be done.

  FIG. 7 shows an example of a processing flow in the reconstruction calculation unit 103. First, a compound eye image is acquired (step S1). Next, a reference single-eye image for parallax detection is set from the single-eye images including the vein pattern extracted in the preprocessing (step S2). The parallax of each single-eye image with respect to this reference single-eye image is detected (step S3). However, for a single-eye image that does not include a vein pattern, the parallax is referred to by the equation (2). Then, reconstruction calculation from a compound eye image to a single image is performed using parallax (step S4), and the reconstructed single image is output (step S5). By such reconstruction processing, the structure of the subject buried in the pixels can be reproduced, and a single image with improved resolution can be obtained even when the subject is far away and the resolution is reduced.

  In addition, when the overlap between single-eye images is small, the detected parallax may become very small or become an abnormal value. Therefore, a threshold value regarding the magnitude of the parallax is set, and in step S4, for example, a comparison determination between the parallax and the threshold value is performed. It is also possible to reconstruct a single image by simply stitching them back to each other, and perform a reconstruction process using parallax as described above when the parallax is equal to or greater than a threshold value.

  In order to enable reconstruction processing using parallax between single-eye images as described above, imaging is performed by the imaging device 5 when the distance from the subject to the lens array 3 is within a predetermined allowable range. It is necessary that the adjacent single-eye images share one or more pixels of the common part of the subject. For this reason, when designing the lens array 3, when the distance from the lens array 3 to the subject is minimized within a predetermined allowable distance range, the adjacent single-eye image is always shielded from overlapping. The height of the member 4 and the distance between the lenses 3a of the lens array 3 are set. Alternatively, a transparent flat plate (not shown) for adjusting the distance from the lens array 3 to the subject so as not to be smaller than the minimum value of a predetermined allowable distance range is between the finger (living body) 1 and the lens array 3, for example. When the optical bandpass filter 7 is omitted, the transparent flat plate is installed instead of the upper surface of the optical bandpass filter 7. In this way, since a compound eye image having an overlapping region can always be acquired, there is no need to change the reconstruction processing method by the comparison determination with the parallax threshold as described above. Since the change in the subject distance is reflected in the parallax, the reconstruction process using the parallax can easily cope with the change in the subject distance caused by individual differences in skin thickness.

  In the above, the case where the input of the vein pattern inside the finger and the personal authentication using the vein pattern has been described. However, this embodiment (and each embodiment described below) can also be used for personal authentication by imaging a palm vein pattern or a finger dermis fingerprint pattern, and for non-invasive blood glucose level measurement. The present invention can also be applied to use for imaging internal biological information.

[Second Embodiment]
FIG. 8 is an explanatory diagram of an image input device and a personal authentication device according to the second embodiment of the present invention. The image input apparatus according to the present embodiment captures and inputs an image of the vein 2 of the finger 1 in a stationary state as in the case of the first embodiment, but the image degradation caused by each lens 3a of the lens array 3 is reduced. The point that the correction calculating part 201 and the memory 202 which comprise the correction process means for correction | amendment (MTF correction | amendment) are added is different from the said 1st Embodiment. The memory 202 stores in advance optical transfer function (OTF) data related to the plano-convex lens 3a with the convex surface facing the image surface side.

  FIG. 9 illustrates the relationship between the MTF, which is the gain of the optical transfer function of a plano-convex lens such as the lens 3a, and the subject prospective angle when the convex surface of the lens is directed to the subject side and to the image plane side. .

  FIG. 9A shows the MTF characteristics of a plano-convex lens with the convex surface facing the object side, and FIG. 9B shows the MTF characteristics of a plano-convex lens with the convex surface facing the image surface side like the lens 3a in the present invention. ing. 9A and 9B, the thin solid line, the dotted line, and the alternate long and short dash line represent the difference in the angle of light incident on the lens, that is, the difference in the expected angle of the subject. The thick solid line in FIG. 9B shows the corrected MTF characteristic.

  When the convex surface of the plano-convex lens is directed to the subject side, as shown in FIG. 9A, a high MTF is obtained up to a high spatial frequency band at a certain prospective angle. However, when the prospective angle changes, the MTF and the cut-off frequency are Both change greatly and the image deteriorates. To maintain a high MTF and cut-off frequency over the entire allowable angle, the allowable angle is very narrow, and multiple lenses and complex aspherical shapes are required. It is difficult to maintain the performance over a wide range of angles. In the range where the MTF is finite, the MTF characteristic can be improved by the correction calculation. However, in the case of FIG. 9A, since the change of the MTF is large depending on the expected angle, it is necessary to apply correction for each expected angle. Therefore, the processing load for remarkably increases. In addition, since the MTF is likely to drop to 0 with the included angle, the range of the expected angle that can be corrected is also limited.

  On the other hand, in the case of a plano-convex lens having a convex surface directed to the image surface side, like the lens 3a in the present invention, the MTF is lowered as a whole as seen in FIG. The change is small and the cut-off frequency does not change greatly. On the premise of MTF correction calculation, by optimizing the surface shape so that the convex surface of the lens faces the image surface side and the MTF is finite and uniform, the MTF can be obtained over a wide range of prospective angles with a small processing load. Performance can be maintained. For example, it is possible to easily correct the MTF characteristic as shown by the thick solid line in FIG. The MTF has been described above. However, when the convex surface is directed to the image surface side, in-plane errors such as distortion and curvature are reduced as compared with the case where the convex surface is directed to the subject side. What has been described above also applies to a lens having a negative power surface on the object side and a positive power surface on the image surface side.

  Next, the MTF correction process in the correction calculation unit 201 will be described. Prior to the correction calculation, the correction calculation unit 201 extracts each single-eye image excluding the shadow portion due to the light shielding member 4 in the compound eye image captured by the imaging element 5. Therefore, the pre-processing unit 102 does not need to perform processing for extracting a single-eye image excluding the shadow portion.

  The intensity data of the individual eye image in the image of the subject that has been deteriorated by the lens 3a, that is, the captured compound eye image, is expressed by the following equation (3).

In Expression (3), x and y are position coordinates in a single-eye image, I is intensity data of the single-eye image, S is intensity data of the subject, OTF is optical transfer function data of the lens 3a, and FFT is Fourier transform. The operator, FFT- ^1, is an inverse Fourier transform operator. The optical transfer function data OTF of the lens 3a can be obtained from autocorrelation calculation of the lens pupil function using lens wavefront aberration data obtained at the time of lens design.

  The correction calculation unit 201 uses the optical transfer function data OTF relating to the lens 3a that is calculated and stored in advance in the memory 202, and uses the lens 3a to calculate a compound eye image for each individual eye image by the following equation (4). A single-eye image (compound eye image that is a set of images) with improved MTF is generated by correcting image degradation. In equation (4), R is intensity data of the corrected single-eye image, and α is a constant for suppressing zero division and noise amplification during division.

  Note that the fact that the optical transfer function does not change according to the light beam angle is equivalent to the fact that the optical transfer function does not change even when the lens is installed at an inclination, so that an inclination installation error when assembling the imaging device 5 is established. It is possible to obtain an effect that the influence of is extremely small. Also, if the light condensing performance by the lens is high, the image plane position is slightly shifted in the optical axis direction, and the condensing point is widened to increase the image degradation. However, as shown in FIG. In a lens with low light condensing performance, the spread of the condensing point does not become relatively large even if the position of the image plane is slightly shifted in the optical axis direction. Therefore, it is possible to greatly reduce the influence of the distance error between the lens and the image plane. When the apparatus is assembled by combining the lens array 3 and the subject member 4, the lens 3 a having a convex surface facing the image surface side is fitted into the corresponding opening (hole) of the light shielding member 4. There is also an advantage that it is easy to remove.

  The correction by frequency filtering using FFT has been described above, but the same correction processing can be performed by deconvolution using a point spread function pattern. Since the processing using the point spread function pattern is simpler than the Fourier transform, the apparatus cost can be reduced when a dedicated processing circuit is manufactured. Further, optical transfer function data for correction calculation is calculated for each of a plurality of different expected angles and stored in the memory 202, and the corresponding optical transfer function data is used for each image region corresponding to the expected angles. May be corrected. It should be noted that in-plane errors such as curvature and distortion can be corrected by estimating the error level in advance.

  According to the modification of this embodiment, correction processing means for correcting image degradation (MTF correction) by the lens 3 a is provided in the previous stage of the connection processing unit 101, and the connection processing is performed by the image sensor 5. Correction processing is performed on each single-eye image of each previous compound-eye image.

  According to another modification of the present embodiment, the correction processing means for correcting image degradation (MTF correction) by the lens 3a is arranged at the subsequent stage of the post-processing unit 104, and for a reconstructed single image. Image deterioration correction processing (MTF correction) is performed. Since the correction process for a single image only needs to be performed once, there is an advantage that the calculation time can be generally shortened as compared with the case where the correction process is performed in units of individual images. However, since the optical transfer function data used for the correction process is related to each lens 3a and should be applied to the correction of individual eye images in a compound eye image, in the correction process for a single image, Compared with the case where correction processing is performed for each eye image, a certain increase in correction error is unavoidable.

  As described in the first embodiment, it is not always necessary to move the light source 6. Further, the lens array 3, the light shielding member 4, and the optical bandpass filter 7 of the imaging optical system 100 are configured and fixed to a size that covers the entire subject, and only the imaging device 5 is moved in the direction of the arrow 15. However, the image may be taken.

[Third Embodiment]
The optical transfer function of the lens also changes depending on the subject distance. In particular, in the case of an imaging optical system in which the subject is close to the lens array 3 as in the image input device of the present invention, the change in the optical transfer function accompanying the change in subject distance increases. Cheap.

  FIG. 10 illustrates the change in the MTF characteristics due to the difference in the subject distance in the plano-convex lens with the same convex surface as in the case of FIG. 9B facing the image surface side. In FIG. 10, the MTF characteristics when the subject distance is A, B, and C at a predetermined subject prospect angle are indicated by a thin solid line, a dotted line, and a one-dot chain line. As can be seen from this example, when the variation in the subject distance cannot be ignored, the correction error becomes large if the image deterioration correction (MTF correction) is performed based on the optical transfer function data at the specific subject distance. . In order to reduce the correction error, it is preferable to prepare optical transfer function data at different subject distances in advance and select the optical transfer function data used for image degradation correction according to the subject distance. According to such image degradation correction according to the subject distance, it is possible to correct to an appropriate MTF characteristic as shown by a thick solid line in FIG.

  FIG. 11 is an explanatory diagram of an image input device and a personal authentication device according to the third embodiment of the present invention. The image input apparatus according to the present embodiment divides and captures and inputs an image of the vein 2 of the finger 1 in a stationary state, but in order to perform image deterioration correction according to the subject distance as described above, A subject distance detection unit (distance detection means) 301 for detecting a subject distance (distance from the subject 2 to the lens array 3) is added. The memory 202 stores in advance optical transfer function data relating to the lens 3a at a plurality of different subject distances. In the correction calculation unit 201, optical transfer function data corresponding to the subject distance closest to the subject distance detected by the subject distance detection unit 301 is read from the memory 202, and based on this optical transfer function data, the stitching processing unit 101 reads the optical transfer function data. Image degradation correction (MTF correction) is performed for each single-eye image on the generated compound eye image. The other configuration is the same as that of the second embodiment.

  As described in relation to the first embodiment, the overlapping area between single-eye images changes according to the subject distance (see FIG. 4). Therefore, the subject distance can be calculated from the principle of triangulation using the overlapping region, that is, the detected parallax. The subject distance detection unit 301 according to the present embodiment employs such a method, detects a parallax between single-eye images in a compound eye image picked up by the image sensor 5, and uses this parallax to perform triangulation The subject distance is calculated according to the principle. In order to detect the subject distance, it is only necessary to detect the parallax between the two single-eye images, and it is not necessary to detect the parallax between all the single-eye images. The method for detecting parallax between single-eye images may be the method already described.

  According to the modification of this embodiment, correction processing means for correcting image degradation (MTF correction) by the lens 3 a is provided in the previous stage of the connection processing unit 101, and the connection processing is performed by the image sensor 5. Correction processing is performed on the single-eye image of each previous compound-eye image.

  According to another modification of the present embodiment, the correction processing means for correcting image degradation (MTF correction) by the lens 3 a is disposed after the post-processing unit 104 and is detected by the subject distance detection unit 301. Image degradation correction processing (MTF correction) is performed on the reconstructed single image using optical transfer function data corresponding to the subject distance.

  As described in the first embodiment, it is not always necessary to move the light source 6. Further, the lens array 3, the light shielding member 4, and the optical bandpass filter 7 of the imaging optical system 100 are configured and fixed to a size that covers the entire subject, and only the imaging device 5 is moved in the direction of the arrow 15. However, the image may be taken.

[Fourth Embodiment]
FIG. 12 is an explanatory diagram of an image input device and a personal authentication device according to the fourth embodiment of the present invention. Contrary to the case of the first to third embodiments, the image input apparatus according to the present embodiment moves the finger 1 from the tip (nail side end) to the arrow 16 while the imaging optical system 100 is stationary. The image of the vein pattern of the entire finger is input by continuously capturing the image of the vein 2 inside the finger at predetermined time intervals while moving it to slide in the direction. The imaging optical system 100 has the same configuration as that of the first embodiment except that the imaging optical system 100 does not move. However, the imaging element drive unit 22 (FIG. 2) is configured so that the imaging element 5 captures images at predetermined time intervals. Driven by.

  Each compound eye image captured at a predetermined time interval is pre-processed by the pre-processing unit 102 and then input to the reconstruction calculation unit 103. The reconstruction calculation unit 103 reconstructs a single image from each input compound eye image by using the parallax between individual images as described in detail in relation to the first embodiment. Each reconstructed single image (reconstructed image) is subjected to post-processing such as removal of reconstruction noise by the post-processing unit 104, and then input to the reconstructed inter-image parallax detection unit 401. The reconstructed image parallax detector 401 detects the parallax between the reconstructed images. The stitching processing unit 401 uses the parallax between the reconstructed images detected by the reconstructed image parallax detecting unit 401 to perform a process of stitching together a plurality of reconstructed images to generate one single image. .

  FIG. 13 is an explanatory diagram of an example of a method for detecting a parallax between reconstructed images and a process for joining reconstructed images using parallax. FIG. 13 (a) shows a reconstructed image from one compound eye image, and FIG. 13 (b) shows a reconstructed image from a compound eye image captured thereafter. 1 ′ is an image of the finger 1, and 2 ′ is an image of the vein 2 that is a subject. FIG. 13C shows the intensity distribution at the cross section A in the reconstructed image of FIG.

  The parallax detection unit 401 between reconstructed images obtains the same intensity distribution while sequentially moving the cross section of the reconstructed image shown in FIG. 13B, and sequentially compares it with the intensity distribution at the cross section A, thereby calculating the intensity distribution. Is obtained. The cross section B and the cross section A correspond to the same position of the subject. The reconstructed inter-image parallax detection unit 401 detects the distance between the cross section A and the cross section B as the parallax between the two reconstructed images. The stitching processing unit 402 shifts one reconstructed image by the amount of the parallax and stitches it with the other reconstructed image. That is, the reconstructed images in FIGS. 13A and 13B are joined so that the cross sections A and B coincide. By shortening the imaging time interval compared to the movement speed of the finger 1 and performing a detailed joining process, it is possible to join the reconstructed images accurately even if the movement speed of the finger 1 is not constant.

  It should be noted that a parallax detection algorithm that uses the sum of squares of luminance deviations between images shown in the equation (1) may be applied to parallax detection between reconstructed images. In this case, only a partial area of the reconstructed image may be used for parallax detection, and the processing load is reduced compared to the case of using the entire area. However, as the area used for parallax detection increases, the processing load increases, but the accuracy of splicing increases.

  The form of capturing a vein image while moving the finger 1 as in the present embodiment can reduce the size of the imaging optical system 100 including the imaging element 5 as compared with the region to be imaged of the finger. Further, since a mechanism for moving the imaging optical system 100 is unnecessary, it is advantageous in terms of reduction in apparatus cost, downsizing of the apparatus, improvement in reliability, and the like as compared with a configuration in which the imaging optical system is moved.

  According to the modification of the present embodiment, correction processing means for correcting image degradation (MTF correction) by the lens 3 a is provided in the previous stage of the preprocessing unit 102, and each compound eye image captured by the image sensor 5 is provided. Correction processing is performed on the single-eye image.

  According to another modification of the present embodiment, the correction processing means for correcting image degradation (MTF correction) by the lens 3 a is provided after the post-processing unit 104 or after the splicing processing unit 402. .

  According to another modification of the present embodiment, a subject distance detection unit similar to the subject distance detection unit 301 illustrated in FIG. 11 is provided, and the preceding stage of the preprocessing unit 102, the subsequent stage of the postprocessing unit 104, or A correction processing unit that performs an image degradation correction process (MTF correction) using optical transfer function data corresponding to the subject distance detected by the subject distance detection unit is provided after the joining processing unit 402.

[Fifth Embodiment]
FIG. 14 is an explanatory diagram of an image input device and a personal authentication device according to the fifth embodiment of the present invention. The image input apparatus according to the present embodiment slides the finger 1 from the tip (nail side end) in the direction of the arrow 16 with the imaging optical system 100 stationary as in the case of the fourth embodiment. The image of the vein pattern of the entire finger is input by continuously capturing images of the vein 2 inside the finger at predetermined time intervals.

  The difference from the fourth embodiment is that a parallax between compound eye images is detected, and a single image is reconstructed after connecting the compound eye images using the detected parallax.

  That is, a plurality of compound eye images continuously captured by the imaging element 5 at predetermined time intervals are sequentially input to the compound eye image parallax detection unit 501 to detect the parallax between compound eye images. This parallax detection is performed by a method that pays attention to the cross-sectional intensity distribution as described with reference to FIG. 13, or a method that uses the sum of squares of the luminance deviation between images shown in the equation (1). Done. For parallax detection, it is preferable to use a compound eye image from which a shadow portion by the light shielding member 4 is removed. Note that only a partial area of the compound eye image may be used for parallax detection.

  Each captured compound eye image is input to the stitching processing unit 502 via the compound eye image detection unit 501. The stitching processing unit 502 uses the parallax detected by the compound-eye image parallax detection unit 501 to stitch the input compound-eye image into one compound-eye image, thereby generating one compound-eye image. .

  The compound eye image generated by the stitching process is preprocessed by the preprocessing unit 102 and then reconstructed into a single image by the reconstruction calculation unit 103. This reconstructed image is post-processed by the post-processing unit 104 and then input to the authentication calculation unit 105 as an image of the subject.

  According to the modification of the present embodiment, correction processing means for correcting image degradation (MTF correction) by the lens 3a is provided before or after the compound-eye image parallax detection unit 501, or before the stitching processing unit 502. Is provided.

  According to another modification of the present embodiment, correction processing means for correcting image degradation (MTF correction) by the lens 3 a is provided at the subsequent stage of the post-processing unit 104.

  According to another modification of the present embodiment, subject distance detection means similar to the subject distance detection unit 301 shown in FIG. 11 is provided, and the preceding or subsequent stage of the preprocessing unit 102 and the subsequent stage of the joining processing unit 502 are provided. Alternatively, correction processing means for performing image degradation correction processing (MTF correction) using optical transfer function data corresponding to the subject distance detected by the subject distance detection means is provided at the subsequent stage of the post-processing unit 104.

[Sixth Embodiment]
FIG. 15 is an explanatory diagram of an image input device and a personal authentication device according to the sixth embodiment of the present invention. As in the case of the first embodiment, the image input apparatus according to the present embodiment performs imaging while moving the imaging optical system in the direction of the arrow 15 with the finger still.

  In FIG. 15, reference numeral 600 denotes an imaging optical system, and the basic configuration is the same as that of the imaging optical system 100 in each of the above embodiments. However, a lens array 3 in which lenses 3 a are arranged in a one-dimensional array is used. As the element 5, a line imaging element in which the light receiving elements 5a are arranged in one row is used. Although not shown, the light shielding member 4 in which the openings corresponding to the lenses 3a are arranged in a line is also included. Although not shown, there is also a light source 6 for irradiating the finger with near-infrared light.

  Such an imaging optical system 600 is integrally moved by a driving mechanism as shown in FIG. 2, for example. Each time the imaging optical system 600 moves by a predetermined amount of movement, a line compound eye image in which a compound eye image of the subject image formed by the lens array 3 is captured by the image sensor 5 (a line compounded only in the main scanning direction) Image) are sequentially input to the stitching processing unit 601. The stitching processing unit 601 creates a two-dimensional compound eye image by stitching the input line compound eye images in order in the sub-scanning direction. However, this two-dimensional compound eye image is an image that is compounded only in the main scanning direction as shown in FIG.

  The predetermined movement amount of the image pickup optical system is selected to be one pixel size or more in the main scanning direction of the image pickup device 5. If the predetermined amount of movement is made substantially equal to the size of one pixel, a two-dimensional compound eye image without missing lines can be obtained.

  The two-dimensional compound eye image is preprocessed by the preprocessing unit 102 and then input to the reconstruction calculation unit 103. The reconstruction calculation unit 103 detects the parallax in the main scanning direction between the single-eye images in the input two-dimensional compound eye image, and performs the pixel arrangement operation similar to that described in FIG. 6 using the parallax. Reconstruct a single image. The reconstructed image is post-processed by the post-processing unit 104 and then input to the authentication calculation unit 105.

  According to the modification of this embodiment, the lens array 3 is replaced with a lens array having a size sufficient to cover the entire subject area to be imaged, in which lenses are arranged in a two-dimensional array. The optical bandpass filter is also enlarged. These elements are fixed, and the image pickup device 5 picks up an image every time the image pickup device 5 moves by a predetermined amount while moving only the image pickup device 5 in the direction of the arrow 15. The line compound eye images thus captured are sequentially stitched in the sub-scanning direction by the stitching processing unit 601 to generate a two-dimensional compound eye image. However, this two-dimensional compound eye image is an image compounded in the main scanning direction and the sub-scanning direction as shown in FIG. Therefore, the reconstruction calculation unit 103 reconstructs a single image using parallax between the single-eye images in the main and sub-scanning directions.

  According to another modification of the present embodiment, correction processing means for correcting image degradation (MTF correction) by the lens 3a is provided at the front stage, the rear stage, or the rear stage of the post-processing unit 104 of the splicing processing unit 601. It is done.

  According to another modification of the present embodiment, subject distance detection means similar to the subject distance detection unit 301 shown in FIG. 11 is provided, and the upstream or downstream stage of the stitching processing unit 601 or the post-processing unit 104. In the subsequent stage, correction processing means for performing image deterioration correction processing (MTF correction) using optical transfer function data corresponding to the subject distance detected by the subject distance detection means is provided.

[Seventh Embodiment]
FIG. 17 is an explanatory diagram of an image input device and a personal authentication device according to the seventh embodiment of the present invention. As in the case of the fourth embodiment, the image input apparatus according to the present embodiment fixes the imaging optical system and moves the finger so that it slides in the direction of the arrow 16 from the tip side. Take images at time intervals.

  In FIG. 17, reference numeral 700 denotes an imaging optical system, and the basic configuration is the same as that of the imaging optical system 600 in the fifth embodiment, but two-line imaging in which the light receiving elements 5 a are arranged in two rows as the imaging element 5. An element is used. Although not shown, there is also a light source 6 for irradiating the finger with near infrared light.

  While sliding the finger, the image pickup device 5 of the image pickup optical system 700 continuously picks up a compound eye image of the subject image formed by the lens array 3 at a predetermined time interval and takes two line compound eye images (in the main scanning direction). (Line images that are compounded only in the image) are sequentially input to the stitching processing unit 701.

  The splicing processing unit 701 separates each input two-line compound eye image one line at a time and sequentially stitches them in the sub-scanning direction to generate two two-dimensional compound eye images. However, each two-dimensional compound eye image is an image that is compounded only in the main scanning direction as shown in FIG. The speed fluctuation compensation processing unit 702 uses the parallax in the sub-scanning direction (the direction of the arrow 16) between the two two-dimensional compound images generated by the stitching processing unit 701 to compensate for fluctuations in the movement speed of the finger. A two-dimensional compound eye image is generated.

  That is, as shown in FIG. 18, in the stitching processing unit 701, a two-line compound eye image captured by the image sensor 5 is converted into a first line compound eye image and a second line compound eye corresponding to each column of the light receiving elements 5a. The first and second connected images (two-dimensional compound eye images) 610A and 610B are generated by separating them into images and connecting them in the sub-scanning direction. The speed fluctuation compensation processing unit 702 uses the parallax in the sub-scanning direction between the two images from the first and second stitched images 610A and 610B, and the final stitched image in which the fluctuation of the finger moving speed is compensated. A (two-dimensional compound eye image) 611 is generated.

  Specific processing contents of the speed fluctuation compensation processing unit 702 will be described. FIG. 19 is an explanatory diagram thereof.

  In order to compensate for fluctuations in the movement speed of the finger (correction of line interval errors in the line image due to speed variations), the connection distance interval (unit: pixels) of the line compound eye image in the first connection image 610A is adjusted. At this time, by obtaining a position where the cross-sectional intensity distribution of the line compound eye image at an arbitrary position in the first joint image 610A is closest to the cross-sectional intensity distribution in the second joint image 610B, the finger movement direction of both images is obtained. , That is, the parallax in the sub-scanning direction is detected in units of pixels. The parallax is Δ (unit is pixel), the pixel size is β (unit is μm / pixel), and the distance in the finger movement direction between the columns of the light receiving elements 5a in the imaging element 5, that is, the line interval is L (unit is μm, for example). Then, the line compound eye image in the first connection image 610A is connected again at an interval of {Δ−L / β} (unit is pixel).

  This reconnection is processing for arranging line compound eye images on the virtual image space 612 at intervals of {Δ−L / β}. When the moving speed of the finger is slower than the imaging time interval, the value of the interval {Δ−L / β} becomes small and the arrangement interval of the line compound eye images becomes dense. When the interval is smaller than the size of one pixel, an average value of a plurality of line compound eye images to be arranged at the pixel position is arranged. Conversely, when the finger moving speed is faster than the imaging time interval, the value of the interval {Δ−L / β} increases, and the arrangement interval of the line compound eye image becomes sparse, which causes missing pixels in the arrangement space. When this occurs, the missing pixel luminance is estimated and arranged by interpolation processing such as linear interpolation or cubic interpolation using the previous and subsequent pixel luminance data. By such processing, even if the relationship between the finger moving speed and the imaging time interval is unknown, the connecting interval error (pixel unit) of the line compound eye image is compensated. When the finger moving speed fluctuates, the parallax Δ changes in accordance with the fluctuation, so that the fluctuation of the finger moving speed is compensated for by the pixel arrangement process.

  By performing the process of arranging all the line compound eye images in the first tethered image 610A in the virtual image space 612 as described above, the final movement in which fluctuations in the movement speed of the finger are compensated in the virtual image space 612 is performed. A connected image 611 is generated.

  Note that the stitching processing unit 701 inputs the line compound eye images to be stitched together as the first stitching image 610A to the speed fluctuation compensation processing unit 702 as they are without stitching, and creates only the second stitching image 610B. You may make it do. The speed fluctuation compensation processing unit 702 can treat each line compound eye image input without being connected as each line compound eye image in the first connection image 610A, and perform the same processing. Such a mode is naturally included in the present embodiment.

  The final stitched image generated by the speed fluctuation compensation processing unit 702, that is, the two-dimensional compound eye image in which the fluctuation of the finger movement speed is compensated, is an image that is compounded only in the main scanning direction as shown in FIG. is there. This two-dimensional compound eye image is pre-processed by the pre-processing unit 102 and then reconstructed into a single image by the reconstruction processing unit 103 using the parallax between the single-eye images in the main scanning direction. This reconstructed image is post-processed by the post-processing unit 104 and then input to the authentication calculation unit 105.

  A standard color line image sensor can be used as the two-line image sensor 5 used in the present embodiment. For example, as shown in FIG. 20, there is a color line imaging device in which filters of R, G, and B colors are arranged. Since this color line image sensor has two light receiving element lines, it can be diverted as the two-line image sensor 5 in the present embodiment simply by removing the color filter.

  According to the modification of the present embodiment, as shown in FIG. 21, the imaging optical system 600 is divided into two imaging optical systems 600A and 600B. Each of the imaging optical systems 600A and 600B is different from the imaging optical system 600 in that a line imaging element in which only one row of light receiving elements 5a is arranged as the imaging element 5 is used. When the imaging optical system 600 is divided into the imaging optical systems 600A and 600B in this way, the line interval L can be increased, and therefore, between the images obtained by joining the line compound eye images captured by the imaging optical systems 600A and 600B. Large parallax can be secured. In addition, a standard line imaging device can be used as the imaging device 5 of the imaging optical systems 600A and 600B.

[Eighth Embodiment]
FIG. 22 is an explanatory diagram of an image input apparatus according to the eighth embodiment of the present invention. In this embodiment, in order to obtain compound eye image data from which a bias component by external light (bias light) other than near-infrared light emitted from the light source 6 is removed, a bias component removal processing unit (bias component removal processing means) 751 is provided. In addition, a control unit (control unit) 752 for controlling the driving of the light source 6 and the like is provided. Other configurations are the same as those in the first to seventh embodiments or modifications thereof, although not shown. Note that the optical bandpass filter of the imaging optical system may be omitted. When the optical bandpass filter is omitted, a transparent flat plate member for adjusting the subject distance or protecting the lens array 3 can be provided at the position of the optical bandpass filter.

(First embodiment)
According to the first example of the present embodiment, the control unit 752 causes the light source 6 to emit light intermittently by controlling on and off the drive current of the light source 6. That is, the light source 6 is caused to emit or extinguish intermittently. Then, the compound eye images at the time of light emission and extinction of the light source 6 are respectively picked up by the image sensor 5 and the respective compound eye images are taken into the bias component removal processing unit 751. The timing signal is supplied to the image sensor driving unit 22 and the bias component removal processing unit 751. The bias component removal processing unit 751 takes the difference between the compound eye image captured when the light source 6 emits light and the compound eye image captured when the light source 6 extinguishes, thereby removing the bias component due to external light. The compound eye image which consists only of the component by 6 illumination light is produced | generated. This compound eye image is treated as a compound eye image captured by the image sensor 5.

(Second embodiment)
According to the second example of the present embodiment, the control unit 752 modulates the driving current of the light source 6 in a sine wave shape, thereby changing the intensity of near infrared light emitted from the light source 6 in a sine wave shape. Since external light (bias light) is also superimposed on the near-infrared light, if the light is incident on the image sensor 5 as it is, light intensity modulation as shown in FIG. 23 is continuously obtained for each pixel. . The intensity of the pixel at an arbitrary position (x, y) in the image is expressed by the following equation (5).

  In Expression (5), I is the intensity of the pixel, A is the intensity of the external light, that is, the bias light, B is the modulation amplitude of the illumination light by the light source 6, and φ is the modulation phase of the illumination light.

  For example, when imaging is performed at time points t1, t2, t3, and t4 shown in FIG. 23 at time intervals that divide the modulation period into four equal parts, the image intensity acquired at each time point is expressed as follows.

  The modulation amplitude of the illumination light can be obtained by the following equation (10) using the equations (6) to (9). Therefore, an image obtained by removing the influence of the bias light (bias component) can be generated by performing the calculation of the expression (10) in the bias component removal processing unit 751 for the image captured at each time point. This image is treated as an image captured by the image sensor 5.

  In the above description, an image is captured at a sampling interval that divides the modulation period into four. However, the number of samplings may be increased, or discrete Fourier transform may be used for the calculation for extracting the modulation amplitude. Increasing the number of samplings makes it possible to generate an image from which the bias component has been removed more strictly.

[Ninth Embodiment]
The personal authentication apparatus according to the present invention can be easily incorporated into various electronic devices because the entire imaging optical system can have a thin structure. Then, the operation of the electronic device can be controlled according to the result of the authentication processing by the personal authentication device, and for example, the user of the electronic device can be restricted.

  FIG. 24 shows an example of a small information terminal (for example, PDA) and a notebook personal computer incorporating a personal authentication device according to the present invention. In FIG. 24, reference numeral 800 denotes the personal authentication device of the present invention, but only the part where the finger is placed is exposed on the surface. A person who intends to use this small information terminal or notebook personal computer places his / her finger on or slides the exposed portion of the personal authentication device 800 to read the vein pattern for authentication. On the device side, for example, if the person is authenticated as the same person as the person registered in advance, the login is permitted, and if the person is not authenticated, the login is denied.

It is explanatory drawing of the 1st Embodiment of this invention. It is a schematic diagram which shows an example of the moving mechanism of an imaging optical system. It is a figure which shows the example of a simulation of the compound eye image imaged. It is a figure for demonstrating the change of the duplication area | region between adjacent single-eye images by the difference in to-be-photographed object distance. This represents the change in E accompanying the change in parallax when the parallax in the x and y directions between the single images is taken on the x and y axes and the square sum E of the pixel luminance difference between the single images is taken on the z axis. FIG. It is a schematic diagram for demonstrating the pixel arrangement | positioning method in the process which reconstructs a single image from a compound eye image. It is a flowchart which shows an example of the processing flow for reconstructing a single image from a compound eye image. It is explanatory drawing of the 2nd Embodiment of this invention. FIG. 6 is an MTF characteristic diagram showing a relationship between a subject prospective angle and an MTF when a convex surface of a plano-convex lens is directed toward the subject side and when directed toward the image plane side. FIG. 6 is an MTF characteristic diagram showing the relationship between subject distance and MTF when the convex surface of a plano-convex lens is directed to the image plane side. It is explanatory drawing of the 3rd Embodiment of this invention. It is explanatory drawing of the 4th Embodiment of this invention. It is explanatory drawing of the parallax detection method which paid its attention to cross-sectional intensity distribution. It is explanatory drawing of the 5th Embodiment of this invention. It is explanatory drawing of the 6th Embodiment of this invention. It is a figure which shows the two-dimensional image compounded only about the main scanning direction. It is explanatory drawing of the 7th Embodiment of this invention. It is explanatory drawing of a joining process and a speed fluctuation compensation process. It is process explanatory drawing of a speed fluctuation compensation process part. It is explanatory drawing of the line image sensor for color. It is explanatory drawing of the modification of the 7th Embodiment of this invention. It is explanatory drawing of the 8th Embodiment of this invention. It is timing explanatory drawing in the case of intensity-modulating illumination light to sine wave shape and sampling an image with the phase interval which divides the modulation period into 4 equally. It is a schematic diagram which shows the example of the electronic device with which the personal authentication apparatus which concerns on this invention was integrated.

Explanation of symbols

1 finger (living body)
2 Veins (subject)
DESCRIPTION OF SYMBOLS 3 Lens array 3a Lens on lens array 4 Light-shielding member 5 Image pick-up element 6 Light source 7 Optical band pass filter 8 Stage 9 Pulse motor 10 Ball screw 100 Imaging optical system 101 Joint processing part 102 Pre-processing part 103 Reconstruction calculating part 104 Post-processing unit 105 Authentication calculation unit 106 Registration data memory 201 Correction calculation unit 202 Optical transfer function data memory 301 Subject distance detection unit 401 Reconstructed image parallax detection unit 402 Splicing processing unit 501 Compound eye inter-parallax detection unit 502 Splicing processing Unit 601 splicing processing unit 700 imaging optical system 701 splicing processing unit 702 speed fluctuation compensation processing unit 600a, 600B imaging optical system 751 bias component removal processing unit 752 control unit 800 personal authentication device incorporated in electronic device

Claims (16)

In an image input device for inputting an image of a subject existing inside a stationary living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Moving means for moving at least the image sensor of the imaging optical system in a plane perpendicular to the optical axis of the lens;
Stitching processing means for creating a single compound eye image by performing a process of stitching a plurality of compound eye images picked up by the image sensor when the image sensor moves to a plurality of different positions in the plane;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for each single-eye image for the compound eye image generated by the splicing processing means;
Reconstructing processing means for generating a single reconstructed image by performing processing for reconstructing a single image from the compound eye image corrected by the correcting means using parallax between the single-eye images;
An image input device for inputting a reconstructed image generated by the reconstructing processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. An image input apparatus characterized by that. In an image input device for inputting an image of a subject existing inside a stationary living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Moving means for moving at least the image sensor of the imaging optical system in a plane perpendicular to the optical axis of the lens;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for each single-eye image for the compound eye image captured by the imaging device;
When the image sensor is moved to a plurality of different positions in the plane, the image is picked up by the image sensor, and a plurality of compound images corrected by the correction process are connected to generate one compound eye image. , Stitching processing means,
Reconstructing processing means for generating a single reconstructed image by performing processing for reconstructing a single image from the compound eye images generated by the stitching processing means using the parallax between the single-eye images. And
An image input device for inputting a reconstructed image generated by the reconstructing processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. An image input apparatus characterized by that. In an image input device for inputting an image of a subject existing inside a stationary living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Moving means for moving at least the image sensor of the imaging optical system in a plane perpendicular to the optical axis of the lens;
Stitching processing means for creating a single compound eye image by performing a process of stitching a plurality of compound eye images picked up by the image sensor when the image sensor moves to a plurality of different positions in the plane;
Reconstructing processing means for generating a single reconstructed image by performing processing for reconstructing a single image from a compound eye image generated by the splicing processing means using parallax between the single-eye images;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for the reconstructed image generated by the reconstruction processing means,
An image input device for inputting the reconstructed image corrected by the correction processing means as the image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. An image input apparatus characterized by that. In an image input device for inputting an image of a subject existing inside a stationary living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Moving means for moving at least the image sensor of the imaging optical system in a plane perpendicular to the optical axis of the lens;
Processing for reconstructing a single image from the plurality of compound eye images captured by the image sensor when the image sensor moves to a plurality of different positions in the plane, using the parallax between the single-eye images Reconstructing processing means for generating a plurality of reconstructed images by performing
A stitching processing means for performing a process of stitching a plurality of reconstructed images generated by the reconstruction processing means to generate a single image;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for a single image generated by the stitching processing means,
An image input device for inputting a single image corrected by the correction processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. An image input apparatus characterized by that. In an image input device for inputting an image of a subject existing inside a stationary living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Moving means for moving at least the image sensor of the imaging optical system in a plane perpendicular to the optical axis of the lens;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for each single-eye image for the compound eye image captured by the imaging device;
When the image sensor moves to a plurality of different positions in the plane, the parallax between the single-eye images is used from each of a plurality of compound-eye images captured by the image sensor and corrected by the correction processing unit. Reconstructing processing means for reconstructing a single image and generating a plurality of reconstructed images;
Stitching processing means for performing a process of stitching together the plurality of reconstructed images generated by the reconstruction processing means to generate a single image;
An image input device for inputting a single image generated by the stitching processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. An image input apparatus characterized by that. In an image input device for inputting an image of a subject existing inside a stationary living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Moving means for moving at least the image sensor of the imaging optical system in a plane perpendicular to the optical axis of the lens;
Processing for reconstructing a single image from the plurality of compound eye images captured by the image sensor when the image sensor moves to a plurality of different positions in the plane, using the parallax between the single-eye images Reconstructing processing means for generating a plurality of reconstructed images by performing
Correction processing means for performing image deterioration correction processing by the lens based on optical transfer function data relating to the lens prepared in advance for a plurality of reconstructed images generated by the reconstruction processing means;
Stitching processing means for generating a single image by stitching together the plurality of reconstructed images corrected by the correction processing means;
An image input device for inputting a single image generated by the stitching processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. An image input apparatus characterized by that. In an image input device for inputting an image of a subject existing inside a moving living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
A plurality of reconstructed images are obtained by performing a process of reconstructing a single image using parallax between the single-eye images from each of a plurality of compound-eye images continuously captured by the image sensor at predetermined time intervals. Reconfiguration processing means to generate,
Reconstructed image parallax detection means for detecting parallax between a plurality of reconstructed images generated by the reconstruction processing means;
A stitching processing means for generating a single image by performing a process of stitching together a plurality of reconstructed images generated by the reconstruction processing means using the parallax detected by the parallax detection means between the reconstructed images When,
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for a single image generated by the stitching processing means,
An image input device for inputting a single image corrected by the correction processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. An image input apparatus characterized by that. In an image input device for inputting an image of a subject existing inside a moving living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for each single-eye image for the compound eye image captured by the imaging device;
A process of reconstructing a single image using parallax between the single-eye images from each of a plurality of compound-eye images continuously captured at a predetermined time interval and corrected by the correction unit. Reconstructing processing means for generating a plurality of reconstructed images, and
Reconstructed image parallax detection means for detecting parallax between a plurality of reconstructed images generated by the reconstruction processing means;
A stitching processing means for generating a single image by performing a process of stitching together a plurality of reconstructed images generated by the reconstruction processing means using the parallax detected by the parallax detection means between the reconstructed images And
An image input device for inputting a single image generated by the stitching processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. An image input apparatus characterized by that. In an image input device for inputting an image of a subject existing inside a moving living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
A plurality of reconstructed images are obtained by performing a process of reconstructing a single image using parallax between the single-eye images from each of a plurality of compound-eye images continuously captured by the image sensor at predetermined time intervals. Reconfiguration processing means to generate,
Correction processing means for performing image deterioration correction processing by the lens based on optical transfer function data relating to the lens prepared in advance for a plurality of reconstructed images generated by the reconstruction processing means;
Reconstructed image parallax detecting means for detecting parallax between a plurality of reconstructed images corrected by the correction processing means;
A splicing processing unit that generates a single image by performing a process of splicing a plurality of reconstructed images corrected by the correction processing unit using the parallax detected by the reconstructed inter-parallax detection unit; Have
An image input device for inputting a single image generated by the stitching processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. An image input apparatus characterized by that. In an image input device for inputting an image of a subject existing inside a moving living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
A compound eye inter-image parallax detecting means for detecting a parallax between a plurality of compound eye images continuously captured by the imaging device at a predetermined time interval;
A stitching processing means for generating a single compound eye image by performing a process of stitching a plurality of compound eye images captured by the image sensor using the parallax detected by the compound eye inter-image parallax detection means;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for each single-eye image for the compound eye image generated by the splicing processing means;
Reconstructing processing means for generating a single reconstructed image by performing processing for reconstructing a single image from the compound eye image corrected by the correcting processing means using the parallax between the single-eye images. ,
An image input device for inputting a reconstructed image generated by the reconstructing processing means as an image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. An image input apparatus characterized by that. In an image input device for inputting an image of a subject existing inside a moving living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for each single-eye image for the compound eye image captured by the imaging device;
A compound-eye image parallax detection unit that detects parallax between a plurality of compound-eye images that are continuously captured at predetermined time intervals by the imaging device and corrected by the correction unit;
A stitching processing means for generating a single compound eye image by performing a process of stitching a plurality of compound eye images captured by the image sensor using the parallax detected by the compound eye inter-image parallax detection means;
Reconstructing processing means for generating a single reconstructed image by performing processing for reconstructing a single image from the compound eye images generated by the stitching processing means using the parallax between the single-eye images. And
An image input device for inputting a reconstructed image generated by the reconstructing processing means as an image of the subject,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. An image input apparatus characterized by that. In an image input device for inputting an image of a subject existing inside a moving living body ,
A light source for irradiating the living body with near-infrared light,
A lens array for forming an image of the subject, wherein a plurality of lenses having a surface having zero or negative power on the living body side and a surface having positive power on the image surface side are arrayed ,as well as,
For capturing a compound eye image that is a set of images of the subject (hereinafter referred to as a single eye image) formed by each of the plurality of lenses provided on the image plane side of the lens array. An imaging optical system including an imaging element;
A compound eye inter-image parallax detecting means for detecting a parallax between a plurality of compound eye images continuously captured by the imaging device at a predetermined time interval;
A stitching processing means for generating a single compound eye image by performing a process of stitching a plurality of compound eye images captured by the image sensor using the parallax detected by the compound eye inter-image parallax detection means;
Reconstructing processing means for generating a single reconstructed image by performing processing for reconstructing a single image from a compound eye image generated by the splicing processing means using parallax between the single-eye images;
Correction processing means for performing correction processing of image degradation by the lens based on optical transfer function data relating to the lens prepared in advance for the reconstructed image generated by the reconstruction processing means,
An image input device for inputting the reconstructed image corrected by the correction processing means as the image of the subject ,
Subject distance calculating means for detecting a distance from the subject to the lens array (hereinafter, subject distance) based on parallax between single-eye images in a compound eye image captured by the imaging element;
The correction processing means selects optical transfer function data to be used for the correction process from a plurality of optical transfer function data relating to the lens prepared in advance, according to the subject distance detected by the subject distance detection means. An image input apparatus characterized by that. Means for controlling the light source to emit light intermittently;
Near-infrared light emitted by the light source by taking the difference between the compound eye image captured by the image sensor in the light emitting state of the light source and the compound eye image captured by the image sensor in the extinguished state of the light source. Bias component removal processing means for generating a compound eye image from which a bias component other than light is removed,
13. The image input apparatus according to claim 1, wherein the compound eye image after the bias component removal generated by the bias component removal processing unit is handled as a compound eye image picked up by the image pickup device. . Means for controlling the light source to change the intensity of near-infrared light applied to the living body in a sinusoidal shape;
Irradiated by the light source by a calculation process from a plurality of compound eye images respectively captured by the image sensor at a plurality of different phase time points within a sinusoidal change period of the intensity of near infrared light irradiated to the living body by the light source. Bias component removal processing means for generating a compound eye image in which a bias component due to light other than near infrared light is removed, and
13. The image input apparatus according to claim 1, wherein the compound eye image after the bias component removal generated by the bias component removal processing unit is handled as a compound eye image picked up by the image pickup device. . The image input device according to any one of claims 1 to 14,
A personal authentication apparatus comprising authentication processing means for performing personal authentication processing based on an image of a subject input by the image input apparatus. 16. An electronic device comprising the personal authentication device according to claim 15, wherein operation is controlled according to a result of personal authentication processing by the personal authentication device.

Images