JP7303022B2 - PORTABLE ELECTRONIC DEVICE AND IMAGE AUTHENTICATION METHOD - Google Patents

Description

One aspect of the present invention relates to a portable electronic device or the like capable of executing image authentication using a camera.

Conventionally, some electronic devices such as smartphones and tablet devices have a function of unlocking by a method such as face authentication. For example, Patent Document 1 discloses an authentication system that compares the front face data of a user using a mobile communication terminal with the front face data registered in an authentication server, and connects the line only when they match. A method is disclosed. Patent Literature 2 discloses a mobile phone terminal that performs face authentication, iris authentication, and voiceprint authentication to authenticate an authorized user of the camera-equipped mobile phone terminal. Japanese Patent Laid-Open No. 2002-200001 discloses an authentication device and the like that improve the accuracy of face authentication by using imaging using visible light and imaging using infrared light.

JP-A-2000-232680 JP-A-2006-93934 JP 2014-78052 A

Electronic devices having a function of unlocking by a method such as face recognition, as described above, are required to improve authentication accuracy and security. In order to improve the accuracy of authentication, there is a method of obtaining a fine image, for example, by providing a configuration with an autofocus mechanism. However, with this method, it may take time to take an image using autofocus, and the time required for authentication may become longer. There is a demand for a configuration that enables authentication in a short time while improving authentication accuracy.

The present invention employs the following means to solve the above problems. In the following description, reference numerals and the like in the drawings are added in parentheses to facilitate understanding of the invention, but each component of the present invention is not limited to these added descriptions. It should be interpreted broadly to the extent that a trader can technically understand.

One means of the present invention is
an imaging device (42);
a lens;
A diaphragm blade (433) capable of adjusting the amount of light irradiated to the imaging device;
A blade driving section (432) for driving the diaphragm blades;
a blade control section (431) that controls the blade drive section;
An authentication unit (10) that performs image authentication based on an image captured by the imaging device during unlocking processing,
The aperture blades are held in a minimum aperture state before the unlocking process,
It is a portable electronic device.

According to the portable electronic device having the above configuration, by setting the aperture blades to the minimum aperture state before the unlocking process, an image with a deep depth of field can be captured during the unlocking process. An image with a deep depth of field looks sharp as a whole and appears to be in focus. By capturing an authentication image with a deep depth of field in this way, feature points necessary for image authentication can be obtained appropriately, and authentication accuracy can be improved. In addition, since the minimum aperture state is maintained before unlocking, the time required for authentication can be shortened.

Preferably, in the portable electronic device,
The blade control section operates the blade control section so that the aperture blades are in an open state during imaging.

According to the portable electronic device configured as described above, the diaphragm is opened when an image is captured, so that a bright image can be captured.

Preferably, in the portable electronic device,
The aperture blades are in the minimum aperture state when the blade drive section is de-energized,
Before the unlocking process, the blade drive section is in a non-energized state.

According to the portable electronic device configured as described above, the aperture blades can be maintained in the minimum aperture state while the blade drive section is kept in a non-energized state before the unlocking process. Then, during the unlocking process, it is possible to take an image while keeping the aperture blades in the minimum aperture state. As a result, power consumption in the locked state can be reduced, authentication accuracy can be improved, and the time required for authentication can be shortened.

Preferably, in the portable electronic device,
It does not have an autofocus mechanism.

According to the portable electronic device having the above configuration, since it does not have an autofocus mechanism but has a diaphragm mechanism, the configuration can be made simpler than a configuration having an autofocus mechanism. Further, the depth of field can be deepened by narrowing down the diaphragm blades of the diaphragm mechanism, and a clear image can be captured even with a configuration that does not have autofocus. Furthermore, it becomes possible to adjust the amount of light taken in during imaging.

Preferably, in the portable electronic device,
The blade control section keeps the blade driving section in a non-energized state until an input operation is performed, and turns the blade driving section in a conductive state when the input operation is performed.

According to the portable electronic device configured as described above, the power saving state is maintained until an input operation is performed, and after the input operation is performed, the diaphragm blades are driven by the blade driving section to enable the aperture to be adjusted. can.

Preferably, in the portable electronic device,
Operates in a standby mode in which the blade control unit is energized and the blade drive unit is de-energized until the input operation is performed;
When the input operation is performed, it operates in an active mode in which the blade drive section is energized in addition to the blade control section.

Alternatively, another means of the present invention is
a camera module including an imaging device, a lens, aperture blades capable of adjusting the amount of light irradiated to the imaging device, a blade driving unit for driving the aperture blades, and a blade control unit for controlling the blade driving unit;
An image authentication method in a portable electronic device comprising an authentication unit that performs image authentication,
activating the camera module with the aperture blades in a minimum aperture state;
capturing an image with the imaging device while maintaining the aperture blades in the minimum aperture state;
and the authentication unit performing image authentication based on the image.
It is an image authentication method.

According to the image authentication method described above, an image with a deep depth of field can be photographed without operating the diaphragm blades maintained in the minimum aperture state, for example, during the unlocking process. An image with a deep depth of field looks sharp as a whole and appears to be in focus. By capturing an authentication image with a deep depth of field in this way, feature points necessary for image authentication can be obtained appropriately, and authentication accuracy can be improved. In addition, since the minimum aperture state is maintained before unlocking, the time required for authentication can be shortened.

FIG. 1 is a block diagram showing the configuration of the smartphone of the embodiment. FIG. 2 is a flowchart showing unlocking processing of the smartphone according to the first embodiment. FIG. 3 is a flow chart showing imaging processing of the smartphone according to the second embodiment. FIG. 4 is a flowchart showing imaging processing of the smartphone according to the third embodiment.

The portable electronic device of the present invention, for example, has a configuration in which a smartphone is provided with an aperture mechanism, the aperture mechanism is set to the minimum aperture state before face authentication for unlocking the locked state, and the aperture state is maintained in the minimum aperture state. One of the features is that it is configured to perform imaging for face recognition. This is intended to enable unlocking processing in a short period of time.

A portable electronic device according to the present invention will be specifically described according to the following configuration with reference to the drawings. However, the embodiment described below is merely an example of the present invention, and the technical scope of the present invention should not be construed in a limited manner. In addition, in each drawing, the same code|symbol is attached|subjected to the same component, and the description may be abbreviate|omitted.
1. Embodiment 1
2. Embodiment 2
3. Embodiment 3
4. Features of the present invention 5 . Supplementary matter

<1. Embodiment 1>
In this embodiment, a smart phone will be described as a specific example of a portable electronic device. However, the portable electronic device of the present invention is not limited to smartphones, and can be applied to tablet devices, notebook PCs, and other portable electronic devices. This embodiment is characterized by the configuration of the smartphone and the unlocking process.

<Smartphone configuration>
FIG. 1 is a diagram showing the configuration of a smartphone 1 of this embodiment. As shown in FIG. 1 , the smartphone 1 includes a processor 10 , a memory 11 , a display 20 , a display control section 21 , a touch panel 30 , a touch panel control section 31 and a camera module 40 . The camera module 40 includes a control section 41, an imaging device 42, an aperture mechanism 43, and a lens (not shown). The diaphragm mechanism 43 includes a blade control section 431 , a blade driving section 432 and diaphragm blades 433 .

<Processor 10>
The processor 10 controls the overall processing of the smartphone 1 including the memory 11 , the display controller 21 , the touch panel controller 31 , the controller 41 of the camera module 40 , and the blade controller 431 of the aperture mechanism 43 . The processor 10 is, for example, a CPU (Central Processing Unit) formed of a semiconductor integrated circuit. The processor 10 performs various types of processing in the smartphone 1, but in this specification, processing that is characteristic of the embodiment will be described, and description of other known processing will be omitted. A specific operation of the processor 10 will be described later.

<Memory 11>
The memory 11 is a rewritable RAM or the like, and is capable of storing various information through processing by the processor 10 and outputting the stored information. The memory 11 may be part of the same semiconductor integrated circuit as the processor 10, or may be configured separately from the processor 10. FIG.

The memory 11 stores, for example, data related to feature points of a registered user's face used for face authentication. The memory 11 also stores software related to various processes performed by the smartphone 1, image data, and the like.

<Display 20, Display Control Unit 21>
The display 20 is a display device arranged on one side of the plate-shaped smartphone, and displays an image in a manner that is visible to the user. The display 20 is a liquid crystal display device, an organic EL display device, or the like, but is not limited to these. The display control unit 21 controls images displayed on the display 20 .

<Touch Panel 30, Touch Panel Control Unit 31>
The touch panel 30 includes a film or the like arranged on the surface of the display 20, and is configured to be capable of detecting user's touch. The touch panel 30 is an example of an input detection mechanism. The touch panel 30 outputs the detected user's operation to the processor 10 via the touch panel control section 31 . The touch panel control section 31 controls the operation of the touch panel 30 .

<Camera module 40>
The camera module 40 includes a control section 41 , an imaging device 42 and an aperture mechanism 43 . The camera module 40 operates in power save mode or normal mode. The range in which power is supplied differs between the power saving mode and the normal mode, so that the power saving mode consumes less power. The power saving mode is also called sleep mode or standby mode. In sleep mode, the range in which power is supplied is smaller than in standby mode, and power consumption is even lower than in standby mode.

<Image sensor 42>
The imaging element 42 is, for example, a photoelectric conversion element such as a CMOS sensor or a CCD, detects light emitted through a lens, converts it into an electric signal, and outputs the electric signal to the control unit 41 . One or more lenses (not shown), an aperture blade 433, an ND filter (not shown), and the like are arranged between the imaging device 42 and the subject to be imaged. Further, a shutter blade may be arranged between the imaging device 42 and the subject.

The lens refracts light that has entered from the subject side and causes the light to travel toward the imaging element 42 side. The ND filter blocks part of the amount of light traveling toward the imaging element 42 side. The ND filter is movably arranged in the optical path from the subject to the imaging element 42, like the aperture blades 433 described later. The ND filter is inserted into or removed from the optical path under the control of the controller 41 .

<Control unit 41>
The control unit 41 controls the operation of the imaging device 42 . The control unit 41 converts the electrical signal input from the imaging device 42 into image data and outputs the image data to the processor 10 . Further, the control unit 41 controls processing such as imaging in the camera module 40 .

<Aperture mechanism 43>
The diaphragm mechanism 43 includes a blade control section 431 , a blade driving section 432 and diaphragm blades 433 . The diaphragm mechanism 43 operates in sleep mode, standby mode, and active mode. The power consumption of the diaphragm mechanism 43 is lowest in the sleep mode and highest in the active mode. In the sleep mode, power is supplied only to a portion of the blade control section 431, and power supply to other portions is stopped. That is, other portions of the blade control section 431 and the blade driving section 432 are in a non-energized state. In the standby mode, power is supplied to the blade control section 431, but the power supply to the blade drive section 432 is stopped and is in a non-energized state. In the active mode, power is supplied to the entire system including the blade control section 431 and the blade drive section 432 .

<Blade control unit 431>
The blade control section 431 controls the operation of the blade driving section 432 .

<Blade driving unit 432>
The blade drive unit 432 is a motor such as a VCM (Voice Coil Motor), and drives the diaphragm blades 433 . The blade drive section 432 may be a drive mechanism other than the VCM.

<Aperture blade 433>
Aperture blades 433 are arranged on the optical path from the subject to the imaging element 42 . The aperture blade 433 is configured including a plurality of blade members. The diaphragm blade 43 is configured to be able to adjust the opening amount of the opening formed by a plurality of blade members. The opening amount of the opening of the aperture blade 433 is adjusted by the blade control section 431 operating the blade driving section 432 . When the diaphragm blades 433 are narrowed down, that is, when the opening amount of the opening becomes small, the amount of light passing through decreases, and the amount of light irradiated to the imaging element 42 decreases. Conversely, when the diaphragm blades 433 are opened, that is, when the opening amount of the aperture increases, the amount of light passing through increases, and the amount of light irradiated to the imaging element 42 increases. The aperture blades 433 take at least two states, a minimum aperture state and a maximum aperture state, with the minimum aperture size being the minimum aperture size and the maximum aperture size being the maximum aperture size. The opening amount of the aperture of the aperture blade 433 may be limited to two states, a minimum aperture state and a maximum aperture state, or may be other states.

In this embodiment, the smartphone 1 changes between the locked state and the normal state. A state of the smartphone 1 is controlled by the processor 10 . When the smartphone 1 is in the locked state, part including at least the display 20 has stopped operating and is in a low power consumption state. At this time, the camera module 40 is in power saving mode (sleep mode or standby mode), and the aperture mechanism 43 is in sleep mode. When the user tries to use the smartphone 1, the processor 10 executes unlocking processing by face authentication in order to switch the smartphone 1 from the locked state to the normal state.

<Unlock processing>
FIG. 2 is a flowchart showing unlocking processing executed by the smartphone 1 of this embodiment.

<S100>
When the unlocking process starts, the processor 10 activates the camera module 40 to change from the power saving mode to the normal mode (active mode). At this time, the aperture mechanism 43 remains in the sleep mode, and the aperture blades 433 of the aperture mechanism 43 are maintained at the minimum aperture (S100). That is, in the aperture mechanism 43, power is supplied only to a part of the blade control section 431 before and after the unlocking process, and the other parts of the blade control section 431 and the blade driving section 432 remain in a non-energized state.

<S101>
Next, the control unit 41 operates the imaging device 42 to capture an image of the subject. Here, the user's face is imaged. At this time, since the aperture blade 433 is in the minimum aperture state, the depth of field is deep, and the whole image seems to be in focus, so that a clear image is captured. The control unit 41 outputs data of the captured image to the processor 10 .

<S110>
Next, the processor 10 performs face authentication by comparing the captured face image data with face authentication data stored in advance in the memory 11 . A known method is used for face authentication here.

<S111>
If the face authentication fails (N in S110), the processor 10 notifies the user that the authentication has failed (S111), and returns to the process of S101. Notification of authentication failure to the user is performed by displaying on the display 20 that authentication has failed. Notification of authentication failure may be made by voice, LED, or the like.

<S112, S113>
If the face authentication is successful (Y in S110), the lock state of the smartphone 1 is released and the normal state is set (S112). As a result, the smartphone 1 is ready for use by the user. After that, the processor 10 changes the camera module 40 from the normal mode (active mode) to the power saving mode (standby mode) (S113). This completes the unlocking process.

On the other hand, during normal imaging that is not unlock processing, the aperture blades 433 are opened to increase the amount of light taken in, and a bright image can be obtained.

<2. Embodiment 2>
Next, Embodiment 2 of the present invention will be described. The smartphone 1 of the present embodiment has substantially the same configuration as the smartphone 1 of Embodiment 1, but is partially different in operation. In particular, the smartphone 1 of the present embodiment is characterized by imaging processing when the user takes an image while the smartphone 1 is operating in a normal state. For example, the user activates application software for capturing an image with a camera on the smartphone 1 and captures an image using the camera module 40 .

<Image processing>
FIG. 3 is a flowchart showing imaging processing executed by the smartphone 1 of this embodiment.

<S200>
When the imaging process is started, the control unit 41 operates the imaging device 42 to acquire luminance information of the imaging target (S200). The control unit 41 outputs the acquired brightness information to the processor 10 .

<S210, S211>
Next, the processor 10 compares the brightness of the subject to be imaged with the brightness of the background based on the brightness information to check whether the subject is too dark with respect to the background (S210). If the subject is too dark for the background (Y in S210), the processor 10 operates the blade driving section 432 via the blade control section 431 to open the aperture blades 433 (S211). As a result, the amount of light captured by the imaging device 42 increases, and the subject becomes brighter and can be captured.

<S220, S221>
Next, the processor 10 compares the brightness of the subject to be imaged with the brightness of the background based on the brightness information to check whether the subject is too bright relative to the background (S220). If the subject is too bright relative to the background (Y in S220), the processor 10 operates the blade driving section 432 via the blade control section 431 to narrow down the aperture blades 433 (S221). As a result, the amount of light captured by the imaging element 42 is reduced, and the subject becomes darker and can be acquired.

<S230, S231>
Next, the processor 10 compares the brightness of the subject to be imaged with the brightness of the background based on the brightness information, and checks again whether the subject is too bright relative to the background (S230). If the subject is too bright relative to the background (Y in S230), the processor 10 inserts an ND filter into the optical path to reduce the amount of light passing toward the imaging device 42. As a result, the amount of light captured by the imaging element 42 is further reduced, and the subject becomes darker and can be captured.

<S240>
Through the above process, the control unit 41 adjusts the amount of light taken into the imaging element 42 so that the subject has a brightness suitable for imaging, and then the control unit 41 performs imaging of the subject with the imaging element 42 . The control unit 41 outputs the captured image to the processor 10 .

With the above processing, it is possible to capture an image of the subject after appropriately adjusting the brightness. In the above process, the process of opening and closing the aperture blades 433 and the process of inserting the ND filter are each performed once, but these processes may be repeated multiple times.

<3. Embodiment 3>
Next, Embodiment 3 of the present invention will be described. The smartphone 1 of the present embodiment has substantially the same configuration as the smartphone 1 of Embodiment 1, but is partially different in operation. In particular, the smartphone 1 of the present embodiment is characterized by imaging processing when the user takes an image while the smartphone 1 is operating in a normal state.

<Image processing>
FIG. 4 is a flowchart showing imaging processing executed by the smartphone 1 of this embodiment.

<S300>
When the imaging process is started, the processor 10 activates the camera module 40 and changes from the power saving mode (standby mode) to the normal mode (active mode). The processor 10 further changes the aperture mechanism 43 of the camera module 40 from sleep mode to standby mode (S300). As a result, the blade control section 431 of the diaphragm mechanism 43 is energized, but the blade driving section 432 is kept in a non-energized state.

<S301>
Next, the control unit 41 operates the imaging element 42 to acquire an image of the subject and outputs it to the processor 10 . The processor 10 causes the display 20 to display this image via the display control section 21 (S301).

<S310, S311, S312>
Here, when the user touches a portion whose brightness is to be adjusted in the image displayed on the display 20, processing is performed to adjust the brightness of the portion touched by the user to an appropriate brightness. Specifically, the touch panel control unit 31 detects whether or not the user has performed an input operation to adjust the brightness of the captured image (S310). When an input operation by the user is detected (Y in S310), the processor 10 changes the diaphragm mechanism 43 from standby mode to active mode (S311). As a result, the blade driving portion 432 of the aperture mechanism 43 is energized. Further, the blade control unit 431 operates the blade driving unit 432 to adjust the state of the diaphragm blades 433 so that the brightness of the portion touched by the user reaches a predetermined brightness. That is, the blade control unit 431 narrows down the aperture blades 433 to reduce the amount of light incident on the imaging device 42 when the location touched by the user is too bright, and when the location touched by the user is too dark. , the blade driving section 432 is operated so as to open the diaphragm blades 433 and increase the amount of light incident on the imaging device 42 .

<S320>
After that, when the user performs an image capturing operation, the processor 10 operates the image sensor 42 via the control unit 41 to capture an image of the subject (S320).

With the above processing, it is possible to capture an image of the subject after appropriately adjusting the brightness of the portion touched by the user on the image of the subject displayed on the display.

<4. Features of the present invention>
The present invention, which has been described by exemplifying the first to third embodiments, has the following features.

In the smartphone 1 of Embodiment 1, the aperture blades 433 are held in the minimum aperture state before the unlocking process. When the image is captured in the lock release process, the image is captured with the diaphragm blades 433 set to the minimum aperture state. You can make it look like By capturing an image in this state, it is possible to acquire a clear image, and it is possible to appropriately acquire feature points necessary for image authentication, so that it is possible to improve the authentication accuracy of face authentication. In addition, since the minimum aperture state is maintained before unlocking, the time required for face authentication can be shortened.

In the first embodiment, face authentication is performed as the unlocking process, but image authentication other than the face may be performed. For example, instead of face authentication, palm print authentication using a captured image or image authentication using fingerprint authentication may be performed.

Further, in the smartphone 1 of Embodiment 1, by opening the diaphragm blades 433 when capturing an image, it is possible to acquire a bright image by increasing the amount of captured light.

In addition, in the smartphone 1 of Embodiment 1, the diaphragm blades 433 are in the minimum aperture state when the blade drive unit 432 is not energized. ing. As a result, the aperture blades 433 can be maintained in the minimum aperture state while the blade driving section 432 is kept in the non-energized state before the unlocking process. Then, during the unlocking process, it is possible to take an image while keeping the aperture blades 433 in the minimum aperture state. As a result, power consumption in the locked state can be reduced, authentication accuracy can be improved, and the time required for authentication can be shortened.

In addition, since the smartphone 1 of Embodiment 1 does not have an autofocus mechanism and has a diaphragm mechanism 43, the configuration can be simpler than a configuration having an autofocus mechanism. Further, the depth of field can be deepened by narrowing down the diaphragm blades 433 of the diaphragm mechanism 43, and a clear image can be captured even with a configuration without autofocus. Furthermore, it becomes possible to adjust the amount of light taken in during imaging.

In addition, in the smartphone 1 of Embodiment 3, the blade driving unit 432 is in the non-energized state until the user performs an input operation, and the blade driving unit 432 is in the energized state when the input operation is performed. Therefore, after the user performs an input operation, the aperture blades 433 can be driven, and the aperture can be adjusted. Further, the aperture mechanism 43 may operate in the standby mode until the user performs an input process, and operate in the active mode when the input operation is performed.

<4. Supplementary matter>
A specific description has been given above of a plurality of embodiments of the present invention. These embodiments are specific examples as one configuration example and one operation example of the present invention, and the scope of the present invention is not limited to these embodiments, and can be grasped by those skilled in the art based on the same technical idea. It is interpreted broadly to the extent possible.

For example, the processes of Embodiments 1-3 may be executed in combination with each other. Specifically, when capturing an image, a combination of the processing of the second embodiment and the processing of the third embodiment may be performed.

The portable electronic device of the present invention is suitably applied to smartphones and the like that unlock by image authentication such as face authentication.

1 Smartphone 10 Processor 11 Memory 20 Display 21 Display control unit 30 Touch panel 31 Touch panel control unit 40 Camera module 41 Control unit 42 Imaging device 43 Aperture mechanism 431 Blade control unit 432 Blade drive Part 433... Aperture blade

Claims (6)

an imaging device;
a lens;
A diaphragm blade capable of adjusting the amount of light irradiated to the imaging device;
a blade drive unit that drives the aperture blades;
a blade control unit that controls the blade drive unit;
an authentication unit that performs image authentication based on an image captured by the imaging device during unlocking processing,
The aperture blades are held in a minimum aperture state before the unlocking process,
portable electronic devices. The blade control unit operates the blade control unit so that the aperture blades are in an open state during normal imaging that is not unlock processing .
A portable electronic device according to claim 1 . The aperture blades are in the minimum aperture state when the blade drive section is de-energized,
Before the unlocking process, the blade driving unit is in a non-energized state,
The portable electronic device according to claim 1 or 2. does not have an autofocus mechanism,
The portable electronic device according to any one of claims 1 to 3. The blade control unit keeps the blade drive unit in a non-energized state until an input operation is performed, and puts the blade drive unit in an energized state when the input operation is performed.
The portable electronic device according to any one of claims 1 to 4. Operates in a standby mode in which the blade control unit is energized and the blade drive unit is de-energized until the input operation is performed;
When the input operation is performed, it operates in an active mode in which the blade drive unit is energized in addition to the blade control unit.
A portable electronic device according to claim 5 .

Images