JP4229201B2 - Input device, information device, and control information generation method - Google Patents

Description

  The present invention relates to an input device, an information device including the same, and a control information generation method.

  The input device is used for an operation unit of an electronic device (information device or information device). For example, when the user operates the input device, the electronic device uses the control information (operation information) output from the input device to move the pointer displayed on the display unit or scroll the image on the display unit. I will let you. For this input device, it is necessary not to deteriorate the operability of the user.

As an input device, for example, there is one that improves operability when an arbitrary position in a three-dimensional space is designated. In this input device, first, a reference point is set. When the indicated position does not appear on the screen, after moving the viewpoint by combining movement around the reference point and movement along a straight line connecting the reference point and the viewpoint, Redisplay the three-dimensional space from the viewpoint after movement. When the indicated position appears on the screen, the cursor is moved on the screen (for example, Patent Document 1). By enabling such an operation, it is not necessary to perform a so-called 6-axis operation in this input device.
Japanese Patent Laid-Open No. 5-40571

  However, the input device disclosed in Patent Document 1 is difficult to apply to a portable information device. In addition, when applied to a portable information device, it is necessary to have a configuration that can be operated by a battery and can be downsized. Furthermore, it is desirable that the input device having the above-described functions can be applied not only to a three-dimensional CAD device and a virtual actual experience device that perform advanced information processing, but also to a mobile phone or PDA with a smaller processing load.

  The present invention has been made in view of the above technical problems, and an object of the present invention is to provide an input device capable of improving operability more than ever without increasing the processing load. An information device and a control information generation method are provided.

  Another object of the present invention is to provide an input device, an information device, and a control information generation method that are ultra-compact and ultra-light and can improve operability more than ever.

  In order to solve the above problems, the present invention is an input device that generates control information by moving an object to be detected, the image capturing unit capturing an image of the object to be detected, and the image capturing unit capturing the image A feature point extraction unit that extracts feature points of the image, a difference calculation unit that obtains a difference between a reference position and the position of the feature point, and a control information output unit that outputs control information corresponding to the difference Concerning input device including.

  In the present invention, the feature point of the image of the moving object to be detected is extracted and the control information corresponding to the difference from the reference position is output. Therefore, the comparison process using the feature point may be performed. Therefore, it is possible to reduce the processing load of the input device that outputs the control information by moving the image of the detected object.

  In the input device according to the present invention, the reference position may be a position of a feature point extracted from an image of the detected object at least one frame before.

  According to the present invention, since the image of the object to be detected of one or more previous frames is used, it is possible to capture an image at a frame period and output control information in units of frames. Thereby, more accurate control information can be generated.

  The present invention is also an input device that generates control information by moving an object to be detected, the image capturing unit capturing an image of the object to be detected, and an area of the image captured by the image capturing unit. An area calculation unit to be obtained; a differential calculation unit for obtaining a difference between an area of the image of the detection object at least one frame before and an area obtained by the area calculation unit; and control information corresponding to the difference The present invention relates to an input device including a control information output unit for outputting.

  According to the present invention, the difference between frames is obtained for the area of the captured image of the detected object, and the control information corresponding to the difference is output, so that the new position is not limited to the movement of the position of the captured image. It becomes possible to perform a control instruction in the axial direction. Thereby, it becomes possible to improve operability more than before.

  In the input device according to the present invention, the image capturing unit has a detection surface, captures an image of an object to be detected that is in contact with the detection surface, and the control information output unit has an axial direction perpendicular to the detection surface. Control information corresponding to the amount of movement of the detected object can be output.

  According to the present invention, the amount of movement of the detection object in the direction of the axis (Z axis) perpendicular to the detection surface can be specified by the change in the area of the image of the detection object in contact with the detection surface. Control information (Δz) corresponding to not only the movement in the first and second axial directions perpendicular to each other but also the movement in the axial direction perpendicular thereto can be generated. Thereby, it becomes possible to improve operability more than before.

  Further, the present invention is an input device that generates control information by moving a detection object, having an detection surface, an image capturing unit that captures an image of the detection object that is in contact with the detection surface, and the image A center-of-gravity position calculation unit for obtaining a center-of-gravity position of an image captured by the capturing unit, a feature point extraction unit for extracting a feature point of the image, a center of gravity position of the image of the detected object at least one frame before, A center-of-gravity position movement detection unit for obtaining a first difference (ΔX1, ΔY1) from the center-of-gravity position obtained by the center-of-gravity position calculation unit; and a position of a feature point of the image of the detected object at least one frame before A feature point position movement detection unit that obtains a second difference (ΔX2, ΔY2) from the feature point position extracted by the feature point extraction unit; and on the detection surface based on the first and second differences. First or orthogonal to each other A rotation angle detection unit that detects a rotation angle around two axes (X axis or Y axis), and the rotation angle detection unit calculates the first difference from the first axial component (ΔX1) of the first difference. A rotation angle (γ) around the second axis (Y axis) is obtained by subtracting the first axial component (ΔX2) of the second difference, and the second axial direction of the first difference A rotation angle (α) about the first axis (X axis) is obtained by subtracting the second axial component (ΔY2) of the second difference from a component (ΔY1), and the first or second The present invention relates to an input device that outputs control information corresponding to a rotation angle around an axis.

  In the present invention, the rotation angle around the first axis is obtained by subtracting the second axial component (ΔY2) of the second difference from the second axial component (ΔY1) of the first difference, The rotation angle around the axis of 2 is obtained by subtracting the second axial component (ΔY2) of the second difference from the second axial component (ΔY1) of the first difference. As a result, a processing load is applied to the control information (α, γ) about the first and second axes related to the three-dimensional movement (rotation) of the detected object from the two-dimensional detected object image. Can be generated. Therefore, it is possible to provide an input device that can improve operability more than ever.

  In the input device according to the present invention, the rotation angle detection unit is characterized in that the position of the feature point of the image of the detected object at least one frame before is extracted by the feature point extraction unit around a reference point. The angle rotated to the position of the point is obtained as a rotation angle around the third axis perpendicular to the detection surface, and control information corresponding to the rotation angle around the third axis can be output.

  According to the present invention, the rotation angle around the reference point can be obtained for the feature point of the image of the object to be detected. Therefore, the rotation angle around the third axis perpendicular to the detection surface can be obtained with less processing load. Can be generated, and an input device capable of improving operability more than ever can be provided.

  In the input device according to the present invention, a first coefficient given from the first axial component (ΔX2) of the second difference to a rotation angle (γ) about the second axis (Y axis). The control information (Δx) corresponding to the movement in the first axis (X axis) direction is output by subtracting the value weighted in (a), and the second axial component (2) of the second difference ( By subtracting a value weighted by a given second coefficient (b) from the rotation angle (α) around the first axis (X axis) from ΔY2), the second axis (Y axis) direction Control information (Δy) corresponding to the movement can be output.

In the input device according to the present invention, a given rotation angle (γ) about the second axis (Y axis) can be obtained from the first axial component (ΔX _G or ΔX _M ) of the first difference. The control information (Δx) corresponding to the movement in the first axis (X-axis) direction is output by subtracting the value weighted by the first coefficient (a), and the second difference of the first difference is output. By subtracting a value weighted by a given second coefficient (b) from the axial component (ΔY _G or ΔY _M ) to the rotation angle (α) around the first axis (X axis), the second The control information (Δy) corresponding to the movement in the direction of the axis (Y axis) can be output.

  According to the present invention, the movement of the detected object in the first and second axial directions is detected in consideration of the rotation angle in each axial direction with respect to the movement of the feature point or the gravity center position of the image of the detected object. Therefore, the optimal movement in the first and second axial directions can be detected in accordance with the manner in which the object to be detected is moved (such as a heel in which the user moves the object to be detected). Thereby, when outputting the control information around the first or second axis, the control information in the first or second axial direction can be output with high accuracy, and the operability is improved more than ever. It is possible to provide an input device that can handle the above.

  In the input device according to the present invention, the detected object may be a fingerprint.

  According to the present invention, since an ultra-small or ultra-lightweight fingerprint sensor can be used, an input device that improves operability more than ever can be mounted on a portable information device.

  The present invention also relates to an information device including any one of the input devices described above and a processing unit that performs control processing based on control information from the input device.

  According to the present invention, it is possible to provide an information device capable of improving operability more than before without increasing the processing load.

  Further, the present invention is a control information generation method for generating control information using the captured image of the detected object, wherein a feature point of the image of the detected object is extracted, a reference position, a position of the feature point, And a control information generation method for outputting control information corresponding to the difference.

  The present invention is also a control information generation method for generating control information using a captured image of a detected object, wherein the area of the image of the detected object in contact with the detection surface is obtained, and the area and at least one frame or more Related to a control information generation method for obtaining a difference with the area of the image of the previous detection object and outputting control information corresponding to the difference according to the movement of the detection object in a direction perpendicular to the detection surface To do.

  Further, the present invention is a control information generation method for generating control information using a captured image of a detected object, wherein the center of gravity position of the image of the detected object in contact with the detection surface is obtained and the feature point of the image is extracted And calculating a difference between the position of the center of gravity and the position of the center of gravity of the image of the detected object at least one frame before, and the position of the feature point and the feature point of the image of the detected object at least one frame before A rotation angle around the first or second axis orthogonal to each other on the detection surface is detected based on the difference between the center of gravity position and the difference between the feature points, The present invention relates to a control information generation method for outputting corresponding control information.

  Also, in the control information generation method according to the present invention, the detection is performed based on an angle at which the position of the feature point of the image of the object to be detected at least one before is rotated around the reference point to the position of the extracted feature point. A rotation angle about a third axis orthogonal to the first and second axes of the surface can be obtained.

  In the control information generation method according to the present invention, the control corresponding to the movement in the first or second axial direction is performed using the difference between the feature points and the rotation angle around the first or second axis. Information can be output.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the drawings. The embodiments described below do not unduly limit the contents of the present invention described in the claims. Moreover, not all of the configurations described below are essential constituent requirements of the present invention.

1. Input Device FIG. 1 shows an outline of the configuration of an input device according to this embodiment. The input device 10 in the present embodiment can output control information (operation information) in six axis directions based on changes in the captured image. Therefore, the input device 10 includes an image capture unit 20, an image analysis unit 30, a difference calculation unit 40, and a control information output unit 50.

  The image capturing unit 20 captures a two-dimensional or three-dimensional object to be moved by a user as an image that is two-dimensional information via a detection surface (sensor surface), and generates image information for each frame. .

  The image analysis unit 30 analyzes the image captured by the image capture unit 20 and outputs the analysis result to the difference calculation unit 40. More specifically, the image analysis unit 30 includes a feature point extraction unit 32, a centroid position calculation unit 34, and an area calculation unit 36.

  The feature point extraction unit 32 extracts feature points of the image captured by the image capture unit 20. Here, the feature point refers to a position (region) having a form peculiar to the image that can be referred to in order to identify the movement distance, the movement direction, or the rotation angle by comparing the two images before and after the movement.

  The center-of-gravity position calculation unit 34 obtains the center-of-gravity position of the image captured by the image capturing unit 20 or a position equivalent thereto. More specifically, the area center position of the image of the object to be detected or a position equivalent thereto is obtained.

  The area calculating unit 36 obtains the area of the image captured by the image capturing unit 20 or a value equivalent thereto. More specifically, the area of the image of the object to be detected or a value equivalent thereto is obtained.

  The difference calculation unit 40 refers to a past image (a given reference position of the image) at least one frame before stored in the memory 48, compares it with the analysis result from the image analysis unit 30, and the difference Is output to the control information output unit 50. More specifically, the difference calculation unit 40 includes a feature point movement detection unit 42, a gravity center position movement detection unit 44, and an area change detection unit 46.

  The feature point movement detection unit 42 includes the feature points of the image of the detected object in the frame extracted by the feature point extraction unit 32 and the images of the previous detected object stored in the memory 48 at least one frame before. The feature point is compared, and the change in the position is detected as a difference (second difference). When the difference is obtained, the feature point of the image of the detected object in the current frame is stored in the memory 48 and used to detect the change in the position of the feature point of the image of the detected object in the next frame.

  The center-of-gravity position movement detection unit 44 determines the center-of-gravity position of the detected object image of the frame obtained by the center-of-gravity position calculating unit 34 and the past detected object image stored at least one frame or more in the memory 48. The center of gravity position is compared, and a change in the position is detected as a difference (first difference). When the difference is obtained, the barycentric position of the image of the detected object in the frame is stored in the memory 48, and is used for detecting the change in the barycentric position of the image of the detected object in the next frame.

  The area change detection unit 46 calculates the area of the image of the detected object in the frame obtained by the area calculation unit 36 and the area of the image of the previous detected object stored in the memory 48 at least one frame before. The change is detected and the change is detected as a difference. When the difference is obtained, the area of the image of the detected object in the current frame is stored in the memory 48, and is used to detect a change in the area of the image of the detected object in the next frame.

  The control information output unit 50 outputs control information corresponding to a difference from an image of the following detected object at least one frame before by the difference calculation unit 40. This control information is control information in the six-axis direction as described above.

  FIG. 2 is an explanatory diagram of control information in six axis directions. The control information in the six-axis direction means that the X axis and the Y axis (first and second axes) orthogonal to each other on the detection surface (sensor surface) 22 of the image capturing unit 20 (or on a plane parallel to the detection surface). ) Direction changes Δx, Δy, Z axis (third axis) direction change Δz perpendicular to the detection surface, rotation angle α around the axis in the X axis direction, rotation angle γ around the axis in the Y axis direction , Information instructed for the six-axis direction of the rotation angle β around the axis in the Z-axis direction. X axis direction change Δx, Y axis direction change Δy, Z axis direction change Δz, X axis direction axis rotation angle α, Z axis direction axis rotation angle β, Y axis direction axis rotation As shown in FIG. 2, a (+) direction and a (−) direction are defined for the rotation angle γ.

  In this embodiment, in order to obtain the rotation angle around each axis with a smaller processing load in addition to the movement of each of the X, Y, and Z axes defined on the detection surface, the above-described detected object The movement of the center of gravity position of the image and the movement of the feature point of the image of the detected object are used. Therefore, the control information output unit 50 includes an X axis movement detection unit 52, a Y axis movement detection unit 54, an X axis rotation detection unit 56, a Y axis rotation detection unit 58, a Z axis rotation detection unit 60, and a Z axis movement detection. The control information corresponding to the output of each part is output including the part 62.

  The X-axis movement detection unit 52 detects the movement of the detection surface of the detected object in the X-axis direction. At that time, the X-axis movement detection unit 52 considers the rotation of the detected object around the axis in the Y-axis direction, and the detected surface of the detected object even when the detected object is slid on the detection surface and rotated. The amount of movement Δx in the X-axis direction is detected. More specifically, the X-axis movement detection unit 52 sets the movement amount in the X-axis direction of the feature point of the image of the detected object as ΔX2 (first axial component of the second difference), and the axis in the Y-axis direction. Assuming that the rotation angle is γ (rotation angle about the second axis), the amount of movement Δx in the X-axis direction of the detection surface of the detected object that can be expressed as in equation (1) is obtained.

Δx = ΔX2-a · γ (where 0 ≦ a ≦ 1) (1)
Here, the coefficient a (given first coefficient) is a weighting coefficient of the rotation angle γ around the axis in the Y-axis direction. When the weighting coefficient a is 0, it means that the rotation angle around the axis in the Y-axis direction is not taken into consideration at all. Therefore, it is suitable for Δx output when the detection object moves without rotating on the detection surface. In addition, when the detection surface is rotated without sliding, it is suitable for outputting the rotation angle γ and setting Δx to “0”. Further, when the weighting coefficient a is 1, it is suitable for a case where the object to be detected only rotates and does not move by sliding the detection surface. It is desirable that the weighting coefficient a is determined by taking into account the user's habit of operation in the X-axis direction.

  Similarly, the Y-axis movement detection unit 54 detects the movement of the detection surface of the detected object in the Y-axis direction. At that time, the Y-axis movement detection unit 54 considers the rotation of the detected object around the axis in the X-axis direction, and the detected surface of the detected object even when the detected object is slid on the detection surface and rotated. Is detected in the Y-axis direction. More specifically, the Y-axis movement detection unit 54 sets the amount of movement of the feature point of the image of the detected object in the Y-axis direction to ΔY2 (second axial component of the second difference) and the X-axis direction axis. Assuming that the rotation angle is α (rotation angle around the first axis), the amount of movement Δy in the Y-axis direction of the detection surface of the object to be detected, which can be expressed as equation (2), is obtained.

Δy = ΔY2−b · α (where 0 ≦ b ≦ 1) (2)
Here, the coefficient b (given second coefficient) is a weighting coefficient of the rotation angle α around the axis in the X-axis direction. When the weighting coefficient b is 0, it means that the rotation angle around the axis in the X-axis direction is not considered at all, which is suitable for Δy output when the detection object moves on the detection surface without rotating. Further, when the detection surface is rotated without sliding, the rotation angle α is output, and this is suitable for setting Δy to “0”. Further, when the weighting coefficient b is 1, it is suitable for a case where the object to be detected only rotates and does not move by sliding the detection surface. It is desirable that the weighting coefficient b is determined by taking into account the user's habit of operation in the Y-axis direction.

  The X axis rotation detection unit 56 detects a rotation angle α around the axis in the X axis direction. More specifically, on the condition that the rotation radius is sufficiently large or the rotation angle is sufficiently small, the amount of movement in the Y-axis direction of the center of gravity position of the image of the object to be detected is expressed as ΔY1 (first difference of the first difference). 2), the rotation angle α around the axis in the X-axis direction, which can be expressed as equation (3), is obtained.

α≈− (ΔY1−ΔY2) (3)
Here, the negative sign depends on the positive and negative directions of the rotation angle defined in FIG. Therefore, if the positive and negative directions of the rotation angle α in FIG. 2 are reversed, the sign of the expression (3) becomes positive.

  In addition, from the equations (2) and (3), the X-axis rotation output unit 56 determines the amount of movement ΔY2 in the Y-axis direction of the feature point of the image of the detected object (the difference obtained by the feature point movement detection unit 42). ) And the amount of movement in the Y-axis direction of the center of gravity position of the image of the object to be detected, the rotation angle α about the axis in the X-axis direction is obtained based on ΔY1 (difference obtained by the center-of-gravity position movement detection unit 44). be able to.

  The Y-axis rotation detection unit 58 detects a rotation angle γ around the axis in the Y-axis direction. More specifically, on the condition that the radius of rotation is sufficiently large or the angle of rotation is sufficiently small, the amount of movement in the X-axis direction of the center of gravity position of the image of the object to be detected is set to ΔX1 (first difference of the first difference). 1), the rotation angle γ around the axis in the Y-axis direction, which can be expressed as in equation (4), is obtained.

γ≈ΔX1−ΔX2 (4)
The Z-axis rotation detection unit 60 detects a rotation angle β around the axis in the Z-axis direction. More specifically, when the feature points of the image of the object to be detected before and after the movement are associated with each other when the feature points are rotated around a given reference point, the rotation angle β is obtained as the rotation angle β. .

  It should be noted that, based on the equations (1) and (4), the Y-axis rotation detection unit 58 moves the feature point of the image of the detected object in the X-axis direction ΔX2 (difference obtained by the feature point movement detection unit 42). And calculating the rotation angle γ about the axis in the Y-axis direction based on ΔX1 (difference obtained by the centroid position movement detection unit 44) of the movement amount in the X-axis direction of the center-of-gravity position of the image of the object to be detected. Can do.

  The Z-axis movement detection unit 62 detects movement in a direction perpendicular to the detection surface of the detection object. More specifically, the amount of movement Δz in the Z-axis direction is obtained by focusing on the fact that the captured image of the detected object changes according to the pressure on the detection surface. If the area of the image of the object to be detected before movement is S1, and the area of the image of the object to be detected after movement is S2, the amount of movement Δz in the Z-axis direction that can be expressed as in equation (5) is obtained.

Δz = c · (S1-S2) (5)
Here, the coefficient c is a coefficient for associating the change in the area of the image of the detection object with the movement amount in the Z-axis direction. The coefficient c is preferably determined by taking into account the user's habit of operation in the Z-axis direction.

  Hereinafter, such an input device will be described in detail. The input device described below uses a fingerprint sensor, but the present invention is not limited to this.

  FIG. 3 shows an outline of an external configuration of an input device using a fingerprint sensor. Here, a case where the input device in the present embodiment is mounted on an IC card (information device in a broad sense) 64 is shown. The IC card 64 has a CPU and a memory element inside. As a result, the IC card 64 can improve the confidentiality and store more advanced information by information processing. By using the input device according to the present embodiment, it is possible to perform information processing that reflects various operations of the user with an ultra-compact and ultra-lightweight configuration.

  In FIG. 3, a fingerprint image is captured by bringing a user's finger (detected object in a broad sense) 66 on which a fingerprint pattern is formed into contact with a detection surface 22 of a fingerprint sensor as an input device. The corresponding control information is output by the movement of the finger 66 in the six-axis direction by the user detected in the three-dimensional space defined on the detection surface 22. In the IC card 64, processing based on the control information is performed. For example, when the IC card 64 is provided with a liquid crystal panel, display control such as movement of a pointer displayed on the liquid crystal panel and scrolling of a display image is performed. When the input device is applied to a three-dimensional CAD device, the rotation of the object to be operated or the movement of the viewpoint is controlled.

  FIG. 4 shows a hardware configuration example of the input device. In the input device 68, a CPU 72, a ROM 74, a RAM 76, and a fingerprint sensor interface (InterFace: I / F) circuit 78 are connected to the bus 70. A fingerprint sensor 80 is connected to the fingerprint sensor I / F circuit 78. A USB I / F circuit 82 is connected to the bus 70. The USB I / F circuit 82 is externally connected to a USB standard host device or peripheral device such as a personal computer 84.

  Here, the function of the image capturing unit 20 shown in FIG. 1 is realized mainly by the fingerprint sensor 80 and the fingerprint sensor I / F circuit 78. The fingerprint image captured by the fingerprint sensor 80 is stored in the RAM 76 via the fingerprint sensor I / F circuit 78. The functions of the image analysis unit 30, the difference calculation unit 40, and the control information output unit 50 shown in FIG. 1 are realized by the CPU 72 and the software program stored in the ROM 74 or RAM 76. The RAM 76 implements the function of the memory 48 shown in FIG.

1.2 Fingerprint Sensor FIG. 5 shows an example of the fingerprint sensor 80. In FIG. 5, there are M (M is an integer of 2 or more) power supply lines 200 and N (N is an integer of 2 or more) output lines 202. A capacitance detection element 204 is provided at each intersection of the M power lines 200 and the N output lines 202. The electrostatic capacitance detection device shown in FIG. 5 204 is illustrated as a closed circuit when a finger is in contact, and a variable capacitance C _F, which changes depending on the uneven pattern of the finger, the signal amplifying element e.g. signal amplification MIS type A thin film semiconductor device (hereinafter abbreviated as a signal amplification TFT) 206. When the finger is not in contact with the capacitance detection element 204, a ground terminal of the variable capacitance C _F is open. The variable capacitor C _F will be described later.

  Each of the M power supply lines 200 is connected to the drains D of the N signal amplification TFTs 206 arranged along the corresponding row. Further, each of the M power supply lines 200 is connected to the common power supply line 212 via each of the M power supply pass gates 210. That is, the power supply pass gate 210 is formed of a MIS type thin film semiconductor device, its source S is connected to the power supply line 200, and its drain D is connected to the common power supply line 212. In the power supply selection circuit 220, in addition to the M power supply pass gates 210 and the common power supply line 212, a power supply shift register 222 is provided. The gates G of the M power supply pass gates 210 are connected to the power supply selection output line 224 of the power supply shift register 222.

  Each of the N output lines 202 is connected to the sources S of M signal amplification TFTs 206 arranged along a corresponding column. Each of the N output lines 202 is connected to the common output line 232 through each of the N output signal pass gates 230. That is, the output signal pass gate 230 is formed of a MIS type thin film semiconductor device, the drain D is connected to the output line 202, and the source S is connected to the common output line 232. In the output signal selection circuit 240, an output signal shift register 242 is provided in addition to the N output signal pass gates 230 and the common output line 232 described above. The gate G of the output signal pass gate 230 is connected to the output selection output line 244 of the output signal shift register 242.

  FIG. 6 is a cross-sectional view of the capacitance detection element 204 shown in FIG. 5 and illustrates a state in which a finger is not touched. The capacitance detection element 204 includes a signal detection element 208 in addition to the signal amplification TFT 206 which is the signal amplification element described above.

  In FIG. 6, over the insulating layer 250, a semiconductor film 252 having a source region 252A, a drain region 252B, and a channel region 252C therebetween is formed. A gate insulating film 254 is formed over the semiconductor film 252, and a gate electrode 256 is formed in a region facing the channel region 252C with the gate insulating film 254 interposed therebetween. The semiconductor film 252, the gate insulating film 254, and the gate electrode 256 constitute a signal amplification TFT 206. The power supply pass gate 210 and the output signal pass gate 230 are also formed in the same manner as the signal amplification TFT 206.

  The signal TFT 206 is covered with a first interlayer insulating film 260. A first wiring layer 262 corresponding to the output line 202 shown in FIG. 5 is formed on the first interlayer insulating film 260. The first wiring layer 262 is connected to the source region 252A of the signal TFT 206.

  The first wiring layer 262 is covered with a second interlayer insulating film 264. A second wiring layer 266 corresponding to the power supply line 200 shown in FIG. 5 is formed on the second interlayer insulating film 264. The second wiring layer 266 is connected to the drain region 252B of the signal amplification TFT 206. As a structure different from that in FIG. 6, the second wiring layer 266 may be formed on the first interlayer insulating film 260 and the first wiring layer 262 may be formed on the second interlayer insulating film 264.

  A capacitance detection electrode 270 is further formed on the second interlayer insulating film 264, and a capacitance detection dielectric film 272 is formed over the capacitance detection electrode 270. The capacitance detection dielectric film 272 is located on the outermost surface of the fingerprint sensor 80 and also functions as a protective film, and a finger is brought into contact with the capacitance detection dielectric film 272. The capacitance detection electrode 270 and the capacitance detection dielectric film 272 constitute a signal detection element 208.

1.2.1 Fingerprint Detection Operation Fingerprint detection is performed by bringing a finger into contact with the capacitance detection dielectric film 272 shown in FIG. At this time, the start switch (for example, pressure sensitive switch) 42 of the fingerprint sensor 80 is activated, the power supply in the input device 68 is activated, and the fingerprint sensor 80 is automatically supplied with power. Alternatively, the input device 68 may be set in the personal computer 84 and power may be supplied from the power supply unit of the personal computer 84.

  In the present embodiment, a power supply voltage is supplied to one power supply line 200 selected from among the M lines shown in FIG. 5, and a signal at that time is output from one output line 202 selected from among N lines. , The signals are sequentially taken out from the M × N capacitance detection elements 204.

  Fingerprint detection operations are broadly divided into (1) a case where a peak (convex part) of a fingerprint pattern is in contact with the capacitance detection dielectric film 272, and (2) a valley (concave part) of the fingerprint pattern is opposed to the capacitance detection dielectric film 272. There is a case to do.

(1) When a peak (convex portion) of a fingerprint pattern contacts the capacitance detection dielectric film 272 FIG. 7 shows an equivalent circuit of the capacitance detection element 204 in this case. Reference numeral 300 corresponds to a crest of a fingerprint of a human body, and a ground electrode 300 is formed opposite to the capacitance detection electrode 270 of FIG. 6 with the dielectric film 272 interposed therebetween. Here, the power supply voltage Vdd is supplied from the common power supply line 212. Code _{C T} is the transistor capacitance of the signal amplifying TFT 206, a capacitance between the code _{C D} is the detection electrode 270 and the ground electrode (finger) 300.

Here, the gate electrode length of the signal amplification TFT 206 is L (μm), the gate electrode width is W (μm), the gate insulating film thickness is tox (μm), the relative dielectric constant of the gate insulating film is εox, and the vacuum dielectric is Let εo be the rate. At this time, the transistor capacitance _CT is expressed by equation (6).

C _T = εo · εox · L · W / tox (6)
Further, the area S of the capacitance detection electrode 270 (μm ² ), the thickness of the capacitance detection dielectric film 272 is td (μm), and the relative dielectric constant of the capacitance detection dielectric film is εd. At this time, the capacitance _CD is as shown in equation (7).

C _D = εo · εd · S / td (7)
In the equivalent circuit of FIG. 7, the voltage V _GT applied to the gate of the signal amplification TFT 206 is expressed by the following equation.

V _GT = Vdd / (1 + C _D / C _T ) (8)
If the capacitance C _D is set sufficiently larger than the transistor capacitance C _T (for example, C _D > 10 × C _T ), the denominator of the equation (8) becomes infinite and approximated as the following equation.

V _GT ≈ 0 (9)
As a result, the signal amplification TFT 206 is turned off because almost no voltage is applied to its gate. Therefore, the current I flowing between the source and drain of the signal amplification TFT 206 is extremely small. By measuring this current I, it can be determined that the measurement location is a peak (convex portion) of the fingerprint pattern.

(2) Case where Valley (Recess) of Fingerprint Pattern Opposes Capacitance Detection Dielectric Film 272 FIG. 8 shows an equivalent circuit of the capacitance detection element 204 in this case. Reference numeral 302 corresponds to a valley of a human body fingerprint. In this case, in addition to the capacitor C _D shown in FIG. 7, between the valley of the dielectric film 272 and a fingerprint, a new capacitance C _A to the air as a dielectric is formed.

In the equivalent circuit of FIG. 8, the voltage V _GV applied to the gate of the signal amplification TFT 206 is expressed by the following equation.
V _GV = Vdd / {[1+ (1 / C _T )] × 1 / [(1 / C _D ) + (1 / C _A )]} (10)
If the capacitance C _D is set sufficiently larger than the transistor capacitance C _T (for example, C _D > 10 × C _T ), the equation (10) is approximated as the following equation.

V _GV ≈ Vdd / [1+ (C _A / C _T )] (11)
Further, if the transistor capacitance C _T is sufficiently larger than the capacitance C _A formed by the fingerprint valley (for example, C _T > 10 × C _A ), the equation (11) is approximated as follows: The

V _GV ≈ Vdd (12)
As a result, the signal amplification TFT 206 is turned on because the power supply voltage Vdd is applied to its gate. Therefore, the current I flowing between the source and drain of the signal amplification TFT 206 becomes extremely large. By measuring this current I, it can be determined that the measurement location is a valley (concave portion) of the fingerprint pattern.

Thus, the variable capacitance C _F is shown in FIG. 5, the capacitor C _D next time the mountain of the fingerprint is in contact with the capacitance detection dielectric film 272, when the valleys of the fingerprint was opposed to the capacitance detection dielectric film 272 is It becomes the sum of capacitance C _D and capacitance C _A, and the capacitance changes according to the unevenness of the fingerprint. By detecting the current based on the capacitance change according to the unevenness of the fingerprint, the peak or valley of the fingerprint can be detected.

  A fingerprint pattern can be detected by performing the above operation in a time-sharing manner with the M × N capacitance detection elements 204. More specifically, after detecting the fingerprint irregularities of the capacitance detection elements located in each column of the first row in order, the fingerprint irregularities of each pixel are detected, such as detecting the fingerprint irregularities of the second row. Will be detected. As a result, for example, a fingerprint image as shown in FIGS. 9A and 9B can be obtained. In this embodiment, the fingerprint sensor 80 is used to periodically capture a fingerprint image.

When a positive power supply is used as the power supply voltage Vdd, the signal amplification TFT 206 may be formed of an enhancement type N-type transistor in which the drain current does not flow when the gate voltage is near zero. When C _D > 10 × C _T is satisfied, 0 <Vmin <0.1 × Vdd, where Vmin is the gate voltage (minimum gate voltage) at which the drain current in the transfer characteristic of the signal amplification TFT 206 becomes the minimum value. Should be satisfied.

When a negative power supply is used for the power supply voltage Vdd, the signal amplification TFT 206 may be formed of an enhancement type P-type transistor in which the drain current does not flow when the gate voltage is near zero. When C _D > 10 × C _T is satisfied, 0.1 × Vdd <Vmin <0, where Vmin is a gate voltage (minimum gate voltage) at which the drain current in the transfer characteristic of the signal amplification TFT 206 is minimum. Should be satisfied.

  In the present embodiment, control information is output using the fingerprint image thus captured. At this time, the processing load is reduced by using the feature points of the captured fingerprint image.

  FIGS. 9A and 9B show examples of fingerprint feature points. FIG. 9A illustrates an example of a fingerprint branch point. FIG. 9B illustrates an example of fingerprint endpoints. For example, a fingerprint branch point is extracted from the fingerprint image captured by the fingerprint sensor 80. 9A and 9B, the fingerprint image represents the form of a ridge line that is a convex portion of the fingerprint. Here, the branching point of the fingerprint is a part where the ridgeline of the fingerprint branches into two or more ridgelines. The end point of the fingerprint is a portion where the ridge line of the fingerprint ends.

  Since the fingerprints do not have the same form, the distribution of branch points or end points varies depending on the individual. Therefore, if the branch points or end points of the fingerprint image can be obtained, only the distributions of the obtained branch points or end points need to be compared. Therefore, the amount of information to be compared is reduced, and the load of the comparison process can be reduced.

1.3 Operation Flow FIG. 10 shows an example of the processing flow of the input device according to this embodiment. A program for executing the processing shown in FIG. 10 is stored in the ROM 74 or RAM 76. The CPU 72 performs processing according to this program.

  First, the CPU 72 initializes each variable prior to processing as an input device (step S400).

  Next, it is determined whether or not the user's finger (object to be detected) is in contact with the detection surface of the fingerprint sensor 80 (step S401). When it is determined that they are not in contact (step S401: N), the process returns to step S400.

  On the other hand, when it is determined in step S401 that contact is made (step S401: Y), a fingerprint image is captured as described above (step S402). The captured fingerprint image for one frame is stored in the RAM 76.

  Then, the CPU 72 reads the fingerprint image for one frame stored in the RAM 76, and obtains the position of the center of gravity of the fingerprint image (step S403). Here, the center-of-gravity position is the center position in terms of the area where the fingerprint pattern exists in the captured fingerprint image. The center-of-gravity position is obtained as the center position of the area of the fingerprint image obtained by performing the integration calculation in the X-axis direction and the Y-axis direction, for example.

For example, if the coordinates of the center of gravity position are (X _G , Y _G ), the following equation is obtained.

  Here, S is the area of the fingerprint image, F (i) is the number of detected dots on the output line i, and G (j) is the number of detected dots on the power line j.

  Subsequently, it is compared with the barycentric position of the fingerprint image of the previous frame (step S404). The X component of the difference from the centroid position of the fingerprint image of the previous frame is output as the amount of movement of the centroid position in the X-axis direction, and the Y component is output as the amount of movement of the centroid position in the Y-axis direction as ΔY1. The barycentric position of the fingerprint image of the previous frame is stored in the RAM 76. The centroid position of the fingerprint image of the previous frame is updated with the centroid position of the fingerprint image of the frame obtained in step S403 after the comparison processing in step S404. Although the fingerprint image of the previous frame is used as the comparison target here, the present invention is not limited to this, and a fingerprint image of two or more frames in the past (an image of an object to be detected two or more frames before) may be used.

  Next, the area of the captured fingerprint image is obtained (step S405). The area of the fingerprint image can be obtained by performing the above integral calculation. The area can be equivalently obtained from the number of dots (number of pixels) detected as a peak or valley portion of the fingerprint image.

  The obtained fingerprint image area is compared with the fingerprint image area of the previous frame (step S406). As indicated by the above equation (5), the movement amount Δz of the user's finger in the Z-axis direction is obtained in accordance with the change in the area of the fingerprint image before and after the movement. The area of the fingerprint image of the previous frame is stored in the RAM 76. The area of the fingerprint image of the previous frame is updated with the area of the fingerprint image of the frame obtained in step S405 after the comparison processing in step S406. Although the fingerprint image of the previous frame is used as the comparison target here, the present invention is not limited to this, and a fingerprint image of two or more frames in the past (an image of an object to be detected two or more frames before) may be used.

  FIGS. 11A to 11C show an example of changes in the fingerprint image when the finger is moved in the Z-axis direction. It is assumed that the fingerprint image 452 in the frame f shown in FIG. 11B is the origin position in the fingerprint image detection area 450. When the fingerprint image captured in the frame (f + 1) is the fingerprint image 454 shown in FIG. 11A, the user moves his / her finger in the direction of lifting from the detection surface of the fingerprint sensor 80 (Z-axis (−) direction). It corresponds to the case. At this time, since the pressure with which the finger presses the detection surface is decreased, the area where the finger contacts the detection surface is reduced. On the other hand, when the fingerprint image captured in the frame (f + 1) is the fingerprint image 456 shown in FIG. 11C, the user presses the finger in the direction in which the detection surface of the fingerprint sensor 80 is pressed (Z axis (+) direction). This corresponds to the case of movement. At this time, since the finger is moved in a direction in which the pressure of pressing the detection surface increases, the area where the finger contacts the detection surface increases. Therefore, by associating the movement amount Δz of the finger in the Z-axis direction with the increase / decrease in the area of the fingerprint image, it is possible to detect the movement direction and the movement amount of the finger in the Z-axis direction.

  By using the fingerprint sensor 80 having the configuration shown in FIG. 5, the area of the captured fingerprint image can be obtained.

  FIG. 12 is a diagram showing a technique for reducing the load of the fingerprint image area calculation process. Assume that the fingerprint image 458 is captured by scanning in the X-axis and Y-axis directions within the fingerprint image detection region 450 by the fingerprint sensor 80 shown in FIG. At that time, in the X-axis direction, an output in which the peak or valley of the fingerprint is detected by the output line O1 at which detection of the peak or valley of the fingerprint is started and the output line O2 at which the peak or valley of the fingerprint is not detected is detected. The number of lines Oc can be specified. For the Y-axis direction, similarly, the number Dc of detected power supply lines is specified by the power supply line D1 at which the detection of the peak or valley of the fingerprint is started and the power supply line D2 at which the peak or valley of the fingerprint is not detected. can do. Therefore, a value Sr equivalent to the area of the fingerprint image 458 can be obtained from the number Oc of output lines and the number Dc of power supply lines.

Oc = O2-O1 + 1 (16)
Dc = D2-D1 + 1 (17)
Sr = Oc · Dc (18)
Thus, by specifying the power supply line and the output line of the fingerprint sensor 80, a rectangle circumscribing the outline of the captured fingerprint image can be easily obtained, thereby reducing the processing load for calculating the area of the fingerprint image. be able to.

In step S403, the barycentric position of the fingerprint image 458 can be specified based on the intermediate position of the output lines O1 to O2 and the intermediate position of the power supply lines D1 to D2. By doing so, the load of the calculation process of the center of gravity position of the fingerprint image 458 can be greatly reduced. Here, if the coordinates of the center of gravity position are (X _M , Y _M ), they can be obtained as in the following equation.

X _M = (O1 + O2) / 2 (19)
Y _M = (D1 + D2) / 2 (20)
Returning to FIG. 10, the description of the processing flow will be continued. When the movement amount Δz in the Z-axis direction is obtained in step S406, the feature point of the fingerprint image is subsequently extracted (step S407). Here, the feature point means an end point or a branch point of a mountain (ridgeline portion) of a fingerprint. Then, the distribution of the feature points is obtained. Thereby, the movement of the fingerprint image on the detection surface can be detected by the movement of the feature points (or the distribution of the feature points).

  Next, the feature point of the fingerprint image obtained in step S407 is compared with the feature point of the fingerprint image of the previous frame (step S408). Then, the fingerprint image feature points of both frames or their distributions are collated, and the movement amount ΔX2 in the X-axis direction, the movement amount ΔY2 in the Y-axis direction, and the rotation about the axis in the Z-axis direction so that the similarity is maximized. Find the angle β.

FIGS. 13A and 13B schematically show an example of feature point matching processing. 13A and 13B, the feature points P _f , Q _f , and R _f extracted from the fingerprint image in the frame f are the feature points P _{f + 1} and Q _{f + 1 of} the fingerprint image captured in the frame (f + 1). , R _{f + 1} . In the collation process, as shown in FIG. 13A, the CPU 72 at least three feature points P _f , Q _f , and R _f among the three or more extracted feature points respectively correspond to feature points P _{f + 1.} , Q _{f + 1} , R _{f + 1 so as} to match the movement amounts ΔX2, ΔY2 in the X-axis direction and the Y-axis direction. Further, in the collation process, as shown in FIG. 13B, the CPU 72 corresponds to feature points corresponding to at least three feature points P _f , Q _f , and R _f among the three or more extracted feature points. The reference point e and the rotation angle β around the axis in the Z-axis direction around the reference point e are obtained so as to coincide with P _{f + 1} , Q _{f + 1} , and R _{f + 1} .

  14A to 14C show examples of changes in the fingerprint image when the finger is moved around the axis in the Z-axis direction. Within the fingerprint image detection area 450, the fingerprint image 462 shown in FIG. 14A is a finger in the (−) direction around the Z-axis direction with reference to the fingerprint image 460 in the frame f shown in FIG. This corresponds to the case of moving. A fingerprint image 464 shown in FIG. 14C corresponds to the case where the finger is moved in the (+) direction around the axis in the Z-axis direction.

  Therefore, for example, by performing collation processing as shown in FIG. 13B, the change in the fingerprint image shown in FIGS. 14A to 14C can be obtained as the rotation angle β around the axis in the Z-axis direction. it can.

  Returning to FIG. When the movement vector of the fingerprint image (more specifically, the movement direction and movement amount of the feature point of the fingerprint image in the X-axis direction and the Y-axis direction) is obtained in step S408, the CPU 72 performs the X-axis direction according to the equation (4). The rotation angle α around the axis of Y and the rotation angle γ around the axis in the Y-axis direction are obtained according to the equation (3). Subsequently, the CPU 72 obtains a movement amount Δx in the X-axis direction according to the equation (1) and a movement amount Δy in the Y-axis direction according to the equation (2) (step S409). Next, when ending (step S410: Y), the series of processing ends (end), but when not ending (step S410: N), the process returns to step S401.

  FIGS. 15A to 15C show an example of changes in the fingerprint image when the finger is rotated and moved around the axis in the Y-axis direction. Here, an example is shown in which the user rotates his / her finger around the axis in the Y-axis direction while sliding the detection surface of the fingerprint sensor 80. Therefore, in FIGS. 15A to 15C, the gravity center position of the fingerprint image hardly changes.

  Within the fingerprint image detection area 450, the fingerprint image 472 shown in FIG. 15A is pointed in the (−) direction around the axis in the Y-axis direction with reference to the fingerprint image 470 in the frame f shown in FIG. This corresponds to the case where is rotated. That is, the position of the center of gravity of the fingerprint image hardly changes, and the feature point of the fingerprint image moves in the (+) direction of the X axis. A fingerprint image 474 shown in FIG. 15C corresponds to the case where the finger is rotated in the (+) direction around the axis in the Y-axis direction. That is, the position of the center of gravity of the fingerprint image hardly changes, and the feature point of the fingerprint image moves in the (−) direction of the X axis. In this case, the rotation angle γ corresponding to the moving amount of the feature point of the fingerprint image shown in FIGS. 15A to 15C is obtained by the above-described equation (4).

  FIGS. 16A to 16C show an example of changes in the fingerprint image when the finger is rotated and moved around the axis in the X-axis direction. Here, an example is shown in which the user rotates the finger around the axis in the X-axis direction while sliding the detection surface of the fingerprint sensor 80. Therefore, in FIGS. 16A to 16C, the gravity center position of the fingerprint image hardly changes.

  Within the fingerprint image detection area 450, the fingerprint image 482 shown in FIG. 16A is a finger in the (−) direction around the axis in the X-axis direction with reference to the fingerprint image 480 in the frame f shown in FIG. The fingerprint image 484 shown in FIG. 16C corresponds to the case where the finger is rotated in the (+) direction around the axis in the X-axis direction. That is, the position of the center of gravity of the fingerprint image hardly changes, and the feature point of the fingerprint image moves in the X-axis direction. In this case, the rotation angle α corresponding to the movement amount of the feature point of the fingerprint image shown in FIGS. 16A to 16C is obtained by the above-described equation (3).

  FIGS. 17A to 17C show examples of changes in the fingerprint image when the finger is moved in the X-axis direction. Here, an example is shown in which the user slides the finger on the detection surface of the fingerprint sensor 80 and moves the finger in the X-axis direction. Accordingly, in FIGS. 17A to 17C, the movement vector of the center of gravity of the fingerprint image and the movement vector of the feature point of the fingerprint image substantially coincide.

  In the fingerprint image detection area 450, the fingerprint image 492 shown in FIG. 17A is moved in the (−) direction of the X axis with reference to the fingerprint image 490 in the frame f shown in FIG. 17B. The fingerprint image 494 shown in FIG. 17C corresponds to the case where the finger is moved in the (+) direction of the X axis direction. In this case, the movement amount Δx in the X-axis direction substantially coincides with the movement amount ΔX2 of the feature point of the fingerprint image. In practice, however, the rotation angle γ around the axis in the Y-axis direction is taken into account as shown in equation (1). The calculated value will be obtained.

  FIGS. 18A to 18C show an example of changes in the fingerprint image when the finger is moved in the Y-axis direction. Here, an example is shown in which the user moves his / her finger in the Y-axis direction while sliding the detection surface of the fingerprint sensor 80. Accordingly, in FIGS. 18A to 18C, the movement vector of the center of gravity position of the fingerprint image and the movement vector of the feature point of the fingerprint image substantially coincide.

  In the fingerprint image detection area 450, the fingerprint image 502 shown in FIG. 18A is moved in the (−) direction in the Y-axis direction with reference to the fingerprint image 500 in the frame f shown in FIG. 18B. The fingerprint image 504 shown in FIG. 18C corresponds to the case where the finger is moved in the (+) direction of the Y-axis direction. In this case, the movement amount Δy in the Y-axis direction substantially coincides with the movement amount ΔY2 of the feature point of the fingerprint image, but actually, the rotation angle α about the axis in the X-axis direction is taken into account as shown in the equation (2). The calculated value will be obtained.

  As described above, according to the input device of the present embodiment, control information in six-axis directions can be generated using a captured image of a detected object such as a fingerprint. Accordingly, it is possible to provide an input device capable of giving control instructions (for example, movement instructions) in six axis directions. In particular, by using the above-described capacitance type fingerprint sensor, it is possible to provide an input device that is highly sensitive, ultra-compact, ultra-light, and has low power consumption and capable of instructing control in six axes.

(Modification)
In the above-described embodiment, the movement amounts Δx and Δy of the detection surface of the object to be detected are detected using the movements (ΔX2, ΔY2) of the feature points of the fingerprint image. However, the present invention is not limited to this.

For example, using the X-coordinate (X _G ) and Y-coordinate (Y _G ) of the centroid position obtained by using the equations (13) to (15) as the centroid position of the fingerprint image, detection is performed at least one frame before. difference (ΔX _G, ΔY _G) between the center of gravity position of the object image may be obtained. In this case, the movement amounts Δx and Δy of the detection surface of the detection object in the X-axis and Y-axis directions can be expressed by the following equations instead of the equations (1) and (2).

Δx = ΔX _G −a · γ (21)
Δy = ΔY _G −b · α (22)
Further, for example, the X-coordinate (X _G ) and Y-coordinate (Y _G ) of the centroid position obtained by using the equations (19) and (20) are used as the centroid position of the fingerprint image. A difference (ΔX _G , ΔY _G ) from the position of the center of gravity of the image of the detected object may be obtained. In this case, the movement amounts Δx and Δy of the detection surface of the detection object in the X-axis and Y-axis directions can be expressed by the following equations instead of the equations (1) and (2).

Δx = ΔX _M −a · γ (23)
Δy = ΔY _M −b · α (24)
In any case, control information in the six-axis directions can be generated as in the present embodiment. In particular, by using the above-described capacitance-type fingerprint sensor, it is possible to provide an input device that is highly sensitive, ultra-compact, ultra-light, and has low power consumption and capable of instructing control in six axes. it can.

2. Information Device FIG. 19 shows an example of a configuration block diagram of an IC card to which the input device of this embodiment is applied. The IC card 600 includes an input device 610 using the above-described fingerprint sensor, an image generation unit (a processing unit that performs control processing of a predetermined control target in a broad sense) 620, and a display unit 630. The input device 610 is the input device described in FIG. 1 or FIG. The image generation unit 620 is realized by a CPU and a software program stored in a ROM or RAM. Display unit 630 is realized by an LCD panel and a driving circuit thereof.

  Here, the image generation unit 620 generates image data based on the control information output from the input device 610 (performs control processing in a broad sense). More specifically, the input device 610 generates image data of an image that changes in response to a movement instruction in six axis directions. The display unit 630 displays an image based on the image data generated by the image generation unit 620.

  In the IC card 600 having such a configuration, the user moves the pointer image displayed on the display unit 630 by moving the fingerprint image of the finger in the six-axis direction to the input device 600 and giving a movement instruction. The image of the part 630 can be scrolled.

  Here, although an IC card has been described as an example of the information device, the input device according to the present embodiment can be applied to a PDA, a mobile phone, a three-dimensional CAD device, a virtual reality experience device, an electronic musical instrument, or the like.

  The present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the gist of the present invention.

  For example, in this embodiment, an input device using a fingerprint sensor has been described. However, the present invention is not limited to this, and an image of a two-dimensional or three-dimensional object other than a fingerprint is captured and control information is output in the same manner. Can do. The present invention can also be applied to an input device that does not have a detection surface.

  In this embodiment, the control information in the 6-axis direction is output in correspondence with the difference from the previous frame (at least one frame before the previous frame), but the present invention is not limited to this. For example, the difference may be integrated for each frame from the reference origin position, and an absolute value may be output as control information in the six-axis directions.

  In the invention according to the dependent claims of the present invention, a part of the constituent features of the dependent claims can be omitted. Moreover, the principal part of the invention according to one independent claim of the present invention can be made dependent on another independent claim.

1 is a configuration block diagram of an input device in the present embodiment. Explanatory drawing of the control information of 6 axial directions. The external appearance block diagram which shows the outline | summary of the input device using a fingerprint sensor. The block diagram which shows the hardware structural example of an input device. The circuit block diagram which shows an example of a fingerprint sensor. Sectional drawing of an electrostatic capacitance detection element. The equivalent circuit diagram of an electrostatic capacitance detection element in case a pile of fingerprints contacts. The equivalent circuit schematic of an electrostatic capacitance detection element in case the valley of a fingerprint opposes. 9A and 9B are explanatory diagrams showing an example of feature points of a fingerprint. The flowchart which shows an example of the processing flow of an input device. 11A to 11C are explanatory diagrams illustrating examples of changes in the fingerprint image when the finger is moved in the Z-axis direction. Explanatory drawing of the method of reducing the load of the calculation process of the area of a fingerprint image. FIGS. 13A and 13B are explanatory diagrams of an example of a feature point matching process. 14A to 14C are explanatory diagrams illustrating examples of changes in the fingerprint image when the finger is moved around the axis in the Z-axis direction. FIGS. 15A to 15C are explanatory diagrams illustrating examples of changes in the fingerprint image when the finger is rotated and moved around the axis in the Y-axis direction. FIGS. 16A to 16C are diagrams illustrating examples of changes in the fingerprint image when the finger is rotated and moved around the axis in the X-axis direction. FIGS. 17A to 17C are explanatory diagrams illustrating examples of changes in the fingerprint image when a finger is moved in the X-axis direction. 18A to 18C are explanatory diagrams illustrating examples of changes in the fingerprint image when the finger is moved in the Y-axis direction. The block diagram which shows the structural example of an IC card.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Input device, 20 Image acquisition part, 22 Detection surface, 30 Image analysis part, 32 Feature point extraction part, 34 Gravity center position calculation part, 36 Area calculation part,
40 difference calculation unit, 42 feature point movement detection unit, 44 center of gravity position movement detection unit, 46 area change detection unit, 48 memory, 50 control information output unit,
52 X-axis movement detector, 54 Y-axis movement detector,
56 X-axis rotation detection unit, 58 Y-axis rotation detection unit,
60 Z-axis rotation detection unit, 62 Z-axis movement detection unit

Claims (11)

An input device that generates control information by moving an object to be detected,
An image capturing unit that captures an image of an object to be detected using a fingerprint sensor;
An area calculating unit for obtaining an area of an image in a predetermined frame of the detected object captured by the image capturing unit;
A storage unit for storing the area of the image of the detection object obtained by the area calculation unit in units of frames;
Differential calculation for obtaining a difference between the area of the image of the detected object stored in the storage unit at least one frame before the predetermined frame and the area in the predetermined frame obtained by the area calculation unit And
A control information output unit for outputting control information corresponding to the difference;
Including
The detected object is a fingerprint,
The fingerprint sensor has a plurality of power lines, a plurality of output lines, and a plurality of capacitance detection elements provided at intersections of the plurality of power lines and the plurality of outputs,
Each of the plurality of electrostatic capacitance detection elements has a variable capacitance that changes depending on the concave / convex pattern of the fingerprint, and a signal amplification element,
The area calculation unit is captured obtains the area by the number of the number and power lines of the output line specified by the uneven pattern of the fingerprints, the input device comprising that you store the area in the storage unit. In claim 1,
The image capturing unit has a detection surface, captures an image of an object to be detected that is in contact with the detection surface,
The control information output unit
An input device that outputs control information corresponding to a movement amount of the detected object in an axial direction perpendicular to the detection surface. In claim 1,
The image capturing unit has a detection surface, captures an image of a detection object in contact with the detection surface,
The input device is
A center-of-gravity position calculating unit for obtaining a center-of-gravity position of an image captured by the image capturing unit;
A feature point extraction unit for extracting feature points of the image;
A center-of-gravity position movement detection unit for obtaining a first difference between the center-of-gravity position of the image of the detected object at least one frame before and the center-of-gravity position obtained by the center-of-gravity position calculation unit;
A feature point position movement detection unit for obtaining a second difference between the position of the feature point of the image of the detected object at least one frame before and the position of the feature point extracted by the feature point extraction unit;
Based on the first and second differences, a rotation angle detection unit that detects a rotation angle around the first or second axis orthogonal to each other on the detection surface;
Including
The rotation angle detector
Subtracting the first axial component of the second difference from the first axial component of the first difference to obtain a rotation angle about the second axis, and the first difference of the first difference Subtracting the second axial component of the second difference from a second axial component to determine a rotation angle about the first axis;
An input device that outputs control information corresponding to a rotation angle around the first or second axis. In claim 3,
The rotation angle detector
The angle at which the position of the feature point of the image of the detected object at least one frame before is rotated to the position of the feature point extracted by the feature point extraction unit around the reference point is perpendicular to the detection surface. Obtained as a rotation angle around the axis of 3,
An input device that outputs control information corresponding to a rotation angle around the third axis. In claim 3 or 4,
Corresponds to movement in the first axial direction by subtracting a value weighted by a given first coefficient from the first axial component of the second difference to the rotation angle around the second axis Output control information,
Corresponding to movement in the second axial direction by subtracting a value weighted by a given second coefficient from the second axial component of the second difference to the rotation angle around the first axis An input device that outputs the control information. In claim 3 or 4,
Corresponding to movement in the first axial direction by subtracting a value weighted by a given first coefficient from the first axial component of the first difference to the rotation angle around the second axis Output control information,
Corresponding to the movement in the second axial direction by subtracting a value weighted by a given second coefficient from the rotation angle around the first axis from the second axial component of the first difference An input device that outputs the control information. An input device according to any one of claims 1 to 6,
A processing unit that performs control processing based on control information from the input device;
An information device comprising: A control information generation method for generating control information using an image of a detected object captured using a fingerprint sensor,
The detected object is a fingerprint,
The fingerprint sensor has a plurality of power lines, a plurality of output lines, and a plurality of capacitance detection elements provided at intersections of the plurality of power lines and the plurality of outputs,
Each of the plurality of electrostatic capacitance detection elements has a variable capacitance that changes depending on the concave / convex pattern of the fingerprint, and a signal amplification element,
The control information generation method includes:
The area of the image of the detected object in contact with the detection surface is obtained by the number of output lines and the number of power supply lines specified by the captured fingerprint uneven pattern, and the area is stored in units of frames.
Obtaining the difference between the area of the image of the detected object in the predetermined frame and the area of the stored image of the detected object at least one frame before the predetermined frame ;
A control information generation method that outputs control information corresponding to the difference in accordance with movement of the detected object in a direction perpendicular to the detection surface. In claim 8,
Obtaining the position of the center of gravity of the image of the detected object in contact with the detection surface and extracting the feature point of the image;
The difference between the center of gravity position and the center of gravity of the image of the detected object at least one frame before is obtained, and the position of the feature point and the position of the feature point of the image of the detected object at least one frame before Find the difference between
Based on the difference between the center of gravity position and the difference between the feature points, a rotation angle around the first or second axis orthogonal to each other on the detection surface is detected,
A control information generation method characterized by outputting control information corresponding to the rotation angle. In claim 9,
The position of the feature point of the image of the object to be detected at least one before is rotated to the position of the extracted feature point around the reference point, and the first and second axes of the detection surface A control information generation method characterized in that a rotation angle about a third axis orthogonal to each other is obtained. In claim 9 or 10,
Control information corresponding to movement in the first or second axial direction is output using the difference between the feature points and the rotation angle around the first or second axis. Generation method.

Images