JP5807464B2 - Image identification apparatus and image identification method - Google Patents

Description

  The present invention relates to an image identification apparatus and an image identification method for determining whether a data image to be identified is of the same type or a different type from a master image that is a reference. It relates to the field of form type identification.

  In a standard form data processing system, normally, when performing automatic reading (OCR) of characters entered on a form, form definition data such as entry position information of each character is also required. Therefore, when a plurality of types of standard forms are targeted, since the form definition data differs for each form type, it is necessary to identify the form type of the input image prior to OCR.

  In the past, it was common to print a unique mark or symbol for each form type on the form and identify the form type by recognizing its presence, but a dedicated form design is required. The problem is that it takes time and cost to switch forms when systematizing existing business, and it cannot be identified if noise, collapse, or blur occurs on marks or symbols.

  In order to solve this problem, for example, Patent Document 1 discloses a method for identifying a form type based on collation of ruled line information. This method does not require any special marks or symbols, and comprehensively uses the information of the entire ruled line, so it can be identified even if there is noise, collapse, or blur in part of the input image, but there is no ruled line It cannot be applied to forms.

  Therefore, in the method disclosed in Patent Document 2, a plurality of corresponding points including not only ruled lines but also pre-printed characters are found between the input image (data image) and the reference image (master image), and corresponding points are obtained. A coordinate conversion is obtained, and form type identification is performed by comparing the degree of difference calculated based on the conversion coefficient or the coefficient with a threshold value.

  However, with the method described above, it is difficult to correctly identify similar form types with small differences in design and layout. Among the geometrical displacement and distortion effects brought about by coordinate transformation, there are essential factors due to “form design / layout differences” that occur between different form types, and “on-form manuscripts” that occur regardless of the form type. Fluctuation factors due to text entry / stamping / dirt ”and“ noise, crushing, blurring, and tilt on image data ”are mixed. Usually, the influence of essential factors is much greater, so that a sufficiently correct identification can be made.

  However, since the effect of the essential factor is small between similar form types and is often the same as the fluctuation factor, the possibility of erroneous identification as a result increases.

  The present invention has been made in view of the above problems, and the object of the present invention is not to require a dedicated form, can be applied to a form without a ruled line, and can correctly identify a similar form. An object of the present invention is to provide an image identification device and an image identification method.

  In order to achieve this object, the present invention has the following features.

Image identification apparatus according to the present invention, the input and the master image input means for inputting a plurality of master images, a data image input means for inputting data image, a set of mask area designated for the master image A mask area input means, an image matching means for matching the master image and the data image, a difference extraction means for extracting the difference excluding the mask area between the matched master image and the data image, and extracted A difference output unit that outputs a difference between the master image and the data image based on the magnitude of the difference, and the difference extraction means assigns the foreground attribute to the pixels in the mask area on the master image and Master image subtraction processing to extract the difference obtained by subtracting the master image from the image, and data from the master image by adding background attributes to the pixels in the mask area on the master image A data image subtraction process for extracting a difference obtained by subtracting the image, and the difference output unit from the difference extracted by the master image subtraction process and the difference extracted by the data image subtraction process. The second difference degree is output, the first difference degree and the second difference degree are added, and the master image having the smallest addition result among the plurality of master images is identified as the same type as the data image. , characterized in that.

  According to the present invention, it is possible to correctly identify similar form types for a wide variety of standard forms including forms with no special marks, symbols, and ruled lines.

1 is a configuration diagram of an image identification device according to a first embodiment of the present invention. It is a figure which shows the flowchart by the 1st Embodiment of this invention. It is a figure which shows the foreground mask process of this invention. It is a figure which shows the alignment deformation | transformation of this invention. It is a figure which shows the thickening process of this invention. It is a figure which shows the subtraction process of this invention. It is a figure which shows the master image subtraction result of this invention. It is a figure which shows the flowchart by the 2nd Embodiment of this invention. It is a figure which shows the data image subtraction result of this invention. It is a figure which shows the margin process by the 3rd Embodiment of this invention. It is a block diagram which shows the computer by the 4th Embodiment of this invention.

(First embodiment)
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a configuration diagram of an image identification device according to a first embodiment of the present invention.

  The image identification device includes a master image input unit 101 for inputting a master image, a data image input unit 102 for inputting a data image, an image matching unit 103 for aligning the master image and the data image, and designation for the master image. A mask area input unit 104 that inputs a set of mask areas to be performed, a difference extraction unit 105 that extracts a difference excluding the mask area between the matched master image and the data image, and a size of the extracted difference And a difference output unit 106 that outputs a difference between the master image and the data image.

  The image identification apparatus also includes a master image buffer 111 for temporarily storing data, a data image buffer 112, an affine transformation coefficient buffer 113, a mask area buffer 114, a temporary image buffer A115, and a temporary image buffer B116.

  FIG. 2 is a diagram showing a flowchart according to the first embodiment of the present invention. The processing operation of the present invention will be described below with reference to the flowchart of FIG.

  First, a master image of a certain blank form is input to the master image input unit 101 configured by a scanner or the like (step S201), and is stored in the master image buffer 111 as black and white binary digital image data. Similarly, a data image corresponding to some completed form is input to the data image input unit 102 configured by a scanner or the like (step S202), and is stored in the data image buffer 112 as monochrome binary digital image data.

  A set of mask areas, which are rectangular areas in which characters can be entered on the master image, is input to the mask area input unit 104 as a data string composed of the start point coordinates and end point coordinates of each rectangle (step S203), and the mask area buffer. 114 is stored.

  Subsequently, the image matching unit 103 obtains affine transformation for matching the master image and the data image on the buffer with each other (step S204). In this embodiment, only the affine transformation that maps the data image plane to the master image plane is obtained, and the inverse transformation is not obtained. A known method is used for the process for obtaining the affine transformation, and the description is omitted here. If an appropriate affine transformation cannot be obtained because the difference between the master image and the data image is too large (step S204, No), the data image is rejected at this point (step S205). .

  When an appropriate affine transformation is found (step S204, Yes), the transformation formula is as follows.

[Formula 1]
X = a * x + b * y + e
Y = c * x + d * y + f
(X, y): coordinates (X, Y) on the data image: coordinates a, b, c, d, e, f on the corresponding master image: affine transformation coefficients

  The coefficients a to f in the formula are stored in the affine transformation coefficient buffer 113.

  Subsequently, as shown in FIG. 3, the difference extraction unit 105 converts the image data in which all pixel values included in any of the mask areas are replaced with black as the foreground with respect to the input master image by the temporary image buffer. A115 is created (step S206). This is equivalent to replacing all pixel values in the mask area with 1 in the case of a binary image expressed by white pixels = 0 and black pixels = 1. Visually, the area where characters can be written on the master image is blacked out.

  Subsequently, image data in which the pixel values of the coordinates (X, Y) of the temporary image buffer A corresponding to Equation 1 are arranged from the data image coordinates (x, y) is created in the temporary image buffer B116. At this time, the coordinates (X, Y) obtained from Equation 1 are rounded off after the decimal point. If this is beyond the range of the master image, the pixel value is regarded as white.

  As shown in FIG. 4, the image data on the temporary image buffer B is image data having the same width and height as the data image, and the result of deforming the master image to match the data image based on the affine transformation. (Step S207).

  Subsequently, a thickening process is performed on the foreground mask + aligned master image on the temporary image buffer B. Here, as an example, as shown in FIG. 5, for each black pixel on the buffer, all 3 × 3 pixels centering on this pixel are made black (step S208). The thickening process is performed in order to absorb the matching error of the affine transformation in the subsequent subtraction process. The thickened result image data is overwritten in the temporary image buffer A by discarding the existing contents.

Subsequently, the master image subjected to the foreground mask + alignment deformation + thickening process is subtracted from the data image (step S209). Image subtraction means that the pixel value Cs of the coordinate (x, y) in the data image buffer is Cd, and the pixel value of the same coordinate in the temporary image buffer A is Cm.
Cs = white if Cd = black, Cm = black Cs = white if Cd = white, Cs = white if Cm = black Cs = black if Cd = black, Cm = white Cs = black if Cd = white, Cm = white = Calculation determined by white (FIG. 6).

  The result of subtraction is image data with the same width and height as the data image. Except for the pre-printed part of the form and the mask area where it is known that characters can be entered in advance, “ It is considered that the difference that exists only in the completed form remains as a black pixel. Since the black pixel portion on the master image side is slightly widened by the thickening process described above, a slight matching error caused by the estimation error of the affine transformation coefficient and the rounding off of the coordinate value is absorbed in the widened black pixel range. The subtraction result obtained in this way is overwritten in the temporary image buffer B by discarding the existing contents.

  Subsequently, the dissimilarity output unit 106 counts the number of black pixels on the temporary image buffer B (step S210), and in order to suppress the influence of the image size on the result, the value normalized by the area of the data image is changed. It outputs as a degree (step S211). That is,

[Formula 2]
S = N / (W * H)
S: degree of difference N: number of black pixels on temporary image buffer B W: width of data image H: height of data image

  In this way, the degree of difference S between the data image and one master image is obtained. If the data image is the same form type as the master image, the subtraction result image should be nearly white, so S is low. On the other hand, if the data image has a form type that is different in design and layout from the master image, a considerable number of black pixel blocks of a size that can be clearly confirmed visually remain in the subtraction result image even if they are similar forms. Since S should be done, S becomes high (FIG. 7).

  After obtaining the degree of difference for all master images by applying the above method, the image type (form) is determined by determining that the form type of the master image having the lowest degree of difference is the form type of the data image. Species identification) is realized.

(Second Embodiment)
Since the first half of the process is the same as that of the first embodiment, the description thereof is omitted. The difference from the first embodiment is that the difference extraction processing unit calculates a second difference T that is different from the difference S obtained in the first embodiment and uses it for image identification.

  FIG. 8 is a diagram showing a flowchart according to the second embodiment of the present invention. The processing operation of the present invention will be described below with reference to the flowchart of FIG.

  Here, first, the first difference S is calculated according to the first embodiment. Subsequently, the difference extraction unit discards the existing contents in the temporary image buffer A with the image data in which all the pixel values included in any of the mask areas are replaced with white as the background for the input master image. Overwriting (step S301). This is equivalent to replacing all pixel values in the mask area with 0 in the case of a binary image represented by white pixels = 0 and black pixels = 1. Visually, an area where characters can be written on the master image is painted white.

  Subsequently, as in the first embodiment, image data obtained by deforming the master image to match the data image based on the affine transformation is overwritten in the temporary image buffer B by discarding the existing contents (step S302). ).

  Subsequently, the same thickening process as in the first embodiment is performed on the data image in the data image buffer (step S303). The thickened result image data is overwritten in the temporary image buffer A by discarding the existing contents.

Subsequently, the data image subjected to the thickening process is subtracted from the master image subjected to the background mask + alignment deformation process (step S304). Similarly to the first embodiment, this subtraction is also the result of subtraction at the same coordinates when the pixel value at a certain coordinate (x, y) in the temporary image buffer B is Cm and the pixel value at the same coordinate in the temporary image buffer A is Cd. Pixel value Cs of
Cs = white if Cm = black, Cd = black Cs = white if Cm = white, Cd = black Cs = black if Cm = black, Cd = white Cs = black if Cm = white, Cd = white = Operation determined by white.

  The subtraction result is image data with the same width and height as the data image, and it is considered that the difference that “existed in the unfilled form but not in the filled form” remains as a black pixel. . Since the black pixel portion on the data image side is slightly widened by the above-described thickening process, a slight matching error due to the estimation error of the affine transformation coefficient and the rounding off of the coordinate value is absorbed in the widened black pixel range. The subtraction result obtained in this way is overwritten in the temporary image buffer B by discarding the existing contents.

  Subsequently, the difference output unit counts the number of black pixels on the temporary image buffer B (step S305). In order to suppress the influence of the image size on the result, the value normalized by the area of the data image is set to the second value. Is output as the difference degree (step S306). That is,

[Formula 3]
T = K / (W * H)
T: Second degree of difference K: Number of black pixels on temporary image buffer B W: Width of data image H: Height of data image

  In this way, the second difference T between the data image and one master image is obtained. If the data image is the same form type as that of the master image, the subtraction result image should be nearly white, so T is low. On the other hand, if the data image has a form type that is different in design and layout from the master image, a considerable number of black pixel blocks of a size that can be clearly confirmed visually remain in the subtraction result image even if they are similar forms. T should be high (FIG. 9).

  After obtaining the dissimilarities S and T by applying the above method for all master images, the form type of the master image having the lowest sum (S + T) is determined to be the form type of the data image. Thus, image identification (form type identification) is realized.

(Third embodiment)
When scanning a master image or data image document with a scanner, etc., large shadows appear near the edges of the image due to paper distortion, punch holes, how to place the document on the scanner table, and other factors. There is. If the document is a FAX reception document, the reception date and time and the sender may be printed near the top and bottom. Any of these can be fluctuation factors that disturb image identification based on the difference.

  The third embodiment is for eliminating such a fluctuation factor. Since most of the processing is the same as in the first or second embodiment, description thereof is omitted. The difference from the first or second embodiment is that, at the beginning of the difference extraction unit, a process of replacing a margin part having a predetermined width from the ends of the master image and the data image with a white pixel is executed.

  Specifically, it is performed as a subsequent process after the affine transformation of Expression 1 is obtained. In this embodiment, it is assumed that the margin width is 15 millimeters, and the horizontal resolution Hx (dpi) and the vertical resolution Hy (dpi) match between the data image and the master image. Then, the horizontal margin pixel width Mx and the vertical margin pixel width My obtained by converting millimeters into pixels are as follows.

[Formula 4]
Mx = Hx * 15 / 25.4 (rounded down)
My = Hy * 15 / 25.4 (rounded down)

  Then, for the image data on the master image buffer and the data image buffer, all pixel values included in the area outside the dotted line illustrated in FIG. 10 are replaced with white. Thereafter, the process follows the first or second embodiment.

(Fourth embodiment)
The present invention can also be executed by a program on a computer. FIG. 11 is a block diagram showing a computer according to the fourth embodiment of the present invention.

  A program related to the present invention recorded on a recording medium such as a CD-ROM 401 is loaded onto the memory 404 at the time of execution through the CD-ROM drive 402 shown in FIG. The processing steps of the program are sequentially executed according to the command. Digital image data corresponding to a master image and a data image is stored in advance on the hard disk 403, or taken in through a scanner (not shown) at the time of execution, and then loaded on the memory 404 for reference.

  The determination result of the inspection is stored in the memory 404, written to the hard disk 403 as necessary, output to the display 406, sent to the network via the communication device 407, or on paper via a printer (not shown). Is printed.

DESCRIPTION OF SYMBOLS 101 Master image input part 102 Data image input part 103 Image matching part 104 Mask area | region input part 105 Difference extraction part 106 Difference output part

JP 2003-109007 A Japanese Patent No. 3932201

Claims (8)

Master image input means for inputting a plurality of master images;
And a data image input means for inputting the data image,
Mask area input means for inputting a set of mask areas designated for the master image;
Image alignment means for aligning the master image and the data image;
A difference extracting means for extracting a difference excluding the mask region between the matched master image and the data image;
A difference output unit that outputs a difference between the master image and the data image based on the size of the extracted difference;
Equipped with a,
The difference extraction means includes a master image subtraction process for extracting a difference obtained by subtracting the master image from the data image by assigning a foreground attribute to pixels in the mask region on the master image, and the master image on the master image. A data image subtraction process for extracting a difference obtained by subtracting the data image from the master image by adding a background attribute to the pixels in the mask region, and
The difference output unit outputs a first difference from a difference extracted by the master image subtraction process and a second difference from a difference extracted by the data image subtraction process,
Adding the first dissimilarity and the second dissimilarity and identifying that the master image having the smallest addition result among the plurality of master images is the same type as the data image;
An image identification apparatus characterized by the above. In the image matching means, the affine transformation for mapping the data image plane to the master image plane, or the inverse transformation thereof , is obtained .
For the pixel on the image after the affine transformation or its inverse transformation, the pixel value of a pixel in a predetermined region centered on the pixel is changed according to the pixel value of the pixel.
The image identification apparatus according to claim 1. In the difference extraction means, margin processing is performed for adding a background attribute to pixels included in a margin of a width that is constant or dynamically determined from the edges of the master image and the data image. The image identification device according to claim 1 . Inputting a plurality of master images, data image, and a set of mask area specified with respect to the master image,
Aligning the master image and the data image;
A master image subtraction step for extracting a difference obtained by subtracting the master image from the data image by assigning a foreground attribute to pixels in the mask region on the master image;
A data image subtraction step for extracting a difference obtained by subtracting the data image from the master image by giving a background attribute to the pixels in the mask area on the master image;
Outputting a first difference from the difference extracted by the master image subtraction step and a second difference from the difference extracted by the data image subtraction step;
Adding the first dissimilarity and the second dissimilarity, and identifying the master image having the smallest addition result among the plurality of master images as being of the same type as the data image. A feature of the image identification method. The step of aligning the master image and the data image includes obtaining an affine transformation for mapping a data image plane to a master image plane, or an inverse transformation thereof .
For the pixel on the image after the affine transformation or its inverse transformation, changing a pixel value of a pixel in a predetermined region centered on the pixel according to the pixel value of the pixel;
The image identification method according to claim 4 , further comprising: The step of extracting the difference excluding the mask region between the matched master image and the data image is a foreground on the data image, a corresponding point on the master image is a background, and the point is 6. The image identification method according to claim 4 , further comprising a step of obtaining a set of points not included in any of the mask regions. The step of extracting the difference excluding the mask area between the matched master image and the data image is a background on the data image, and a corresponding point on the master image is a foreground, and The image identification method according to claim 4 , further comprising a step of obtaining a set of points whose points are not included in any of the mask regions. The step of extracting the difference excluding the mask area between the aligned master image and the data image has a width that is constant or dynamically determined from the edges of the master image and the data image. The image identification method according to claim 4 , wherein a difference is extracted except for a margin.

Images