JPH0789361B2 - Form registration device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document processing system, and more particularly to a form registration system suitable for registering a form in advance for the purpose of reading characters on a tabular document.

[Background of the Invention]

In the conventional character reading device (hereinafter abbreviated as OCR),
Since it is necessary to print the reading area in a color that OCR does not feel (dropout color), the printing cost becomes high. Further, it is necessary to describe the position of the character reading area as the distance from the edge of the document, and to specify the number of characters in that area, the character type, the check method, etc., which is complicated. Hereinafter, the position information of the character reading area, the number of characters in the area, the character type, the check method, and the like are referred to as format information. The character recognition function is required not only for dedicated OCR, but also for document files and OA workstations, but the above problems hindered its widespread use.

In the related art, for example, Japanese Patent Laid-Open Publication No. Showa 58-20718
(Published December 2, 1983), a method is disclosed in which a document other than drop-out color can be used by removing a fixed image stored in advance in a memory from an input image. ing. However, this method is similar to the conventional one in that it is necessary to specify the format information for the remaining images from which the standard images have been removed.

[Object of the Invention]

An object of the present invention is to automatically understand the structure of an input document and automatically generate a document format, thereby significantly expanding the range of a target document and simplifying the labor for creating format information. To resist.

Another object of the present invention is to provide means for enabling continuous reading of standard documents of different types by automatically identifying the type of a standard input document and using prestored format information. Especially.

[Outline of Invention]

In order to achieve such an object, in the present invention, a frame portion and a relationship between the frame portions are extracted from the image of the identification target document (form), and further, an image inside the frame and selection information for the property of the frame (hereinafter referred to as “selection information”). (Referred to as “menu”), the character of the frame is entered, and the information of the frame itself is understood from the property of the frame selected by the user to understand the structure of the form. Is to make a decision.

That is, the frame has a data frame in which data is scheduled to be entered and an item frame in which data is not normally entered,
If the distinction between the item frame and the data frame and the attribute of the item frame are known,
Use the fact that you can understand the structure of the form. Since the item name is usually printed (or handwritten) in the item frame, the item name is displayed and a person gives the type / attribute to automatically understand the structure of the form.

Example of Invention

 Hereinafter, the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of an apparatus for implementing the document understanding system of the present invention. Each unit of the device is connected to the bus 1, and the entire operation is controlled by the control unit 2. Information (document image) on the document 3 is scanned by the photoelectric conversion device 4,
Further, it is digitized and stored in the memory 51 via the bus 1. The memory 51 forms a part of the memory 5 together with 52, 53 and 54 described later. A well-known high-efficiency encoding process may be performed at the time of digitalization, whereby the storage capacity of the memory for storing the document image can be saved. Further, in the following description, it is assumed that one pixel is binarized into one bit, but one pixel may be represented by multiple values, or color information may be provided by photoelectric conversion by a color scanner.

Document processing has a registration mode and an identification mode. Mode selection is performed by man-machine interaction with the control unit 2 using the keyboard 6 and the display 7. First, the case of the registration mode will be described. A flowchart of the registration mode processing is shown in FIG. This processing is executed by the software of the control unit 2. In FIG. 2, 201 is an input of a document image to be identified, and the image on this document is stored in the memory 51. 202
Then, the input document image is normalized such as position correction processing and tilt correction processing, and the resulting image is stored in the memory.
Stored in 52. Line extraction processing is performed at 203 on this normalized image, and the extracted line pattern is stored in the memory 53. FIG. 3 is a diagram for explaining the processing contents of line pattern extraction. A horizontal line pattern as shown in FIG. 3B is obtained by horizontal line extraction from an input image as shown in FIG. A vertical line pattern such as (C) is obtained by extracting the direction lines, and an image such as (D) is obtained by the logical sum of (B) and (C). Hereinafter, (D) is called a line pattern. In 204, the coordinate point sequence on the contour is extracted from the line pattern by contour extraction. This coordinate point sequence is obtained by the number of contours. In the following, only the inner contour is processed among the contours. Here, the inner contour is a boundary of a black background that surrounds a white background, and refers to a contour that goes around clockwise when the white background is viewed on the right and the black background is viewed on the left. In 205, a rectangle is extracted from this inner contour. Hereinafter, the inner contour forming a rectangle is called a frame. In 206, the order of the frames is rearranged from the upper left to the lower right using the coordinates of the four corners of the frame. In 207, when there are a plurality of extracted frames, the mutual relationship between the frames, that is, the relation code Cmn between the frames m and n is extracted. The following is an example of the definition of the relation code Cmn.

Cmn = 0: All of the following conditions are not satisfied = 1: m includes n = -1: n includes m = 2: m is directly above n = -2: m is directly below n = 3: m is n To the left =-3: m is to the right of n = 4: m is above n = -4: m is below n = 5: m is to the left of n = -5: m is to the right of n 208 Then, the image of the area corresponding to the extracted frame is taken out from the original image, and this image is sequentially sent to the display 7 to display the image in the frame. In 209, a menu regarding the nature of this frame is displayed on the display 7. At 210, the user's response to this menu is read from the keyboard. The characteristics of the frame include distinction between types (item frame or data frame) and attributes (blank, person's name, place name, date, item name, human relationship). Attribute menu display and response input are performed only for the item frame. Alternatively, it may be automatically determined that the inside of the frame is blank, and the blank frame may be automatically determined as the data frame.
At 211, the structure Am of the frame m, the relation code Cmn between the frames, and the like are compared with the knowledge about the structure of the document to understand the structure of the input document. Knowledge is expressed in "if-then-" form, that is, conditions and conclusions. An example of knowledge is shown below. However, a, b,
c indicates the frame number.

1. (Condition) Aa = 'Person name'& Box a is item frame (Conclusion) Item name of box a = 'Person name' 2. (Condition) Aa = 'Blank'& Ab = 'Person name'& Cab = -3 ( Conclusion) Frame a is a data frame that is subordinate to frame b Attribute of frame a = 'Person name' 3. (Condition) Aa = 'Blank'& Ab = 'Place name: Prefecture'& Cab = -4 Ac = 'Place name'& Cac = -3 (Conclusion) Frame a is a data frame belonging to frames b and c Attribute of frame a = 'Place name: prefecture' 4. (Condition) Aa = 'Date of year' (Conclusion) Frame a is independent data Frame Attribute of frame a = 'Time: date' 5. (Condition) Aa = 'Blank' (Conclusion) Frame a is an independent data frame Item name of frame a = 'Miscellaneous' Attribute of frame a = 'Miscellaneous' That is, the attribute of the data frame is automatically determined by the item frame to which it belongs.

The concrete realization method of the above knowledge is "if ~ then
It is a subroutine written as a "~" statement, and the collation with the knowledge can be performed by substituting m and n for a and b and giving the result of checking whether the condition is satisfied to the outside as an argument. When the condition is satisfied, the conclusion is also given to the outside as an argument. Knowledge If, for example, the frame numbers m and n that meet the conditions a and b of 1. are found, the frame number m is substituted for the frame number a of the conclusion and registered as the structural data of the input document. The structural data includes the frame relation code, frame type (item frame, data frame, or both), subordinate relationship, item name, attribute code, and the like. If the knowledge corresponds to a plurality of pieces of knowledge, it may be determined that the knowledge having the earliest appearance order has priority. By determining the remedy (5. in the above case) when it does not apply to other knowledge as the lowest priority knowledge, the structure for all the frames is determined. Once all the boxes have been matched against the knowledge and the structure has been determined,
Registration of structural data is completed. At 212, the mask data is determined. The mask data indicates the position of a character (corresponding to a unit name such as the date, etc. in 4. above) that is printed in the data frame. To determine the mask data, for example, a connected area of black pixels in the frame may be extracted and a circumscribed rectangle of a sufficiently large connected area may be obtained. twenty one
At 3, the structure data, the physical data of the frame (coordinates of the four corners of the frame, etc.) and the mask data are written to the output device 9. Structural data, frame physical data, and mask data correspond to the conventional OCR format data. At the time of writing, the identification number and file name for the input document are input using the keyboard 6 and the display 7.

This is the end of the description of the registration mode. This registration mode processing is performed in advance to store format information such as structural data, but it is also possible to perform it in combination with the identification mode processing described below.

Next, processing of the input target document in the identification mode will be described. FIG. 4 shows a flow chart of processing in the identification mode. 40
1 is the input document image capture, 402 is normalization, 403 is line pattern extraction, 404 is contour extraction, 405 is rectangle extraction, 40
6 is frame ordering, 407 is relation code extraction, and
Same as 202-207. In 408, the structural data and physical data of the sample document are sequentially read into the memory 53 and compared with the structural data and physical data of the input document in the memory 54 to determine which sample document the input document matches. The input document is read using the structural data and physical data of the sample document as follows. 409 is a loop for repeatedly executing the following 410 to 414 for the frame. 410 is a judgment to read only the data frame. At 411, only the area inside this data frame is extracted. In 412, the characters printed in advance by the mask data are deleted (only when the mask data exists). In 413, a character type determined by the attribute code is selected and a recognition dictionary (standard pattern for character recognition) is set. At 414, the area inside the frame is sent to the recognition unit 8 to recognize the character inside the frame.
At 415, select the word dictionary determined by the attribute code,
The word matching is performed to detect or correct misreading or nonreading of the character recognition result. At 417, a necessary part of the character recognition result and the structure data is edited and written to the output device 9. As an example of editing, if the attribute of the item frame to which the data frame belongs is “personal name” and “corresponds to 2. of the above knowledge” and the recognition result of this frame is “Taro Yamada”, then “personal name” = Write it out as one record like "Taro Yamada".
Independent item / data frame with item name = 'Time: year / month / day', unit name = 'year / month / day' (corresponding to knowledge 4.) If the output is 58529, the edited output will be "Date" = "May 29, 58". In the latter example, the position coordinate of the character is taken out from the recognition unit 8, and the position is determined by the comparison with the mask data and the coordinate to edit.

The embodiment of the present invention has been described above. It is easy to add processing performed by a normal character recognition device or a graphic processing device to this embodiment, for example, display / correction of a frame extraction result, display / correction of a character recognition result, or the like. Also,
Multiple thresholds for line extraction and rectangle determination are provided for frame recognition, and when the result of automatic determination is difficult, the optimum value is selected in a man-machine manner or the binarization level is changed and retried is added. May be. Further, when the type of document is determined in advance in the identification mode, the number of the sample document may be given from the keyboard so that the collation of structural data and the like may be omitted.

Although the line pattern is extracted from the document and the frame is extracted from the document in the present embodiment, it is also possible to directly extract the contour from the original document pattern and then extract the frame. In this case, the frame is extracted. Contours other than can be used later for other processing such as character cutting and character recognition.

Further, in the present embodiment, the description of the document structure is limited to the frame format, but it is also possible to extend it to use ruled lines such as a solid line or a broken line and circles. For example, a ruled line composed of a solid line can be detected by extracting a slender outer contour, and an underlined title character can be recognized by cutting out an area above the ruled line. The circle can be detected by displaying the outline coordinate series in polar coordinates and detecting a straight line in this space.

Also, the characters outside the frame can be read and used for the structure description. It is also effective to add color attributes to the structure description using color information.

Further, the present invention is not limited to the case where the photoelectric conversion is directly performed on the document, and the invention can be applied to the document image read from the image file.

〔The invention's effect〕

As described above, according to the present invention, it is possible to automatically understand the structure of the input document with a simple structure and automatically generate the document format. As in the conventional art, the work of creating the format information by humans is performed. Is unnecessary, and the tabular document printed in black can be read, so there is no need to reprint in dropout color, and the preparation work can be simplified.
The range of target documents can be greatly expanded.

Further, according to the present invention, it is possible to automatically identify the type of a standard input document and use prestored format information,
It enables continuous reading of different types of standard documents.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an apparatus for implementing the document processing system of the present invention, FIGS. 2 and 4 are flow charts for explaining the processing in the control unit of FIG. 1, and FIG. It is a figure explaining a processing content. 1 ... bus, 2 ... control unit, 3 ... document, 5 ... memory, 6 ... keyboard, 7 ... display, 8 ... character recognition unit, 9 ... output device.

 ─────────────────────────────────────────────────── ─── Continuation of front page (72) Masao Dono Inventor Masao Dono 2880, Kozu, Odawara City, Kanagawa Prefecture Odawara Plant, Hitachi Ltd. (56) Reference JP 61-59568 (JP, A) JP 61 -54569 (JP, A) JP-A-56-137480 (JP, A) 7th. International Conference on Pattern Recognition, Proceedings Vol. 1, P.I. 347-349, July 30-August2 1984, "HIERARCHICAL REPRE SENTATION OF OPTICA LLY SCANNED DOCUMEN TS"

Claims (1)

[Claims] 1. A means for holding inference information for determining the table structure of a tabular document, a means for inputting a document image, a means for extracting a frame in the document image, and a means for mutually extracting the extracted frames. Means for detecting positional relationship information of the frame, means for extracting an image of an area corresponding to the frame from the document image, means for displaying an image in a frame included in the area corresponding to the extracted frame, A means for displaying a plurality of attribute information for a frame as a selection item and selecting one attribute information from the plurality of displayed attribute information, the attribute information selected from the selection item, and the positional relationship information between the frames. And a means for applying the above inference information to determine table structure data of a tabular document, and a means for registering the table structure data.