JP4849883B2 - Row direction determination program, method and apparatus - Google Patents

Description

  The present invention relates to an OCR (Optical Character Recognition) technique, and more particularly to a technique for determining either vertical writing or horizontal writing.

  The OCR technique is a technique for recognizing a layout for text image data read by a scanner or the like and performing character recognition on a text area. In recent years, document management systems using OCR technology have attracted attention for storing, retrieving and reusing document image data such as forms. In the document management system, when character recognition is performed on a text area, a line is first obtained from the arrangement of characters. However, depending on the arrangement of characters in the text area, the line direction may be erroneously determined.

  Conventional techniques for determining the row direction include the following.

For example, Japanese Patent Laid-Open No. 8-263587 discloses a character string direction estimation technique using language information. Specifically, a histogram of an image projected in the vertical direction and the horizontal direction is generated, and a direction in which the interval between the images is narrow is detected from the histogram to determine the row direction. When the character spacing is almost the same in either direction, character recognition is performed in two directions, and the character string of the recognition result is decomposed into phrases using a word dictionary. The number of phrases at this time is compared in the vertical and horizontal directions, and the direction with the smaller number of phrases is output. For example,
In an image including bank transfer, in the horizontal direction, “bank” and “transfer” constitute a phrase, and the number of phrases is two. In the vertical direction, there are four “silver”, “shake”, “row”, and “include”, and the horizontal direction is selected as the row direction.

  Japanese Patent Application Laid-Open No. 08-63545 discloses the following technique. That is, the circumscribed line of the character area specified as the processing target is extracted, and the character direction is generated for each circumscribed line. Character recognition processing is performed for each circumscribed line and each character direction to generate a character lattice, and language processing is performed for each character lattice to determine the constituent word rate or the independent word content rate of each character lattice. The character direction and line direction of the specific area are determined based on the constituent word rate or the independent word content rate obtained for each character lattice.

  Further, Japanese Patent Laid-Open No. 07-220027 discloses the following technique. That is, the circumscribed line of the character area specified as the processing target is extracted, and the character direction is generated for each circumscribed line. Character recognition processing is performed for each circumscribed line and each character direction to generate a character lattice, and language processing is performed for each character lattice to obtain the number of words constituting the character lattice or the number of independent words whose notation length is 1. The character direction and line direction of the specific region are determined based on the number of constituent words or the number of independent words obtained for each character lattice.

  Furthermore, Japanese Patent Application Laid-Open No. 2000-20638 discloses a character string direction determination method capable of reliably determining vertical writing / horizontal writing. Specifically, an image reading process in which a document in which a character string is described is decomposed into pixels and read as image data, and image data read in the image reading process on the assumption that the character string of the document is vertically written From the first character string extraction process for extracting the first character string in the vertical direction from the character string and the character string extracted by the first character string extraction process, a plurality of characters constituting this character string are cut out and recognized. A first word recognition process, a first word search process for searching a word spelled by a plurality of characters recognized in the first character recognition process by referring to a word dictionary, and a character string of a document is written horizontally Assuming that there is a second character string extraction process for extracting a character string at the beginning of the horizontal direction from the image data read by the image reading process, and a character string extracted by the second character string extraction process Cut out multiple characters that make up this string A second character recognition process for recognizing, a second word search process for searching for a word spelled by a plurality of characters recognized in the second character recognition process with reference to a word dictionary, and the first and second And vertical / horizontal determination processing for determining whether the direction of the character string described in the document is vertical writing or horizontal writing based on the search result of the word search processing.

  Japanese Patent Laid-Open No. 08-194773 discloses the following technique. In other words, a circumscribed rectangle is extracted for each character from the input document image, and the degree of overlap between each circumscribed rectangle is calculated for each row direction and column direction of the document image, and the degree of overlap between the row direction and the column direction is compared. Then, the first horizontal writing / vertical writing determination step for determining whether the document image is written horizontally or vertically and the distance between the centers of the respective circumscribed rectangles adjacent to each other in the row direction and the column direction of the document image are obtained. And a second horizontal / vertical writing determination step of the document image based on the average value of the center-to-center distance in the column direction, and the first and second determination steps are selected according to the number of characters to be processed. Determine whether horizontal writing or vertical writing. Thereafter, the coordinate transformation of the circumscribed rectangle is performed, and the inclination of the document image is detected and the inclination is corrected, and then the character is cut out and recognized.

  Further, Japanese Patent Laid-Open No. 62-54380 discloses the following technique. That is, by simply scanning the pseudo-square area of the input image in the vertical and horizontal directions to obtain a histogram for the pixels forming the character part, and comparing the average value of the character spacing lengths obtained from this histogram, In this way, the row direction of the input image is extracted.

  Japanese Patent Application Laid-Open No. 61-235990 discloses the following technique. That is, the input image is scanned in the vertical and horizontal directions to obtain a histogram of the pixels forming the character portion, and the average value of the lengths of the character intervals in the vertical and horizontal directions obtained from the histogram is compared. This extracts the line direction of the document and edits the recognition candidate characters. In other words, the rectangle of the recognition target character that has been cut out is sequentially input to the recognition unit, and for each pixel of the cut out character, it is checked whether there are M or more consecutive pixels including the pixel of interest in the direction indicated by the arrow, and a direction code is set To do. For each direction code, the connectivity of each pixel is examined to extract a stroke, and features such as the number, position, and length of the stroke are extracted. The extracted feature amount is collated with the feature amount of characters registered in the dictionary, and the most similar character is set as a recognition candidate character.

  Further, Japanese Patent Application Laid-Open No. 2004-272798 discloses an image reading apparatus that reads an image including character information. Specifically, the apparatus groups continuous black pixel areas constituting a character and includes group circumscribing rectangle information of the black pixel continuous areas included in the read black-and-white two-tone monochrome image. Grouped by the labeling processing means, the line extraction processing means for extracting the line rectangle information from the position information of the group circumscribed rectangles of the grouped black pixel continuous areas extracted by the labeling processing means, and the labeling processing means A punctuation mark identifying means that identifies punctuation marks, periods, and commas from the position and size of the continuous area of black pixels, and a line direction that determines the line direction from the positional relationship of punctuation marks, periods, and commas with respect to the line rectangle of characters in the image Determination means.

Japanese Patent Laid-Open No. 2002-56398 discloses the following technique. That is, a white pixel run as a background of a document image is extracted in both the vertical and horizontal directions, and white pixel runs adjacent to a white pixel run equal to or greater than a predetermined threshold are integrated into a white pixel region. Create a rectangular frame with both vertical and horizontal directions, extract a rectangular frame with a predetermined width or more in both the vertical and horizontal directions with the rectangular frame, and extract the one with the larger number of extracted vertical and horizontal rectangular frames. It is determined as the character string direction of the document. The white pixel run is substantially information related to the pitch.
Japanese Patent Laid-Open No. 08-263587 Japanese Patent Application Laid-Open No. 08-63545 Japanese Patent Application Laid-Open No. 07-220027 JP 2000-20638 A Japanese Patent Laid-Open No. 08-194773 JP 62-54380 A JP 61-235990 A JP 2004-272798 A JP 2002-56398 A

  Some of the techniques described above disclose a process that uses word information as language information when determining the row direction. However, when processing is performed on a portion where no word exists, the determination may be wrong. In addition, when the line direction is determined using the pitch, it is not possible to correctly determine the character area described at the same pitch in both the vertical and horizontal directions. Furthermore, when the line direction is determined from the positional relationship of punctuation marks, periods, and commas, it is not possible to determine those that do not exist.

  As described above, in the conventional technique, the accuracy rate for the determination in the row direction is not high.

  Therefore, an object of the present invention is to provide a novel technique for improving the accuracy rate in determining the row direction.

  The row direction determination method according to the first aspect of the present invention is based on the first number of recognized characters by the vertical character recognition process for the vertical or horizontal character group reading result and the horizontal character recognition process for the read result. The step of obtaining the second number of recognized characters and the n-gram data storage unit for storing the appearance probability of n (n is an integer of 2 or more) continuous characters, and n characters by vertical character recognition processing Calculating a first average appearance probability of the consecutively recognized characters and a second average appearance probability of the n consecutively recognized characters by the character recognition processing in the horizontal direction, the first number of recognized characters, and the second recognition When the number of characters satisfies a predetermined condition, a determination step of determining vertical writing or horizontal writing according to the magnitude of the first average appearance probability and the second average appearance probability is included. By using n-grams in this way, the row direction can be accurately determined even when a word that is not a word is recognized.

  Further, the predetermined condition described above may be such that the ratio of the first recognized character number and the second recognized character number is less than a threshold value. Generally, even if character recognition is performed in the wrong direction, the number of recognized characters decreases, and if character recognition is performed in the correct direction, the number of recognized characters increases. As described above, when the ratio between the first recognized character number and the second recognized character number is less than the threshold value, it indicates that the determination cannot be made only by the recognized character number. As described above, the determination is based on the n-gram. To do.

  Furthermore, when the first number of recognized characters and the second number of recognized characters do not satisfy a predetermined condition, the step of determining vertical writing or horizontal writing according to the size of the first number of recognized characters and the number of second recognized characters May be further included.

  The row direction determination method according to the second aspect of the present invention specifies a circumscribed rectangle of a character or a part of a character from the reading result of a vertically or horizontally written character group, and averages the vertical overlap between adjacent circumscribed rectangles. And a horizontal average overlapping degree between adjacent circumscribed rectangles, and an n-gram data storage unit that stores the appearance probability of n (n is an integer of 2 or more) consecutive characters, and the reading result Calculating a first average appearance probability of n consecutively recognized characters by character recognition processing in the vertical direction and a second average appearance probability of n consecutively recognized characters by character recognition processing in the horizontal direction; A step of determining vertical writing or horizontal writing according to the magnitude of the first average appearance probability and the second average appearance probability when the vertical average overlapping degree and the horizontal average overlapping degree satisfy a predetermined condition; including. By using n-grams in this way, the row direction can be accurately determined even when a word that is not a word is recognized.

  Further, the predetermined condition described above may be such that the ratio of the average overlap in the vertical direction and the average overlap in the horizontal direction is less than the threshold. In general, when the degree of duplication is calculated in the wrong direction, the degree of duplication decreases, and when the degree of duplication is calculated in the correct direction, the degree of duplication increases. As described above, when the ratio of the average overlap in the vertical direction and the average overlap in the horizontal direction is less than the threshold value, it indicates that the determination cannot be made only with the overlap, and as described above, the n-gram base is used. Judgment.

  Further, when the vertical average horizontal overlap and the horizontal average overlap do not satisfy the predetermined condition, vertical writing or horizontal writing is performed depending on the size of the vertical average horizontal overlapping and the horizontal average overlapping. The method may further include a step of determining.

  The row direction determination method according to the third aspect of the present invention includes a first average certainty factor of a vertical character recognition result with respect to a reading result of a vertical or horizontal character group and a second character recognition result of a horizontal direction. And calculating a vertical writing or a horizontal writing based on the magnitude of the first average certainty and the second average certainty. If the character recognition is performed in the wrong direction as described above, the certainty level is lowered. If the character recognition is performed in the correct direction, the certainty level is increased. Therefore, if the processing described above is performed, the correct line direction is specified. be able to.

  A program for causing a computer to execute this method can be created, and this program is stored in a storage medium or storage device such as a flexible disk, a CD-ROM, a magneto-optical disk, a semiconductor memory, or a hard disk. . Moreover, it may be distributed as a digital signal via a network or the like. The intermediate processing result is temporarily stored in a storage device such as a main memory.

  According to the present invention, it is possible to improve the accuracy rate for determination in the row direction.

[Embodiment 1]
FIG. 1 shows a functional block diagram of the row direction determination device according to the first embodiment of the present invention. The row direction determination apparatus according to the first embodiment includes, for example, a scanner 1 that optically reads a document that includes vertically or horizontally written text, and image data storage that stores image data of a character group read by the scanner 1. Part 3 and an n-gram data storage that holds data (n-gram data) about the appearance probability of n (n is an integer of 2 or more) consecutive characters generated in advance from a large amount of text data A character recognition processing unit 5 that performs character recognition processing or the like using data stored in the n-gram data storage unit 7 for at least a part of the image data stored in the image data storage unit 3, and character recognition processing A histogram of black pixels is generated from the character recognition result data storage unit 9 for storing the processing results of the unit 5 and the image data stored in the image data storage unit 3 to generate vertical and horizontal lines. Stored in the line number determination unit 13 for calculating the number, the line number data storage unit 15 for storing the calculation result of the line number determination unit 13, the character recognition result data storage unit 9, and in some cases the line number data storage unit 15. A row direction determination unit 11 that determines the row direction using the processed data.

  In the present embodiment, it is assumed that the n-gram data storage unit 7 holds data about bigrams as n-grams. In addition, not only a word but the case where the word which is not a word continues is registered into the n-gram, and the appearance probability is registered. Therefore, it is possible to perform processing even on a portion that does not include a word.

  In the present embodiment and the following embodiments, it is assumed that one text area is either horizontal writing or vertical writing, and there is no mixed vertical and horizontal area. In addition, the following processing is performed after the vertical and horizontal mixed area is divided into a vertical area and a horizontal area by a prior layout recognition technique.

  Next, processing of the row direction determination device according to the first embodiment will be described with reference to FIGS. First, a document including a text to be processed is read as image data by the scanner 1, and the read image data is stored in the image data storage unit 3. Then, the character recognition processing unit 5 performs vertical character recognition processing on at least a part of the image data stored in the image data storage unit 3, and recognizes the number of recognized characters Nv, the number of line feeds Cv, and the average appearance of n-grams. The probability Pv is acquired and stored in the character recognition result data storage unit 9 (step S1).

  More specifically, the number of characters that can be recognized as a result of performing the character recognition process in the vertical direction is counted as the recognized character number Nv, and when a line break is detected during the character recognition process in the vertical direction, the number of line breaks Cv is detected. Increment by the number of times. Since it is the number of line breaks, Cv = 1 for 2 lines and Cv = 2 for 3 lines. For characters that could be recognized, the n-gram data storage unit 7 is searched for every two characters (generally n characters), the corresponding appearance probability is obtained, and the average value of the obtained appearance probabilities is obtained. calculate. When it is detected that there are a plurality of lines, the appearance probability is not acquired for two characters straddling the lines. In addition, when it is detected that there are a plurality of lines, the number of recognized characters may be different in each line. In this case, for example, a statistical value such as an average value of the number of recognized characters may be calculated. good.

  Similarly, the character recognition processing unit 5 performs a horizontal character recognition process on at least a part of the image data stored in the image data storage unit 3, and recognizes the number of recognized characters Nh, the number of line breaks Ch, and the average of n grams. The appearance probability Ph is acquired and stored in the character recognition result data storage unit 9 (step S3). The specific processing is the same as that described for the vertical direction.

  Then, the line direction determination unit 11 calculates max (Nv, Nh) / min (Nv, Nh) using the data stored in the character recognition result data storage unit 9, and max (Nv, Nh) / It is determined whether or not min (Nv, Nh)> threshold (experimentally 3.25 is known to be preferable) is satisfied (step S5). If Nv> Nh, Nv / Nh is calculated, and if Nv <Nh, Nh / Nv is calculated and compared with a threshold value. If Nv = Nh, it is 1, and it is determined that the condition of step S5 is not satisfied. Usually, if the character recognition process is performed in the correct direction, a larger number of characters can be recognized, and if the character recognition process is performed in the wrong direction, only a smaller number of characters can be recognized. In this way, when the number of recognized characters differs vertically and horizontally by exceeding a threshold multiple (3.25 times), it indicates a case where the correct direction and the wrong direction are clear.

  Therefore, if it is determined that the condition of step S5 is satisfied, the row direction determination unit 11 checks whether Nv> Nh (step S7), and if Nv> Nh, the row direction is the vertical direction. Judgment is made (step S9). For example, when an image as shown in FIG. 3A is processed, if character recognition is performed in the vertical direction, for example, “one”, “繼”, “finished”, “skin”, “study”, “place” are displayed on the first line. In the second line, the characters “角”, “horn”, “field”, “win”, “tsukasa”, and “sha” are recognized and each has 6 characters, so Nv = 6. On the other hand, when character recognition is performed in the horizontal direction, it is misaligned and recognized as one character, and Nh = 1. Therefore, it is determined that the condition is satisfied in steps S5 and S7, and the vertical direction is determined in step S9.

  Conversely, if Nh> Nv, the row direction is determined to be the horizontal direction (step S11). For example, in the case of processing an image as shown in FIG. 3B, if character recognition is performed in the vertical direction, for example, “middle”, “base”, “book”, “war” on the first line, and “ The characters “遜”, “mind”, “table”, “light” and “mouth” are recognized, and on average, Nh = 4.5. On the other hand, when character recognition is performed in the vertical direction, the character is misaligned and thus recognized as one character and Nv = 1. Accordingly, the condition is satisfied in step S5, but the condition is not satisfied in step S7, and the horizontal direction is determined in step S11.

  Then, the process ends. Normally, when the line direction is specified, the character recognition processing unit 5 performs character recognition processing in the specified line direction on the entire image data stored in the image data storage unit 3. To.

  On the other hand, if it is determined that the condition of step S5 is not satisfied, the line direction determination unit 11 reads the number of vertical line breaks Cv stored in the character recognition result data storage unit 9, and whether Cv = 0. Judgment is made (step S13). If Cv = 0, it indicates that the vertical direction is one line, so that the row direction is determined to be the vertical direction (step S19). For example, when processing image data as shown in FIG. 3C, the number of line breaks Cv = 0, so the row direction is determined to be the vertical direction.

  On the other hand, if not Cv = 0, the horizontal line feed number Ch stored in the character recognition result data storage unit 9 is read out, and it is determined whether Ch = 0 (step S15). If Ch = 0, it indicates that the horizontal direction is one line, so the row direction is determined to be the horizontal direction (step S21). For example, when processing image data as shown in FIG. 3D, the line feed number Ch = 0, so the line direction is determined to be the horizontal direction.

  Note that the number of lines is not determined by the number of line breaks, and the processing result by the line number determination unit 13 may be used. Specifically, a black pixel (character pixel) in a portion to be processed in image data is projected in the vertical direction, and a histogram is generated by counting the number of pixels for each projection position. Is detected. Similarly, a black pixel in a portion to be processed in image data is projected in the horizontal direction and the number of pixels is counted for each projection position to generate a histogram, and a character break is detected according to the frequency. For example, if a horizontally written “university” is projected in the vertical direction as shown in FIG. 4A, a histogram as shown in FIG. 4B is generated, and a frequency of zero or an error level is generated. It can be seen that there is a break at the part where it is. For example, a value obtained by multiplying the mode value by 0.1 is used as a threshold value for determining whether or not there is a break. In the case of FIG. 3C, a break is detected in the histogram by the horizontal projection, and no break occurs in the histogram by the vertical projection, so it is determined that the vertical direction is one line. Similarly, also in the case of FIG. 3D, a break is detected in the histogram by the vertical projection, and no break is generated in the histogram by the horizontal projection, so it is determined that the horizontal direction is one line.

  The line number determination result (both in the vertical direction and the horizontal direction) by the line number determination unit 13 is stored in the line number data storage unit 15, and the line number determination result is stored based on the line number determination result stored in the line number data storage unit 15. The direction determination unit 11 may determine whether the vertical direction is one line or the horizontal direction is one line.

  When it is determined that neither vertical nor horizontal is one line, the vertical n-gram average appearance probability Pv is compared with the horizontal n-gram average appearance probability Ph to determine whether Ph ≧ Pv is satisfied ( Step S17). If it is satisfied, it is presumed that the horizontal direction is more accurately recognized, so the line direction is determined to be the horizontal direction (step S21). On the other hand, if the condition of step S17 is not satisfied, it is presumed that the character in the vertical direction can be recognized more accurately, so the row direction is determined as the vertical direction (step S19).

  For example, if image data as shown in FIG. 3 (e) is a processing target, there are almost no vertical and horizontal deviations in the characters, and if all characters are recognized correctly, words in the horizontal direction are more meaningful. Therefore, the appearance frequency increases in the n-gram. Therefore, in the example shown in FIG. 3E, the row direction is determined as the horizontal direction. If image data as shown in FIG. 3 (f) is a processing target, there is a possibility that the character is recognized as one character in the horizontal direction because there is a character shift in the horizontal direction. If the line is clear and all the characters are recognized correctly, the words that are more meaningful in the vertical direction are consecutive, and the frequency of occurrence is higher in the n-gram. Therefore, in the example shown in FIG. 3F, the row direction is determined to be the vertical direction.

  By performing the processing as described above, the row direction can be determined more accurately. According to the inventor's experiment, the accuracy rate was 97.3%.

[Embodiment 2]
Next, a functional block diagram of the row direction determination apparatus according to the second embodiment will be described with reference to FIG. In addition, the same code | symbol is attached | subjected about what performs the same process as FIG. The row direction determination apparatus according to the second embodiment includes a scanner 1, an image data storage unit 3 for storing image data read by the scanner 1, and a sequence of n consecutive characters generated in advance from a large amount of text data. Data stored in the n-gram data storage unit 7 for at least a part of the image data stored in the image data storage unit 3 and the n-gram data storage unit 7 holding data (n-gram data) about the appearance probability. At least a part of the image data stored in the image data storage unit 3, the character recognition result data storage unit 29 for storing the processing result of the character recognition processing unit 25, the character recognition result data storage unit 29 for storing the processing result of the character recognition processing unit 25 A line number determination unit 13 that generates a histogram of black pixels from the number of pixels and calculates the number of vertical and horizontal lines; A storage unit 15, a redundancy calculation unit 33 that calculates the redundancy between characters from at least a part of the image data stored in the image data storage unit 3, and data storage of the redundancy calculated by the redundancy calculation unit 33 A row direction determination unit that determines vertical writing or horizontal writing using data stored in the redundancy data storage unit 35, the character recognition result data storage unit 29, the line number data storage unit 15, and the redundancy data storage unit 35. 31.

  Next, the processing flow of the row direction determination apparatus shown in FIG. 5 will be described with reference to FIGS. 6 and 7. First, a document including a text to be processed is read as image data by the scanner 1, and the read image data is stored in the image data storage unit 3. Then, the line number determination unit 13 performs black pixel histogram processing by projecting black pixels (pixels for characters) vertically and horizontally for at least part of the image data stored in the image data storage unit 3, The number of lines is determined by counting a range exceeding a threshold value (for example, a value obtained by multiplying the mode value by 0.1), and stored in the line number data storage unit 15 (step S31).

  The processing described in relation to FIGS. 4 (a) and 4 (b) is performed, and a portion where only a frequency equal to or less than the threshold is detected is determined as a character break, and a range exceeding the threshold is counted or a character is counted. The number of lines is determined by the number of breaks + 1.

  The row direction determination unit 31 determines whether there is a direction with a line number of 1 using the number of vertical lines and the number of horizontal lines stored in the line number data storage unit 15 (step S33). . If there is a direction with a line number of 1 in the vertical or horizontal direction, the direction with the line number of 1 is specified as the row direction (step S37). For example, in the case of FIGS. 3C and 3D, the direction in which the number of lines is determined to be 1 is specified as the row direction. Then, the process ends.

  On the other hand, if it is determined that there is no direction having the number of lines 1, the redundancy calculation unit 33 determines the circumscribed rectangle of the character included in at least a part of the image data stored in the image data storage unit 3. Based on the circumscribed rectangle of the character, the degree of overlap Ov in the vertical direction and the degree of overlap Oh in the horizontal direction between the adjacent circumscribed rectangles are calculated and stored in the degree-of-redundancy data storage unit 35 (step S35). ). Specifically, as shown in FIG. 7, the circumscribed rectangles of characters such as rectangles 101 to 109 are specified. In FIG. 7, the circumscribed rectangle is simplified to be one rectangle per character, but a plurality of rectangles may be specified for each character. For each adjacent circumscribed rectangle in the horizontal direction, the degree of overlap (the length of the overlapping distance when the rectangle is projected on the boundary between the rectangles) is specified, and the average is calculated. Specifically, rectangle 101 and rectangle 102 overlap (specifically, the sum of overlap) 201, rectangle 102 and rectangle 103 overlap 202, rectangle 104 and rectangle 105 overlap 203, rectangle 105 and rectangle 106 overlap 204, The overlap 205 between the rectangle 107 and the rectangle 108 and the overlap 206 between the rectangle 108 and the rectangle 109 are specified, and the average is calculated. Further, for the adjacent circumscribed rectangles in the vertical direction, the degree of overlap (the length of the overlapping distance when the rectangles are projected on the boundaries between the rectangles) is specified, and the average is calculated. Specifically, rectangle 211 and rectangle 104 overlap 211, rectangle 104 and rectangle 107 overlap 214, rectangle 102 and rectangle 105 overlap 212, rectangle 105 and rectangle 108 overlap 215, rectangle 103 and rectangle 106 overlap 213, The overlap 216 between the rectangle 106 and the rectangle 109 is specified, and the average is calculated. Such a technique is disclosed in, for example, Japanese Patent Application Laid-Open No. 10-63776, and details are not described further.

  Then, the row direction determination unit 31 calculates max (Ov, Oh) / min (Ov, Oh) using the data stored in the redundancy data storage unit 35, and max (Ov, Oh) / min (Ov). , Oh) is larger than a threshold value (experimentally 1.4 is known to be preferable) (step S39). If Ov> Oh, Ov / Oh is calculated, and if Ov <Oh, Oh / Ov is calculated and compared with a threshold value. If Ov = Oh, it is 1 and it is determined that the condition of step S39 is not satisfied. Usually, if the degree of overlap is calculated in the correct direction, the rectangles are aligned in one direction, so the degree of overlap is high. If the degree of overlap is calculated in the wrong direction, the rectangles vary and the degree of overlap is low. In this way, when the vertical and horizontal overlaps differ by more than a threshold value (1.4 times), the correct direction and the wrong direction are clearly shown.

  Therefore, when the condition of step S39 is satisfied, the row direction determination unit 31 determines whether or not Ov> Oh (step S41). When this condition is satisfied, the row direction is determined as the vertical direction. (Step S43). On the other hand, when the condition of step S41 is not satisfied, the row direction is determined as the horizontal direction (step S45). For example, in the case shown in FIG. 3A, the vertical direction is determined in step S43, and in the case shown in FIG. 3B, the horizontal direction is determined in step S45.

  If the condition of step S39 is not satisfied, the character recognition processing unit 25 performs character recognition in the vertical direction for at least a part of the image data stored in the image data storage unit 3, and recognizes the recognized character. An n-gram average appearance probability Pv is acquired using the n-gram data storage unit 7 and stored in the character recognition result data storage unit 29 (step S47).

  More specifically, the character recognition process is performed in the vertical direction to recognize the characters, and for the characters that can be recognized, the n-gram data storage unit 7 is provided every two characters (generally, n characters). , The corresponding appearance probability is acquired, and the average value of the acquired appearance probabilities is calculated. When it is detected that there are a plurality of lines, the appearance probability is not acquired for two characters straddling the lines.

  Similarly, the character recognition processing unit 25 performs horizontal character recognition processing on at least a part of the image data stored in the image data storage unit 3, and the n-gram data storage unit 7 for the recognized character. Is used to acquire the n-gram average appearance probability Ph and store it in the character recognition result data storage unit 29 (step S49).

  Then, the line direction determination unit 31 compares the vertical n-gram average appearance probability Pv stored in the character recognition result data storage unit 29 with the horizontal n-gram average appearance probability Ph, and Ph ≧ Pv is satisfied. It is determined whether or not (step S51). If so, it is presumed that the horizontal direction is more accurately recognized, so the line direction is determined to be the horizontal direction (step S55). On the other hand, if the condition of step S51 is not satisfied, it is presumed that the character in the vertical direction can be recognized more accurately, so the row direction is determined as the vertical direction (step S53). 3 (e) and FIG. 3 (f), as described in the first embodiment, in the example as shown in FIG. 3 (e), the row direction is determined as the horizontal direction, In the example shown in FIG. 3F, the row direction is determined as the vertical direction.

  By performing the processing as described above, the row direction can be determined more accurately. According to the inventors' experiment, the accuracy rate was 99.6%.

[Embodiment 3]
Next, FIG. 8 shows a functional block diagram of the row direction determining apparatus according to the third embodiment. In addition, the same code | symbol is attached | subjected about what performs the same process as 1st Embodiment. The row direction determination apparatus according to the third embodiment includes a scanner 1, an image data storage unit 3 that stores image data read by the scanner 1, and at least one of image data stored in the image data storage unit 3. A line number determination unit 13 that generates pixel histograms from the unit and calculates the number of vertical and horizontal lines; a line number data storage unit 15 that stores the calculation result of the line number determination unit 13; and the image data storage unit 3. A character recognition processing unit 41 that performs character recognition processing or the like on at least a part of the image data, a character recognition result data storage unit 42 that stores a processing result of the character recognition processing unit 41, a character recognition result data storage unit 42, and a line A row direction determination unit 43 that determines vertical writing or horizontal writing using the data stored in the numerical data storage unit 15.

  Next, the processing flow of the row direction determination apparatus shown in FIG. 8 will be described with reference to FIG. First, a document including a text to be processed is read as image data by the scanner 1, and the read image data is stored in the image data storage unit 3. Then, the line number determination unit 13 performs black pixel histogram processing by projecting black pixels (pixels for characters) vertically and horizontally for at least part of the image data stored in the image data storage unit 3, The number of lines is determined by counting a range exceeding a threshold value (for example, a value obtained by multiplying the mode value by 0.1), and stored in the line number data storage unit 15 (step S61).

  The processing described in relation to FIGS. 4 (a) and 4 (b) is performed, and a portion where only a frequency equal to or less than the threshold is detected is determined as a character break, and a range exceeding the threshold is counted or a character is counted. The number of lines is determined by the number of breaks + 1.

  The row direction determination unit 43 determines whether there is a direction with a line number of 1 using the number of vertical lines and the number of horizontal lines stored in the line number data storage unit 15 (step S63). . If there is a direction with a line number of 1 in the vertical or horizontal direction, the direction with the line number of 1 is specified as the row direction (step S65). For example, in the case of FIGS. 3C and 3D, the direction in which the number of lines is determined to be 1 is specified as the row direction. Then, the process ends.

  On the other hand, if it is determined that there is no direction with the number of lines 1, the character recognition processing unit 41 performs character recognition processing in the vertical direction on at least a part of the image data stored in the image data storage unit 3. Then, the average value Rv of the certainty factor obtained simultaneously with the character recognition is calculated and stored in the character recognition result data storage unit 42 (step S67). For example, in the case of the image data shown in FIG. 3A, the recognition results of the vertical character recognition are, for example, “one” “繼” “done” “skin” “study” “place” and “所”. ”“ Horn ”“ field ”“ win ”“ tsukasa ”“ company ”, for example, the average value Rv of confidence is calculated as 706. Since the certainty factor is described in detail in, for example, Japanese Patent Laid-Open No. 2000-306045, details are not described here.

  Further, the character recognition processing unit 41 performs character recognition processing in the horizontal direction on at least a part of the image data stored in the image data storage unit 3, and calculates an average value Rh of certainty factor obtained simultaneously with the character recognition. Then, it is stored in the character recognition result data storage unit 42 (step S69). In the case of the example of FIG. 3A, the recognition result of the character recognition in the horizontal direction is “例 え ば”, for example, and the average value Rh of the certainty factor is calculated as 625, for example.

  Thereafter, the row direction determination unit 43 compares the average value Rv of the certainty in the vertical direction stored in the character recognition result data storage unit 42 with the average value Rh of the certainty in the horizontal direction, and Rv ≧ Rh is established. (Step S71). If Rv ≧ Rh holds as in the example of FIG. 3A, the row direction determination unit 43 determines that the vertical direction is the row direction (step S73). On the other hand, when it is determined that the relationship of step S71 does not hold as in the case of FIG. 3B, the horizontal direction is determined as the row direction (step S73).

  In this way, a direction in which the degree of certainty about the result of the character recognition process becomes higher is specified.

  Although the embodiment of the present invention has been described above, the present invention is not limited to this. For example, the example of Japanese is shown above, but the present invention can be applied to languages such as Korean and Chinese that may use both vertical writing and horizontal writing.

  Moreover, although the example which performs a character recognition process etc. about at least one part of image data was shown, the vertical and horizontal lengths, such as a square, do not necessarily need to be the same.

  Furthermore, the threshold values described for the processing described above may need to be adjusted depending on the environment and language.

  The row direction determination device (except for the scanner 1) described above is a computer device as shown in FIG. 10, and includes a memory 2501 (storage device), a CPU 2503 (processing device), a hard disk drive (HDD) 2505, and the like. A display control unit 2507 connected to the display device 2509, a drive device 2513 for a removable disk 2511, an input device 2515, and a communication control unit 2517 for connecting to a network are connected by a bus 2519. An operating system (OS: Operating System) and an application program for performing processing in the present embodiment are stored in the HDD 2505, and are read from the HDD 2505 to the memory 2501 when executed by the CPU 2503. If necessary, the CPU 2503 controls the display control unit 2507, the communication control unit 2517, and the drive device 2513 to perform necessary operations. Further, data in the middle of processing is stored in the memory 2501 and stored in the HDD 2505 if necessary. In the embodiment of the present invention, an application program for performing the processing described above is stored in the removable disk 2511 and distributed, and is installed in the HDD 2505 from the drive device 2513. In some cases, the HDD 2505 may be installed via a network such as the Internet and the communication control unit 2517. Such a computer apparatus realizes various functions as described above by organically cooperating hardware such as the CPU 2503 and the memory 2501 described above, the OS, and necessary application programs.

(Appendix 1)
Obtaining a first number of recognized characters by a vertical character recognition process for a result of reading a vertical or horizontal character group, and a second number of recognized characters by a horizontal character recognition process for the read result;
Referring to an n-gram data storage section that stores the appearance probability of n (n is an integer of 2 or more) continuous characters, a first average appearance probability of n consecutive recognition characters by the vertical character recognition processing Calculating a second average appearance probability of n consecutively recognized characters by the horizontal character recognition process;
When the first number of recognized characters and the second number of recognized characters satisfy a predetermined condition, vertical writing or horizontal writing is determined according to the magnitude of the first average appearance probability and the second average appearance probability. A determination step to:
A row direction determination method executed by a computer.

(Appendix 2)
The row direction determination method according to claim 1, wherein the predetermined condition is that a ratio of the first number of recognized characters to the second number of recognized characters is less than a threshold value.

(Appendix 3)
When the first number of recognized characters and the second number of recognized characters do not satisfy the predetermined condition, vertical writing or horizontal writing is performed according to the size of the first recognized character number and the second recognized character number. The row direction determination method according to appendix 1, further comprising a step of determining.

(Appendix 4)
The determination step includes
The line direction determination method according to appendix 1, including a step of determining vertical writing or horizontal writing based on the specific direction when the line number determination result represents one line in the specific direction.

(Appendix 5)
In the line number determination, the number of line breaks in the vertical character recognition process and the horizontal character recognition process, or the vertical pixel histogram and the character pixel by the vertical projection of the character pixels in the read result The direction determination method according to appendix 4, wherein the number of lines is determined by a horizontal pixel histogram by projection in the horizontal direction.

(Appendix 6)
Identifies the circumscribed rectangle of a character or part of a character from the result of reading vertical or horizontal characters, and calculates the vertical average overlap between adjacent circumscribed rectangles and the average horizontal overlap between adjacent circumscribed rectangles And steps to
Referring to an n-gram data storage unit that stores the appearance probability of n (n is an integer of 2 or more) continuous characters, the first of n consecutively recognized characters by the character recognition processing in the vertical direction for the read result Calculating an average appearance probability and a second average appearance probability of n consecutively recognized characters by the horizontal character recognition process;
When the vertical average overlap and the horizontal average overlap satisfy a predetermined condition, vertical writing or horizontal writing is performed depending on the magnitude of the first average appearance probability and the second average appearance probability. A determining step;
A row direction determination method executed by a computer.

(Appendix 7)
The row direction determination method according to appendix 6, wherein the predetermined condition is that a ratio of the average overlap degree in the vertical direction and the average overlap degree in the horizontal direction is less than a threshold value.

(Appendix 8)
If the average vertical overlap and the horizontal average overlap do not satisfy a predetermined condition, vertical writing is performed according to the size of the vertical average overlap and the horizontal average overlap. The row direction determination method according to appendix 6, further comprising a step of determining horizontal writing.

(Appendix 9)
Whether vertical writing or horizontal writing is determined based on the direction determined as one row by the vertical pixel histogram by the vertical projection of the character pixels and the horizontal pixel histogram by the horizontal projection of the character pixels in the reading result. The line direction determination method according to appendix 6, further comprising:

(Appendix 10)
Calculating a first average certainty of a vertical character recognition result and a second average certainty of a horizontal character recognition result with respect to a reading result of a vertically or horizontally written character group;
Determining vertical writing or horizontal writing based on the magnitude of the first average certainty factor and the second average certainty factor;
A row direction determination method executed by a computer.

(Appendix 11)
Whether vertical writing or horizontal writing is determined based on the direction determined as one row by the vertical pixel histogram by the vertical projection of the character pixels and the horizontal pixel histogram by the horizontal projection of the character pixels in the reading result. The line direction determination method according to appendix 10, further comprising:

(Appendix 12)
A program for causing a computer to execute the row direction determination method according to any one of appendices 1 to 11.

(Appendix 13)
Means for acquiring a first recognized number of characters by vertical character recognition processing for a reading result of a vertical or horizontal character group and a second number of recognized characters by horizontal character recognition processing for the reading result;
Referring to an n-gram data storage section that stores the appearance probability of n (n is an integer of 2 or more) continuous characters, a first average appearance probability of n consecutive recognition characters by the vertical character recognition processing And means for calculating a second average appearance probability of n consecutively recognized characters by the horizontal character recognition process;
When the first number of recognized characters and the second number of recognized characters satisfy a predetermined condition, vertical writing or horizontal writing is determined according to the magnitude of the first average appearance probability and the second average appearance probability. Determination means to perform,
A row direction determination device having:

(Appendix 14)
Identifies the circumscribed rectangle of a character or part of a character from the result of reading vertical or horizontal characters, and calculates the vertical average overlap between adjacent circumscribed rectangles and the average horizontal overlap between adjacent circumscribed rectangles Means to
Referring to an n-gram data storage unit that stores the appearance probability of n (n is an integer of 2 or more) continuous characters, the first of n consecutively recognized characters by the character recognition processing in the vertical direction for the read result Means for calculating an average appearance probability and a second average appearance probability of n consecutively recognized characters by the character recognition processing in the horizontal direction;
When the vertical average overlap and the horizontal average overlap satisfy a predetermined condition, vertical writing or horizontal writing is performed depending on the magnitude of the first average appearance probability and the second average appearance probability. Means for determining;
A row direction determination device having:

(Appendix 15)
Means for calculating a first average certainty of a vertical character recognition result and a second average certainty of a horizontal character recognition result with respect to a vertical or horizontal character group reading result;
Means for determining vertical writing or horizontal writing based on the magnitude of the first average certainty factor and the second average certainty factor;
A row direction determination device having:

It is a functional block diagram concerning the 1st embodiment of the present invention. It is a figure which shows the processing flow in the 1st Embodiment of this invention. (A) thru | or (f) is a figure which shows the example of the concrete image data in embodiment of this invention. (A) And (b) is a figure for demonstrating the histogram of a black pixel. It is a functional block diagram concerning a 2nd embodiment of the present invention. It is a figure which shows the processing flow in the 2nd Embodiment of this invention. It is a figure for demonstrating calculation of a duplication degree. It is a functional block diagram concerning a 3rd embodiment of the present invention. It is a figure which shows the processing flow in the 3rd Embodiment of this invention. It is a functional block diagram of a computer.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Scanner 3 Image data storage part 5, 25, 41 Character recognition process part 7 n-gram data storage part 9, 29, 42 Character recognition result data storage part 11, 31, 43 Line direction determination part 13 Line number determination part 15 Number of lines Data storage unit 33 Multiplicity calculation unit 35 Multiplicity data storage unit

Claims (9)

Obtaining a first number of recognized characters by a vertical character recognition process for a result of reading a vertical or horizontal character group, and a second number of recognized characters by a horizontal character recognition process for the read result;
Referring to an n-gram data storage section that stores the appearance probability of n (n is an integer of 2 or more) continuous characters, a first average appearance probability of n consecutive recognition characters by the vertical character recognition processing Calculating a second average appearance probability of n consecutively recognized characters by the horizontal character recognition process;
When the ratio of the larger number of recognized characters to the smaller number of recognized characters of the first number of recognized characters and the second number of recognized characters is less than a threshold , the first average appearance probability and the second An average appearance probability is compared, and when the first average appearance probability is greater than the second average appearance probability, it is determined as vertical writing, and the second average appearance probability is greater than the first average appearance probability. If it is also larger, the step of judging as horizontal writing ,
-Direction determination program for causing a computer to execute. Identifies the circumscribed rectangle of a character or part of a character from the result of reading vertical or horizontal characters, and calculates the vertical average overlap between adjacent circumscribed rectangles and the average horizontal overlap between adjacent circumscribed rectangles And steps to
Referring to an n-gram data storage unit that stores the appearance probability of n (n is an integer of 2 or more) continuous characters, the first of n consecutively recognized characters by the character recognition processing in the vertical direction for the read result Calculating an average appearance probability and a second average appearance probability of n consecutively recognized characters by the horizontal character recognition process;
When the ratio of the larger average overlap degree to the smaller one of the average overlap degree in the vertical direction and the average overlap degree in the horizontal direction is less than a threshold , the first average appearance probability and the first 2 if the first average appearance probability is greater than the second average appearance probability, it is determined as vertical writing, and the second average appearance probability is the first average appearance probability. A step of determining horizontal writing if greater than the probability ,
-Direction determination program for causing a computer to execute. When the ratio of the larger average overlap to the smaller one of the average overlap in the vertical direction and the average overlap in the horizontal direction is larger than a threshold , the average overlap in the vertical direction and the average overlap Compared with the average overlap in the horizontal direction, if the average overlap in the vertical direction is larger than the average overlap in the horizontal direction, it is determined as vertical writing, the average overlap in the horizontal direction is the average in the vertical direction The row direction determination program according to claim 2 , further causing the computer to execute a step of determining horizontal writing when the degree of overlap is greater . Means for acquiring a first recognized number of characters by vertical character recognition processing for a reading result of a vertical or horizontal character group and a second number of recognized characters by horizontal character recognition processing for the reading result;
Referring to an n-gram data storage section that stores the appearance probability of n (n is an integer of 2 or more) continuous characters, a first average appearance probability of n consecutive recognition characters by the vertical character recognition processing And means for calculating a second average appearance probability of n consecutively recognized characters by the horizontal character recognition process;
When the ratio of the larger number of recognized characters to the smaller number of recognized characters of the first number of recognized characters and the second number of recognized characters is less than a threshold , the first average appearance probability and the second An average appearance probability is compared, and when the first average appearance probability is greater than the second average appearance probability, it is determined as vertical writing, and the second average appearance probability is greater than the first average appearance probability. If it is too large, means to determine horizontal writing ,
A row direction determination device having: A means for identifying a circumscribed rectangle of a character with respect to a reading result of a vertically or horizontally written character group, and calculating an average overlap in the vertical direction between adjacent circumscribed rectangles and an average overlap in the horizontal direction between adjacent circumscribed rectangles;
Referring to an n-gram data storage unit that stores the appearance probability of n (n is an integer of 2 or more) continuous characters, the first of n consecutively recognized characters by the character recognition processing in the vertical direction for the read result Means for calculating an average appearance probability and a second average appearance probability of n consecutively recognized characters by the character recognition processing in the horizontal direction;
When the ratio of the larger average overlap degree to the smaller one of the average overlap degree in the vertical direction and the average overlap degree in the horizontal direction is less than a threshold , the first average appearance probability and the first 2 if the first average appearance probability is greater than the second average appearance probability, it is determined as vertical writing, and the second average appearance probability is the first average appearance probability. If it is greater than the probability , means to determine horizontal writing ,
A row direction determination device having: Means for calculating a first average certainty of a vertical character recognition result and a second average certainty of a horizontal character recognition result with respect to a vertical or horizontal character group reading result;
The number of lines in the vertical or horizontal direction based on the vertical pixel histogram based on the vertical projection of the character pixels and the horizontal pixel histogram based on the horizontal projection of the character pixels in the reading result of the vertical or horizontal character group If there is a direction with a line number of 1 in the vertical or horizontal direction, the direction with the line number of 1 is specified as the row direction, and the line is set in the vertical or horizontal direction. When it is determined that there is no direction having a number of 1, the first average certainty factor is compared with the second average certainty factor, and the first average certainty factor is greater than the second average certainty factor. Is determined to be vertical writing, and when the second average certainty is greater than the first average certainty , means for determining horizontal writing ;
A row direction determination device having: Obtaining a first number of recognized characters by a vertical character recognition process for a result of reading a vertical or horizontal character group, and a second number of recognized characters by a horizontal character recognition process for the read result;
Referring to an n-gram data storage section that stores the appearance probability of n (n is an integer of 2 or more) continuous characters, a first average appearance probability of n consecutive recognition characters by the vertical character recognition processing Calculating a second average appearance probability of n consecutively recognized characters by the horizontal character recognition process;
When the ratio of the larger number of recognized characters to the smaller number of recognized characters of the first number of recognized characters and the second number of recognized characters is less than a threshold , the first average appearance probability and the second An average appearance probability is compared, and when the first average appearance probability is greater than the second average appearance probability, it is determined as vertical writing, and the second average appearance probability is greater than the first average appearance probability. If it is also larger, the step of judging as horizontal writing ,
A row direction determination method executed by a computer. Identifies the circumscribed rectangle of a character or part of a character from the result of reading vertical or horizontal characters, and calculates the vertical average overlap between adjacent circumscribed rectangles and the average horizontal overlap between adjacent circumscribed rectangles And steps to
Referring to an n-gram data storage unit that stores the appearance probability of n (n is an integer of 2 or more) continuous characters, the first of n consecutively recognized characters by the character recognition processing in the vertical direction for the read result Calculating an average appearance probability and a second average appearance probability of n consecutively recognized characters by the horizontal character recognition process;
When the ratio of the larger average overlap degree to the smaller one of the average overlap degree in the vertical direction and the average overlap degree in the horizontal direction is less than a threshold , the first average appearance probability and the first 2 if the first average appearance probability is greater than the second average appearance probability, it is determined as vertical writing, and the second average appearance probability is the first average appearance probability. A step of determining horizontal writing if greater than the probability ,
A row direction determination method executed by a computer. When the ratio of the larger average overlap to the smaller one of the average overlap in the vertical direction and the average overlap in the horizontal direction is larger than a threshold , the average overlap in the vertical direction and the average overlap Compared with the average overlap in the horizontal direction, if the average overlap in the vertical direction is larger than the average overlap in the horizontal direction, it is determined as vertical writing, the average overlap in the horizontal direction is the average in the vertical direction The row direction determination method according to claim 8 , further comprising the step of determining horizontal writing when the degree of overlap is greater .

Images