JP4943501B2 - Image processing apparatus and method, and optical character identification apparatus and method - Google Patents

Description

  The present invention relates to a technical field of an image processing apparatus and method capable of extracting and recognizing a character portion included in an image such as a color image, and an optical character identification apparatus and method.

  In recent years, with the spread of multi-channel broadcasting and the spread of the high-speed Internet, the image and video content that we can handle has become enormous. Such images often include characters, such as telops in TV broadcasts, signboard characters in photographs, and the like. When it is desired to manage an image efficiently, it is an effective method to recognize these characters included in the image, convert them into character codes, and use them as image metadata.

  As a technique for recognizing characters in an image, there is optical character recognition (hereinafter referred to as “OCR” (Optical Character Recognition) as appropriate). In this method, a black and white two-color image is input and character recognition is performed using black as a character pixel and white as a background pixel. Therefore, in order to recognize characters in a general color image by OCR, it is necessary to extract only the character portion and convert it into a monochrome image. However, when the amount of images handled is enormous, it is difficult to perform all of this conversion work manually, and automation is desired.

  Patent Document 1 and the like disclose the following method. First, in a class image clustered by color components, a pixel connection region in which pixels are connected is extracted, and a rectangle that has the smallest area and size among rectangles circumscribing the pixel connection region (hereinafter referred to as “minimum” as appropriate). (Referred to as a circumscribed rectangle). Next, characterity (a degree that can be regarded as a character) is determined from the positional relationship of the acquired minimum circumscribed rectangle. This is because the character parts in the image are almost the same size and tend to line up at almost the same height, so the same characteristic can be seen in the minimum circumscribed rectangle that circumscribes the character part. Because it is.

  Alternatively, in Patent Document 2 or the like, a character portion included in an image is extracted from a pixel region of the same color having a narrow width using a mathematical morphology by using the property that the character portion is composed of elongated line segments. The method of extracting as is disclosed.

JP 2002-42055 A "Study on character region extraction from scene image" (Rep., IEICE Technical Report PRU95-221 (1996-03), pp.39-46)

  As described above, in Patent Document 1 and the like, when a set of minimum circumscribed rectangles of pixel connection areas are arranged at substantially the same height, the pixel connection areas are likely to be characters, in other words, the degree that can be regarded as characters is high. Deciding. However, this is effective when one character such as English is composed of almost one pixel connection region. On the other hand, when one character is composed of a plurality of pixel connection regions as in Japanese, there is a technical problem that it is difficult to determine the character likeness based on the positional relationship of the minimum circumscribed rectangle.

  Further, as described above, in Patent Document 2 and the like, basically, a long and narrow line segment is determined as a character part. Therefore, noise affected by a background part having the same property and shape can be distinguished from a character part. It becomes technically difficult. Further, since it is necessary to set the maximum value of the line segment width of the character in advance, there is a possibility that the extraction ratio of the true character parts may be lowered depending on how the maximum value is set.

  The present invention has been made in view of the above-described conventional problems, for example, an image processing apparatus and method capable of appropriately extracting a character portion from an image such as a color image, and an optical character. It is an object to provide an identification apparatus and method.

  In order to solve the above-described problem, the image processing apparatus according to claim 1 detects a line segment width of a pixel connection portion where a plurality of pixels including a character portion indicating a character are connected on a plane of an image. Means and identification means for identifying the character portion based on a statistical distribution of the detected line segment width.

  In order to solve the above-described problem, an optical character identification device according to claim 14 has a line segment width of a pixel connection portion where a plurality of pixels including a character portion indicating a character are connected on a plane of an image. Detection means for optical detection, and identification means for identifying the character portion based on a statistical distribution of the detected line segment width.

  In order to solve the above-described problem, the image processing method according to claim 15 is a detection for detecting a line segment width of a pixel connection portion where a plurality of pixels including a character portion indicating a character are connected on a plane of the image. A step of identifying the character portion based on a statistical distribution of the detected line segment width.

  In order to solve the above-mentioned problem, an optical character identification method according to claim 16, wherein a line segment width of a pixel connection portion where a plurality of pixels including a character portion indicating a character are connected on a plane of an image is determined. A detection step for optical detection; and an identification step for identifying the character portion based on a statistical distribution of the detected line segment width.

  The operation and other advantages of the present invention will become apparent from the embodiments described below.

  Hereinafter, an image processing apparatus and method according to an embodiment of the present invention as the best mode for carrying out the invention will be described in order.

(Embodiment of Image Processing Device)
Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described.

  An embodiment according to the image processing apparatus of the present invention includes: a detecting unit configured to detect a line width of a pixel connection portion in which a plurality of pixels including a character portion indicating a character are connected on a plane of the image; Identification means for identifying the character portion based on a statistical distribution of line segment widths.

  According to the embodiment of the image processing apparatus of the present invention, the target image may be, for example, a color image including a character portion indicating a character (so-called character part). Alternatively, it may be a black and white image including a character portion. Alternatively, the target image may be a still image including a character part. Alternatively, it may be a moving image including a character part. Alternatively, one image including these character portions may be grouped into a plurality of images (so-called class images, clusters, and cluster images) depending on attributes and properties such as luminance components and chromaticity. The image may include a background portion (so-called background part) indicating the background image in addition to the character portion indicating the character.

  First, the line width of a pixel connection portion (so-called part) in which a plurality of pixels including a character portion indicating a character are connected is detected by the detection means on the plane of the image. Here, the pixel connection part according to the present embodiment is a binary image in which pixels are defined by binary identification information such as “0b (where“ b ”means binary”) and “1b”, for example. Connected pixels formed by being connected if the identification information of other pixels adjacent to one pixel is the same, such as black (1b), for example, and not connected if the identification information is different Means an image area to be a set of In addition, “detection” according to the present invention typically means directly performing “detection”, “measurement”, “measurement”, or the like on a statistic or parameter indicating the line segment width of a pixel connection portion. means. Furthermore, it may include indirectly performing “calculation”, “calculation”, “estimation”, and the like on the statistics and parameters indicating the line segment width of the pixel connection portion.

  Next, the character portion is identified from the image by the identifying means based on the statistical distribution of the detected line segment width. Here, “identification” according to the present invention typically means directly “identification”, “selection”, “distinguishment”, “extraction”, “recognition”, “selection” of a character part in an image. "," Selection "and so on. In addition, it means that the character part in the image means indirectly “identification”, “selection”, “distinguishment”, “extraction”, “recognition”, “selection”, “selection” May be. Therefore, for example, it is possible to determine the character likelihood based on a statistical distribution of line widths such as a frequency distribution indicating the frequency of appearance of the line width. Here, the “characteristic” according to the present invention means a degree that can be regarded as a character. The reason why the character-likeness can be determined is that, in the character shown in the image, it is highly likely that the character width is constituted by a substantially constant line segment width.

  As a result, for example, even when the character color or the character size is not known in advance, the character part in the image can be appropriately identified based on the statistical distribution of the line segment width.

  Specifically, as described above, in the characters shown in the image, the character width is based on the fact that the possibility of being configured with a substantially constant line segment width is remarkably high. However, it is possible to identify a character part constituted by a plurality of pixel connection parts. Furthermore, it is possible to eliminate almost or completely the need to preset the character size when identifying the character portion. Furthermore, even if the proportion of noise affected by the background portion is large, the character portion can be appropriately identified. Specifically, for example, a character portion with a small area such as a muddy point is displayed even if the ratio of noise affected by the background portion is large in the appearance frequency of the line segment width of the pixels constituting all the pixel connection portions. It is possible to identify appropriately.

  In one aspect of the embodiment of the image processing apparatus according to the present invention, the identification unit correlates the detected line segment width and the appearance frequency of the detected line segment width as the statistical distribution. Based on the relationship, the character portion and the background portion indicating the background image are identified.

  According to this aspect, for example, even when the character color or the character size is not known in advance, the character portion in the image is based on the correlation between the line width and the appearance frequency of the line width. Can be accurately identified with high accuracy.

  In another aspect of the embodiment of the image processing apparatus of the present invention, the true line segment width of the true character portion based on the frequency distribution indicating the appearance frequency of the line segment width as the statistical distribution. A predicting unit for predicting a line segment width, wherein the identifying unit compares the predicted true line segment width with the appearance frequency of the line segment width in one pixel connection part among a plurality of pixel connection parts; The character portion is identified based on

  According to this aspect, for example, even when the character color or the character size is not known in advance, the predicted true line segment width is compared with the appearance frequency of the line segment width in each pixel connection portion. Based on the above, the character portion in the image can be accurately identified with high accuracy.

  In the aspect according to the prediction means described above, the prediction means may predict a plurality of prediction values as the true line segment width.

  By configuring in this way, by having a plurality of predicted line segment widths, the character portion can be reduced even for special character fonts in which the vertical and horizontal character widths are significantly different, such as Mincho. It can be identified with high accuracy. As a result, even if the language of the character or the font cannot be predicted in advance, the character portion can be accurately identified with high accuracy.

  In the aspect which concerns on the prediction means mentioned above, the said identification means WHEREIN: When the ratio of the appearance frequency of the said predicted value is larger than a predetermined threshold in the appearance frequency of the line segment width in the said one pixel connection part, the said one pixel connection part May be identified as the character portion.

  If comprised in this way, a character part can be identified more accurately and appropriately based on the comparison with the ratio of the appearance frequency of a predicted value, and a predetermined threshold value. Here, the “predetermined threshold value” according to the present invention is a statistical value such as the appearance frequency, the distribution of the appearance frequency, the number, the degree of regularity, etc. In such a distribution, it may be specifically defined by experiment, theoretical, empirical, simulation, or the like so that a desired character portion can be identified.

  In another aspect of the embodiment of the image processing apparatus of the present invention, the detection means is a boundary that is a pixel that matches one of a plurality of contour patterns defined by a shape pattern of an edge of the pixel connection portion. Calculation means for calculating the line segment width using a pixel as a start point or an end point is provided.

  According to this aspect, the line width can be calculated with high accuracy by the calculation means while measuring the line width in units of the number of pixels with the boundary pixel as the start point or the end point.

  In the aspect according to the calculation means described above, the calculation means may calculate the line segment width along a direction defined corresponding to each of the plurality of contour patterns.

  According to this configuration, the line width is calculated with higher accuracy while the line width is measured in units of the number of pixels along each direction defined corresponding to each contour pattern by the calculation means. Is possible.

  In another aspect of the embodiment of the image processing apparatus of the present invention, a cluster dividing means for dividing the image into a plurality of clusters in advance, and the divided plurality of clusters into a binary image that can be expressed in binary. Conversion means for converting each is further provided.

  According to this aspect, one image including a character portion is divided into a plurality of clusters (so-called class images and cluster images) by the cluster dividing means according to attributes and properties such as luminance components and chromaticity. Next, the plurality of clusters are converted into binary images that can be expressed in binary by the conversion means. Therefore, even when the color of the character is not known in advance, the character portion in the image can be accurately and appropriately identified by the converted binary image.

  In another aspect of the embodiment of the image processing apparatus according to the present invention, the identification unit includes the appearance frequency of a plurality of contour patterns defined by the shape pattern of the edge of the pixel connection portion, or the distribution of the appearance frequencies of the contour patterns. The character portion is identified based on

  According to this aspect, for example, even when the character color or the character size is not known in advance, the character portion in the image is increased based on the appearance frequency of the contour pattern or the distribution of the appearance frequency of the contour pattern. Accurate and appropriate identification is possible. In particular, the character portion in the image may be identified by comparing the appearance frequency variance with a predetermined threshold. Here, the “predetermined threshold value” according to the present invention is an individual value obtained by experimental, theoretical, empirical, simulation, or the like so that a desired character portion can be identified in a statistical distribution such as the appearance frequency. It may be specifically defined.

  In another aspect of the embodiment of the image processing apparatus of the present invention, the identification unit identifies the character portion based on the degree of regularity of the shape in a plurality of pixel connection portions.

  According to this aspect, for example, even when the color of the character or the size of the character is not known in advance, the character portion in the image is highly accurate based on the degree of regularity of the shape in the plurality of pixel connection portions. And can be properly identified. In particular, the character portion in the image may be identified by comparing the degree of regularity of the shape in the plurality of pixel connection portions with a predetermined threshold value. Here, the “predetermined threshold value” according to the present invention is, for example, experimental so that a desired character portion can be identified in various statistical distributions indicating the degree of regularity of shapes in a plurality of pixel connection portions. It may be specified individually, theoretically, empirically, or by simulation.

  In another aspect of the embodiment of the image processing apparatus of the present invention, the identifying means identifies the character portion based on the number of pixel connection portions.

  According to this aspect, for example, even when the color of the character or the size of the character is not known in advance, the character portion in the image is accurately and appropriately based on the number of the plurality of pixel connection portions. Can be identified. In particular, the character portion in the image may be identified by comparing the number of pixel connection portions with a predetermined threshold value. Here, the “predetermined threshold value” according to the present invention is an experimental, theoretical, empirical, or the like so that a desired character portion can be identified in various statistical distributions indicating the number of pixel connection portions. Alternatively, it may be specifically defined by simulation or the like.

  In another aspect of the embodiment of the image processing apparatus of the present invention, the identification unit is configured to calculate the distance between the center of gravity of one pixel connection portion and the center of gravity of another pixel connection portion among a plurality of pixel connection portions. The character portion is identified based on the size.

  According to this aspect, for example, even when the character color or the character size is not known in advance, the size of the distance between the centroid of one pixel connection portion and the centroid of another pixel connection portion. Based on the above, the character portion in the image can be accurately identified with high accuracy. In particular, the character portion in the image may be identified by comparing the distance between the center of gravity of one pixel connection portion and the center of gravity of another pixel connection portion with a predetermined threshold. Here, the “predetermined threshold” according to the present invention refers to a desired character portion in various statistical distributions indicating the distance between the center of gravity of one pixel connection portion and the center of gravity of another pixel connection portion. In order to be able to be identified, it may be specifically defined by experiment, theoretical, empirical, simulation or the like.

  In another aspect of the embodiment of the image processing apparatus of the present invention, the identifying means primarily identifies a primary character portion among a plurality of pixel connection portions as the character portion, and identifies the character portion. The secondary character portion is further identified secondarily based on the similarity of the attribute or property with the primary character portion.

  According to this aspect, the character portion can be identified with high accuracy and appropriately by identifying the character portion by a two-stage method which is a different identification method.

(Embodiment of optical character recognition device)
Hereinafter, an optical character identification device according to an embodiment of the present invention will be described.

  An embodiment of the optical character identification device according to the present invention is a detection means for optically detecting a line segment width of a pixel connection portion where a plurality of pixels including a character portion indicating a character are connected on a plane of an image. And identification means for identifying the character portion based on a statistical distribution of the detected line segment width.

  According to the embodiment of the optical character identification device of the present invention, first, the line width of the pixel connection portion where a plurality of pixels including the character portion indicating the character are connected on the plane of the image by the detection means. Are detected optically.

  Next, the character portion is identified from the image by the identifying means based on the statistical distribution of the detected line segment width.

  As a result, in the optical character identification device, for example, even if the character color or the character size is not known in advance, the character part in the image is appropriately determined based on the statistical distribution of the line segment width. Can be identified.

(Embodiment of Image Processing Method)
Hereinafter, an image processing method according to an embodiment of the present invention will be described.

  According to an embodiment of the image processing method of the present invention, a detection step of detecting a line segment width of a pixel connection portion where a plurality of pixels including a character portion indicating a character are connected on the plane of the image, and the detected An identification step of identifying the character portion based on a statistical distribution of line segment widths.

  According to the embodiment of the image processing method of the present invention, it is possible to receive various benefits of the above-described embodiment of the image processing apparatus of the present invention.

  Incidentally, in response to the various aspects of the embodiment of the image processing apparatus of the present invention described above, the embodiment of the image processing method of the present invention can also adopt various aspects.

(Embodiment of optical character identification method)
Hereinafter, an optical character identification method according to an embodiment of the present invention will be described.

  An embodiment of the optical character identification method according to the present invention includes a detection step of optically detecting a line width of a pixel connection portion where a plurality of pixels including a character portion indicating a character are connected on a plane of an image. And an identification step for identifying the character portion based on a statistical distribution of the detected line segment width.

  According to the embodiment of the optical character identification method of the present invention, it is possible to receive various benefits of the above-described embodiment of the optical character identification device of the present invention.

  Incidentally, in response to the various aspects of the embodiment of the optical character identification device of the present invention described above, the embodiment of the optical character identification method of the present invention can also adopt various aspects.

  Such an operation and other advantages of the present embodiment will be clarified from examples described below.

  As described above, according to the embodiment of the image processing apparatus and method of the present invention, the detection unit and the process, and the identification unit and the process are provided. As a result, for example, even when the character color or the character size is not known in advance, the character part in the image can be appropriately identified based on the statistical distribution of the line segment width.

  As described above, according to the embodiment of the optical character identification device and method of the present invention, the detection unit and the process, and the identification unit and the process are provided. As a result, in the optical character identification device, for example, even if the character color or the character size is not known in advance, the character part in the image is appropriately determined based on the statistical distribution of the line segment width. Can be identified.

It is the block diagram which showed the basic composition of the image processing apparatus 100 based on a present Example. It is the schematic diagram which showed typically the width | variety of the character shown in the image used as the object of an image processing based on a present Example. 3 is a flowchart illustrating a series of overall image processing performed by the image processing apparatus according to the present exemplary embodiment. It is the schematic diagram which showed typically the several class image produced | generated after clustering the image used as the object of an image processing based on a present Example. It is the schematic diagram which showed typically the relationship between the pixel which concerns on a present Example, and part ID. It is the schematic diagram which showed typically the position of the black pixel in the outline unit, and the position of a white pixel based on a present Example. It is the schematic diagram which showed typically a mode that the boundary pixel to which the outline unit ID was provided based on a present Example was started, and the number of the black pixels continuous in the edge direction was measured. One and other graphs (FIG. 8 (a) and FIG. 8A) showing the frequency distribution indicating the frequency of appearance of the line segment width for all parts when selecting an element of a histogram having a high frequency of appearance according to the present embodiment. FIG. 8B). According to the present embodiment, a frequency distribution (FIG. 9A) showing the frequency of appearance of the line width of the pixel group constituting all parts extracted from the original class image, the pixel group constituting the true character part The frequency distribution (FIG. 9B) showing the appearance frequency of the line width and the frequency distribution (FIG. 9C) showing the appearance frequency of the line width constituting the background part. It is the schematic diagram which showed typically the change from a background part to a character part candidate based on a present Example. It is the schematic diagram which showed typically a mode that the character image was produced | generated from all the class images based on a present Example. It is the one and other schematic diagram (Drawing 12 (a) and Drawing 12 (b)) which showed a mode that a character portion concerning a comparative example was extracted. It is the schematic diagram which showed typically a mode that the character image was produced | generated from all the class images based on 2nd Example. It is the schematic diagram which showed typically a mode that the character image was produced | generated from all the class images based on 3rd Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image processing apparatus 110 Character area extraction means 120 Pixel clustering means 130 Class image creation means 140 Character selection means 141 Parts extraction means 142 Parts line segment width calculation means 143 Character parts line segment width prediction means 144 Character parts candidate selection means 145 Character parts Determination means 150 Character image generation means 160 Character recognition means

  Embodiments of the present invention will be described below with reference to the drawings.

(Example of image processing apparatus)
An embodiment of an image processing apparatus according to the present invention will be described with reference to FIGS.

(1) Basic Configuration First, the basic configuration of the image processing apparatus 100 according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram illustrating the basic configuration of the image processing apparatus 100 according to the present embodiment.

  As shown in FIG. 1, the image processing apparatus 100 according to the present embodiment includes a character area extraction unit 110, a pixel clustering unit 120, a class image creation unit 130, a character selection unit 140, a character image generation unit 150, and a character. Recognizing means 160 is provided. The character selection unit 140 includes a part extraction unit 141, a part line segment width calculation unit 142, a character part line segment width prediction unit 143, a character part candidate selection unit 144, and a character part determination unit 145. Yes. A specific example of the “detecting means” according to the present invention is constituted by the part line segment width calculating means 142. A specific example of the “identifying means” according to the present invention is configured by the character part candidate selecting means 144. A specific example of the “cluster dividing unit” according to the present invention is configured by the pixel clustering unit 120. A specific example of the “conversion unit” according to the present invention is configured by the class image creation unit 130.

(2) Operating principle-Overall image processing-
Next, referring to FIG. 2 and FIG. 3, as an operation principle of the image processing apparatus 100 according to the present embodiment, for example, a character portion (that is, a character part) is extracted from an image such as a color image, and the character is extracted from the image. A series of flow of image processing for recognizing a part (that is, a character part) will be described. FIG. 2 is a schematic diagram schematically showing the width of characters shown in an image to be subjected to image processing according to the present embodiment. FIG. 3 is a flowchart illustrating a series of overall image processing performed by the image processing apparatus according to the present exemplary embodiment.

  As an overview of the image processing according to the present embodiment, first, the pixels constituting the image such as a color image including a character portion (that is, a character part) indicating a character are grouped according to an attribute such as a luminance component, and the like. For each group, a pixel connection area, that is, a part is extracted. Here, the “pixel connection region” according to the present embodiment is, for example, a pixel in which pixels are defined by binary identification information such as “0b (where“ b ”means binary”) and “1b”. In the value image, connected if the identification information of other pixels adjacent to one pixel is the same, such as black (1b), for example, and connected if the identification information is different. This means an image area that is a set of pixels. Next, it is possible to detect the line width of the extracted part and determine the characterity (that is, the degree that can be regarded as a character) based on the frequency distribution indicating the appearance frequency of the line width. is there. This is because, as shown in FIG. 2, in the character shown in the image, the possibility that the character width is constituted by a substantially constant line segment width is remarkably high. Thereby, even when the character color or the character size is not known in advance, the character part can be appropriately extracted from the image based on the frequency distribution indicating the appearance frequency of the line segment width.

  Next, referring to FIG. 3, an outline of a series of overall image processing steps for extracting a character part from an image and recognizing the character part from the image will be outlined.

  As shown in FIG. 3, first, an image such as a color image including a character portion is input to the image processing apparatus 100 (step S10). The color image attributes and properties may take various forms. Specifically, the size of the color image, the format for constructing the color image, and the compression format such as JPEG and GIF may take various forms. The input color image is not limited to a still image, and may be a moving image such as a television image.

  Next, a rough character region is extracted from the input color image by the character region extracting means 110 (step S20). Here, the “character area” according to the present embodiment means a part of an image area that can include a character part in an image. Unless otherwise specified, this “character area” is simply referred to as an image, and the subsequent processing steps are performed only on this “character area”.

  Next, the pixels constituting the image are grouped into groups (that is, classes or clusters) by the luminance component by the pixel clustering means 120 (so-called clustering) (step 30). Here, the number of classes is two or more, and there is no upper limit. Note that the clustering is not limited to the luminance component of the pixel, and may be performed using, for example, a color component.

  After the clustering is performed, the class image creation means 130 decomposes the original unprocessed image for each class and generates a binary image (hereinafter referred to as “class image” as appropriate) for each class. . Here, the “binary image” according to the present embodiment means that the pixels constituting the image are, for example, white (for example, “0b”: where “b” means a subscript indicating a bit) and black (for example, “ 1b ") means an image shown in one of two pieces of identification information. The subsequent image processing from step S40 to step S80 may be performed independently for each class image.

  Next, a pixel connection area (hereinafter referred to as “parts” as appropriate) is extracted from the class image by the part extraction unit 141 included in the character selection unit 140 (step S40).

  Next, the extracted line segment widths of the parts are calculated by the part line segment width calculating unit 142 (step S50). In addition, this step S50 is comprised by step S51, step S52, and step S53 so that it may mention later.

  Next, the frequency distribution (so-called histogram) indicating the frequency of appearance of the line segment width is generated by the character part line segment width predicting unit 143 based on all the line segment widths in the calculated plurality of parts, and is displayed on the image. The line width of the character part that is the included character part is predicted (step S60).

  Next, the character part candidate selecting means 144 compares the predicted line width of the character part with the frequency distribution generated for each part and indicating the frequency of appearance of the line width for each part, A plurality of parts are selected as candidate character parts and candidate background parts that can be estimated with a relatively high probability as true background parts (step S70). Here, “candidate character parts” according to the present embodiment means character parts that can be estimated with a relatively high probability as true character parts. Called. Further, the “candidate background part” according to the present embodiment means a background part that can be estimated with a relatively high probability as a true background part, and is hereinafter referred to as a “background part candidate” as appropriate.

  Next, the character part determining unit 145 again selects a plurality of background part candidates that are preferably changed to the character part candidates (in other words, those that should be “upgraded”), and the relative character is selected as a true character part. Parts that can be estimated with high probability are finally determined (step S80). In particular, the change from the background part candidate to the character part candidate may be performed based on the degree of similarity between the brightness of the background part candidate and the brightness of the character part candidate. Alternatively, the position of the center of gravity for each part may be compared, and the distance of the position of the center of gravity for each part may be determined based on the size.

  Next, the character image generator 150 generates a character image by superimposing all the character parts determined in each class image (step S90). Here, the “character image” according to the present embodiment may mean a binary image in which pixels constituting the character part are displayed in black and pixels not constituting the character part are displayed in white.

  Next, the character recognition unit 160 inputs the generated character image into an OCR (Optical Character Recognition) device, and character recognition is performed (step S100).

(3) Principle of operation-Details of image processing-
Next, referring to FIG. 3 described above as appropriate in addition to FIGS. 4 to 11, a character portion (that is, a character part) is extracted from a color image as an operation principle of the image processing apparatus 100 according to the present embodiment. The details of each process constituting the image process for recognizing the character portion (ie, the character part) from the image will be described.

(3a) 1st process group (from step S10 to S30)
First, referring to FIG. 3 described above as appropriate in addition to FIG. 4, the image processing apparatus 100 according to the present embodiment performs image processing from input of an original color image to grouping (so-called clustering). The first half process from a certain step S10 to S30 will be described. FIG. 4 is a schematic diagram schematically showing a plurality of class images generated after the images to be subjected to image processing are clustered according to the present embodiment.

(Step S10: Image input)
As shown in FIG. 3 described above, first, an image such as a color image including a character portion is input to the image processing apparatus 100 (step 10). Specifically, a general color image may be input to the image processing apparatus 100, and the input color image may be stored in storage means such as a memory or storage means (not shown), for example. The size and compression format of the color image are not particularly limited. If the image is a compressed image such as JPEG, the following various processes from step S20 to S100 are performed after decoding. The input image is not limited to a still image, and may be a moving image such as a television image.

(Step S20: Extraction of character area)
Next, a rough (approximate) character region is extracted from the input color image by the character region extraction means 110 (step 20). Specifically, since a character portion in an image usually includes many edges (that is, high-frequency components of the image), a rough character region may be extracted based on the edge distribution degree. In addition, since a character is often composed of a single color or several colors, a rough character region may be extracted based on the degree of density of pixels having similar luminance. These extraction methods are known as known techniques. When the target is not a still image but a moving image such as a TV video, the character portion has the property of being continuously displayed for several seconds at the same position in the screen. As in 2004-364234), the character region may be extracted by observing the edge extraction result of each frame in the time direction. Unless otherwise specified, this “character area” is simply referred to as an image, and the subsequent processing steps are performed only on this “character area”.

(Step S30: Pixel Clustering by Luminance Component)
Next, the pixel clustering means 120 groups the pixels constituting the image into groups (classes) based on the luminance components (so-called clustering) (step 30). Specifically, the pixels constituting the image are clustered by luminance components and classified into a plurality of classes. The number of classes to be classified is at least 2 classes, and no upper limit is specified. However, when the number of classes is too large, the load of image processing increases, so 3 to 5 classes may be preferably set. The pixel clustering method is well known, and the discriminant analysis method (Otsu, “Automatic threshold selection method based on discriminant and least square criterion”, Theory of Science, vol.63-D, no.4, pp.349 -356, 1980) or dynamically depending on surrounding pixels (salt, “dynamic binarization method for detecting characters in a scene”, IEICE D, vol.J71-D , no.5, pp.863-873, 1988). Pixel clustering may be performed based on, for example, color components in addition to or instead of luminance components. After clustering, as shown in FIG. 4, some of the plurality of pixels belonging to each class are displayed as black (for example, “1b”), and the other pixels among the plurality of pixels are white ( For example, binary images displayed as “0b”) may be created for the number of classes. Hereinafter, in order to simplify the description, a binary image whose class ID (class identification number) is “N (where“ N ”is a natural number)” is referred to as a “class N image”. When there is no need to specify the class ID, it is simply referred to as “class image”. Specifically, three class images are created from the original image shown in the upper part of FIG. 4, for example, composed of black, white, and gray. That is, the three class images are a “class 1 image” whose class ID is “1” shown in the lower left part of FIG. 4 and a class ID which is “2” in the lower center part of FIG. In addition, a “class 2 image” having a white character portion, and a “class 3 image” having a class ID “3” shown in the lower right part of FIG. 4 and a black character portion. It is.

  The subsequent image processing from step S40 to step S80 may be performed independently for each class image that is a binary image.

(3b) Second processing group (Steps S40 and S50)
Next, referring to FIG. 3 described above as appropriate in addition to FIG. 5 to FIG. 7, image processing from part extraction to part line width calculation in the image processing apparatus 100 according to the present embodiment. The middle process of steps S40 and S50 will be described. FIG. 5 is a schematic diagram schematically showing the relationship between the pixel according to the present embodiment and the part ID. FIG. 6 is a schematic diagram schematically showing the positions of black pixels and white pixels in the contour unit according to the present embodiment.

(3b-1: Part extraction: Step S40)
First, as shown in FIG. 3 described above, the part extraction unit 141 included in the character selection unit 140 extracts a pixel connection region (hereinafter referred to as “part” as appropriate) from the class image (step S40). ). Specifically, a part of a pixel indicating black in the class image (hereinafter referred to as “black pixel” as appropriate) is extracted. For example, in the class image shown in FIG. 5, 11 parts are extracted. Specifically, eleven parts corresponding to the radical or part of the radical of the Chinese character are extracted from the “forest park” which is the character shown in the class image in FIG. The black pixel connection condition may be, for example, a four-connection method in which black pixels located on the top, bottom, left, and right adjacent to a reference black pixel are connected. Alternatively, the black pixel connection condition may be an eight connection method in which black pixels located obliquely in addition to the upper, lower, left, and right adjacent to the reference black pixel are connected. When a part is extracted, a part ID is assigned to each pixel, and this data is stored in a storage means (not shown). Here, “part ID” according to the present embodiment means an identification number that can uniquely associate each pixel with a part to which each pixel belongs.

  Specifically, referring to FIG. 5, for example, in the case where a character “Forest Park” is shown in the class image, the relationship between the pixel and the part ID is focused on the character portion “Forest”. explain. When the pixels around the “tree” that is the three character parts included in the “forest” that is the character part shown in the upper part of FIG. 5 are enlarged, “8” shown in the lower part of FIG. As indicated by the pixel “× 8”, a part ID is given. That is, in the pixel of “8 × 8”, when the pixel located at the lower left corner is defined as the pixel (1, 1) on the X axis and the Y axis, the pixel (4, 7), the pixel (4, 8), The four pixels that are the pixel (5, 7) and the pixel (5, 8) are assigned “1” that is the part ID of the upper character portion “tree” in the character portion “forest”, respectively. In substantially the same manner, pixel (1, 2), pixel (2, 2), pixel (3, 2), pixel (1, 3), pixel (2, 3), pixel (3, 3) are 6 Each pixel is assigned “2”, which is the part ID of the lower left character portion “tree” in the character portion “forest”. In substantially the same manner, pixel (6, 2), pixel (7, 2), pixel (8, 2), pixel (6, 3), pixel (7, 3), pixel (8, 3) 6 Each pixel is assigned “3”, which is the part ID of the character portion “tree” on the lower right side in the character portion “forest”.

(3b-2 Calculation of part line width: part S50)
Next, as shown in FIG. 3 described above, the extracted line segment widths of the plurality of parts are calculated by the part line segment width calculating unit 142 (step S50). Specifically, the parts extracted as in step S40 described above can be broadly classified into character parts that constitute characters (or a part thereof) and other parts (hereinafter referred to as “background parts” as appropriate). . As shown in FIG. 2 above, as the characteristics seen in the character part, when the line width constituting the part indicates the same character (or character string), the appearance frequency of the line width having almost the same value is remarkable. Get higher.

  In the present embodiment, paying attention to this basic feature, based on the frequency distribution indicating the frequency of appearance of the line segment width constituting the part, it is determined whether each extracted part is a character part, Only character parts are selected and identified from all the extracted parts.

(3b-2-1 Measurement of part line width: step S51)
In this step S50, first, the line segment width of the part is measured. For this measurement, eight types of pixel filters as shown in FIG. 6 (hereinafter referred to as “contour unit” as appropriate) are used. A specific example of the “contour pattern” according to the present invention is constituted by the contour unit.

  Eight types of contour unit IDs are assigned to the contour unit, and each pixel is composed of a white pixel or a black pixel. Specifically, as shown in FIG. 6, in the contour unit whose contour unit ID is “1”, 3 × 3 pixels of three pixels in the X-axis direction and three pixels in the Y-axis direction. When the pixel whose reference position is the pixel in the lower left corner in the range is the pixel (1, 1), the pixel (1, 1), the pixel (2, 1), the pixel (3, 1), the pixel (1, 2), the pixel (2, 2), and the pixel (3, 2) are black pixels, and the pixel (1, 3), the pixel (2, 3), and the pixel (3, 3) are white pixels. is there. In a similar manner, in the contour unit whose contour unit ID is “2”, pixel (1, 1), pixel (2, 1), pixel (3, 1), pixel (2, 2), pixel (3 2) and the pixel (3, 3) are black pixels, and the pixel (1, 2), the pixel (1, 3), and the pixel (2, 3) are white pixels. In a similar manner, in the contour unit whose contour unit ID is “3”, pixel (2, 1), pixel (3, 1), pixel (2, 2), pixel (3, 2), pixel (2 3) and the pixel (3, 3) are black pixels, and the pixel (1, 1), the pixel (1, 2), and the pixel (1, 3) are white pixels. In a similar manner, in the contour unit whose contour unit ID is “4”, pixel (3, 1), pixel (2, 2), pixel (3, 2), pixel (1, 3), pixel (2 3) and the pixel (3, 3) are black pixels, and the pixel (1, 1), the pixel (1, 2), and the pixel (2, 1) are white pixels. In a similar manner, in the contour unit whose contour unit ID is “5”, pixel (1, 2), pixel (2, 2), pixel (3, 2), pixel (1, 3), pixel (2 3) and the pixel (3, 3) are black pixels, and the pixel (1, 1), the pixel (1, 2), and the pixel (1, 3) are white pixels. In a similar manner, in the contour unit whose contour unit ID is “6”, pixel (1, 1), pixel (1, 2), pixel (2, 2), pixel (1, 3), pixel (2 3) and the pixel (3, 3) are black pixels, and the pixel (2, 1), the pixel (3, 1), and the pixel (3, 2) are white pixels. In a similar manner, in the contour unit whose contour unit ID is “7”, pixel (1, 1), pixel (2, 1), pixel (1, 2), pixel (2, 2), pixel (1 3) and the pixel (2, 3) are black pixels, and the pixel (3, 1), the pixel (3, 2), and the pixel (3, 3) are white pixels. In a similar manner, in the contour unit whose contour unit ID is “8”, pixel (1, 1), pixel (2, 1), pixel (3, 1), pixel (1, 2), pixel (2 2) and the pixel (1, 3) are black pixels, and the pixel (3, 2), the pixel (2, 3), and the pixel (3, 3) are white pixels.

(Measurement of line width by 3b-2-2 contour unit)
Next, with reference to FIG. 7, a method for measuring the line segment width by the contour unit will be described. FIG. 7 is a schematic diagram schematically showing a state in which the number of black pixels continuous in the edge direction is measured starting from the boundary pixel to which the contour unit ID is assigned according to the present embodiment. It is.

(3b-2-2-1 Assigning of contour unit ID to boundary pixel: step S52)
First, the black pixels constituting each extracted part are distinguished into internal pixels and boundary pixels. An internal pixel means a pixel in which the 3 × 3 pixel range adjacent to itself including itself is a black pixel. Alternatively, the internal pixel may mean a pixel in which surrounding 3 × 3 pixels have the same part ID. On the other hand, a boundary pixel means a pixel that cannot be defined as an internal pixel.

  Next, the 3 × 3 pixel arrangement centering on the boundary pixel is compared with the pixel arrangement of each contour unit, and if there is a completely identical pixel arrangement, the outline unit ID corresponding to the coincident pixel arrangement Is added to the boundary pixel.

  If the 3 × 3 pixel arrangement centering on the boundary pixel does not match the pixel arrangement of any contour unit, “0” is given. In other words, when “0” is given to a boundary pixel, it can be said that there is no contour unit corresponding to the boundary pixel. This is performed for all boundary pixels.

(3b-2-2-2 Counting from boundary pixels: Step S53)
Next, the boundary pixel to which any contour unit ID other than “0” is assigned is used as the starting point, and the number of black pixels continuous in the edge direction of the contour unit is counted. Here, the “edge direction” according to the present embodiment means a direction perpendicular to the boundary line between white and black of the contour unit and a direction from the white side to the black side. Specifically, for example, the edge direction of the contour unit whose contour unit ID is “1” is a downward direction that is a direction from the white side to the black side. The edge direction of the contour unit whose contour unit ID is “2” is the direction of 45 degrees on the lower right. The edge direction of the contour unit whose contour unit ID is “3” is the direction of 90 degrees to the right. The edge direction of the contour unit whose contour unit ID is “4” is the direction of 45 degrees on the upper right. The edge direction of the contour unit whose contour unit ID is “5” is the direction of 0 degrees above. The edge direction of the contour unit whose contour unit ID is “6” is the direction of 45 degrees on the upper left. The edge direction of the contour unit whose contour unit ID is “7” is the direction of 90 degrees to the left. The edge direction of the contour unit whose contour unit ID is “8” is the direction of 45 degrees on the lower left.

  The count (measurement) is stopped (stopped) when there are no black pixels, and the number of black pixels continuously arranged from the start point is measured. This measured value is used as the line segment width, and is given to the boundary pixel that is the starting point. This measurement process is performed for all boundary pixels. However, the measurement process is performed on the boundary pixel to which the contour unit ID other than “0” is assigned. A specific example is shown in FIG. This is an enlargement of a part of the character part of the part ("I" shape). In the enlarged view of the pixel with the “contour unit ID” located in the center of FIG. 7, the pixel represented by “•” is the internal pixel and the pixel represented by the number of the contour unit ID is the boundary pixel. It is. Among these, the boundary pixel to which the contour unit ID other than “0” is assigned is used as a start point, and the number of black pixels continuous in the edge direction is measured. A state in which the measurement value is given to the boundary pixel that is the start point is an enlarged view of the “line segment width” located on the right side of FIG. In the specific example shown in FIG. 7, the number of continuous black pixels in the horizontal direction and the oblique direction may be measured by a substantially similar method. Alternatively, in each of the square pixels in the specific example shown in FIG. 7, the distance between the pixels in the diagonal direction is “root twice” the distance between the horizontal (or vertical) pixels (in other words, the horizontal Since the product of the distance between the (or vertical) pixels and the square root of 2 (so-called twice the root), when measuring black pixels in an oblique direction, the measurement value (count value) is “ The line width can be obtained by multiplying the route twice.

  More specifically, in the upper center portion of FIG. 7, an 8 × 8 pixel range obtained by enlarging the pixels on the upper side of the character portion “A” is shown, and “2” is given as the contour unit ID. Five boundary pixels are shown. In addition, five boundary pixels assigned “6” as the contour unit ID are shown. In addition, twelve boundary pixels assigned “0” as the contour unit ID are shown. In addition, 22 internal pixels represented by “·” are shown. Here, in the 8 × 8 pixel range of 8 pixels in the X-axis direction and 8 pixels in the Y-axis direction, the pixel whose reference position is the pixel in the lower left corner is the pixel (1, 1). Then, the number of black pixels having the line segment width is measured using the pixel (4, 7) as the boundary pixel as a starting point. First, since the pixel (4, 7) as the boundary pixel is given “2” as the contour unit ID as shown in the upper center part of FIG. 7, the edge direction is 45 degrees on the lower right. Direction. When the number of black pixels is measured in the direction of 45 degrees on the lower right, the pixel itself (4, 7), pixel (5, 6), pixel (6, 5), pixel (7, 4) It can be seen that there are a total of four. Accordingly, 8 × 8 pixels in the X-axis direction and 8 × 8 pixels in the Y-axis direction are used to give a measurement value corresponding to the line width to the boundary pixel shown in the upper right side of FIG. In the pixel range of 8, in substantially the same manner, when the pixel whose reference position is the pixel in the lower left corner is the pixel (1, 1), the pixel (4, 7) as the boundary pixel has a measurement value as “4” is assigned. In this case, since black pixels in an oblique direction are measured, a measurement value (count value) obtained by “double the route” may be added as a line segment width.

  In substantially the same manner, an 8 × 8 pixel range obtained by enlarging the lower pixel of the character portion “I” is shown in the lower center portion of FIG. 7, and “3” is assigned as the contour unit ID. Eight boundary pixels are shown. In addition, eight boundary pixels assigned “7” as the contour unit ID are shown. In addition, 16 internal pixels represented by “·” are shown. Here, in the 8 × 8 pixel range, when the pixel whose reference position is the pixel in the lower left corner is the pixel (1, 1), the line segment with the pixel (6, 2) as the boundary pixel as the starting point Measure the number of black pixels that will be the width. First, the pixel (6, 2), which is a boundary pixel, is given “7” as the contour unit ID as shown in the lower center part of FIG. 7, so the edge direction is 90 degrees to the left. Direction. When the number of black pixels in the direction of 90 degrees to the left is measured, the pixel itself (6, 2), pixel (5, 2), pixel (4, 2), pixel (3, 2), which is the pixel itself, is measured. It can be seen that there are a total of four. Accordingly, in the 8 × 8 pixel range for giving the measurement value corresponding to the line width to the boundary pixel shown in the lower right portion of FIG. When the pixel serving as the reference position is the pixel (1, 1), the pixel (6, 2) that is the boundary pixel is given “4” as the measurement value.

(3c) Third processing group (steps S60 to S70)
Next, in addition to FIG. 8 and FIG. 9, with reference to FIG. 3 described above as appropriate, from prediction of line widths of character parts to selection of character part candidates in the image processing apparatus 100 according to the present embodiment. The third processing group of steps S60 to S70, which is the image processing of FIG. Here, FIG. 8 shows one and other graphs showing the frequency distribution indicating the frequency of appearance of the line segment width for all parts when selecting an element of a histogram having a high frequency of appearance according to the present embodiment. (FIG. 8A and FIG. 8B). FIG. 9 shows a frequency distribution (FIG. 9 (a)) indicating the frequency of appearance of the line width of the pixel group constituting all parts extracted from the original class image according to the present embodiment, constituting a true character part. The frequency distribution (FIG. 9B) showing the appearance frequency of the line width of the pixel group to be performed and the frequency distribution (FIG. 9C) showing the appearance frequency of the line width constituting the background part.

(3c-1) (Step S60: Prediction of line width of character part)
As shown in FIG. 2 above, as the characteristics seen in the character part, when the line width constituting the part indicates the same character (or character string), the appearance frequency of the line width having almost the same value is remarkable. Get higher.

  In the present embodiment, paying attention to this basic feature, based on the frequency distribution indicating the frequency of appearance of the line segment width constituting the part, it is determined whether each extracted part is a character part, Only character parts are selected and identified from all the extracted parts. As the previous stage, in this step S60, the line segment width of the character part is predicted.

(Histogram of line width of all parts)
First, a frequency distribution (histogram) indicating the frequency of appearance of line widths for all parts is created from the line widths measured from all the extracted parts, and the frequency of occurrence is significantly higher in the frequency distribution. Select only “s” histogram elements (that is, line segment widths) to be peak points. As shown in FIG. 8 (a), the specific selection method is to select the number “s” elements, and to select the upper “s” -th element of the frequency of appearance frequency (that is, the line segment width). ) May be used. Alternatively, as shown in FIG. 8B, a technique may be used in which all elements within a predetermined range centered on the first element (that is, the line segment width) of the frequency of appearance frequency may be selected. Or, basically, another method of preferentially selecting an element having a relatively high frequency of appearance may be used. In FIG. 8A, “s” is set to “3”, but the present invention is not limited to this. However, if the “s” -th is too large, the accuracy in sorting the character parts, which is the latter half of the process, is affected, so it is preferable to suppress the “s” -th to about several points. The element thus selected, that is, the line segment width is set to WDk (1 ≦ k ≦ s) and used as a predicted value of the line segment width of the character part. In addition, by selecting several “s” to select, even for character fonts with greatly different line widths in the vertical and horizontal directions, such as characters in Mincho font, for example, There is an effect that the predicted value of the line width of the part can be set appropriately.

(3c-2) Step S70 Selection of character part candidates
Next, a line width histogram is created for each part.

(Histogram of line width for each part)
As described above, FIG. 9A shows a frequency distribution indicating the frequency of appearance of the line widths of the pixel groups constituting all parts extracted from the original class image, and FIG. FIG. 9C shows a frequency distribution indicating the frequency of appearance of the line width of the pixel group constituting the character part, and FIG. 9C shows the frequency distribution indicating the frequency of appearance of the line width constituting the background part.

(Qualitative trend)
In the frequency distribution indicating the appearance frequency of the line segment width of the pixel group constituting the true character part shown in FIG. 9B, the line of the pixel group constituting all the parts shown in FIG. 9A. In the range of the line width of WDk (1 ≦ k ≦ s), which is the range of the line width that can be determined based on the frequency of appearance of the width, the frequency of the appearance frequency tends to be remarkably increased.

  On the other hand, in the case of the background part shown in FIG. 9C, the frequency of appearance frequency is dispersed on average, and the frequency of appearance frequency is noticeable in the above-described range of the line segment width from WD1 to WD3. We cannot read the tendency to increase. In the range of the line segment width from WD1 to WD3, candidate character parts (hereinafter referred to as the following) that can be estimated with a relatively high probability as true character parts based on the tendency that the frequency of appearance frequency is remarkably increased. If necessary, this is called “character part candidate”). In addition, in the range of the line segment width from WD1 to WD3, candidate background parts that can be estimated with a relatively high probability as true background parts based on the tendency that the frequency of appearance frequency is remarkably increased ( Hereinafter, the “background part candidate” is appropriately selected.

  Specifically, in the appearance frequency of the line segment width of the pixels constituting the part p, the ratio of the frequency of the appearance frequency up to WDk (1 ≦ k ≦ s) to the total frequency is calculated, and this is set to a predetermined value. The character part candidate and the background part candidate may be selected depending on whether or not the threshold value T is greater.

(Determination of character parts and background parts)
When the following conditional expression (1) is satisfied, the part p is determined as a character part candidate.

…… （１）
他方、次の条件式（２）を満たす場合、パーツｐを背景パーツ候補と判断する。

(1)
On the other hand, when the following conditional expression (2) is satisfied, the part p is determined as a background part candidate.

…… （２）
ここでＦｐ（Ｗ）はパーツpの線分幅ヒストグラムにおける要素Wの度数を指し、Ｎｐは全度数の合計を指す。

(2)
Here, Fp (W) indicates the frequency of the element W in the line segment width histogram of the part p, and Np indicates the sum of all frequencies.

(3d) Fourth processing group (steps S80 to S100)
Next, referring to FIG. 3 described above as appropriate in addition to FIG. 10 and FIG. 11, step S80 which is image processing from the determination of the character parts to the character recognition in the image processing apparatus 100 according to the present embodiment. To S100 will be described. FIG. 10 is a schematic diagram schematically showing the change from the background part to the character part candidate according to the present embodiment. FIG. 11 is a schematic diagram schematically illustrating how character images are generated from all class images according to the present embodiment.

(3d-1) Step S80 Determination of character parts
As shown in FIG. 3 described above, in step S70, character part candidates are selected based on the line segment width of the parts. In addition to or instead of this, in step S80, it is preferable to compare the character part candidate selected in step S70 with the background part candidate, and make a new determination as a character part candidate from the background part candidates ( In other words, it is preferable to identify parts.

  Specifically, a background part candidate having a small distance (for example, Euclidean distance) between the center of gravity with the character part candidate may be determined again as a character part candidate, and the background part candidate may be changed as a character part candidate. In addition to or instead of this, a background part candidate having a relatively high similarity in various attributes, properties, and characteristics such as luminance or chromaticity is newly determined as a character part candidate, and the background part candidate is determined as a character part. You may change as a candidate.

  This is shown in FIG. Specifically, the original image including the character part and the background part shown at the top in FIG. 10 is selected in step S70 based on the line width of the part. The character part “Hayashi” shown in the center of FIG. 10 is selected as a character part candidate. Next, in step S80, as described above, based on the distance between the centers of gravity of the character part candidates and the similarity of various attributes such as luminance or chromaticity, properties, and characteristics, the lower side in FIG. Parts that were background part candidates, such as “Tree” in the lower right part of “Forest”, part of “Public”, part of “Gion”, etc. Be changed. In this way, the character part candidate selected in step S70 and the character part candidate changed from the background part candidate in step S80 are determined as the final true character parts.

As a result, it is possible to extract a true character part almost or completely. Specifically, for example, a character part with a small area, such as a muddy point, is an accurate character part candidate even if the proportion of noise affected by the background part is large in the appearance frequency of the line width of the pixel group constituting all parts. It is possible to judge.
(3d-2) Step S90 Generation of Character Image As shown in FIG. 3 described above, steps S40 to S80 are independent of the class image (ie, binary image) classified in step S30. Are performed respectively. Then, in the processing performed for each of these class images, a character image is generated using all the selected character parts.

  An example is shown in FIG. As shown in FIG. 11, when the original image is clustered into N by luminance and character parts are selected in each class image, a character image in which those character parts are integrated into one is generated.

As a result, a character image including all true character parts can be generated even for an image having a character string composed of a plurality of character colors. The character image may be a binary image in which a true character part is black and a part other than the true character part is white for input to an optical character recognition (OCR) device described later. Or, when character recognition by the character recognition (OCR) device is not required, the color of the true character part and the color of the part other than the true character part are not limited to this, and conform to other image formats. You may do it.
(3d-3) Step S100 Character recognition
Next, as shown in FIG. 3 described above, in step S100, the generated character image is recognized by an OCR (optical character recognition) device to obtain a character code. Existing techniques may be used for the OCR technology itself. The identified character code may be assigned to the original image as metadata, for example, and applied to image search or the like.

(4) Examination of operation and effect of this embodiment Next, the operation and effect of the embodiment of the image processing apparatus according to the present invention will be examined with reference to FIG. FIG. 12 is one and other schematic diagrams (FIGS. 12A and 12B) schematically showing how the character portion is extracted according to the comparative example.

  In one comparative example, the character portion is extracted by the following method. First, in a class image clustered by color components, a pixel connection region in which pixels are connected is extracted, and a rectangle that has the smallest area and size among rectangles circumscribing the pixel connection region (hereinafter referred to as “minimum” as appropriate). (Referred to as a circumscribed rectangle). Next, character-likeness (degree that can be regarded as a character) is determined from the positional relationship of the acquired rectangles. This is because the character parts in the image are almost the same size and tend to line up at almost the same height, so the same characteristic can be seen in the minimum circumscribed rectangle that circumscribes the character part. Because it is.

  Certainly, in one comparative example, when the set of minimum circumscribed rectangles of the pixel connection area is arranged at substantially the same height, it is determined that the pixel connection area seems to be a character, in other words, it can be regarded as a character. Yes. However, this is effective when one character such as English is composed of almost one pixel connection region. On the other hand, when one character is composed of a plurality of pixel connection regions as in Japanese, there is a technical problem that it is difficult to determine the character likeness based on the positional relationship of the minimum circumscribed rectangle. . Specifically, in the English characters shown in FIG. 12A, “P”, “A”, “R”, and “K” have substantially the same height. However, in the Japanese characters shown in FIG. 12B, of the three “trees” in “forest”, the upper “tree” and the lower “tree” have a height. Is different.

  Alternatively, in another comparative example, the character portion included in the image is extracted from a pixel region of the same color having a narrow width using a mathematical morphology by using the property that the character portion is composed of elongated line segments. As extracted.

  However, in this other comparative example, basically, a long and narrow line segment is determined as a character part. Therefore, it is technically possible to distinguish noise influenced by a background part having the same properties and shape from a character part. It becomes difficult. Further, since it is necessary to set the maximum value of the line segment width of the character in advance, there is a possibility that the extraction ratio of the true character parts may be lowered depending on how the maximum value is set.

  On the other hand, in the present embodiment, as described in steps S10 to S30 described above, the pixels constituting the image input to the image processing apparatus are classified according to various attributes and properties such as luminance components and chromaticity. Clustering is performed to create a class image in which pixels are decomposed for each class. Then, as described in steps S40 and S50 described above, for each class image, parts (that is, pixel connection regions) are extracted and the line widths of these parts are measured. Create a histogram showing the frequency of appearance.

  Next, in the characters shown in the image, it is highly likely that the character width is composed of a substantially constant line segment width. Therefore, if the class image contains many character parts, It is considered that the line segment width (that is, the element of the histogram) having a remarkably high appearance frequency approximates the line segment width of the true character part. Therefore, as described in step S60 described above, the line width having a remarkably high appearance frequency is set as the predicted value of the line width of the character part. Then, as described in step S70 described above, the character part candidates are selected by comparing the predicted value with the appearance frequency of the line segment width in each part.

  As a result, the character parts can be accurately extracted even when the language of the character and the font cannot be predicted in advance. Also, by having multiple predicted line segment widths, it is possible to extract character parts with high accuracy even for special character fonts where the vertical and horizontal character widths are very different, such as Mincho. is there.

  Subsequently, as described in steps S80 to S100 described above, when character parts are extracted for each class, a character image in which these character parts are combined into one is generated.

  As a result, even for an image having a character string composed of a plurality of character colors, all character parts can be extracted appropriately and with high accuracy.

(5) Second embodiment of image processing apparatus
Next, a second embodiment of the image processing apparatus according to the present invention will be described with reference to FIG. FIG. 13 is a schematic diagram schematically showing how character images are generated from all class images according to the second embodiment.

  In the second embodiment, when character parts are extracted for each class, dispersion (variation) of contour unit IDs assigned to the parts may be taken into consideration. In other words, by calculating the degree of dispersion of the contour unit ID assigned to the part, the character likelihood (that is, the degree that can be regarded as a character) may be determined. This is because, in such background parts, similar shapes (patterns) are often regularly arranged. Therefore, in step S40 in FIG. 3 described above, the contour unit ID is almost equal to each part. This is because the tendency to be given in the same way is high.

  Specifically, the character string portion in the image has a relatively high tendency to be composed of parts having different shapes. On the other hand, the background part often has a plurality of parts having similar shapes (patterns) even if it is difficult to distinguish the characters from characters only by the shape of the individual parts. Therefore, by considering the degree of dispersion of the contour unit ID, it is possible to distinguish a background part that is difficult to distinguish from a character part by using only individual shapes, and reject it at the stage of generating a character image.

  This will be specifically described with reference to FIG. In the class 2 image shown at the center in FIG. 13, in addition to the true character part, the background part is extracted as the character part. This is because the appearance frequency of the line segment width of the pixels constituting the part is remarkably high in the appearance frequency of several elements (ie, line segment width) such as 1, 2, etc. This is because there is a characteristic that the level that can be regarded as being accidentally is at a high level.

  Next, generalize using mathematical formulas. For example, in class C, N character parts are extracted. Here, if the number of contour units k (1 ≦ k ≦ 8) given to the part i (1 ≦ i ≦ N) is Uik, the average Mk of appearance frequencies of the contour unit k is expressed by the following equation (3 )

…… （３）
次に、輪郭ユニットkの出現頻度の分散を、次の変数（４）で示した場合、次の式（５）によって算出される。

(3)
Next, when the variance of the appearance frequency of the contour unit k is indicated by the following variable (4), it is calculated by the following equation (5).

…… （４）

(4)

…… （５）
上述の式（５）で示された輪郭ユニットkの出現頻度の分散を、全ての輪郭ユニット ｋ（１≦ｋ≦８） で算出し、それを加算したものを、次の変数（６）で示した場合、次の式（７）によって求められる。

(5)
The variance of the appearance frequency of the contour unit k shown in the above equation (5) is calculated for all the contour units k (1 ≦ k ≦ 8), and the sum is added to the following variable (6). In the case shown, it is obtained by the following equation (7).

…… （６）

(6)

…… （７）
と求められる。この尚、以下、上述の変数（６）で示された変数を、Ｄａと称す。このＤａをクラスCにおける輪郭ユニットIDが出現する出現頻度の分散とする。文字のようにパーツ形状が様々に変化するものであれば、Ｄａは相対的に大きくなる。他方、パーツ形状の変化量が相対的に小さければ、Ｄａ相対的に小さくなる。よって輪郭ユニットIDが出現する出現頻度の分散Ｄａが、例えば所定閾値より小さい場合、背景パーツである可能性が高いので、文字画像を生成する段階で棄却する。具体的には、図１３中ではクラス２で抽出された文字パーツ候補の輪郭ユニットIDの分散度が、相対的に小さいため、文字画像の生成段階で棄却されている。

(7)
Is required. Hereinafter, the variable indicated by the variable (6) is referred to as Da. This Da is defined as a distribution of appearance frequencies in which contour unit IDs in class C appear. If the part shape changes variously like characters, Da becomes relatively large. On the other hand, if the amount of change in the part shape is relatively small, Da is relatively small. Therefore, when the appearance frequency variance Da at which the contour unit ID appears is smaller than a predetermined threshold, for example, it is highly possible that the contour unit ID is a background part. Specifically, in FIG. 13, since the degree of dispersion of the contour unit IDs of the character part candidates extracted in class 2 is relatively small, it is rejected at the character image generation stage.

  In this example, the character parts are selected based on the degree of dispersion of the contour unit IDs as the degree of distribution of the character parts. In addition to or instead of this, the character parts may be selected as a result of simply calculating the similarity of the shapes between the parts.

(6) Third embodiment of image processing apparatus
Next, a third embodiment of the image processing apparatus according to the present invention will be described with reference to FIG. FIG. 14 is a schematic diagram schematically showing how character images are generated from all class images according to the third embodiment.

  In the third embodiment, when extracting character parts for each class, the number of parts may be taken into consideration. In other words, when the number of character parts extracted from the class image is extremely small, there is a high possibility that it is a background part, and therefore it may be rejected at the character image generation stage. This is because the original character string portion is often composed of a plurality of parts, and the possibility of an increase in the number of character parts is remarkably high.

  Specifically, since the character string portion in the image is basically composed of a plurality of characters, the number of character parts tends to be relatively large. Therefore, the number of character parts extracted for each class can be measured, and according to the degree, the background parts and the character parts can be distinguished and rejected at the stage of generating the character image. In particular, it is possible to more accurately distinguish a background part and a character part that are likely to be erroneously recognized as a character part only based on an individual shape.

  This will be specifically described with reference to FIG. In this example, in addition to characters, background parts in the class 2 image are extracted as character parts. This is because the appearance frequency of the line segment width of the pixels constituting the part is remarkably high in the appearance frequency of several elements (ie, line segment width) such as one, and it is regarded as a character (ie, a character) This is because there is a characteristic that the degree of the possibility is accidentally at a high level. As a result, the class 2 image in FIG. 14 has a small number of extracted character part candidates, and therefore can be rejected when the character image is generated.

  In the present embodiment, the method using the contour unit is used to detect the line segment width. However, a method using a mathematical algorithm may be used instead of the contour unit.

  In this embodiment, the image processing apparatus has been described. For example, a DVD recorder, an HDD (Hard Disc Drive) recorder, a digital camera, a car navigation system, a PC (Personal Computer) compatible with TV (Television) recording, The present invention can be applied to all electronic devices that can store (store), manage, and view images such as mobile phones.

  The present invention is not limited to the above-described embodiments, and can be appropriately changed without departing from the gist or concept of the invention that can be read from the claims and the entire specification. The method and the optical character recognition apparatus and method are also included in the technical scope of the present invention.

  The image processing apparatus and method according to the present invention can be used in an image processing apparatus that can extract and recognize a character portion included in an image such as a color image. The optical character identification device and method according to the present invention can be used for an optical character identification device that can extract and recognize a character portion included in an image such as a color image.

Claims (14)

Detecting means for detecting a line segment width of a pixel connection part in which a plurality of pixels including a character part indicating a character are connected on a plane of an image;
Identification means for identifying the character portion based on a statistical distribution of the detected line segment width ,
The detecting means is a calculating means for calculating the line segment width using a boundary pixel that is a pixel that coincides with any one of a plurality of contour patterns defined by a shape pattern of an edge of the pixel connection portion as a start point or an end point. Have
Said identifying means, in addition to the statistical distribution of the calculated line width, based on the variance of the frequency of appearance frequency or the contour pattern of said plurality of contour patterns, Rukoto identifies the character portion An image processing apparatus. The identification means, as the statistical distribution, based on the correlation between the detected line width and the appearance frequency of the detected line width, the character portion and a background portion indicating a background image The image processing apparatus according to claim 1 , wherein: Based on a frequency distribution indicating the frequency of occurrence of the line segment width as the statistical distribution, further comprising a predicting unit that predicts a true line segment width that is a line segment width of a true character part,
The identifying means identifies the character portion based on a comparison between the predicted true line segment width and the frequency of appearance of a line segment width in one pixel connection portion among a plurality of pixel connection portions. The image processing apparatus according to claim 1 . The image processing apparatus according to claim 3 , wherein the prediction unit predicts a plurality of predicted values as the line width. The identifying means identifies the one pixel connection portion as the character portion when the ratio of the appearance frequency of the predicted value is greater than a predetermined threshold in the line segment width appearance frequency in the one pixel connection portion. The image processing apparatus according to claim 3 . The image processing apparatus according to claim 1 , wherein the calculation unit calculates the line segment width along a direction defined corresponding to each of the plurality of contour patterns. Cluster dividing means for dividing the image into a plurality of clusters in advance;
The image processing apparatus according to claim 1 , further comprising conversion means for converting each of the plurality of divided clusters into a binary image that can be expressed in binary . Claim wherein the identification means, in addition to the statistical distribution of the calculated line width, based on the degree of shape regularity in a plurality of pixels connecting portion, and wherein the identifying the character portion the image processing apparatus according to 1. Said identifying means, in addition to the statistical distribution of the calculated line width, based on some number of the plurality of pixels connecting portion, according to claim 1, characterized in that identifying said character portion Image processing apparatus. In addition to the statistical distribution of the calculated line segment width , the identification unit is configured to calculate a distance between a centroid of one pixel connection portion and a centroid of another pixel connection portion among a plurality of pixel connection portions. The image processing apparatus according to claim 1 , wherein the character portion is identified based on a size. The identifying means identifies, as the character portion, one character portion of a plurality of pixel connection portions as a candidate based on a statistical distribution of the calculated line segment width, and is identified as the candidate . the distance between the centers of gravity of the one character portion, with respect to the chromaticity of the one of the character portion luminance, or the one character portion, based on the similarity, identify the true character portion of the plurality of pixels connecting portion The image processing apparatus according to claim 1 , wherein: Detecting means for optically detecting a line segment width of a pixel connection portion in which a plurality of pixels including a character portion indicating a character are connected on a plane of an image;
Identification means for identifying the character portion based on a statistical distribution of the detected line segment width ,
The detecting means is a calculating means for calculating the line segment width using a boundary pixel that is a pixel that coincides with any one of a plurality of contour patterns defined by a shape pattern of an edge of the pixel connection portion as a start point or an end point. Have
Said identifying means, in addition to the statistical distribution of the calculated line width, based on the variance of the frequency of appearance frequency or the contour pattern of said plurality of contour patterns, Rukoto identifies the character portion An optical character recognition device. A detection step of detecting a line segment width of a pixel connection portion where a plurality of pixels including a character portion indicating a character are connected on a plane of an image;
An identification step of identifying the character portion based on a statistical distribution of the detected line segment width ,
The calculation step of calculating the line segment width using a boundary pixel that is a pixel matching one of a plurality of contour patterns defined by a shape pattern of an edge of the pixel connection portion as a start point or an end point. Have
The identification step, in addition to the statistical distribution of the calculated line width, based on the variance of the frequency of appearance frequency or the contour pattern of said plurality of contour patterns, Rukoto identifies the character portion An image processing method characterized by the above. A detection step for optically detecting a line width of a pixel connection portion where a plurality of pixels including a character portion indicating a character are connected on a plane of an image;
An identification step of identifying the character portion based on a statistical distribution of the detected line segment width ,
The calculation step of calculating the line segment width using a boundary pixel that is a pixel matching one of a plurality of contour patterns defined by a shape pattern of an edge of the pixel connection portion as a start point or an end point. Have
The identification step, in addition to the statistical distribution of the calculated line width, based on the variance of the frequency of appearance frequency or the contour pattern of said plurality of contour patterns, Rukoto identifies the character portion An optical character identification method characterized by the above.

Images