JP4966397B2 - Information processing apparatus and information processing method - Google Patents

Description

  The present invention relates to an information processing apparatus and an information processing method for determining the effectiveness of features extracted from patterns such as characters, graphics, and sounds.

  There is a pattern recognition device that reads a form on which patterns such as characters and figures are drawn with a scanner and recognizes the original pattern drawn on the form.

  In this pattern recognition apparatus, at the time of pattern recognition, an area in which a pattern to be recognized exists is cut out from an image, and features extracted from the image of the area are pre-stored in the pattern dictionary. It is determined whether the patterns are similar to each other, and the pattern drawn on the form is specified.

  For this reason, it is extremely important to verify how effective the feature extracted from the image is for pattern identification.

  As a conventional technique related to feature extraction, for example, when a feature is formed for each pixel of an image to be processed, and the feature satisfies a predetermined condition of a qualitative property or a quantitative property, the image to be processed corresponding to the feature A technique for coloring these pixels has already been disclosed (see, for example, Patent Document 1).

  In the technique of this Patent Document 1, qualitative properties are used as density information, position information, direction information, and the like. The quantitative property is the size of the feature amount. In Patent Document 1, information necessary for elucidating the cause of misrecognition can be obtained by viewing a colored image.

JP 2008-3730 A

  However, in the case of the conventional technique described in Patent Document 1, the nature of the feature extracted from the image to be processed is expressed, but it is not known how effective the feature is for image identification. There's a problem.

  In the case of the technique disclosed in Patent Document 1, the applicable feature extraction method is limited to a method of forming a feature in units of pixels, and thus cannot be applied when forming a feature for a set of pixels. It was.

  In the above-mentioned prior art document 1, there is a description that the image to be processed can be colored according to the evaluation value of the feature obtained by the feature evaluation or the effectiveness of the feature obtained by the analysis of the identification process. Although there is no specific description of the technology.

  The present invention has been made to solve such problems, and how effective the feature of a pattern calculated by a certain feature extraction method is for pattern recognition, or how much the feature affects misrecognition of a pattern. It is an object of the present invention to provide an information processing apparatus and an information processing method capable of providing quantitative information for determining whether or not the information has been received.

In order to solve the above problems, an information processing apparatus according to the present invention includes an acquisition unit that acquires a pattern to be processed, and a pattern obtained from the acquisition unit divided into a plurality of patterns according to a predetermined rule. A feature calculation unit that calculates a feature vector indicating the feature of each divided pattern , a dictionary storage unit that stores the type of identification candidate and a set of representative vectors of the identification candidate, and the dictionary storage unit The set of representative vectors corresponding to the type is acquired, and the similarity or difference between the processing target and the identification candidate obtained from the set and the feature vector is decomposed so as to correspond to each component of the feature vector. , the degraded identification influence calculation unit which each component is calculated as the degree of influence on the identification of the pattern of the values the feature vector, the calculated degraded by the identification influence calculation unit Color information corresponding to the obtained value, associating the color information with the divided patterns, and generating an image indicating the correspondence between the color information and the divided patterns, And an image output unit that outputs an image generated by the image generation unit .

An information processing method of the present invention, acquisition unit, feature calculation unit, identification influence calculation unit, the image generating unit, an information processing method in an information processing apparatus having an image output unit, a pattern to be processed acquired by the acquiring unit , after dividing the pattern in which the acquisition unit to obtain a plurality of patterns by a predetermined rule, a feature vector indicating a feature of each pattern the divided calculate said feature calculation unit, of the identification candidate type and identification a set of the representative vector the identification influence calculation unit obtains in which the set of representative vectors of the candidate corresponding to the type from the dictionary storage unit stored to correspond, obtained from the feature vector and said population if the The similarity or dissimilarity between the processing target and the identification candidate is decomposed so as to correspond to each component of the feature vector, and each component of the feature vector is used to identify the pattern. Calculated as the degree of sound, said image generating unit a color information corresponding to a value obtained by the decomposition calculated by identifying influence calculation unit generates, correspondence to the patterns of the color information and the divided, the color An image showing correspondence between color information of information and each divided pattern is generated, and the image output unit outputs the image generated by the image generation unit .

  According to the present invention, it is possible to quantitatively determine how effective a pattern feature calculated by a certain feature extraction method is for pattern recognition, or how much the feature has affected pattern misrecognition. Information can be provided.

It is a figure which shows the structure of the image processing system of 1st Embodiment. It is a flowchart which shows operation | movement of the image processing system of 1st Embodiment. It is a flowchart which shows a feature vector calculation process. It is a flowchart which shows an identification influence degree calculation process. It is a figure which shows the data stored in the identification dictionary storage part. It is a flowchart which shows the other example of an identification influence degree calculation process. It is a figure which shows the other example of the data of an identification dictionary storage part. It is a flowchart which shows the production | generation process of a visualization image group. It is a schematic diagram of the image and numerical value obtained in the process of 1st Embodiment. It is a block diagram which shows the structure of the image processing system of 2nd Embodiment. It is a flowchart which shows operation | movement of the image processing system of 2nd Embodiment. It is a flowchart which shows a feature vector calculation process. It is a figure which shows the division area of the image referred in the case of a feature vector calculation process. It is a flowchart which shows an identification influence degree calculation process. It is a flowchart which shows the production | generation process of a visualization image group. It is a block diagram which shows the structure of the image processing system of 3rd Embodiment. It is a flowchart which shows operation | movement of the image processing system of 3rd Embodiment. It is a flowchart which shows a partial space projection process.

Hereinafter, an image processing system according to an embodiment will be described in detail with reference to the drawings. (First embodiment)
FIG. 1 is a block diagram showing the configuration of the image processing system according to the first embodiment of the information processing apparatus of the present invention.

  As shown in FIG. 1, this image processing system includes a scanner 1 as an input device, a computer 2 connected to the scanner 1, and a display device 3 as an output device connected to the computer 2. Yes. The computer 2 includes general-purpose hardware such as a CPU, a memory, a hard disk device, and an external interface.

  The scanner 1 is a scanner such as a flatbed type or an automatic paper feed type, for example. The scanner 1 reads a form set in a reading unit such as a line sensor, and an image of the read form (hereinafter referred to as an “input image” or an image to be processed). Is input to the computer 2. In addition, an input device such as a keyboard and a mouse is connected to the computer 2, and an instruction by a user operation is input to the computer 2. The input device may be a setting file in which arbitrary instruction information is set in addition to the keyboard. The display device 3 displays information output from the computer 2.

  The computer 2 includes an image reception unit 21, an image storage unit 22, a feature extraction unit 23, an identification dictionary storage unit 25, an identification influence degree calculation unit 24, an image generation unit 26, an image output unit 27, and the like. The image storage unit 22 and the identification dictionary storage unit 25 are realized by a memory or a hard disk device. Other configurations, that is, the image reception unit 21, the feature extraction unit 23, the identification influence calculation unit 24, the image generation unit 26, the image output unit 27, and the like are realized as one of the functions of the image processing program installed in the hard disk device. Is done.

  The image receiving unit 21 receives (acquires) an image to be processed input from the scanner 1 (hereinafter referred to as “input image”), and stores the received (acquired) image in the image storage unit 22. To do. That is, the image reception unit 21 functions as an acquisition unit that acquires a pattern to be processed.

  The feature extraction unit 23 reads the input image from the image storage unit 22, performs calculation to represent the feature of the subject in the vector format from the pixel information of the input image, and stores the vector (feature vector) in the memory.

  The feature extraction unit 23 functions as a feature calculation unit that calculates a first feature vector indicating the feature of each divided region obtained by dividing the image according to a predetermined rule.

  The identification dictionary storage unit 25 stores a plurality of pairs of a subject type of an image and information (type identification information) characterizing the type. The type of subject of the image indicates a number (character type) of “1” if the character is “1”, for example, and “cat” corresponds to this if it is a cat image. The type identification information is a vector group (type representative vector group) representing the type. However, the type representative vector group can be composed of only one representative vector. That is, the identification dictionary storage unit 25 functions as a dictionary storage unit in which the type of reference pattern and the type representative vector group corresponding to the type are stored in association with each other.

  The identification influence calculation unit 24 uses the feature vector of the input image extracted by the feature extraction unit 23 and the type identification information in the identification dictionary storage unit 25 to decompose the processing target pattern and the reference pattern. The degree of similarity or the degree of difference is calculated, and the decomposed value is calculated as an identification influence degree indicating how much each component of the feature vector affects the identification of the input image.

  The image generation unit 26 associates an area (divided area) obtained by dividing the spatial area of the input image with the identification influence degree, and generates color information according to the identification influence degree, thereby visualizing the association ( Visualized image) is generated and stored in the memory. The spatial area of the input image is a set of all pixels constituting the image.

The input image may be divided into regions with random shapes as shown in FIG. 13, or may be regularly divided into small rectangular regions of the same size.
Since the pattern to be processed is a pixel of the input image, a pattern obtained by dividing the pattern to be processed is obtained from the divided area. Hereinafter, the divided pattern is referred to as a divided pattern. That is, the image generation unit 26 generates color information according to the identification influence degree calculated by the identification influence degree calculation unit, associates the color information with each division pattern, and color information between the color information and the division pattern. An image showing the correspondence of is generated.

  The image output unit 27 reads the image visualized by the image generation unit 26 from the memory, outputs the image to the display device 3, and displays the image on the screen of the display device 3. Since the magnitude of the identification influence corresponding to each divided area of the input image to be identified is expressed in a format that can be visually recognized by the color information, it is possible to determine what kind of the divided area works effectively for identification. It becomes possible to grasp visually. As a result, a guideline for identifying a feature effective for identifying a pattern to be processed is clarified, which is useful for improving a feature extraction method, for example.

  Next, the operation of the image processing system according to the first embodiment will be described with reference to FIGS. In this image processing system, when the scanner 1 reads a form image and inputs it to the computer 2, the form image is received by the image receiving unit 21 and stored in the image storage unit 22 (step A-ST1). .

  Specifically, the data is input to the reading computer 2 by the scanner 1 connected to the computer 2 and written to the hard disk device. As a method not using the scanner 1, for example, an image (input image) on which a pattern to be processed is drawn in advance is stored in the image storage unit 22 of the hard disk device of the computer 2, and the input image is read from the hard disk device and stored in the memory. You may write to. The input image is stored in a storage device such as a server computer or NAS on the network, and the input image in the storage device is sent to the computer 2 via the network. 22 may be stored.

  Next, the feature extraction unit 23 reads an input image from the memory, calculates a feature vector of the input image (step A-ST2), and stores it in the memory.

  Subsequently, the identification influence degree calculation unit 24 uses the feature vector stored in the memory in step A-ST2 and the type representative vector group stored in the identification dictionary storage part 25, that is, one or more identification influence degree sequences. Is calculated (step A-ST3) and stored in the memory.

  The image generation unit 26 reads the input image from the memory, generates a visualized image by associating the divided area of the input image with the identification influence level (step A-ST4), and stores the visualized image in the memory.

  The image output unit 27 reads out the visualized image generated by the image generation unit 26 from the memory and outputs it to the display device 3 (step A-ST5).

  Here, the details of the processing of each step will be described. First, the feature vector calculation process in step A-ST2 will be described.

  As shown in FIG. 3, the feature extraction unit 23 converts the input image into a grayscale image. Specifically, with respect to all the pixels of the input image, information that distinguishes colors such as red, blue, and green included in the pixel value is removed and only the light and dark information is left, thereby converting the input image into a grayscale image ( Step A-ST2-1).

Next, the feature extraction unit 23 calculates a gradient vector of pixel values for all pixels of the input image converted to grayscale. The gradient vector is a vector indicating in which direction the lightness / darkness value of the pixel changes on the two-dimensional plane. Arbitrary coordinate in the gray scale image (px, py) (however px and py is an integer) when the pixel values in the I _px, and _py, coordinates (p _x, p _y) gradient vector for the pixel of g _{px and py} are calculated by the following (formula 1) (step A-ST2-2).
g _{px, py} = ((I _{px + 1, py} -I _{px-1, py} ) / 2 (I _{px, py + 1} -I _{px, py + 1} ) / 2) ^t (Formula 1)
However, when the coordinates (p _x, p _y) points to the pixels of the upper and lower left and right edges of the input image, since (Equation 1) In not required gradient vector, for the pixel, the gradient vector g _{px, py} = 0 And

Feature extraction unit 23 equally divides the spatial region of the input image converted into gray-scale lateral w _c pieces, the vertical h _c number (step A-ST2-3). w _c and h _c are constants. That is, the feature extraction unit 23 divides the pattern according to a predetermined rule (in this example, the pattern is equally divided into w _{c in} the horizontal direction and h _{c in} the vertical direction) to generate a plurality of divided regions.

At this time, the respective b _x the vertical position and horizontal position of the divided _{region, b y (0 ≦ b x} <w c, 0 ≦ b y <h c, is provided that b _x and b _y are integers), and the A number (bx + bywc) is assigned to the area. Hereinafter, the set of the divided areas is denoted as R. Since R is a set of (w _c h _c −1) divided regions,
R = {R ₀ , R ₁ ,..., R _wchc−1 }.

とする。ベクトルの成分の番号は０から順に始まるものとしており、

はそれぞれＲ_iにおける勾配ベクトルの第０番成分の平均、第１番成分の平均である。 The feature extraction unit 23 calculates the average of gradient vectors corresponding to all the pixels in each region for each region divided in step A-ST2-3 (step A-ST2-4).
Hereinafter, the average gradient vector in R _i (0 ≦ i <| R |, | R | = w _c h _c , where i is an integer)

And The vector component numbers start from 0 in order,

Are the average of the 0th component and the average of the 1st component of the gradient vector at R _i, respectively.

  The feature extraction unit 23 generates a feature vector by concatenating the average gradient vectors obtained in step A-ST2-4 (step A-ST2-5) and stores it in the memory. That is, the feature extraction unit 23 calculates the feature vector of each divided region from the average value of the gradient vectors.

Specifically, the feature vector x is obtained in the following (formula 2) format.

は、上記分割した領域のうち、ｂ_x＝０，ｂ_y＝０である領域における平均勾配ベクトルの第０番成分、という特徴の種類を意味する。 When the feature vector is calculated as described above,

Means the feature type of the 0th component of the average gradient vector in the region where b _x = 0 and b _y = 0 among the divided regions.

  Here, the detailed operation of the identification influence degree calculation process (the process of step A-ST3) performed by the identification influence degree calculation unit 24 will be described.

  At this time, as shown in FIG. 5, it is assumed that the identification dictionary storage unit 25 stores numerical values (types of subjects of the image) and type representative vector groups (type identification information) in association with each other.

  In the first embodiment, in the type representative vector group, a set of feature vectors obtained by a process equivalent to the feature extraction unit 23 from a group of images in which the same type is assigned to the subject is referred to as Karhunen-Loeve expansion (hereinafter referred to as “KL expansion”). ), And the eigenvector group is calculated in advance. The eigenvector group means an axis (base) of a space (subspace) that approximates the distribution of the set of feature vectors on the feature space.

  The identification influence degree calculation unit 24 displays a screen for allowing the user of the computer 2 to input the type (integer value) of the subject. Then, on this screen, when the user inputs the type of the subject (integer value) from the keyboard, the identification influence calculation unit 24 accepts the input integer value (step A-ST3-1 in FIG. 4), The type identification information (eigenvector group) corresponding to the received numerical value is acquired from the identification dictionary storage unit 25 (step A-ST3-2).

である。但しｒはφに含まれる固有ベクトルの総数である。 Hereinafter, the eigenvector group is denoted as φ as a set of eigenvectors. In other words,

It is. Where r is the total number of eigenvectors included in φ.

ここで＜x ，φ_i ＞はx とφ_i の内積を表す。識別影響度算出部２４は下記（式４）により入力画像と前記画像の被写体の種別との類似度を分解した形式で算出し（ステップA-ST3-4）、該分解した類似度を識別影響度としてメモリに書き込む。

上記（式４）において、s_ｉはxの第i成分の識別影響度であり、

また、ｄは特徴空間の次元数であり、第１実施形態においては、特徴空間の次元数ｄ＝２ｗ_ｃｈ_ｃである。 The discriminating influence calculation unit 24 reads the feature vector x calculated in step A-ST2 from the memory, and orthogonally projects the vector onto the partial space formed by φ (step A-ST3-3) to obtain a projection vector x ′. . However, the similarity between the input image and the type of subject of the image can be decomposed by describing the projection vector as a vector on the feature space. The calculation method of x ′ is expressed by an expression (Expression 3).

Here, <x, φ _i > represents the inner product of x and φ _i . The discriminating influence calculation unit 24 calculates the similarity between the input image and the type of subject of the image according to the following (Equation 4) (step A-ST3-4), and uses the decomposed similarity to discriminate influence. Write to memory as a degree.

In the above (Equation 4), s _i is the discriminative influence degree of the i-th component of x,

Further, d is the number of dimensions of the feature space, and in the first embodiment, the number of dimensions of the feature space is d = 2w _c h _c .

  Next, with reference to FIG. 6 and FIG. 7, another operation example of the identification influence degree calculation process (the process of step A-ST3) performed by the identification influence degree calculation unit 24 will be described.

  At this time, as shown in FIG. 7, the identification dictionary storage unit 25 stores numerical values (types of subject of the image, hereinafter simply referred to as types) and sets of vector values (type identification information) correspondingly. It shall be. However, the type identification information is an average (average vector) of feature vectors obtained by a process equivalent to the feature extraction unit 23 from a group of images in which the same type is assigned to the subject.

  That is, the type representative vector group (type identification information) stored in the identification dictionary storage unit 25 is a vector (average vector) obtained by averaging the feature vector groups calculated from the image group belonging to the type to which the type identification information corresponds.

  Subsequently, the identification influence degree calculation unit 24 reads the type (integer value) of the subject of the image stored in advance in the hard disk (step A-ST3′-1), and type identification information (average vector) corresponding to the integer value ) Is acquired from the identification dictionary storage unit 25 (step A-ST3'-2). Hereinafter, the average vector is denoted as T.

  A difference vector x ″ = T−x between the feature vector x and the average vector T is calculated (step A-ST3′-3).

（式５）において、s_ｉは特徴空間の第i軸の識別影響度であり、

For all components of the feature vector x, the degree of difference between the input image and the type of subject of the image is calculated in the form of decomposition according to the following (Equation 5) (step A-ST3-4). Write to memory as identification impact.

In (Expression 5), s _i is the discriminative influence degree of the i-th axis of the feature space,

  Since the sum of the discriminating influences calculated in steps A-ST3-1 to A-ST3-4 coincides with the similarity based on the subspace method, the discriminating influence is calculated based on the feature vector corresponding to the similarity based on the subspace method. It means each component influence degree.

  In addition, since the sum of the discriminative influences calculated in steps A-ST3'-1 to A-ST3'-4 coincides with the square of the Euclidean distance in the feature space of x and T, the discriminative influence is This means the degree of influence of each component of the feature vector on the degree of difference due to the Euclidean distance.

  When calculating the discriminating influence level sequence as described above, it should be noted that "the discriminating influence degree is targeted for similarity based on the subspace method" or "the discriminating influence degree is targeted for the degree of difference due to the Euclidean distance". And Further, reference values used for identification of input images such as similarity and dissimilarity are collectively indicated as an identification evaluation scale or simply as an evaluation scale.

  Regarding the data structure of the identification dictionary storage unit 25, the data format of the type is an integer value in the above description, but other data formats such as a character string that can distinguish the type of the input image may be used.

  Next, the visualization image generation process (the process of step A-ST4) by the image generation unit 26 will be described in detail with reference to the flowchart of FIG.

The image generation unit 26 executes the following sub-steps A-ST4-1 to A-ST4-6 (flowchart of FIG. 8) to visualize the image group I _v = {I _{v (0)} , I _{v (1)} } Is generated and stored in the memory.

First, the image generation unit 26 initializes the visualized images I _{v (0)} and I _{v (1)} . Specifically, each visualized image is a color image having red, green, and blue color channels (color divisions), and each visualized image has a width w _c pixel and a height h _c pixel, and the visualized image I _{v ( 0)} , 0 is substituted into the pixel values of all pixels of all color channels of _{Iv (1)} . w _c and h _c are the same as w _c and h _c used in step A-ST2-3.

と記す。但しｉは１か０であり、ｃは０か１か２であり、ｃの各値は０＝赤、１＝緑、２＝青であるものとする。 Thereafter, the pixel value at an arbitrary coordinate (p _x , p _y ) in the color channel c of the i-th visualized image is determined.

. However, i is 1 or 0, c is 0 or 1 or 2, and each value of c is 0 = red, 1 = green, and 2 = blue.

に、ステップA-ST3で算出した第（ｐ_ｘ＋（ｐ_ｙ＋ｉｈ_ｃ）ｗ_ｃ）番の識別影響度を代入する。該操作を式で表すと下記（式６）となる。

なお記号「←」は代入操作を意味する。 Subsequently, the image generation unit 26 generates color information corresponding to the identification influence degree by writing the identification influence degree in the red channel of the visualized image (step A-ST4-2). Specifically, the pixel value of the pixel of the coordinates (p _x , p _y ) of the red channel of the i-th visualized image I _{v (i)}

The identification impact degree of the (p _x + (p _y + ih _c ) w _c ) number calculated in step A-ST3 is substituted. This operation is expressed by the following formula (formula 6).

The symbol “←” means an assignment operation.

The image generation unit 26 converts the range of pixel values of each pixel of each visualized image into a range [l _low , l _high ] that can be taken by the pixel value of the input image (step A-ST4-3). For example, the pixel value is converted in the following (formula 7).

In this (Equation 7), l _min and l _max mean the maximum and minimum pixel values of the pixels in the red channel of the i-th visualized image immediately after execution of step A-ST4-2.

  The image generation unit 26 extracts red, green, and blue color components from the pixel values of the input image, assigns the extracted color components of the input image to red, green, and blue channels (sections), respectively, and the input image is a color image. (Step A-ST4-4).

  If the input image is a grayscale image or a black and white image (an image in which pixels are represented by white or black), the value v of each pixel of the input image is set as the value of the color component of each of the red, green, and blue channels. In addition, if the input image is originally a color image having red, green, and blue channels, no particular conversion process is performed.

  The image generation unit 26 adjusts the size (size) of each visualized image by expanding or contracting each visualized image to the same size as the input image (step A-ST4-5). At this time, the nearest neighbor method is used as a pixel value interpolation method when the image is expanded or contracted. In addition, other methods such as a cubic spline interpolation method may be used as a pixel value interpolation method during image expansion / contraction.

  Finally, the image generation unit 26 performs synthesis with the input image for each visualized image (step A-ST4-6), and stores the synthesized image group in the memory. The synthesized image group becomes a new visualized image group.

  Alpha blending is used to synthesize the visualized image and the input image. In the alpha blending method, an image to be synthesized is weighted, and the weighted images are added to generate a synthesized image. The weight is a real value between 0 and 1, and the sum of the weights assigned to each image is 1. In the image generation unit 26, the weight is an arbitrary value determined in advance. A new visualized image synthesized and generated by the alpha blending method is displayed with a higher luminance value of red in a divided region having a higher identification influence in the input image.

  In the description of step A-ST4 above, the identification influence level is written in the red channel (red section). Alternatively, it may be written in the blue channel (blue section) and the green channel (green section), or multiple channels. (Section) may be written simultaneously.

  In addition, the image may be synthesized by another method as long as the pixel value of the visualized image is reflected in the synthesis result.

  Finally, the output process of the visualized image (the process of step A-ST5) by the image output unit 27 will be described in detail. The image output unit 27 outputs the visualized image to the display device 3. Each visualized image is output to, for example, the display device 3 and displayed on the screen of the display device 3. In addition, each visualized image may be sent to a printer connected to the computer 2 and printed. Each visualized image may be stored in an external storage device such as an external hard disk device connected to the computer 2. Furthermore, each visualized image may be transmitted to another computer via a network and stored.

  As described above, according to the image processing system of the first embodiment, each component of the feature vector is input using the feature vector obtained from the input image and the information representative of the type of the image (eigen vector, average vector, etc.). Since the degree of influence on the value related to image identification (identification influence degree) is calculated, an index or a guideline for searching for an effective feature for image identification in an image identification apparatus using the feature vector calculation method is obtained. be able to. Further, by visualizing the identification influence degree sequence and synthesizing it with the input image, it is possible to provide a clue for intuitively understanding the above-described index or guideline using vision. By visualizing how much each component (feature type) of the feature vector affects the evaluation scale for identification, a clue for determining which type of feature is effective for identification can be provided.

  FIG. 9 is a schematic diagram of images and numerical values obtained in the process of the first embodiment. As shown in FIG. 9A, when an input image is divided into three vertically and horizontally to generate nine divided regions, the feature vector components of each divided region are divided as shown in FIG. 9B. Two types of feature values are obtained for each region (a horizontal direction component and a vertical direction component of the gradient vector).

Assume that the identification influence degree of each divided region calculated from this feature value is a value as shown in FIG. As a result of visualization from the identification influence degree, that is, the generated visualization image, two images are obtained as shown in FIG. The left visualized image is lV ₍₀₎ , and the right visualized image is lV ₍₁₎ .

  A feature obtained from a divided region having a high luminance value of a visualized image means that the evaluation scale is affected. Therefore, the number of visualized images is two, and each indicates the identification influence degree regarding different types of feature values. Finally, as a result of combining the input image and the visualized image, an image as shown in FIG. 9E is obtained.

(Second Embodiment)
Next, a second embodiment will be described. FIG. 10 is a block diagram showing the configuration of the image processing system of the second embodiment. In addition, the same code | symbol is attached | subjected to the same structure as 1st Embodiment, and the description is abbreviate | omitted.

  As shown in FIG. 10, the image processing system includes a computer 2 connected to the feature extraction device 40 via a network, and a scanner 1 and a display device 3 connected to the computer 2. The feature extraction device 40 extracts a feature vector to be processed from the image and transmits the extracted feature vector to the computer 2.

  The computer 2 includes a feature acquisition unit 28, an identification dictionary storage unit 25, an identification influence degree calculation unit 24, an image generation unit 26, an image output unit 27, and a feature vector storage unit 30.

The feature acquisition unit 28 acquires a feature vector to be processed from the feature extraction device 40 and stores it in the feature vector storage unit 30 provided in the hard disk device or memory of the computer 2. Note that the feature vector is not only acquired from the outside, but may be stored, for example, in a hard disk device built in the computer 2 in advance and read out.
The identification influence degree calculation unit 24 calculates an identification influence degree sequence using the feature vector obtained from the feature vector storage unit 30 and the type identification information obtained from the identification dictionary storage unit 25, and stores the calculated identification influence degree sequence in the memory. To remember. The image generation unit 26 images the identification influence degree sequence and stores the image in the memory.

The operation of the image processing system according to the second embodiment will be described with reference to FIGS.
In the case of the second embodiment, the feature acquisition unit 28 reads out a feature vector stored in advance in the feature vector storage unit 30 of the hard disk device connected to the computer 2, for example, and acquires a feature vector to be processed. (Step B-ST1 in FIG. 11) and stored in the memory.

  In this example, feature vectors are taken into the computer 2 from a feature extraction device 40 provided separately from the computer 2 via a network such as a USB cable and an interface or a LAN, but the feature extraction device 40 may be built in the computer 2. Good.

The feature vector is calculated by the feature extraction device 40 executing the following steps FT-ST1 to FT-ST4 (flowchart in FIG. 12), and the feature acquisition unit 28 acquires it.
Similar to step A-ST4-4 of the first embodiment, the feature extraction device 40 converts the input image into a color image (R, G, B image) having a red channel, a green channel, and a blue channel (steps). FT-ST1).

Subsequently, as shown in FIG. 13, the spatial region of the input image is divided into, for example, six regions R _{o to} R ₅ (step FT-ST2). However, the regions R _{o to} R ₅ are allowed to overlap, and the overlapping regions are indicated by hatching in FIG. In FIG. 13, w and h mean the width and height dimensions (number of pixels) of the input image, respectively.

Divided for each region _R o to R _5, and calculates the average of the pixel values of all pixels in each region _R o to R ₅ (step FT-ST3).

赤、緑、青チャネルの画素値の平均である。 Hereinafter, the average pixel value in the region R _i (0 ≦ i <5, where i is an integer) is expressed in a vector format.

This is the average of the pixel values of the red, green, and blue channels.

All the average pixel values obtained in step FT-ST3 are connected to generate a feature vector (step FT-ST4) and stored in the memory. Specifically, the feature vector x is obtained in the following (formula 8) format.

  The identification influence degree calculation unit 24 calculates an identification influence degree sequence using the feature vector and type identification information acquired by the feature extraction device 40 (step B-ST2), and stores it in the memory. Specifically, this step executes the following sub-steps B-ST2-1 to BST2-3 (the flowchart in FIG. 14). However, the data structure of the identification dictionary storage unit 25 is as shown in FIG. The number of dimensions of each eigenvector is the number of dimensions d (= 18) of the feature vector x acquired in step B-ST1.

The identification impact calculation unit 24 acquires the type ωα (integer value) from the user's keyboard input or the setting file on the hard disk, and reads the eigenvector group for the type ωα.
Then, the identification influence degree calculation unit 24 calculates an identification influence degree sequence using the eigenvector group for the type ωα read from the identification dictionary storage part 25 and the feature vector x read from the feature vector storage part 30 (step B -ST2-1). Specifically, the processes of sub-steps A-ST3-1 to A-ST3-4 are executed. However, since the feature vector x is calculated in steps FT-ST1 to FT-ST4, the dimension number d = 18 of the feature space.

Next, the identification influence degree calculation unit 24 acquires the type ωβ from the user's keyboard input or the setting file on the hard disk, and reads the eigenvector group for the type ωβ.
Then, the identification influence degree calculation unit 24 calculates an identification influence degree sequence using the eigenvector group for the type ωβ (integer value) read from the identification dictionary storage part 25 and the feature vector x (step B-ST2-2). . Specifically, the processes of sub-steps A-ST3-1 to A-ST3-4 are executed. (However, d = 18).

  The identification influence degree calculation unit 24 calculates a final identification influence degree sequence using the identification influence degree sequences calculated in step B-ST2-1 and step B-ST2-2, respectively (step B-ST2-3). Store in memory.

Thereafter, the identification influence degree corresponding to the i-th component (0 ≦ i <18, where i is an integer) of the feature vector in the identification influence degree sequence calculated in step B-ST2-1 and step B-ST2-2. _These are denoted as s _{α, i} and s _{β , i} , respectively.

この他、（式９）の代わりに下記（式１０）を用いて識別影響度を算出してもよい。

また、（式９）の代わりに下記（式１１）を用いて識別影響度を算出してもよい。

The discriminating influence degree calculation unit 24 calculates the final discriminating influence degree by (Equation 9) for each component of the feature vector.

In addition, the discriminating influence degree may be calculated using (Equation 10) below instead of (Equation 9).

Further, the identification influence degree may be calculated by using the following (Expression 11) instead of (Expression 9).

The image generation unit 26 visualizes the identification influence degree sequence (step B-ST3 in FIG. 11), and stores the image in the memory. At this time, the image generation unit 26 executes the following sub-steps B-ST3-1 to B-ST3-2 (flowchart of FIG. 15) to visualize the image group I _v = {I _{v (0)} , I _{v ( 1)} , I _{v (2)} } is generated and stored in the memory. However, the visualized images I _{v (0)} , I _{v (1)} , and I _{v (2)} are assumed to be gray scale images.

The image generation unit 26 initializes the three visualized images I _{v (0)} , I _{v (1)} , and I _{v (2)} (step B-ST3-1). Specifically, each visualized image is a gray scale image, and the size of each visualized image is a width w _v pixel and a height h _v pixel, and 0 is substituted for the pixel value of all the pixels of all the visualized images. The width w _v and the height h _v are predetermined values.

In the subsequent steps, each visualized image is handled in units of divided areas. The divided areas are the six areas R _{o to} R ₅ having the shape shown in FIG. 13 described in step B-ST2. However, the width w of the visualized image shown in FIG. 13 is replaced with w _v and the height h is replaced with h _v . Hereinafter, the j-th (0 ≦ j <5) region in the i-th visualized image is denoted as I _{v (i, j)} . However, i is 0, 1 or 2, and j is an integer.

The image generation unit 26 assigns an identification influence sequence to each visualized image (step B-ST3-2), and writes the visualized image in the memory. Specifically, the (3j + i) th identification influence degree calculated in step B-ST2 is added to Iv _{(i, j)} . This step is expressed by the following (Expression 12).

The reason why the discriminating influence is added is that the plurality of spatial regions R _{o to} R ₅ shown in FIG.
Finally, the image output unit 27 outputs the visualized image generated by the image generation unit 26 as in Step A-ST5 shown in the first embodiment (Step B-ST4 in FIG. 11).

  As described above, according to the second embodiment, the feature vector x obtained from the feature extraction device 40 outside the computer 2 and stored in the feature vector storage unit 30, and the type ωα read from the identification dictionary storage unit 25. In addition, from the eigenvector group for the type ωα, the identification influence of each component of the feature vector for the type ωα and the type ωα is obtained, and the difference or ratio of the classification influence for the type ωα and the type ωα is output for each divided region of the input image. . That is, since the difference or ratio of the identification influence between the two types is output, it is possible to compare the identification influence between the types such as which type has the higher identification influence. For example, when the type ωα is a correct answer type and the type ωβ is a type that is easily misidentified by a pattern recognition (image identification) device other than the computer 2, the cause of misidentification in the image identification device other than the computer 2 is searched. Can be obtained. In addition, since the identification impact column is visualized, information such as what kind of feature represents the difference between the two types can be visually observed by observing the difference or ratio of the identification impact between the types. The feature extraction method and the evaluation scale calculation algorithm in the image identification apparatus that can be intuitively known and use the evaluation scale that is the target of the identification influence level are improved.

(Third embodiment)
An image processing system according to a third embodiment will be described with reference to the drawings. FIG. 16 is a block diagram illustrating a configuration of an image processing system according to the third embodiment.

  As shown in FIG. 16, the image processing system according to the third embodiment includes a computer 2 connected to the feature extraction device 40 via a network, and a scanner 1 and a display device 3 connected to the computer 2. ing. The feature extraction device 40 transmits the feature vector to be processed to the computer 2.

  The computer 2 includes an identification influence degree calculation unit 24, an identification dictionary storage unit 25, a feature acquisition unit 28, a subspace projection unit 29, a feature vector storage unit 30, and an identification influence degree output unit 31.

  The feature acquisition unit 28 acquires a feature vector to be processed from the feature extraction device 40 and stores it in the feature vector storage unit 30 provided in the hard disk device or memory of the computer 2. Note that the feature vector is not only acquired from the outside, but may be stored, for example, in a hard disk device built in the computer 2 in advance and read out.

  The identification influence degree calculation unit 24 calculates an identification influence degree sequence using the eigenvector group (type identification information) for the type ωα read from the identification dictionary storage unit 25 and the feature vector x read from the feature vector storage unit 30. And store it in the memory. The subspace projection unit 29 receives the input type (integer value), and acquires the eigenvector group (type identification information) corresponding to the received type from the identification dictionary storage unit 25. The subspace projection unit 29 is inside the discrimination influence calculation unit 24. The subspace projection unit 29 reads out the feature vector x from the memory, orthonormally projects the vector onto the subspace formed by φ, obtains the projection vector x ′, and outputs the projection vector x ′ to the discrimination influence degree calculation unit 24. The identification influence degree output unit 31 reads out the identification influence degree sequence from the memory and outputs it to the outside. The functions of the other components are the same as those in the first and second embodiments, and a description thereof is omitted.

  The operation of the third embodiment will be described in detail with reference to the flowcharts of FIGS. 17, 4, and 18.

  In the case of the third embodiment, the feature acquisition unit 28 acquires from the feature extraction device 40 outside the computer 2 and stores the feature vector stored in advance in the feature vector storage unit 30 of the hard disk device, thereby Is obtained (step C-ST1 in FIG. 17) and stored in the memory.

  The identification influence degree calculation unit 24 calculates an identification influence degree sequence using the feature vector and the type identification information corresponding to the two types (step C-ST2), and stores it in the memory. Specifically, the subspace projection unit 29 is called, and the sub-steps C-ST2-1 to C-ST2-3 shown in the flowchart of FIG. 18 are executed using the processing results to calculate the identification influence degree sequence. At this time, it is assumed that the identification dictionary storage unit 25 stores the data shown in FIG. The number of dimensions of each eigenvector of the eigenvector group is the same as the feature vector acquired in step C-ST1.

The discriminating influence calculation unit 24 first calls the subspace projection unit 29, and substitutes the projection vector x ′ output from the subspace projection unit 29, that is, the return value, into the vector x ′ _α (step C-ST2-1). .

Subsequently, the identification influence calculation unit 24 calls the subspace projection unit 29 and substitutes the projection vector x ′ output from the subspace projection unit 29, that is, the return value, into the vector x ′ _β (step C-ST2- 2).

Then, the identification influence degree calculation unit 24 calculates an identification influence degree sequence using these two projection vectors (vector x ′ _α and vector x ′ _β ) (step C-ST2-3), and writes it in the memory.
Thereafter, the i-th component (0 ≦ i <18, where i is an integer) of the vector x _'alpha and the vector x' _beta, respectively, x _'.alpha.i, x' referred to _.beta.i.

この他、（式１３）の代わりに（式１４）を用いて識別影響度を算出してもよい。

また、（式１３）の代わりに（式１５）を用いて識別影響度を算出してもよい。

For example, with respect to all the components of the feature vector x, one identification influence degree is calculated by the following (formula 13).

In addition, the identification influence degree may be calculated using (Expression 14) instead of (Expression 13).

Further, the identification influence degree may be calculated using (Expression 15) instead of (Expression 13).

  The subspace projection unit 29 executes processing steps A-ST3-1 to A-ST3-3 in the flowchart shown in FIG. However, in the third embodiment, the feature vector x in step A-ST3-3 is the feature vector acquired in processing step C-ST1 in FIG. With the above processing, the subspace projection unit 29 ends the subspace projection and outputs the projection vector x ′ to the discrimination influence calculation unit 24.

  In the type identification influence calculation processing in step C-ST2, the identification influence calculated by the identification influence calculation unit 24 is a feature vector with respect to a difference or ratio between the types of evaluation scales targeted by the identification influence. Means the influence of each component.

  The identification influence degree output unit 31 outputs the identification influence degree sequence calculated by the identification influence degree calculation unit 24 to the outside (step C-ST3). As an example, the identification influence degree sequence is output to the display device 3 as a numeric string, and the numeric string of the identification influence degree sequence is displayed on the screen of the display device 3.

  In addition, the identification influence degree sequence may be output to a printer connected to the computer 2 as, for example, a numeric string, and the numeric string of the identification influence degree sequence may be printed. Further, the identification influence degree sequence may be stored in a hard disk device connected to the computer 2. Further, the identification influence degree sequence may be transmitted to another computer via the network.

  As described above, according to the third embodiment, it is possible to obtain a guideline for searching for the cause of erroneous identification in pattern recognition (image identification) performed by other than the computer 2. Specifically, since the influence of each component of the feature vector on the difference or ratio of the evaluation scale targeted by the identification influence is output, it is possible to know how many types of features represent the difference between the types. It becomes possible.

  For example, if one type is the correct answer type and the other one type is highly likely to be misidentified, the difference between the two types is output, so that what type of features affected the misidentification Can be visualized. Therefore, by observing the visualized image, it becomes possible to obtain a clue to know what kind of feature is caused by misidentification, and in the image identification apparatus using the evaluation scale that is the target of the identification influence degree It leads to improvement of the feature extraction method and evaluation scale calculation algorithm.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  That is, the present invention is not limited to the above-described embodiment, and may be modified in the implementation stage by reducing or combining the components without departing from the scope of the invention. For example, each component may be realized by a program installed in a storage such as a hard disk device of a computer, and the above program is stored in a computer-readable electronic medium: electronic media, and the program is transferred from the electronic medium to the computer. The computer may realize the functions of the present invention by causing the computer to read them.

  Examples of the electronic medium include a recording medium such as a CD-ROM, flash memory, and removable media. Further, the configuration may be realized by distributing and storing components in different computers connected via a network, and communicating between computers in which the components are functioning.

  In the above-described embodiment, an image is a processing target. However, if the image can be described as a pattern, the processing target may be other than an image, and includes, for example, an audio waveform.

  DESCRIPTION OF SYMBOLS 1 ... Scanner, 2 ... Computer, 3 ... Display apparatus, 21 ... Image reception part, 22 ... Image memory | storage part, 23 ... Feature extraction part, 24 ... Identification influence calculation part, 25 ... Identification dictionary storage part, 26 ... Image generation 27: Image output unit, 28: Feature input unit, 29 ... Subspace projection unit, 30 ... Feature vector storage unit, 31 ... Discrimination influence output unit

Claims (5)

An acquisition unit for acquiring a pattern to be processed;
A feature calculation unit that calculates a feature vector indicating a feature of each divided pattern after dividing the pattern obtained from the acquisition unit into a plurality of patterns according to a predetermined rule ;
A dictionary storage unit in which the types of identification candidates and a set of representative vectors of the identification candidates are stored correspondingly;
The set of representative vectors corresponding to the type is acquired from the dictionary storage unit, and the similarity or difference between the processing target and the identification candidate obtained from the set and the feature vector is used as each component of the feature vector. An identification influence calculation unit that decomposes correspondingly and calculates the decomposed value as a degree that each component of the feature vector affects pattern identification ;
Color information corresponding to the decomposed value calculated by the identification influence degree calculation unit is generated, the color information is associated with the divided patterns, and color information of the color information and the divided patterns An image generation unit for generating an image indicating correspondence;
An information processing apparatus comprising: an image output unit that outputs an image generated by the image generation unit . The set of representative vectors is
A base vector group of a subspace of the feature space in which the feature vector exists;
The degree to which each component of the feature vector affects pattern identification is determined by the value of each component of the projection vector when the projection vector obtained by projecting the feature vector onto the partial space is described on the feature space. claim 1 Symbol placement of the information processing apparatus, characterized in that a reference. The set of representative vectors is a base vector group of a subspace of the feature space where the feature vector exists,
The degree to which each component of the feature vector influences the pattern identification is determined when the projection vector obtained by projecting the feature vector onto the partial space is described on the feature space. The information processing apparatus according to claim 1, wherein a value of each component of the difference vector is used as a reference. The identification influence calculation unit
Decompose the degree of similarity or dissimilarity with the pattern to be processed for a plurality of identification candidates, and each component of the feature vector affects the pattern identification with a numerical value representing the magnitude relationship between the decomposed values for the plurality of identification candidates claim 1 Symbol placement of the information processing apparatus, and calculates a degree of. In an information processing method in an information processing apparatus having an acquisition unit, a feature calculation unit, an identification influence calculation unit , an image generation unit, and an image output unit ,
The acquisition unit acquires a pattern to be processed,
After the pattern obtained by the acquisition unit is divided into a plurality of patterns according to a predetermined rule, the feature calculation unit calculates a feature vector indicating the feature of each divided pattern , and the type of identification candidate and the identification candidate The identification influence degree calculation unit acquires the set of representative vectors corresponding to the type from the dictionary storage unit in which the set of representative vectors is stored correspondingly, and the processing obtained from the set and the feature vector The degree of similarity or difference between the target and the identification candidate is decomposed so as to correspond to each component of the feature vector, and the decomposed value is calculated as a degree that each component of the feature vector affects pattern identification ,
The image generation unit generates color information corresponding to the decomposed value calculated by the identification influence degree calculation unit, associates the color information with the divided patterns, and the color information and the divided patterns. An image showing the correspondence of color information with
The information processing method , wherein the image output unit outputs an image generated by the image generation unit .

Images