JP3355015B2 - Image processing method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method for improving recognition performance when recognizing an object existing in an image by using digital image processing technology.

[0002]

2. Description of the Related Art FIG. 39 shows, for example, "Model Matching Using Reference Points Accumulating Tentacle Features" (Matsuo Sano, Shinichi Meguro,
Akira Ishii: Explanatory diagram showing a conventional image processing method shown in IEICE Pattern Recognition and Understanding Study Group material PRU86-90). In the figure, O is a reference point for describing the shape of the model of the detection target graphic, x is a vector toward the reference point O at each point on the boundary of the model graphic, and φ is the direction of the gradient of the boundary line. It is.

Next, the operation will be described. In this image processing method, pattern matching by generalized Hough transform is performed. First, a reference point O for describing the shape of the model of the graphic to be detected is selected, and a coordinate system having the reference point O as an origin is determined. Then, at each point on the boundary of the model graphic, vector toward the reference point _{O x i = (r i,} α i) and determining the direction phi _i of the gradient of the border. Note that r is the length of the vector, and α is the direction angle. This vector x
Is regarded as a function of the gradient direction φ, and the shape of the model is represented as a table called an R table shown below.

[0004]

[Table 1]

According to the R table shown in Table 1, for example, the vector x (2Δφ) is x (2Δφ) = (r (2Δφ), α
(2Δφ)) = {x ₂₁ , x ₂₂ ,...}.

Here, coordinate conversion for a two-dimensional figure is as follows:
It is represented by four parameters of translation (x, y), rotation angle θ, and enlargement / reduction ratio s. A four-dimensional space spanned by these parameters is defined as a parameter space.
Further, it is assumed that the input image of the conversion includes the edge points extracted by the edge detection or the like, and the gradient direction ψ is obtained. Each edge point (u _j , v _j ) on the image (direction of gradient ψ
Perform the following operations for _j ). That is, for all combinations of (θ, s) in the parameter space, x and y satisfying the following expressions (1) and (2) are obtained, and the elements (x, y, θ, s) of the parameter space are obtained. ) Is increased by one, and (x, y, θ, s) having a peak is obtained.

X = u _j + r (ψ _j −θ) × cos [α (ψ _j −θ) + θ] (1) y = v _j + r (ψ _j −θ) × sin [α (ψ _j −θ) + θ] (2)

[0008]

Since the conventional image processing method is configured as described above, it includes a large number of parameters for pattern matching by generalized Hough transform, and the search space becomes large. Therefore, there has been a problem that the operation cost increases and the stability decreases.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and has as its object to provide an image processing method that reduces processing costs and improves stability.

[0010]

[0011]

According to the first aspect of the present invention, there is provided an image processing method comprising: performing preprocessing on an original image;
Is evaluated, and the center of gravity of the object is evaluated based on the evaluation result.
Is determined, and the obtained object is centered on the obtained center of gravity.
Recognizing attributes such as size, type (shape), and posture
In addition, the localization of the target is evaluated in parallel at different scales using multiple evaluation functions, and the localization of the target is reevaluated by integrating the multiple evaluation results obtained. The position of the center of gravity of the object is determined from the result.

According to a second aspect of the present invention, in the image processing method, the localization of an object is evaluated in parallel at different scales using a plurality of evaluation functions, and the object is evaluated from a plurality of obtained evaluation results. Are extracted in parallel, and a plurality of candidate center-of-gravity position candidates are obtained.

According to a third aspect of the present invention, there is provided an image processing method for converting an input image into a plurality of images having different resolutions and using the same evaluation function for each resolution-converted image. The localization is evaluated, the localization of the target is re-evaluated by integrating a plurality of obtained evaluation results, and the position of the center of gravity of the target is determined from the re-evaluation result.

According to a fourth aspect of the present invention, there is provided an image processing method for converting an input image into a plurality of images having different resolutions and using the same evaluation function for each resolution-converted image. The localization is evaluated, and the barycentric positions of the objects are extracted in parallel from the obtained plurality of evaluation results to obtain the barycentric position candidates of the plurality of objects.

According to a fifth aspect of the present invention, there is provided the image processing method according to the first aspect of the present invention, wherein the preprocessing is performed on the original image and then the localization of the object is performed.
Performs an evaluation and determines the center of gravity of the object from the evaluation result
And the size of the object around the obtained center of gravity,
Recognize attributes such as type (shape), posture, etc.
The spread amount around the position where the target exists
When examining radially in multiple directions, the center of gravity of the target
Input image that extends radially in multiple directions around
Generates one-dimensional image data and changes depending on the amount of spread
Multiply-accumulate with a one-dimensional function
Execute and calculate the amount of spread at which the calculation result is maximal
The spread of the elephant
One-dimensional function that changes depending on the amount and the input image
Result of multiply-accumulate with one-dimensional data
It will be maximum when it matches the size of the object in that direction
This is set in advance as described above, and the target attribute is recognized by paying attention to the obtained spread amount in each direction.

[0016] In the image processing method according to the invention of claim 6, the amount of spread of the subject examined in parallel radially in a plurality of directions around the center of the plurality of candidate positions where the target is present, obtained The attribute of each target candidate is extracted in parallel by paying attention to the spread amount in each direction, and they are integrated to extract the final attribute of the target.

[0017] In the image processing method according to the invention of claim 7, the target upon spreading of extraction, of the input image that extends radially into a plurality of directions around the center of gravity position of the object in one dimension Generates data, executes a product-sum operation with a one-dimensional function that varies depending on the spread amount, while changing the spread amount, and sets the spread amount at which the calculation result reaches a maximum as the target spread amount in that direction. At this time, the result of the product-sum operation of the one-dimensional function that changes depending on the spread amount and the one-dimensional data in a certain direction of the input image indicates that the spread amount matches the size of the target in that direction. It is set in advance so that it becomes a maximum when it is done.

[0018]

[0019]

[0020]

[0021] In the image processing method according to the invention of claim 8 is to extract the maximum value of the evaluation results of the obtained localization as the center-of-gravity position of the object.

According to a ninth aspect of the present invention, in the image processing method, an image having an evaluation value of locality of an object as a luminance distribution is binarized by a predetermined threshold value. The zero-order and first-order moments are calculated to extract the position of the center of gravity of the object.

[0023] In the image processing method according to the invention of claim 10 includes a luminance distribution of the central small area set around the center of gravity of the object, surrounding the small region set around the center small region Compare the luminance distribution of the target, determine whether the surrounding small area is the target or the background, fuse the surrounding small area determined to be the target with the center small area, expand the central small area sequentially to the boundary of the target, and Is to extract the size of.

[0024] In the image processing method according to the invention of claim 11 binarizes the image evaluation results of localization of target luminance distribution with a predetermined threshold value, in that the binary image The zeroth, first, and second moments are calculated to extract the target attribute.

[0025]

[0026]

According to the image processing method of the present invention, the position of the center of gravity of the target extracted by focusing on the localization of the target is determined.
In addition to recognizing the attributes of the object at the center, re-evaluating the locality by integrating the localization evaluation results of the objects evaluated in parallel on different scales, and re-evaluating the locality, the center of gravity of the object is obtained from the re-evaluation To reduce the size of the search space
To reduce processing costs and improve matching stability
Allowed, to recognize the target of various sizes.

The image processing method according to the second aspect of the present invention is a method of extracting a plurality of candidate center-of-gravity positions by extracting the center-of-gravity position of each object from each evaluation result of the localization of the object evaluated in parallel at different scales. Are obtained in parallel so that objects of various sizes can be recognized.

The image processing method according to the third aspect of the present invention evaluates the locality of an object using the same evaluation function for a plurality of resolution-converted images having different resolutions, and integrates the evaluation results. By extracting the position of the center of gravity of the object from the re-evaluation result of the re-evaluation of the locality, objects of various sizes can be recognized.

[0029] In the image processing method in the invention of claim 4 is to evaluate the localization of the target with the same evaluation function for resolution of different resolution-converted image, of the object from the evaluation results By extracting the barycentric positions and obtaining a plurality of barycentric position candidates in parallel, objects of various sizes can be recognized.

Further, the image processing method in the invention of claim 5, the subject extracted by focusing on the localization of the target heavy
In addition to recognizing the attributes of the object around the center of gravity and calculating the spread of the object in multiple directions around the center of gravity of the object,
For one-dimensional data of the input image extended in a radial pattern and the amount of spread
The product-sum operation with a one-dimensional function that changes depending on the spread amount
And calculate the amount of spread that maximizes the result.
The size of the search space can be reduced by extracting as the amount of spread of the object in the direction and extracting the attributes of the object from them.
Reduce processing costs, improve matching stability,
A search space can be reduced in size to reduce processing costs and improve matching stability. Also,
It is possible to stably extract the amount of waste.

[0031] In the image processing method in the invention of claim 6, radially in a plurality of directions around the center of the plurality of candidate positions that exist in the subject by examining the amount of spread of each target in parallel, than those By extracting the attributes of each target candidate in parallel, integrating them, and extracting the final target attributes, it is possible to reduce the size of the search space, reduce processing costs, and improve matching stability. Improve.

[0032] In the image processing method in the invention of claim 7, the one-dimensional data of the input image extending radially from the center of gravity of the subject, the one-dimensional function that varies depending on the amount of spread By calculating the product-sum operation while changing the spread amount and extracting the spread amount at which the result is maximum as the target spread amount in that direction, it is possible to stably extract the spread amount.

[0033]

[0034]

[0035]

[0036] In the image processing method in the invention of claim 8, by extracting the local maximum value of the resulting localization evaluation results as the center-of-gravity position of the object, stably performed by the extraction of the centroid position Is possible.

In the image processing method according to the ninth aspect of the present invention, the zero-order and first-order moments are calculated in a binarized image of an image having a localization evaluation value as a luminance distribution. It is possible to stably extract the position of the center of gravity of the object from the evaluation result of the presence.

The image processing method according to the tenth aspect of the present invention is the image processing method according to the present invention, wherein the luminance distribution between the center small area set at the center of gravity and the surrounding small area set around the center is determined based on a comparison. If the area is determined to be the target, the surrounding small area is fused to the central small area, and the central small area is sequentially expanded to the boundary of the target to extract the size of the target, thereby obtaining the target size. Extraction can be performed stably.

In the image processing method according to the eleventh aspect , a zero-order, first-order, or second-order moment is calculated in a binarized image of an image having an evaluation value of locality as a luminance distribution. Thereby, it is possible to stably extract a target attribute.

[0040]

【Example】Reference Example 1. Reference Example 1 to Help Understand the Invention The illustration about the figure
I will tell. Figure 1Reference Example 1Embodiment of Image Processing Method by Using
FIG. 2 is a flowchart showing the target extraction process.
6 is a flowchart showing details of the process. Also, FIG.
A block diagram showing a configuration example of an image processing apparatus to which the image processing method is applied.
FIG. 4 is a lock diagram, and FIG.
FIG.

In FIG. 3, reference numeral 1 denotes an image input unit for taking in an image by a camera or the like and digitizing the image. Reference numeral 2 denotes an image digitized by the image input unit 1 (hereinafter referred to as an original image). This is a pre-processing unit that performs pre-processing for removing high-frequency noise and a slow change in luminance in an image. 3 evaluates the degree of local concentration (hereinafter, referred to as locality) of elements constituting the object in the image pre-processed by the pre-processing unit 2 (hereinafter, referred to as pre-processed image), and Reference numeral 4 denotes a target extracting unit for extracting the position of the center of gravity of the target by focusing on the position, size, type (shape), posture, etc. of the target with the center of gravity of the target obtained by the target extracting unit 3 as the center. This is a target recognition unit for extracting attributes. Reference numeral 5 denotes an original image input from the image input unit 1 to the pre-processing unit 2, reference numeral 6 denotes a pre-processed image subjected to pre-processing by the pre-processing unit 2, and reference numeral 7 denotes a target extracted by the target extraction unit 3. The barycenter position 8 is the attribute of the target extracted by the target recognition unit 4.

In FIG. 4, reference numeral 9 denotes the degree of local concentration of the constituent elements of the object in the pre-processed image 6 pre-processed by the pre-processing unit 2, that is, the evaluation of the locality of the object. The localization evaluation unit 10 performs the extraction of the target center of gravity position 7 by paying attention to the localization evaluation result by the localization evaluation unit 9,
Reference numeral 11 denotes a center of gravity extraction unit that outputs to the object recognition unit 4. Reference numeral 11 denotes a localization evaluation result sent from the localization evaluation unit 9 to the center of gravity extraction unit 10.

Next, the operation of the image processing apparatus shown in FIG. 3 will be described with reference to the flowchart of FIG. An image based on an analog signal captured by a camera or the like in the image input unit 1 is subjected to analog / digital conversion to become an original image 5 based on a digital signal (step ST1). The original image 5 is sent to the pre-processing unit 2 and subjected to pre-processing, and the pre-processed image 6 from which high-frequency noise and a slow luminance change in the image have been removed.
(Step ST2), and are input to the target extraction unit 3 and the target recognition unit 4. The target extraction unit 3 extracts the center-of-gravity position 7 of the target by focusing on the locality of the input preprocessed image 6,
It is sent to the object recognition unit 4 (step ST3). The target recognizing unit 4 extracts the target attribute 8 from the position of the center of gravity 7 of the target and the pre-processed image 6 input from the pre-processing unit 2 and outputs the same to the outside (step ST4).

Here, in step ST13, when the target extracting unit 3 extracts a plurality of targets, the target recognizing unit 4
Then, in step ST4, it is also possible to recognize the attributes 8 of a plurality of objects in parallel with the existence position of each object as a center.

The image input to the target extracting unit 3 and the image input to the target recognizing unit 4 use the same preprocessed image 6 which has been subjected to the same preprocessing. It is also possible to generate two types of processed preprocessed images, and input one of them to the target extraction unit 3 and the other to the target recognition unit 4.

Next, the operation of the target extracting section 3 shown in FIG. 4 will be described in more detail with reference to the flowchart of FIG. First, the locality evaluation unit 9 evaluates locality, which is the degree of local concentration of elements constituting a target in the pre-processed image 6 input to the target extraction unit 3 (step S).
T5). The obtained localization evaluation result 11 is the centroid extraction unit 1
0, and the center-of-gravity extraction unit 11 evaluates its locality as 1
The position of the center of gravity 7 of the object is extracted from the object recognition unit 1, and is extracted by the object recognition unit 4.
(Step ST6). In the example shown, M
Objects exist.

Here, for example, the input image is a binary image, and the points of "1" constituting the object are densely present.
It is assumed that the "1" points that make up the background are sparse. For such a binary image, a local region is considered around a certain point of interest, and the sum of the luminances in the local region (in this case, since it is a binary image, it is equal to the sum of the number of "1" points) Is calculated, the sum is large in a portion where the points of “1” are densely present like the target portion, and small in a portion where the points of “1” are sparsely present like the background portion. Become. The locality evaluation unit 9 adds the luminance values in the local area around the point of interest in this manner, thereby obtaining the degree of local concentration of the elements constituting the target,
That is, the localization of the target is evaluated, and the result is output as the localization evaluation result 11.

Next, it is assumed that the input image is a multi-valued image, and that the points constituting the object have high luminance and the points constituting the background have low luminance. When the sum of the luminance is calculated in a local area around a point of interest for such a multi-valued image, the sum of the luminance is large in a portion where a point having high luminance exists, such as a target portion. And becomes smaller in a portion where a point having low luminance exists, such as a background portion. The locality evaluation unit 9 evaluates the locality of the target by adding the luminance values in the local area around the point of interest in this way, and outputs the result as a locality evaluation result 11.

Further, when the locality evaluation unit 9 evaluates the locality of the object, the processing of adding the luminance values in the local area around the point of interest corresponds to the average value filter processing. here,
Instead of a simple average filter processing for simply averaging the brightness in the local area, execute the product-sum operation of the weight coefficient proportional to the Gaussian function centered on the point of interest and the brightness in the local area. Thus, the closer to the point of interest, the greater the contribution of the luminance of that point, and the farther away from the point of interest, the smaller the contribution of the luminance of that point. This corresponds to Gaussian filter processing. The locality evaluation unit 9 can also be realized by such a Gaussian filter process. Here, the two-dimensional Gaussian function of the standard deviation σ is represented by the following equation (3).

[0050]

(Equation 1)

When an average value filter is used to evaluate the locality of an object, more global information can be obtained by increasing the local area, and more detailed information can be obtained by reducing the local area. Is obtained. When a Gaussian filter is applied, more global information can be obtained by increasing the standard deviation σ, and more detailed information can be obtained by reducing σ. As described above, by changing the local region of the average filter and the standard deviation σ of the Gaussian filter, the localization evaluation unit 9 capable of evaluating the localization of an object on various scales is realized. Can be.

[0052]Embodiment 1 FIG.  Next, the present inventionExample 1Will be described with reference to FIG. FIG.
IsExample 1Of Object Extraction Processing in Image Processing Method
FIG. 6 is a flowchart showing details, and FIG.
Internal structure of the target extraction unit 3 in the image processing apparatus to which the method is applied
It is a block diagram showing composition.

In FIG. 6, 9-1 to 9-L denote L evaluations (L is an integer of 1 or more) of localization in the preprocessed image 6 preprocessed by the preprocessing unit 2. This is a localization evaluation unit performed on the scale shown in FIG. Reference numeral 12 denotes an integration unit that integrates the localization evaluation results evaluated on the L different scales by the respective localization evaluation units 9-1 to 9-L and re-evaluates the locality. A centroid extracting unit that extracts the target centroid position 7 from the localization re-evaluation result obtained by the integration unit 12. 11-1 to 11-L are localization evaluation results evaluated at L different scales by the localization evaluation units 9-1 to 9-L, and 13 is obtained by the integration unit 12. It is the re-evaluation result of the obtained localization.

Next, the operation will be described with reference to the flowchart of FIG. The pre-processed image 6 input to the target extraction unit 3 is input to L localization evaluation units 9-1 to 9-L, and the localization at different scales by different L evaluation functions respectively. Is evaluated (step ST
11-1 to ST11-L). Each of these localization evaluation units 9-
Evaluation results of L localization obtained in 1 to 9-L 11-
1 to 11-L are sent to the integration unit 12, and the integration unit 12 integrates them and performs re-evaluation of locality (step ST1).
2). The re-evaluation result 13 of the locality integrated by the integration unit 12 is input to the center-of-gravity extraction unit 10, and the target center-of-gravity position 7 is extracted and output to the object recognition unit 4 (step ST1).
3). Note that, in this case as well, an example in which M objects exist is shown.

Here, the evaluation function used in each of the localization evaluation units 9-1 to 9-L that evaluates the localization of the object on L different scales is, for example, an average value filter. Can be realized by changing the size of the local region and the standard deviation σ of the Gaussian filter.

The integration of the plurality of localization evaluation results 11-1 to 11-L in the integration unit 12 is realized by a method such as an addition calculation, an average calculation, a maximum value calculation, and a median value calculation. Can be.

[0057]Embodiment 2. FIG.  Next, the present inventionExample 2Will be described with reference to FIG. FIG.
IsExample 2Of Object Extraction Processing in Image Processing Method
FIG. 8 is a flowchart showing details, and FIG.
Internal structure of the target extraction unit 3 in the image processing apparatus to which the method is applied
It is a block diagram showing composition. In FIG. 8, 14 is local
Sex evaluation sections 9-1 to 9-L are respectively different scales
Subject from evaluation results 11-1 to 11-L of evaluated locality
L center-of-gravity extraction units for extracting the centroid positions of
15-1 to 15-L are calculated by these respective centroid extracting units 14.
L different object centroid positions with different extracted scales
It is a candidate.

Next, the operation will be described with reference to the flowchart of FIG. The preprocessed image 6 input to the target extraction unit 3 is input to L localization evaluation units 9-1 to 9-L, and each of the localization evaluation units 9-1 to 9-L individually Are evaluated at different scales by using L evaluation functions (step ST21-1).
-ST21-L). Each of these localization evaluation units 9-1 to 9-
L localization evaluation results 11-1 to L obtained in L
11-L is sent to the center-of-gravity extractor 14 prepared corresponding to each of them.
Based on 1-1 to 11-L, L possible center-of-gravity position candidates 15-1 to 15-L are extracted (step ST12-1 to ST12-1).
ST12-L), and output it to the target recognition unit 4. In the illustrated example, there are M _{1 to} M _L objects.

Also in this case, each of the locality evaluation units 9-1 to 9-, which evaluates the locality of the object on L different scales.
The evaluation function used for each of L can be realized by, for example, changing the size of the local region of the average filter or the standard deviation σ of the Gaussian filter.

The number of objects and their locations are different for each scale, and different L types are obtained. These results are used in subsequent processes to determine attributes such as the size, shape, and posture of the objects. After recognition, they are integrated.

[0061]Embodiment 3 FIG.  Next, the present inventionExample 3Will be described with reference to FIG. FIG.
IsExample 3Of Object Extraction Processing in Image Processing Method
FIG. 10 is a flowchart showing details, and FIG.
Of the target extraction unit 3 in the image processing apparatus to which the image processing method is applied
FIG. 3 is a block diagram illustrating a configuration.

In FIG. 10, reference numerals 16-1 to 16-L denote resolution conversion units for converting the input preprocessed image 6 into images of L different resolutions, respectively, and reference numeral 17 denotes each of the resolution conversion units 16-1 to 16-L. A localization evaluation unit that evaluates the locality of the target with respect to the resolution-converted images converted to the respective resolutions in 16-L. 18-1 to 18-L are resolution-converted images converted to different resolutions, and 11-1 to 18-L.
11-L is each of these resolution converted images 18-1 to 18-L
It is the evaluation result of the localization obtained from. In addition, 12 is an integration unit that integrates the plurality of localization evaluation results 11-1 to 11-L, 13 is a re-evaluation result of the localization obtained by the integration unit 12, and 10 is this localization evaluation result. A center-of-gravity extractor 7 for extracting the position of the center of gravity of the target from the re-evaluation result 13.
The position of the center of gravity of the object extracted by 0 corresponds to the part denoted by the same reference numeral in FIG.

Next, the operation will be described with reference to the flowchart of FIG. The pre-processed image 6 input to the target extraction unit 3 includes L resolution conversion units 16-1 to 16-L
Are input to the resolution conversion units 16-1 to 16-L.
18-L (steps ST31-1 to ST3-1).
1-L). These resolution converted images 18-1 to 18-L
Are sent to the localization evaluation units 17 prepared corresponding to the respective components, and the localization of the target is evaluated in each of the localization evaluation units 17 (steps ST32-1 to ST32-L). The L localization evaluation results 11-1 to 11-L obtained by the respective localization evaluation units 17 are sent to the integration unit 13, where they are integrated and re-evaluated. (Step ST3
3). The re-evaluation result 13 of the locality is input to the center-of-gravity extracting unit 10, and the center-of-gravity position 7 of the object is extracted and the object recognizing unit 4
(Step ST34).

Here, the resolution converters 16-1 to 16-L
In order to convert the input pre-processed image 6 into a plurality of different resolution-converted images 18-1 to 18-L, for example, the input pre-processed image 6 is thinned out for each of a different number of pixels. Can be realized.

The resolution converters 16-1 to 16-
In L, the preprocessed image 6 is converted into a plurality of resolution-converted images 18-1 to 18-1.
At the time of conversion to 18-L, the luminance value of a pixel in the preprocessed image 6 is set to a local area of a certain size centered on the pixel, and is replaced with the average value of the luminance values of the pixels in the local area. It may be realized by doing so.

The resolution conversion units 16-1 to 16-
In L, the preprocessed image 6 is converted into a plurality of resolution-converted images 18-1 to 18-1.
At the time of conversion to 18-L, the luminance value of a pixel in the preprocessed image 6 is set to a local area of a certain size centered on the pixel, and is replaced with the median value of the luminance value of the pixel in the local area. Alternatively, it may be realized.

Further, for example, when the input pre-processed image 6 is a multi-valued image, in order to convert it into a plurality of resolution-converted images 18-1 to 18-L having different resolutions, the resolution converter 16- It can be realized by a method of converting the luminance gradation of the pre-processed image 6 into images having different luminance gradations from 1 to 16-L.

A plurality of different resolution-converted images 18-
The plurality of locality evaluation units 17 prepared corresponding to 1 to 18-L can evaluate the locality of each resolution-converted image 18-1 to 18-L using the same evaluation function. Since the same processing may be performed on each of the resolution-converted images 18-1 to 18-L, L types of resolution-converted images 18-1 to 18-L may be used.
-It is not necessary to change the configuration for each L, and the same configuration can be used.

The integration of the plurality of localization evaluation results 11-1 to 11-L in the integration unit 12 is realized by a method such as addition calculation, average calculation, maximum value calculation, and median value calculation. Can be.

[0070]Embodiment 4. FIG.  Next, the present inventionExample 4Will be described with reference to FIG. Figure 1
1 isExample 4Object Extraction Processing in Image Processing Method by Image Processing
6 is a flowchart showing details of the process. Also, FIG.
Object extraction unit in an image processing apparatus to which the image processing method is applied
FIG. 12 is a block diagram showing the internal configuration of FIG.
14 is a centroid extracting unit, and 16-1 to 16-L are resolutions
Conversion unit, 17 is a localization evaluation unit, 6 is a preprocessed image, and 11-
1 to 11-L are localization evaluation results, 15-1 to 15-L
Is the candidate position of the center of gravity of the object, and 18-1 to 18-L are
These are the same images as those shown in FIG. 8 or FIG.
It is equivalent to the part which did.

Next, the operation will be described with reference to the flowchart of FIG. The pre-processed image 6 input to the target extraction unit 3 includes L resolution conversion units 16-1 to 16-.
L, the images are converted into resolution-converted images 18-1 to 18-L having different resolutions (step ST41-).
1 to ST41-L). These resolution converted images 18-1
To 18-L are sent to the corresponding locality evaluation unit 17 to evaluate the locality of the target (step ST42-1).
-42-L). The L localization evaluation results 11-1 to 11-L obtained by the respective localization evaluation units 17 are sent to the corresponding centroid extraction units 14, and each centroid extraction unit 14 uses them. L candidate centroid position candidates 15-1 to 15-L
Is extracted (steps ST43-1 to ST43-L).
Note that, also in this case, an example in which M _{1 to} M _L objects exist respectively is shown.

Here, the resolution converters 16-1 to 16-L
In order to convert the input pre-processed image 6 into a plurality of different resolution-converted images 18-1 to 18-L, for example, the input pre-processed image 6 is thinned out for every different number of pixels. It can be realized by a method such as replacing the average value or the median value of the luminance values of the pixels in the local region.

If the input pre-processed image 6 is a multi-valued image, for example, the resolution converter 16-1 is converted by a method of converting the brightness gradations of the pre-processed image 6 into images having different brightness gradations. 16-L, it can be converted into a plurality of resolution-converted images 18-1 to 18-L having different resolutions.

A plurality of different resolution-converted images 18-
It is not necessary to prepare different evaluation functions for evaluating the locality of the object corresponding to 1 to 18-L, and the same processing may be performed on each of the resolution converted images 18-1 to 18-L. Therefore, it is not necessary to change the configuration of each of the L types of resolution-converted images 18-1 to 18-L, and the localization evaluation units 17 can use the same configuration.

In addition, the number of objects and their locations can be different for each scale in L types, and these results are used in subsequent processing to modify attributes such as the size, shape, and posture of the objects. After recognition, they are integrated.

[0076]Embodiment 5 FIG.  Next, the present inventionExample 5Will be described with reference to FIG. Figure 1
3 isExample 5Recognition Processing in Image Processing Method by Image Processing
FIG. 14 is a flowchart showing details of
It is explanatory drawing which shows the principle of recognition. In FIG. 14, 21
24 to 24 are examples of objects, and 25 to 28 are each object
Each direction around 21 to 24 (8 directions in the illustrated example)
This is an extraction result of the spread amount of the direction.

FIG. 15 is a block diagram showing the internal configuration of the object recognition unit 4 in the image processing apparatus to which the above-described image processing method is applied. In FIG. 15, reference numeral 31 denotes each of directions (T in the illustrated example) around the target from the input preprocessed image 6 and the center of gravity position 7 of the target (in the illustrated example, there are M objects). A spread amount extraction unit that extracts the spread amount,
Reference numeral 32 denotes an attribute extraction unit that extracts attributes 8 such as the size, shape, and posture of the target based on the spread amount extracted by the spread amount extraction unit 31, and outputs the attributes to the outside. Also, 33
Are extracted by the spread amount extraction unit 31 and are extracted by the attribute extraction unit 3
2 is the spread amount of the object sent to 2.

Next, the operation will be described with reference to the flowchart of FIG. The pre-processed image 6 input from the pre-processing unit 2 to the target extracting unit 4 is sent to the spread amount extracting unit 31 in each direction around the target center of gravity 7 of the target sent from the target extracting unit 3. Then, the target spread amount 33 is extracted radially (step ST51). Next, the attribute extraction unit 32 extracts the attribute of the target from the spread amount in each direction of the target extracted by the spread amount extraction unit 31 (step ST52).

Next, the principle of object attribute extraction will be described with reference to FIG. For each of the objects indicated by 21 to 24, when the amount of spread in eight directions around the center of gravity is extracted, the result of extraction of the amount of spread indicated by 25 to 28 is obtained. Here, when the target has a cross shape like 21, the extraction result of the spread amount is as 25. On the other hand, if the target is a cross shape similar to the target 21 such as 22 and the posture or the same, the extraction result of the spread amount is as shown in 26. Comparing the extraction results 26 and 25 of these spread amounts, it is understood that the spread amounts in each direction are in a proportional relationship. Thus, the size of the object can be known from the average of the spread amounts in each direction.

When the object has a cross shape as shown at 21, the result of extraction of the spread amount has four peaks as shown at 85. On the other hand, when the target is a rectangle such as 23, the extraction result of the spread amount has only two peaks as indicated by 27. Thus, the type (shape) of the target can be recognized from the distribution of the spread amount in each direction. The shape of the target can be recognized in more detail by increasing the number of directions in which the surrounding spread amount is checked and checking the spread amount more finely.

When the object is a rectangle having a horizontal direction such as 23, the result of extracting the spread amount has a peak in the direction 0 and the direction 4 as indicated by 27. On the other hand, if the target is a rectangle such as 24 obliquely oriented in the same shape as 23, the extraction result of the spread amount has peaks in the directions 3 and 7 as indicated by 27. As described above, the change in the posture (rotation) of the target can be recognized by shifting the distribution of the spread amount to the adjacent direction.

Here, if the shape and orientation of the target object are determined in advance, the spread amount extraction unit 31
It is possible to reduce the number of directions when extracting the spread amount of the object radially in each of the surrounding directions. For example, if the target is known to be a circle, this can be realized by a configuration in which only the amount of spread in any one direction is checked. Further, for example, when it is known that the target is a horizontally installed rectangle, it can be realized by a configuration in which only the spread amount in four directions, up, down, left, and right is checked.

[0083]Embodiment 6 FIG.  Next, the present inventionExample 6Will be described with reference to FIG. Figure 1
6 isExample 6Recognition Processing in Image Processing Method by Image Processing
FIG. 17 is a flowchart showing details of the above-mentioned image.
Of the attribute extraction unit 4 in the image processing apparatus to which the processing method is applied
It is a block diagram which shows a part structure.

In FIG. 17, reference numerals 31 are prepared in correspondence with L possible center-of-gravity position candidates 15-1 to 15-L.
15-L (in the example shown, there are M _{1 to} M _L targets for each scale), and extracts L spread amounts for extracting the spread amount in each direction around the target for each scale. Department. Reference numeral 32 denotes an attribute extracting unit which is provided corresponding to each of the spread amount extracting units 31 and extracts target attributes from the spread amounts extracted for each scale.
Is an integration unit that integrates target attributes for each scale extracted by each attribute extraction unit 32. In addition, 33-1 to 33
−L is the spread amount of the target in a plurality of directions (T directions in the illustrated example) around the center of gravity of the target extracted by each spread amount extraction unit 31 for each scale; 3
5-L is the attribute of the target candidate extracted by each attribute extraction unit 32, and 8 is integrated by the integration unit 34 (M in the illustrated example).
Attributes).

Next, the operation will be described with reference to the flowchart of FIG. The pre-processed image 6 input to the target extraction unit 4 is extracted at each scale in each spread amount extraction unit 31 and each candidate center of gravity position 15-1 to 15- of the target input to the target extraction unit 4 is extracted. The spread amounts 33-1 to 33-L in the respective directions around the target around L are extracted (steps ST61-1 to ST61-L). next,
The attribute extracting unit 32 extracts the target attributes 35-1 to 35-L of each scale from the spread amounts 33-1 to 33-L of the targets of each scale (step ST62-1).
-ST62-L). The integrating unit 34 extracts the target attribute 8 by integrating the target attributes 35-1 to 35-L of each scale (step ST6).
3) Re-recognize attributes such as the size, shape, and posture of the target.

Here, each spread amount extraction unit 31 describes the contents of the process of extracting the spread amount of the target radially in each direction around the target, based on the input L types of scales. Since it is not necessary to change every 1-15-L, all can be realized with the same configuration.

Each attribute extracting unit 32 also outputs the target spread amount 3 of each scale input from each spread amount extractor 31.
Since it is not necessary to change the content of the process of extracting the target attribute for each of 3-1 to 33-L, all of them can be realized with the same configuration.

[0088]Embodiment 7 FIG.  Next, the present inventionExample 7Will be described with reference to FIG. Figure 1
8 isExample 7Extraction in Image Processing Method by Image Processing
FIG. 19 is a flowchart showing details of the processing, and FIG.
It is explanatory drawing which shows the principle of spread amount extraction. In this FIG.
41 has a spread of R in a certain direction from the position of the center of gravity.
Is a one-dimensional model Ie (r) of the object
An example of a one-dimensional function p (r; s) depending on the amount s,
3 depends on the one-dimensional model Ie (r) and the spread s
Q of the product-sum operation with the calculated one-dimensional function p (r; s)
(S). Here, a one-dimensional model I of these objects
e (r), a one-dimensional function p (r;
s) and the product-sum operation result q (s) are, for example,
Expressions (4) to (6) are given below.

[0089]

(Equation 2)

FIG. 20 is a block diagram showing the internal configuration of the spread amount extraction unit 31 in the image processing apparatus to which the above-described image processing method is applied. In FIG. 20, 44-1 to 44-4
−T is an input preprocessed image radially in a plurality of (T in the illustrated example) directions around the center of gravity 7 of the input target (in the illustrated example, there are M objects). 6
It is a one-dimensional data generation unit for generating one-dimensional data from. A product-sum operation unit 45 executes a product-sum operation of the one-dimensional data output from the one-dimensional data generation unit in each direction and a one-dimensional function that changes depending on the spread amount while changing the spread amount. Numeral 46 denotes a local maximum value extracting unit for extracting the local maximum value of the result of the product-sum operation to extract the spread amount of the object in each direction. Also, 47-1 to 47-T
Is one-dimensional data in each direction generated from the input preprocessed image 6, and 48-1 to 48-T are one-dimensional data 47-1 to 47-T in each direction and change depending on the spread amount. And the result of the product-sum operation with the one-dimensional function
9-T is the extracted spread amount in each direction.

Next, the operation will be described with reference to the flowchart of FIG. In this spread amount extraction unit 31,
When the pre-processed image 6 from the pre-processing unit 2 and the target center of gravity position 7 from the target extraction unit 3 are input, the one-dimensional data generation unit 4
4-1 to 44-T are one-dimensional data 47-1 to 47-T in each direction from the input pre-processed image 6 radially in each direction around the target center of gravity 7 of the target. It is generated (steps ST71-1 to ST71-T). Next, in each product-sum operation unit 45, the one-dimensional data 4 in each direction is obtained.
The product-sum operation of 7-1 to 47-T and a one-dimensional function that changes depending on the spread amount is executed while changing the spread amount (steps ST72-1 to ST72-T).
T). Then, in each local maximum value extraction unit 46, the product-sum operation results 48-1 to 48-T obtained by each product-sum operation unit 45
Are obtained, and the spread amounts 49-1 to 49-T in each direction are extracted by obtaining the spread amounts at which the respective maximum values are obtained (steps ST73-1 to 73-T). However, the product-sum operation results 48-1 to 4-4 of the one-dimensional function that changes depending on the spread amount and the one-dimensional data 47-1 to 47-T in a certain direction.
8-T is set in advance so that it becomes maximum when the spread amount matches the size of the target in that direction.

The processing in the case where the one-dimensional function p (r; s) dependent on the spread amount s, for example, the one shown in the above equation (5) will be described with reference to FIG. In FIG. 19, a one-dimensional model Ie (r) of an object having a spread of R in a certain direction from a center of gravity indicated by 41 and a one-dimensional function p (r; s) dependent on the spread amount s indicated by 42 When the product-sum operation is calculated, the product-sum operation result q (s) is as shown in 43. By extracting the spread amount s at which the product-sum operation result q (s) indicated by 43 is maximized, the target size R can be obtained.

Here, in the description of the processing with reference to FIG. 19, the one using the equation (5) as the one-dimensional function p (r; s) depending on the spread amount s is shown. Also in this case, the size R of the target can be obtained by extracting the spread amount s at which the product-sum operation result q (s) is maximized. In this case, as the one-dimensional function p (r; s) depending on the used spread amount s, for example, the following equations (7) to (9) can be considered.

[0094]

(Equation 3)

[0095]Reference example 2.  Next, the present inventionReference example 2 to help understandingIn the figure
Will be described. FIG.Reference Example 2Image processing method
FIG. 2 is a flowchart showing the contents of pre-processing in FIG.
2 is a preprocessing in the image processing apparatus to which the above image processing method is applied.
FIG. 2 is a block diagram illustrating an internal configuration of a processing unit 2. FIG.
51, the spatial frequency in the input original image 5 is predetermined.
And extract it as the pre-processed image 6.
This is a spatial filter processing unit that outputs.

Next, the operation will be described with reference to the flowchart of FIG. When the original image 5 is input from the image input unit 1 to the pre-processing unit 2, the spatial filter processing unit 5
1 performs a spatial filtering process on the original image 5 (step ST81). As a result, only the portion of the input original image 5 where the spatial frequency is within a predetermined range is emphasized, and the preprocessed image 6 from which high-frequency noise, a slow luminance change, and the like have been removed is obtained. The obtained preprocessed image 6
Is output to the target extraction unit 3 and the target recognition unit 4.

Here, as the spatial filter processing section 51,
By applying an average filter that replaces the luminance value of a certain point of interest with an average value of luminance in a local region centered on the point of interest, or a median filter that replaces the luminance value of the luminance value of the original point, the high-frequency Noise can be removed.

Also, by using a Gaussian filter as the spatial filter processing section 51, high frequency noise in the original image 5 can be removed.

Also, a differential filter is operated as the spatial filter processing unit 51 to perform a spatial filter process of replacing a luminance value of a certain point of interest with a difference value between peripheral pixels centered on the point of interest. Slow luminance changes in the original image 5 can be removed.

Further, the spatial filter processing section 51 applies a smoothing filter such as a Gaussian filter, an average filter, a median filter, and the like, and then applies a differential filter. A large change in luminance can be removed at the same time.

Also, as a spatial filter processing section 51, a differential filter is operated, and then a smoothing filter such as a Gaussian filter, an average filter, a median filter or the like is operated, so that high-frequency noise in the original image 5 can be reduced. Slow luminance changes and the like can be removed at the same time.

[0102]Reference example 3.  next,Reference Example 3Will be described with reference to FIG. FIG.Reference example
3Showing the contents of preprocessing in the image processing method by
FIG. 24 shows an example in which the above image processing method is applied.
Showing the internal configuration of the pre-processing unit 2 in the image processing apparatus.
FIG. In FIG. 24, 52 is an input original image
5 is binarized to a predetermined threshold value, and the binarized
As a binarization processing unit.

Next, the operation will be described with reference to the flowchart of FIG. When the original image 5 is input from the image input unit 1 to the pre-processing unit 2, the binarization processing unit 52 performs binarization processing on the original image 5 using a predetermined threshold (step ST82). . As a result, a preprocessed image 6 in which the target and the background in the input original image 5 are separated is obtained and sent to the target extraction unit 3 and the target recognition unit 4.

Here, the threshold value of the binarization processing is automatically obtained based on, for example, an evaluation function obtained from a luminance histogram of the input original image 5 to stably separate the target and the background. can do.

[0105]Reference example 4.  next,Reference example 4Will be described with reference to FIG. FIG.Reference example
4Showing the contents of preprocessing in the image processing method by
FIG. 26 shows an example in which the above image processing method is applied.
Showing the internal configuration of the pre-processing unit 2 in the image processing apparatus.
FIG. In FIG. 26, reference numeral 51 denotes spatial filter processing
Section, 52 is a binarization processing section, 5 is an original image, 6 is a pre-processed image
These are given the same reference numerals in FIG. 22 or FIG.
It is equivalent to the part which did. 53 is a space fill
The spatial filter sent from the data processor 51 to the binarization processor 52
It is a ruta output image.

Next, the operation will be described. This will be described with reference to the flowchart of FIG. When the original image 5 is input from the image input unit 1 to the preprocessing unit 2, first, the spatial filter processing unit 51 performs a spatial filter process on the original image 5 (step ST83). As a result, only the part of the input original image 5 where the spatial frequency is within a certain range is emphasized, and the spatial filter output image 53 from which high-frequency noise, a slow luminance change, and the like have been removed is obtained. This spatial filter output image 53 is sent to the binarization processing section 52,
The binarization processing unit 52 responds to this by a predetermined threshold value.
A value conversion process is performed (step ST84). As a result, the target and the background in the input original image 5 are separated from each other, and a preprocessed image 6 from which high-frequency noise, a slow change in luminance, and the like are removed is obtained. Output to

Here, a spatial filter processing section 51 applies a smoothing filter such as an average filter, a median filter, and a Gaussian filter, and then performs a binarization processing in a binarization processing section 52. By removing the influence of high-frequency noise in the original image 5, the target can be separated from the background.

After the differential filter is acted on by the spatial filter processing section 51, the binarization processing section 52 performs binarization processing to remove the influence of a slow luminance change in the image. The subject can be separated from the background.

In the spatial filter processing section 51, the differential filter is operated after the smoothing filter is operated.
Alternatively, the smoothing filter is operated after the differential filter is operated, and then the binarization processing is performed by the binarization processing unit 52, thereby simultaneously removing the influence of high-frequency noise and a slow luminance change in the image. The object can be separated from the background.

Here, the threshold value of the binarization processing is automatically obtained based on, for example, an evaluation function obtained from a luminance histogram of the input original image 5 to stably separate the target and the background. can do.

[0111]Embodiment 8 FIG.  Next, the present inventionExample 8Will be described with reference to FIG. FIG.
7 isExample 8Of Center of Gravity in Image Processing Method by Image Processing
FIG. 28 is a flowchart showing the contents of the processing.
It is explanatory drawing which shows the principle of a center-of-gravity position extraction. Figure 28
61 has the evaluation result of the localization of the object as a luminance distribution.
Image 62, the center of gravity of the target extracted from the image 62
It is an image of a location.

FIG. 29 is a block diagram showing the internal configuration of the center-of-gravity extracting unit 10 in the image processing apparatus to which the above-described image processing method is applied. In FIG. 29, reference numeral 63 denotes a local maximum value extracting unit that obtains the local maximum value from the localization evaluation result 11 of the target evaluated by the locality evaluating unit 9 and extracts the center of gravity position 7 of the target.

Next, the operation will be described with reference to the flowchart of FIG. 27 and the explanatory diagram of FIG. The localization evaluation unit 9 outputs the localization evaluation result 1 to the centroid extraction unit 10.
When 1 is input, the local maximum value extracting unit 63 extracts a pixel whose luminance shows a local maximum value from the image 61 having the evaluation result of the locality as a luminance distribution (step ST91). The positions of the pixels indicated by x in the image 62 whose luminance has the maximum value are respectively extracted as the position of the center of gravity 7 of the target and output to the target recognition unit 4.

Here, if the maximum value is simply obtained, a large number of maximum values are extracted. Therefore, a certain threshold value is set, and only when the obtained maximum value exceeds the threshold value, the maximum value is set. It can be set so as to be extracted as the position of the center of gravity of the object.

The threshold value for obtaining the maximum value can be automatically determined from the maximum value and the minimum value of the image 61 having the luminance value and the distribution of the evaluation result of the input locality.

The threshold value for obtaining the local maximum value is determined by the image 61 having the input locality evaluation result as a luminance distribution.
Can be automatically determined based on the evaluation function obtained from the histogram of.

[0117]Embodiment 9 FIG.  Next, the present inventionExample 9Will be described with reference to FIG. FIG.
0 isExample 9Of Center of Gravity in Image Processing Method by Image Processing
FIG. 31 is a flowchart showing the contents of the processing.
It is explanatory drawing which shows the principle of a center-of-gravity position extraction. Figure 31
61 has the evaluation result of the localization of the object as a luminance distribution.
Image 64 shows the evaluation result of the locality as a luminance distribution.
Image obtained as a result of binarizing the image 61 with a predetermined threshold
And 65 is a binarized image 64
Labeled result image, 66 is the result of this labeling
Is calculated from the image 65 of FIG.
Is an image resulting from the extraction.

FIG. 32 is a block diagram showing the internal configuration of the center-of-gravity extracting unit 10 in the image processing apparatus to which the above-described image processing method is applied. In FIG. 32, reference numeral 67 denotes a binarization processing unit for binarizing the evaluation result 11 of the locality of the object evaluated by the localization evaluation unit 9 with a predetermined threshold value, and 68 denotes the binarization processing. In the binarization processing result obtained by the processing section 67, a moment calculation section for calculating the 0th and 1st moments, and 69 is a center of gravity for extracting the target center of gravity position 7 from the result of the moment calculation by the moment calculation section 68. It is a position extraction unit. 70 is a binarization processing result sent from the binarization processing section 67 to the moment calculation section 68;
Reference numeral 71 denotes a moment calculation result sent from the moment calculator 68 to the center-of-gravity position extractor 69.

Next, the operation will be described with reference to the flowchart of FIG. When the localization evaluation result 9 is input from the localization evaluation unit 9 to the center of gravity extraction unit 10,
The binarization processing section 67 binarizes the evaluation result 11 of the locality with a predetermined threshold value (step ST92). Next, the obtained binarization processing result 70 is sent to the moment calculating unit 86, where labeling is performed to calculate the zero-order and first-order moments (step ST93), and the center-of-gravity position extracting unit 69
Extracts the target center of gravity position 7 from the moment calculation result 71 (step ST94) and outputs it to the target recognition unit 4.

Next, the principle of extracting the position of the center of gravity will be described with reference to FIG. The image 61 having the localization evaluation result 11 as a luminance value is binarized by a predetermined threshold value to become a binarized image 64. Here, the threshold value for the binarization can be automatically obtained based on, for example, an evaluation function obtained from a histogram of the luminance of the input image.

The obtained binarized image 64 is labeled, and an image 65 of the labeling result shown in 65 is obtained.
Calculate the zero-order and first-order moments. Here, the zero-order moment m00 and the first-order moment m1
0 and m01 are represented by the following equations (10) to (12), respectively. Here, lx and ly are the horizontal and vertical sizes of the image, respectively, and I (i, j) = 1,
I (i, j) = 0 (background part) in a part other than the target.

[0122]

(Equation 4)

From the calculated zero-order and first-order moments m00, m10, and m01, the position indicated by x in the image 66 of the result of the center-of-gravity position extraction is extracted as the center-of-gravity position 7 of each object. The coordinates of the center of gravity (Gx, Gy
) Is represented by the following equations (13) and (14).

Gx = m10 / m00 (13) Gy = m01 / m00 (14)

[0125]Embodiment 10 FIG.  Next, the present inventionExample 10Will be described with reference to FIG. Figure
33 isExample 10Of object in image processing method by
FIG. 3 is a flowchart showing the contents of the size extraction processing, and FIG.
4 and FIG. 35 are views showing the principle of the size extraction of this object.
FIG. In FIG. 34, reference numeral 81 denotes an image to be recognized.
82 to 84 are images of intermediate results of the processing, respectively.
A plane 85 is a processing result screen. Also, 86 recognizes
The object 87 is the position of the center of gravity of the object 86, and 88 is
Center small area set around the center of gravity position 87 of 8
9 is a peripheral small area set around the central small area 88.
You. In FIG. 35, reference numeral 91 denotes a screen to be recognized.
Where 92 to 94 are screens showing the results of the processing in progress,
Reference numeral 95 denotes a processing result screen.

Next, the operation will be described with reference to the flowchart of FIG. 33 and the explanatory diagram of FIG. When the processing is started, first, the localization of the object 86 to be recognized is evaluated, and the center of gravity 8
7 is extracted (step ST101). Next, a center small area 88 is set around the extracted center of gravity position 87 (step ST102), and a surrounding small area 89 is set around the center small area 88 (step ST10).
3). Next, the average of the luminance values of the pixels in the central small area 88 and the average of the luminance values of the pixels in the peripheral small area 89 are examined to determine whether the peripheral small area 89 is the target or the background. (Step ST104). If the average of the two brightness values is not significantly different, it is determined that the surrounding small area 89 is the target, and is integrated into the central small area 88 (step ST105). Thereafter, the process proceeds to step ST103.
The size of the object 86 is extracted by repeating the above processing until a boundary of the object is found (step S).
T106).

Here, as shown in FIG. 34, a small rectangular area is set as a peripheral small area 89 and is sequentially extended around the central small area 88, so that when extracting a rectangular target 86, the target 86 to be extracted is extracted. In the up, down, left, and right directions around the current central small area 88,
Can be set or changed. For example, if the object 86 is close to a square (up → down → left → right → ...)
It is only necessary to expand equally up, down, left, and right as in ()). Also,
If the target 86 is a horizontally long rectangle (left → right → left → right →
Upper → lower → left → right → left → right → upper → lower → ...)
The expansion in the left-right direction may be set to be larger than the expansion in the vertical direction. Conversely, if the object 86 is a vertically long rectangle (up → down → top → bottom → left → right → top → bottom → top → bottom →
(Left → right →...), The expansion in the vertical direction may be set to be larger than the expansion in the horizontal direction. As described above, the size of the rectangular object 86 can be extracted by sequentially expanding a minute rectangular area as the surrounding small area 89.

Further, as shown in FIG. 35, for example, the peripheral small area 89 is sequentially set in a spiral shape around the central small area 88 so that the size of the object 86 can be extracted.

[0129]Embodiment 11 FIG.  Next, the present inventionExample 11Will be described with reference to FIG. Figure
36 isExample 11Object Recognition in Image Processing Method
FIG. 37 is a flowchart showing the contents of the processing.
FIG. 3 is an explanatory diagram illustrating the principle of object recognition. In FIG.
101 is a screen of the pre-processed image, and 102 is the localization of the target.
This is an image having the evaluation results of the characteristics in a luminance distribution. 103
Image 102 having this localization evaluation result as a luminance distribution
Binarization, which is the image resulting from binarizing the image with a predetermined threshold
An image 104 is a rasterized image 103 of the binarized result.
Image of belling result, 105 is position and size of target
It is an image as a result of extracting attributes such as shape, posture, and the like.

FIG. 38 is a block diagram showing a configuration of an object recognition unit in an image processing apparatus to which the above-described image processing method is applied. In FIG. 38, reference numeral 105 denotes a localization evaluation unit for evaluating the localization of the target from the pre-processed image 6. Reference numeral 106 denotes the localization evaluation result 11 of the target evaluated by the localization evaluation unit 105. This is a binarization processing unit that binarizes at a predetermined threshold value. Reference numeral 107 denotes a moment calculation unit that calculates the 0th, 1st, and 2nd moments in the binarization processing result obtained by the binarization processing unit 106.
Is an attribute extraction unit for extracting the position of the center of gravity, the size, the shape, and the attributes of political parties from the result of the moment calculation by the moment calculation unit 107. Reference numeral 109 denotes a localization evaluation result sent from the localization evaluation unit 105 to the binarization processing unit 106; 110, a binarization processing result sent from the binarization processing unit 106 to the moment calculation unit 107; Is a moment calculation result sent from the moment calculation unit 107 to the attribute extraction unit 108.

Next, the operation will be described with reference to the flowchart in FIG. When the preprocessed image 6 is input to the target recognition unit, the locality evaluation unit 105 evaluates the locality of the input preprocessed image 6 (step ST).
111). The evaluation result 109 of the locality is sent to the binarization processing unit 106, binarized by a predetermined threshold value (step ST112), and sent to the moment calculation unit 107.
In the moment calculation unit 107, the binarization processing result 110
, The 0th, 1st and 2nd moments are calculated (step ST113), and the attribute extraction unit 1
08 extracts attributes such as the position, size, shape, and posture of the center of gravity of the object from the moment calculation result 111 (step ST114).

Next, the principle of such attribute extraction is shown in FIG.
This will be described with reference to FIG. The localization of the target is evaluated on the screen 101 of the preprocessed image, and an image 102 having the evaluation result as luminance and the localization evaluation result as a luminance distribution is obtained. The image 102 having the localization evaluation result as a luminance distribution is binarized by a predetermined threshold value to become a binarized image 64. Here, the threshold value for the binarization can be automatically obtained based on, for example, an evaluation function obtained from a histogram of the luminance of the input image.

Next, the obtained binarized image 64 is labeled, and an image 65 as a labeling result is obtained.
Next, the first and second moments are calculated. Here, the zero-order moment m00 and the first-order moment m
10, m01 and secondary m20, m02
Are represented by the following equations (15) to (20), respectively. Here, lx and ly are the horizontal and vertical sizes of the image, respectively, and I (i, j) = 1 in the target portion, and I (i, j) = 0 (background portion) in portions other than the target.

[0134]

(Equation 5)

Then, the calculated zero-order, first-order, and second-order moments m00, m10, m01, m20, and m11 are calculated.
From m02 and m02, attributes such as the position of the center of gravity, size, shape, and posture of each object are extracted as in the image 105 of the attribute extraction result. The coordinates (Gx, Gy) of the center of gravity are represented by the above-described equations (13) and (14), and the inclination θ of the main axis is represented by the following equation (21). θ = Tan ⁻¹ {2M11 / (M20−M02)} (21) where M20 = m20−m10 ² / m00 M11 = m11−m10 · m01 / m00 M02 = m02−m01 ² / m00

[0136]

[0137]

According to the first aspect of the present invention, the object
After extracting the position of the center of gravity focusing on the locality, the extracted pair
The size, shape, and posture of the target, centered on the elephant's center of gravity
In recognizing such attributes, the localization of the target is evaluated in parallel at different scales using multiple evaluation functions, and the obtained localization evaluation results are integrated to re-evaluate the localization of the target After that, the position of the center of gravity of the target is extracted from the re-evaluation result .
Can reduce scale, reduce processing cost, match
Of an image processing method that can improve the stability of
There is fruit. Further, there is an effect that it is possible to recognize objects of various sizes when extracting the objects.

According to the second aspect of the present invention, the localization of an object is evaluated in parallel at different scales using a plurality of evaluation functions, and the center of gravity of the object is evaluated from the obtained localization evaluation results. position to extract, since a plurality of the gravity center position candidate extracted <br/> by Uni configuration you in parallel, when extracting an object, the effect of making it possible to recognize the subject of various sizes is there.

According to the third aspect of the present invention, the input image is converted into a plurality of different resolutions, and the localization of the object is evaluated for each resolution image using the same evaluation function. Then, after integrating the obtained localization evaluation results and re-evaluating the localization of the target, the position of the center of gravity of the target is extracted from the re-evaluation result. This has the effect that objects of various sizes can be recognized.

According to the fourth aspect of the present invention, the input image is converted into a plurality of different resolutions, and the localization of the object is evaluated for the images of each resolution using the same evaluation function. Since a plurality of barycenter position candidates are extracted in parallel from each evaluation result, there is an effect that it is possible to recognize objects of various sizes when extracting objects.

[0141] According to the invention described in claim 5, pairs
After extracting the position of the center of gravity focusing on the localization of the elephant,
The size, shape,
When recognizing attributes such as posture, set the center of gravity of the target
The input image that extends radially in multiple directions around
One-dimensional data and one-dimensional data that changes depending on the amount of spread
And the product of these
The spread amount at which the sum operation result is maximized is
Extract as the amount of caulking and extract the target attribute based on it
So that when recognizing the target, the search
Space, reducing processing costs and
An image processing method that can improve the stability of
Has an effect. Further, when recognizing the target, the size of the search space can be reduced, and the processing cost can be reduced and the stability of matching can be improved. In addition,
When extracting the amount of spread
There is an effect that can be.

According to the sixth aspect of the present invention, the spread amount of each object is checked in parallel in a plurality of directions around the plurality of candidate positions where the object exists, and based on that, the spread amount of each object is determined. By extracting candidate attributes in parallel and integrating them, the target attributes are configured to be extracted, so that when recognizing the target, the size of the search space can be reduced, reducing processing costs, This has the effect of improving the stability of matching.

According to the seventh aspect of the present invention, one-dimensional data of an input image radially extended in a plurality of directions around the center of gravity of the object and changes depending on the spread amount. A one-dimensional function is generated, and while changing the spread amount, the spread amount at which the product-sum operation result reaches a maximum is extracted as the target spread amount in that direction, so the target spread amount is extracted. This has the effect that the extraction can be performed stably.

[0144]

[0145]

[0146]

According to the eighth aspect of the present invention, the maximum value of the obtained localization evaluation result is extracted as the position of the center of gravity of the target. In determining the position of the center of gravity, there is an effect that the position of the center of gravity can be stably extracted.

Further, according to the ninth aspect of the present invention, an image having a localization evaluation result as a luminance distribution is subjected to binarization processing, and the obtained binary image is used for the 0th and 1st order of the object. Since the position of the center of gravity of the object is extracted by calculating the next moment, there is an effect that the position of the center of gravity can be stably extracted when determining the position of the center of gravity of the object from the evaluation result of the locality.

According to the tenth aspect of the present invention,
The luminance distribution is compared between the center small area set at the center of gravity and the plurality of surrounding small areas set around the center small area to determine whether the surrounding small area is a target or a background. For example, a process for fusing the surrounding small area to the central small area is performed, and the central small area is sequentially expanded to the boundary of the target to extract the size, so when extracting the size of the target, There is an effect that the extraction can be performed stably.

According to the eleventh aspect of the present invention,
The image having localization evaluation results as a luminance distribution is binarized, and the 0th, 1st, and 2nd moments of the object are calculated in the obtained binarized image to obtain the position of the center of gravity of the object, Since the attributes such as the size, shape, and posture are simultaneously extracted, there is an effect that the attribute of the target can be stably extracted when the target is recognized.

[Brief description of the drawings]

FIG. 1 is a flowchart illustrating an image processing method according to a reference example 1 .

FIG. 2 is a flowchart showing a flow of a target extraction process of the reference example .

FIG. 3 is a block diagram illustrating a configuration of an image processing apparatus to which the image processing method according to the reference example is applied.

FIG. 4 is a block diagram illustrating a configuration of a target extraction unit of the image processing apparatus.

FIG. 5 is a flowchart showing a target extraction process in the image processing method according to the first embodiment of the present invention.

FIG. 6 is a block diagram illustrating a configuration of a target extraction unit of the image processing apparatus to which the image processing method according to the embodiment is applied.

FIG. 7 is a flowchart showing a target extraction process in the image processing method according to the second embodiment of the present invention.

FIG. 8 is a block diagram illustrating a configuration of a target extraction unit of an image processing apparatus to which the image processing method according to the embodiment is applied.

FIG. 9 is a flowchart illustrating a target extraction process in the image processing method according to the third embodiment of the present invention.

FIG. 10 is a block diagram illustrating a configuration of a target extraction unit of an image processing apparatus to which the image processing method according to the embodiment is applied.

FIG. 11 is a flowchart illustrating an object extraction process in the image processing method according to the fourth embodiment of the present invention.

FIG. 12 is a block diagram illustrating a configuration of a target extraction unit of an image processing apparatus to which the image processing method according to the embodiment is applied.

FIG. 13 is a flowchart showing an object extraction process in the image processing method according to the fifth embodiment of the present invention.

FIG. 14 is an explanatory diagram showing the principle of object extraction in the embodiment.

FIG. 15 is a block diagram illustrating a configuration of a target extraction unit of the image processing apparatus to which the image processing method according to the embodiment is applied.

FIG. 16 is a flowchart showing an object recognition process in the image processing method according to the sixth embodiment of the present invention.

FIG. 17 is a block diagram illustrating a configuration of a target recognition unit of an image processing apparatus to which the image processing method according to the above embodiment is applied.

FIG. 18 is a flowchart illustrating a spread amount extraction process in the image processing method according to the seventh embodiment of the present invention.

FIG. 19 is an explanatory diagram illustrating a principle of extracting a spread amount in the embodiment.

FIG. 20 is a block diagram illustrating a configuration of a spread amount extraction unit of the image processing apparatus to which the image processing method according to the embodiment is applied.

FIG. 21 is a flowchart showing pre-processing in an image processing method according to Reference Example 2 .

FIG. 22 is a block diagram illustrating a configuration of a preprocessing unit of an image processing apparatus to which the image processing method according to the reference example is applied.

FIG. 23 is a flowchart illustrating preprocessing in an image processing method according to Reference Example 3 ;

FIG. 24 is a block diagram illustrating a configuration of a preprocessing unit of an image processing apparatus to which the image processing method according to the reference example is applied.

FIG. 25 is a flowchart showing pre-processing in the image processing method according to Reference Example 4 .

FIG. 26 is a block diagram illustrating a configuration of a preprocessing unit of an image processing apparatus to which the image processing method according to the reference example is applied.

FIG. 27 is a flowchart illustrating a center-of-gravity position extraction process in the image processing method according to the eighth embodiment of the present invention.

FIG. 28 is an explanatory diagram showing the principle of centroid position extraction in the above embodiment.

FIG. 29 is a block diagram illustrating a configuration of a center-of-gravity extracting unit of the image processing apparatus to which the image processing method according to the above embodiment is applied.

FIG. 30 is a flowchart showing a center-of-gravity point position extracting process in the image processing method according to the ninth embodiment of the present invention.

FIG. 31 is an explanatory diagram showing the principle of center-of-gravity position extraction in the above embodiment.

FIG. 32 is a block diagram illustrating a configuration of a center-of-gravity extracting unit of the image processing apparatus to which the image processing method according to the above embodiment is applied.

FIG. 33 is a flowchart showing target size extraction processing in the image processing method according to Embodiment 10 of the present invention.

FIG. 34 is an explanatory diagram showing an example of the principle of object size extraction in the embodiment.

FIG. 35 is an explanatory diagram showing an example of the principle of object size extraction in the embodiment.

FIG. 36 is a flowchart showing an object recognition process in the image processing method according to the eleventh embodiment of the present invention.

FIG. 37 is an explanatory diagram showing the principle of object recognition in the embodiment.

FIG. 38 is a block diagram illustrating a configuration of a target recognition unit of an image processing apparatus to which the image processing method according to the above embodiment is applied.

FIG. 39 is an explanatory diagram showing the principle of conventional pattern matching.

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-4-282782 (JP, A) JP-A-1-171072 (JP, A) JP-A-2-90374 (JP, A) JP-A-5-1990 159057 (JP, A) JP-A-52-98444 (JP, A) JP-A-1-116853 (JP, U) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T 7/00-7 / 60 G06T 1/00 G01B 11/00-11/30

Claims (11)

(57) [Claims] When recognizing a target existing in an image captured by an imaging unit, the target image is included in a pre-processed image obtained by performing pre-processing on an original image obtained by digitizing the captured image. the localization is the degree of local concentration of elements constituting the different using a plurality of evaluation functions schedule
And evaluate the results of these multiple localizations in parallel.
By re-evaluating the localization of the subject by integrating,
Extract the center of gravity of the object from the re-evaluation result of its locality
And extracting attributes of the object, such as the size, type, and posture of the object, with the center of gravity of the object determined as a center. 2. A method according to claim 1, wherein when recognizing a target existing in the image picked up by the image pickup means, the target image is included in a preprocessed image obtained by performing preprocessing on an original image obtained by digitizing the captured image. the localization is the degree of local concentration of elements constituting the different using a plurality of evaluation functions schedule
And evaluate the localization of each
Finding the center of gravity of the target, and listing a plurality of candidates for the center of gravity
An image processing method characterized by extracting attributes of a target, such as a size, a type, and a posture of the target, with the center of gravity of the target being extracted as a center. 3. An image obtained by performing pre-processing on an original image obtained by digitizing the captured image when recognizing a target present in the image captured by the imaging unit .
Converts pre-processed images into multiple resolution converted images with different resolutions
Into multiple resolution-converted images with different resolutions.
The degree of local concentration of the constituent elements of the object
A certain locality is evaluated in parallel using the same evaluation function.
By integrating multiple localization evaluation results
Re-evaluation of elephant localization
An image processing method comprising: extracting a position of the center of gravity of the object from the object; and extracting attributes of the object such as a size, a type, and a posture of the object with the obtained center of gravity of the object as a center. 4. An image obtained by performing a pre-process on an original image obtained by digitizing the captured image when recognizing a target present in the image captured by the imaging unit .
Converts pre-processed images into multiple resolution converted images with different resolutions
In other words, the object in the plurality of resolution-converted images is
The localization, which is the degree of local concentration of the constituent elements , is the same.
Are evaluated in parallel using the same evaluation function,
Finding the center of gravity of the object from the evaluation result
An image processing method, comprising extracting candidate placements in parallel and extracting target attributes such as the size, type, and posture of the target with the center of gravity of the target determined as a center. 5. When recognizing a target existing in an image captured by an imaging unit, the target image is included in a preprocessed image obtained by performing preprocessing on an original image obtained by digitizing the captured image. The localization, which is the degree of local concentration of the constituent elements, is evaluated, and the position of the center of gravity of the target is extracted by focusing on the obtained localization evaluation result .
Multiple directions around the elephant's location
In obtaining the spread amount of the object radially,
Radial in multiple directions around the center of gravity of the target
To the one-dimensional data of the input image
Generate a one-dimensional function that changes depending on
When the spread amount matches the size of the object in that direction
A sum of product of the one-dimensional function and the one-dimensional data
Set so that the result of the calculation is maximal, and
The one-dimensional function and the one-dimensional data while changing the amount
Is performed, and the result of the product-sum operation is maximized.
Is obtained as the amount of spread in each direction.
And extracting attributes of the object such as the size, type, and posture of the object from the spread amount in each direction . 6. When recognizing a target present in an image picked up by an image pickup means, the target image is included in a preprocessed image obtained by preprocessing an original image obtained by digitizing the captured image. The localization is evaluated as the degree of local concentration of the constituent elements, and the position of the center of gravity of the target is extracted by focusing on the obtained evaluation result of the locality. In extracting the attributes of the object, such as the size, type, and posture of the object, with the center of gravity as the center ,
The location where the target exists is parallel at multiple different scales
If they are extracted, the multiple candidate positions
Spread each object radially in parallel in multiple directions around
Investigate the amount and calculate the spread in each of the multiple directions.
The attributes of the target candidates are extracted in parallel,
Extracting the target attribute by integrating the attributes,
An image processing method characterized by finally recognizing the object . 7. The method according to claim 1, wherein when the spread amount of the object is extracted, one-dimensional data of the input image radially extended in a plurality of directions around the center of gravity of the object and changes depending on the spread amount. A one-dimensional function is generated, and in advance, when the spread amount matches the size of the target in that direction, the result of the product-sum operation of the one-dimensional function and the one-dimensional data is maximized. And performing a product-sum operation of the one-dimensional function and the one-dimensional data while changing the spread amount, and setting the spread amount at which the result of the product-sum operation is maximized to the spread amount in each direction. the image processing method according to claim 6 Symbol mounting, characterized in that the amount. 8. When recognizing a target present in an image captured by an image capturing means, the target image is included in a preprocessed image obtained by performing preprocessing on an original image obtained by digitizing the captured image. The localization is evaluated as the degree of local concentration of the constituent elements, and the maximum value of the obtained localization evaluation result is extracted as the position of the center of gravity of the target. An image processing method characterized by extracting attributes of a target such as a size, a type, and a posture of the target with a center of gravity position as a center. 9. When recognizing a target existing in an image captured by an imaging unit, the target image is included in a preprocessed image obtained by performing preprocessing on an original image obtained by digitizing the captured image. Is evaluated, which is the degree of local concentration of the elements constituting the image, and an image having the obtained evaluation result of the locality as a luminance distribution is determined by a predetermined threshold value by 2
The binarization process is performed, and in the obtained binarized image, the 0th order and the
And the first moment of the object
An image processing method, comprising extracting a position of a center of gravity, and extracting attributes of the object such as a size, a type, and a posture of the object with the obtained center of gravity of the object as a center. 10. When recognizing a target existing in an image captured by an imaging unit, the target image is included in a preprocessed image obtained by performing preprocessing on an original image obtained by digitizing the captured image. The localization is evaluated, which is the degree of local concentration of the constituent elements, and the position of the center of gravity is extracted by focusing on the obtained localization evaluation result.
The brightness distribution in the central small area set around the center position
Luminance distribution in a peripheral small area set around the central small area
And comparing them, the surrounding small area is
To determine whether the object is in the background.
If it is determined that the peripheral small area is the central small area
Is performed, and the central small area is
The size of the object is extracted by sequentially expanding to the boundary.
An image processing method characterized by output. 11. When recognizing a target existing in an image captured by an imaging unit, the target image is included in a preprocessed image obtained by performing preprocessing on an original image obtained by digitizing the captured image. The localization, which is the degree of local concentration of the elements that make up, is evaluated, and the obtained localization evaluation results
Image having a luminance distribution as a luminance distribution is binarized by a predetermined threshold value.
In the obtained binary image, the 0th, 1st, and
And calculating the second moment, the object
Attributes such as the position, size, shape, and posture of the center of gravity
An image processing method characterized by output.