JP3262181B2 - Motion detection device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion detecting device used in various fields requiring motion area detection and motion vector detection.

[Summary of the Invention] The present invention relates to an apparatus for detecting a motion area or a motion vector between two consecutive images, and sets each element between two images as "0" to "1". By combining these values to obtain the similarity between two images, a motion area is detected and a motion vector is detected based on the size. Detection of a region and detection of a motion vector are performed.

[0003]

2. Description of the Related Art Conventionally, as a motion detection method used in various fields requiring motion area detection and motion vector detection, conventionally, a method of mainly examining a correlation of a luminance signal level between two images is often used.

At this time, a method for improving the accuracy of motion detection by examining the correlation between the color signal levels has been proposed, but it is not clear how to realize this method.

On the other hand, as a similar technique, there is an automatic adjustment method of registration of a television camera. In this automatic adjustment method, a registration shift is detected by using a correlation between respective color signals of three primary color signals (RGB signals). These deviations are automatically adjusted.

Further, a method of performing motion detection by applying fuzzy logic is also conceivable, but no concrete proposal for realizing this has yet been made.

[0007]

By the way, in each of the above-described motion detection methods, when a motion area is detected or a motion vector is detected from a television signal, at the present stage, the detection is performed with 100% accuracy. Because it is impossible
Although various measures have been taken according to each application, the basic method is a method of detecting the correlation between the luminance signals of two images or simply detecting the level difference between the luminance signals.

When only the luminance signal is used,
Naturally, when the two images have the same luminance level but different colors, a large detection error occurs in the detection of the motion region and the detection of the motion vector.

In many motion vector detection, when detecting a motion vector between two images, the two images are divided into a plurality of blocks, and a correlation is obtained for each block to detect the motion vector. Often do.

However, in such a method, not only the correlation between the blocks but also the image in the block is expanded into an orthogonal function and the respective coefficients are compared with each other. Sometimes it can be done.

Further, in each of the above-described motion detection methods, it is conceivable to minimize errors in motion detection by using a very large number of elements including a luminance signal and a color signal for detection.

In such a case, how to reflect each element as a whole, that is, how to combine each element is an important issue.

In view of the above circumstances, the present invention makes it possible to appropriately combine a large number of elements including a luminance signal and a chrominance signal at the time of motion detection, and to accurately reflect these elements in motion detection. Accordingly, it is an object of the present invention to provide a motion detection device capable of greatly improving the accuracy of motion region detection and motion vector detection.

[0014]

According to the present invention, there is provided an edge detection circuit for detecting an edge of an input image and generating an edge detection signal indicating the edge intensity. An edge detection unit comprising: a first conversion unit that converts a value of the generated edge detection signal into an edge intensity signal in a range of “0” to “1”; and two continuous images including the image. Difference calculating means for calculating a difference between the luminance signal and the color difference signal of each image to generate a luminance difference signal and a color difference signal, respectively. Separately, using a function that reduces the similarity between two pixels as the absolute value of the inter-frame difference value increases and decreases the similarity as the two pixel values decrease, the range of “0” to “1” is used. Luminance similarity signal and A second conversion unit for converting the luminance similarity signal and the color difference similarity signal to generate a total similarity signal, and a reciprocal thereof to generate a correlation signal; Means, a motion element detecting unit comprising: a reciprocal of the overall similarity signal output from the synthesizing unit to generate a dissimilarity signal; and an edge intensity generated by the edge detecting unit. A multiplier that multiplies the signal by a dissimilarity signal generated by the motion element detection unit to generate a motion area detection signal of the image. According to the second aspect, a continuous image to be compared is captured, and a difference between the captured brightness signal and color difference signal of the current image and the brightness signal and color difference signal of the previous image is calculated to obtain the brightness difference signal and the color difference signal. And first conversion means for converting the calculated luminance difference signal and color difference signal into luminance similarity signal and color difference similarity signal in the range of “0” to “1”, respectively, A block generation unit that fetches a vector value indicating the position of the shift amount and cuts out a luminance signal and a chrominance signal in block units from a previous luminance signal and a chrominance signal based on the fetched vector value; In addition to calculating the orthogonal expansion coefficients of the generated block-based luminance signal and color difference signal, the block luminance signal and the block luminance signal of the current image are calculated. Orthogonal expansion coefficient calculating means for calculating an orthogonal expansion coefficient of the block color difference signal, and difference means for calculating a difference between each orthogonal expansion coefficient of each block in the previous image and each orthogonal expansion coefficient of the current image. The difference signal of each coefficient is set to “0” to
A second conversion unit for converting into a luminance similarity signal and a color difference similarity signal in a range of “1”, and a similarity indicating a total similarity by combining respective output signals of the first and second conversion units. A motion element detecting unit having a first synthesizing unit for generating a degree signal; and a synthesizing unit for synthesizing a luminance similarity signal and a color difference similarity signal constituting a similarity signal output from the first synthesizing unit. And maximum value detection means for detecting a similarity signal indicating the maximum value among the similarity signals for each of the synthesized vector values, and a motion vector based on the detection result and the vector value from the synthesis unit. And a motion vector determining unit having a shift amount detecting means for detecting.

[0015]

In the above arrangement, elements representing features between two consecutive television images are extracted between two consecutive television images, and each of the elements is extracted using a function preset for each of these elements. Converting the values of the elements into numerical values within a predetermined range, and further combining these numerical values to generate numerical values within a predetermined range representing the similarity between two images, and obtaining the relationship between the images using the numerical values. Therefore, a large number of elements including a luminance signal and a chrominance signal can be appropriately combined at the time of motion detection, whereby each of these elements can be accurately reflected in motion detection to detect a motion area and a motion vector. Significantly improves detection accuracy.

[0016]

FIG. 1 is a block diagram showing an example of a motion area detecting circuit to which the first embodiment of the present invention is applied. In this figure, only circuits related to the present invention are disclosed by omitting circuits provided in actual circuits, for example, a filter circuit for an input signal and a processing circuit for performing various processes on a motion detection signal. are doing. This embodiment corresponds to claim 1. Difference calculation means and subtractors 13, 1
7, 21, the first conversion means and the conversion circuit 5, the second conversion means and the conversion circuits 15, 19, and 23, the synthesis means and the multiplier 10, and the third conversion means and the conversion circuit 11, respectively.

The motion area detection circuit shown in FIG. 1 includes an edge detection unit 1, a motion element detection unit 2, and a multiplier 3, and captures images to be compared in units of frames, About the luminance signal Y and the color difference signal R−
A similarity is detected for each of Y and BY, and a motion area detection signal having a value indicating whether or not the image is a motion area based on each similarity and the edge information (edge strength signal) of the image. And output this.

The edge detector 1 detects an edge of an image every time an image is input in frame units, and generates an edge detection signal in the range of "0" to "255" based on the detection result. A detection circuit 4 and a conversion circuit for converting the edge detection signal output from the edge detection circuit 4 into a signal (edge strength signal) in the range of “0” to “1” based on a preset conversion characteristic curve 5, and generates a signal (edge strength signal) indicating the edge strength of the image every time an image is input in frame units, and supplies the signal to the multiplier 3.

In this case, the conversion circuit 5 is, for example, as shown in FIG.
“0” to “25” based on the conversion characteristic curve shown in FIG.
The edge detection signal in the range of “5” is converted into an edge strength signal in the range of “0” to “1” and supplied to the multiplier 3.

The motion element detecting section 2 includes a luminance detecting section 6
And two color difference detectors 7 and 8 and two multipliers 9 and 10
And a conversion circuit 11, and each time an image is input in a frame unit, the similarity is detected for each of the luminance signal Y, the color difference signals RY, and BY for each of these images, and these are detected. After calculating the overall similarity signal indicating each similarity, the reciprocal thereof is calculated to generate a dissimilarity signal.
To supply.

The luminance detecting section 6 includes a frame delay circuit 12 for delaying the luminance signal of the image of the current frame by one frame, a luminance signal of the image of the current frame, and a previous image output from the frame delay circuit 12. Subtractor 13 for obtaining a difference from the luminance signal and extracting a luminance difference signal
And an absolute value circuit 14 that calculates the absolute value of the luminance difference signal output from the subtraction circuit 13 to generate a luminance difference detection signal in the range of “0” to “255”. A conversion circuit 15 for converting the detected luminance difference detection signal into a signal (luminance similarity signal) in the range of “0” to “1” based on a predetermined conversion characteristic curve;
Every time a luminance signal Y of an image is input in frame units, a signal (luminance similarity signal) indicating the similarity between the luminance signal Y and the luminance signal Y of the previous frame is generated, and this signal is multiplied by a multiplier 10. To supply.

In this case, for example, the conversion circuit 15 outputs "0" to "25" based on a conversion characteristic curve shown in FIG.
The luminance difference detection signal in the range of “5” is converted into a luminance similarity signal in the range of “0” to “1” and supplied to the multiplier 10.

The one color difference detecting section 7 has a frame delay circuit 16 for delaying the color difference signal RY of the image of the current frame by one frame, a color difference signal RY of the image of the current frame and the frame. A subtracter 17 for obtaining a difference from the color difference signal RY of the previous image output from the delay circuit 16 and extracting a color difference signal;
Calculates the absolute value of the color difference signal output from
An absolute value circuit 18 for generating a color difference detection signal in the range of “255”, and a color difference detection signal output from the absolute value circuit 18 are converted from “0” to “1” based on a preset conversion characteristic curve. And a conversion circuit 19 for converting the color difference signal RY into a signal (color difference similarity signal) in the range of ".
A signal (color difference similarity signal) indicating the similarity between Y and the color difference signal RY of the previous frame is generated and supplied to the multiplier 9.

In this case, the conversion circuit 19 outputs "0" to "25" based on, for example, a conversion characteristic curve shown in FIG.
The color difference difference detection signal in the range of “5” is converted into a color difference similarity signal in the range of “0” to “1” and supplied to the multiplier 9.

The other color difference detection section 8 is a frame delay circuit 20 for delaying the color difference signal BY of the current frame image by one frame, a color difference signal BY of the current frame image and the frame A subtracter 21 for obtaining a difference from the color difference signal BY of the previous image output from the delay circuit 20 and extracting a color difference signal;
Calculates the absolute value of the color difference signal output from
An absolute value circuit 22 for generating a color difference detection signal in the range of “255”, and a color difference detection signal output from the absolute value circuit 22 are converted from “0” to “0” based on a preset conversion characteristic curve. And a conversion circuit 23 for converting the image into a signal (color difference similarity signal) in the range of 1 ″.
A signal (color difference similarity signal) indicating the similarity between −Y and the color difference signal BY of the previous frame is generated and supplied to the multiplier 9.

In this case, the conversion circuit 23 converts the color difference detection signal in the range of "0" to "255" to "0" based on the conversion characteristic curve shown in FIG. A color difference similarity signal in the range of “1” to “1” is supplied to the multiplier 9.

The multiplier 9 multiplies the color difference similarity signals output from the color difference detectors 7 and 8 and supplies a signal (color difference similarity signal) obtained by the multiplication process to the multiplier 10.

The multiplier 10 multiplies the luminance similarity signal output from the luminance detector 6 by the chrominance similarity signal output from the multiplier 9 and obtains a signal obtained by this multiplication (total similarity). Is supplied to the conversion circuit 11.

The conversion circuit 11 calculates the reciprocal of the overall similarity signal output from the multiplier 10 to generate a difference signal, and supplies the difference signal to the multiplier 3.

The multiplier 3 multiplies the difference signal output from the conversion circuit 11 of the motion element detection unit 2 by the edge intensity signal output from the conversion circuit 5 of the edge detection unit 1 to obtain a luminance. Signal Y and color difference signals RY, BY
A motion area detection signal indicating the degree of difference between each frame image viewed from the image is generated and output.

As described above, in this embodiment, not only the luminance signal Y of the image but also the color difference signals RY and BY of the image are provided.
Is also taken into account, a similarity between images is detected, and these similarities are combined to generate a motion region detection signal indicating the difference between the images. An extremely large number of elements, including color signals, can be appropriately combined, and these elements can be accurately reflected in the detection of the motion area, thereby greatly improving the accuracy of the detection of the motion area.

FIG. 3 is a block diagram showing an example of a motion vector detecting circuit to which the second embodiment of the present invention is applied. This embodiment corresponds to claim 2. Difference operation means and subtractor 41, first conversion means and conversion circuit 42, block generation means and block generation section 35, orthogonal expansion coefficient operation means and orthogonal expansion coefficient calculation circuits 52 and 53, difference means and subtractor 3
8, the second conversion means and the conversion circuit 39, the first synthesis means and the multiplier 43, the maximum value detection means and the maximum value detection circuit 46, and the shift amount detection means and the shift amount detection circuit 47, respectively.

The motion vector detecting circuit shown in this figure has three motion element detecting units 30, 31, 32, a synthesizing unit 33,
A motion vector determination unit 34 is provided. Each of the motion element detection units 30 to 32 compares the luminance and each color difference of the current image with the luminance and each color difference of the previous image on a block basis. Are synthesized and the motion vector of the block image to be compared is comprehensively determined, and this determination result (motion vector signal) is output.

As shown in FIG. 4, the first motion element detecting section 30 includes a block generating section 35, an orthogonal correlation detecting section 36, and an inter-frame correlation detecting section 37. Each time the luminance signal Y is input, the luminance signal Y is stored in a frame unit, and a luminance image in block units (previous shift block luminance image) from a luminance image stored based on a vector value indicating the position of the shift amount. And calculates the inter-field correlation between the shifted block luminance image and the block luminance image obtained based on the current image, and the coefficient correlation after orthogonal expansion, and combines these calculation results to generate a luminance similarity detection signal. Is generated and supplied to the synthesizing unit 33.

Each time the luminance signal Y of the image to be subjected to the motion detection is input, the block generation unit 35 stores the luminance signal Y in a frame unit based on the frame memory 50 and the vector value indicating the position of the shift amount. A shift circuit 51 for cutting out a luminance image in block units (previous shift block luminance image) from the luminance image stored in the frame memory 50, and receives a luminance signal Y of an image to be subjected to motion detection. For each frame, this is stored in frame units, and a luminance image in the block unit (previous shift block luminance image) is cut out from the stored luminance image based on the vector value indicating the position of the block, and this is subjected to orthogonal correlation detection. This is supplied to the section 36 and the inter-frame correlation detection section 37.

The orthogonal correlation detecting section 36 fetches the current block luminance image and calculates an orthogonal expansion coefficient of the current block luminance image, and fetches the previous shift block luminance image output from the block generating section 35. And an orthogonal expansion coefficient calculation circuit 53 for calculating the orthogonal expansion coefficient
And a plurality of subtractions for calculating, for each order, the difference between the coefficient for each order output from the orthogonal expansion coefficient calculation circuit 53 and the coefficient for each order output from the orthogonal expansion coefficient calculation circuit 52. And a coefficient difference signal of each order output from each of these subtractors 38, and
A plurality of conversion circuits 39 for converting into luminance similarity signals in the range of “0” to “1”, and a luminance similarity signal output from each of these conversion circuits 39 is multiplied to synthesize one luminance similarity signal. A plurality of multipliers 40 for obtaining the previous shift block luminance image output from the block generation unit 35, calculating the orthogonal expansion coefficient thereof, and obtaining the block luminance image obtained based on the current image. And computes the orthogonal expansion coefficients. The difference between these orthogonal expansion coefficients for each order (coefficient correlation after orthogonal expansion)
Is calculated and converted into a luminance similarity signal in the range of “0” to “1”, and then the luminance similarity signals are combined to generate one luminance similarity signal, which is used to detect inter-frame correlation. To the unit 37.

An inter-frame correlation detecting section 37 calculates a difference between the current block luminance image and the shifted block luminance image output from the block generating section 35 to generate a block luminance difference image, A conversion circuit 42 for converting the block luminance difference image output from the converter 41 into a luminance similarity signal in the range of “0” to “1”, and a luminance similarity signal output from the conversion circuit 42 and the orthogonal correlation detection. A multiplier 43 that multiplies the luminance similarity signal output from the unit 36 to generate a luminance similarity signal indicating overall similarity
And calculates the difference (inter-frame correlation) between the previous shift block luminance image output from the block generation unit 35 and the block luminance image obtained based on the current image, and calculates this as “0”. After converting the luminance similarity signal into a luminance similarity signal in the range of "1" to "1", the luminance similarity signal is multiplied by the luminance similarity signal output from the orthogonal correlation detection unit 36 to obtain an overall similarity. A luminance similarity signal shown in FIG.

The second motion element detecting section 31 is constructed similarly to the first motion element detecting section 30. Each time the color difference signal RY of the image to be motion detected is input, , RY stored in frame units and stored based on a vector value indicating the position of the shift amount.
An RY image in block units (previous shift block RY image) is cut out from the image and this shift block RY image is extracted.
The inter-frame correlation and the coefficient correlation after orthogonal expansion between the Y image and the block RY image obtained based on the current color difference signal RY are calculated, and the calculation results are combined to obtain a color difference similarity detection signal. Is generated and supplied to the synthesizing unit 33.

Further, the third motion element detecting section 32 includes the first motion element detecting section 30 and the second motion element detecting section 3.
Each time the chrominance signal BY of the image to be subjected to motion detection is input, the chrominance signal BY is stored in frame units and stored based on a vector value indicating the position of the shift amount. A block-by-block BY image (previous shift block BY image) is cut out from the current BY image and a block obtained based on the shift block BY image and the current color difference signal BY The inter-frame correlation with the BY image and the coefficient correlation after orthogonal expansion are calculated, and these calculation results are combined to generate a color difference similarity detection signal.
This is supplied to the synthesis unit 33.

The synthesizing unit 33 includes the second motion element detecting unit 3
A multiplier 44 for multiplying the color difference similarity signal output from the first unit 1 with the color difference similarity signal output from the third motion element detection unit 32; and a color difference similarity signal output from the multiplier 44. A multiplier 45 for multiplying the luminance similarity signal output from the first motion element detection unit 30 with the luminance similarity signal output from each of the motion element detection units 30 to 32; The degree signal is multiplied, and the similarity signal obtained by the multiplication processing is supplied to the motion vector determination unit 34.

The motion vector determination section 34 is
Maximum value detection circuit 46 for detecting the similarity signal indicating the maximum value among the similarity signals for each vector output from
And the relationship (shift amount) between the previous shift block image and the current block image is detected based on the detection result of the maximum value detection circuit 46 and the vector value indicating the position (shift amount) of the block. And a shift amount detection circuit 47 for generating the relationship between the previous image and the current image based on the value of the similarity signal output from the synthesizing unit 33 and the vector value indicating the position of the block. (Shift amount) is detected to generate a motion vector signal, which is output.

As described above, in this embodiment, the luminance and each color difference of the current image and the luminance and each color difference of the previous image are compared with the luminance and each color difference of the previous image on a block-by-block basis. Since the comparison result is synthesized and the motion vector of the image to be compared is comprehensively determined and the determination result (motion vector signal) is output, the luminance signal and the color signal are included at the time of motion detection. An extremely large number of elements can be appropriately combined, and these elements can be accurately reflected in motion detection, thereby greatly improving the accuracy of motion area detection and motion vector detection.

In this embodiment, since the orthogonal expansion coefficients are compared for each order by each of the motion element detectors 30 to 32, the importance of a portion having a flat luminance level or color difference level is determined. The optimum similarity signal can be obtained by changing the characteristics of the respective conversion circuits that process the respective orthogonal expansion coefficients, when is relatively small as compared with the other parts or according to the application of the system.

In the above-described first embodiment, the conversion characteristic curves of the conversion circuits 15, 19, and 23 are shown in FIG.
Using the conversion characteristic curves shown in FIG.
The chrominance difference detection signal in the range of "5" is converted into the chrominance similarity signal in the range of "0" to "1". However, the conversion characteristic curves shown in FIG. May be converted into a color difference similarity signal ranging from “0” to “1”.

In this manner, the similarity between the color difference signals RY and BY can be reflected on the similarity between the luminance signals Y only when the difference between the color difference signal levels is large.

In the first and second embodiments, the multipliers 3, 9, 10, 40, 43,
Although the multiplication using 44 and 45 is used,
For example, a synthesis method that selects a larger one or a smaller one may be used in accordance with the purpose or use.

In the above-described second embodiment, each of the motion element detection units 30 to 32 performs orthogonal expansion on an image of each block. Alternatively, another calculation, for example, a differentiation process, may be performed instead of the orthogonal expansion to detect the similarity between the images.

[0048]

As described above, according to the present invention, a large number of elements including a luminance signal and a chrominance signal are appropriately combined at the time of motion detection, and these elements are accurately reflected in motion detection. As a result, the accuracy of motion area detection and motion vector detection can be greatly improved.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an example of a motion region detection circuit to which a first embodiment of the present invention has been applied.

FIG. 2 is a table showing a characteristic example of each conversion circuit shown in FIG. 1;

FIG. 3 is a block diagram illustrating an example of a motion vector detection circuit to which a second embodiment of the present invention has been applied.

FIG. 4 is a block diagram illustrating a detailed circuit example of a motion detection unit illustrated in FIG. 3;

FIG. 5 is a table showing another characteristic example of each conversion circuit used in the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Edge detection part 2 Motion element detection part 3, 9, 10 Multiplier 4 Edge detection circuit 5 Conversion circuit 6 Luminance detection part 7, 8 Color difference detection part 11 Conversion circuit 12, 16, 20 Frame delay circuit 13, 17, 21 Subtraction Units 14, 18, 22 Absolute value circuit 15, 19, 23 Conversion circuit

Continuation of front page (58) Field surveyed (Int.Cl. ⁷ , DB name) H04N 11/00-11/22

Claims (2)

(57) [Claims] 1. An edge detection circuit for detecting an edge of an input image and generating an edge detection signal indicating the edge strength, and setting the value of the generated edge detection signal to "0" to
An edge detection unit including a first conversion unit that converts the image into an edge intensity signal in a range of “1”; capturing two continuous images including the image, and obtaining a luminance signal and a color difference signal between the images. Difference calculating means for calculating a difference to generate a luminance difference signal and a chrominance difference signal, respectively; and for the calculated luminance difference signal and the chrominance difference signal, the larger the absolute value of the inter-frame difference value is,
Using a function that lowers the similarity between two pixels and makes the similarity smaller as the two pixel values are smaller, “0” to
A second conversion unit for converting into a luminance similarity signal and a color difference similarity signal in a range of “1”, and combining the converted luminance similarity signal and the color difference similarity signal to generate an overall similarity signal;
A motion element detecting unit including a synthesizing means for calculating a reciprocal thereof to generate a correlation signal, and a third converting means for calculating a reciprocal of the overall similarity signal output from the synthesizing means to generate a dissimilarity signal A multiplier that multiplies the edge intensity signal generated by the edge detection unit and the dissimilarity signal generated by the motion element detection unit to generate a motion region detection signal of the image. A motion detection device characterized by the above-mentioned. 2. A continuous image to be compared is taken in, and a difference between a taken luminance signal and a color difference signal of the current image and a taken luminance signal and a color difference signal of the previous image is calculated. And first conversion means for converting the calculated luminance difference signal and color difference signal into luminance similarity signal and color difference similarity signal in the range of “0” to “1”, respectively, A block generation unit that captures a vector value indicating the position of the shift amount, and cuts out a luminance signal and a color difference signal in block units from the previous luminance signal and the color difference signal based on the captured vector value, respectively. In addition to calculating the orthogonal expansion coefficients of the generated block-based luminance signal and color difference signal, the block luminance signal and the block luminance signal of the current image are calculated. Orthogonal expansion coefficient calculating means for calculating the orthogonal expansion coefficient of the block color difference signal, and difference means for calculating the difference between each orthogonal expansion coefficient for each block of the previous image and each orthogonal expansion coefficient for the current image. Second conversion means for converting the difference signal of each coefficient into a luminance similarity signal and a color difference similarity signal in the range of “0” to “1”, and output signals of the first conversion means and the second conversion means. First to generate a similarity signal indicating the overall similarity by combining
A motion element detecting unit comprising: a synthesizing unit; a synthesizing unit for synthesizing a luminance similarity signal and a chrominance similarity signal constituting a similarity signal output from the first synthesizing unit; and the synthesized vector. Among the similarity signals for each value, a maximum value detecting means for detecting a similarity signal indicating a maximum value, and a shift amount detecting means for detecting a motion vector based on the detection result and a vector value from the synthesizing unit. A motion vector determination unit comprising: a motion vector determination unit comprising: