JPH07101456B2 - Animal body extraction method and device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an animal for extracting a moving object or recognizing the area by using a frame which constitutes an image signal obtained from an area including a moving object (animal body). The present invention relates to a body extraction method and its device.

[0002]

2. Description of the Related Art As a method for extracting the motion of an object included in an image signal from the image signal, (1) a frame-to-frame change in brightness is calculated for each pixel forming the image signal to obtain a speed. Method.

(2) A method of calculating the amount of movement of the moving object by obtaining the correlation of brightness between frames in the periphery including the moving object region. (3) A method of associating feature points such as contours and edges between frames and calculating the amount of movement thereof. (4) A method of calculating the speed by performing subtraction between frames, setting a region other than 0 as a motion region, and obtaining the time shift of the motion region.

(5) A method of extracting a moving object by passing a real image signal through a temporal filter and a spatial filter in cascade. Etc. are known. As a technique disclosed in the above-mentioned spatiotemporal filtering, for example, p. Jay. Bart, et al. Written "Mechanisms for Isolation Component Pattern in the Sequential Analysis of Multiple Motion" Proceedings Eye EE Workshop on Visual Motion. PP.
187-193, 1991 (P.J. Burt, Raj.
esh Hingorani and Raymond
J. Kolezynski "Mechanisms
for Isolating Component P
atterns in the Sequential
Analysis of Multiple Mot
ion ”Proc. IEEE Workshop on
Visual Motion "PP. 187-19
3, 1991).

[0005]

The above-mentioned conventional methods have the following problems. In (1), since calculation is performed for all pixels, the amount of calculation is large and the scale is large to realize real-time processing. In addition, it is necessary to divide the moving area based on the speed of each pixel, which complicates the processing.

In (2), the correlation processing requires a large amount of calculation and is large in scale for realizing the real-time processing. Further, it is difficult to determine the moving object region to be subjected to the correlation processing. In (3), the features are often different for each target, and it is difficult to extract the features without losing generality (it is necessary to perform the feature extraction according to the target). In (4), when the image input device itself such as a camera may move, the background of the input image differs between frames, and the entire area of the image becomes nonzero. Further, the subtraction result becomes non-zero due to the change of the light ray (light quantity).
Therefore, in order to make the subtraction result 0, it is necessary to devise the background processing.

Further, in (5), since the real number signal is passed through the temporal filter and the spatial filter in cascade, the object moving in the opposite direction cannot be distinguished. Therefore, the background may be extracted depending on the movement of the image input device itself. Further, in the above-described conventional technology, it is difficult to extract only an object moving in a certain direction in real time without losing generality of the object (which can be performed by the same processing method for any object). It was

An object of the present invention is to solve the above-mentioned problems of the prior art and to extract a moving body or to recognize a region of the moving body in real time by using a relatively small-scale circuit means. An object is to provide a moving body extraction method and an apparatus thereof.

[0009]

In order to achieve the above object, the present invention applies a complex number filtering using a spatio-temporal filter that extracts only a certain speed to an image signal obtained from an image input device, and makes it desirable. It is characterized by extracting a signal of a moving object. That is, the present invention provides a complex number composed of a time direction Hilbert transform means 2 for Hilbert transforming an original image signal 1 including a moving object in the time direction, the original image signal 1 and the image signal Hilbert transformed in the time direction. A complex number spatiotemporal filtering unit 3 for filtering a signal is provided, and the fact that the original image signal is a function of horizontal and vertical components and time, and the spatiotemporal frequency component of a moving object has a slope proportional to its velocity is used. Then, the moving object is extracted by applying a spatiotemporal filter.

In the present invention, the slope in the spatio-temporal frequency domain corresponding to a certain velocity of the moving body is u ₀ , the slope corresponding to a slower velocity is u _-1 , and the slope corresponding to a higher velocity is u _1. comprising a filtering means to the respective gradients u _0, u _-1, corresponding to u _1, 1 of the original image signal 1
By applying three types of spatiotemporal filters in a frame and updating the filter output having the maximum output among the three types of spatiotemporal filters as a filter corresponding to the slope u ₀ in the next frame, the moving object The feature is that the movement is stably extracted regardless of the change in speed.

The present invention constructs an image signal of a moving object.
A moving body that extracts the movement of the moving body from the frame
In the extraction device, the image signal is hillbelled in the time direction.
Time-direction Hilbert transformer for converting the time direction
Gives a time delay equivalent to the conversion time of the Hilbert transformer
The delay circuit and the time direction Hilbert transformer
The Hilbert transform image signal which has been subjected to the Rult transformation and the delay circuit
Extract horizontal image signal from delayed delayed image signal
Horizontal direction moving object extraction filter circuit, and the time direction
Hilbert transform Hilbert transform with Hilbert transformer
From the image signal and the delayed image signal delayed by the delay circuit
A vertical moving object extraction filter for extracting the vertical image signal.
And a horizontal circuit and a vertical component of the image signal.
And the space-time circumference of a moving object as a three-dimensional time
It is useful that the wavenumber component has a slope proportional to its movement speed.
By applying a spatiotemporal filter using
An apparatus for extracting a moving object, which is characterized by extracting.

[0012]

In the above-mentioned structure of the present invention, (1) a complex number signal is created from an input image (original image), and this is passed through a complex number space-time filter. At this time, the spatiotemporal filter is selected so as to pass the frequency region corresponding to the desired motion velocity.

The filtering is performed independently in the horizontal direction-time direction and the vertical direction-time direction of the image. That is,
Hilbert transform the input image in the time direction

[0014]

[Equation 1]

Is created. The filter output g (x, t) in the horizontal-time direction is

[0016]

[Equation 2]

And the filter output g (y, t) in the vertical-time direction is

[0018]

[Equation 3]

It is given by. here,

[0020]

[Equation 4]

Is the Hilbert transform of f (x, t) in the t direction, and * is the convolution integral. Also, W _RE (x,
t) and W _IM (x, t) respectively represent the real part and the imaginary part of the impulse response of the horizontal-temporal filter, and W _RE (y,
t) and W _IM (y, t) represent the real and imaginary parts of the impulse response of the vertical-time direction filter. After all, the filter output G (x, y, t) in the horizontal direction-vertical direction-time direction is

[0022]

[Equation 5]

It becomes Next, W _RE (x, t), W _IM (x, t), W
Set _RE (y, t) and W _IM (y, t). That is, when the image of the moving object is frequency-decomposed in the spatiotemporal region, the component of the spatial frequency ωsp at the velocity 0 (stationary state) is transferred to the straight line of the gradient u such that ωt = u · ωsp at the horizontal velocity u. .

Utilizing this fact, the filter W (ωx,
ωt) is set so that ωt = u · ωx and its surroundings pass through the band. Similarly, for the velocity v in the vertical direction, the filter W (ωy, ωt) is set to pass ωt = u · ωy and its periphery. Therefore, W _RE (x, t), W
_IM (x, t) is the inverse Fourier transform of W (ωx, ωt), W _RE (y,
t) and W _IM (y, t) are obtained by the inverse Fourier transform of W (ωy, ωt).

Further, processing for countermeasures against speed fluctuation and acceleration of the moving object of interest is further performed. That is,
The objects to be observed include speed fluctuations due to changes in the environment and acceleration of the objects themselves. Therefore, the slopes u and v are not constant, and one pair of filters is not sufficient. In the present invention, 1
A filter that adapts to the velocity before the frame (a filter that passes through the slope u or v and its periphery), a filter for the slower one, and a filter for the faster one are used.

The output of each of the above filters is g ₋₁ ,
g ₀ , g _1, and the _one with the maximum output is g (x, t) or g (y,
Adopted as t).

[0027]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of a moving object extraction system and its apparatus according to the present invention will be described in detail below with reference to the drawings. FIG. 1 is a functional block diagram showing the configuration of a moving object extraction system according to the present invention, in which 1 is an original image signal from an image input means such as a camera, 2 is a time direction Hilbert transform means, and 3 is a complex number spatiotemporal filtering means. Reference numeral 4 is a moving object extraction signal.

In the figure, the time direction Hilbert transform means 2 Hilbert transforms the original image signal 1 including the moving object in the time direction. A complex number signal is formed by the original image signal 1 and the image signal subjected to the Hilbert transform in the time direction, and the complex number signal is filtered by the complex number space-time filtering means 3. Note that DL is a delay unit that gives a delay corresponding to the processing time in the time-direction Hilbert transform unit 2.

In this way, the fact that the original image signal is a function of the horizontal and vertical components and the time is used to take advantage of the fact that the spatiotemporal frequency component of a moving object has a slope proportional to its velocity. By applying a filter, it is possible to obtain a moving object extraction signal indicating a moving object from the original image signal. FIG. 2 is an explanatory view of a spatiotemporal filter used in the moving object extraction method and the apparatus thereof according to the present invention, in which ω _x (ω _y ) is a horizontal (vertical) spatial frequency coordinate, ω _t is a temporal frequency coordinate, u is the velocity (pixel / frame) of the moving object.

In the figure, when the image signal of the moving object is frequency decomposed in the spatiotemporal domain, the component of the spatial frequency ωsp at the velocity 0 (stationary state) is ωt at the velocity u in the horizontal direction.
= U · ωsp Transfer to a straight line with a slope u. Utilizing this, filters corresponding to several types of inclination u are prepared in advance. For example, u ≧ 6, 6> u ≧ 3, 3> u ≧
1, 1>u> -1, -3 <u ≦ -1, -6 <u ≦ -3,
Prepare for 7 kinds of inclinations of u ≦ −6 (pixel / frame).

By the way, in the case of an object with u> 1 or u <-1, frequency aliasing occurs. For example, in the case of u = 2.5 shown in 8, the frequency folding 9
Occurs. In order to prevent the frequency aliasing component from being extracted in this filtering, the filter gain in the region of ωsp ≧ π / umax is 0 when umax is the maximum velocity of the moving object.

After all, seven kinds of filters of W0 to W6 as shown in FIG. 2 are subjected to inverse Fourier transform, and seven kinds of W _RE (x, t), W
_IM (x, t) is stored in the memory as a filter table. Similarly, W _RE (y, t) and W _IM (y, t) are also stored in the memory as a table. Next, the moving object detection apparatus according to the present invention will be described.

FIG. 3 is a block diagram for explaining the configuration of an embodiment of the moving object extracting apparatus according to the present invention, in which 1 is an original image signal, 10 is a delay circuit, 20 is a time direction Hilbert transformer, and 30 is horizontal. Directional moving object extraction filter circuit, 31 is a vertical direction moving object extraction filter circuit, 50 is a horizontal / vertical synthesizing circuit, and 4 is a moving object extraction signal. In the configuration of the figure, the input original image signal 1 is supplied to the delay circuit 10 and the time direction Hilbert transformer 3. The delay circuit 10 is
The time direction Hilbert transformer 20 is added to the input original image signal 1.
A delay amount corresponding to the delay time generated by the processing in (1) is given.

The delayed image signal output from the delay circuit 10 and the Hilbert-transformed image signal Hilbert-transformed in the time direction by the time-direction Hilbert transformer 20 are the horizontal moving object extraction filter circuit 30 and the vertical moving object extraction filter circuit. It is input to 31. The filter output x from the horizontal moving object extraction filter circuit 30 and the filter output y from the vertical moving object extraction filter circuit 31 are combined in the horizontal / vertical combining circuit 50, and the low frequency of the image of the object that makes a desired motion is combined. The component (animal body extraction signal output) 4 is obtained.

FIG. 4 is a block diagram for explaining the structure of the horizontal direction extraction filter circuit in FIG.
A delayed image signal which is the output of the delay circuit 10 in
Is the Hilbert transform image signal power output from the temporal Hilbert transformer 3, 30A-1, 30A-2, 30A-3
Is a real part convolution circuit, 30B-1, 30B-2, 30
B-3 is an imaginary part convolution circuit, 30C-1, 30C-
2, 30C-3 is a subtractor, 30D is a maximum output detector, 3
Reference numeral 0E is a filter type switch, 30F is a filter table, 30G is a switch, and 30H is an output image signal (filter output image signal x) of the horizontal extraction filter circuit.

In the figure, the delayed image signal 11 has weights W _{RE # i-1} , W from the filter table 30F in the real part convolution circuits 30A-1, 30A-2, 30A-3.
_The convolution operation processing is performed by _{RE # i} and W _{RE # i + 1} . Further, the time-direction Hilbert-transformed image signal 21 has the weights W _{IM # i -1,} W _{IM # i} , W from the filter table 30F in the imaginary part convolution circuits 30B-1, 30B-2, 30B-3.
_The convolution operation is processed by _{IM # i + 1} .

Real part convolution circuits 30A-1 and 30A-
2, 30A-3 and imaginary part convolution circuit 30B-1, 30
The respective output signals of B-2 and 30B-3 are subtracted by subtractors 30C-1, 30C-2 and 30C-3. Subtracted filter output image signals αx, βx, γ
x is input to the maximum output detector 30D. Where W
_{RE # i} and W _{IM # i} are the real and imaginary parts of the impulse response of the filter that extracts the slope u of the frequency component and its vicinity. W _{RE # i-1} and W _{IM # i-1} are the real and imaginary parts of the impulse response of the filter that extracts the slower than the velocity u,
W _{RE # i + 1} and W _{IM # i + 1} are the real part and the imaginary part of the filter that extracts those faster than the speed u.

For example, as W _{RE # i,} W _{IM # i} , 6> u ≧ 3
W _{RE # i-1,} W if a filter to extract
_{As IM # i-1} , a filter for extracting 3> u ≧ 3, W _{RE # i + 1, and} W _{IM # i + 1} for u ≧ 6 is used. In this circuit, among the three filter outputs αx, βx, γx obtained in a certain frame, the one having the maximum value is selected in the maximum output detector 30D, and the one having the maximum value is selected by the switch 30G as the output image signal 30H (filter It is output as the output image signal x). In addition, if αx is the maximum in this frame, the filter type switching unit 30E determines the current W _{RE # i-2} → the next frame W _{RE # i-1 the} current W _{IM # i-2} → the next frame. W _{IM # i-1} Current W _{RE # i-1} → Next frame W _{RE # i} Current W _{IM # i-1} → Next frame W _{IM # i} Current W _{RE # i} → Next frame of W _RE if _{# i + 1 W IM # i} + 1 βx is the largest of the current W _{IM # i} → the next frame, as it is γx is currently if the maximum W _{RE # i} → W _{RE #} of the next frame _i-1 current W _{IM # i} → next frame W _{IM # i-1} current W _{RE # i + 1} → next frame W _{RE # i} current W _{IM # i + 1} → next frame W _{IM # i} Current W _{RE # i + 2} → Next frame W _{RE # i + 1} Current W _{IM # i + 2} → Next frame W _{IM # i + 1} Filter table 30F Issue a command to.
The filter table 30F stores the W _{RE and} W _IM updated in the next frame in the respective convolution circuits 30A-1 and 30A-.
2, 30A-3 and imaginary part convolution circuit 30B-1, 30
B-2, 30B-3.

FIG. 5 is a block diagram for explaining the configuration of the vertical moving object filter circuit in FIG. 3, 11 is the delayed image signal output from the delay circuit 10 in FIG.
Reference numeral 1 denotes a Hilbert-transformed image signal output from the time-direction Hilbert transformer 20, 31A-1, 31A-2, 31A-.
3 is a real part convolution circuit, 31B-1, 31B-2, 3
1B-3 is an imaginary part convolution circuit, 31C-1, 31C-
2, 31C-3 is a subtractor, 31D is a maximum output detector, 3
1E is a filter type switching device, 31F is a filter table, 31G is a switch, and 31V is an output image signal (filter output image signal y) of the vertical direction extraction filter circuit.

In the figure, the delayed image signal 11 has weights W _{RE # i-1} , W from the filter table 31F in the real part convolution circuits 31A-1, 31A-2, 31A-3.
_The convolution operation processing is performed by _{RE #} i and W _{RE # i + 1} . The time-direction Hilbert-transformed image signal 21 has a weight W in the imaginary part convolution circuits 31B-1, 31B-2, 31B-3.
_The convolution calculation processing is performed by _{IM # i-1,} W _{IM # i,} W _{IM # i + 1} .

Real part convolution circuits 31A-1, 31A-
2, 31A-3 and imaginary part convolution circuit 31B-1, 31
The corresponding output signals of B-2 and 31B-3 are subtracted by subtractors 31C-1, 31C-2 and 31C-3. Subtracted filter output image signals αy, βy, γ
y is input to the maximum output detector 31D. Here, W _{RE # i and} W _{IM # i} are the real part and the imaginary part of the impulse response of the filter that extracts the slope u of the frequency component and its vicinity, as in the horizontal direction extraction filter described above. W _{RE # i-1,}
W _{IM # i-1} is the real and imaginary parts of the impulse response of the filter that extracts the slower than the speed u, and W
_{RE # i + 1 and} W _{IM # i + 1} are a real part and an imaginary part of a filter that extracts ones faster than the speed u.

For example, 6> u ≧ 3 as W _{RE # i} and W _{IM # i}
W _{RE # i-1,} W if a filter to extract
_{As IM # i-1} , a filter for extracting 3> u ≧ 1, W _{RE # i + 1, and} W _{IM # i + 1} for u ≧ 6 is used. In this circuit, among the three filter outputs αy, βy, γy obtained in a certain frame, the one having the maximum value is selected in the maximum output detector 31D, and the one having the maximum value is selected by the switch 31G as the output image signal 31V (filter It is output as the output image signal y). If αy is the maximum in this frame, the filter type switching unit 31E changes the current W _{RE # i-2} → the next frame W _{RE # i-1 the} current W _{IM # i-2} → the next frame. W _{IM # i-1} Current W _{RE # i-1} → Next frame W _{RE # i} Current W _{IM # i-1} → Next frame W _{IM # i} Current W _{RE # i} → Next frame of W _RE if _{# i + 1 W IM # i} + 1 βy the maximum current of the W _{IM # i} → the next frame, as it is γy is currently if the maximum W _{RE # i} → W _{RE #} of the next frame _i-1 current W _{IM # i} → next frame W _{IM # i-1} current W _{RE # i + 1} → next frame W _{RE # i} current W _{IM # i + 1} → next frame W _{IM # i} Current W _{RE # i + 2} → Next frame W _{RE # i + 1} Current W _{IM # i + 2} → Next frame W _{IM # i + 1} Filter table 31F Issue a command to.
The filter table 31F uses the convolution circuits 31A-1 and 31A- to update W _RE and W _IM updated in the next frame.
2, 31A-3 and imaginary part convolution circuit 31B-1, 31
B-2, 31B-3.

As a result, the moving body can be extracted or the area of the moving body can be recognized in real time using a relatively small-scale circuit means.

[0044]

As described above, according to the present invention,
Extract a desired moving object from an object that moves at different speeds in different directions on the screen of the image signal that is composed of an image signal frame that includes the moving object obtained from an image input device such as a camera, or specify the area can do.

Since the moving body is extracted by the spatiotemporal filtering, the hardware configuration of the moving object detecting device is simple and real-time processing can be realized. Furthermore, since the spatiotemporal filter is updated every frame, it is possible to stably extract the moving object or specify its area even when the speed of the moving object changes.

When the image input device itself such as a camera moves, not only the relative speed of the moving object to be observed may change but also the background image moves. According to the present invention, Only the desired moving object can be extracted or its area can be specified without being affected by this.

[Brief description of drawings]

FIG. 1 is a functional block diagram showing a configuration of a moving object extraction system according to the present invention.

FIG. 2 is an explanatory diagram of a spatiotemporal filter used in the moving object extraction method and device according to the present invention.

FIG. 3 is a block diagram illustrating the configuration of an embodiment of the moving object extracting apparatus according to the present invention.

FIG. 4 is a block diagram illustrating a configuration of a horizontal direction extraction filter circuit in an embodiment of the moving object extraction device according to the present invention.

FIG. 5 is a block diagram illustrating a configuration of a vertical moving object filter circuit in an embodiment of the moving object extracting device according to the present invention.

[Explanation of Codes] FIG. 1 1 Original image signal 2 Temporal Hilbert transform means 3 Complex number spatiotemporal filtering means 4 Moving object extraction signal FIG. 2 8 Slope of velocity u of u> 1 or u <−1 9 Folding of frequency of slope 8 Figure 3 1 Original image signal 4 Moving object extraction signal 10 Delay circuit 20 Time direction Hilbert transformer 30 Horizontal direction moving object extraction filter circuit 31 Vertical direction Moving object extraction filter circuit 50 Horizontal and vertical synthesis circuit Figure 4 11 Delayed image signal 21 Time direction Hilbert transform image signal 30A-1 to 30A-3 Real part convolutional circuit 30B-1 to 30B-3 Imaginary part convolutional circuit 30C-1 to 30C-3 Subtractor 30D Maximum output detector 30E Filter type selector 30F Filter table 30G switch 30H output image signal (filter output image signal x) 5 11 delays image Signal 21 Time direction Hilbert transform image signal 31A-1 to 31A-3 Real part convolutional circuit 31B-1 to 31B-3 Imaginary part convolutional circuit 31C-1 to 31C-3 Subtractor 31D Maximum output detector 31E Filter type switcher 31F Filter table 31G switch 31V Output image signal (filter output image signal y)

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical indication H04N 7/18 G K

Claims (2)

[Claims] 1. A time-direction Hilbert transform means for Hilbert transforming an original image signal including a moving object in a time direction, and a complex signal composed of the original image signal and the image signal Hilbert-transformed in the time direction is filtered. A spatio-temporal filter is provided by using a complex number spatio-temporal filtering means, and using the fact that the spatio-temporal frequency component of a moving object has a slope proportional to its speed as a function of the horizontal and vertical components and time of the image signal. A moving object extraction method for extracting the moving object by applying the object. 2. From the frames that make up the image signal of the moving object
In a moving body extraction device that extracts the movement of the moving body, Time direction for Hilbert transforming the image signal in time direction
A Hilbert transformer, When it corresponds to the conversion time of the time direction Hilbert converter
A delay circuit that provides a delay between Hilbert transformed by the time direction Hilbert transformer
The delayed image delayed by the Roberto transform image signal and the delay circuit
Horizontal motion to extract horizontal image signal from image signal
An object extraction filter circuit, Hilbert transformed by the time direction Hilbert transformer
The delayed image delayed by the Roberto transform image signal and the delay circuit
Vertical motion to extract vertical image signal from image signal
And an object extraction filter circuit, The image signal is divided into a horizontal component, a vertical component, and a three-dimensional time.
, The spatiotemporal frequency component of a moving object is its motion
Spatio-temporal fill using the fact that the slope is proportional to the speed
It is a special feature to extract the animal body by applying
A moving body extraction device.