JPH07101458B2 - Animal body extraction method and device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for extracting a moving body using an image or recognizing an area of the moving body in an image, and more particularly to a technique for extracting a moving body using a spatiotemporal filter. BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving object extraction method and an apparatus for extracting a desired moving object from objects moving in any direction at any speed.

[0002]

2. Description of the Related Art Conventionally, as a method for extracting the movement of an object included in an image signal captured by a television camera or the like, (1) a frame of brightness for each pixel forming the image signal has been used. A method of calculating the speed change and calculating the speed.

(2) A method of calculating the amount of movement of the moving object by obtaining the correlation between frames in the periphery including the moving object region. (3) A method of associating feature points such as contours and edges between frames and calculating the amount of movement thereof. (4) A method of calculating the speed by performing subtraction between frames, setting a region other than 0 as a motion region, and obtaining the time shift of the motion region.

(5) A method of extracting a moving object by independently applying a temporal filter and a spatial filter in which each impulse response is a real number. Etc. are known. As a technique disclosed in the above-mentioned spatiotemporal filtering, for example, p. Jay. Bart, et al. Written "Mechanisms for Isolation Component Pattern in the Sequential Analysis of Multiple Motion" Proceedings Eye EE Workshop on Visual Motion. PP.
187-193, 1991 (P.J. Burt, Raj.
esh Hingorani and Raymond
J. Kolezynski "Mechanisms
for Isolating Component P
atterns in the Sequential
Analysis of Multiple Mot
ion ”Proc. IEEE Workshop on
Visual Motion "PP. 187-19
3, 1991).

[0005]

The above-mentioned conventional methods have the following problems. In the case of (1), since calculation is performed for all pixels, the amount of calculation is large, and the processing circuit becomes large in scale for realizing real-time processing. Also, it is necessary to divide the real moving area based on the speed of each pixel,
Processing is complicated.

In (2), the correlation processing requires a large amount of calculation, and the processing circuit becomes large in scale for realizing the real-time processing. Also,
It is difficult to determine the moving object area for correlation processing. (3) is
Features are often different for each target, and it is difficult to perform feature extraction without losing generality (it is necessary to perform feature extraction according to the target). In (4), when the image input device itself such as a camera may move, the background of the input image differs between frames, and the entire area of the image becomes nonzero. Also, the subtraction result is 0 even if the light ray (light quantity) changes.
Other than Therefore, in order to make the subtraction result 0, it is necessary to devise the background processing.

Further, in (5), when an image signal is independently applied to a temporal filter and a spatial filter whose impulse responses are real numbers, objects moving in opposite directions cannot be distinguished. Therefore, the background may be extracted depending on the movement of the image input device itself. In addition, since the spatial frequency pass band is not limited, 1 pixel / frame (1 pixel / fr
ame) Objects that move above are indistinguishable.

In the above-mentioned conventional technique, only the object moving in a certain direction can be extracted in real time without losing the generality of the object (which can be done by the same processing method for any object). It was difficult. An object of the present invention is to solve the above-mentioned problems of the prior art and extract a moving body in real time using a relatively small-scale circuit means, or extract a moving body capable of recognizing a region of the moving body. To provide a method and its device.

In order to achieve the above object, the present invention is characterized in that an image signal obtained from an image input device is subjected to spatiotemporal filtering for extracting only a certain speed to extract a signal of a desired moving object. To do. That is, the present invention is
Spatio-temporal filtering means for convolving the image signal input from the input terminal 1 in the horizontal-temporal direction and the vertical-temporal direction, and storing the impulse response real part for storing the real parts of the impulse responses of the plurality of spatio-temporal filters. The input time series two-dimensional image signal is convoluted with the predetermined real part selected from the impulse response real part storing means for storing the real part of the impulse response of the spatiotemporal filter by the spatiotemporal filtering means. it the performs spatio-temporal filtering, the image signal the input horizontal component, and three-dimensional vertical component and time, dynamic
The spatiotemporal frequency component of a living object is proportional to its velocity
The image signal is controlled by utilizing the inclination.
It is characterized in that the moving object is extracted from the input image signal by passing through a limited plurality of pass bands .

Further, the image signal is three-dimensionally composed of the horizontal component, the vertical component and the time, and the spatiotemporal frequency component of the moving object has a slope proportional to its velocity. In the moving object extraction method for obtaining the extracted signal of the moving object in the input image signal, the spatio-temporal filter corresponds to the inclination of the velocity, and a type that passes a nearby region centering on the inclination, spatial frequency Direction, low frequency pass type, spatial frequency ωsp,
When the time frequency is ωt, ωsp> 0 and ωt> 0 are passed, or ωsp> 0 and ωt> 0 and ωt
With sp <0 and ωt <0 as symmetric types ,
Only a limited number of different types of
With multiple filters consisting of overband,
Multiple objects including moving objects of 1 pixel or more per frame
It is characterized in that an object having a desired motion is extracted from the object of FIG.

Further, when the gradient in the spatio-temporal frequency domain corresponding to a certain velocity is u0, the gradient corresponding to a slower velocity is u-1, and the gradient corresponding to a higher velocity is u1, these gradients u0, u -1 and u1, three types of spatiotemporal filters are applied in one frame, and the filter having the maximum output among the three types is updated as a filter corresponding to the gradient u0 in the next frame. The feature is that the object whose speed changes can be stably extracted.

[0012] Then, the apparatus for extracting the moving body,
The image signal is divided into horizontal, vertical, and temporal 3D
However, the spatiotemporal frequency component of a moving object is
In the horizontal direction-time direction
Horizontal Object Extraction Filter Circuit for Convolutional Integration
And vertical direction-vertical direction for performing convolution integration in the time direction
The moving object extraction filter circuit and the horizontal moving object extraction filter
Filter circuit and the output of the vertical moving object extraction filter circuit.
Combining force signals from three dimensions: horizontal, vertical, and time
It has a horizontal / vertical synthesis circuit that generates a moving object extraction signal
It is characterized by doing .

Further, the apparatus is characterized in that
The output filter has three convolution integrals corresponding to a certain speed.
Circuit and the maximum value of the outputs of the three convolutional integration circuits
The maximum value detector to detect and the maximum value detector
The output of the convolutional integrator circuit is selected and output.
Switch and the maximum value detection signal of the maximum value detector
A filter type corresponding to each of the slopes u0, u-1, u1
Filter type switch for switching, and the filter
Each said convolution integral updated by a type switcher
With a filter table that stores the weight given to the circuit
However, the vertical moving object extraction filter
Corresponding three convolution integrator circuits and said three convolutions
A maximum value detector that detects the maximum value of the output of the integrating circuit, and
The convolution integration times detected as maximum by the maximum detector
Switch for selectively outputting the output of the path, and the maximum value detector
To each of the slopes u0, u-1, u1 according to the maximum value detection signal of
Filter type switch to switch the corresponding filter type
Updated by the switch and the filter type switch
Stores the weight given to each of the convolutional integration circuits
And a horizontal table for extracting the moving object in the horizontal direction.
Filter and three vertical tatami mats of the vertical moving object extraction filter.
Of the convolutional integrators, the coefficient of the central convolutional integrator
Is the convolution coefficient with the maximum output in the previous frame and
However, the coefficient is a spatiotemporal filter corresponding to multiple velocity ranges.
Is the impulse response in a certain speed range, and the other 2
The coefficients of the two convolution integrators are slower than the certain speed mentioned above.
Impulse response over a wide speed range and certain speeds mentioned above
A impulse response at a faster speed range, the engagement
The number is updated every frame, the horizontal moving object extraction file
And the vertical moving object extraction filter .

[0014]

In the above structure, the time-series two-dimensional image f (x, y, t) which is the input image signal and the real part w (x, t), w (y, t) of the impulse response of the space-time filter are Gx (x, y, t) = f (x, y, t) * w (x, t) gy (x, y, t) = f (x, y, t) * w (y, t) In this way, by performing convolution integration in the horizontal direction-time direction and the vertical direction-time direction, spatiotemporal filtering is performed to extract an object moving at a desired speed. Here, * indicates convolution integral.

The horizontal direction-time direction and vertical direction-
From the filtering result in the time direction, the output G (x,
y, t) is set to G (x, y, t) = √ {(gx (x, y, t) · gy (x, y, t))}.

At this time, the spatiotemporal filtering means 2 is
It is chosen to limit the passband in the spatiotemporal frequency domain, as shown below. That is, the real part w (x, t), w (y, of the impulse response of this spatiotemporal filter
t) is set as follows. When Fourier transform is performed on the image of the moving object in the spatiotemporal domain, the component of spatial frequency ωsp at velocity 0 (stationary state) is ωt = at velocity u in the horizontal direction.
Transfer to a straight line with a slope u of u · ωsp. Utilizing this fact, the filter w (ωx, ωt) is set so that ωt = u · ωx and its periphery pass through the band.

Similarly, for the velocity v in the vertical direction, the filter w (ωy, ωt) is set so that ωt = uωy and its peripheral band pass. At this time, in order to prohibit the extraction of the object moving in the opposite direction, in the filter for u> 0, the region of ωsp> 0 and ωt <0 and ωsp <0
The gain is 0 in the region where 0 and ωt> 0. In the filter for u <0, the gain is 0 in the region of ωsp> 0 and ωt> 0 and the region of ωsp <0 and ωt <0. Further, in order to prevent extraction of folding components of an object moving at or above the maximum permissible velocity umax (umax ≧ 1), the pass band of the spatial frequency ωsp is limited to the low frequency region of ωsp <π / umax.

The filter w (ωx determined in this way
, Ωt) and w (ωy, ωt) are inverse Fourier transformed, and their real parts are w (x, t) and w (y, t). Note that the object to be observed may be accompanied by a speed change due to a change in the environment and the acceleration of the object itself. Therefore, the slopes u and v are not constant, and one pair of filters is not enough. In the present invention, a filter that adapts to the speed one frame before (a filter that passes the slope u or v and its periphery) is used, a filter for a slower speed and a filter for a higher speed. The respective outputs are g-1, g0, g1, and the one with the maximum output is the optimum g (x,
t) or g (y, t).

[0019]

Embodiments of the present invention will now be described in detail with reference to the drawings. FIG. 1 is a schematic block diagram for explaining a moving object extraction method according to the present invention, in which 1 is an input terminal of an image signal, 2 is a spatiotemporal filtering means, and 3 is an impulse for storing the real part of the impulse response of the spatiotemporal filter. The response real part storage means 4 is an output terminal for a moving object extraction signal.

In the figure, the image signal input from the input terminal 1 is a time-series two-dimensional image signal f (x, y, t), and this two-dimensional image signal is spatiotemporal filtering means 3
In the above, the spatiotemporal filtering is performed with a predetermined real part w (x, t), w (y, t) selected from the impulse response real part storage means 3 for storing the real part of the impulse response of the spatiotemporal filter, and the input is made. Taking advantage of the fact that the spatiotemporal frequency component of a moving object has a slope proportional to its velocity when the image signal is three-dimensional with horizontal component, vertical component and time,
The moving object extraction signal obtained by extracting the moving object in the input image signal is obtained at the output terminal 4.

FIG. 2 is an explanatory view of a spatiotemporal filter used in the moving object extraction method according to the present invention, in which ω _x (ω _y ) is a spatial frequency coordinate in the horizontal direction (vertical direction), ω _t is a temporal frequency coordinate, u is the speed of the moving object (pixel / frame: pixel /
frame). In the figure, when the image signal of the moving object is frequency-decomposed in the spatiotemporal domain, the component of the spatial frequency ωsp at the velocity 0 (stationary state) is on the straight line of the gradient u such that ωt = u · ωsp at the horizontal velocity u. Transfer to. Utilizing this, filters corresponding to several types of inclination u are prepared in advance by batch processing.

For example, u ≧ 6, 6> u ≧ 3, 3> u ≧
1, 1>u> -1, -3 <u ≦ -1, -6 <u ≦ -3,
Prepare for 7 kinds of inclinations of u ≦ −6 (pixel / frame). By the way, in the case of an object with u> 1 or u <-1,
Frequency wrapping occurs. For example, u = 8
In the case of 2.5, frequency folding 9 occurs as shown.

In order to prevent the frequency aliasing component from being extracted in this filtering, the maximum allowable velocity of the moving body is set to uma.
When x is set, the filter gain in the region of ωsp ≧ π / umax is 0. After all, seven types of filters of w0 to w6 as shown in FIG. 2 are subjected to inverse Fourier transform, and seven types of w (x, t) are stored in the memory as a filter table. w
Similarly, (y, t) is also stored in the memory as a table.

FIG. 3 is a block diagram for explaining the configuration of an embodiment of the moving object extracting apparatus according to the present invention, in which 1 is an image input terminal, 21 is a horizontal moving object extracting filter circuit, and 22 is a horizontal moving object extracting filter circuit.
Is a vertical direction object extracting filter circuit, 23 is a horizontal / vertical combining circuit, and 4 is an object extracting signal output terminal. In the configuration shown in the figure, the input image signal is a horizontal moving object extracting filter circuit 21 and a vertical moving object extracting filter circuit 2.
Entered in 2. Horizontal moving object extraction filter circuit 21
And the filter output from the vertical direction moving object extraction filter circuit 22 are combined in the horizontal / vertical combining circuit 23 by calculation of √ (x · y), and are combined at a low frequency of the image of the object that makes a desired motion. The components are obtained at the moving object extraction signal output terminal 4.

FIG. 4 is a block diagram showing the configuration of the horizontal direction extraction filter circuit in FIG.
2, 213 is a convolution circuit, 214 is a maximum output detector,
Reference numeral 215 is a switch, 216 is a filter type selector, and 31 is a filter table. In the figure, the input image signal has a weight w in the convolution circuit 211.
_{# i-1} is convolved so that α _x is the maximum output detector 21
4 is input. Similarly, the convolution circuit 212 convolves with the weight w _#i , the result β _x, and the convolution circuit 213.
The result γ _x convolved with the weight w _{# i + 1} at is input to the maximum output detector 214.

Here, w _#i is the real part of the impulse response of the filter that extracts the slope u of the frequency component and its vicinity. w _{# i-1} is the real part of the impulse response of the filter that extracts slower than the speed u, and w _{# i + 1} is the real part of the filter that extracts faster than the speed u. For example, when a filter that extracts 6 <u ≦ 3 is used as w _#i , a filter that extracts 3 <u ≦ 1 as w _{# i−1 and} u ≧ 6 as w _{# i + 1} is used. Used.

In this circuit, obtained in a certain frame
Three filter outputs α_x, Β_x, Γ _xIs the maximum output detector
The one with the maximum value is selected at 214 and the switch
The one with the maximum value is the horizontal direction
The force signal x is output to the horizontal / vertical synthesis circuit 23.
In addition, the filter type switching unit 11 is
Where α_xIf is the maximum, the current w_{# i-2} → w of the next frame_{# i-1} Current w_{# i-1} → w of the next frame_#i Current w_#i → w of the next frame_{# i + 1} β_xIf is the maximum γ_xIf is the maximum, the current w_#i → w of the next frame_{# i-1} Current w_{# i + 1} → w of the next frame_#i Current w_{# i + 2} → w of the next frame_{# i + 1} The filter table 31 is instructed so that F
The filter table 31 will be
Each of the convolution circuits 211 to 211 using the updated w value as a weight.
Give to 213.

FIG. 5 is a block diagram showing the configuration of the vertical direction extraction filter circuit in FIG.
2, 223 is a convolution circuit, 224 is a maximum output detector,
225 is a switch, 226 is a filter type switch, and 32 is a filter table. In the figure, the input image signal has a weight w in the convolution circuit 221.
_{# i-1} is convolved so that α _y is the maximum output detector 22
4 is input. Similarly, the convolution circuit 222 performs convolution with the weight w _#i , β _y, and the convolution circuit 223.
The result γ _y convolved with the weight w _{# i + 1} at is input to the maximum output detector 224.

Here, w _#i is the real part of the impulse response of the filter that extracts the slope u of the frequency component and its vicinity. w _{# i-1} is the real part of the impulse response of the filter that extracts slower than the speed u, and w _{# i + 1} is the real part of the filter that extracts faster than the speed u. For example, when a filter that extracts 6 <u ≦ 3 is used as w _#i , a filter that extracts 3 <u ≦ 1 as w _{# i−1 and} u ≧ 6 as w _{# i + 1} is used. Used.

In this circuit, it is obtained in a certain frame
Three filter outputs α_y, Β_y, Γ _yIs the maximum output detector
At 224, the one with the maximum value is selected and the switch
According to 225, the one with the maximum value appears in the vertical direction.
The force signal y is output to the horizontal / vertical synthesis circuit 23.
In addition, the filter type switch 226 is
, Α_yIf is the maximum, the current w_{# i-2} → w of the next frame_{# i-1} Current w_{# i-1} → w of the next frame_#i Current w_#i → w of the next frame_{# i + 1} β_yIf is the maximum γ_yIf is the maximum, the current w_#i → w of the next frame_{# i-1} Current w_{# i + 1} → w of the next frame_#i Current w_{# i + 2} → w of the next frame_{# i + 1} To the filter table 32. F
The filter table 32 will be
Each of the convolution circuits 221 to 221 uses the updated w value as a weight.
Give to 223.

With this, it is possible to extract the moving body or recognize the region of the moving body in real time by using a relatively small-scale circuit means.

[0032]

As described above, according to the present invention,
A desired moving object can be extracted from objects moving at different speeds in different directions on the screen, and different on the screen of the image signal composed of an image signal frame including a moving object obtained from an image input device such as a camera. A desired moving object can be extracted from an object that moves at different speeds in different directions, or its region can be specified.

Further, since the moving object is extracted by the spatiotemporal filtering, the hardware configuration of the moving object detecting device is simple and the real-time processing can be realized. Furthermore, since the spatiotemporal filter is updated every frame, it is possible to stably extract the moving object or specify its area even when the speed of the moving object changes.

When the image input device itself such as a camera moves, not only the relative speed of the moving object to be observed may change but also the background image moves. According to the present invention, , It is possible to extract only a desired moving object or specify its area without being affected by this.

[Brief description of drawings]

FIG. 1 is a schematic block diagram illustrating a moving object extraction method according to the present invention.

FIG. 2 is an explanatory diagram of a spatiotemporal filter used in the moving object extraction method according to the present invention.

FIG. 3 is a block diagram illustrating the configuration of an embodiment of a moving object extracting apparatus according to the present invention.

4 is a block diagram showing a configuration of a horizontal direction extraction filter circuit in FIG.

5 is a block diagram showing a configuration of a vertical direction extraction filter circuit in FIG.

[Explanation of symbols] Fig. 1 1 Input terminal of image signal 2 Spatio-temporal filtering means 3 Storage means of impulse response real part 4 Output terminal of moving object extraction signal Fig. 2 8 Gradient of u = 2.5 9 u = 2.5 Frequency folding diagram 3 1 Image input terminal 21 Horizontal direction moving object extraction filter circuit 22 Vertical direction moving object extraction filter circuit 23 Horizontal / vertical synthesis circuit 4 Moving object extraction signal output terminal diagram 4 211, 212, 213 Convolution circuit 214 Maximum output detection 215 Switch 216 Filter type changer 31 Filter table Fig. 221, 222, 223 Convolution circuit 224 Maximum output detector 225 Switch 226 Filter type changer 32 Filter table

Claims (5)

[Claims] 1. Spatio-temporal filtering means for convolving the image signal in the horizontal-temporal direction and vertical-temporal direction, and impulse response real part storage means for storing real parts of impulse responses of a plurality of spatio-temporal filters. By convoluting the input time series two-dimensional image signal with the predetermined real part selected from the impulse response real part storing means for storing the real part of the impulse response of the spatiotemporal filter by the spatiotemporal filtering means. An object with motion by performing spatiotemporal filtering to make the input image signal three-dimensional with horizontal components, vertical components and time .
The spatiotemporal frequency component of the body has a slope proportional to its velocity
By utilizing this, the image signals are
A moving object extraction method characterized in that the moving object is extracted from the input image signal by passing it through several pass bands . 2. An image signal is three-dimensionally composed of a horizontal component, a vertical component and time, and a spatiotemporal filter is applied by utilizing that the spatiotemporal frequency component of a moving object has a slope proportional to its velocity. In the moving object extraction method for obtaining an extracted signal of the moving object in the input image signal, the spatiotemporal filter corresponds to the slope of the velocity,
A type that passes a nearby region centered on this slope, a type that passes a low frequency band in the spatial frequency direction, and a spatial frequency ω
where sp and time frequency are ωt, ωsp> 0 and ωt>
Type that passes only 0, or ωsp> 0 and ωt> 0
And a ωsp <0 and symmetric types and .omega.t <0, these territories
A restricted plurality of types that only pass through the zone
By having multiple filters with different passbands,
Includes a moving object whose degree is 1 pixel or more per frame
A moving object extraction method characterized by extracting an object having a desired motion from a plurality of objects including polarities in the direction of motion. 3. The slope according to claim 2, wherein the slope in the spatiotemporal frequency domain corresponding to a certain velocity is u0, the slope corresponding to a slower velocity is u-1, and the slope corresponding to a higher velocity is u1.
Then, in correspondence with these inclinations u0, u-1, and u1,
By applying three types of spatiotemporal filters in one frame and updating the filter having the maximum output among the three types as a filter corresponding to the slope u0 in the next frame, an object whose speed changes can be stabilized. A method for extracting a moving object, which is characterized in that it can be extracted by using the above method. 4. An image signal having a horizontal component, a vertical component, and a time component.
And the spatiotemporal frequency component of a moving object
Use the inclination that is proportional to the speed to move horizontally
− Horizontal moving object extraction function that performs convolution integration in the time direction
Filter circuit and convolutional integration in the vertical-time direction.
Vertical moving object extraction filter circuit
Object extraction filter circuit and vertical moving object extraction filter
The output signals of the digital circuit to synthesize horizontal, vertical, and time
Horizontal / vertical combination that generates a three-dimensional moving object extraction signal
An animal body extraction device having a synthetic circuit. 5. The horizontal moving object extractor according to claim 4,
The output filter has three convolution integrals corresponding to a certain speed.
Circuit and the maximum value of the outputs of the three convolutional integration circuits
The maximum value detector to detect and the maximum value detector
The output of the convolutional integrator circuit is selected and output.
Switch and the maximum value detection signal of the maximum value detector
A filter type corresponding to each of the slopes u0, u-1, u1
Filter type switch for switching, and the filter
Each said convolution integral updated by a type switcher
With a filter table that stores the weight given to the circuit
However, the vertical moving object extraction filter corresponds to a certain speed.
Three convolutional integration circuits, and the three convolutional integration circuits
A maximum value detector for detecting the maximum value of the output of the circuit, and
Of the convolutional integration circuit detected as maximum by the large value detector
The switch that selects and outputs the output and the maximum of the maximum value detector.
Corresponds to each of the slopes u0, u-1, u1 according to the large value detection signal
Switching the selected filter type Switching the filter type
And the filter type switching device
Storing the weight given to each convolutional integration circuit
A horizontal table, the horizontal direction moving object extraction filter and the vertical direction moving object.
The center of each of the three convolutional integration circuits of the extraction filter
The coefficient of the convolutional integrator of has the maximum output in the previous frame.
The value is a convolution coefficient, and the coefficient is a spatio-temporal filter corresponding to a plurality of velocity ranges.
Is the impulse response in a certain velocity range of
The coefficient of the convolutional integrator circuit is slower than the certain speed described above.
From the impulse response in degrees range and the certain speed mentioned above
It is an impulse response in a fast velocity range, and the coefficient is updated every frame, and the horizontal moving object is extracted.
Each of the filter and the vertical direction moving object extraction filter
Animal body extraction device and performing in.