JPS6033350B2 - correlation tracking device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention performs digital correlation calculation between analog-to-digital converted images of images output from an imaging device, and tracks a desired target or scene within the field of view of the imaging device. The present invention relates to a correlation tracking device.

In this type of device, as shown in Fig. 1, horizontal and vertical correlation regions 1 and 2 are provided in a cross shape with a width of m pixels, and a luminance signal is converted from analog to digital using Q bits of pixels in each region. was extracted, and m pixels were added to the two-dimensional luminance signal in the width direction to convert it into a one-dimensional multi-value luminance signal.

For the purpose of real-time processing of correlation calculations and miniaturization of the device, fixed scales and values have been set for this multivalued luminance signal and converted into a binary luminance signal.

In this binarized luminance signal, as shown in FIG.
The luminance signal of the part corresponding to is used as a reference signal for correlation calculation and is memorized in this memo.

Next, a correlation area is created on the aircraft for the image input from the green image device, and a one-dimensional binary luminance signal of that area is obtained.
The correlation coefficient was calculated by performing a 1-bit correlation calculation with the self-reference signal. Among the correlation coefficients obtained in this way, the coordinates on the screen that maximize the correlation coefficient are found, and the horizontal and vertical correlation windows are moved to that position to locate the desired target or scene within the field of view of the imaging device. I was tracking it. In this device, in a room where lighting conditions can be kept constant, the brightness of the target or scene within the field of view of the imaging device is constant, so when converting to a one-dimensional binary luminance signal, The optimal fixed threshold and value could be easily set manually. On the other hand, when used outdoors, the brightness is not constant depending on weather conditions and the time of use, and the contrast between the target and the background changes greatly, so when converting to a one-dimensional binary luminance signal, It is necessary to vary the threshold and value according to the conditions of use, but it is difficult to manually set the threshold and value to follow changes in brightness and contrast. In order to eliminate such a drawback, the present invention has a correlation window having a length of n pixels and a width of m pixels, and a luminance signal obtained by adding m pixels in the width direction of the correlation window to the center of the correlation window and the length from the center. △n pixels are sampled at equal intervals, each sampling value is weighted depending on its position, the sum is calculated, and the binarization threshold of the added luminance signal is set to the value, and the one-dimensional value obtained thereby is A binarized luminance signal is used to detect the maximum value of the correlation coefficient and the coordinates on the screen that give the maximum value, and based on this, a predetermined target or scene is tracked. A detailed explanation will be given below with reference to the drawings. FIG. 2 is a diagram illustrating the operating principle of the automatic value setting method according to the present invention.

In the following description, since the same correlation calculation is performed independently in the horizontal and vertical directions for the scanning line, the horizontal direction will be explained as an example. Figure 2a shows a correlation window with a length of n pixels and a width of m pixels, Figure 2b shows a luminance signal obtained by adding m pixels in the width direction, and Figure 2c shows the center of the correlation length n and its width. Length from center△
The summed luminance signal sampled at equal intervals by n pixels is shown in Fig. 2d, and the weighting function for each sampling value is shown as
FIG. 2e also shows the product of the sampling value and the weighting function, respectively. For convenience, the pixel at the upper left end of the horizontal correlation window 3 in FIG. 2a is assumed to be (1, 1) in the coordinate system, and the luminance signal of the pixel at coordinates (i, j) is expressed as P:j.

Then, the luminance signal Pm(i) obtained by adding m pixels in the correlation width direction
is given by ZPij.

This additional brightness bond j = No. 1 P(i) = i = Geiju k - sampled at equal intervals with length △n pixels from the back's position and clothing calendar
The luminance signal at the position of △n (k; earth 1, ±2, ±3--) is multiplied by the weighting function g(i) shown in the equation tl'.
(i) = Gei No. i: Gei + k△n The sum of each weighted added luminance signal P(i) and g(i) is determined.

In this invention, the value obtained as described above is used as a binarization threshold value for the added luminance signal obtained on the next input screen.

To simplify the explanation, a horizontal correlation window with a length n pixels and a width m pixels is set for the rectangular target as shown in Fig. 3, and the sampling position is set at the center of the correlation window and △n
=n2, that is, the case where it is set at the left end and right end of the correlation window will be explained.

3a shows the horizontal correlation window 3 and the rectangular target image 5 on the screen, FIG. 3b shows the video signal Vs captured using the imaging device, and FIG. 3c shows the video signal Vs. Figure 3 d shows the added luminance signal 2Ni after being added in the width direction of the correlation window, and Figure 3 e shows the digital video signal Ni obtained by analog-to-digital conversion using Q bits. FIG. 3f shows the weighting function g(i) of the added luminance signal ZNi sampled at the position 2Ni, and FIG. 3g shows the product of the added luminance signal 2Ni and the weighting function g(i). In FIG. 3c, the digital video signals of the target image and the background are assumed to be Nt and Nb.

Then, as shown in FIG. 3d, the luminance signals after adding m pixels in the width direction of the correlation window become mNt and mNb. If this added luminance signal is sampled at each layer of i=n2 and i=1,n, it will be mN as shown in Figure 3e.
t and mNb are obtained. The weighting function g(i) is as shown in FIG. The above added luminance signal 2Ni and weighting function g(i)
As shown in Figure 3 (g), the product is the target image 'maan Nt, the background criticism Nb and the ground, and their sum is 学 + (character + 学) = 学 + 学.

This is the value shown by the dotted line in Figure 3d,
Binarize this and set the value to 6.

In addition, when the target and background digital video signals within the correlation window are the same, that is, when NteNb, the actual binary value becomes "year", and all the binary luminance signals are "
However, if the contrast between the target and the background becomes 0 as described above, tracking cannot be performed using any tracking method, so it is inconvenient to use the binarization setting method of the present invention. does not occur.

FIG. 4 is a block diagram of an embodiment of a correlation tracking device using a binarization threshold and automatic value setting method according to the present invention.

In FIG. 4, a video signal supplied from an imaging device 7 is passed through an analog-to-digital conversion circuit 8 and converted into a digital signal with Q bits, and then a region dividing circuit 9 divides a predetermined position on the screen into horizontal and vertical directions. Divide into correlated regions.

Next, the adder circuit 10 is used for each correlation region to convert the two-dimensional multi-value signal into a one-dimensional multi-value signal.

This added luminance signal is passed through a sampling circuit 11 and sampled at the center of the correlation length and at intervals of Δn pixels from the center. Each sampled value is passed through a weighting calculation circuit 12 to be weighted, and then an addition circuit 13 is used to calculate the sum. This value is temporarily stored in the buffer memory 14 and used as the binarization threshold value for the next screen. When the next screen is input to the adder circuit 1, the added luminance signal, which is the output of the adder circuit 10 obtained in the same way, is sent to the digital comparator 1.
Buffer memory 4 obtained as above is connected to the A terminal of 5.
The binarization threshold values, which are the stored contents, are respectively input to the B terminal of the digital comparator 15, and the AZB calculation is performed to binarize the added luminance signal.

The binarized luminance signal obtained for each correlation area is passed through a switch 16, and the signal of the portion corresponding to the correlation window is stored in a memory 17 as a reference signal.

For the next screen, the one-dimensional binarized luminance signal obtained in the same manner is applied to the horizontal and vertical correlators 18 and 18V, and the reference signal is read out from the memory 17 and correlated in the horizontal and vertical directions. 18V and 18V respectively to calculate the correlation coefficient.

This correlation coefficient is passed through the maximum value detection circuit 19 to detect the maximum value of the correlation coefficient and output a coordinate signal on the screen giving the maximum value, which is fed back to the area dividing circuit 9, and is sent horizontally and vertically on the screen. By moving the position of the correlation window in the direction, the desired target or scene within the field of view of the imaging device 7 is captured by the electronic roof.

It can be tracked with. Further, using the tracking coordinate signal output circuit 20, a tracking display signal is generated by multiplying the video signal supplied from the imaging device 7 by the display signals of the horizontal and vertical correlation windows, and an error signal is output to the servo system. By controlling the attitude of the imaging device 7, a desired target or scene can be kept stationary on the screen and tracked. In addition, in order to perform real-time processing, a cross-shaped case in which correlation windows are set in the horizontal and vertical directions and correlation calculations are performed independently has been described above. It can also be used when using Further, although the above description has been made regarding a target having a simple brightness distribution, it goes without saying that the present invention is not limited to this and can be similarly applied to targets or scenes having a complex brightness distribution.

As described above, in the correlation tracking device according to the present invention, the summed luminance signal is sampled for each window in the correlation window in the horizontal and vertical directions and at equal intervals from the center of the correlation window,
An automatic threshold that sets the sum of the values obtained by weighting the sampled values as the binarization threshold/value of the next screen,
Obtain a one-dimensional binarized luminance signal using the value setting method, part 1
The maximum value of the correlation coefficient and the coordinates on the screen that give the maximum value are detected using the dimensional binary luminance signal, and a predetermined target or scene is tracked based on this, so it is possible to This has the advantage that it is possible to easily extract a luminance signal that follows changes in contrast between the target and the background, and to stably track a desired target or scene within the field of view of the imaging device.

[Brief explanation of the drawing]

Fig. 1 is a diagram showing the arrangement of the correlation region and the correlation window when a cross-shaped correlation window is used, Fig. 2 is a diagram showing the operating principle of the automatic threshold setting method of the correlation tracking device according to the present invention, and Fig. 3 is a diagram showing the arrangement of the correlation region and the correlation window when a cross-shaped correlation window is used. FIG. 4 is an explanatory diagram of the threshold/value calculation method when the center of the correlation window and the positions at both ends of the correlation window with respect to the target are selected as sampling points. FIG. 4 is a configuration diagram of an embodiment of the correlation tracking device according to the present invention. In the figure, 1 is a horizontal correlation area, 2 is a vertical correlation area, 3...
horizontal correlation window, 4 is vertical correlation window, 5 is rectangular target image, 6 is 2
digitization threshold, value, 7 is an imaging device, 8 is an analog-to-digital conversion circuit, 9 is an area division circuit, 10 is an addition circuit, 1
1 is a sampling circuit, 12 is a weighting calculation circuit, 13
is an adder circuit, 14 is a buffer memory, 15 is a digital comparator, 16 is a switch, 17 is a memory, 1
8th is a horizontal correlator, 18V is a vertical correlator, 19 is a maximum value detection circuit, and 20 is a tracking coordinate signal output circuit. It should be noted that the same or corresponding parts in the figures are indicated by the same reference numerals. Figure 4 Figure 1 Figure 2 Figure 3

Claims (1)

[Claims] 1. A correlation coefficient between a screen obtained by analog-to-digital conversion of a video signal supplied from an imaging device and a reference screen stored in advance is determined, the maximum correlation point is detected, and the correlation coefficient of the imaging device is determined. In a correlation tracking device that tracks a desired target or scene within the field of view, correlation windows with a length of n pixels and a width of m pixels are provided in each horizontal and vertical direction on the scanning line, and each correlation window has a width direction. A one-dimensional multivalued brightness signal is obtained by adding the brightness signal of each pixel to Sample the signal, and each sampled value P(i)
The weighting function g(i)g expressed by the following relational expression for
(i)=1/2; i=n/2 ▲There are mathematical formulas, chemical formulas, tables, etc.▼P(i)・g(i) multiplied by △n(K=±1,±2,±3...) This sum is set as the threshold value for binarizing the one-dimensional multilevel luminance signal obtained for the next screen, and the resulting 1
A correlation coefficient is calculated using a dimensional binary luminance signal, the maximum value of the correlation coefficient and the coordinates on the screen that give the maximum value are detected, and a predetermined target or scene is tracked based on that. A correlation tracking device characterized by: