JPH0239153B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a key signal adjustment device that can be applied to digital video signal processing devices such as digital chroma key devices and digital montage devices.

In these video signal processing devices, it is necessary to form a key signal corresponding to a partial area of one image. A well-known chromakey device has a configuration as shown in FIG. In the figure, 1 and 2 are color television cameras that photograph the foreground and background, respectively, 3 and 4 are gate circuits to which a foreground color video signal and a background color video signal are supplied, and 5 is a gate circuit for these gate circuits 3 and 4. A key signal generating circuit 6 that generates a key signal is a mixing circuit that mixes the outputs of the gate circuits 3 and 4 and leads it to an output terminal 7. As illustrated in FIG. 2A, a foreground 10 in which a subject 9 (for example, a person) is positioned in front of a back screen 8 painted with a back color, for example, blue, is photographed by a color television camera 1;
In the key signal generation circuit 5, a key signal is generated by calculating the three primary color components (R, G, B) in this foreground color video signal and converting the hue difference into an amplitude difference. That is, as shown in FIG. 2C, a key signal for turning on the gate is generated only at the subject 9, and this is supplied to the gate circuit 3.
On the other hand, as shown in FIG. 2D, a key signal is formed to turn on the gate only in areas other than the subject 9,
This is supplied to the gate circuit 4. Therefore, a signal of the image shown in FIG. 2E, in which the subject 9 is removed from the background 11 shown in FIG. 2B taken by the television camera 2, is generated from the gate circuit 4, and the gate circuit By mixing the signal corresponding to the object 9 from 3 in the mixer 6, the object 9 is mixed with the background 1 at the output terminal 7 as shown in FIG. 2F.
1 can be obtained.

In such a chromakey device, the edge portion of the generated key signal does not necessarily correspond to the boundary between the back screen 8 and the subject 9. Therefore, it is necessary to adjust the timing of the edge portion of the key signal.

This invention makes it possible to arbitrarily adjust the timing of the front edge and rear edge of the digital key signal when the video signal is digitized and the generated key signal is also digital. be. Another object of the present invention is to realize a digital key signal adjustment device that can adjust the timing of the edge portion not only to an integral multiple of the sampling period of the digital key signal but also within a range smaller than one period of this sampling period. be. If the present invention is applied to a digital chroma key device, it is possible to effectively prevent color fringing in which a back color remains on the side of the subject 9 at the boundary between the subject 9 and the background 11 in a composite image.

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the present invention is applied to a digital chromakey device for digital video signals of Y, U, and V signal systems will be described below with reference to the drawings.

In FIG. 3 showing the overall configuration of the digital chromakey device, 12 is foreground color video data.
This is the interface to which FG.VID and background color video data BG.VID are supplied together with their respective timing reference signals TRS. This color video data is the output of the color television camera (R,
For example, the luminance signal Y and color difference signals U and V formed by matrix calculation of (14:
Each component is sampled at a sampling frequency of 7:7). The interface 12 provides respective timing reference signals.
By looking at the timing signals (horizontal synchronization signal, vertical synchronization signal, etc.) decoded from the TRS, the phases of the two color video data FG.VID and BG.VID are set appropriately and outputted to the subsequent stage.

13 is a back color data forming circuit.
Back color data is formed from the foreground color video data FG.VID, and this back color data is sent to the key signal forming circuit 14 and the color canceller 1.
6.

The key signal forming circuit 14 performs a comparison operation for each corresponding sample between the back color data and the foreground color video data FG.VID, and generates a key signal of a predetermined level. The key signal itself generated in this way contains many disturbances and cannot be used as is, so as described later, the key processor 15 performs waveform shaping such as adjusting the grip, the edge timing of this clip output, and adjusting the gain. Processing is performed and key processor 1
A key signal KEY is obtained from 5.

The color canceller 16 uses this key signal.
Based on KEY, foreground color video data FG.VID
Remove the back collar from inside. For example, subject 9
When the background color is transparent, the transparent background color is removed. Specifically, the back color data is amplitude-modulated using the key signal KEY, and the modulated output is subtracted from the foreground color video data FG.VID. This back color removal is done by U and V.
The luminance signal Y is simply passed through.

This color canceller 16 includes a delay circuit 1.
Foreground color video data FG.VID is supplied via 7. The delay circuit 17 has a delay amount corresponding to the time required for the above-mentioned waveform processing in the key processor 15.

Then, the output CAN of the color canceller 16.
VID and background color video data BG.VID are supplied to the mixer 18, and mixing of the two is performed based on the key signal KEY. This mixing consists of two color video data CAN.VID and BG.
In addition to the method of simply switching and outputting VID, a crossfade method can be used in which the level of one is gradually decreased while the level of the other is gradually increased at the boundary between the two. The output of this mixer 18 is supplied to an interface 20 via a digital filter 19. The digital filter 19 is for adjusting the waveform of the output of the mixer 18.

The interface 20 receives the color canceled CAN.VID from the color canceller 16 and the composite color video data KYD.VID from the digital filter 19.
VID, respective timing reference signals, and key signals
This is for outputting the KEY to the outside.

Furthermore, a microprocessor 21, a CRT monitor 22, and a console 23 are provided, which translate the user's key input from the console 23 and transmit it to the inside of the system, and perform the calculation processing required in each circuit block. It is designed so that you can get used to it.

The digital chromakey device described above is operated by a sampling clock having a frequency corresponding to the sampling rate of the color difference data.

The processing in the key signal generation circuit 14 is as follows:
There are several methods, but for example, as shown in Figure 4, a reference point (U ₀ , V ₀ ) corresponding to the back color is specified on the (u, v) chromaticity coordinates, and this reference point (U ₀ , V ₀ ), the linear combination (K
A key signal is generated by calculating =|x|+|y|). Here, x=(U-U ₀ )cos θ+(V-V ₀ )sin θ y=(V-V ₀ )cos θ-(U-U ₀ )sin θ.

FIG. 5 shows the configuration of the key processor 15, in which a key signal K from the key signal generation circuit 14 is supplied to clip circuits 24 and 25. The clip circuit 24 forms a key signal HKEY for hard keying, the clip circuit 25 forms a key signal SKEY for soft keying,
One key signal HKEY is supplied to the non-additive mixer 27 and selector 28, and the other key signal SKEY is similarly supplied to the non-additive mixer 27 and selector 28 via the phase shift circuit 26. The non-destructive mixer 27 compares the values of the two key signals HKEY and SKEY and outputs the larger one.

The key signal output from the selector 28 is supplied to an edge timing adjustment circuit 29, and the timing of the edge, that is, the portion having a slope is adjusted.
This edge timing adjustment circuit 29 is shown in FIG.
The configuration is such that adjustments can be made in units of the clock sampling period t as shown in FIG. 2, and adjustments within this period t. The adjustment modes include a shift mode in which the key signal is moved in parallel in units of clock period t, as shown in FIG. 6B, and a shift mode in which the key signal is moved inward in units of clock period t, as shown in FIG. 6C. Coarse adjustment such as deflating (compression) or expanding outward (expansion), and the sixth
As shown in Figure D, there is a fine adjustment in which the edges are compressed within the clock period t. This edge timing adjustment circuit 29 will be described in detail later.

The key signal output from the edge timing adjustment circuit 29 is passed through the filter 30 to the key signal KEY.
is extracted as. This filter 30 reduces the influence of quantization errors in the key signal processing up to the previous stage, and also
Limit the key signal band to avoid aliasing noise when modulating the video signal with KEY.

For control and arithmetic processing in the key processor 15 as described above, an I/O controller 31 is provided.
Data, addresses and control signals are supplied to each circuit.

Hard keying and soft keying will be briefly explained with reference to FIG. For example, in front of the back screen 8, there is a transparent tip as the subject 9.
When photographing the foreground 10, which is set as A key signal K having a slightly smaller value is generated from the key signal forming circuit 14. In FIG. 7, the signal is shown as an analog waveform for convenience of explanation, but in the digital chroma key device described above, one sample consisting of 8 bits is data that is sequentially positioned at a sampling period t. Then, in the clip circuit 24, the base clip level BL and the peak clip level PLh
A clipping operation is performed using the threshold level as the threshold level, and a key signal HKEY for hard keying as shown in FIG. 7B is formed. Further, in the clip circuit 25, a clip operation is performed with the base clip level BL and the peak clip level PLs (>PLh) as threshold levels, and the seventh
Key signal for soft keying as shown in Figure C
SKEY is formed. In this manner, soft keying can form a key signal that corresponds well to the back color that is visible through the transparent subject 9 or to the reflected light of the back screen reflected on the subject 9.

FIG. 8 shows the basic configuration of color cancellation and mixing performed using the key signal KEY. First, when the level range of the key signal KEY from the lowest value to the highest value is 1, and the relative level of its instantaneous value is k, the arithmetic circuit 32
It is converted into (1-k) by being supplied to (1-k). Taking the key signal SKEY shown in FIG. 7C as an example, it is converted into the key signal SKEY' shown in FIG. 7D. This key signal KEY' is supplied to a multiplier 33 and modulates the back color signal D _B from the back color data forming circuit 13. This multiplier 33
The output of FG.VID is supplied to a subtracter 34 and subtracted from the foreground color video data FG.VID. Therefore, the subtracter 34 generates video data CAN.VID that corresponds to the subject 9 out of the color video data FG.VID and from which the back color in the subject 9 has been removed. The above-mentioned operation is nothing but that performed by the color canceller 16 in FIG. 3.

Also, the multiplier 35 inputs the video data CAN.
VID is modulated by the key signal KEY, and
In the multiplier 36, the background video data BG.VID is modulated by the key signal KEY′, and both multipliers 3
The outputs of 5 and 36 are added in an adder 37. In this output video data KYD.VID, in the case of a transparent subject 9 as described above, the background image can be seen through. Furthermore, due to the gradient of the edge of the key signal KEY, a crossfade is performed at the boundary between the subject 9 and the background 11, in which the image is gradually switched from one side to the other, giving a natural feel to the boundary between the images. You can also.

The edge timing adjustment circuit 29 included in the key processor 15 will be described in detail with reference to FIG.

This edge timing adjustment circuit 29 shifts the 1-sample 8-bit digital key signal KEY supplied from the selector 28 and sends it to the coarse adjustment circuit 38.
A fine adjustment circuit 39 shown enclosed by a broken line is provided at the subsequent stage, and a common control logic circuit 40 is provided for these. This control logic circuit 40 is supplied with data and control signals from a microprocessor via an I/O control circuit 31, and is supplied with shift amounts,
Coarse adjustment on/off, fine adjustment on/off, switching between expansion or compression, amount of adjustment, etc. are controlled.

First, the shift and coarse adjustment circuit 38 will be explained. It includes three RAMs 42, 43, 4
4 is provided. In this example, each RAM is assumed to have a capacity for four samples, since it is possible to shift or adjust by a maximum of four clock cycles. RAM 42 is for shift mode, RAM 43 is for coarse adjustment of the front edge, and RAM 44 is for coarse adjustment of the rear edge. RAM42, 43, 44 are
Write address formed by control logic circuit 40
Write operations are commonly controlled by WA.

The RAM 42 also includes a control logic circuit 40.
A read address _RA0 formed by RA0 is given.
RAM can be written and read within one memory cycle, and regarding address control of RAM 42, write address WA and read address
By making a difference between RA ₀ and 0, the input key signal
An output obtained by shifting KEY by (1 to 4) clock periods can be obtained.

Further, the read address formed by the control logic circuit 40 for each of the RAMs 43 and 44 is
RA ₁ and RA ₂ are supplied and this read address
The amount of expansion or compression is defined by controlling RA ₁ and RA ₂ and setting the amount of delay occurring in the RAM to a predetermined value.

The output of RAM42 is output via latch 45.
RAM43 and latches 46 and 4
7 and 48 to the RAM 44. 9th
The latches shown all introduce a delay of one sampling clock. Therefore, the data written to RAM 43 is in a phase advance relative to the output MID of latch 48. This RAM
The output of 43 is supplied via latches 49 and 50 to latch 56 of fine adjustment circuit 39. Also, RAM
The output of 44 is supplied via latches 51 and 52 to latch 56 of fine adjustment circuit 39. In this case, the control signals TK ₁ ,
TK ₂ , HLD are supplied to each of the latches 50, 52, and 56, and the output of either the latch 50 or the latch 52 is selected based on the trend of the key signal waveform, and the data update of the latch 56 is stopped. Ru.

Leading edges and trailing edges indicating the trend of the waveform are detected by comparing the output PRE of latch 49 with PRE', which is delayed by one clock period by latch 53, in level comparator 54. In other words, the detection signal CT becomes H when the two levels are equal and is in a flat region, the detection signal UP becomes H when the slope is rising (PRE>PRE'), that is, the front edge, and the detection signal UP is high when the slope is falling (PRE<PRE'). ') That is, H at the rear edge
A detection signal DW is generated and supplied to the control logic circuit 40. When performing this level comparison, it is practical to use the upper 6 bits of the output PRE of the latch 49 to widen the range to be determined as a flat area. These detection signals CT, UP, and DW are the output of latch 49.
It is synchronized with PRE.

A level comparator 55 is also provided, and the latch 4
The output PRE of latch 9 and the output FLW of latch 51 are compared in level to form a detection signal GT. This detection signal GT is used to prevent an unnatural waveform as a result of enlarging or compressing the edge portion during rough adjustment. This level comparator 55 includes I/
A mode switching signal is supplied via the O controller 41, and in the expansion mode, it becomes H when (FLW≧PRE), and in the compression mode, it becomes H when (FLW>PRE).
A detection signal GT that becomes H is generated when .

In the shift and coarse adjustment circuit 38 described above, when the coarse adjustment is off, the control signals are set to (TK ₁ =L, TK ₂ =H) according to instructions from the microprocessor.
The key signal read from RAM 44 is always supplied to latch 56 via latches 51 and 52. By controlling the read address _RA0 relative to the write address WA in the RAM 42, the key signal KEY can be shifted (delayed) by an integral multiple of the period of the sampling clock CK.

When the coarse adjustment is on, the microprocessor instructs expansion or compression and the amount thereof, and supplies them to the control logic circuit 40, and the operation mode of the level comparison circuit 55 is switched. In other words, the control logic circuit 40 has the following logic: When expanding: TK ₁ = UP, TK ₂ = DWM, GT HLD = ₁ + ₂ When compressing: TK ₁ = DW, GT TK ₂ = UPM, HLD = ₁ + ₂ Control signals TK ₁ , TK ₂ , HLD
is generated. Here, DWM and UPM are
DW and UP are each delayed by the phase difference between PRE and FLW. Further, the control amount of expansion and compression is determined by the latch 4 defined by successive addresses RA ₁ and RA ₂ from the control logic circuit 40.
The outputs PRE and FLW of 9 and 51 are the output of latch 48.
It is determined by the phase difference it has with respect to MID. During expansion, the front edge of PRE and the rear edge of FLW are selected by control signals TK ₁ and TK ₂ , and during compression,
The front edge of FLW and the back edge of PRE are control signals.
Selected by TK ₁ and TK ₂ . Therefore
By controlling the phase difference that PRE and FLW each have with respect to MID, the amount of expansion or compression can be controlled independently for the leading and trailing edges.

As an example, for the sampling clock shown in FIG. 10A, the output PRE of latch 49, the output PRE' of latch 53, the output MID of latch 48, and the output FLW of latch 51 are as shown in FIG. 10B. The enlargement operation when the illustrated waveform represents discrete sample data in an analog manner will be described. This figure 10B
As is clear from the waveforms, in the operation shown in FIG. 10, each of the RAMs 43 and 44 is controlled so that PRE and FLW have a leading phase difference and a lagging phase difference of one clock period, respectively, with respect to MID. The PRE
The phase difference between FLW and FLW is set to be two clock cycles.

The level comparison circuit 54 compares the levels of PRE and PRE', and the detection signals CT and PRE' shown in FIG.
Both UP and DW occur. Also, when expanding,
The level comparison circuit 55 generates a detection signal GT shown in FIG. 10E, which becomes H when (FLW≧PRE). These detection signals are transmitted to the control logic circuit 40.
The control signals TK ₂ , TK ₁ , and HLD as shown in FIG. 10F are formed by the above-mentioned logical formula. During the H period in which the control signals TK ₂ and TK ₁ include the rising edge, sample data included in each waveform of FLW and PRE is selected by the latches 52 and 50, and in the H period including the rising edge of HLD, the sample data is The position will be held.

The control signals TK ₂ , TK ₁ , shown in FIG. 10F
The sample data selected and held by the HLD is shown by white circles in FIGS. 10B and 10C, and as shown in FIG. 10C, both the front edge and the rear edge are A key signal EAK is obtained which is expanded by one clock period.

In addition, FIG. 11 shows a key signal EAK in which both the leading edge and trailing edge are compressed for one clock period.
This is a time chart showing the operation when forming a . The sampling clock shown in FIG. 11A, the waveform shown in FIG. 11B, and the detection signal shown in FIG. 11D are the same as those shown in FIG. 10 in the aforementioned enlargement operation. However, the operation has been changed so that the level comparison circuit 55 generates the detection signal GT shown in FIG. Therefore, the control signal as shown in Figure 11F
TK ₂ , TK ₁ and HLD are formed. Therefore,
The sample data marked with white circles in FIGS. 11B and 11C is selected and held, and as shown in FIG. 11C, both the front edge and the rear edge are
A key signal EAK compressed by one clock period can be formed.

Next, the fine adjustment circuit 39 will be explained. The key signal obtained at the output of the latch 56 is supplied to the buffer memory 57, RAM 58, and RAM 59.
These outputs are taken out via latch 60. Buffer memory 57, RAM58, 59
is the control signal formed by the control logic circuit 40
NC, ALE, and ATE are supplied as output control signals, and an output appears from each during the period when the control signal is H. The RAM 58 is a table for front edge conversion, and is loaded with conversion data from the microprocessor via the I/O controller 41. The RAM 59 is a table for post-edge conversion and is similarly loaded with conversion data from the microprocessor. In this example, compression is performed as a fine adjustment.
Therefore, the converted data is a value obtained by attenuating each sample data of the key signal provided from latch 56 by a predetermined amount.

First, when the fine adjustment is off, the control logic circuit 40
In this case, ALE=L, ATE=L, and NC=H, and an output always appears from the buffer memory 57, which is taken out as an output via the latch 60.

Further, when the fine adjustment is on, the logic for forming the control signal differs depending on whether the coarse adjustment operation of the preceding stage shift and coarse adjustment circuit 38 is on or off. When coarse adjustment is off, ALE=UPD ATE=DW・CTD+DWD・When coarse adjustment is on and enlargement is performed, ALE=TK ₁ D ATE=TK ₂ NC=+ When coarse adjustment is off and compression is performed, ALE=TK ₂ D ATE= Each control signal is formed by the logical formula TK ₁ NC=+. In the above formula
UPD, CTD, and DWD are the detection signals of UP, CT, and DW delayed by two clock cycles, and TK ₁ D and TK ₂ D are the control signals TK ₁ and TK during coarse adjustment, respectively. ₂ delayed by one clock period.

The fine adjustment operation when the coarse adjustment is off will be explained with reference to the time chart in FIG. There is. Detection signals CT, UP, and DW shown in FIG. 12D are generated from the level comparison circuit 54 at a timing synchronized with this key signal PRE. This detection signal is delayed by two clock cycles, CTD, UPD,
The DWD is shown in Figure 12E. buffer memory 57
and RAMs 58 and 59 have PRE latches 50,
A key signal shown in FIG. 12C delayed by 56 is provided. The key signals shown in FIGS. 12B, 12C, and 12G have consecutive samples, but are shown as analog signals for ease of understanding. Also, in the waveform indicated by the broken line in FIG. 12C, the waveform corresponding to the previous edge is RAM5.
8, and the data corresponding to the rear edge is the conversion data output from RAM 59.

When the coarse adjustment is off, the control signals ALE, ATE, NC shown in FIG.
are formed. This control signal causes the first
The sample data indicated by white circles in FIG. 2C is output from any of the buffer memory 57, RAM 58, or RAM 59, and as shown in FIG. is formed.

Note that the conversion data loaded into RAM 58 and RAM 59 respectively can be expanded within one clock cycle by loading data that increases the original data (maximum value is 255 with 8 bits). can.

FIG. 13 shows the configuration of another embodiment of the fine adjustment circuit 39. In this other embodiment, the ROM 67 generates a coefficient corresponding to the slope of the edge of the input key signal, and the multiplication circuit 63 multiplies this coefficient by each sample of the key signal.
For this multiplier circuit 63, latches 61, 62
A key signal is supplied via. Also, latch 6
The slope of the edge of the key signal is detected by 4 and subtraction circuit 65, and the detected signal is supplied to ROM 67 via latch 66 as an address. The coefficients generated in this ROM 67 are supplied to a multiplier circuit 63 via a latch 68. The sign of the slope is indicated by the most significant bit of the detection signal.

The fine adjustment circuit 39 having the configuration shown in FIG.
Since the predetermined level is uniformly attenuated regardless of the magnitude of the slope, as shown in FIG. 14A, the amount of compression τ ₁ where the slope is large and the amount of compression τ ₂ where this is small
are different, and (τ ₂ >τ ₁ ), resulting in variations in the amount of compression. On the other hand, in the configuration shown in FIG. 13, the slope is detected and the ROM 67 generates a multiplication coefficient that becomes larger as the slope becomes larger. Therefore, as shown in FIG. can be kept constant (τ ₁ =τ ₂ ).

As can be understood from the description of the embodiments described above, according to the present invention, the timing of an edge portion corresponding to a part of an image is controlled within the period of the sampling clock, like a digital key signal in a chromakey device. Adjustments can be made to expand or compress by an amount. Furthermore, in this invention, there is no need to form a clock with a frequency that is an integral multiple of the sampling clock, and it is only necessary to attenuate or increase each sample data of the key signal, so the timing system of the system can be improved. It has the advantage of not being complicated. However, with the present invention, the amount of expansion or compression can be adjusted independently for the leading edge and trailing edge of the key signal.

[Brief explanation of drawings]

1 and 2 are block diagrams showing the general structure of a conventional chromakey device and a schematic diagram used to explain its operation, and FIG. 3 is an overall diagram of an embodiment of a digital chromakey device to which the present invention is applied. 4 is a schematic diagram used to explain the key signal generation, FIG. 5 is a block diagram showing the structure of the key processor, and FIGS. 6 and 7 are waveform diagrams used to explain the key processor. FIG. 8 is a block diagram showing the general configuration of a color canceller and mixer, FIG. 9 is a block diagram of an embodiment of an edge timing adjustment circuit to which the present invention is applied, and FIG.
11 and 12 are time charts used to explain the operation of the edge timing adjustment circuit.
3 and 14 are block diagrams of other embodiments of the fine adjustment circuit included in the edge timing adjustment circuit and waveform diagrams used for explanation thereof. 14 is a key signal forming circuit, 15 is a key processor, 18 is a mixer, 29 is an edge timing adjustment circuit, 38 is a shift and coarse adjustment circuit, 39 is a fine adjustment circuit, 40 is a control logic circuit, 54, 55
is a level comparison circuit.

Claims (1)

[Scope of Claims] 1. Detection means for detecting a leading edge and a trailing edge of a key signal formed based on the hue of a video signal, data indicating a shift amount of the transition timing of the leading edge and trailing edge, and the above-mentioned edge timing adjustment means that is supplied with an output signal of the detection means and shifts the transition timings of the front edge and the rear edge of the key signal in a desired direction independently of each other based on data indicating the shift amount; A key signal adjustment device characterized by comprising: