JP2685580B2 - Image synthesis device - Google Patents

Description

The present invention relates to an image synthesizing apparatus for simply synthesizing television images.

[Conventional technology]

When performing television broadcasting, a method of inserting a prepared insertion image (inset image) into a part of the studio set is used.

In such a case, the rectangular portion of R ₀ , R ₁ , R ₂ , R ₄ of the studio set 401 as shown in FIG.
When a photograph is taken from the front directly at 402, a similar rectangular image is obtained on the display screen 403. From this state, rotate the television camera 402 to the right, left, up, down,
When the zoom factor is converted, the display screen 40
The image of 3 is transformed as indicated by reference numeral 404, for example. This is referred to as primary perspective transformation, and assuming that the new coordinates on the display screen 403 are X, Y and the original coordinates are x, y, the above X, Y can be expressed by the following conversion formula.

X = m · (dx + ey + f) / (ax + by + c) Y = m · (gx + hy + i) / (ax + by + c) (1) When this primary perspective transformation is performed by digital signal processing, first the original coordinate value x of the display screen 403 is set. , y is used to perform the operation according to the equation (1). Next, among the obtained coordinate values, the image data of the original coordinate having the coordinate value closest to the new coordinate X, Y of the display screen 403 is set as the image data of the new coordinate, or a plurality of neighboring pixels ( The original value of the original image data is calculated, and this average value is used as the image data of the new coordinate. Therefore, the horizontal pixel count of the display screen 403 is P _H and the vertical pixel count is P _H.
When it is set to _V , it is necessary to repeat the calculation of the above formula (1) P _H × P _V times per screen, which makes high-speed processing difficult.

Therefore, in the conventional art, the studio set 401 is photographed directly from the front, and the inserted image is reduced and enlarged to perform image combination.

[Problems to be solved by the invention]

However, in reality, there are cases where it is desired to rotate the television camera and change the zoom magnification to shoot. For this reason, there has been a demand for the development of an image synthesizing apparatus capable of synthesizing a natural image by inserting an inserted image in the area indicated by the hatching of the image transformed as shown by reference numeral 404 in FIG.

Therefore, it is an object of the present invention to provide an image synthesizing device capable of obtaining a natural synthetic image even when a television camera is rotated and the zoom magnification is changed for shooting.

[Means for solving the problem]

An image synthesizing apparatus according to the present invention stores first and second sensors for detecting a rotation angle and a zoom magnification of a television camera, and image data of an inserted image to be inserted into an actual image obtained by the television camera. The memory and the coordinate data of the four corners on the display screen where the image data in the memory should be inserted into the above-mentioned actual image.
And a quadrangular data calculation means for calculating based on the output of the second sensor, and a start of reading the image data from the memory when inserting the image data in the memory into the image data of the actual image on the display screen from the calculation result. The read control data calculation means for obtaining the address data and the magnification data indicating the magnification of the address step for each address in the horizontal and vertical directions of the memory, and the image data in the memory corresponding to the horizontal and vertical directions of the display screen. A modified image creating means for creating a modified image by performing reading based on the start address data and the magnification data; and an inserting means for inserting the modified image created by the modified image into an actual image obtained by the television camera. Characterize.

[Action]

Since the image synthesizing apparatus according to the present invention is configured as described above, the coordinate data of the four corners on the display screen into which the image data of the inserted image should be inserted by the four-corner data calculating means, that is, the reference numeral in FIG. The coordinate data of the four corners (Q _{0 to} Q ₃ ) of the area indicated by the hatching of the image transformed as indicated by 404 is calculated. Further, from the relationship between the area including the four corners and the area of the memory in which the image data of the inserted image is stored and the magnification ratio in the horizontal and vertical directions, the start address data and the address step ratio for reading the image data from the memory are determined. If these are obtained for each of the horizontal and vertical addresses of the memory and the image data is read from the memory based on these data, the image data will be read in the horizontal and vertical addresses. A deformed image that is reduced or enlarged is obtained for each image, and the deformed image is in a shape that enters a square area including the above-mentioned four corners. Therefore, the deformed image is inserted into the real image obtained by the television camera by the inserting means, and the appropriate combination is performed.

〔Example〕

Hereinafter, referring to FIG. 1 to FIG. 3 of the accompanying drawings,
An image synthesizing apparatus according to an embodiment of the present invention will be described.

FIG. 1 is a block diagram showing an image synthesizing apparatus according to an embodiment of the present invention. In the figure, the television camera 101
Two rotary encoders (first sensors) 103 and 104 are attached to the pan head 102. The rotary encoder 103 detects a rotation angle θ _x with the X axis as the rotation axis in the Cartesian coordinate system in the figure, and the rotary encoder 104 detects a rotation angle θ _y with the Y axis as the rotation axis. A potentiometer (second sensor) 105 is attached to the lens, and the projection magnification m is detected. Rotary encoder 103,10
The output signals of the 4 and potentiometer 105 are guided to the conversion circuit 106, digitally converted into, for example, 16-bit data that can be processed by the computer 107, and transferred to the computer 107.

The television camera 101 photographs the studio set 108 having a rectangular window in the center of the panel, and sends the image data obtained by this photographing to the inserting means 109. When the television camera 101 is placed at the reference position and the studio set 108 is photographed from directly in front with a projection magnification of m = 1, the coordinate values of the four corners of the frame 110 of the studio set 108 on the display screen are R ₀ (x _R0 , y _R0 ), R ₁ (x _R1 , y _R1 ), R ₂ (x _R2 , y _R2 ), R ₃ (x _R3 , y _R3 ), and the television camera 101 with the X axis as the rotation axis.
The above-mentioned frame on the display screen when the projection magnification is m by rotating θ _y with the _x and Y axes as rotation axes, respectively.
Let Q ₀ (x _Q0 , y _Q0 ), Q ₁ (x _Q1 ,, y _Q1 ), Q ₂ (x _Q2 , y _Q2 ), and Q ₃ (x _Q3 , y _Q3 ) be the coordinate values of the four corners of 110. x _Q = m · (x _R cos θ _y −z _R sin θ _y ) / (x _R cos θ _x · sin
θ _y −y _R sin θ _x + z _R cos θ _x · cos θ _y ) y _Q = m · (x _R sin θ _x · sin θ _y + y _R cos θ _x + z _R sin θ
_x・ cos θ _y ) / (x _R cos θ _x・ sin θ _y −y _R sin θ _x + z _R
cos θ _x · cos θ _y ) (where Q = Q _{0 to} Q ₃ , R = R _{0 to} R ₃ , z _R ; Z-axis coordinate value of the reference position) (2).

The computer 107 follows the rotation angle θ _x ,
Based on the data of θ _y and zoom magnification m, the above (2)
The coordinate values of the four corners are calculated using the formula. It is assumed that x _R , y _R , and z _R have been previously given as data to the main memory.

On the other hand, the memory 111 stores the image data of the insertion image 111A. This inserted image 111A has a rectangular shape as shown, and the four corner addresses are P ₀ (0,0), P ₁ (W, 0), P ₂ (W, h), and P ₃ (0, h). Then, in order to transform the insertion image 111A into a quadrangle having Q ₀ , Q ₁ , Q ₂ , and Q ₃ as quadrangles, the horizontal (H) direction and the vertical direction (V) direction at the time of memory reading from the memory 111 are changed. The start address S _H (y), S _V (x) and the magnifications M _H (y) and M _V (x) may be represented by the following equation (3).

_{M H (y) = {(} x Q1 + x Q0) / W} {(h-y) / h} + {(x Q2 -x Q3) / W} (y / h) S H (y) = - M _H (y) ・ [x _Q0 {h−y) / h} + x _Q3 (y / h)] M _V (x) = {(x _Q3 −x _Q0 ) / h} {(x _Q1 −x) (x _Q1 − x _Q0 )} + {(x _Q2 −x _Q1 ) / h} {(x−x _Q0 ) (x _Q1 −x _Q0 )} S _V (x) = −M _V (x) ・ [y _Q0 { _{(x Q1 -x) / (x} Q1 -x Q0)} + y Q1 {(x-x Q0) / (x Q1 -x Q0)} ] ... (3) Kopyuta 107 based on a program in the main memory ( Using the equation 3), the start address data and the magnification data are calculated for each address in the horizontal and vertical directions. Number of operations at this time, the number of horizontal pixels of the display screen P _H, the number of vertical pixels may be repeated one screen per P _H + P _V times when the P _V, greatly compared to conventional P _H × P _V times The number of calculations can be reduced, and therefore high speed processing is possible.
The data of W and h are given and stored in the main memory of the computer 107 in advance. The start address data and the magnification data thus obtained are sent to the modified image creating means 112. Thus, the computer 107
Functions as a square data calculation means and a read control data calculation means.

 FIG. 2 shows a detailed configuration of the modified image creating means 112.

Since the deformed image creating means 112 performs the deformation in the horizontal direction and the vertical direction, the memory access units 200 _H , 200 _V
Has the same configuration, only the inside of the memory access unit 200 _H is shown in detail. The buffer memories 201 and 202 store the image data of the transformed image, and need not be physically two, and a magnification register for storing magnification data.
203 and a start address register 204 for storing start address data are each configured by RAM. The counter 205 also generates vertical address data. The output of the oscillator 206, which generates the read clock of the image data corresponding to the pixel, is the latch circuit 207 and the counter 20.
Given to 8. The counter 208 is an h-ary counter, and the MSB of its output is given to the counter 205 and the selector 209. The contact of the selector 209 is normally tilted to the contact a side, and is tilted to the contact b side only while a pulse is being applied. The counter 205 is a W-adic counter, the output of which is given to the magnification register 203 and the start address register 204, and magnification data and start address data corresponding to the respective addresses are output from these registers. The output of the scale factor register 203 is given to the adder 210, and the output of the start address register 204 is given to the contact b of the selector 209. Note that the horizontal scale factor data M to the scale factor register 203 is
_The computer 107 sets _H (y) and horizontal start address data S _H (y) in the start address register 204. The adder 210 adds the reciprocal of the scale factor data output from the scale factor register 203 to the output of the latch circuit 207, thereby making the contact a of the selector 209 a.
Send the output data to the side. The output of the latch circuit 207 is supplied to the adder 210 as described above, and the memory 1
It is horizontal address data for accessing 11. The outputs of the counters 208 and 205 are given to the buffer memory 201 as horizontal address data and vertical address data, respectively. Selectors 211 and 212 are memory access units 200 _H and 200 _V
This is a configuration for selecting either one of the outputs. The control signal R / W is a read / write switching signal, and the selector 211
.. are supplied from the computer 107 together with the switching signal S of 214.

The deformed image creating unit 112 configured as described above operates as follows with respect to the horizontal deformation of the insertion image 111A. This will be described with reference to FIG.

The computer 107 sets magnification data M _H (y) and start address data S _H (y) in the magnification register 203 and the start address register 204 for each vertical address. Further, the read signal is activated for R / W ₁ of the memory 111, the write signal is activated for R / W ₂ of the buffer memory 201, and S of the selectors 211 and 212 selects the output of the memory access unit 200 _H , and the counter 205 and 208 are respectively set to "W" and "h" by the initialization by the computer 107, and the operation is started. At this time, the MSB of the counter 208 becomes H (= “1”), and the selector 209
Is turned to the contact b side, and the output of the counter 205 is set to "0". Then, the start address register 20
The start address data S _H (0) is output corresponding to the address “# 0” of 4 and set in the latch circuit 207 via the selector 209. Then, during this time, the counter 208 is incremented by the pulse of the oscillator 206, and the output thereof becomes “0”. Therefore, the address (S _H (0), 0) is accessed in the memory 111, and the address (0,0) is accessed in the buffer memory 201.
Is accessed.

Here, if S _H (0) is a negative value, the image data is not read from the memory 111, and the data corresponding to the black image data is output. When the counter 208 becomes "0", its MSB becomes "0", and the selector 209 selects the contact a. In the adder 210, the address "#
The reciprocal of the magnification data M _H (0) corresponding to “0” is added to the output S _H (0) of the latch circuit 207, and the result is latched in the latch circuit 207. As a result, the address ( _SH (0) + 1 / _MH (0), 1) is accessed in the memory 111, and the address (0,1) is accessed in the buffer memory 201.

Thereafter, similarly, only the horizontal address is incremented, and the reading of the image data from the memory 111 and the writing of the image data to the buffer memory 201 are continued. When the counter 208 reaches "h", the counter 205 is incremented to "1", the selector 209 selects the contact b, and the start address data S _H
(1) is set in the latch circuit 207. At this time, the address (S _H (1), 0) is accessed in the memory 111,
In the buffer memory 201, the address (1,0) is accessed. Furthermore, the counter 208 is incremented addresses of the memory _{111 (S H (1) +} / 1M H (1), 1) the address of the buffer memory 201 and (1,1) is accessed.

After that, the same operation is repeated, and the address (S _H (h) + W / M _H (h)) of the memory 111 and the address (W, h) of the buffer memory 201 are accessed and the modified image 201A is buffered. It is created in the memory 201. In the end, this horizontal deformation reduces the step size of the horizontal address as the horizontal magnification of the inserted image 111A increases, and decreases the number of times of reading from the outside (negative address) of the memory 111. As the magnification becomes smaller, the step width of the address in the horizontal direction is increased and the operation of increasing the number of times of reading from the outside (negative address) of the memory 111 is performed by using the start address data S _H (y) and the magnification data M _H ( y). As shown in the figure, while the image data is being read so as to extend across l ₀ to l _{W in} FIG. 3 in the horizontal direction, the reading is moved in the direction of the arrow.

Since it is expected that the result of the addition by the adder 210 will not be an integer, the adder 210 is provided with any one of the functions of rounding up, rounding down, and rounding down.

Next, using the image data in the buffer memory 201 obtained in this way, the memory access unit 200 _V performs vertical image transformation. In this case, the memory 111 is associated with the buffer memory 201, the buffer memory 201 is associated with the buffer memory 202, and the horizontal direction and the vertical direction are reversed,
Memory access section 200 _V
It can be configured similarly to the 0 _H internal structure.

The image data of the modified image 202A created by the operation of the modified image creating unit 112 is stored in the buffer memory 202 as shown in FIG. After all, the vertical deformation here is such that the larger the horizontal magnification of the deformed image 201A in the buffer memory 201 is, the smaller the step size of the vertical address is, and the outside (negative address) of the buffer memory 111 is changed. The operation of increasing the number of times of reading from the outside of the buffer memory 202 (negative address) and increasing the step size of the address in the vertical direction as the magnification is decreased
S _V (x) and magnification data M _V (x) are used. As shown in the drawing, the reading is shifted in the arrow direction while the image data is being read so as to extend in the vertical direction from d ₀ to d _h in FIG.

The inserting means 109 has obtained the video signal of the real image 130 of FIG. 1 obtained by being photographed by the television camera 101,
The horizontal synchronizing signal and the vertical synchronizing signal of this video signal are detected, based on these, the image data of the deformed image 202A is read from the buffer memory 202 of the deformed image creating means 112, and the image data of the actual image is obtained by the known superimposing method. And sends the created video signal to, for example, the CRT display device 113. This makes the frame of studio set 108
It is possible to obtain a natural composite image in which the inserted image 111A of the memory 111 of the portion 110 is inserted.

It should be noted that if memory reading is performed as in the present embodiment, thin lines and the like of the inserted image 111A may not appear in the deformed image, but thin lines are rarely used in the inserted image used in television. The present embodiment is particularly effective for simple processing.

〔The invention's effect〕

As described above in detail, according to the image synthesizing apparatus of the present invention, the coordinate data of the four corners on the display screen where the image data of the inserted image should be inserted by the four-corner data calculating means, that is, the reference numeral in FIG. The coordinate data of the four corners (Q _{0 to} Q ₃ ) of the area indicated by the hatching of the image transformed like 404 is calculated. Further, from the relationship between the horizontal and vertical magnifications of the area including the four corners and the area of the memory in which the image data of the inserted image is stored, the start address data and the address increment of the image data are read from the memory. The magnification ratio is obtained, and these are obtained for each address in the horizontal and vertical directions of the memory, and if the image data is read from the memory based on these data, the image data can be obtained in the horizontal and vertical directions. A deformed image as if it was reduced or enlarged is obtained for each of the addresses, and this deformed image is in the shape of a square area including the above four corners. Then, the inserting unit inserts the deformed image into the actual image obtained by the television camera, and the appropriate combination is performed. Therefore, it is possible to obtain a natural composite image even when the television camera is rotated and the zoom magnification is changed.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an image synthesizing apparatus according to an embodiment of the present invention, FIG. 2 is a configuration diagram showing a main part of the embodiment of FIG. 1, and FIG. 3 is related to an embodiment of the present invention. FIG. 4 is a diagram for explaining the operation of the image synthesizing device, and FIG. 4 is an explanatory diagram of the primary perspective transformation. 101 ... Television camera, 103, 104 ... Rotary encoder (first sensor), 105 ... Potentiometer (second sensor), 106 ... Conversion circuit, 107 ... Computer, 108 ... Studio set, 109 ... Insertion means, 111 ... Memory, 112 ... deformed image creating means, 113 ... CRT display device, 200 _H (200 _V ) ... memory access unit, 201,202 ... buffer memory, 203 ... magnification register, 20
4 …… Start address register, 205,208 …… Counter, 207 …… Latch circuit, 209,211-214 …… Selector.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Yasuaki Kintsugu 1-10-11 Kinuta, Setagaya-ku, Tokyo Inside the broadcasting technology research institute of Japan Broadcasting Corporation (72) Masaki Hayashi 1-10-11 Kinuta, Setagaya-ku, Tokyo Issue Broadcasting Technology Institute, Japan Broadcasting Corporation

Claims (2)

(57) [Claims] 1. A first sensor for detecting a rotation angle of a television camera, a second sensor for detecting a zoom magnification of the television camera, and an insertion image to be inserted into an actual image obtained by the television camera. Image data is stored in the memory, and coordinate data of four corners on the display screen in which the image data in the memory is to be inserted into the actual image obtained by the television camera is stored in the first and second sensors. Image data from the memory when inserting the image data in the memory into the image data of the actual image on the display screen from the calculation result of the four-sided data calculation unit based on the output The start address data and the scaling factor data indicating the scaling factor of the address step are read in the horizontal and vertical directions of the memory. A read-out control data calculation unit for each dress, and a deformed image for creating a deformed image by reading out image data in the memory corresponding to the horizontal and vertical directions of the display screen based on the start address data and the magnification data. An image synthesizing apparatus comprising: a creating unit and an inserting unit that inserts the modified image created by the modified image creating unit into an actual image obtained by the television camera. 2. The modified image creating means starts access to the memory from start address data, and for the subsequent address data, data obtained by adding the reciprocal data of the magnification data to the previously used address data is used. The image synthesizing apparatus according to claim 1, wherein.