JPS6324353B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to, for example, video special effects, and relates to an image processing device that obtains an image composed of rectangular pixels (mosaic) from a digitally converted video signal.

Conventionally, in order to obtain a mosaic image made up of square pixels as a video special effect, it has been common to prepare an original picture made up of square pixels and image it with a television camera. In order to turn this image into a moving image that moves in real time, it is necessary to process each frame one by one as in the production of so-called animation, which requires a lot of man-hours and costs. Also,
Because the types of original images are limited, it is not possible to create mosaic images from ordinary photographs or materials as seen with a camera, and there are significant limitations in terms of materials. moreover,
The size of the mosaic unit of the mosaic image is determined by the original image, so it is difficult to compose complex patterns.
It tends to be a repeating pattern with little variation. Moreover, in the case of creating a moving image, the above-described method has the problem that the changes in the image cannot be seen until the image is completed, which often results in unintended results and that the interest as a special effect is lost.

This invention has been made based on the above circumstances, and it reads out image data stored in an image storage section in synchronization with the scanning timing of a television screen, and reconstructs the read out image data to create a mosaic material. This aims to remove the constraints on the size of the mosaic unit and the constraints on making it into a video, and to obtain a practical mosaic special effect. The present invention aims to provide an image processing device that does not generate unnecessary black pixels in a mosaic image by extracting pixels from other parts.

An embodiment of the present invention will be described below with reference to the drawings.

First, the outline of the device will be explained. In FIG. 1, the video input signal V1 is A/D (analog/
(digital) converter 11. This A/
The D converter 11 has a horizontal synchronizing pulse period of 63.5 μs.
192 cycles of color subcarrier included in 52 μs
The sampling period is set to 93 ns so that sampling is performed at three points per cycle, and the video input signal V1 is converted into a digital signal of, for example, 8 bits according to this sampling period. Therefore, there are 576 sampling points per scanning line, and 576 pieces of image data are provided. Hereinafter, one image data will be referred to as one dot. The output signal of this A/D converter 11 is S/P
The signal is supplied to the (serial/parallel) converter 12 .
This S/P converter 12 performs serial/parallel conversion for 3 cycles of color subcarriers, that is, 9 dots at a time, and the conversion output of this S/P converter 12 is 3 cycles in accordance with the control of the write controller 13. One cycle is addressed and stored in the image storage unit 14. The image storage section 14 has a storage capacity for, for example, one field, and the output of the S/P conversion section 12 is stored as shown in FIG. 2.
In FIG. 2, H is the horizontal direction and V is the vertical direction, and 242 addresses are set in the V direction including the blanking period. When image data is read from the image storage section 14, it is done in accordance with address data stored in the address control section 15. This read data is supplied to a P/S (parallel/serial) converter 16. This P/S conversion section 16 is a buffer section for timing adjustment between the image storage section 14 and the subsequent stage, and like the S/P conversion section 12, the number of dots that can be processed at one time is, for example, 9 dots. The conversion output of this P/S converter 16 is supplied to a D/A (digital/analog) converter 17.
The analog-converted video signal is output from the A converter 17. Note that the image data in the image storage section 14 is not lost when read out once, but is retained for at least one field time.

On the other hand, in order to obtain a mosaic image, the screen is divided into, for example, a plurality of mosaic units, and each mosaic unit is composed of predetermined representative data. Figure 3 explains address conversion to obtain a mosaic image, where x and y indicate the size of one unit of mosaic in the horizontal direction H and vertical direction V, respectively. In this case, x = 7, y =5.
One point of representative data is selected for each mosaic unit determined to have this size. For example, if the central part of the mosaic unit is the representative data, the representative data in the mosaic unit M1 is a dot (H4, V3). Now, if the readout scan of the image storage unit 14 comes within the mosaic unit M1, then the representative data (H
4, V3) and successively extract representative data (H4, V3) from this, the mosaic unit M1
can all be converted into representative data (H4, V3). That is, when the address control unit 15 reads out and scans inside the mosaic unit M1, only the V3 line is addressed, and the P/S is determined from the read image data of the V3 line.
The converter 16 may continuously extract only the representative data (H4, V3) under the control of the data controller 18. By converting each mosaic unit in this way, it is possible to obtain a mosaiced video output signal V2 corresponding to the video input signal V1. The address control section 15 and data control section 18 are controlled by a mosaic control section 19, and the values of the sizes x and y of the mosaic unit are given by this mosaic control device 19.

Next, each part will be further explained. First, how to define representative data will be explained.

Various methods can be considered for determining representative data constituting a mosaic unit, but when representative data is determined on the boundary line of a mosaic unit, various drawbacks occur. That is, as shown in Figure 4a, there is a possibility that the data obtained by sampling the blanking level (point B) in the video signal becomes the representative data, and in this case, unnecessary black pixels may appear on the reconstructed mosaic screen. Become. Figure 4b shows this,
When point B in FIG. 14A is representative data, at least the shaded area becomes a black pixel. This black pixel is not noticeable when the mosaic unit is small, but when it is made large, the black pixel area is expanded, and when one screen is composed of one pixel (one representative data), it is always a black pixel. Therefore, in the present application, the representative data is set at a position other than the boundary line of each mosaic unit to prevent the above-mentioned inconvenience. Note that this representative data is the address control unit 1.
5. Set in advance by the data control unit 18.

Next, write control of the image storage section 14 will be explained.

Access to the image storage section 14 on the writing side and access on the reading side are performed alternately.
Therefore, all data in the image storage section 14 is rewritten within a unit time specific to the device. This is no exception to the representative data of each mosaic unit. For this reason, in FIG. 3, representative data (H4,
V3) may be rewritten midway through.
In this case, two pieces of representative data exist within one reconstructed mosaic unit, and a horizontal boundary line occurs within one mosaic unit. Such a phenomenon is not noticeable when the values of the mosaic unit x and y are small, but when the values of x and y are large, the boundary line becomes clear and the screen is divided into two parts, with each mosaic unit being divided into two, above and below the boundary. As a result, data changes are accompanied by a so-called flickering phenomenon, resulting in an unsightly screen. Fifth
The figure explains this, and shows a case where there is a difference of 1H (one V line) between the write side scan and the read side scan, and the read side takes the lead. Here, in the mosaic unit M1, if the representative data (H4, V3) before writing is M1-1 and after writing is M1-2, the upper half of the boundary dividing the mosaic unit into two (indicated by the dotted line) is before writing. Data M1-1 is read out, and rewritten data M1 is read from (H4, V4) one line later.
-2 is read. This is extremely unsightly as it also occurs in other mosaic units.

For this reason, the image storage section 14 has a configuration shown in FIG. 6. That is, 14a and 14b are storage sections each having a storage capacity for one field, and the duplicated storage sections 14a and 14b are provided with an input changeover switch 14c and an output changeover switch 14d. This switch 14
c and 14d alternately store the storage units 14a and 14, respectively.
b can be selected by switching. Therefore, the storage section that is being written can be made write-only;
Since the storage section being read out can be made read-only, the above-mentioned inconvenience does not occur, and one mosaic unit is composed only of the relevant representative data.

Next, the representative data will be further explained. The representative data mentioned above is composed of one dot. Therefore, if the representative data is affected by noise, it will immediately appear in the mosaic and be difficult to see. This is not so much of a problem when the mosaic unit is small, but when the mosaic unit becomes large, the pixel area is also enlarged, so the effect is magnified, which is inconvenient. Therefore, in practice, a plurality of dots are selected within one mosaic unit, the average value of the data values of each dot is determined, and this is used as representative data of the corresponding mosaic unit. Figure 7 shows an example.
Adjacent (H4, V2), (H3, V3), (H4,
This is a case where five dots, V3), (H5, V3), and (H4, V4) are selected. At this time, if the data values of each dot are A, B, C, D, and E, then the representative data of this mosaic unit is (A+B+C+D
+E)/5. In order to obtain the average value of a plurality of dots in this way, the P/S conversion section 16 and the D/A
A data averaging circuit may be provided between the converter 17 and the converter 17. By averaging a plurality of dots to obtain representative data in this way, it is possible to suppress the influence of noise.

Next, a case will be described in which the sizes x and y of the mosaic unit are arbitrarily changed. Here, the address control section 15, data control section 18, and mosaic control section 19 will be further explained.

The mosaic control section 19 specifies the size of a mosaic unit using, for example, a digital signal of 1 to 31, and the output signal of this mosaic control section 19 is supplied to the address control section 15 and the data control section 18. This address control unit 15 has horizontal direction H and vertical direction V corresponding to the size of the mosaic unit.
Each of them is provided with 1 to 31 storage units, such as PROM (programmable read-only memory). Of these PROMs, those corresponding to the horizontal direction H are, for example, 1 as shown in Figure 8a.
It has a ~64 address configuration. This is because, as described above, the access unit of the image storage unit 14 is three cycles of the color subcarrier, and is defined by the relationship: 576 dots/9 dots=64 dots. Also, in the PROM, vertical direction V
As shown in FIG. 8b, the addresses corresponding to , respectively, have, for example, 1 to 242 addresses. This matches the address structure of the image storage section 14. And PROM in horizontal direction H and vertical direction V
Each has an image storage unit 1 corresponding to the mosaic unit.
4 access address data are stored in advance.

Further, the data control unit 18 is provided with 1 to 31 storage units, such as PROM, corresponding to the size of the mosaic unit. As shown in Figure 9, this PROM corresponds to the number of dots in the horizontal direction.
It has a configuration of 576 addresses, and control data for configuring one mosaic unit with the minimum representative data (for example, 3 dots) is stored in advance.

Accordingly, the address control section 15 and the data control section 18 select corresponding PROMs according to the output signals of the mosaic control section 19, respectively.
That is, for example, if the output signal of the mosaic control section 19 is "1", PROM1 of the address control section 15 and the data control section 18 is selected. and,
The address control section 15 accesses the image storage section 14 using the data stored in the selected PROM, and the data control section 18 controls the P/S conversion section 16 according to the data stored in the PROM.

The operation in the above configuration will be explained. Now, assuming that the digital signal "1" is output as the output signal of the mosaic control unit 19, in this case,
Assume that the mosaic unit is composed of x=7 dots and y=5 dots as shown in FIG.
Furthermore, the number of dots that the P/S converter 16 can process at one time is P=9 dots, and in FIG. 10, P ₁ ,
P ₂ , P ₃ . . . indicate the order of P/S conversion.
In this state, first, PROM1 is selected in each of the address control section 15 and the data control section 18 by the output signal of the mosaic control section 19. This PROM 1 stores, for example, address data shown in FIGS. 8a and 8b and FIG. 9, and these data are accessed at respective scanning timings. Then, the image storage section 14 is read out according to the data stored in the PROM 1 of the address control section 15. That is, when the scanning timing is, for example, mosaic unit M1, the _P1 portion (H
1, V3) to (H9, V3) are read out and supplied to the P/S converter 16. This P/
The S converter 16 is provided with a plurality of buffer registers 16a ₁ to 16an, as shown in FIG _. is supplied to the register _16a1 , for example. Representative data is repeatedly taken out from this register _16a1 by the selection circuit 16c. This selection circuit 16c is controlled by the output signal P _R of the data control section 18. That is, the registers 16a ₁ to 16an and the selection circuit 16c are
It is as shown in the figure. In addition, Figure 12 is the first
This shows a part of Figure 1. In this diagram,
The register _16a1 is a so-called circular shift register, and the data in 16d is circularly shifted to 16e. This register _16a1 sequentially holds the data corresponding to the H direction of the V3 line.
In order to extract the representative data corresponding to the scanning of the mosaic unit M1, the switch 16g of the selection circuit 16c is first turned on. Therefore, H
The data of 4, H5, and H6 are taken out from the switch 16g according to the shift operation, and then the switches 16f and 16h are sequentially turned on, so that the representative data (H4, V3), (H5, V3),
(H6, V3) is extracted and the mosaic unit M1
(H1, V1) to (H7, V1) are converted into representative data.

Next, when scanning moves to mosaic unit M2,
While the image data of the _P1 portion is held in the register _16a1 , the _P2 portion shown in FIG. ₁₀ is stored in the register 16a2.
Image data of portions (H10, V3) to (H18, V3) are read out. And register 16a ₂
As described above, representative data (H11, V3), (H
12, V3) and (H13, V3) are repeatedly read out, and (H8, V1) to (H
15, V1) is converted into the representative data. Thereafter, such operations are sequentially performed to convert each mosaic unit into corresponding representative data.

Next, it is assumed that a signal "n" is output from the mosaic control circuit 19, and the mosaic unit at this time is, for example, x=11 and y=7 as shown by the dotted line in FIG. In this state, first, if the scanning timing is mosaic unit M1, representative data of P1 part (H5, V4), (H6, V4), (H7, V
Image data of (H1, V4) to (H9, V4) including 4) is read out and supplied to the register _16a1 . From this register _16a1 , the operation of the selection circuit 16c described above is controlled by the data control section 18.
This is performed according to the data of PROMn, and (H1, V1) to (H9, V1) of mosaic unit M1 are converted into representative data. In this state, the image storage unit 14 outputs the P2 portion, that is, (H10, V4) to
(H18, V4) is read out and held in the register _16a2 . Then, (H1
0, V1), (H11, V1) are converted to representative data held in register 16a ₁ , and (H12, V1) to (H18, V1) of mosaic unit M2 are converted to representative data (H16, V4), (H17 , V4). In this state, from the image storage unit 14
The P3 portion is read out and this image data is held in the register 16a ₁ or 16a ₃ . Then, (H19, V1) to (H
22, V1) is converted into representative data held in the register _16a2 . Thereafter, such operations are performed for each mosaic unit, and each mosaic unit is converted into corresponding representative data.
Incidentally, the number of registers is actually 2 to 3, which can cover most sizes of mosaic units, and data is taken out from these registers as appropriate by the data control section 18.

As described above, by providing a plurality of registers in the present application, the size of the mosaic unit can be determined regardless of the number of data P that the P/S converter 16 can process at once. That is, for example, when there is one register, the constraint condition holds between P and the mosaic unit x in the H direction that when P>x, P/x=integer, and when P<x, x/P=integer. do. Therefore, if the mosaic unit described above is x = 11 dots and y = 7 dots, when scanning H10 and H11 of mosaic unit M1, it will be converted to representative data of mosaic unit M2, and the mosaic unit It becomes impossible to freely vary the size of the . Therefore, as described above, in the present application, a plurality of registers are provided to make the size of the mosaic unit variable without being restricted by the number P of data that the P/S converter 16 can process at one time.

Next, the mosaic control section 19 will be further explained. As described above, the mosaic control section 19 selects and controls the size of the mosaic unit. In order to make a mosaic image highly practical as a special effect, it is desired that the size of the mosaic unit can be easily varied. Therefore, it is conceivable to use, for example, a fader and use the position information of the fader for this control.

FIG. 13 shows an example of this, in which position information of the fader 20 is supplied to the mosaic control section 19. This position information is A/D converted in the mosaic control section 19, and the digital signals "1" to "31" described above are generated.
Therefore, “1” depending on the position of the feeder 20.
~ “31” digital signal is sent to mosaic control unit 19
The signal is then supplied to the address control section 15 and data control section 18, and these control sections 15 and 18 select PROM1 to PROM31 corresponding to the digital signal. A mosaic image is then generated using the data stored in this PROM.

Further, by using the fader 20, it is possible to obtain a wide variety of special effects. For example, the screen
When changing the screen from P _A to P _B , first,
A screen P _A without mosaic conversion is displayed, and in this state, the fader 20 is operated to convert the screen P _A into, for example, the smallest mosaic image. Then, operate the fader 20 to gradually increase the size of this mosaic unit, and display the screen at the maximum mosaic unit.
Switch from P _A to P _B. At this time, screen P _B is the largest mosaic unit. From this state, the fader 20 is further operated to gradually make the mosaic unit smaller, and finally an image P _B that is not subjected to mosaic conversion is obtained.

In this way, by varying the size of the mosaic unit using the fader 20, it is possible to obtain a wide variety of special effects. In FIG. 13, the same parts as in FIG. 1 are given the same reference numerals.

Next, the write control section 13 will be further explained.

When generating a mosaic image, it is desirable that the representative data have values as stable as possible. For example, when converting a still image or an image similar to a still image with little movement into a mosaic image, even if the values of the representative data differ even slightly, the image will flicker on the playback screen because the pixels are larger. This is due to the nature of the NTSC system. That is, in the NTSC system, the frequency of the color subcarrier is an odd multiple (455/2 times) of 1/2 the line frequency. Therefore, the phase of the color subcarrier for each line differs by 180°. Furthermore, since the number of lines in each frame is 525, which is an odd number, the phases of the color subcarriers at the starting points of consecutive frames also differ by 180°. Therefore, the color subcarriers are in phase once every four fields. FIG. 14 shows the movement of sample points caused by the four-field sequence of the NTSC system, and shows that the writing start point (arrow in the figure) differs from field to field. Figure 15 is the first
4 shows the movement of pixel data contents derived from FIG. As is clear from the figure, even data on the same line has different contents for each field. For example, the data of line number 1 in frame 1 and the data of line number 246 in a different field have a horizontal position shift of 140 ns and a vertical position shift of 1/2 line.
This relationship is the same for frame 2, and 4
If the sequence of fields is considered, the data contents will move as shown by the arrows in FIG. Therefore, the same data is divided into 1 in 4 fields.
You can only get times. For this reason, in mosaic conversion of a signal close to a still image, using data that is updated every field will result in an image with a lot of flickering, which is inconvenient.

Therefore, in this application, the writing cycle is changed so as to be compatible with still images. That is, when obtaining a mosaic image, writing is performed once in four fields. This switching control is performed, for example, by a control signal supplied from the mosaic control section 19 to the write control section 13. In this way, a flicker-free mosaic image can be obtained even in mosaic conversion of a still image or an image close to a still image.

According to the above configuration, the video input signal V1 is digitally converted and stored in the image storage section 14, and in the process of reading out this stored data, it is reconstructed according to the stored data of the address conversion section 18 and the data control section 19, and a mosaic image is created. is being generated. Therefore, it is possible to convert any image, whether it is a moving image or a still image, into a mosaic image, and the number of man-hours required for this conversion is extremely small compared to the conventional method. Furthermore, since this mosaic conversion is instantaneous, it is possible to immediately see how the image changes, which is extremely useful for obtaining special video effects.

Further, the representative data constituting the mosaic unit is set, for example, at the center, which is spaced apart from the boundary of the mosaic unit. Therefore, unnecessary black pixels do not appear in the generated mosaic image, which is convenient.

Further, since the representative data is an average value of a plurality of pixels, there is an advantage that even if noise is mixed in the pixels, the influence of noise is reduced and image quality deterioration can be suppressed.

Further, the image storage unit 14 is the same storage unit 14a,
14b is provided, which is connected to the input selector switch 14.
c. By alternately switching with the output changeover switch 14d, the storage sections 14a and 14b are made write-only and read-only, respectively. Therefore, since no data is written to the storage unit that is in the read state, one mosaic unit is
It is possible to prevent inconveniences caused by individual representative data, and it is possible to obtain a flicker-free mosaic image.

Further, the P/S converter 16 is provided with a plurality of registers 16a, each of which stores different read data. Therefore, the size of the mosaic unit P/S converter 16 can handle the number of data at one time P
can be determined independently.

Further, the size of the mosaic unit is preset in a plurality of PROMs provided in the address control section 15, and is selected as desired by the output signal of the mosaic control section 19. Therefore, the ratio of the H direction and V direction of the mosaic unit is
It can be easily changed by changing the data stored in PROM.

Further, the size of the mosaic unit can be selected by the mosaic control section 19, and this selection control is performed based on the position information of the fader 20. Therefore, by operating the fader 20, it is possible to obtain a mosaic image rich in variation suitable for screen switching, etc.

Furthermore, writing control for the image storage unit 14 is such that writing is performed for each field when the original image is reproduced.
When obtaining a mosaic image, writing is performed once in four fields. Therefore, the representative data does not change even for still images, and a stable mosaic image can be obtained.

As described in detail above, according to the present invention, the image data stored in the image storage unit is read out in synchronization with the scanning timing of the television screen, and the read image data is configured into an image to create a mosaic material. It is possible to remove the restrictions regarding the size of the mosaic unit and the restrictions on making it into a video, and it is possible to obtain practical mosaic special effects. By extracting the pixels, it is possible to provide an image processing device that does not generate unnecessary black pixels in the mosaic image.

[Brief explanation of the drawing]

The drawings show an embodiment of the image processing device according to the present invention, and FIG. 1 is a configuration diagram, FIG. 2 is a diagram for explaining the image storage section in FIG. 1, and FIG. 3 is a mosaic image. Figures 4a and 4b are diagrams shown to explain the state in which the representative data is set as the boundary of the mosaic unit, respectively. Figure 5 is the diagram in which the representative data is selected at the center of the mosaic and the image is stored in the image storage unit. FIG. 6 is a diagram showing a schematic configuration of an image storage unit; FIG. 7 is a diagram showing representative data composed of a plurality of pixels; Figure 8a,
b is a diagram shown to explain the address control section, FIG. 9 is a diagram shown to explain the data control section, FIG. 10 is a diagram shown to explain address conversion to obtain a mosaic image, and FIG. Figure 11 shows the configuration of the P/S converter, and Figure 12 shows the configuration of the P/S converter.
13 is a configuration diagram showing the configuration of FIG. 1 with a fader added, and FIGS. 14a and 14b explain the NTSC signal and the writing timing of the image storage section. FIG. 15 is a diagram shown for explaining the movement status of data contents. 11...A/D conversion unit, 12...S/P conversion unit, 13...Writing control unit, 14...Image storage unit, 15...Address control unit, 16...P/S conversion unit, 17... ...D/A converter, 18...data control section, 19...mosaic control section.

Claims (1)

[Claims] 1. An image storage section in which image data is stored, a means for controlling readout of this storage section in synchronization with scanning timing and specifying the readout address for each mosaic according to the size of the mosaic, and a representative of the mosaic described above. means for selecting data from a portion other than the boundary of the mosaic; the image data for each mosaic read from the image storage section is held in register means for parallel-to-serial conversion; An image processing apparatus characterized in that the image data is replaced with the representative data selected for each mosaic, and the replaced image data is used as new mosaic image data.