JP3022334B2 - Image generation device, moving image decompression mapping device, and multimedia device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention generates computer graphics (hereinafter, referred to as "CG") and decompresses a high-efficiency coded digital moving image (hereinafter, referred to as "compressed moving image decompression"). The present invention relates to an image generation device. More specifically, there is provided an image generation apparatus which realizes real-time generation of CG by a texture mapping method and decompression of a digital moving image which has been encoded with high efficiency using inter-frame motion compensation, with a single configuration apparatus, and an image generation apparatus. The present invention relates to a moving image expansion mapping device and a multimedia device used.

[0002]

2. Description of the Related Art In recent years, home TV game machines, personal computers, workstations and the like have
In order to generate a CG to be displayed and output on an RT display or the like, image data (hereinafter, referred to as a “texture image”) prepared in advance is converted into a polygon area (hereinafter, “polygon”).
, A “texture mapping method” may be used. The use of the texture mapping method has an effect that when drawing a complicated three-dimensional three-dimensional shape, a relatively high-quality image can be obtained even when approximation drawing is performed using a small number of polygons.

[0003] On the other hand, in the case of displaying and outputting moving image data on a CRT display or the like in a personal computer, an audiovisual device, or the like, the moving image data which has been compressed in advance is displayed while being expanded in real time. The use of compressed moving image data has the effect of reducing the capacity of the storage medium and the amount of data transmission.

As one of the techniques for efficiently compressing moving picture data, there is a so-called inter-frame predictive coding technique in which only difference data of image data is encoded over a plurality of frames. For inter-frame prediction coding, "forward prediction coding"
There are “reverse prediction coding” and “bidirectional prediction coding”.

[0005] In an input image sequence, an image to be compressed is an "input image", an image temporally before the input image is a "past image", and an image later than the input image is a "future image". Then, forward prediction encoding encodes difference data between an input image and a past image, and backward prediction encoding
The difference data between the input image and the future image is encoded, and the bidirectional predictive encoding is an encoding using both the past image and the future image. Note that the past image and the future image are collectively referred to as a reference image.

Further, motion prediction processing is performed to increase the compression ratio. The motion prediction process divides an input image into image blocks of a certain size (for example, 16 × 16 pixels) when calculating difference data, as well as differences at the same position in an image. In this method, a position where the difference is minimized is obtained by shifting in the horizontal and vertical directions in the reference image to be processed, and the block position at that time and the difference data are encoded. In this method, “forward prediction”, “reverse prediction”, “bidirectional prediction”, or “no predictive coding” is adaptively performed for each image block so that the compression efficiency is highest. It is switched and encoded. Further, the position of the reference image block may be obtained not only with one-pixel accuracy but also with half-pixel accuracy.

[0007] In recent years, there is a method of using a compression-encoded moving image as a texture image prepared in advance in a texture mapping method. That is, this is a method of mapping the expanded moving image on the polygon. An example of a conventional image generating apparatus that realizes this method is disclosed in Japanese Patent Application Laid-Open No. Hei 6-162165.

In this conventional example, there is provided an image processing method capable of using a moving image as a texture image instead of a still image such as a geometric pattern, a photograph, or a handwritten design which has been conventionally used as a texture image. As a result, a variety of CG images can be generated. The configuration of this conventional example includes a data reproducing unit for reproducing compressed moving image data from an information recording medium, an image data expanding unit for expanding compressed moving image data reproduced from the data reproducing unit, and an expanding unit for expanding the image data expanding unit. Image synthesizing means for receiving the obtained image data via the bus line and synthesizing the image data.

[0009]

However, the configuration of the above conventional example has the following problems.

First, the configuration of a conventional image generating apparatus will be described with reference to FIG. In FIG. 27, system bus 2
Reference numeral 401 denotes a data transmission medium between blocks. CPU2
402 controls the entire system. The main memory 2403 stores programs and data. The image expansion circuit 2404 expands the compressed moving image data. A CD-ROM decoder 2405 reproduces data from a CD-ROM 2409 as an information recording medium. The image synthesis circuit 2406 synthesizes the expanded moving image data on the frame memory 2410 by texture mapping. The coordinate conversion circuit 2407 obtains coordinates for image synthesis. The DMA controller 2408 performs data transfer on the system bus. The CD-ROM 2409 is an information storage medium for storing image data and the like. The frame memory 2410 is a working memory for the image composition processing. The D / A conversion circuit 2412 performs D / A conversion on the signal to display the result of image synthesis.

Next, the operation of the image generating apparatus will be described. CD
-Compressed video data stored in ROM2409 is CD-R
The data is read and reproduced by the OM decoder 2405. The reproduced compressed moving image data is transferred to the image decompression circuit 2404 via the system bus 2401. In the image expansion circuit 2404, the compressed moving image data is expanded. The expanded moving image data is transferred again to the image synthesizing circuit 2406 via the system bus 2401. On the other hand, the drawing coordinates are determined by the CPU 2402 and the coordinate conversion circuit 2407.
Image synthesis circuit 24 via the system bus 2401
Instructed at 06. In the image synthesis circuit 2406, an image decompression circuit
The moving image data transferred from 2404 is synthesized with the coordinates on the frame memory 2410 specified by the CPU 2402 by texture mapping. The image synthesized on the frame memory 2410 is read out by the image synthesis circuit 2406 and is read by the DA conversion circuit.
Transferred to 2412. The DA conversion circuit 2412 converts a digital signal into an analog signal and outputs the analog signal.

FIG. 40 shows an image decompression circuit shown in FIG.
2 shows the internal configuration of 2404. Hereinafter, the operation of the motion compensation processing will be described with reference to FIG.

The compressed moving image data read from the CD-ROM decoder is input to the image expansion circuit 2404 via the system bus 2401 in units of, for example, 16 × 16 pixel image blocks. The compressed image block data input to the image decompression circuit 2404 is first converted into an orthogonal transform coefficient image 3002 by the variable-length inverse encoding circuit 3001. Subsequently, the image data is converted into a difference image block by the orthogonal transformation circuit 3003 and stored in the difference image memory 3004. The compressed image block data input to the image decompression circuit 2404 includes not only difference image components but also motion compensation data necessary for decoding this image block. The motion compensation data is the type of motion prediction processing used when this image block is encoded (that is, “forward prediction”, “reverse prediction”, “bidirectional prediction”, or “ No.), and data indicating the position of a reference image block used for prediction. Based on the content of the motion compensation data, a motion compensation circuit 3013 performs motion compensation processing. From the reference image memory 3012, an image block at a position corresponding to the data indicating the position of the reference image block is stored in the image reading circuit 300.
7, read through 3008. For example, if the image block to be converted is a block coded by forward prediction, the average / selection circuit 3009 uses the past image memory
The image block read from 3005 is selected and input to the adding circuit 3010. In the addition circuit 3010, the past image block data and the difference image block data are added,
This is written into the generated image memory 3011. For an image block encoded by backward prediction, the future image memory 3006
Are selected and added to the difference image block. If the image block to be converted is a block coded by bidirectional prediction, the averaging / selection circuit 3009 determines whether the image block read from the past image memory 3005 and the image block read from the future image memory 3006 are to be converted. The average value for each pixel is calculated, and the result is sent to the addition circuit 3010, added to the difference image block, and stored in the generated image memory 3011. When the data indicating the position of the reference image block is given with 1/2 pixel precision, the image block reading circuits 3007 and 3008
It has a reading function corresponding to half-pixel accuracy, such as reading an image block while calculating the average value of two pixels. The expanded moving image data stored in the generated image memory 3011 is transferred to the image synthesizing circuit via the system bus 2401. In addition, the contents of the generated image memory 3011 are also transferred to the past image memory 3005 or the future image memory 3006 in the reference image memory 3012 for use in subsequent moving image expansion. The above is the configuration inside the conventional image decompression circuit, and the operation of the motion compensation processing.

Next, the operation of the conventional texture mapping process performed in the image synthesizing circuit 2406 and the frame memory 2410 shown in FIG. 27 will be described with reference to FIG.

The moving image data transferred to the image composition circuit 2406 is temporarily stored in the frame memory 2410 as texture image data 3101 when the image composition circuit performs a texture mapping operation. Then, the texture image 3101 is read by the image synthesizing circuit, image-deformed, and stored (mapped) in the generated image 3102 area. One pixel 3104 in the texture image 3101 is input to a first multiplication circuit 3107 in the texture mapping circuit 3106. At the same time, the pixel 3105 in the generated image area 3102 to which this pixel is mapped is input to the second multiplication circuit 3108. In the first multiplication circuit 3107, a multiplier α (where α is a numerical value of 0 or more and 1 or less) is multiplied. In the second multiplication circuit 3108, a multiplier (1-α) is multiplied. The signal is input to the circuit 3109 and added. This result is stored at the position of the pixel 3105 in the generated image 3102. By sequentially performing the above operation for all the pixels of the texture image, a texture mapping process involving translucent synthesis of the original image (which is generally a background image) in the generated image and the texture image is realized. You. The above is the operation of the texture mapping process performed by the conventional image synthesis circuit.

In the above-described image generating apparatus, the compressed moving image data is transferred to the image decompressing circuit 2404 via the system bus 2401 in order to decompress the moving image, and the decompressed moving image data is again transferred via the system bus 2401 to the image. The data is transferred to the combining circuit 2406. The data amount of the expanded moving image data is generally very large (for a moving image of 320 × 240 pixels, about 7M
B / sec), which places a heavy load on the data transfer capability of the system bus 2401. In addition, when performing moving picture compression with high efficiency, an inter-frame motion prediction technique is often used, and it is necessary to provide a frame memory in order to expand a moving picture compressed based on this technique. That is, if the image decompression circuit 2404 and the image synthesis circuit 2406 are provided separately, the frame memory for the image decompression processing and the frame memory 2410 for drawing the image must be separated. Become. Further, since the image decompression circuit 2404 and the image synthesizing circuit 2406 are connected via the system bus 2401, control complexity such as timing control of data transfer and the system configuration become complicated.

The conventional image generating apparatus has many problems as described above.

SUMMARY OF THE INVENTION The present invention has been made in view of such circumstances, and has as its object to expand a digital moving image encoded using inter-frame motion prediction,
By realizing generation of a CG image by a texture mapping method with a single configuration, simplification of the device configuration,
It is an object of the present invention to provide an image generation device capable of realizing a reduction in the circuit scale of a processing device, a reduction in the amount of memory required for processing, and a reduction in the amount of data transfer by eliminating the need for transferring moving image data between memories.

[0019]

An image generating apparatus according to the present invention has at least a motion compensation process or a texture mapping process.
Command signal instructing processing including
A signal indicating the corresponding first pixel position,
And a signal indicating the third pixel position.
Signal, and the command signal indicates a motion compensation process.
The pixel position in the reference image block
And the pixel position in the generated image block
The position of the pixel in the difference image block is defined as the second pixel position.
Position is the third pixel position, and the command signal is
When the mapping process is instructed, the mapping
The position of the pixel in the polygon to be
And the position of the pixel in the texture image as the second pixel position.
A pixel position specifying means for setting the position to the third pixel position;
Receiving a signal indicating the pixel position of the first pixel;
Is output as the first pixel data, and the third image
Receiving a signal indicating the elementary position, and selecting a pixel at the third pixel position
The value is output as third pixel data, and the second pixel position
And the second pixel data, and the second pixel data
An image memory for writing the second pixel data at a pixel position;
Access means, the first pixel data and the third pixel data.
With the second pixel data.
Pixel operation means for outputting the data as
The position specifying means detects that the command signal has a motion compensation process and text.
When instructing simultaneous texture mapping processing
The position of the pixel in the reference image block is referred to as the first pixel
Position and within the generated polygon to be texture mapped
The position of the pixel is set as the second pixel position, and the difference image block is set.
The pixel position in the block is output as the position of the third pixel, thereby achieving the above object.

[0020]

When the command signal indicates a motion compensation process, the second pixel position and the third pixel position may be the same.

When the command signal indicates a motion compensation process, the second pixel position and the third pixel position may be different.

[0023]

Preferably, the pixel operation means receives the command signal, and when the command signal indicates a motion compensation process, or when the command signal indicates a simultaneous process of a motion compensation process and a texture mapping process, The first pixel data and the third pixel data are arithmetically processed as signed precision data, and when the command signal indicates texture mapping processing, the first pixel data and the third pixel data are processed.
And the third pixel data are processed as unsigned precision data.

Preferably, the pixel position specifying means receives the command signal, and when the command signal indicates a motion compensation process, a relative position of the first pixel position in the reference image block. And a pixel position at which the relative position of the third pixel position in the difference image block is the same as the relative position of the second pixel position in the generated image block.

Preferably, the pixel position designation means receives the command signal, and when the command signal indicates a texture mapping process, a pixel position of each pixel on a horizontal line in the generated polygon. Are sequentially calculated, and the pixel position is determined by the first pixel position and the second pixel position.
The pixel position in the texture image corresponding to the first pixel position and the second pixel position is determined, and the pixel position is specified as the third pixel position, and the command When the signal indicates a motion compensation process, the pixel position of each pixel on the horizontal line in the generated image block is sequentially calculated, and the pixel position is designated as the second pixel position. The relative position of a third pixel position in the difference image block and the relative position of the first pixel position in the reference image block are the same as the relative position of the second pixel position in the generated image block The third pixel position and the first pixel position are specified as follows.

The pixel position specifying means receives the command signal, and when the command signal indicates a texture mapping process, sequentially calculates a pixel position of each pixel on a horizontal line in the texture image. Then, the pixel position is designated as the third pixel position, a pixel position in the generated polygon corresponding to the third pixel position is determined, and this pixel position is determined by the first pixel position and the second pixel position.
When the command signal indicates a motion compensation process, the pixel position of each pixel on the horizontal line in the difference image block is sequentially calculated, and the pixel position is designated as the 3 as a pixel position, and
The relative position of the pixel position in the reference image block and the relative position of the second pixel position in the generated image block are the same as the relative position of the third pixel position in the difference image block. The first pixel position and the second pixel position may be specified.

Preferably, the pixel position designating means receives the command signal, and when the command signal indicates that a motion compensation process and a texture mapping process are to be performed simultaneously, , The pixel position of each pixel on the horizontal line is sequentially calculated, and this pixel position is designated as the second pixel position.
A pixel position to be mapped to the pixel data of the pixel position is determined in the difference image block, and this pixel position is designated as the third pixel position, and the relative position of the third pixel position in the difference image block is determined. And the first pixel position is specified such that the relative position of the first pixel position in the reference image block is the same as that of the first pixel position.

The pixel position designating means receives the command signal, and when the command signal indicates that the motion compensation process and the texture mapping process are to be performed simultaneously, the horizontal position in the difference image block is set. The pixel position of each pixel on the line is sequentially calculated, this pixel position is designated as the third pixel position, and the pixel position in the generated polygon corresponding to the third pixel position is obtained. The first pixel position is designated as the second pixel position, and the relative position of the third pixel position in the difference image block is the same as the relative position of the first pixel position in the reference image block. The pixel position may be specified.

Preferably, the pixel operation means receives the command signal, and when the command signal indicates a motion compensation process, the first pixel position in the reference image block from the first pixel position. Is read, the third pixel data is read from the third pixel position in the difference image block, and an operation is performed using the first pixel data and the third pixel data. After performing a block motion compensation process of performing the procedure of writing the second pixel data to the second pixel position indicating the pixel position in the generated image block for all the pixels in one image block, The pixel position specifying means sets the generated image block as a new difference image block, and sets an image block located at a position shifted by one pixel from the reference image block as a new reference image block. Performing image block setting process for specifying a pixel position as a lock.

[0031] The pixel position designating means may perform a pixel block setting process in which an image block shifted by one pixel to the right with respect to the reference image block is set as the new reference image block.

[0032] The pixel position specifying means may perform a pixel block setting process in which an image block shifted downward by one pixel from the reference image block is set as the new reference image block.

The pixel position specifying means performs an image block setting process in which the generated image block is set as a new difference image block, and an image block located at a position shifted by one pixel from the reference image block is set as a new reference image block. The direction of the shift of the reference image block may be different for each image block setting process in the image block setting process repeated twice or more times.

In the case where the pixel position specifying means performs the image block setting processing two or more times, an image block shifted by one pixel to the right with respect to the reference image block is set as the new reference image block. And a case where an image block shifted by one pixel downward with respect to the reference image block is set as the new reference image block. The new reference image block may be used.

Another image generating apparatus according to the present invention comprises at least
Includes motion compensation or texture mapping
Receives command signals for processing and responds to each
Signal indicating the first pixel position, indicating the second pixel position
And outputting a signal indicating a third pixel position.
When the command signal indicates motion compensation processing,
The position of the pixel in the quasi-image block is defined as the first pixel position.
Then, the position of the pixel in the generated image block is referred to as the second pixel position.
And the position of the pixel in the difference image block is
And the command signal is used for texture mapping.
Polygons to be mapped when
The position of the pixel within the first pixel position and the second pixel position
The position of the pixel in the texture image as the third image.
A pixel position specifying means to be a raw position, and the first pixel position
Indicating that the pixel value at the first pixel position is the first pixel value.
And outputs a signal indicating the third pixel position.
And the pixel value at the third pixel position is
And outputs a signal indicating the second pixel position and
And the second pixel data, and the second pixel data is received at the second pixel position.
Image memory access means for writing pixel data of
Operate between the first pixel data and the third pixel data
And outputs the operation result as the second pixel data
Comprising a pixel calculation unit that, the, after performing block motion compensation process is a motion compensation processing for one image block, the pixel position specifying means, and the texture image to generate image blocks generated by the block motion compensation process set, also performs image block setting processing for setting the newly generated polygons, the image memory access means, said generating polygons within said first pixel said in the texture image reading said first pixel data from the position reading said third pixel data from said third pixel positions, said pixel calculation means,
Calculated by using the pixel data of the first pixel data and the third, a process of writing the second pixel data which is the result in the second pixel position in the generated polygons, these processes Is sequentially performed on the pixel data in the generated polygon. Thereby, the above object is achieved.

Still another image generating apparatus according to the present invention is a control means for outputting a command signal for instructing contents to be processed by the image generating apparatus, receiving the command signal, and the command signal instructing a motion compensation process. The pixel position in the reference image block for motion compensation is calculated and output as a first pixel position signal, and the pixel position in the generated image block is calculated and output as a second pixel position signal. A pixel position in a difference image block which is difference data of the generated image block is obtained and output as a third pixel position signal. When the command signal indicates a texture mapping process, a pixel position in the texture image is determined. Is obtained as the third pixel position signal, and the image data at the pixel position indicated by the third pixel position signal is mapped and written. A pixel position in the generated polygon to be processed is output as the first pixel position signal and the second pixel position signal, and when the command signal indicates an image block setting process and an image block position, A pixel position specifying means for newly setting the position of the reference image block, the generated image block, the difference image block, or the generated polygon, the texture image; the first pixel position signal and the second pixel position A signal and the third pixel position signal are input, pixel data is read out from a pixel position indicated by the first pixel position signal, and output as a first pixel data signal, which is indicated by the third pixel position signal. The pixel data is read from the pixel position to be output and output as a third pixel data signal, and is input to the pixel position indicated by the second pixel position signal by the second pixel data signal. An image memory access unit for writing the input pixel data, and the command signal instructs a motion compensation process for the input pixel data indicated by the first pixel data signal and the third pixel data signal. When the command signal indicates the texture mapping process, the pixel data is processed as an unsigned data if the command signal indicates the texture mapping process. Performs the operation indicated by the command signal in
A pixel operation means for outputting an operation result as the second pixel data signal, wherein the control means receives a control command of an image generation operation, interprets the control command, and executes the pixel position designation means, the pixel operation The command signal for instructing the processing contents is output to the means, whereby the above object is achieved.

Preferably, the image memory access means inputs a first pixel position signal and a second pixel position signal, reads pixel data from a pixel position indicated by the first pixel position signal, and A first image memory access unit that outputs the pixel data as one pixel data signal and writes the pixel data input by the second pixel data signal at the pixel position indicated by the second pixel position signal; A second image memory access unit for inputting a pixel position signal, reading pixel data from a pixel position indicated by the third pixel position signal, and outputting the read pixel data as a third pixel data signal; A block image memory that stores a difference image block and outputs pixel data from a pixel position designated by the second image memory access unit; I have.

Preferably, the image memory access means inputs a first pixel position signal, a second pixel position signal, and a fourth pixel position signal, and outputs a pixel position indicated by the first pixel position signal. The pixel data is read out from the pixel position indicated by the fourth pixel position signal and output as a first pixel data signal. The pixel data is read out from the pixel position indicated by the fourth pixel position signal and output as a fourth pixel data signal. A first image memory access unit for writing the pixel data input by the second pixel data signal to the indicated pixel position; a third pixel position signal and a fifth pixel position signal; The pixel data is read from the pixel position indicated by the pixel position signal and output as a third pixel data signal, and the fifth pixel data signal is output to the pixel position indicated by the fifth pixel position signal. And a second image memory access unit for writing the pixel data inputted by the second image memory unit. The texture image or the difference image block is stored in the second image memory access unit. A block image memory for inputting and outputting data, receiving the command signal, and when the command signal indicates an image block setting process and an image block position, a pixel read position in the difference image block or the texture image Are sequentially output as the fourth pixel position signal, the pixel writing positions in the block image memory are sequentially output as the fifth pixel position signal, and the pixel data input as the fourth pixel data signal is By outputting as the fifth pixel data signal, the difference image block or the And an image moving means for moving the tea image on the block image memory.

Preferably, the image memory access means receives a first pixel position signal and a second pixel position signal, and reads out pixel data from a pixel position indicated by the first pixel position signal. A first image memory access unit that outputs the pixel data as one pixel data signal and writes the pixel data input by the second pixel data signal at the pixel position indicated by the second pixel position signal; A pixel position signal is input, pixel data is read from a pixel position indicated by the third pixel position signal, and the read data is output as a third pixel data signal. A second pixel position signal is input, and the second pixel position signal is input. When a pixel position in the block image memory is indicated, pixel data input by a second pixel data signal is written to the block image memory. And a block image memory for storing a texture image or a difference image block, and for inputting and outputting pixel data from a pixel position designated by the second image memory access unit. .

Preferably, when the control unit instructs the simultaneous execution of the motion compensation process and the texture mapping process, the pixel position designating unit obtains a pixel position in a reference image block for motion compensation and obtains a first pixel position signal. , The pixel position in the generated polygon is determined and output as a second pixel position signal, and the pixel position in the difference image block is determined and output as a third pixel position signal.

Preferably, the pixel calculating means includes a multiplier generating section for generating a multiplier in accordance with the calculation indicated by the control means, and the multiplier generating section generates pixel data input by the first pixel data signal. A multiplication unit that multiplies the calculated multiplier, and adds the pixel data input by the third pixel data signal and the operation result of the multiplication unit to obtain a second addition result.
And an adder for outputting the pixel data signal as a pixel data signal.

Preferably, the pixel calculation means includes a multiplier generation unit for generating a first multiplier and a second multiplier according to the calculation indicated by the control means, and a pixel input by the first pixel data signal. A first multiplier for multiplying data by a first multiplier generated by the multiplier generator, and a second multiplier generated by the multiplier generator on pixel data input by the third pixel data signal , And an adder that adds the operation result of the first multiplier and the operation result of the second multiplier and outputs the addition result as a second pixel data signal.

The multiplying unit or the first multiplying unit is 1
It may be possible to perform only the calculations of 、 １, 倍, / and ８.

The number of pixels of image data that can be stored in the block image memory may be the same size as the reference image block, the generated image block, and the difference image block.

The image data that can be stored in the block image memory may be rectangular image data of 16 pixels in each of vertical and horizontal directions.

Preferably, the block image memory comprises:
It is realized at least on the same semiconductor element as the pixel operation means.

The moving image decompression mapping apparatus of the present invention performs a difference operation on a reference image block before encoding by motion prediction processing, and performs orthogonal transformation and variable length encoding on the obtained difference image block before encoding. an input compressed image block is, the variable-length-decoding means for performing variable-length-decoding process to the compressed image block, enter the result of the variable-length-decoding process, by performing orthogonal transformation Standard
Orthogonal transformation means for outputting an image block and a difference image block ; and the reference image block which is a result of the orthogonal transformation means.
And the above- described image generating apparatus, which inputs the input image and the difference image block , performs a motion compensation process, and performs a texture mapping process using image data of a generated image block as a result of the motion compensation process. The decompression process and the texture mapping process are simultaneously performed on the compressed moving image data, thereby achieving the above object.

The multimedia device of the present invention has a CPU
And a main memory; a video decompression mapping device according to claim 27; and a frame memory, wherein the frame memory is connected to the video decompression mapping device and stores at least a reference image and a generated image. While decompressing the data, the image of the decompression result is simultaneously subjected to texture mapping on the surface of the three-dimensional object, thereby achieving the above object.

According to the present invention, the compression moving image decompression processing using the motion compensation processing and the texture mapping processing are processed by a single memory and a single processing device, so that the processing which is a problem of the prior art can be performed in many ways. Points that require memory,
The present invention has been made focusing on the point that a large amount of data transfer from the motion compensation processing device to the texture mapping processing device is required.

By newly devising the processing procedure of the motion compensation processing, the procedure of the memory access, and the method of arranging data in the memory, the motion compensation processing is performed by the apparatus for performing the texture mapping processing and the apparatus having a configuration similar to the memory configuration. Is made possible. As a result, both the texture mapping process and the motion compensation process can be performed by a single image generation device and a single image memory.

The basic texture mapping processing apparatus uses a texture image which is rectangular image data and a generated image which is an image being generated, and uses a texture image corresponding to image data constituting polygons in the generated image. The processing is performed by reading out the image data and the image data already drawn in the generated image from the image memory, adding the weight of the already drawn image data and the texture image data, and writing the generated image again in the image memory. .

In consideration of such an operation procedure of the texture mapping processing, a processing procedure of a motion compensation processing described below has been devised.

The reference image and the difference image of the motion compensation processing are stored in the image memory, and the image data in the difference image is read out at the same time as the image data in the difference image is read out.
After multiplying the reference image data by a coefficient as required, the reference image data is added to the difference image data and written into the generated image in the image memory. By doing so, it is possible to perform the motion compensation processing by a device having a configuration similar to the texture mapping processing device.

Further, in the motion compensation processing, bidirectional motion compensation processing may be performed using two reference images, the future and the past, or the position designation of the reference image may be performed with half-pixel accuracy.

In the bidirectional motion compensation processing, in the present invention, the motion compensation processing is performed according to the above-described procedure using one of the two reference images and the difference image, and the result is written back to the image memory. This is realized by regarding the rewritten image as a difference image and performing a similar motion compensation process between the image and another reference image. At this time, by setting the coefficient by which the reference image data is multiplied to 0.5, accurate bidirectional motion compensation processing becomes possible.

If the reference image position with half-pixel accuracy has been designated, the reference image is read from a position shifted by half a pixel from the designated reference image position with half-pixel accuracy, and this image data is read. On the other hand, the motion compensation process is performed by adding the difference image data and writing back to the image memory. Further, the motion compensation process result is newly set as the difference image. And the motion compensation process is repeated.
The motion compensation process is repeated for a reference image shifted by half a pixel vertically and horizontally (or up and down or left and right) with respect to a reference image of position designation with half pixel accuracy, and at the same time, a coefficient multiplied by the reference image data is controlled. By doing so, it is possible to simultaneously obtain the reference image for specifying the half-pixel position by averaging the upper, lower, left, and right reference image data, and to perform the motion compensation process using the reference image and the difference image.

As described above, by devising a new processing method in which the motion compensation processing is realized by adding the weight of the reference image and the difference image, writing back to the memory, and repeating this processing, the texture mapping processing and the processing are performed. This can be realized by a similar processing procedure, a memory access procedure, and a method of arranging data in a memory.

As a result, the motion compensation processing and the texture mapping processing can be realized by a single image processing device and a single memory configuration, which is necessary when performing both the motion compensation processing and the texture mapping processing. Compared to each dedicated processing unit and dedicated memory,
It is possible to greatly reduce the amount of data transfer between memories at the same time as the amount of circuits and memory of the processing device is reduced.

Further, the motion compensation process and the texture mapping process, which were conventionally performed sequentially, can be simultaneously performed.

The operation will be described below.

According to the present invention, with the above-mentioned configuration, it is possible to realize the expansion processing of the digital moving image compressed using the motion compensation prediction and the texture mapping processing to the polygon in the CG generation with a single configuration. is there. Furthermore, by simultaneously performing the compressed moving image decompression process and the texture mapping process, it is possible to map a moving image decompressed by the compressed moving image decompression process onto a polygon as a texture image of the texture mapping process.

In the compression of an image by motion compensation prediction, a similar block is searched for from a block in a past image (hereinafter referred to as a past image block) in a block area in the image being compressed, and this past image block is Compression is performed by taking the difference from the image block being compressed. Further, a similar block (hereinafter, this block is referred to as a future image block) is searched not only for the past image but also for the future image, and a difference between the block and the similar block is obtained. Further, the positions of the past image block and the future image block in the image are specified with an accuracy of half a pixel.

The expansion of the image thus compressed is performed by extracting a reference image block (past image block or future image block) serving as a reference for motion prediction from the past image or future image, and extracting the image in the currently reproduced image. Processing is performed by adding difference image data (hereinafter, referred to as a difference image block) between a block (hereinafter, referred to as a generated image block) and a reference image block to a reference image block. The decompression process is performed by performing the same process for all blocks constituting the entire image. Such decompression processing is called motion compensation processing. In the motion prediction process at the time of compression encoding, when a past image is used, when a future image is used, when both a past image and a future image are used, or when a reference image is not used, a motion prediction process is performed. An optimal prediction method may be selected for each block. In order to cope with this, it is necessary to switch the motion compensation method for each processing of the image block also in the motion compensation processing.

In the following description, the extension processing is performed from the past image block in the forward direction, the extension processing is performed from the future image block in the backward direction, and the expansion is performed from either the future or the past reference image block. Performing the process is referred to as one-way motion compensation, and performing the decompression process using both the past image block and the future image block is referred to as bidirectional motion compensation. Also, pixel data in the past image block is P (x, y), pixel data in the future image block is F (x, y), pixel data in the reference image block is R (x, y), and the difference image block is Pixel data in D (x,
y), the pixel data in the generated image block is denoted by G (x, y). Here, x and y represent pixel positions in the image block.

On the other hand, in the texture mapping process, the drawing pixel position in the generated image and the reference pixel position in the texture image are obtained based on the vertex coordinate information of the polygon, and the pixel data at the reference pixel position in the texture image is read out. The pixel data is processed by writing the pixel data to the drawing pixel position in the generated image. An image of the entire screen is generated by performing texture mapping on all the polygons constituting the screen in the same manner.

Further, the motion compensation processing is performed on one image block, and the texture mapping processing is performed using the generated image block as a texture image, so that the moving image decompression processing and the texture mapping processing are performed in parallel. Becomes possible. Further, according to the present invention, the motion compensation processing and the texture mapping processing can be performed simultaneously.

The image generating apparatus of the present invention has a pixel operation means for performing an operation using two image data. The type of the image data to be operated by the pixel operation means differs according to the contents to be processed. That is, when performing the motion compensation processing, the motion compensation processing operation is performed using the image data of the reference image and the image data of the difference image, and when performing the texture mapping processing, the image data of the texture image is calculated. The texture mapping is performed using the image data in the polygon. As a result, the image generation device of the present invention can perform the motion compensation process and the texture mapping process with the same device, and as a result, can reduce the circuit scale. In addition, in the conventional dress,
This eliminates the need to transfer the necessary data, and reduces the amount of memory required for processing.

In the following, each process of the basic image generation device will be described in detail, and then the processes and features of another image generation device of the present invention, and a moving image decompression mapping device using these image generation devices, The operation of the system will be described.

First, the basic motion compensation processing of the image generating apparatus will be described.

The forward motion compensation process or the backward motion compensation process, that is, the one-way motion compensation process, is processed as follows.

The image data G (x, y) of the image block generated by the one-way motion compensation is, as shown in (Equation 1), the image data D (x, y) of the difference image block and the Image data R of the reference image block in the reference image used as the reference
It is obtained by adding (x, y).

[0072]

G (x, y) = R (x, y) + D (x, y) The reference image and the difference image are stored in the image memory. Pixel data in the difference image block of the difference image is sequentially designated by the pixel position designation means, read out by the image memory access means, and input to the pixel calculation means. From the reference image, a reference image block serving as a reference for motion estimation of the currently generated image block is selected, and pixel data in the reference image block is sequentially designated by the pixel position designation means and read out by the image memory access means. And input to the pixel operation means. In the pixel operation means, two input pixel data are added, and the operation result is sequentially written to the pixel position designated by the pixel position designation means in the generated image block in the image memory via the image memory access means. By performing the calculation shown in (Equation 1) in this manner, one-way motion compensation processing, that is, forward motion compensation processing and backward motion compensation processing, is performed.

Next, the bidirectional motion compensation processing will be described.

In the bidirectional motion compensation, the average of the result of the forward motion compensation and the result of the backward motion compensation may be obtained, and is processed by performing the operation shown in (Equation 2).

[0075]

G (x, y) = ((P (x, y) + D (x, y)) + (F (x, y) +
D (x, y))} / 2 In the present invention, the operation of (Equation 2) is transformed into (Equation 3), and the processing is performed according to this.

[0076]

G (x, y) = (P (x, y) / 2 + D (x, y)) + F (x, y) / 2 That is, for the difference image block, By adding 0.5 times of the pixel data, and further adding 0.5 times of the pixel data of the future image block to this result, the bidirectional motion compensation processing is realized.

The image memory stores a past image, a future image, and a difference image as reference images for motion compensation. Pixel data in the difference image block of the difference image is sequentially designated by the pixel position designation means, read out by the image memory access means, and input to the pixel calculation means. This input pixel data is D (x, y). From the past image, the past image block which is the basis of the motion prediction of the currently generated image block is selected, and the pixel data in the past image block is sequentially designated by the pixel position designation means and read out by the image memory access means. And input to the pixel operation means. This input pixel data is P (x, y). In the pixel calculation means, P (x, y) / 2 + D (x, y) is obtained, and the calculation result is obtained by accessing the pixel position designated by the pixel position designation means in the generated image block in the image memory. It is written sequentially via the means. When the processing in one generated image block is completed, the image block just generated is regarded as a difference image block, and the pixel data in the difference image block is sequentially designated by the pixel position designation means and read out by the image memory access means. It is input to the calculating means. This input pixel data is P (x, y) / 2 + D (x, y). From the future image, a future image block serving as a reference for motion prediction of the image block currently being generated is selected, and pixel data in the future image block is sequentially designated by the pixel position designation means and read out by the image memory access means. And input to the pixel operation means. This input pixel data is F (x, y). In the pixel operation means, {P (x, y) / 2 + D (x, y)} + F (x, y) / 2
Is calculated, and the calculation result is sequentially written to the pixel position designated by the pixel position designation means in the generated image block in the image memory via the image memory access means. As described above, the bidirectional motion compensation processing is realized by performing the calculation shown in (Equation 3).

Next, the motion compensation processing with half-pixel accuracy will be described. In the motion compensation with half-pixel accuracy, the position designation of the reference image block may be shifted by half a pixel from the position where the pixel exists. In this case, the specified reference image block is obtained by performing an interpolation operation based on the image blocks located above, below, and left and right by half a pixel from the specified reference image block position, and the motion compensation process is performed based on this. Do.

The operation of the half-pixel accuracy one-way motion compensation processing will be described below. If the position of the reference image block R (x, y) is specified to be shifted by a half pixel in the horizontal direction from the position where the pixel exists, the reference image block (R0 (x, y)), and a reference image block R (x, y) is obtained according to (Equation 4) from a reference image block (R1 (x, y)) which is “right” by a half pixel from the designated position.

[0080]

R (x, y) = (R0 (x, y) + R1 (x, y)) / 2 The position of the reference image block R (x, y) is half the length of the pixel in the vertical direction. If the specified image is shifted by a pixel, the reference image block (R0 (x, y)
), And a reference image block R (x, y) is obtained according to (Equation 5) from a reference image block (referred to as R2 (x, y)) which is “under” by a half pixel from the designated position.

[0081]

R (x, y) = (R0 (x, y) + R2 (x, y)) / 2 The position of the reference image block R (x, y) is half the position of the pixel in both the vertical and horizontal directions. If the specified image is shifted by a pixel, the reference image block (R0
(x, y)), a reference image block (R1 (x, y)) only half a pixel from the specified position (R1 (x, y)), and a reference image block half a pixel (lower left) from the specified position (R2
(x, y)) and the reference image block R (x, y) according to (Equation 6) from the reference image block (referred to as R3 (x, y)) which is “lower right” by half a pixel from the designated position. ).

[0082]

R (x, y) = (R0 (x, y) + R1 (x, y) + R2 (x, y) +
R3 (x, y)) / 4 The half-pixel accuracy motion compensation is based on the image data of the reference image block and the image data of the motion compensation difference image block obtained as described above, Processing is performed according to the bidirectional motion compensation processing procedure.

That is, in the one-way motion compensation, an operation is performed according to (Equation 7).

[0084]

G (x, y) = (R0 (x, y) + R1 (x, y)) / 2 + D (x, y) When the pixel is shifted half a pixel in the horizontal direction G (x, y) = (R0 (x, y) + R2 (x, y)) / 2 + D (x, y) G (x, y) = (R0 (x, y) + R1 (x, y) + R2 (x, y) + R3 (x, y)) / 4
+ D (x, y) In another configuration of the present invention, processing is performed according to (Expression 8) obtained by modifying (Expression 7).

[0085]

G (x, y) = (R0 (x, y) / 2 + D (x, y)) + R1 (x, y) / 2 When the pixel is shifted by half a pixel in the horizontal direction G (x, y) = (R0 (x, y) / 2 + D (x, y)) + R2 (x, y) / 2 G (x, y) / 2 , y) = [{(R0 (x, y) / 4 + D (x, y)) + R1 (x, y) / 4} + R2
(x, y) / 4] + R3 (x, y) / 4 If there is a half pixel shift only in the horizontal or vertical direction,
As can be understood from comparison with (Equation 3), an operation procedure similar to that of the bidirectional motion compensation is employed.

The reference image and the difference image for motion compensation are stored in the image memory. Pixel data in the difference image block of the difference image is sequentially designated by the pixel position designation means, read out by the image memory access means, and input to the pixel calculation means. This input pixel data is D (x, y).
From the reference image block, pixel data R0 (x, y) in the reference image block shifted by a half pixel from the designated position is sequentially designated by the pixel position designation means, read out by the image memory access means, and input to the pixel calculation means. You. In the pixel calculation means, R0 (x, y) / 2 + D (x, y) is obtained, and the calculation result is stored in the image memory at the pixel position designated by the pixel position designation means in the generated image block in the image memory. It is written sequentially via the means. When the processing in one generated image block is completed, the image block just generated is regarded as a difference image block, and the pixel data in the difference image block is sequentially designated by the pixel position designation means and read out by the image memory access means. It is input to the calculating means.
This input pixel data is R0 (x, y) / 2 + D (x, y). Again from the reference image, the pixel data of R1 (x, y) or R2 (x, y) is sequentially instructed by the pixel position specifying means, read out by the image memory access means, and input to the pixel calculating means. In the pixel operation means, {R0 (x, y) / 2 + D (x, y)} + R1 (x, y) / 2 or
{R0 (x, y) / 2 + D (x, y)} + R2 (x, y) / 2 was obtained, and the operation result was indicated by the pixel position designation means in the generated image block in the image memory The data is sequentially written at the pixel positions via the image memory access means. The pixel data G (x, y) can be obtained as described above.

If there is a half pixel shift in both the vertical and horizontal directions, the following procedure is taken.

The image memory stores a reference image for motion compensation and a difference image. Pixel data D (x, y) in the difference image block of the difference image is sequentially read out and input to the pixel calculation means. From the reference image block, pixel data R0 (x,
y) is input to the pixel operation means. In the pixel operation means, R0
(x, y) / 4 + D (x, y) is obtained, and the operation result is written into the generated image block in the image memory. When the processing within one generated image block is completed, the image block just generated is regarded as a difference image block, and the pixel data R0 (x, y) / 4 + D (x, y) in the difference image block are sequentially read. And output to the pixel operation means. From the reference image, the pixel data of R1 (x, y) is sequentially read and input to the pixel operation means. In the pixel calculation means, (R0 (x, y) / 4 + D (x, y)) + R1 (x, y)
/ 4 is calculated, and the operation result is written into the generated image block in the image memory. The generated image block is regarded as a difference image block, and pixel data (R0 (x, y) / 4 + D (x, y)) + R1 (x, y) / 4 in the difference image block are sequentially read. And output to the pixel operation means. From the reference image, R2
The pixel data of (x, y) is sequentially read and input to the pixel operation means. In the pixel calculation means, ((R0 (x, y) / 4 + D (x, y)) +
R1 (x, y) / 4} + R2 (x, y) / 4 is obtained, and the calculation result is written into the generated image block in the image memory. Further, the generated image block is regarded as a difference image block,
The pixel data {(R0 (x, y) / 4 + D
(x, y)) + R1 (x, y) / 4} + R2 (x, y) / 4 are sequentially read and input to the pixel operation means. From the reference image, the pixel data of R3 (x, y) is sequentially read and input to the pixel operation means. In the pixel operation means, [{(R0 (x, y) / 4 + D (x, y)) + R1 (x,
y) / 4} + R2 (x, y) / 4] + R3 (x, y) / 4, and the calculation result is written in the generated image block in the image memory.
In this way, the pixel data G (x, y) of the generated image block
Can be obtained.

In order to perform the half-pixel precision bidirectional motion compensation processing, the half-pixel precision forward motion compensation and the half-pixel precision reverse motion compensation are obtained by the above-described method, and the average is obtained. Although there are many cases corresponding to the specification of the image block positions of the past image block and the future image block, basically, the same processing as the one-way motion compensation may be performed. After adding the image data of the image block shifted by half a pixel from the designated pixel position of the reference image block multiplied by a multiplication coefficient according to the ratio of the interpolation operation to the difference image data, and writing the result as a generated image block It is sufficient to regard the generated image block as a new difference image block, add another reference image block to the generated difference image block, and write the generated image block into the generated image block.

Hereinafter, the texture mapping processing of the basic image generating apparatus of the present invention will be described.

A texture image is stored in the image memory. Based on the vertex coordinates of the polygon to be texture-mapped, the reference pixel position in the texture image and the drawing pixel position in the generated polygon in the generated image are obtained by the pixel position specifying means. The pixel data at the pixel position is read out from the reference pixel position in the texture image via the image memory access unit, and is provided to the pixel operation unit. The pixel operation means outputs the input pixel data as it is as the operation result. The operation result is written to the drawing pixel position in the generated polygon specified by the pixel position specifying unit via the image memory access unit. By performing this process for all pixels in the generated polygon, a texture mapping process is performed.

Further, the pixel data in the generated polygon read from the image memory is input to the pixel calculation means, and the pixel calculation is performed between the pixel data and the texture pixel data read from the texture image, whereby the translucent texture is obtained. It is possible to perform texture mapping of an image and anti-aliasing of polygon edges in the texture mapping.

The texture mapping process of the translucent texture image will be described in detail below. Let α be the translucency of the texture image, A be the pixel data value already written at the rendering pixel position of the generated polygon, and B be the pixel data of the texture image to be mapped. The pixel data A already written at the drawing pixel position is designated by the pixel position designation means, and is read out via the image memory access means. The read pixel data A is input to the pixel operation means. Pixel data B of texture image
The pixel position is designated by the pixel position designation means, and is read out via the image memory access means. The read pixel data B is input to the pixel operation means. The pixel calculation means calculates α × A + (1−α) × B. The pixel data of the calculation result is written to the drawing pixel position in the generated polygon specified by the pixel position specifying unit via the image memory access unit. In this way, texture mapping of the translucent texture image becomes possible. Also,
For the anti-aliasing of the polygon edge, the same processing as that for the translucent texture may be performed on only the pixels constituting the polygon edge. With the above processing, it is possible to perform the mapping of the translucent texture and the anti-aliasing processing of the edge.

Next, the simultaneous execution of the motion compensation process and the texture mapping process, that is, texture mapping of a compressed image will be described.

The compressed image to be subjected to the motion compensation processing is a collection of a plurality of block images which are units of the motion compensation processing. These blocks are represented as B0 to B99.
Further, a texture mapping target polygon is defined for each block image which is a unit of motion compensation. These polygons are represented as P0 to P99. Here, 100 of B0 to B99
The images made of the block images are 10
Assume that an image texture-mapped to zero polygons is generated.

First, a motion compensation process is performed on the B0 image block in accordance with the above-described motion compensation process. Next, the expanded image block is used as a texture image, and this block image is subjected to texture mapping on the polygon P0 according to the above-described texture mapping method. In this manner, the motion compensation process and the texture mapping process are repeated for each block, and all blocks (B0 to
B99), the decompression process from the reference image and the difference image consisting of multiple image blocks and the texture mapping process using this decompression result as a texture image are simultaneously performed in a single configuration. It is possible to do with.

Further, by specifying the pixel position in the pixel position specifying means as described below, it is possible to simultaneously execute the motion compensation processing of the block image and the texture mapping processing. That is, the pixel position in the reference image block is specified as the first pixel position, the pixel position in the generated polygon is specified as the second pixel position, and the third pixel position is specified.
The pixel position in the difference image block is designated as the pixel position of.

The pixel data is read from the first pixel position in the reference image block, and the pixel data in the difference image block corresponding to the pixel data is read from the third pixel position in the difference image block. The pixel data is calculated by the pixel calculation means, and the resulting pixel data is written to the pixel position in the generated polygon specified by the second pixel position. This process is sequentially performed on all pixels in the generated polygon. By performing the processing as described above, it is possible to perform the expansion processing by the motion compensation processing based on the difference image block compressed by the motion prediction and the reference image, and to deform and draw the shape of the generated polygon.

As described above, the image generating apparatus of the present invention can realize both texture mapping processing and compressed moving picture decompression processing by motion compensation with a single configuration, and simultaneously execute these processings. Becomes possible.

The image data of the difference image block in the case of the motion compensation processing and the texture image data in the case of the texture mapping processing may be stored in the block image memory.

By separating the block image memory from the memory in which the generated image or the reference image is stored, the access to the pixel data in the reference image block and the access to the pixel data in the difference image block collide. And
It is possible to avoid collision between access to pixel data in the texture image and access to pixel data in the generated polygon. The size of the difference image block is generally about 8 × 8 pixels to 16 × 16 pixels, and a memory having such a size constitutes an image generating apparatus.
It can be built in LSI, which makes it possible to reduce the size of the circuit and increase the processing speed (access speed of the block image memory).

Further, it is also possible to adopt a configuration that enables the image block of the generated image to be moved to the block image memory. Thus, the above-described basic bidirectional motion compensation processing and half-pixel precision motion compensation processing of the image generation device are performed in the following procedure.

In the processing procedure of the bidirectional motion compensation, the processing is performed according to the above (Equation 3). That is, the bidirectional motion compensation is performed by adding 0.5 times the pixel data of the past image block to the difference image block, and further adding 0.5 times of the pixel data of the future image block to the result. Realize. Hereinafter, the processing of the bidirectional motion compensation will be described in more detail.

First, a forward motion compensation process is performed. The difference image block is stored in the block image memory. Pixel data D (x, y) specified in the third pixel position signal is sequentially read from the block image memory via the second image memory access unit. On the other hand, the pixel data P (x, y) of the past image block designated as the first pixel position signal is sequentially read from the past image block via the first image memory access unit. These pixel data are input to the pixel operation means, and P (x, y) / 2 + D (x, y) is obtained. The pixel position designating means designates a pixel position in the generated image block by the second pixel position signal, and the pixel data obtained by the pixel operation means stores the first pixel data in the designated pixel position in the generated image block. Written via the image memory access unit. This process is sequentially performed on the pixel data of the entire image block.

Next, the image moving means sequentially specifies the pixel positions in the generated image block obtained by performing the forward motion compensation processing as the fourth pixel position signal, and sets the first image memory access unit to The pixel data is read through the memory. On the other hand, the image moving means sequentially designates the pixel positions in the block image memory as a fifth pixel position signal, and writes the pixel data read from the generated image block into the block image memory via the second image memory access unit. Thus, the block image resulting from the forward motion compensation processing is moved to the block image memory.

Next, a backward motion compensation process is performed. Third
Pixel data P (x, y) / 2 + D specified in the pixel position signal of
(x, y) are sequentially read out via the second image memory access unit. On the other hand, from the future image block, the pixel data of the future image block designated as the first pixel position signal
F (x, y) are sequentially read out via the first image memory access unit. These pixel data are input to the pixel operation means, and {P (x, y) / 2 + D (x, y)} + F (x, y) / 2 is obtained.
The pixel position designation means designates the pixel position in the generated image block of the generated image as the second pixel position signal,
The obtained pixel data is written via the first image memory access unit. This process is sequentially performed on the pixel data of the entire image block.

The calculation shown in (Equation 3) is performed as described above. The bidirectional motion compensation processing is performed by performing the above-described processing for all the image blocks constituting the generated image.

Hereinafter, the motion compensation processing with half-pixel accuracy will be described.

First, the difference image block is stored in the block image memory, and the pixel data specified by the third pixel position signal is sequentially read out via the second image memory access unit. On the other hand, pixel data specified as the first pixel position signal is sequentially read from the reference image block via the first image memory access unit. These pixel data are input to the pixel operation means, and the pixel operation is performed. The pixel position designation means designates the pixel position in the generated image block by the second pixel position signal,
The pixel data obtained by the pixel calculation means is written to a designated pixel position in the generated image block via the first image memory access unit. This process is sequentially performed on the pixel data of the entire image block to perform a block motion compensation process.

Next, the image moving means sequentially specifies the pixel positions in the generated image block obtained by performing the above one-way motion compensation processing as the fourth pixel position signal, and sets the first image memory access unit to the first image memory access unit. The pixel data is read through the memory. On the other hand, the image moving means sequentially designates the pixel positions in the block image memory as a fifth pixel position signal, and writes the pixel data read from the generated image block into the block image memory via the second image memory access unit. In this way, the block image resulting from the one-way motion compensation processing is moved to the block image memory.

The above-described block motion compensation processing and the movement of the image block to the block image memory are repeated as many times as necessary for the half-pixel precision motion compensation processing, whereby half-pixel precision motion compensation can be performed. Becomes

By adopting the above configuration, it is possible to realize both texture mapping processing and compressed moving picture decompression processing by half-pixel precision unidirectional / bidirectional motion compensation with a single configuration. Becomes

In still another configuration, it is possible to write the pixel data output from the pixel calculation means to the block image memory via the second image memory access unit. Thus, the bidirectional motion compensation processing and the half-pixel precision motion compensation processing described in the basic configuration are performed in the following procedure.

In the processing procedure of the bidirectional motion compensation, the processing is performed according to the above (Equation 3).

That is, in the bidirectional motion compensation, first, 0.5 times the pixel data of the past image block is added to the motion-compensated difference image block. This is realized by adding five times.

First, a forward motion compensation process is performed. A difference image block is stored in the block image memory, and pixel data D (x, y) specified in the third pixel position signal is stored.
Are sequentially read out via the second image memory access unit. On the other hand, from the past image block, the pixel data P (x, y) of the past image block designated as the first pixel position signal
Are sequentially read out via the first image memory access unit. These pixel data are input to the pixel operation means,
P (x, y) / 2 + D (x, y) is obtained. The pixel position designation means designates the pixel position in the block image memory as the second pixel position signal, and the pixel data obtained by the pixel operation means stores the second pixel position in the designated pixel position in the block image memory. Written via the image memory access unit.
This process is sequentially performed on the pixel data of the entire image block.

Next, a backward motion compensation process is performed. Third
Pixel data P (x, y) / 2 + D specified in the pixel position signal of
(x, y) are sequentially read out via the second image memory access unit. On the other hand, from the future image block, the pixel data of the future image block designated as the first pixel position signal
F (x, y) are sequentially read out via the first image memory access unit. These pixel data are input to the pixel operation means, and {P (x, y) / 2 + D (x, y)} + F (x, y) / 2 is obtained.
The pixel position designation means designates the pixel position in the generated image block of the generated image as the second pixel position signal,
The obtained pixel data is written via the first image memory access unit. This process is sequentially performed on the pixel data of the entire image block.

As described above, the operation shown in (Equation 3) is performed. The bidirectional motion compensation processing is performed by performing the above-described processing for all the image blocks constituting the generated image.

Further, the motion compensation processing with half-pixel accuracy can be realized by performing the following processing. That is, an operation is performed by the pixel operation means using the pixel data of the difference image block stored in the block image memory and the pixel data of the reference image block, and the operation result is written back to the block image memory. Only the last block motion compensation process among the multiple repetitions of the half-pixel accuracy motion compensation process is written into the generated image via the first image memory access means.

By performing the processing as described above, it is not necessary to move the image block data required in the above processing, and it is possible to greatly reduce the number of times of memory access.

Image data in the block image memory is referred to only once for each pixel in the block motion compensation processing.
Since the calculation result is written back to the referenced pixel position, no problem occurs in performing the processing even if the calculation result is overwritten on the image data to be calculated.

As described above, it is possible to realize both texture mapping processing and compressed moving picture decompression processing by half-pixel precision one-way / two-way motion compensation with a single configuration. Furthermore, compared to the case described above, the number of accesses to the block image memory is reduced, so that the processing can be sped up.

As described above, with a single configuration, the compressed moving image decompression process by the motion compensation process and the texture mapping process are realized, thereby simplifying the configuration of the image generating apparatus.
It is possible to reduce the circuit scale, reduce the amount of memory required for processing, and reduce the amount of data transfer by eliminating the need to transfer moving image data between memories.

In the moving picture decompression and mapping apparatus using the above-described image generating apparatus, moving picture compression using variable length coding such as motion prediction of image blocks, orthogonal transform such as DCT, and Huffman coding, as represented by MPEG, etc. A moving image decompression process and a texture mapping process using the decompressed moving image as a texture image are performed on the compression-encoded digital moving image. The details of the processing will be described below.

The compressed moving image is obtained by orthogonally transforming a reference image and performing orthogonal transformation on an image block obtained as a result of difference between compressed reference image data obtained by performing variable length encoding on compressed reference image data and image blocks constituting the reference image.
And variable-length encoded compressed difference image data.

In the moving image decompression mapping apparatus of the present invention, first, the compressed reference image data is subjected to variable-length inverse coding, and orthogonal transformation is performed to obtain a decompression reference image. Here, if texture mapping processing is necessary, texture mapping processing is performed using this image as a texture image. Next, the compressed difference image data is subjected to variable-length inverse coding and subjected to orthogonal transform to obtain a difference image. Furthermore, an expanded image (generated image) is obtained by performing a motion compensation process by the image generation device of the present invention. If texture mapping processing is required, texture mapping processing is performed using this generated image as a texture image. Alternatively, the motion compensation process and the texture mapping process are simultaneously performed on the difference image as in the operation of the image generation device described above.

As described above, the moving picture expansion mapping apparatus of the present invention can perform the texture mapping process using the moving image as the texture image while expanding the compressed moving image. Therefore, it can be realized by a small-scale device.

In a multimedia device using the above-described moving image decompression mapping device, a moving image decompression process and a texture mapping process are performed as follows.

The CPU controls the moving image decompression mapping device and other components of the system and executes an application program in accordance with the program stored in the main memory. The moving image decompression mapping device generates and displays an image on a frame memory based on instructions and data from the CPU.

In this system, the operation of a process in which a compressed moving image is texture-mapped to a three-dimensional polygon and displayed will be described. It is assumed that the storage device of the multimedia device stores compressed moving image data and a three-dimensional polygon shape.

The CPU reads out the compressed moving image data and polygon shape data from the storage device to the main memory. CPU
Converts the three-dimensional polygon shape into a two-dimensional polygon shape on the screen. Next, the compressed moving image data is passed from the CPU to the moving image expansion mapping device, and an image block setting process is instructed. The image generation device in the moving image decompression mapping device sets a generated image area and a reference image in a frame memory. Next, a decompression process is instructed from the CPU.
The moving image decompression mapping device performs a variable-length inverse encoding process and an orthogonal transform process on the given compressed moving image data.
Further, a motion compensation process is performed by an image generation device in the moving image decompression mapping device, and a decompressed image is generated in a generated image area in a frame memory. Next, an image block setting process is instructed from the CPU to the moving image extension mapping device, and the generated image data is set as a texture image and a new generated polygon is set. Then, a two-dimensional polygon shape is given from the CPU, and texture mapping processing is instructed. Based on the set texture image and the generated polygon, a texture mapping process is performed in the image generation device, and a polygon image on which the moving image data is texture-mapped is generated. For one piece of decompressed image data, not only one polygon shape but also a plurality of polygon shapes are given, and it is possible to map to a complicated shape.

In the above description, the texture mapping process is performed after the image is expanded and the expanded image is obtained. However, the moving image expansion process and the texture mapping process can be alternately performed for each image block. . Further, it is possible to simultaneously execute the motion compensation process and the texture mapping process in moving image decompression.

As described above, it is possible to configure and operate the multimedia device, so that the decompressed image data is not transferred on the system bus, and the data transfer load on the system bus is greatly reduced. . Further, since the moving image decompression process and the texture mapping process are performed using the same frame memory, the amount of memory in the system can be reduced.

As described above, according to the present invention, decompression processing of a digital moving image compressed using inter-frame motion prediction, texture mapping processing to polygons in CG generation, and decompression processing and texture mapping processing are simultaneously performed. It is possible to realize an image generation device, a moving image decompression mapping device, and a multimedia device that can be performed. As a result, it is possible to simplify the configurations of devices and devices, reduce the circuit scale, reduce the amount of memory required for processing, and reduce the amount of data transfer by eliminating the need to transfer video data between memories. .

[0135]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of an image processing apparatus, a moving image decompression mapping apparatus, and a multimedia device according to the present invention will be described with reference to the drawings.

By using the present invention, it is possible to expand moving image data compressed based on motion prediction and generate an image by texture mapping using the expanded moving image data as a texture image.

In the motion prediction, the most similar block is obtained from a past or future image of a certain block image in the image, and the difference is obtained to reduce the data amount. In a moving image compression encoding method such as MPEG, as shown in FIG.
182 (eg, an image block of 16 × 16 pixels) is subjected to motion prediction by calculating a difference from the similar image block 181 in the reference image, and is further subjected to orthogonal transform (such as DCT) and variable-length coding (such as Huffman coding). And performs encoding.

Decompression of the moving picture encoded in this way is as follows.
As shown in FIG. 22, the compressed moving image data is first subjected to variable-length inverse coding and converted into an orthogonal transform coefficient image 191, which is then subjected to orthogonal transform to be converted into a difference image 192. Difference image 192
Is divided into image blocks, and a motion compensation process is performed for each block. That is, an image block 193 serving as a reference for motion prediction is extracted from the reference image, and the image data and difference image data are calculated to obtain an image of a motion compensation result. The reference image of the motion prediction process at the time of compression encoding is a past image (forward motion prediction), a future image (reverse motion prediction), and both a past image and a future image (bidirectional motion prediction). In some cases, a reference image is not used (no motion prediction), and an optimal prediction method is selected for each block of motion prediction. In order to cope with this, it is necessary to switch the motion compensation method for each processing of the image block also in the motion compensation processing.

In the image generating apparatus of the present invention, not only the motion compensation processing shown in FIG. 22 but also the deformation of the image by the texture mapping processing shown in FIG. 23, and the motion compensation processing and the texture mapping shown in FIG. Simultaneous processing can be performed by a single device.

For texture mapping of a moving image, for example,
When adding a three-dimensional effect to a moving image in video editing or multimedia equipment (see FIG. 25A), or when attaching a moving image with perspective to an image generated by three-dimensional CG (see FIG. 25). 25 (B)). By using the present invention, texture mapping processing on compressed moving image data can be performed efficiently.

Hereinafter, regarding the multimedia device of the present invention using the above-described image generation device, the system configuration, the configuration and operation of the image generation device, the operation of the moving image decompression mapping device using the image generation device, and the operation of the multimedia device Will be described.

(Embodiment 1) FIG. 1 shows a configuration diagram of a multimedia apparatus according to a first embodiment of the present invention.

The multimedia device 100 according to the present embodiment is configured as follows. The CPU 101 controls the entire system, controls the moving image decompression mapping device, controls the image generation device, controls the disk device, and the like. Main memory
Reference numeral 102 denotes a program for controlling the operation of each unit of the system, an image and other data read from a disk device, and data used in the program, including a program for controlling the image generation device executed by the CPU. .
The disk device 103 stores compressed image data, texture image data, a drawing program, and the like. A moving image decompression mapping device 108 performs a variable length decoding process on the compressed moving image data written by the CPU 101 and outputs an orthogonal transform coefficient image data. An orthogonal transformation unit 106 that performs orthogonal transformation processing and outputs difference image data, an image generation device 104, and a local bus 107 for connecting these to a frame memory 109 are provided. The frame memory 109 stores a reference image, a difference image, and a generated image used by the image generation device 104. The DA converter 110 stores the image generated by the image generator 104 in the frame memory 109.
In order to display on the read-out display device 111, DA conversion is performed. The display device 111 displays the image output from the DA converter 110. The system bus 112 connects the CPU 101, the main memory 102, the disk device 103, and the moving image decompression mapping device 108 to exchange data.

Next, regarding the multimedia device 100,
The respective operations of expanding the compressed moving image, texture mapping, and performing texture mapping using the compressed moving image as texture data will be described.

Here, a description will be given of the operation in which compressed moving image data based on MPEG is stored in the disk device 103, a moving image decompression process is performed while reading the data, and the moving image is displayed as a moving image. This operation procedure is stored in the main memory as a program.

[0146] The compressed moving image data string by MPEG is obtained by performing an intra-frame encoding process on one image without performing motion estimation and a compression encoding process by performing motion estimation on the basis of a past image. It is composed of a P picture and a B picture that has been subjected to motion prediction from both the past image and the future image and has been subjected to compression encoding processing. Here, it is assumed that the moving image is composed of image 1, image 2, image 3, and image 4 in this order. Image 1 is compressed as an I picture,
Image 4 is compressed as a P-picture obtained by performing forward motion prediction using image 1 as a reference image.
It is assumed that the image 4 is compressed as a B picture that has been subjected to bidirectional motion prediction.

First, the CPU 101 reads the compressed data of the image 1 from the disk device 103. This data is an I picture and does not require motion compensation processing. The compressed image is divided into image blocks (written as compressed image blocks), and each block is sent to the moving image decompression mapping device 108. Since the compressed image block has been Huffman-encoded, the variable-length inverse encoding unit 105 performs an inverse transform, and outputs the result to the orthogonal transform unit 106. The orthogonal transform unit 106 performs the inverse DCT transform and writes the result in the generated image in the frame memory 109. By processing all the compressed image blocks in the same manner, an expanded image 1 is generated in the frame memory 109.

Next, the image 1 generated in the frame memory 109 is set as a reference image. The CPU 101 reads the compressed image data of the image 4 from the disk device 103. This data is a P-picture and requires forward motion compensation processing. The compressed image is divided into compressed image blocks, and each block is sent to the moving image decompression mapping device 108. Since the compressed image block is Huffman-coded, inverse transform is performed in the variable-length inverse encoding unit 105, and the result is
Output to The orthogonal transform unit 106 performs the inverse DCT transform and writes the result in the difference image of the frame memory 109.
Based on the reference image block in the reference image (image 1) and the difference image block in the difference image, the image generation device 104 performs a forward motion compensation process to store the image 4 in the frame memory 109.
To be generated.

From this point, the DA converter 110 reads the image 1 in the frame memory 109 and displays it on the display device 111.

Next, the already generated images 1 and 4 are set as reference images. The CPU 101 reads out the compressed image data of the image 2 from the disk device 103. This data is a B picture, and requires bidirectional motion compensation processing. The compressed image is divided into compressed image blocks, and each block is sent to the moving image decompression mapping device 108. Since the compressed image block is Huffman-coded, the variable-length inverse coding unit 105
, And outputs the result to the orthogonal transform unit 106. The orthogonal transform unit 106 performs the inverse DCT transform and writes the result in the difference image of the frame memory 109. Past image (Image 1)
The image generation device 104 performs a bidirectional motion compensation process on the basis of the reference image block in, the reference image block in the future image (image 4), and the difference image block in the difference image to generate an image 2.

From this point, the DA converter 110 reads the image 2 in the frame memory 109 and displays it on the display device 111.

Next, the CPU 101 reads out the compressed image data of the image 3 from the disk device 103. This data is a B picture, and requires bidirectional motion compensation processing. The compressed image is divided into compressed image blocks, and each block is sent to the moving image decompression mapping device 108. Since the compressed image block is Huffman-coded, the variable-length inverse coding unit 105
, And outputs the result to the orthogonal transform unit 106. The orthogonal transform unit 106 performs the inverse DCT transform and writes the result in the difference image of the frame memory 109. Past image (Image 1)
The image generation device 104 performs a bidirectional motion compensation process on the basis of the reference image block in the reference image block, the reference image block in the future image (image 4), and the difference image block in the difference image to generate an image 3.

From this point, the DA converter reads the image 3 in the frame memory 109 and displays it on the display. When the display of the image 3 is completed, the already generated image 4
Is read by the DA converter 110 and displayed on the display device 111.

As described above, in the multimedia device 100, it is possible to display image data compressed and encoded by MPEG while expanding it. This procedure is shown in the timing chart of FIG.

In the above description, the motion prediction method in one picture is fixed to one. However, in MPEG, the picture blocks which are compressed without performing motion prediction are contained in the picture blocks constituting the P picture. In a block or an image block constituting a B picture, the existence of an image block that has been subjected to compression without forward motion prediction or motion prediction is allowed.
Even in such a case, in the present embodiment, it is possible to perform the decompression processing by performing the above-described processing while resetting the motion compensation method for each image block.

Here, the texture image is stored in the disk device.
A method for reading out the stored image, performing texture mapping on the polygon, and displaying the generated image will be described. This operation procedure is stored in the main memory as a program.

First, the CPU 101 reads a texture image from the disk device 103 and writes it in the frame memory 109 via the moving image decompression mapping device. The CPU 101 specifies the polygon position (vertex coordinates) in the generated image generated in the frame memory 109 to the image generating device 104, specifies the position of the texture image in the frame memory to the image generating device 104, and executes the texture mapping. Give instructions. The image generation device 104 draws a generated polygon in the generated image from the texture image in the frame memory 109.
When the texture mapping of all polygons in the generated image is completed, the completed generated image is stored in the frame memory 109.
Is read by the DA converter 110 and displayed on the display device 111. To generate the next image, the CPU 101 writes new necessary texture data in the frame memory 109, and then instructs the image generating device 104 to specify a polygon position and perform texture mapping.

Here, a description will be given of a process in which compressed moving image data by MPEG is stored in the disk device 103, a moving image decompression process is performed while reading the data, a texture mapping process is performed at the same time, and a moving image mapped to a polygon is displayed. . This operation procedure is stored in the main memory 102 as a program. A moving image is composed of an image 1 and an image 2 in this order. It is assumed that the image 1 is compressed as an I-picture and the image 2 is compressed as a P-picture obtained by performing forward motion prediction using the image 1 as a reference image. It is assumed that the image 1 is a reference image for the motion compensation processing of the image 2, but the image 2 is not a reference image for another motion compensation processing.

First, the CPU 101 reads the compressed data of the image 1 from the disk device 103. This data is an I picture and does not require motion compensation processing. The compressed image is divided into image block (hereinafter, referred to as a compressed image block) units,
Each block is sent to the moving image decompression mapping device 108. Since the compressed image block has been Huffman-encoded, the variable-length inverse encoding unit 105 performs an inverse transform, and outputs the result to the orthogonal transform unit 106. The orthogonal transform unit 106 performs an inverse DCT transform and writes the result in the texture image in the frame memory 109. Here, the CPU 101 causes the image generation device 104 to specify the image block expanded in the above description as a texture image and to perform texture mapping by designating the position of the generated polygon. By processing all the compressed image blocks in the same manner, a decompressed image 1 and a generated image obtained by texture-mapping this image and mapping it to polygons are obtained in the frame memory 109.

At this point, the DA converter 110 reads the image 1 in the frame memory 109 and displays it on the display device 111.

Next, the CPU 101 reads the compressed data of the image 2 from the disk device 103. This data is a P picture, and it is necessary to perform a forward motion compensation process using image 1 as a reference image. The compressed image is divided into image blocks (written as compressed image blocks), and each block is sent to the moving image decompression mapping device 108. Since the compressed image block has been Huffman-encoded, the variable-length inverse encoding unit 105 performs an inverse transform, and outputs the result to the orthogonal transform unit 106. The orthogonal transform unit 106 performs an inverse DCT transform and writes the result into a difference image block in the difference image of the frame memory 109. Here, the CPU 101 designates the difference image block as a difference image block, designates the reference image block of the reference image (image 1) already written in the frame memory 109, and specifies the position of the generated polygon in the generated image. And instructs the image generation device 104 to simultaneously execute the moving image decompression process and the texture mapping. Image generation device
At 104, texture mapping is performed on the pixel data on which the motion compensation processing has been performed in the generated polygon, while performing forward motion compensation processing based on the difference image and the reference image. By processing all the compressed image blocks in the same manner, a generated image in which the expanded image 2 is texture-mapped to a polygon is obtained in the frame memory 109. The result of this processing is only the image that has been texture-mapped and deformed, but as described above, image 2 does not become a reference image for motion compensation of the next image, and therefore no problem occurs.

At this point, the DA converter 110 reads out the image 2 in the frame memory 109 and displays it on the display 111.

As described above, in the multimedia device shown in FIG. 1, it is possible to realize a process of decompressing a compressed moving image such as MPEG, a texture mapping process, and a process of performing texture mapping using the compressed moving image as texture data.

Hereinafter, the configuration and operation of the image generating apparatus 104 used in the above-described multimedia device will be described in detail. The image generation device 104 is configured as shown in FIG.

The control circuit 201 interprets the command designated by the command input signal 20a, and
A command signal for controlling operations of the image moving circuit 206 and the pixel operation circuit 203 is output. The pixel position designation circuit 202
In accordance with an instruction (command signal) from the control circuit 201, a pixel position (first pixel position) in the reference image block or the generated polygon is obtained and output as a first pixel position signal 20b. Is obtained as the second pixel position signal 20c, and the pixel position (third pixel position) in the difference image block or the texture image is obtained as the third pixel position signal 20d. Output. The image memory access circuit 204 receives the first pixel position signal 20b, reads out pixel data at a specified pixel position in the frame memory 109, and reads the first pixel data signal 20e.
And receives the second pixel position signal 20c and outputs the second pixel data signal 2 to the designated pixel position in the frame memory 109.
A first image memory access unit 212 that writes pixel data input as 0f, receives a fourth pixel position signal 20h, reads out pixel data at a specified pixel position in the frame memory 109, and outputs it as a fourth pixel data signal 20j. And receives the third pixel position signal 20d, reads out the pixel data at the designated pixel position in the block image memory 205 and outputs it as the third pixel data signal 20g, and receives the fifth pixel position signal 20i in the block image memory. And a second image memory access unit 211 for writing the pixel data input as the fifth pixel data signal 20k at the designated pixel position. Pixel operation circuit 2
03 inputs a first pixel data signal 20e and a third pixel data signal 20g, and outputs the result of calculating these data as a second pixel data signal 20f. In this embodiment,
The pixel operation circuit 203 generates a first multiplier and a second multiplier according to an instruction from the control circuit 201, and outputs the first multiplier and the second multiplier to the first multiplier and the second multiplier, and a multiplier generator 207. A first multiplier 209 that multiplies the first multiplier and the first pixel data specified by the second multiplier, and a second multiplication that multiplies the second multiplier and the third pixel data specified by the multiplier generator 207 Part 208
And an adder 210 for adding the operation result of the first multiplier 209 and the operation result of the second multiplier 208. When the instruction (command signal) of the control circuit 201 indicates a motion compensation process, or when the pixel calculation circuit 203 indicates a simultaneous process of the motion compensation process and the texture mapping process, the pixel operation circuit 203 outputs the signed data. When the calculation is performed with precision and the texture mapping process is instructed, the calculation is performed with unsigned data precision.

The image moving circuit 206 generates a fourth pixel position and a fifth pixel position for moving the image block in the frame memory 109 to the block image memory 205 in accordance with an instruction from the control circuit 201. Fourth pixel position signal 20
The pixel data specified by h is read from the frame memory 109 via the image memory access circuit 204, and this image data is stored in the pixel position in the block image memory 205 specified by the fifth pixel position signal 20i. Write through.

The image generating apparatus 104 configured as described above
The operation of will be described below. The image generation device 104 is a CP
The operation is instructed by a command from U101, and the control circuit 2
The operation is performed by interpreting the command in 01 and outputting a command signal for controlling each circuit of the image generating apparatus 104 according to the command. In the following description, a control procedure by issuing a command to the control circuit 201 by the CPU 101 and an operation procedure of each circuit in the image generating apparatus 104 under the control will be described.

The operation of the motion compensation processing in the image generation device 104 will be described below. FIGS. 3 and 4 show a control procedure of the image generation device 104 by the CPU 101.
5 and 6 show the operation procedure of each circuit. 5 and 6, the dashed arrows indicate the flow of data / control signals, and the solid arrows indicate the flow of processing.

First, the CPU 10 performs the motion compensation processing.
The control procedure of the image generation device 104 according to 1 will be described with reference to FIGS.

In STEP 301, a reference image (past image or future image) as a reference for motion compensation is set in the frame memory 109, and a difference image is written. Reference image is disk device 10
3 and written into the frame memory 109 by the CPU 101, and an image that has already been generated by the image generating device 104.

At STEP 311, it is determined whether or not to perform the motion compensation process. If no motion compensation process is performed, the process proceeds to STEP 312; otherwise, the process proceeds to STEP 321.

At STEP 312, an image block setting process is instructed to the image generating device 104. With this instruction, the position of the generated image block in the generated image and the position of the differential image block in the differential image are specified.

In STEP 313, the block motion compensation processing is controlled according to FIG.

In STEP 321, it is determined whether or not the process is a forward motion compensation process.
The process proceeds to step 22; otherwise, the process proceeds to step 331.

In STEP 322, an image block setting process is instructed to the image generating device 104. With this instruction, the position of the past image block in the past image, the position of the generated image block in the generated image, and the position of the difference image block in the difference image are indicated.

In STEP 323, the motion compensation processing of the block is controlled according to FIG.

In STEP 331, it is determined whether or not the process is a backward motion compensation process.
The process proceeds to step 32; otherwise, the process proceeds to step 341.

In STEP 332, an image block setting process is instructed to the image generating device 104. With this instruction, the position of the future image block in the future image, the position of the generated image block in the generated image, and the position of the difference image block in the difference image are indicated.

In STEP 333, the block motion compensation processing is controlled according to FIG.

In STEP 341, an image block setting process is instructed to the image generating device 104. With this instruction, the position of the past image block in the past image, the position of the generated image block in the generated image, and the position of the difference image block in the difference image are indicated.

At STEP 342, the motion compensation processing of the block is controlled according to FIG.

In STEP 343, an image block setting process is instructed to the image generating device 104. With this instruction, the position of the future image block in the future image, the position of the generated image block in the generated image, and the position of the difference image block in the difference image are indicated.

At STEP 344, the motion compensation processing of the block is controlled according to FIG.

At STEP 302, it is determined whether or not the motion compensation processing has been completed for all the image blocks in the generated image. If completed, proceed to STEP 303; if not completed
The process proceeds to STEP 311, and the process is performed for the next image block.

At STEP 306, it is determined whether or not the motion compensation processing has been completed for all the images.
The processing advances to 301, and processing is performed for the next image.

Next, the control procedure of the image generation device 104 by the CPU 101 for performing the motion compensation processing of the image block will be described in detail with reference to FIG.

At STEP 351, the image generation device 104 is instructed to perform an image block setting process. Thereby, the difference image block to be processed next is moved from the difference image in the frame memory to the block image memory.

At STEP 361, it is determined whether or not to perform motion compensation with half-pixel accuracy. If motion compensation with half-pixel accuracy is not performed, the process proceeds to STEP 362. If motion compensation with half-pixel accuracy is performed, the process proceeds to STEP 371.

In STEP 362, the image generation device 104 is instructed to perform a block motion compensation process. As a result, the position of the reference image block in the reference image is
Is instructed.

At STEP 371, it is determined whether or not the position designation of the reference image block with half-pixel accuracy is only in the horizontal direction. If only in the horizontal direction, the process proceeds to STEP 372; otherwise, the process proceeds to STEP 381.

In STEP 372, the image generation device 104 is instructed to perform a block motion compensation process. As a result, the position of the reference image block in the reference image is
Is instructed. The position of the reference image block indicates a position shifted to the left by 0.5 pixel from the position of half pixel precision designated for motion compensation.

In STEP 373, an image block setting process is instructed to the image generating device 104. Thus, it is instructed to move the generated image block to the block image memory 205 in order to use it as a difference image block for the next motion compensation processing.

At STEP 374, the image generation device 104 is instructed to perform a block motion compensation process. As a result, the position of the reference image block in the reference image is
Is instructed. The position of the reference image block indicates a position shifted to the right by 0.5 pixel as compared with the position of half pixel precision designated for motion compensation.

At STEP 381, it is determined whether or not the position designation of the reference image block with half-pixel accuracy is only in the vertical direction. If only in the vertical direction, the process proceeds to STEP 382; otherwise, the process proceeds to STEP 391.

At STEP 382, the image generation device 104 is instructed to perform a block motion compensation process. As a result, the position of the reference image block in the reference image is
Is instructed. The position of the reference image block indicates a position shifted by 0.5 pixel from the position of half pixel precision designated for motion compensation.

At STEP 383, an image block setting process is instructed to the image generating device 104. Thus, it is instructed to move the generated image block to the block image memory 205 in order to use it as a difference image block for the next motion compensation processing.

At STEP 384, the image generation device 104 is instructed to perform a block motion compensation process. As a result, the position of the reference image block in the reference image is
Is instructed. The position of the reference image block indicates a position shifted by 0.5 pixel from the position of half pixel precision designated for motion compensation.

At STEP 391, the image generation device 104 is instructed to perform a block motion compensation process. As a result, the position of the reference image block in the reference image is
Is instructed. The position of the reference image block indicates a position shifted by 0.5 pixel to the left by 0.5 pixel from the position of half pixel accuracy designated for motion compensation.

At STEP 392, an image block setting process is instructed to the image generating device 104. Thus, it is instructed to move the generated image block to the block image memory 205 in order to use it as a difference image block for the next motion compensation processing.

In STEP 393, the image generation device 104 is instructed to perform a block motion compensation process. As a result, the position of the reference image block in the reference image is
Is instructed. The position of the reference image block indicates a position shifted by 0.5 pixel to the right by 0.5 pixel as compared to the position of half pixel accuracy designated for motion compensation.

At STEP 394, an image block setting process is instructed to the image generating device 104. Thus, it is instructed to move the generated image block to the block image memory 205 in order to use it as a difference image block for the next motion compensation processing.

At STEP 395, the image generation device 104 is instructed to perform a block motion compensation process. As a result, the position of the reference image block in the reference image is
Is instructed. The position of the reference image block indicates a position shifted by 0.5 pixel to the left by 0.5 pixel to the left from the position of half pixel accuracy designated for motion compensation.

At STEP 396, an image block setting process is instructed to the image generation device 104. Thus, it is instructed to move the generated image block to the block image memory 205 in order to use it as a difference image block for the next motion compensation processing.

At STEP 397, the image generation device 104 is instructed to perform a block motion compensation process. As a result, the position of the reference image block in the reference image is
Is instructed. The position of the reference image block indicates a position shifted by 0.5 pixel to the right by 0.5 pixel below the position of half pixel accuracy designated for motion compensation.

The operation procedure of the image generating apparatus 104 controlled by the CPU 101 as described above to perform the motion compensation processing will be described with reference to FIGS. Here, the CPU 101
The operation procedure will be described for each type of operation instruction.

FIG. 5 shows an operation procedure when a block motion compensation process is instructed from the CPU 101 to the image generation device 104. In this processing, the instruction from the CPU 101 is analyzed by the control circuit 201 of the image generation device 104, and the pixel position designation circuit
At 202, a command signal for controlling the operation of the multiplier generation unit 207 of the pixel operation circuit 203 is output.

The pixel position specifying circuit 202 operates according to the following procedure.

In STEP 401, a pixel position (first pixel position) in the reference image block is obtained, and this pixel position is output to the image memory access circuit 204 as a first pixel position signal 20b.

At the same time, in STEP 402, a pixel position (third pixel position) in the difference image block is obtained, and this pixel position is set as a third pixel position signal 20d.
Output to 04.

At the same time, in STEP 403, a pixel position (second pixel position) in the generated image block is obtained, and this pixel position is used as a second pixel position signal 20c.
Output to 04.

At STEP 404, after the second image data is written at the second pixel position output at STEP 403, it is determined whether or not the processing of all the pixels in the generated image block has been completed. Otherwise, STEP 401, STEP 402, STEP 403
Processing proceeds to

The order in which the pixel position designating circuit 202 designates a pixel is usually focused on one image block among a generated image block, a difference image block, or a reference image block, and is placed on a horizontal line of the block, that is, on a scan line. Are sequentially specified. When the processing for one horizontal line is completed, pixels on the next horizontal line are sequentially specified. When the processing on the pixels on all the horizontal lines is completed, the processing on all the pixels in the image block is completed.

The image memory access circuit 204 operates according to the following procedure.

In STEP 411, the first pixel position signal 20b is input, the image data at the pixel position in the frame memory 109 is read out, and this image data is converted to the first image data signal.
Output as 20e.

In STEP 412, the second pixel position signal 20c and the second image data signal 20f are input, and the frame memory 109
The image data is written to the corresponding pixel position in.

At STEP 413, the third pixel position signal 20d is input, the image data at the pixel position in the block image memory 205 is read, and this image data is output as the third image data signal 20g.

The pixel operation circuit 203 operates according to the following procedure.

In STEP 421, the control circuit 201
The first multiplier and the second multiplier are output from the multiplier generation unit 207 as shown in (Table 1) according to the type of the motion compensation processing instructed via the.

[0219]

[Table 1]

In STEP 422, the first image data signal 20e
The first multiplier 209 multiplies the input image data by the first multiplier.

In STEP 423, the third image data signal 20g
Is multiplied by the second multiplier 208 by the image data input as the second multiplier.

In STEP 424, the addition section 210 adds the results of the first multiplication section 209 and the second multiplication section 208, and outputs the addition result as a second image data signal 20f.

When the CPU 101 instructs the image generating device 104 to perform image block setting processing, the instruction from the CPU 101 is analyzed by the control circuit 201 of the image generating device 104, and the operations of the pixel position specifying circuit 202 and the pixel moving circuit 206 are controlled. Command signal is output.

The pixel position designating circuit stores the positions of the difference image block, the reference image block, the generated image block, and the texture image for which the pixel position must be generated for the next motion compensation processing and the texture mapping processing described later. , The position of the generated polygon is set. The pixel position specifying circuit generates a pixel position according to the set image block position at the time of block motion compensation processing or texture mapping processing.

In the image moving circuit, the operation shown in FIG. 6 is performed to move the image block from the frame memory to the block image memory.

At STEP 501, a pixel position (fourth pixel position) in the generated image block is obtained, and this pixel position is output to the image memory access circuit 204 as a fourth pixel position signal 20h.

In STEP 502, a pixel position (fifth pixel position) in the block image memory is obtained, and this pixel position is
To the image memory access circuit 204 as the pixel position signal 20i.

At STEP 503, after the fifth image data is written at the fifth pixel position output at STEP 502, it is determined whether or not the processing of all the pixels in the generated image block has been completed. If not, the process proceeds to STEP 501 and STEP 502.

In STEP 511, the fourth pixel position signal 20h is input, the image data at the pixel position in the frame memory 109 is read, and this image data is converted to the fourth image data signal.
Output as 20j. The fourth image data signal 20j is output as a fifth image data signal 20k via the image moving circuit.

At STEP 512, the fifth pixel position signal 20i and the fifth image data signal 20k are input, and the image data is written to the corresponding pixel position in the block image memory 205.

According to the above procedure, the image generation device 104 of the present embodiment realizes the motion compensation processing.

According to the operation procedure at the time of the above-described motion compensation processing,
The operation of the multimedia device 100 during the forward motion compensation processing will be specifically described with reference to FIG. Here, for simplicity, the size of the image is 8 × 8 pixels, and the size of the image block for motion compensation is 4 × 4 pixels.

In FIG. 7, a past image 601 is a past image serving as a reference image for forward motion compensation. The past image block 602 is obtained by extracting a block serving as a reference for motion compensation from the past image. Difference image block 603
Is a difference image block for generating a generated image block of 4 × 4 pixels at the upper left of the generated image by motion compensation. The generated image block 604 is a generated image block obtained as a result of performing the forward motion compensation processing. The generated image 605 indicates an image to be generated by the decompression process.

In the following description, pixel positions and image block positions are represented by pixel position numbers assigned to the past image 601 and the generated image 605. For example, an image block of 4 × 4 pixels that is one pixel below the upper left corner of the past image (the block shown by a thick line in the figure) is an image block (0,1) −
Notated as (3,4). The pixels in the block are represented by coordinates in the block. That is, the past image block (0,1)-
The pixel at the upper left corner in (3,4) is denoted as pixel (0,0).

Hereinafter, the operations of the multimedia device 100 and the image generating device 104 will be described in detail.

The past image 601 is stored in the frame memory 109. Further, a storage area in which the generated image 605 is to be stored is also secured in the frame memory 109. CPU1
01 writes the difference image to the frame memory 109 (STEP 30
1).

Since the forward motion compensation processing is performed, the processing
Proceed to EP 322 (STEP 311 → STEP 321).

The CPU 101 instructs the image generation device 104 to perform an image block setting process, and
(0,1)-(3,4) is specified as the past image block 602, the image block (0,0)-(3,3) in the generated image 605 is specified as the generated image block 604, and the difference image block The position of 603 in the frame memory 109 is specified (STEP 322).

The processing shown in FIG. 4 is performed on the designated image block.

At STEP 351, the CPU 101 instructs the difference image block 603 to move to the block image memory as image block setting processing. Upon receiving this instruction, the control circuit 201 controls the image moving circuit 206 to move the difference image block 603 in the frame memory 109 to the block image memory 205 (FIG. 6).

Since the position designation of the past image block 602 (0,1)-(3,4) in the past image 601 does not deviate from the pixel existing position by a half pixel, motion compensation with half-pixel accuracy is not performed (S
TEP361).

The CPU 101 instructs the image generation device 104 to perform a block motion compensation process. The image generation device 104 receives this instruction and performs processing according to FIG.

The pixel position designating circuit 202 calculates a pixel position (first pixel position) (0,0) in the past image block 602, and outputs this pixel position as a first pixel position signal 20b. On the other hand, the pixel position in the difference image block 603 (the third
Pixel position) (0,0), output this pixel position as a third pixel position signal 20d, and calculate the pixel position (second pixel position) (0,0) in the generated image block 604. Then, this pixel position is output as the second pixel position signal 20c (ST
EP 401, STEP 402, STEP 40
3).

The first image memory access unit 212 in the image memory access circuit 204 receives the first pixel position signal 20b and receives the pixel at the pixel position (0,0) from the past image block 602 in the frame memory 109. Data is read out and this pixel data is output as a first pixel data signal 20e (STEP 4
11). The second image memory access unit 211 receives the third pixel position signal 20d, reads the pixel data at the pixel position (0,0) from the difference image block 603 in the block image memory 205, and It is output as the pixel data signal 20g of Step 3 (STEP 413).

In the multiplier generator 207 in the pixel operation circuit 203,
According to (Table 1), 1.0 is generated as the first multiplier,
Is generated as a multiplier of (STEP 421). First multiplication unit
In 209, the pixel data input by the first pixel data signal 20e is multiplied by the first multiplier, and the result is output to the adder 210 (STEP 422). The second multiplier 208 multiplies the pixel data input by the third pixel data signal 20g by the second multiplier and outputs the result to the adder 210 (STEP 423). The adder 210 adds the multiplication results of the first multiplier 209 and the second multiplier 208 and outputs the result as a second pixel data signal 20f.
(STEP 424).

Next, in the first image memory access unit 212, the second pixel position signal 20c and the second pixel data signal 20f
And writes this pixel data at the pixel position (0,0) in the generated image block 604 (STEP 412).

Until all the pixels in the generated image block 604 have been generated, the processing of FIG. 5 is repeated for each pixel in the generated image block 604 (STEP 404).

By performing such processing, a generated image block 604 is generated from the past image block 602 and the motion compensated difference image 603.

The operation of the backward motion compensation processing is different from the above-described forward motion compensation in that a future image is used instead of a past image as a reference image.

Next, the operations of the multimedia device 100 and the image generating device 104 during the bidirectional motion compensation processing will be specifically described with reference to FIG.

In FIG. 8, a past image 701 is a reference image for forward motion compensation. The future image 702 becomes a reference image for backward motion compensation. The past image block 703 is obtained by extracting a block serving as a reference for motion compensation from the past image. A future image block 704 is obtained by extracting a reference block for motion compensation from the future image. The difference image block 705 is 4 × 4 in the upper left of the generated image.
This is a block for generating a generated image block of pixels by motion compensation processing. The generated image block 706 is a block obtained as a result of performing the bidirectional motion compensation processing. Generated image
Reference numeral 707 indicates an image to be generated.

In the case of the bidirectional motion compensation processing, first, the same operations as those in STEP 322 and STEP 323 of the above-described forward motion compensation processing are performed by the CPU 101, and an operation instruction is issued to the image generation device 104 (STEP 341). → STEP 342). Upon receiving this instruction, the image generation device 104 operates according to the same processing procedure as the above-described forward motion compensation processing. However, as shown in Table 1, the multiplier generator
The first multiplier generated at 207 is “0.5”, and the second multiplier is “1.0”. As a result of performing the forward motion compensation processing, a result is obtained in which 0.5 times the pixel data of the past image block 703 and the pixel data of the difference image block 705 are added to the generated image block.

Then, in STEP 332 of the backward motion compensation processing,
The same operation as in STEP 333 is controlled by the CPU 101, and an operation instruction is issued to the image generation device 104 (STEP 343 → STEP 3
44).

In response to this instruction, the image generation device 104 operates in the same procedure as the backward motion compensation process. First, the generated image block generated by the forward motion compensation processing is moved to the block image memory as a differential image block (FIG. 6). Next, motion compensation processing is performed according to the procedure of FIG. However, as shown in Table 1, the first number generated by the multiplier generation unit 207
Is 0.5, and the second multiplier is 1.0. As a result of performing the backward motion compensation processing, a generated image block 706 is obtained.

Next, the process of half-pixel precision motion compensation will be described with reference to FIG.

The entire operation procedure of the forward motion compensation and the bidirectional motion compensation is as described above. However, if the designation of the position of the reference image block in the reference image is shifted by half a pixel from the existing position of the pixel, In order to obtain an image block, an interpolation process must be performed between a plurality of reference image blocks. FIG. 4 shows this processing procedure. Here, a description will be given assuming that the position is specified with half-pixel accuracy in both the vertical and horizontal directions.

The reference image for the motion compensation processing is the reference image.
If the block to be generated at this time is the generated image block 802, the position of the reference image block is designated by a bold line in the reference image 801 in order to make the data of the motion compensation difference image block as close to 0 as possible. It becomes the enclosed rectangular area (0.5,0.5)-(3.5,3.5). However, since the position designation of the reference image block is shifted by a half pixel from the pixel position, such a reference image block is
Cannot be read from Therefore, image blocks shifted by a half pixel from the designated position of the reference image block, that is, image blocks 803 (0,0)-(3,3), image blocks 804 (1,
0)-(4,3), image block 805 (0,1)-(3,4), image block
806 (1,1)-(4,4) is taken out and a reference image block is obtained by interpolating these four image blocks. When the forward or bidirectional motion compensation processing is performed on the reference image block and the motion compensation difference image block obtained as described above, a motion compensation processing with half-pixel accuracy can be realized.

In the present embodiment, interpolation and motion compensation are simultaneously performed on the four image blocks 803, 804, 805, and 806 by the following procedure.

According to the procedure already described (FIG. 4), the processing proceeds to STEP 391 in the motion compensation processing with half-pixel accuracy both vertically and horizontally. Here, STEP 391 and subsequent steps will be described in detail.

In STEP 391, an instruction to perform a motion compensation process on the reference image block 803 is issued to the image generation device 104. In the pixel operation circuit 203 of the image generation device 104, the first multiplier is “0.25” (in the case of bidirectional motion compensation, “0.25”).
125 ”) is generated, the pixel data in the reference image 803 is multiplied by a first multiplier, the pixel data of the difference image block is added thereto, and the result is written to the generated image block.

In STEP 392, the generated image block is moved to the block image memory 205 as a difference image block according to the procedure shown in FIG.

In STEP 393, an instruction to perform a motion compensation process on the reference image block 804 is issued to the image generation device 104. In the pixel operation circuit 203 of the image generation device 104, the first multiplier is “0.25” (in the case of bidirectional motion compensation, “0.25”).
125 ") is generated, the pixel data in the reference image 804 is multiplied by a first multiplier, the pixel data of the difference image block is added thereto, and the result is written to the generated image block.

In STEP 394, the generated image block is moved to the block image memory 205 as a difference image block according to the procedure shown in FIG.

In STEP 395, an instruction to perform a motion compensation process on the reference image block 805 is issued to the image generation device 104. In the pixel operation circuit 203 of the image generation device 104, the first multiplier is “0.25” (in the case of bidirectional motion compensation, “0.25”).
125 ") is generated, the pixel data in the reference image 805 is multiplied by a first multiplier, and the pixel data of the difference image block is added thereto, and written to the generated image block.

In STEP 396, the generated image block is moved to the block image memory 205 as a difference image block according to the procedure shown in FIG.

At STEP 397, an instruction to perform a motion compensation process on the reference image block 806 is issued to the image generation device 104. In the pixel operation circuit 203 of the image generation device 104, the first multiplier is “0.25” (in the case of bidirectional motion compensation, “0.25”).
125 ”) is generated, the pixel data in the reference image 806 is multiplied by a first multiplier, the pixel data of the difference image block is added thereto, and the result is written to the generated image block 802.

The motion compensation processing with half-pixel accuracy only in the vertical or horizontal direction can be realized by performing the same processing as the above processing as shown in FIG.

As described above, according to the present embodiment, it is possible to simultaneously perform the interpolation processing and the motion compensation processing for obtaining the reference image block, and realize the motion compensation processing with half-pixel accuracy.

In the following, the operation of the image generating apparatus 104 during the texture mapping processing will be described. CPU101
FIG. 10 shows a control procedure of the image generation apparatus 104 according to the first embodiment, and FIG. 11 shows an operation procedure of each circuit of the image generation apparatus 104. FIG.
In the figure, broken arrows indicate the flow of data / control signals, and solid arrows indicate the flow of processing.

First, the control procedure of the image generation device 104 by the CPU 101 when performing the texture mapping process is shown in FIG.
Explanation will be made using 0.

In STEP 901, the previously generated image in the frame memory 109 is deleted.

At STEP 902, the texture image is written into the frame memory 109.

In STEP 903, an image block setting process is instructed to the image generating device 104. With this instruction, the position (vertex position) of the generated polygon and the position of the texture image mapped to the generated polygon are specified. Further, the specified texture image is stored in the block image memory 205.
To move to.

At STEP 904, a texture mapping process is instructed to the image generating device 104.

In STEP 905, it is determined whether or not the texture mapping processing for all polygons in the generated image has been completed. If the processing has been completed, the processing proceeds to STEP 906, and if not, the processing proceeds to STEP 903 to perform processing for the next image block.

At STEP 906, it is determined whether or not the texture mapping process has been completed for all the images. If not, the process proceeds to STEP 901 to process the next image.

An operation procedure of the image generating apparatus 104 controlled by the CPU 101 as described above to perform the texture mapping process will be described with reference to FIGS. Here, the operation procedure will be described for each type of operation instruction by the CPU 101.

FIG. 11 shows an operation procedure when texture mapping processing is instructed from the CPU 101 to the image generating apparatus 104. In this process, an instruction from the CPU 101 is analyzed by the control circuit 201 of the image generation device 104, and a command signal for controlling the operations of the multiplier generation unit 207 of the pixel position designation circuit 202 and the pixel calculation circuit 203 is output.

The pixel position specifying circuit 202 operates according to the following procedure.

In STEP 1001, a pixel position (first pixel position) in the generated polygon is obtained, and this pixel position is output to the image memory access circuit 204 as a first pixel position signal 20b.

At the same time, in STEP 1002, the pixel position (third pixel position) of the texture to be mapped to the first pixel position is obtained, and this pixel position is output to the image memory access circuit 204 as a third pixel position signal 20d. I do.

At the same time, STEP 1003 sets the same pixel position (second pixel position) as the first pixel position to the second pixel position signal 20.
Output to the image memory access circuit 204 as c.

At STEP 1004, the second data output at STEP 1003
After the second image data is written at the pixel position of, it is determined whether or not the processing of all the pixels in the generated image block has been completed. If the processing has not been completed, STEP 1001, STEP 1002, and STEP
Processing proceeds to 1003.

The order in which the pixel position designating circuit 202 designates pixels usually focuses on either the generated polygon or the texture image block, and sequentially designates the horizontal line of the block or polygon, that is, the pixels on the scan line. When the processing for one horizontal line is completed,
Pixels on the next horizontal line are sequentially specified. When the processing on the pixels on all the horizontal lines is completed, the processing on all the pixels in the image block is completed.

The image memory access circuit 204 operates according to the following procedure.

In STEP 1011, it is determined whether the translucent texture image is a pixel to be subjected to texture mapping processing or anti-aliasing. If translucent or anti-aliasing is to be performed, the process proceeds to STEP 1012, and the translucent processing is performed. If no anti-aliasing is performed, proceed to STEP 1013.

In STEP 1012, the first pixel position signal 20b is input, the image data at the pixel position in the frame memory 109 is read, and this image data is read as the first image data signal.
Output as 20e.

In STEP 1013, the second pixel position signal 20c and the second image data signal 20f are input, and the frame memory 109
The image data is written to the corresponding pixel position in.

At STEP 1014, the third pixel position signal 20d is input, the image data at the pixel position in the block image memory 205 is read, and this image data is output as the third image data signal 20g.

The pixel operation circuit 203 operates according to the following procedure.

In STEP 1021, the CPU 101 sends the control circuit 20
The translucency α (1 for opaque, 0 for transparent) in the texture mapping process specified via 1 and the translucent coefficient β for edge anti-aliasing (the pixel being processed constitutes the edge of the polygon If not, β = 1) is calculated, αβ is output to the second multiplier as a second multiplier, and (1−αβ) is output to the first multiplier as a first multiplier.

At STEP 1022, the first image data signal 20e
The first multiplier 209 multiplies the input image data by the first multiplier.

In STEP 1023, the third image data signal 20g
Is multiplied by the second multiplier 208 by the image data input as the second multiplier.

In STEP 1024, the addition section 210 adds the results of the first multiplication section 209 and the second multiplication section 208, and outputs the addition result as a second image data signal 20f.

When the CPU 101 instructs the image generating device 104 to perform the image block setting process, it is as described in the motion compensation process. When the movement of the texture image is instructed in the image block setting processing, the operation shown in FIG. 6 is performed in the image movement circuit, and the texture image is moved from the frame memory to the block image memory.

At STEP 501, a pixel position (a fourth pixel position) in the texture image is obtained, and this pixel position is output to the image memory access circuit 204 as a fourth pixel position signal 20h.

In STEP 502, a pixel position (fifth pixel position) in the block image memory is determined, and this pixel position is
To the image memory access circuit 204 as the pixel position signal 20i.

At STEP 503, after the fifth image data has been written at the fifth pixel position output at STEP 502, it is determined whether or not the processing of all the pixels in the generated image block has been completed. If not, the process proceeds to STEP 501 and STEP 502.

In STEP 511, the fourth pixel position signal 20h is input, the image data of the pixel position in the frame memory 109 is read, and this image data is read as the fourth image data signal.
Output as 20j. The fourth image data signal 20j is output as a fifth image data signal 20k via the image moving circuit.

In STEP 512, the fifth pixel position signal 20i and the fifth image data signal 20k are input, and the image data is written to the corresponding pixel position in the block image memory 205.

[0301] By the above procedure, the image generating apparatus of the present embodiment realizes the texture mapping processing.

According to the above-described texture mapping operation procedure, the image generating apparatus 104 during the texture mapping processing
This operation will be specifically described with reference to FIG. 12 and FIG. 10 and FIG. In FIG. 12, an image 1101 represents a texture image. Polygon 1102 texture image
1101 represents a polygon to be pasted. The pixel 1103 is a pixel constituting the polygon 1102. Pixel 1104 is pixel 11
Pixels constituting a texture image to be pasted on 03. The pixel 1105 represents a pixel forming an edge of the polygon 1102. The pixel 1106 represents a pixel of the texture image corresponding to the pixel 1105.

The operation is as follows.

[0304] The texture image 1101 is stored in the frame memory 109.
(STEP 902).

The CPU 101 instructs the image generating device 104 to perform an image block setting process, and determines the polygon vertex coordinates and the translucency of the texture (here, the translucency of the entire texture is 0.6).
Is set in the pixel position specifying circuit 202 and the multiplier generation unit 207 of the pixel operation circuit 203 via the control circuit 201. Also,
Instruct the image moving circuit 206 to move the texture image (S
TEP 903).

The image transfer circuit 206 sequentially outputs the pixel position of the texture image in the frame memory 109 as the fourth pixel position to the first image memory access unit 212 of the image memory access circuit 204 (STEP 501). The first image memory access unit 212 reads the texture image data from the frame memory 109 and outputs it as fourth pixel data (STEP 511). The fourth pixel data is input to the image shift circuit, and this data is output as fifth pixel data. At the same time, the storage pixel position of the texture image in the block image memory is sequentially output from the image transfer circuit as the fifth pixel position (STEP 502). The fifth pixel position and the fifth pixel data are input to the second image memory access unit of the image memory access circuit, and the fifth image data is written to the fifth pixel position of the block image memory (STEP 512) .
By repeating this operation for all pixels in the texture image (STEP 503), the texture image is moved to the block image memory.

Next, the CPU 101 instructs the image generating device 104 to perform a texture mapping process. The image generation device receives this instruction and performs texture mapping according to the procedure of FIG.

In the pixel position specifying circuit 202, the generated polygon 1
The pixel position (first pixel position) in 102 is sequentially calculated, and this pixel position is output as a first pixel position signal 20b.
Now, assume that a pixel 1103 is generated. The pixel position of the pixel 1103 is output as the first pixel position. On the other hand, the pixel position (third pixel position) of the pixel 1104 in the texture image 1101 mapped to the pixel 1103 is calculated, and this pixel position is
Is output as the pixel position signal 20d. Further, the same pixel position as the first pixel position is output as the second pixel position signal 20c. (STEP 1001, STEP 1002 and STEP 1003) To perform translucent texture mapping, the first image memory access unit 212 in the image memory access circuit 204
The first pixel position signal 20b is input, pixel data is read from the generated polygon 1102 in the frame memory 109, and this pixel data is output as a first pixel data signal 20e.
This pixel data becomes background pixel data (STEP 1012).
The second image memory access unit 211 receives the third pixel position signal 20d, reads pixel data from the texture image 1101 in the block image memory 205, and outputs the pixel data as a third pixel data signal 20g. (STEP101
Four).

[0309] In the multiplier generation unit 207 in the pixel operation circuit 203,
According to the translucency of 0.6, a first multiplier of 0.4 is generated and a second multiplier of 0.6 is generated (STEP 1021). The first multiplier 209 multiplies the pixel data input by the first pixel data signal 20e by the first multiplier and adds the result to the adder.
Output to 210 (STEP 1022). The second multiplication unit 208 multiplies the pixel data input by the third pixel data signal 20g by the second multiplier and outputs the result to the addition unit 210 (STEP 1).
023). The adder 210 includes a first multiplier 209 and a second multiplier 209.
The multiplication result of 208 is added and output as the second pixel data signal 20f (STEP1024). As a result, 0.6 × (texture pixel data) + 0.4 × (background pixel data) is output as the second pixel data.

Next, in the first image memory access unit 212, the second pixel position signal 20c and the second pixel data signal 20f
And writes this pixel data at the pixel position of the pixel 1103 in the generated polygon 1102 (STEP 1013). In this way, the pixel data 1104 of the texture image is texture-mapped.

Next, consider the case where a pixel 1105 forming the edge of the generated polygon 1102 is generated. Here, in the pixel 1105, 70% of the pixel area is inside the polygon 1102, and 3
It is assumed that 0% is outside the polygon. In this case, the multiplier generator
In 207, the first multiplier is (1−0.6 × 0.7) = 0.
58 is output as a second multiplier, 0.6 × 0.7 = 0.42, and as a result of operation in the pixel operation circuit, 0.42 × (texture pixel data) +
0.58 × (background pixel data) is output, and this pixel data is written to the pixel 1105 in the generated polygon 1102.

The process of FIG. 11 is repeated for each pixel in the generated polygon 1102 until all the pixels in the generated polygon 1102 are generated (STEP 1004).

By performing the processing as described above, it is possible to realize the texture mapping of the texture image 1101 to the generated polygon 1102.

As described above, according to the present embodiment, the texture mapping of the translucent texture image can be performed including the anti-aliasing processing of the polygon edge portion.

Next, the operation of the image generating apparatus 104 when simultaneously executing the motion compensation processing and the texture mapping processing will be described. 13 and 14 show a control procedure of the image generation device 104 by the CPU 101, and FIG. 15 shows an operation procedure of each circuit of the image generation device 104. In FIG. 15, the dashed arrows indicate the flow of data / control signals, and the solid arrows indicate the flow of processing. In FIGS. 13, 14 and 15, when the same processing as the above-described motion compensation processing (described with reference to FIGS. 3, 4 and 5) is performed, the same processing procedure number is assigned.

The control procedure of the image generation device 104 by the CPU 101 when the motion compensation process and the texture mapping process are executed simultaneously will be described with reference to FIGS. 13 and 14. The description of the steps that take the same procedure as in the case where the motion compensation processing is executed alone is omitted.

When the processing proceeds to STEP 1201, it is determined whether or not the image to be subjected to motion compensation, that is, the generated image, is used as a reference image in the motion compensation processing of the subsequent image.

When used as a reference image, STEP 1
Proceed from step 211 to step 1213. After completing the motion compensation processing (FIG. 4) of the image block currently being processed (STEP 1211), after generating the generated image block, the generated image block is moved to the block image memory 205 for use as a texture image. (STEP 1212), and instructs the image generation device 104 to execute the texture mapping process (STEP 121).
3). When it is necessary to generate a generated image for the subsequent motion compensation processing, the generated image block is generated in this way, and then the texture mapping processing is performed. When compression-encoded by MPEG, the expanded image of an I picture is used for motion compensation of P and B pictures, and the expanded image of a P picture is used for motion compensation of other P and B pictures. In some cases, the above-described processing procedure is applied to such an image. The operation procedure of the image generation device 104 corresponding to this processing procedure is the same as that in FIG. 5 (motion compensation processing) and FIG. 11 (texture mapping processing).

On the other hand, when the image resulting from the motion compensation processing is not used for the subsequent motion compensation processing, such as an MPEG B picture, the motion compensation processing and the texture mapping processing are executed according to the procedure of STEP 1202 (FIG. 14). I do. In FIG. 14, when the same procedure as that of FIG. In the processing procedure of FIG. 14, when a plurality of block motion compensation processes are required, the process is the same as that of FIG. 4 except for the last one motion compensation process. In FIG. 14, the texture mapping process is performed simultaneously with the last block motion compensation process. STEP 1251, STEP 1261, STEP 1
In steps 271 and 1281, the CPU 101 instructs an image block setting process. In this step, the position of the generated polygon is specified, and STEP 1261, STEP 1271 and STEP
In 1281, the generated image block is stored in the block image memory 205.
Move to Thereafter, an instruction is given to simultaneously execute the block motion compensation processing and the texture mapping. The operation procedure of the image generating apparatus 104 when the CPU 101 instructs the image generating apparatus 104 to perform the image block setting process for moving the generated image block to the block image memory 205 is shown in FIG.
As described above.

The operation procedure of the image generating device 104 when the CPU 101 instructs the image generating device 104 to simultaneously execute the block motion compensation processing and the texture mapping will be described with reference to FIG. In the operation procedure of FIG. 15, the same steps as those in the description of FIG. 5 are given the same numbers.

In this process, the instruction from the CPU 101 is analyzed by the control circuit 201 of the image generation device 104, and a command signal for controlling the operation of the multiplier generation unit 207 of the pixel position designation circuit 202 and the pixel calculation circuit 203 is output.

The pixel position specifying circuit 202 operates according to the following procedure.

In STEP 401, a pixel position (first pixel position) in the reference image block is obtained, and this pixel position is output to the image memory access circuit 204 as a first pixel position signal 20b.

At the same time, in STEP 402, the pixel position (third pixel position) in the difference image block is obtained, and this pixel position is set as a third pixel position signal 20d.
Output to 04.

At the same time, in STEP 1301, a pixel position (second pixel position) in the generated polygon is obtained, and this pixel position is used as a second pixel position signal 20c.
Output to

In STEP 404, the second data output in STEP 1301 is output.
After the second image data is written at the pixel position of, it is determined whether or not the processing of all the pixels in the generated polygon has been completed. If not completed, STEP 401, STEP 402, STEP 130
Proceed to step 1.

The image memory access circuit 204 operates according to the following procedure.

In STEP 411, the first pixel position signal 20b is input, the image data at the pixel position in the frame memory 109 is read, and the image data is read out from the first image data signal.
Output as 20e.

At STEP 412, the second pixel position signal 20c and the second image data signal 20f are inputted, and the frame memory 109
The image data is written to the corresponding pixel position in.

In STEP 413, the third pixel position signal 20d is input, the image data at the pixel position in the block image memory 205 is read, and this image data is output as the third image data signal 20g.

The pixel operation circuit 203 operates according to the following procedure.

In STEP 421, the CPU 101 sends the control circuit 201
The first multiplier and the second multiplier are output from the multiplier generation unit 207 as shown in (Table 1) according to the type of the motion compensation processing instructed via the.

In STEP 422, the first image data signal 20e
The first multiplication unit 209 performs multiplication using the image data and the first multiplier input as.

At STEP 423, the third image data signal 20g
The second multiplication unit 208 performs multiplication using the image data and the second multiplier input as.

In STEP 424, the addition unit 210 adds the result of the first multiplication unit 209 and the result of the second multiplication unit 208,
The result of the addition is output as a second image data signal 20f.

According to the above procedure, the image generating apparatus of this embodiment realizes the simultaneous execution of the motion compensation processing and the texture mapping.

In the following, the simultaneous execution of the motion compensation process and the texture mapping process will be described with reference to FIG. 16 (STEP 125 in FIG. 14).
2, STEP 1262, STEP 1272, STEP 1282, FIG. 15) will be described.

The reference image 1401 is stored in the frame memory 109.
Is stored, and the image block to be subjected to the motion compensation processing is set in the pixel position specifying circuit 202 as the reference image block 1402. On the other hand, the difference image 1403 in the frame memory 109
Is set, and the image data of this image block is moved to the block image memory 205. (STEP 1251, STEP 1261, STEP 1271, STEP 1281, FIG. 6) The pixel position designation circuit 202 maps the second pixel position signal 20c indicating the pixel position in the generated polygon 1405 to the second pixel position. The pixel position in the reference image block 1402 and the pixel position in the difference image block 1404 are output as a first pixel position signal 20b and a third pixel position signal 20d. The pixel data at the first pixel position is read from the frame memory 109 via the first image memory access unit 212 (first pixel data), while the pixel data at the third pixel position is stored in the second image memory. The data is read from the block image memory 205 via the access unit 211 (third pixel data). The first pixel data and the third pixel data are respectively multiplied by the multipliers shown in (Table 1) in the pixel operation circuit 203, the respective multiplication results are added, and the addition result is output as the second pixel data. By this operation, pixel data resulting from the motion compensation processing is obtained as second pixel data. The second pixel data is generated polygon 1405
Is written in the pixel position in the frame memory 109 indicated by the second pixel position signal indicating the pixel position in the frame. The second pixel position indicates a pixel position in the generated polygon 1405 to be texture-mapped, and the texture mapping process is achieved by writing the second pixel data. As described above, the motion compensation process and the texture mapping process can be performed simultaneously.

According to the present embodiment, as described above,
A block image memory 205 for storing texture images, motion compensation difference images, and the like;
An image memory access circuit 204 for accessing pixel data of a frame memory 109 in which a reference image, a generated polygon, and a generated image are stored; third pixel data read from a block image memory 205 and read from the frame memory 109 A pixel operation circuit 203 that performs an operation on the obtained first pixel data between the pixels, and a motion compensation process, a texture mapping process, and a simultaneous execution of the motion compensation process and the texture mapping process. By providing a pixel position specifying circuit 202 for generating and specifying a pixel position and a pixel position for writing pixel data of a calculation result, and an image moving circuit 206 for moving an image block in the frame memory 109 to the block image memory 205, Compressed using motion prediction, such as directional motion compensation, bidirectional motion compensation, half-pixel accuracy motion compensation, etc. And expansion processing of the video,
It is possible to realize a texture mapping process including a texture mapping of a translucent texture image and an anti-aliasing process of polygon edges, and a simultaneous execution of a motion compensation process and a texture mapping process.

Although the size of the image block for motion compensation is set to 4 × 4 pixels for the sake of simplicity, the present invention is not limited to this.
It does not exclude the use of other sizes such as × 8 pixels. Further, the size of the image is set to 8 × 8 pixels, but any size can be selected. Furthermore, the processing order of the image blocks described in the procedure of the motion compensation processing according to the present embodiment is not limited to this, and the processing order can be interchanged with each other. Further, in the description of the texture mapping processing of the present embodiment, the translucency of the translucent texture is described as one type, but it is also possible to give different translucency to each pixel constituting the texture.

(Embodiment 2) In the configuration and operation of a multimedia device according to a second embodiment of the present invention, the description of the same parts as in Embodiment 1 will be omitted.

The configuration of the image generating apparatus 104 according to the present embodiment will be described with reference to FIG. The same components as those of the image generating apparatus according to the first embodiment have the same numbers.

The control circuit 201 interprets the command specified by the command input signal 20a, and outputs the pixel position specifying circuit 1501,
This is a control circuit that outputs a command signal for controlling operations of the image moving circuit 206 and the pixel operation circuit 203. The pixel position specifying circuit 1501 follows the instruction of the control circuit 201, and sets the pixel position (first pixel position) in the reference image block or the generated polygon.
And outputs it as a first pixel position signal 20b, obtains a pixel position (second pixel position) in a generated image block or a generated polygon and outputs it as a second pixel position signal 150a,
Pixel position in the difference image block or texture image (third
Is obtained and output as a third pixel position signal 20d. The image memory access circuit 1502 receives the first pixel position signal 20b, reads out the pixel data at the specified pixel position in the frame memory 109, outputs it as the first pixel data signal 20e, and receives the second pixel position signal 150a. When the second pixel position signal 150a points to a pixel position in the frame memory 109, the pixel data input as the second pixel data signal 150b is written to a designated pixel position in the frame memory 109, and the fourth pixel position A first image memory access unit 1504 which receives a signal 20h and reads out pixel data at a specified pixel position in the frame memory 109 and outputs it as a fourth pixel data signal 20j, and a block image memory which receives a third pixel position signal 20d The pixel data at the designated pixel position in 205 is read and output as a third pixel data signal 20g, and the second pixel position signal 150a is received in response to the second pixel position signal 150a. When pointing to a pixel position in the image memory 205, the pixel data input as the second pixel data signal 150b is written to a designated pixel position in the block image memory 205, and the block image memory receives the fifth pixel position signal 20i. And a second image memory access unit 1503 for writing the pixel data input as the fifth pixel data signal 20k at a designated pixel position in the above.

The pixel operation circuit 203 receives the first pixel data signal 20e and the third pixel data signal 20g, and outputs the result of operation of these data as a second pixel data signal 150b. In the present embodiment, the pixel operation circuit 203 generates a first multiplier and a second multiplier according to an instruction (command signal) of the control circuit 201, and outputs a multiplier and a multiplier to the multiplier.
The first multiplier 209 multiplies the first pixel data specified by the multiplier generator 207 with the first pixel data, and the third multiplier multiplies the second multiplier specified by the multiplier generator 207 with the third pixel data. The operation result of the second multiplier 208 and the first multiplier 209 is
And an adder 210 for adding the operation result of the multiplier 208.

When the instruction (command signal) from the control circuit 201 indicates a motion compensation process,
Alternatively, when the simultaneous processing of the motion compensation processing and the texture mapping processing is instructed, the calculation is performed with signed data precision, and when the texture mapping processing is instructed, the calculation is performed with unsigned data precision. Do.

The image moving circuit 206 generates a fourth pixel position and a fifth pixel position for moving an image block in the frame memory 109 to the block image memory 205 in accordance with an instruction (command signal) of the control circuit 201. Let it. The pixel data specified by the fourth pixel position signal 20h is read from the frame memory 109 via the image memory access circuit 1502, and the image data is read from the pixel position in the block image memory 205 specified by the fifth pixel position signal 20i. Via the image memory access circuit 1502.

[0347] The image generating apparatus 104 configured as described above.
The operation of will be described below. The image generation device 104 is a CP
The operation is instructed by a command from U101, and the control circuit 2
The operation is performed by interpreting the command at 01 and outputting a command for controlling each circuit of the image generating device 104 according to the command. In the following description, a control procedure by issuing a command to the control circuit 201 by the CPU 101 and an operation procedure of each circuit in the image generating apparatus 104 under the control will be described.

First, the image generating apparatus 104 according to the present embodiment
The operation at the time of the motion compensation processing will be described. FIGS. 18 and 19 show a control procedure of the image generation device 104 by the CPU 101.
FIG. 20 shows an operation procedure of each circuit of the image generation device 104.
18, FIG. 19 and FIG. 20, FIG.
The same processing steps as those in FIGS. 4 and 5 are given the same numbers.

First, the CPU 10 performs the motion compensation processing.
FIG. 18 and FIG.
9 will be described.

In STEP 301, a reference image (past image or future image) as a reference for motion compensation is set in the frame memory 109, and a difference image is written. Reference image is disk device 10
3 and written into the frame memory 109 by the CPU 101, and an image that has already been generated by the image generating device 104.

At STEP 311, it is determined whether or not to perform the motion compensation process. If no motion compensation process is performed, the process proceeds to STEP 1601; otherwise, the process proceeds to STEP 321.

In STEP 1601, an image block setting process is instructed to the image generating device 104. Thereby, the position of the generated image block in the block image memory and the position of the difference image block in the difference image are indicated.

In STEP 1602, block motion compensation processing is controlled according to FIG.

In STEP 321, it is determined whether or not the process is a forward motion compensation process.
The process proceeds to step 611; otherwise, the process proceeds to step 331.

[0355] In STEP 1611, an image block setting process is instructed to the image generating device 104. With this instruction, the position of the past image block in the past image, the position of the generated image block in the block image memory, and the position of the difference image block in the difference image are indicated.

At STEP 1612, block motion compensation processing is controlled according to FIG.

In STEP 331, it is determined whether or not the process is a backward motion compensation process.
The process proceeds to step 621; otherwise, the process proceeds to step 341.

[0358] In STEP 1621, an image block setting process is instructed to the image generating device 104. With this instruction, the position of the future image block in the future image, the position of the generated image block in the block image memory, and the position of the difference image block in the difference image are indicated.

At STEP 1622, the motion compensation processing of the block is controlled according to FIG.

In STEP 1631, an image block setting process is instructed to the image generating device 104. With this instruction, the position of the past image block in the past image, the position of the generated image block in the block image memory, and the position of the difference image block in the difference image are indicated.

At STEP 1632, the motion compensation processing of the block is controlled according to FIG.

At STEP 1633, an image block setting process is instructed to the image generating device 104. With this instruction, the position of the future image block in the future image, the position of the generated image block in the block image memory, and the position of the difference image block in the difference image are indicated.

At STEP 1634, block motion compensation processing is controlled according to FIG.

[0364] In STEP 302, it is determined whether or not the motion compensation processing for all image blocks in the generated image has been completed. If completed, proceed to STEP 303; if not completed
The process proceeds to STEP 311, and the process is performed for the next image block.

In STEP 303, it is determined whether or not the motion compensation processing for all the images has been completed.
The processing advances to 301, and processing is performed for the next image.

Next, referring to FIG. 19, the image generating apparatus 104 by the CPU 101 for performing the motion compensation processing of the image block.
Will be described in detail.

At STEP 351, the image generation device 104 is instructed to perform an image block setting process. Thereby, the difference image block to be processed next is moved from the difference image in the frame memory 109 to the block image memory 205.

At STEP 361, it is determined whether or not motion compensation with half-pixel accuracy is to be performed. If motion compensation with half-pixel accuracy is not performed, the process proceeds to STEP 1651. If motion compensation with half-pixel accuracy is performed, the process proceeds to STEP 371.

At STEP 1651, an image block setting process is instructed, and the generated image in the frame memory 109 is designated as a generated image block generation position.

At STEP 362, the image generation device 104 is instructed to perform a block motion compensation process. As a result, the position of the reference image block in the reference image is
Is instructed.

In STEP 371, it is determined whether or not the position designation of the reference image block with half-pixel accuracy is only in the horizontal direction. If only in the horizontal direction, the process proceeds to STEP 372; otherwise, the process proceeds to STEP 381.

At STEP 372, the image generation device 104 is instructed to perform a block motion compensation process. As a result, the position of the reference image block in the reference image is
Is instructed. The position of the reference image block indicates a position shifted to the left by 0.5 pixel from the position of half pixel precision designated for motion compensation.

In STEP 1661, an image block setting process is instructed to the image generating device 104. Thereby, the inside of the generated image in the frame memory 109 is specified as the generation position of the generated image block.

At STEP 374, the image generation device 104 is instructed to perform a block motion compensation process. As a result, the position of the reference image block in the reference image is
Is instructed. The position of the reference image block indicates a position shifted to the right by 0.5 pixel as compared with the position of half pixel precision designated for motion compensation.

At STEP 381, it is determined whether or not the position designation of the reference image block with half-pixel accuracy is only in the vertical direction. If only in the vertical direction, the process proceeds to STEP 382; otherwise, the process proceeds to STEP 391.

At STEP 382, the image generation device 104 is instructed to perform a block motion compensation process. Thereby, the position of the reference image block in the reference image is determined by the image generation device 104.
To instruct. The position of the reference image block indicates a position shifted by 0.5 pixel from the position of half pixel precision designated for motion compensation.

[0377] In STEP 1671, an image block setting process is instructed to the image generating device 104. Thereby, the inside of the generated image in the frame memory 109 is specified as the generation position of the generated image block.

In STEP 384, the image generation device 104 is instructed to perform a block motion compensation process. Thereby, the position of the reference image block in the reference image is determined by the image generation device 104.
To instruct. The position of the reference image block indicates a position shifted by 0.5 pixel from the position of half pixel precision designated for motion compensation.

In STEP 391, the image generation device 104 is instructed to perform a block motion compensation process. As a result, the position of the reference image block in the reference image is
Is instructed. The position of the reference image block indicates a position shifted by 0.5 pixel to the left by 0.5 pixel from the position of half pixel accuracy designated for motion compensation.

In STEP 393, the image generation device 104 is instructed to perform a block motion compensation process. Thereby, the position of the reference image block in the reference image is determined by the image
To instruct. The position of the reference image block indicates a position shifted by 0.5 pixel to the right by 0.5 pixel as compared to the position of half pixel accuracy designated for motion compensation.

In STEP 395, the image generation device 104 is instructed to perform a block motion compensation process. Thereby, the position of the reference image block in the reference image is determined by the image
To instruct. The position of the reference image block indicates a position shifted by 0.5 pixel to the left by 0.5 pixel to the left from the position of half pixel accuracy designated for motion compensation.

In STEP 1681, the image generation device 104 is instructed to perform an image block setting process. Thereby, the inside of the generated image in the frame memory 109 is specified as the generation position of the generated image block.

At STEP 397, the image generation device 104 is instructed to perform a block motion compensation process. Thereby, the position of the reference image block in the reference image is determined by the image generation device 104.
To instruct. The position of the reference image block indicates a position shifted by 0.5 pixel to the right by 0.5 pixel below the position of half pixel accuracy designated for motion compensation.

An operation procedure of the image generating apparatus 104 controlled by the CPU 101 as described above to perform the motion compensation processing will be described with reference to FIG.

FIG. 20 shows the operation procedure when the CPU 101 instructs the image generation device 104 to perform the block motion compensation processing.
Shown in In this processing, the instruction from the CPU 101 is analyzed by the control circuit 201 of the image generation device 104, and a command signal for controlling the operations of the pixel position designation circuit 1501 and the multiplier generation unit 207 of the pixel calculation circuit 203 is output.

The pixel position designation circuit 1501 operates according to the following procedure.

In STEP 401, a pixel position (first pixel position) in the reference image block is obtained, and this pixel position is output to the image memory access circuit 1502 as a first pixel position signal 20b.

At the same time, in STEP 402, a pixel position (third pixel position) in the difference image block is obtained, and this pixel position is set as a third pixel position signal 20d.
Output to 502.

At the same time, in STEP 1701, a pixel position (second pixel position) in the generated image block is obtained, and this pixel position is output to the image memory access circuit 1502 as a second pixel position signal 150a. The second pixel position indicates a pixel position in the frame memory 109 or a pixel position in the block image memory as instructed by the pixel block setting process.

At STEP 404, after the second image data is written at the second pixel position output at STEP 403, it is determined whether or not the processing of all the pixels in the generated image block has been completed. Otherwise, STEP 401, STEP 402, STEP 170
Proceed to step 1.

The image memory access circuit 1502 operates according to the following procedure.

First, the first image memory access unit operates as follows.

In STEP 411, the first pixel position signal 20b is input, the image data at the pixel position in the frame memory 109 is read, and this image data is read as the first image data signal.
Output as 20e.

In STEP 1711, the second pixel position signal 150a
Is input, and it is determined whether or not this pixel position indicates the inside of the frame memory 109. If the second pixel position indicates the inside of the frame memory 109, the procedure proceeds to STEP 412;
Processing proceeds to EP 411.

In STEP 412, the second pixel position signal 150a and the second image data signal 150b are input, and the frame memory 10
The image data is written to the corresponding pixel position in 9.

The second image memory access unit operates as follows.

In STEP 413, the third pixel position signal 20d is input, the image data at the pixel position in the block image memory 205 is read, and this image data is output as the third image data signal 20g.

In STEP 1712, the second pixel position signal 150a
Is input, and it is determined whether or not this pixel position indicates the inside of the block image memory 205. The second pixel position is a block image memory
If it indicates 205, the procedure proceeds to STEP 1713; otherwise, the procedure proceeds to STEP 413.

In STEP 1713, the second pixel position signal 150a
And the second image data signal 150b are input, and the image data is written to the pixel position in the block image memory 205.

The pixel operation circuit 203 operates according to the following procedure.

At STEP 421, the control circuit 201
The first multiplier and the second multiplier are output from the multiplier generation unit 207 as shown in (Table 1) according to the type of the motion compensation processing instructed via the.

In STEP 422, the first image data signal 20e
The first multiplier 209 multiplies the input image data by the first multiplier.

In STEP 423, the third image data signal 20g
Is multiplied by the second multiplier 208 by the image data input as the second multiplier.

In STEP 424, the addition section 210 adds the results of the first and second multiplication sections 209 and 208, and outputs the addition result as a second image data signal 150b.

When the CPU 101 instructs the image generating apparatus 104 to perform image block setting processing, the instruction from the CPU 101 is analyzed by the control circuit 201 of the image generating apparatus 104, and the operations of the pixel position specifying circuit 1501 and the pixel moving circuit 206 are controlled. Is done.

The pixel position designating circuit 1501 includes the positions of a difference image block, a reference image block, and a generated image block for which pixel positions must be generated for the next motion compensation processing and texture mapping processing (the procedure will be described later).
Alternatively, the positions of the texture image and the generated polygon are set. The pixel position specifying circuit 1501 generates a pixel position according to the set image block position at the time of the block motion compensation processing and the texture mapping processing. In the present embodiment, the position of the generated image block may be specified not only in the frame memory 109 but also in the block image memory 205. Accordingly, the pixel position in the frame memory 109 is set as the second pixel position. Whether to generate or generate a pixel position in the block image memory 205 is controlled.

In the image moving circuit 206, the operation shown in FIG. 6 is performed, and the image block is moved from the frame memory 109 to the block image memory 205. The details of the operation are as described in the first embodiment, and a description thereof will not be repeated.

With the above procedure, the image generating apparatus 104 according to the present embodiment realizes the motion compensation processing.

The operation of the forward motion compensation process will be specifically described with reference to FIG. 7 according to the operation procedure of the above-described motion compensation process. Here, the size of the image is set to 8 × 8 for simplicity.
4 × 4 pixels, size of image block for motion compensation
Pixels.

The operation is as follows.

[0411] The past image 601 is stored in the frame memory 109. Further, a storage area in which the generated image 605 is to be stored is also secured in the frame memory 109. CPU1
01 writes the difference image to the frame memory 109 (STEP 30
1).

Since the forward motion compensation processing is performed, the processing
Proceed to EP 1611 (STEP 311 → STEP 321).

[0413] The CPU 101 instructs the image generating device 104 to perform the image block setting process, and
(0,1)-(3,4) is specified as the past image block 602, the block image memory is specified as the generated image block, and the position of the difference image block 603 in the frame memory 109 is specified (STEP 322). .

The processing shown in FIG. 19 is performed on the designated image block.

[0415] At STEP 351, the CPU 101 instructs the difference image block 603 to move to the block image memory as image block setting processing. Upon receiving this instruction, the control circuit 201 controls the image moving circuit 206 to move the difference image block 603 in the frame memory 109 to the block image memory 205 (FIG. 6).

Since the position designation of the past image block 602 (0,1)-(3,4) in the past image 601 does not deviate from the pixel existing position by a half pixel, motion compensation with half-pixel accuracy is not performed (S
TEP361).

[0417] The CPU 101 instructs the image generating apparatus 104 to perform an image block setting process (STEP 1651). With this instruction, the image generation device 104 specifies the position of the generated image block 604 in the generated image 605.

[0418] The CPU 101 instructs the image generation device 104 to perform the block motion compensation processing (STEP 1652). Image generation device 10
4 receives this instruction and performs processing according to FIG.

[0419] The pixel position designation circuit 1501 calculates a pixel position (first pixel position) (0, 0) in the past image block 602, and outputs this pixel position as a first pixel position signal 20b. On the other hand, the pixel position in the difference image block 603 (the third
Pixel position) (0,0), output this pixel position as a third pixel position signal 20d, and calculate the pixel position (second pixel position) (0,0) in the generated image block 604. Then, this pixel position is output as the second pixel position signal 150a (STEP 40).
1, STEP 402, STEP 1701).

In the first image memory access unit 1504 in the image memory access circuit 1502, the first pixel position signal 20b
Is input, and the past image block 602 in the frame memory 109 is input.
From the pixel position (0,0), and outputs the pixel data as a first pixel data signal 20e (STEP
411). The second image memory access unit 1503 receives the third pixel position signal 20d, reads pixel data at the pixel position (0, 0) from the difference image block 603 in the block image memory 205, and It is output as the pixel data signal 20g of Step 3 (STEP 413).

[0421] In the multiplier generation unit 207 in the pixel operation circuit 203,
According to (Table 1), 1.0 is generated as the first multiplier,
Is generated as a multiplier of (STEP 421). First multiplier 2
In step 09, the pixel data input by the first pixel data signal 20e is multiplied by the first multiplier, and the result is output to the adding unit 210 (STEP 422). The second multiplier 208 multiplies the pixel data input by the third pixel data signal 20g by the second multiplier and outputs the result to the adder 210 (STEP 423). The adder 210 adds the multiplication results of the first multiplier 209 and the second multiplier 208 and outputs the result as a second pixel data signal 150b.
(STEP 424).

Next, in the first image memory access unit 1504, the second pixel position signal 150a and the second pixel data signal
0b is input, it is determined that the second pixel position indicates the inside of the frame memory (STEP 1711), and the second pixel data is written to the pixel position (0, 0) in the generated image block 604 (STEP 41).
2). On the other hand, the second image memory access unit 1503
It is determined that the pixel position does not point in the block image memory (STEP 1711).

The process of FIG. 20 is repeated for each pixel in the generated image block 604 until all the pixels in the generated image block 604 have been generated (STEP 404).

[0424] By performing the processing as described above, the generated image block 604 is generated from the past image block 602 and the motion compensation difference image 603.

[0425] The operation of the backward motion compensation process is different from the above-described forward motion compensation only in that a future image is used instead of a past image as a reference image.

Next, the operation at the time of the bidirectional motion compensation processing will be described. Here, only the points different from the bidirectional motion compensation processing in the first embodiment will be described.

For bidirectional motion compensation processing, backward motion compensation processing is performed after forward motion compensation processing. ST
In the image block setting process of EP 1631, the inside of the block image memory 205 is designated as the generation location of the generated image block. In the forward motion compensation process of STEP 1632, the generated image block is generated in the block image memory 205.
That is, the image block resulting from the forward motion compensation processing is stored in the block image memory 205. This image data is image data of a difference image block in the backward motion compensation process performed after the forward motion compensation process, and is already stored in the block image memory 205 when the forward motion compensation process is completed. The process of moving the generated image block to the block image memory for the backward motion compensation process, which is required in the first embodiment, becomes unnecessary.
In the backward motion compensation processing, since the position of the generated image block is specified in the frame memory 109, the result of the bidirectional motion compensation processing is generated in the generated image in the frame memory 109.

Next, the operation during the half-pixel precision motion compensation processing will be described with reference to FIG. Here, as in the first embodiment,
A description will be given on the assumption that the position is specified with half-pixel accuracy in both the vertical and horizontal directions.

According to the present embodiment, 4
Interpolation and motion compensation of one image block 803, 804, 805 and 806 are performed simultaneously.

[0430] By the image block setting processing, an instruction is given to write the generated image block into the block image memory 205.

[0431] In STEP 391, the image data of the reference image block 803 is multiplied by 0.25 according to (Table 1), and this is added to the difference image block read from the block image memory 205, and written to the block image memory 205.

In STEP 393, the image data of the reference image block 804 is multiplied by 0.25 in accordance with (Table 1), and the image data read out from the block image memory 205 is added and written to the block image memory 205.

[0433] In STEP 395, the image data of the reference image block 805 is multiplied by 0.25 according to (Table 1), added to the image data read from the block image memory 205, and written into the block image memory 205.

Here, the position of the generated image block is changed within the generated image in the frame memory 109 (STEP 1681).

[0435] In STEP 397, the image data of the reference image block 804 is multiplied by 0.25 according to (Table 1), and the image data read from the block image memory 205 is added. Is written to.

The one-way motion compensation process has been described above. In the case of the bidirectional motion compensation process, the same block motion compensation process as the one-way motion compensation process is repeated eight times. In the case of the bidirectional motion compensation processing, the first multiplier by which the reference image data is multiplied becomes 0.125, and the generated image blocks of the first seven block motion compensation processings are written to the block image memory 205, and In the block motion compensation processing of the above, the result is written to a generated image block in the generated image of the frame memory 109.

As described above, in the present embodiment, it is possible to simultaneously perform the interpolation processing and the motion compensation processing for obtaining the reference image block.

As described above, according to the present embodiment, a motion compensation process with half-pixel accuracy can be realized.

[0439] The texture mapping processing in this embodiment can be realized by the same processing procedure as in the first embodiment. The components of the image generating apparatus according to the first embodiment and the components according to the second embodiment, the pixel position specifying circuit 202 is used for the pixel position specifying circuit 1501, the image memory access circuit 204 is used for the image memory access circuit 1502, and the first image memory is used. The access unit 212 is assigned to the first image memory access unit 1504,
If the image memory access unit 211 of the second embodiment is associated with the first image memory access unit 1503, the operation of the texture mapping process of the second embodiment is the same as that of the first embodiment. Therefore, description of the operation of the texture mapping process is omitted here.

According to the image generating apparatus of this embodiment, the texture mapping of the translucent texture image can be processed including the anti-aliasing processing of the polygon edge portion.

The simultaneous execution of the motion compensation process and the texture mapping process in the present embodiment is the same as the feature of the motion compensation process of the present embodiment described above, except for the points described below. This is the same as the operation procedure of simultaneous execution.

The difference from the operation procedure of Embodiment 1 is the following two points.

(I) In one or more block motion compensation processes required for the motion compensation process, in the last block motion compensation process (or simultaneous execution of the block motion compensation and the texture mapping process), the reference image block The calculation result of the image data of the difference image block is written to the pixel position forming the generated polygon in the frame.

(Ii) In the block motion compensation processing before the last block motion compensation processing (or simultaneous execution of block motion compensation and texture mapping processing), the calculation result of the image data of the reference image block and the difference image block is stored in the block image memory 205. Write to.

By operating as described above, it is possible to simultaneously execute the motion compensation processing and the texture mapping processing.
When performing the block motion compensation processing a plurality of times, it is not necessary to move the image block from the frame memory 109 to the block image memory 205, and the processing is speeded up as compared with the image generation apparatus of the first embodiment.

[0446] The operation timing and processing time of the motion compensation process and the process of simultaneously executing the motion compensation process and the texture mapping process in the image generation device 104 will be described below. Here, the operation of the image generation device 104 will be considered in a case where a moving image compressed according to the MPEG-1 standard is expanded, and in a case where expansion of the compressed moving image and texture mapping are performed simultaneously.

The image (frame) of the MPEG-1 standard is 320 ×
It is composed of 240 pixels. 16 motion prediction blocks
It consists of × 16 pixels. As a result, one frame is composed of 20 × 15 = 300 image blocks. Further, 30 frames of image are included in one second. Here, it is assumed that the number of pixels of the polygon generated by the texture mapping process is 256 pixels, and that the reference image and the difference image have been written in the frame memory 109 before the motion compensation process.

The performance of each block of the image generating apparatus according to the present embodiment is assumed as follows.

The first image memory access unit 1504 is configured so that two pixels can be simultaneously read from and written to the frame memory 109 in parallel, and further configured so that a read or write cycle can be executed by one clock. Suppose you have

The block image memory 205 is constituted by a dual port memory, and has a second image memory access unit.
Reference numeral 1503 is assumed to be able to simultaneously process reading of data from the block image memory 205 and writing of data to the block image memory 205. The first multiplying unit 209, the second multiplying unit 208, and the adding unit 210 can perform an operation in one clock. Each circuit of the image generation device 104 is configured to enable a pipeline operation. That is,
Memory access, multiplication, addition, and pixel location generation are processed in parallel. The image generation device 104 is configured by an LSI that operates with a clock of 50 MHz. In addition,
These performances are sufficiently achievable with current technology.

The following two types of image processing will be described based on the above assumption.

(I) When performing MPEG-1 P picture expansion processing and while performing expansion, texture mapping processing is performed. It is assumed that the image blocks constituting the P picture are all compressed using forward motion prediction, and the motion compensation of all the image blocks is performed with half-pixel accuracy in both the up, down, left, and right directions. This considers the case where the processing amount is maximized.

(Ii) When performing MPEG-1 B picture expansion processing and while performing expansion, texture mapping processing is performed. It is assumed that the image blocks constituting the B picture are all compressed using bidirectional motion prediction, and the motion compensation of all the image blocks is performed with half-pixel accuracy in both the up, down, left, and right directions. This considers the case where the processing amount is maximized.

The procedure of (i) the image motion compensation processing is as follows.
It is as follows.

1: Move the difference image block from the frame memory 109 to the block image memory 205 2: Read one pixel of pixel data from within the difference image block (block image memory 205) 3: Inside the reference image block (frame memory 109) 4: Multiply the difference image data by a multiplier 5: Multiply the reference image data by a multiplier 6: Add the multiplication result 7: Write the pixel data of the operation result to the block image memory 205 8: Steps 2 to 7 are repeated for each pixel in the block (total of 256 times (= 16 × 16)) 9: Steps 2 to 8 are repeated twice by shifting the reference image block position by one pixel to the right and downward in order 10: Reference image Shift the block position to the lower right 11: Read one pixel data from the difference image block 12: Read one pixel data from the reference image block 13: Multiply the difference image data by a multiplier 14: Multiply the reference image data by a multiplier 15: Add the multiplication result 16: Write the pixel data of the operation result to the frame memory 109 17: Perform the processing 11 to 16 in the block Repeat for each pixel 18: Repeat processes 1 to 17 for each block constituting the image (300 times) Operation timings of processes 1 to 17 are shown in the timing chart of FIG.

From FIG. 28, the operation time of each step is as follows.

1 16 clocks 2-8 259 clocks 2-9 777 clocks (= 259 × 3) 11-17 259 clocks 18 315600 clocks (= (16 + 259 × 4) ×
300) One clock cycle of an LSI operating at 50 MHz is
20 ns, from which the processing time is 6.312 ms
Becomes

For decompression of an MPEG-1 image, a processing speed of 30 frames / second is sufficient.

That is, the processing time of one frame is 33 ms.
If it is below, the required performance is satisfied. From this, it is understood that the motion compensation processing of the image generation device 104 of the present embodiment can perform the motion compensation processing at a sufficient speed.

Next, the case where the motion compensation processing and the texture mapping processing are simultaneously executed for the image (i) will be described.

In the case of a P picture, it is necessary to leave a generated image for motion compensation of a subsequent B picture, so the processing procedure is as follows.

1: Perform processing 1 to 17 of motion compensation processing 2: Move generated image block to block image memory 205 3: Read one pixel of pixel data from texture image (block image memory 205) 4: Generate polygon 5: multiply the texture image data by a translucent multiplier 6: multiply the generated polygon image data by a translucent multiplier 7: add the multiplication result 8: calculate the operation result Write the pixel data into the generated polygon of the frame memory 109 9: Repeat steps 3 to 8 for each pixel in the block (256 times) 10: Repeat steps 1 to 9 for each block constituting the image (300 times) Operation timings 1 to 9 are shown in a timing chart of FIG.

From FIG. 29, it can be seen that the operation time of each step is as follows.

1 ... 1052 clocks (777 + 259) 2 ... 16 clocks 3-9 ... 259 clocks 1-10 ... 398100 clocks = (1052 + 16 + 259)
× 300 The processing time per frame is 7.962 ms,
It can be seen that the processing speed required for the above-described MPEG-1 decompression is satisfied.

Next, the (ii) image motion compensation processing will be described.

[0466] The processing procedure is as follows.

1: Move the difference image block from the frame memory 109 to the block image memory 205 2: Read out one pixel of pixel data from within the difference image block (block image memory 205) 3: Inside the reference image block (frame memory 109) 4: Multiply the difference image data by a multiplier 5: Multiply the reference image data by a multiplier 6: Add the multiplication result 7: Write the pixel data of the operation result to the block image memory 205 8: Steps 2 to 7 are repeated for each pixel in the block (total 256 times (= 16 × 16)) 9: Steps 2 to 8 are repeated three times by shifting the reference image block position by one pixel to the right, to the bottom, to the lower right in order 10 : Set future image as reference image 11: Read one pixel of pixel data from difference image block (block image memory 205) 12: Reference image block (frame 13: multiply the differential image data by a multiplier 14: multiply the reference image data by a multiplier 15: add the multiplication result 16: add the pixel data of the operation result to the block image memory 205 Write 17: Repeat steps 11 to 16 for each pixel in the block (total of 256 times (= 16 × 16)) 18: Repeat the steps 11 to 17 twice by shifting the reference image block position by one pixel to the right and down sequentially 19 : Shift the reference image block position to the lower right 20: Read one pixel data from the difference image block (block image memory 205) 21: Read one pixel data from the reference image block (frame memory 109) 22: Difference Multiply the image data by a multiplier 23: Multiply the reference image data by a multiplier 24: Add the multiplication result 25: Write the pixel data of the operation result to the frame memory 109 26: Process 2 Repeat steps 0 to 25 for each pixel in the block 27: Repeat steps 1 to 26 for each block forming the image (300 times) The operation timing of steps 1 to 17 is shown in the timing chart of FIG.

From FIG. 30, the operation time of each step is as follows.

1 16 clocks 2-8 259 clocks 2-9 1036 clocks (= 259 × 4) 11-17 259 clocks 11-18 777 clocks (= 259 × 3) 20-26 259 clocks 27 … 626400 clocks (= (16 + 259 × 4 + 25
9 × 4) × 300) One clock cycle of an LSI operating at 50 MHz is
20 ns, from which the processing time is 12.528 m
s.

For decompression of an MPEG-1 image, a processing speed of 30 frames / second is sufficient.

That is, it is sufficient if the processing time of one image is 33 ms or less. From this, it can be seen that the motion compensation processing of the image generation device of the present invention can perform the motion compensation processing at a sufficient speed.

[0472] Next, the simultaneous execution of the motion compensation processing and the texture mapping processing for the image (ii) will be described.

[0473] In the case of a B picture, there is no need to leave a generated image for subsequent motion compensation, so the following processing procedure is used.

1: Perform processing 1 to 19 of motion compensation processing 2: Set generated polygon 3: Read one pixel data from block image memory 205 4: Read pixel data from reference image block (frame memory 109) Read one pixel 5: Multiply the differential image data (= texture image data) by a multiplier 6: Multiply the reference image data by a multiplier 7: Add the multiplication result 8: Generate pixel data of the calculation result in the frame memory 109 9: Processes 3 to 8 are repeated for each pixel in the block (256 times) 10: Processes 1 to 9 are repeated for each block constituting the image. (300 times) The only difference from the case where only the motion compensation process is performed is that the writing destination of the generated pixel data in processes 3 to 8 is a polygon, and the operation timing of processes 1 to 9 is shown in FIG.
0 is the same as the timing chart.

The operation time of each step is as follows.

1 ... 1829 clocks (= 16 + 259 × 4 + 259 × 3) 3-9 ... 259 clocks 1-10 ... 626400 clocks (= (1829 + 259) × 300) One clock cycle of the LSI operating at 50 MHz is as follows.
20 ns, from which the processing time is 12.528 m
s, which indicates that the processing speed is sufficient.

As described above, the image generating apparatus of the present invention
When implemented as a circuit operating at 0 MHz, in the decompression processing of an image compressed by MPEG-1, not only can the motion compensation processing be performed in real time, but also the decompression image is regarded as a texture image, and the motion compensation processing and texture mapping processing are performed. Can be processed in real time.

When the operation clock is set to 50 MHz, the motion compensation processing of the compressed image and the texture mapping of the image as a result of the motion compensation can be executed in a maximum of about 13 ms, so that the processing time per frame is 33 ms (1 ms).
(s / 30 frames), there is a margin of 20 ms. Using this time, texture mapping processing for other texture images and motion compensation processing for higher resolution images can be performed.

According to the present embodiment, as described above,
A block image memory 205 for storing a texture image, a difference image, an intermediate image for motion compensation processing, and the like, and a pixel image for the block image memory 205 and a frame memory 109 for storing a reference image, a generated polygon, and a generated image. An image memory access circuit 1502 to access, a pixel operation circuit 203 for performing an operation between the third pixel data read from the block image memory 205 and the first pixel data read from the frame memory 109, between pixels, A pixel position specifying circuit that generates and specifies the pixel position of the pixel data to be operated and the pixel position to write the pixel data of the operation result in accordance with processing such as motion compensation processing, texture mapping processing, and simultaneous execution of motion compensation processing and texture mapping processing. 1501 and move the image block in the frame memory 109 to the block image memory 205 By providing the image moving circuit 206, decompression processing of a moving image compressed using motion prediction, such as forward motion compensation, bidirectional motion compensation, and motion compensation with half-pixel accuracy, and texture mapping of a translucent texture image, Texture mapping processing including anti-aliasing processing of polygon edges, and simultaneous execution of motion compensation processing and texture mapping processing can be realized.

Further, according to the present embodiment, since the intermediate result of the motion compensation processing is stored in the block image memory 205, the number of times of memory access is greatly reduced as compared with the image generating apparatus of the first embodiment, so that high-speed operation is achieved. Processing can be realized. Further, since the block image memory 205 only needs to store one difference image block, the block image memory 205 may have a small capacity, and can be integrated on the LSI together with the motion compensation processing / texture mapping processing circuit. This means that the access speed of the block image memory 205 can be increased, which means that epoch-making high-speed processing can be realized as compared with the first embodiment.

In the present embodiment, the size of an image block for motion compensation is set to 4 × 4 pixels for simplicity of explanation, but the present invention is not limited to this. The size is not limited to 4 × 4 pixels. It does not exclude the use of other sizes such as pixels. Further, the size of the image is set to 8 × 8 pixels, but any size can be selected. Furthermore, the processing order of the image blocks described in the procedure of the motion compensation processing according to the present embodiment is not limited to this, and the processing order can be interchanged with each other. Further, in the description of the texture mapping processing according to the present embodiment, the translucency of the translucent texture is described as one type, but it is also possible to give different translucency to each pixel constituting the texture. Further, in the present embodiment, the backward motion compensation process is not described, but the backward motion compensation process has the same processing procedure as the forward motion compensation process, and thus has the configuration of the image generation device described in the present embodiment. It is feasible.

(Embodiment 3) FIG. 31 shows the configuration of an image generation system 3100 according to the present invention. Hereinafter, the configuration and operation of the image generation system 3100 will be described in the following order.

[0486] 1. 1. Configuration of image generation system 3100 2. Configuration of image generation device 3114 3. Texture mapping process in image generation device 3114 4. Moving image decompression processing in image generation device 3114 Conclusion 1. Configuration of Image Generation System 3100 The image generation system 3100 includes a CPU 3111 and a main memory.
3112, external storage device 3113, image generation device 3114, digital / analog converter (DAC) 3115, display device
3116 and a system bus 3117 are included.

The CPU 3111 controls the entire system, controls the image generating device 3114, controls the external storage device 3113, and the like.

The main memory 3112 includes a program for controlling the image generating apparatus executed by the CPU 3111,
It stores a program describing the operation of each part of the system, images and other data read from the external storage device 3113, and data used in the program.

[0486] The external storage device 3113 stores compressed image data, texture image data, a drawing program, and the like.

[0487] The image generating device 3114 will be described later.

The DAC 3115 converts the image data output from the image generation device 3114 into a format that can be displayed on the display device 3116.
A conversion is performed.

[0489] The display device 3116 displays an image output from the DAC 3115.

[0490] The system bus 3117 connects the CPU 3111, the main memory 3112, the external storage device 3113, and the image generating device 3114 to each other, and is used for exchanging data among them.

[0491] 2. Configuration of Image Generation Device 3114 FIG. 32 shows a configuration of the image generation device 3114 shown in FIG.

The first image memory 3121 has an area for storing image data for forward motion compensation prediction, an area for storing image data for backward motion compensation prediction and texture image data, And an area for storing currently generated image data.

The pixel position specifying circuit 3127 specifies one of a plurality of pixels stored in the first image memory 3121 by using the first image address signal 3120a. When the position of the pixel stored in the first image memory 3121 is specified by the pixel position specifying circuit 3127, the first image memory 31
21 reads out the data corresponding to the designated position and outputs the data to the second image memory 3122 as a first image data signal 3120d.

The pixel position specifying circuit 3127 uses the second image address signal 3120b to generate the first image memory 3121
Specifies one of the plurality of pixels stored in the. The first image memory 31 is operated by the pixel position specifying circuit 3127.
When the position of the pixel stored in 21 is designated, the first image memory 3121 reads the data corresponding to the designated position, and uses that data as the second image data signal 3120e in the first multiplication unit 3123. And writes the generated image data signal 3120g output from the adder 3125 at the designated position.

The second image memory 3122 stores the image data written by the first image data signal 3120d read from the first image memory 3121. The pixel position specifying circuit 3127 specifies one position of a plurality of pixels stored in the second image memory 3122 by using the third image address signal 3120c. The second image memory 3122 includes:
The data corresponding to the designated position is read, and the read data is used as a third image data signal 3120f in the second multiplication unit 312.
Output to 4.

[0496] The image data input circuit 3126 uses the image data input signal 3120h to convert the texture image data or the motion compensation difference image data designated by the CPU 3111 into the first image data.
Is written to the image memory 3121.

[0497] The first multiplication unit 3123 is provided with the first image memory 31.
The image data read from 21 as the second image data signal 3120e is multiplied by the first multiplication coefficient specified by the multiplier generation unit 3128, and the multiplication result is output to the addition unit 3125.

[0498] The second multiplying unit 3124 has a function of the second image memory 31.
The image data read from 22 as the third image data signal 3120f is multiplied by the second multiplication coefficient specified by the multiplier generation unit 3128, and the multiplication result is output to the addition unit 3125.

[0499] The adding unit 3125 adds the first multiplied image data signal output from the first multiplying unit 3123 and the second multiplied image data signal output from the second multiplying unit 3124, and performs the addition. The result is output to the first image memory 3121 as a generated image data signal 3120g.

The pixel position specifying circuit 3127 outputs the first image address signal 3120a, the second image address signal 3120b, and the third image address signal 3120c to the first image memory 3121. The first image address signal 3120a is determined based on the pixel position of the texture image to be texture-mapped and the pixel position of the reference image block in the motion compensation processing.
Is used to indicate the read pixel position of the image memory 3121. The second image address signal 3120b indicates the read / write pixel position of the first image memory 3121 according to the pixel position inside the polygon to be texture-mapped and the pixel position inside the motion compensated difference image data in the moving image decompression process. Used for Third image address signal 31
Reference numeral 20c is used to indicate a pixel position to be read from the second image memory 3122 according to a pixel position in a texture image to be texture-mapped or a pixel position in a reference image block at the time of motion compensation.

[0501] The multiplier generation unit 3128 includes a first multiplier signal indicating a first multiplier coefficient and a second multiplier signal according to the translucency of the texture image in the texture mapping, the translucency in the anti-aliasing process, and the processing procedure in the motion compensation process. And a second multiplier signal indicating the multiplier coefficient of The first multiplier signal is output to first multiplier 3123, and the second multiplier signal is output to second multiplier 3124.

[0502] The image data output circuit 3129 sequentially reads out image data stored in the first image memory 3121 for image display, and outputs the read image data to the DAC 3115.

[0503] Hereinafter, an image generating apparatus having the above-described configuration will be described.
The operation of 3114 will be described.

[0504] 3. Texture mapping process in image generating device 3114 Hereinafter, texture mapping process in image generating device 3114 will be described.

FIG. 34 shows a procedure of a texture mapping process when a texture image is pasted on one polygon.

When the CPU 3111 issues a command to start texture mapping (STEP 3401), the image data input circuit 3
Reference numeral 126 denotes a first image memory 3121 for storing texture image data.
(STEP 3402).

The pixel position specifying circuit 3127 calculates the position of each pixel constituting the polygon to which the texture is to be pasted based on the vertex coordinates of the polygon, and outputs a signal indicating the position as a second image address signal 3120b. Preparations are made (STEP 3403).

The pixel position specifying circuit 3127 obtains the pixel position of the texture image to be pasted on the polygon, and outputs a signal indicating the position as the first image address signal 3120a (STEP 3404).

The multiplier generator 3128 generates a first multiplier and a second multiplier depending on whether the texture image is translucent or whether the generated pixel forms an edge of a polygon (STEP 3405). ). The translucency of the texture image is α
In the case of (1 for opaque, 0 for transparent), a second multiplier having the value α is generated and input to the second multiplier 3124, and the first multiplier having the value (1−α) is generated. It is generated and input to the first multiplication unit 3123. If the pixel currently being processed forms a polygon edge, a translucent coefficient β for anti-aliasing of the edge is calculated, and a multiplication coefficient having a value αβ is calculated by the second multiplication unit 3124. And a multiplication coefficient having a value (1−αβ) is output to the first multiplication unit.
Output to 3123. If the texture image is not translucent and the polygon anti-aliasing process is not performed, the multiplier “0” is input to the first multiplier 3123, and the multiplier “1” is input to the second multiplier 3124. Is entered.

Next, the image data (that is, the texture image) at the pixel position designated by the first image address signal 3120a is read from the first image memory 3121 and written to the second image memory 3122 (STEP 3406). .

In accordance with the second image address signal 3120b indicating the position of each pixel in the polygon obtained in STEP 3403, the pixel data of one corresponding pixel is read from the first image memory 3121 and is read out from the second image data. The signal is input to the first multiplier 3123 as a signal 3120e (STEP 3407). This image data is image data existing at the position where the texture image is to be pasted before the texture image is pasted. Hereinafter, in the present specification, such image data will be referred to as "background image data".

At the same time, the pixel position specifying circuit 3127 outputs the address in the second image memory 3122 of the texture image to be attached to the pixel position indicated by the second image address signal 3120b as the third image address signal 3120c. Then, one pixel corresponding to the texture image is read from the second image memory 3122 (STEP 3408). Second image memory 312
The image data read from 2 is input to the second multiplier 3124 as a third image data signal 3120f.

[0513] The first multiplying unit 3123 includes the first image memory 31.
Multiplier generator for background image data read from 21
Multiply by the first multiplier specified by 3128 (STEP 34
09).

[0514] The second multiplication unit 3124 is provided with the second image memory 31.
Multiplier generator 3128 for image data read from 22
Multiply by the second multiplier specified by (STEP 341)
0).

[0515] The adding unit 3125 adds the multiplication result output from the first multiplication unit 3123 and the multiplication result output from the second multiplication unit 3124 (STEP 3411).

[0516] The result of addition by the addition unit 3125 is written as a generated image data signal 3120g in the generated image pixel position of the first image memory 3121 (STEP 3412). The generated image pixel position is a pixel indicated by the second image address signal 3120b. This means that the mapping process for one pixel in the polygon has been completed.

At STEP 3413, it is determined whether all the pixels in the polygon have been processed. If all the pixels have not been processed, the processing of STEP 3407 to STEP 3412 is repeated.

[0518] This means that the texture mapping processing for one polygon has been completed (STEP 341).
Four).

By performing the above operation on all the polygons constituting the generated image, the image generating apparatus 3414 realizes the texture mapping process.

FIG. 35 (a) shows a specific flow of image data in the above-described texture mapping processing.
35A, the same components as those in FIG. 32 are denoted by the same reference numerals.

That is, reference numeral 3121 indicates a first image memory, reference numeral 3122 indicates a second image memory, reference numeral 3123 indicates a first multiplication unit, and reference numeral 3124 indicates a second multiplication unit.
, And reference numeral 3125 denotes an addition unit.

[0522] Reference numerals 3151 and 3153 indicate texture images. Reference numeral 3152 indicates a generated image (polygon) to which the texture image 3151 is pasted. Assume that the texture image 3151 has been read into the first image memory 3121 in the above-described operation STEP 3402.

In operation STEP 3406, the first image memory 312
The texture image 3151 in 1 is stored in the second image memory 3122.
Is forwarded to

When processing a translucent polygon having the transparency α, the background images in the polygon 3152 are sequentially read out in steps 3407 to 3412, and are multiplied by (1−α) by the first multiplier 3123. The pixel to be pasted at this pixel position is read from the texture image 3153 in the second image memory 3122, multiplied by α by the second multiplier 3124, and input to the adder 3125. The generated image data output from the adding unit 3125 is written at the same pixel position in the polygon 3152.

In FIG. 35B, a horizontal line indicating the order of pixels to be read / written is written by overlapping the polygon 3152. Pixels in the polygon 3152 are read in the order of the horizontal lines (1) to (8), and the generated image data is written at the positions of the read pixels. Further, FIG.
The texture images 3153 correspond to the order shown in FIG.
The order in which the pixels within are read out. Straight line (1) ~
The pixels in the texture image 3153 are read in the order of (8).

The second output from the pixel position specifying circuit 3127
Image address signal 3120a and third image address signal 312
The order of the pixel positions specified by 0b and
This corresponds to the pixel access order shown in FIGS. 35 (b) and (c).

FIG. 35D shows a state of the polygon 3152 after the texture image 3153 is pasted. FIG.
In (d), the background image is omitted for simplicity of explanation.

In the above description, the size of the texture image to be transferred to the second image memory 3122 is set to a size necessary and sufficient for pasting on one polygon. However, the size of the texture image transferred to the second image memory 3122 can be set to an arbitrary size regardless of the size of the polygon.

If the size of the second image memory 3122 is smaller than the size of the texture image pasted on one polygon, the size of the texture image that can be transferred to the second image memory 3122 at a time is one polygon. Not enough to stick to. Therefore, the texture image to be attached to one polygon is divided into a plurality of
Need to transfer to 3122. In this case, the transfer operation STEP from the first image memory 3121 to the second image memory 3122
3406 is incorporated into the operation of STEP 3407 to STEP 3412,
The transfer operation may be performed a plurality of times as needed.

When the size of the second image memory 3122 is larger than the size of the texture image to be pasted on one polygon, the operation STEP 3406 is performed once,
It is possible to transfer all texture images necessary for pasting to one polygon. Thereby, the first
Since the number of accesses to the image memory 3121 is reduced, the processing speed is improved.

[0531] 4. Moving Image Decompression Process in Image Generation Device 3114 The following describes the video decompression process in the image generation device 3114. Decompression of a moving image compressed by the inter-frame motion compensation predictive coding method is sequentially performed for each image block constituting an image.

FIG. 36 shows the procedure of a moving image decompression process for one image block.

When an image block decompression command is issued from the CPU 3111 (STEP 3601), the image data input circuit 3126
The motion compensation difference data of this image block is stored in the first image memory 3121 (STEP 3602). Note that the first image memory 3121 stores a forward predicted image or a backward predicted image for an image that is expanded as necessary. These predicted images are read from the external storage device 3113 and read from the external storage device 3113.
There are a case where the image has been written to the image generation device 3114 by the PU 3111 and a case where the image has already been generated by the image generation device 3114.

The pixel position specifying circuit 3127 calculates the position of each pixel constituting the image block to be decompressed (that is, the pixel position where the motion compensation difference data is written), and sends a signal indicating the position to the second image address. The first as signal 3120b
Prepare to output to the image memory 3121 (STEP 36
03).

Next, the pixel position specifying circuit 3127 obtains the pixel position of the image of the predicted image data block from the first predicted image according to the value of the motion vector, and outputs a signal indicating the position to the first image address signal 3120a. As the first image memory 3
Output to 121 (STEP 3604).

According to the type of the predicted image used for the motion prediction and the value of the motion vector, how many times the predicted image data block is read and added to the generated image block is calculated, and the number of times calculated in this manner is calculated. A second multiplier is generated based on the second multiplier (STEP 3605). A detailed calculation method of the second multiplier will be described later. Note that the first multiplier is always set to “1” at the time of moving image decompression processing.

Next, the image data at the pixel position specified by the first image address signal 3120a (that is, the image data of the predicted image data block) is stored in the first image memory.
3121 and write it to the second image memory 3122 (S
TEP 3606).

According to the second image address signal 3120b indicating the position of each pixel constituting the image block obtained in STEP 3603, the pixel data of one corresponding pixel (ie,
The motion compensation difference data) is read from the first image memory 3121, and is input to the first multiplier 3123 as a second image data signal 3120e (STEP 3607).

[0539] At the same time, the pixel position specifying circuit 3127 sets the address in the second image memory 3122 of the predicted image corresponding to the motion compensation difference data at the pixel position indicated by the second image address signal 3120b as the third image address. The signal is output as a signal 3120c, and one pixel of the corresponding predicted image is read from the second image memory 3122 (STEP 3608). Second image memory 312
The image data read from 2 is input to the second multiplier 3124 as a third image data signal 3120f.

[0540] The first multiplying unit 3123 includes the first image memory 31.
Multiplier generator 3128 for the difference data read from 21
Multiplies the first multiplier specified by (STEP 360
9). As set in STEP 3605, the first multiplier is always “1”.

[0537] The second multiplication unit 3124 is provided with the second image memory 31.
Multiplier generator 3128 for image data read from 22
Multiply by the second multiplier specified by (STEP 361)
0).

The adding section 3125 adds the multiplication result output from the first multiplication section 3123 and the multiplication result output from the second multiplication section 3124 (STEP 3611).

The result of the addition by the adding section 3125 is written as a generated image data signal 3120 g in the generated image pixel position of the first image memory 3121 (STEP 3612). The generated image pixel position is a pixel indicated by the second image address signal 3120b. This completes the process for one predicted image of one pixel in the image block.

In STEP 3613, it is determined whether or not all the pixels in the image block have been processed. If all the pixels have not been processed, the processing of STEP 3607 to STEP 3612 is repeated.

[0545] In STEP 3614, it is determined whether or not all necessary prediction images have been processed. If the image block being processed still needs to be added to another predicted image, the processing of STEP 3604 to STEP 3613 is repeated.

[0546] Thus, the decompression processing for one image block is completed (STEP 3615).

By performing the above operation on all the image blocks constituting the generated image, the image generating device 34
14 implements the moving image decompression process.

FIG. 37 (a) shows a specific flow of image data in a moving image decompression process when a decompression target image block is compressed using two forward and backward predicted images. In FIG. 37A, the same components as those in FIG. 35A are denoted by the same reference numerals.

That is, reference numeral 3121 indicates a first image memory, reference numeral 3122 indicates a second image memory, reference numeral 3123 indicates a first multiplication unit, and reference numeral 3124 indicates a second multiplication unit.
, And reference numeral 3125 denotes an addition unit.

[0550] Reference numeral 3171 indicates forward predicted image data, and reference numeral 3172 indicates backward predicted image data. Reference numeral 3173 indicates an area where generated image data is stored. Reference numeral 3174 indicates a forward predicted image data block, reference numeral 3175 indicates a backward predicted image data block, reference numeral 3176 indicates a predicted image data block,
Reference numeral 3177 indicates a generated image data block.

FIG. 37A shows an image generating apparatus in a state in which the processing of STEP 3602 has been completed, that is, a state in which the motion compensation difference data block has been loaded into the first image memory 3121.
The state of the image data in 3114 is shown. That is, the content of the generated image data block 3177 is motion compensation difference data.

FIG. 37 (b) shows the result of processing using forward predicted image blocks. That is, this is the result of performing the processing from STEP 3604 to STEP 3613 shown in FIG. 36 for the forward prediction image block. The pixel data in the motion compensation difference data block stored in the generated image data block 3177 is represented by D (x, y), and the pixel data stored in the forward prediction image data block 3174 is denoted by F (x, y). When expressed as, as a result of processing using the forward prediction image block,
Image data G1 (x, y) as shown in (Equation 9) is written to the generated image data block 3177.

Here, γ is a multiplier given to the second multiplier 3124, and in this case, is 0.5.

[0554]

G1 (x, y) = D (x, y) + F (x, y) × γ FIG. 37 (c) shows the result obtained by further processing this result using a backward prediction image block. Show. That is, FIG.
Are the results of performing the processing STEP 3604 to STEP 3613 shown in FIG. The pixel data stored in the backward prediction image data block 3175 is represented by R (x, y).
As a result of processing using the backward prediction image block, image data G2 (x, y) as shown in (Equation 10) is written to the generated image data block 3177.

[0555]

G2 (x, y) = G1 (x, y) + R (x, y) × γ Here, γ is a multiplier given to the second multiplier 3124, and in this case, is 0.5.

[0556] By transforming (Equation 10) into (Equation 11), the pixel value of the generated image data block becomes 2
It can be seen that the value is obtained by adding the motion compensation difference data to the average of two prediction image blocks.

[0557]

G2 (x, y) = G1 (x, y) + R (x, y) × γ = {D (x, y) + F (x, y) × γ} + R (x, y ) × γ = {F (x, y) + R (x, y)} × γ + D (x, y) Since γ = 0,5, G2 (x, y) = (F (x, y) ) + R (x, y)} / 2 + D (x, y) In FIGS. 37A to 37C, the image block to be decompressed is compressed using two predicted images in the forward and reverse directions. The operation of the decompression process in the case of the above has been described. Next, the operation of the decompression process in the case where the image block to be decompressed is compressed using one forward or backward predicted image will be described.

Processing shown in FIG. 36, STEP 3604 to STEP 36
13 is performed for one predicted image block in the forward or backward direction. The result G1 (x, y) is as shown in (Equation 12).

[0559]

G1 (x, y) = D (x, y) + F (x, y) × γ Here, γ is 1.

Next, the decompression processing when the value of the motion vector is given in a unit of a half pixel will be described. In this case as well, for the forward and / or backward predicted images used for compression, processing STEPs 3604 to STE
The repetition of P 3613 is exactly the same, but for a predicted image in which the value of the motion vector is given in half-pixel units, a predicted image block shifted by one pixel
Steps 3604 to 3613 are repeatedly performed. And
The multiplier γ to be given to the second multiplier 3124 is changed according to the number of repetitions and whether the predicted image is in either the forward direction or the backward direction or both in the forward direction and the backward direction.

FIGS. 38 (a) to 38 (c) show predicted image blocks when a motion vector value is given in half-pixel units. 38A to 38C, it is assumed that one image block is composed of 4 × 4 pixels. The horizontal direction is indicated by x, and the vertical direction is indicated by y. A circle represents one pixel constituting the predicted image block. Written above the image block
The numbers 16 to 20 and the numbers 21 to 25 written to the right of the image block mean addresses indicating pixel positions.

As shown in FIG. 38 (a), for a predicted image block in which a motion vector is specified in units of half pixels in the horizontal direction, processing is first performed on the predicted block Q0 (x, y), The process is performed on the block Q1 (x, y).
Thus, the processing is performed twice in total.

As shown in FIG. 38 (b), for a predicted image block in which a motion vector is specified in half-pixel units in the vertical direction, processing is first performed on the predicted block Q0 (x, y), The process is performed on the block Q2 (x, y).
Thus, the processing is performed twice in total.

As shown in FIG. 38 (c), with respect to a predicted image block in which a motion vector is specified in half a pixel in both the vertical and horizontal directions, the predicted blocks Q0 (x, y), Q1 (x, y), Q2
(x, y) and Q3 (x, y) are processed four times in total. that time,
The values of γ set in the second multiplication unit 3124 are shown in (Table 2).

[0565]

[Table 2] As shown in Table 2, the value of γ changes according to the number of prediction images used when the original image block is compressed.

[0566] In (Expression 13), among the image blocks compressed using both the forward and backward predicted images, the forward predicted image block is specified in half-pixel units in the vertical and horizontal directions.
FIG. 14 shows the progress and results of the decompression process in the case where the predicted image block in the reverse direction is specified in the horizontal direction in half-pixel units. G1 (x, y), G2 (x, y), G3 (x, y), G4 (x, y), G5 (x,
y) represents the pixel value of the generated image block in the middle of the decompression process, and G6 (x, y) represents the final result of the decompression process. Also, F0 (x,
y), F1 (x, y), F2 (x, y) and F3 (x, y) are, as can be inferred from FIG. 38 (c), shifted by half pixels in the prediction block of the forward prediction image. Represents the pixel value of one image block, R0
(x, y) and R1 (x, y) represent pixel values of two image blocks shifted by half a pixel in the prediction block of the backward prediction image.

[0567]

G1 (x, y) = D (x, y) + F0 (x, y) × 0.125 G2 (x, y) = G1 (x, y) + F1 (x, y) × 0.125 G3 ( x, y) = G2 (x, y) + F2 (x, y) x 0.125 G4 (x, y) = G3 (x, y) + F3 (x, y) x 0.125 G5 (x, y) = G4 (x, y) + R0 (x, y) × 0.25 G6 (x, y) = G5 (x, y) + R1 (x, y) × 0.25 When the result G6 (x, y) is transformed, )become that way.
According to this, the pixel value of the generated generated image data block is an average of a plurality of predicted image blocks shifted by one pixel read out according to the value of the motion vector for each of the forward and backward predicted images. Is obtained, and the value is obtained by adding the average value of both of the predicted image blocks and the motion compensation difference data.

[0568]

G6 (x, y) = [(F0 (x, y) + F1 (x, y) + F2 (x, y)
+ F3 (x, y)} / 4+ {R0 (x, y) + R1 (x, y)} / 2] / 2 +
By performing similar processing for combinations other than D (x, y) (Equation 13), image block decompression processing can be realized.

5. Conclusion As described above, according to the present invention, a process of decompressing a moving image compressed using inter-frame motion compensation prediction encoding,
Texture mapping including translucent texture image texture mapping and polygon edge anti-aliasing can be performed using a single image generation device 3114.

[0570] Note that the above-described image generating apparatus 3114 is capable of performing anti-aliasing processing of a translucent texture image or polygon edge and performing decompression processing using a plurality of predicted images. A multiplication unit 3124 was provided.

However, in the texture mapping process, if the texture image is always opaque (that is, the translucency is always 1), the second multiplying unit 3124 may be omitted. In this case, the multiplier generation unit 3128 includes the first multiplication unit
The multiplier "0" is output to 3123. As a result, the generated image data signal output from the adding unit 3125 does not include any components of the background image, and includes only components of the texture image.

Also, in the decompression processing, the image block is compressed using only one of the forward prediction image data and the backward prediction image data, and the value of the motion vector is specified in units of one pixel. The second
May be omitted. In this case, the multiplier generator
3128 outputs the multiplier “1” to the first multiplication unit 3123. As a result, the generated image data signal output from the adding unit 3125 includes the result of adding one piece of predicted image data and the motion compensation difference image data.

As described above, even in an image generating apparatus having a configuration in which the second multiplying unit 3124 is removed, both texture mapping processing and moving picture decompression processing can be realized under specific conditions. The processes that can be realized are all basic processes.

(Embodiment 4) According to the following order,
The configuration and operation of another image generation system 3100 'according to the present invention will be described.

[0575] 1. 1. Configuration of image generation system 3100 ′ 2. Configuration of image generation device 3114 ′ 3. Texture mapping process in image generation device 3114 ' 4. Moving image decompression process in image generation device 3114 ' Conclusion 1. Configuration of Image Generation System 3100 'The configuration of the image generation system 3100' is the same as the configuration of the image generation system 3100 shown in FIG. 31 except for the image generation device 3114 '. Therefore, the same components are denoted by the same reference numerals, and the description thereof is omitted here.

[0576] 2. Configuration of Image Generation Apparatus 3114 'FIG. 33 shows a configuration of the image generation apparatus 3114'. The configuration of the image generation device 3114 ′ is the same as the configuration of the image generation device 3114 shown in FIG. 32 except that a FIFO memory 3130 is added. Therefore, the same components are denoted by the same reference numerals, and the description thereof is omitted here.

The second image data signal 31 read from the position specified by the second image address signal 3120b
20e is stored in the FIFO before being input to the first multiplication unit 3123.
The data is input to the memory 3130 and held there. Further, the image generation device 3114 ′ is configured so that the image data input signal input from the image data input circuit 3126 can be directly written into the FIFO memory 3130. Therefore, the image data input circuit 3126 writes the texture image data or the motion compensation difference image data to the first image memory 3121 or the FIFO memory 3130 according to the image data input signal. The FIFO memory 3130 is an image data input circuit
Image data written by an image data input signal output from 3126 or image data input as a second image data signal 3120e from the first image memory 3121 is stored and output as a fourth image data signal. .
Hereinafter, the operation of the image generation device 3114 'having the above-described configuration will be described. The operation of the image generation device 3114 'is the same as the operation of the image generation device 3114 except for a part. Therefore, hereinafter, portions different from the operation of the image generation device 3114 will be described in detail.

[0578] 3. Texture mapping process in image generating device 3114 'Hereinafter, the texture mapping process in the image generating device 3114' will be described.

The procedure of the texture mapping process when pasting a texture image to one polygon is shown in FIG.
As shown in FIG.

The FIFO memory 3130 stores the second image data signal read from the position specified by the second image address signal 3120b (the image data in the polygon at the generated image position before the texture is pasted). That is, before inputting the background image data) to the first multiplication unit 3123, it serves as a buffer memory that passes through temporarily.

The operation of the FIFO memory 3130 is included in the processing STEP 3407 in FIG. In STEP 3407, the image data of one corresponding pixel is converted into a second image data signal 3120e from the first image memory 3121 according to the second image address signal 3120b indicating each pixel position in the polygon obtained in STEP 3403. The data is read out, and once input via the FIFO memory 3130 to the first multiplying unit 3123 as a fourth image data signal.

[0583] Other operations in the texture mapping process are the same as the operations of the image generating device 3114 shown in FIG. Thereby, it is possible to realize texture mapping processing including texture mapping of a translucent texture image and anti-aliasing processing of polygon edges.

[0583] 4. Moving Image Decompression Process in Image Generating Device 3114 'Next, the moving image decompression process in the image generating device 3114' will be described.

FIG. 39 shows the operation procedure of the decompression process for one image block. The operation shown in FIG.
The operation is the same as that shown in FIG. 36 except for EP 3902, STEP 3093, STEP 3907-1, and STEP 3907-2. 39, the same operations as those in FIG. 36 are denoted by the same reference numerals.

When an image block decompression command is issued from the CPU 3111 (STEP 3601), the image data input circuit 3126
The motion compensation difference data of the image block is stored in the FIFO memory 3130 (STEP 3902). Note that the first image memory 3121 stores a forward predicted image or a backward predicted image for an image that is expanded as necessary. These predicted images are read from the external storage device 3113 and read from the CP.
There are a case where the image is written to the image generating device 3114 ′ by the U3111 and a case where the image is already generated by the image generating device 3114 ′.

The pixel position specifying circuit 3127 calculates the position of each pixel constituting the image block to be decompressed (ie, the pixel position where the motion compensation difference data has been written), and sends a signal indicating the position to the second image address. The first as signal 3120b
Prepare to output to image memory 3121 (STEP 39)
03). The order of the pixel positions indicated by the second image address signal 3120b corresponds to the order of reading out the motion compensation difference data of the image block to be decompressed from the FIFO memory 3130.

Next, the pixel position specifying circuit 3127 obtains the pixel position of the image of the predicted image data block from the first predicted image according to the value of the motion vector, and outputs a signal indicating the position to the first image address signal 3120a. As the first image memory 3
Output to 121 (STEP 3604).

According to the type of the predicted image used for the motion prediction and the value of the motion vector, how many times the predicted image data block is read and added to the generated image block is calculated, and the number of times thus calculated is calculated. A second multiplier is generated based on the second multiplier (STEP 3605). The detailed calculation method of the second multiplier is as described in the third embodiment. The first
Is always set to “1” at the time of moving image decompression processing.

Next, the image data at the pixel position designated by the first image address signal 3120a (that is, the image data of the predicted image data block) is stored in the first image memory.
3121 and write it to the second image memory 3122 (S
TEP 3606).

According to the second image address signal 3120b indicating the position of each pixel constituting the image block obtained in STEP 3603, the pixel data of one corresponding pixel (ie,
The motion compensation difference data) is read out from the FIFO 3130 and is input to the first multiplication unit 3123 as a fourth image data signal (STEP 3907-2).

[0591] The processing of STEP 3608 to STEP 3613 is the same as that of the third embodiment.

In STEP 3614, it is determined whether or not processing has been performed on all necessary predicted images. If the image block being processed still needs to be added to another predicted image, the processing of STEP 3604 to STEP 3613 is repeated.

In this embodiment, in the processing of STEP 3907 for the second and subsequent predicted images, the first image memory 3131
Is read out from the pixel position in the generated image area, and is temporarily input to the first multiplier 3123 via the FIFO memory 3130 as a fourth image data signal (STEP
3907-1 to STEP 3907-2).

[0591] 5. Conclusion As described above, in the present embodiment, the motion compensated difference image data block first written from the image data input circuit 3126 is not stored in the first image memory 3121, but is directly stored in the FIFO memory 3130. To the second image data signal from the first image memory 3121.
The operation of reading as 3120e can be omitted. As a result, the number of accesses to the first image memory 3132 can be reduced, and the processing speed can be increased. This effect is prominent when the image data block compressed using only one of the forward prediction image and the backward prediction image in the motion compensation prediction is expanded. That is,
In the case where the FIFO memory 3130 is not provided, the number of accesses to the first image memory 3121 required to decompress the image data block compressed by using one predicted image depends on the read and write times. Two times, a total of four times. On the other hand, in the case of the configuration in which the FIFO memory 3130 is provided, the number of accesses to the first image memory 3121 required to expand the image data block compressed using one Each time is a total of two times. By providing the FIFO memory 3130 in this manner, the number of accesses to the first image memory 3121 can be halved.

[0595] Note that the above-described image generation device 3114 'is designed to perform a second process in order to enable anti-aliasing processing of a translucent texture image or polygon edge and to perform expansion processing using a plurality of predicted images. The multiplication unit 3124 was provided.

However, in the texture mapping process, if the texture image is always opaque (ie, the translucency is always 1), the second multiplication unit 3124 may be omitted. In this case, the multiplier generation unit 3128 includes the first multiplication unit
The multiplier "0" is output to 3123. As a result, the generated image data signal output from the adding unit 3125 does not include any components of the background image, and includes only components of the texture image.

Also, in the decompression processing, the image block is compressed using only one of the forward prediction image data and the backward prediction image data, and the value of the motion vector is specified in units of one pixel. The second
May be omitted. In this case, the multiplier generator
3128 outputs the multiplier “1” to the first multiplication unit 3123. As a result, the generated image data signal output from the adding unit 3125 includes the result of adding one piece of predicted image data and the motion compensation difference image data.

As described above, even in an image generating apparatus having a configuration from which the second multiplying unit 3124 is removed, under a specific condition, both the texture mapping process and the moving image decompressing process can be realized. The processes that can be realized are all basic processes.

(Embodiment 5) Hereinafter, an image generating apparatus 4110 according to the present invention will be described.

The image generation device 4110 includes a pixel operation circuit 4120 and an image memory 4130. The image generation device realizes the texture mapping process and the motion compensation process with a single configuration.

Hereinafter, referring to FIG. 42, image generation apparatus 41
The procedure of the texture mapping process in 10 will be described. The numbers 1 to 3 in FIG. 42 correspond to the numbers indicating the processing procedures described below.

0: The texture image is written in the image memory 4130.

[0603] The generated image area is set in the image memory 4130.

1: Pixel data is read from the texture image.

The pixel data of the texture mapping target coordinates of the texture mapping target polygon in the generated image is read.

2: Texture pixel data and polygon pixel data are each multiplied by a coefficient, and the multiplication results are added.

3: The pixel data resulting from the addition is written to the texture mapping target coordinates in the generated image.

4: Processes 1 to 3 are repeated until all pixels in the texture mapping target polygon have been processed.

[0609] Referring to FIG. 43, image generation apparatus 41 will now be described.
The procedure of the motion compensation processing in 10 will be described. Numbers 1 to 4 and 7 in FIG. 43 correspond to the numbers indicating the processing procedures described below.

0: The difference image is written in the image memory 4130.

The reference image is written in the image memory 4130.

[0691] The generated image area is set in the image memory 4130.

1: An image block to be subjected to motion compensation is extracted from the difference image.

(In FIG. 43, the image block is copied. Instead of copying the image block, the location of the image block may be designated.) A reference image block in the reference image is set.

[0615] A generated image block in the generated image is set.

2: Read pixel data from difference image block.

[0617] Pixel data is read from the reference image block.

3: Multiply the pixel data read from the reference image block by a coefficient.

(Coefficient is whether motion compensation processing is bidirectional,
This depends on whether the position of the reference image block is specified with half-pixel accuracy. The sum of the multiplication result and the pixel data read from the difference image block are added.

4: The pixel data resulting from the addition is written in the generated image block.

5: Steps 2 to 4 are repeated until the processing of all the pixels in the image block is completed.

6: When the motion compensation processing of the image block is completed, the processing is repeated for all the image blocks in the generated image.

7: In the case of bidirectional motion compensation processing or when the position of the reference image block is specified with half-pixel accuracy, a new reference image block is set.

[0624] The generated image block is extracted as a difference image block.

(In FIG. 43, the image block is copied. Instead of copying the image block, the location of the image block may be designated.) 8: Bidirectional motion compensation processing and reference image block Processes 2 to 7 are repeated according to the position designation with half-pixel accuracy.

[0626] Referring to FIG. 44, image generation apparatus 41 will now be described.
The procedure of another motion compensation process in 10 will be described. FIG.
Correspond to the numbers indicating the processing procedures described below.

0: A difference image is written in the image memory 4130.

The reference image is written in the image memory 4130.

[0629] A generated image area (the same area as the difference image) is set in the image memory 4130.

1: An image block serving as a reference for motion compensation is extracted from the reference image.

(In FIG. 44, the image block is copied. Instead of copying the image block, the location of the image block may be designated.) A difference image block in the difference image is set.

The same image block as the difference image block is set as a generated image block.

2: Read pixel data from difference image block.

[0634] Pixel data is read from the reference image block.

3: The pixel data read from the reference image block is multiplied by a coefficient.

(Coefficient is whether motion compensation processing is bidirectional,
This depends on whether the position of the reference image block is specified with half-pixel accuracy. The sum of the multiplication result and the pixel data read from the difference image block are added.

[0637] 4: The pixel data resulting from the addition is written at the same coordinates as the read coordinates of the difference pixel data in the generated image block.

5: Steps 2 to 4 are repeated until the processing of all the pixels in the image block is completed.

6: When the motion compensation processing of the image block is completed, the processing is repeated for all the image blocks in the generated image.

7: In the case of bidirectional motion compensation processing or when the position of the reference image block is specified with half-pixel accuracy, a new reference image block is taken out.

(In FIG. 44, the image block is copied. Instead of copying the image block, the location of the image block may be designated.) The generated image block is set as the difference image block.

8: Processes 2 to 7 are repeated in accordance with bidirectional motion compensation processing and half-pixel precision position designation of the reference image block.

[0643] Hereinafter, with reference to FIG.
The processing procedure in the case where the texture mapping process and the motion compensation process are performed at the same time in 10 will be described. Numbers 1 to 4 in FIG. 45 correspond to the numbers indicating the processing procedures described below.

[0644] 0: A difference image is written in the image memory 4130.

[0645] The reference image is written in the image memory 4130.

[0646] The generated image area is set in the image memory 4130.

1: An image block to be subjected to motion compensation is extracted from the difference image.

(In FIG. 45, the image block is copied. Instead of copying the image block, the location of the image block may be designated.) A reference image block in the reference image is set.

[0649] A polygon to be texture mapped is set in the generated image.

2: Read pixel data from difference image block.

[0655] Pixel data is read from the reference image block.

(3) The pixel data read from the reference image block is multiplied by a coefficient.

(Coefficient is whether motion compensation processing is bidirectional,
This depends on whether the position of the reference image block is specified with half-pixel accuracy. The sum of the multiplication result and the pixel data read from the difference image block are added.

4: The pixel data resulting from the addition is written into the generated polygon.

5: Steps 2 to 4 are repeated until the processing of all the pixels in the generated polygon is completed.

6: Upon completion of the generated polygon processing, the processing is repeated for all polygons in the generated image.

[0657]

The image generating apparatus of the present invention realizes not only forward or backward motion compensation processing but also bidirectional motion compensation processing, half-pixel precision motion compensation processing, and texture mapping processing with a single unit. it can.

[0658] Furthermore, the simultaneous and parallel processing of the motion compensation processing and the texture mapping processing can be realized by a device having a single configuration.

Also, by storing the intermediate result of the motion compensation processing in the block image memory, the number of memory accesses required for the motion compensation processing can be greatly reduced.

Further, by integrating the block image memory and the pixel operation means on the LSI, it is possible to greatly speed up memory access required for motion compensation processing and texture mapping processing.

As described above, by realizing an image generating apparatus capable of performing a compressed moving image decompression process by a motion compensation process, a texture mapping process, and a simultaneous and parallel process of these processes with a single configuration, Reduction of the data transfer amount by simplifying the configuration of the video decompression mapping device and the multimedia device using the generation device, reducing the circuit scale, reducing the amount of memory required for processing, and eliminating the need to transfer video data between memories Can be realized.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a multimedia device according to a first embodiment of the present invention.

FIG. 2 is a configuration diagram of an image generation device according to the first embodiment.

FIG. 3 is a control flowchart for controlling the operation of the image generating apparatus by the CPU when performing a motion compensation process in the first embodiment.

FIG. 4 is a part of a control flowchart for controlling operation of the image generating apparatus by the CPU when performing motion compensation processing in the first embodiment.

FIG. 5 is an operation flowchart of each circuit in the image generation apparatus when performing a motion compensation process in the first embodiment.

FIG. 6 is an operation flowchart of each circuit in the image generation device when performing an image block setting process in the first embodiment.

FIG. 7 is an explanatory diagram of a forward motion compensation process according to the present invention.

FIG. 8 is an explanatory diagram of a bidirectional motion compensation process according to the present invention.

FIG. 9 is an explanatory diagram of a half-pixel precision motion compensation process according to the present invention.

FIG. 10 is a control flowchart for controlling operation of the image generating apparatus by the CPU when performing texture mapping processing in the present invention.

FIG. 11 is an operation flowchart of each circuit in the image generation device when performing the texture mapping process in the present invention.

FIG. 12 is an explanatory diagram of a texture mapping process according to the present invention.

FIG. 13 is a control flowchart for controlling the operation of the image generating apparatus by the CPU when simultaneously executing the motion compensation process and the texture mapping process in the first embodiment.

FIG. 14 is a part of a control flowchart for controlling the operation of the image generating apparatus by the CPU when simultaneously executing the motion compensation process and the texture mapping process in the first embodiment.

FIG. 15 is an operation flowchart of each circuit in the image generation device when the motion compensation process and the texture mapping process are simultaneously executed in the first embodiment.

FIG. 16 is an explanatory diagram of simultaneous execution of motion compensation processing and texture mapping processing according to the present invention.

FIG. 17 is a configuration diagram of an image generation device according to a second embodiment.

FIG. 18 is a control flowchart for controlling operation of the image generating apparatus by the CPU when performing motion compensation processing in the second embodiment.

FIG. 19 is a part of a control flowchart for controlling the operation of the image generating apparatus by the CPU when performing the motion compensation processing in the second embodiment.

FIG. 20 is an operation flowchart of each circuit in the image generation device when performing a motion compensation process in the second embodiment.

FIG. 21 is an explanatory diagram of a procedure of an image compression process using a motion prediction process.

FIG. 22 is an explanatory diagram of a decompression processing procedure of a compressed moving image compressed and encoded by motion prediction.

FIG. 23 is an explanatory diagram of a texture mapping process.

FIG. 24 is a conceptual explanatory diagram of simultaneous execution of a motion compensation process and a texture mapping process.

FIG. 25 is an explanatory diagram of an application of texture mapping using moving image data as a texture image.

FIG. 26 is an operation timing chart of the multimedia device of the second embodiment.

FIG. 27 is a configuration diagram of an image generation system in a conventional example.

FIG. 28 is a diagram showing a timing chart when a motion compensation process is performed on an MPEG-1 P picture.

FIG. 29 is a diagram showing a timing chart when a motion compensation process and a texture mapping process are simultaneously performed on an MPEG-1 P picture.

FIG. 30 is a diagram showing a timing chart when a motion compensation process or a motion compensation process and a texture mapping process are simultaneously performed on an MPEG-1 B picture;

FIG. 31 is a diagram showing a configuration of an image generation system 3100 according to the present invention.

FIG. 32 is a diagram showing a configuration of an image generation device 3114 included in the image generation system 3100.

FIG. 33 is a diagram showing a configuration of an image generation device 3114 ′ included in the image generation system 3100 ′.

FIG. 34 is a flowchart illustrating a procedure of a texture mapping process when attaching a texture image to one polygon in the image generation device 3114.

35A is a diagram showing a specific flow of image data in a texture mapping process, FIG. 35B is a diagram showing an order of pixels to be read / written, and FIG. 35C is an order to read pixels in a texture image (D) is a diagram showing the state of the polygon after the texture image is pasted.

FIG. 36 is a flowchart showing a procedure of a moving image decompression process for one image block in the image generation device 3114.

37A is a diagram showing a specific flow of image data in a moving image decompression process, FIG. 37B is a diagram showing a result of processing using a forward prediction image block, and FIG. FIG. 14 is a diagram illustrating a result of processing further using the backward prediction image block on the result of FIG.

FIGS. 38A to 38C are diagrams illustrating a predicted image block when a motion vector value is given in half-pixel units.

FIG. 39 is a flowchart showing a procedure of a moving image decompression process for one image block in the image generation device 3114 ′.

40 is a diagram showing a configuration inside an image decompression circuit 2404 shown in FIG. 27.

FIG. 41 is a diagram for explaining the operation of a conventional texture mapping process performed in the image synthesis circuit 2406 and the frame memory 2410 shown in FIG. 27;

FIG. 42 is a diagram illustrating a procedure of a texture mapping process in the image generation device 4110.

FIG. 43 is a diagram illustrating a procedure of a motion compensation process in the image generation device 4110.

FIG. 44 is a diagram illustrating a procedure of another motion compensation process in the image generation device 4110.

FIG. 45 is a view for explaining a processing procedure when the texture mapping processing and the motion compensation processing are simultaneously performed in the image generation device 4110.

[Explanation of symbols]

 101 CPU 104 Image generation device 105 Variable-length inverse coding unit 106 Orthogonal transformation unit 108 Video expansion mapping device 109 Frame memory 201 Control circuit 202 Pixel position designation circuit 203 Pixel calculation circuit 204 Image memory access circuit 205 Block image memory 206 Image movement circuit 207 Multiplier generation unit 208 Second multiplication unit 209 First multiplication unit 210 Addition unit 211 Second image memory access unit 212 First image memory access unit 1501 Pixel position designation circuit 1502 Image memory access circuit 1503 Second image Memory access unit 1504 First image memory access unit

Continuation of front page (56) References JP-A-5-35867 (JP, A) JP-A-7-73333 (JP, A) JP-A-7-230555 (JP, A) Les Kohn et al., “Out-of- UltraSPARC, which increases the operating frequency by omitting the order execution function, Nikkei Electronics January 30, 1995, Nikkei BP, January 30, 1995, No. 627, p. 109-120 (58) Field surveyed (Int. Cl. ⁷ , DB name) G06T 15/00-15/70 JICST file (JOIS)

Claims (28)

(57) [Claims] 1. A least motion compensation process or receives a command signal for instructing the processing including texture mapping, a first signal indicating the pixel positions corresponding respectively, signals indicative of a second pixel position, and a third A signal indicating a pixel position is output, and when the command signal indicates a motion compensation process, a position of a pixel in a reference image block is set as the first pixel position, and a position of a pixel in a generated image block is set as the first pixel position. The second pixel position is set, the position of the pixel in the difference image block is set as the third pixel position, and when the command signal indicates a texture mapping process, the position of the pixel in the polygon to be mapped is set as the second pixel position. A pixel position designating unit that sets the position of a pixel in the texture image as the third pixel position, the pixel position being the first pixel position and the second pixel position; And outputs a pixel value at the first pixel position as first pixel data.
And outputs the pixel value at the third pixel position as third pixel data, receives the signal indicating the second pixel position and the second pixel data, An image memory access means for writing the second pixel data at the pixel position of, and an operation between the first pixel data and the third pixel data, and using the operation result as the second pixel data and a pixel calculation means for outputting, the pixel position specifying means, said command signal is a motion compensation processing
And texture mapping processing at the same time
The pixel position in the reference image block
1 pixel position, and the
The position of the pixel in Ligon is defined as the second pixel position, and the difference image
The pixel position in the image block is output as the position of the third pixel.
Power, image generator. Wherein when said command signal instructs the motion compensation process, the second pixel position and the third pixel positions are the same, the image generating apparatus according to claim 1. Wherein when the command signal indicates a motion compensation process, the second pixel position and the third pixel positions are different, the image generating apparatus according to claim 1. Wherein said pixel calculation unit receives the command signal, when the command signal indicates the simultaneous processing of a case or motion compensation and texture mapping instructs the motion compensation process, wherein The first pixel data and the third pixel data are arithmetically processed as signed precision data, and when the command signal indicates a texture mapping process, the first pixel data and the third pixel data are processed. 2. The image generating apparatus according to claim 1 , wherein the pixel generating unit performs arithmetic processing on the pixel data as unsigned precision data. Wherein said pixel position specifying means receives the command signal, when the command signal indicates a motion compensation process, the relative position of the first of the reference image block of pixel positions The relative position of the third pixel position in the difference image block and the second pixel position,
2. The image generating apparatus according to claim 1 , wherein a pixel position at which the relative position of the pixel position in the generated image block is the same as the pixel position is specified. Wherein said pixel position specifying means receives the command signal, the command signal, if instructs the texture mapping process, the pixel position of each pixel on the horizontal lines in said generating polygon Calculating sequentially, designating the pixel positions as the first pixel position and the second pixel position, and further calculating pixel positions in the texture image corresponding to the first pixel position and the second pixel position. Determining the pixel position as the third pixel position,
When the command signal indicates a motion compensation process, the pixel position of each pixel on the horizontal line in the generated image block is sequentially calculated, and the pixel position is designated as the second pixel position, Further, the relative position of the third pixel position in the difference image block and the relative position of the first pixel position in the reference image block are the same as the relative position of the second pixel position in the generated image block. The image generating apparatus according to claim 5 , wherein the third pixel position and the first pixel position that are the same are designated. 7. The pixel position specifying means receives the command signal, and when the command signal indicates a texture mapping process, specifies a pixel position of each pixel on a horizontal line in the texture image. The pixel position is sequentially calculated, the pixel position is designated as the third pixel position, a pixel position in the generated polygon corresponding to the third pixel position is obtained, and this pixel position is determined by the first pixel position and the third pixel position. When the command signal indicates a motion compensation process, the pixel position of each pixel on the horizontal line in the difference image block is sequentially calculated, and the pixel position is designated as the pixel position. A third pixel position is designated, and a relative position of the first pixel position in the reference image block and a relative position of the second pixel position in the generated image block are determined by the third pixel position. The image generating apparatus according to claim 5 which specifies the relative position of the same to become such the first pixel position and the second pixel position in the difference image block of pixel positions. Wherein said pixel position specifying means receives the command signal, the command signal, if instructs to perform the motion compensation and texture mapping process at the same time, in said generating polygon The pixel position of each pixel on the horizontal line is sequentially calculated, the pixel position is designated as the second pixel position, and the pixel position mapped to the pixel data of the second pixel position is defined in the difference image block. This pixel position is designated as the third pixel position, and the relative position of the third pixel position in the difference image block and the relative position of the first pixel position in the reference image block are determined. the image generating apparatus according to claim 1 specifying the first pixel position as the same. 9. The pixel position specifying means receives the command signal, and, if the command signal indicates that a motion compensation process and a texture mapping process are to be simultaneously performed, the pixel position specifying means includes a command for executing the motion compensation process and the texture mapping process. Is sequentially calculated, the pixel position is designated as the third pixel position, and the pixel position in the generated polygon corresponding to the third pixel position is determined. A position is designated as the second pixel position, and the second pixel position is such that the relative position of the third pixel position in the difference image block is the same as the relative position of the first pixel position in the reference image block. claim 1 specifies a pixel position
An image generation device according to claim 1. Wherein said pixel calculation unit receives the command signal, the command signal, if instructs the motion compensation process, the first from the first pixel position of the reference image block Reading pixel data, reading the third pixel data from the third pixel position in the difference image block, performing an operation using the first pixel data and the third pixel data, and The said second
After performing a block motion compensation process of performing the procedure of writing the pixel data of the above-mentioned pixel data to the second pixel position indicating the pixel position in the generated image block for all the pixels in one image block, but the generated image block and the new difference image block and performs image block setting process for specifying the pixel position an image block in a position shifted by the reference image block and one pixel as the new reference image block, according to claim 1 An image generation device according to claim 1. Wherein said pixel position specifying means, image according to claim 10 for the pixel block setting processing for an image block is shifted by one pixel in the right direction with respect to the reference image block and the new reference image block Generator. 12. The pixel position specifying means, image according to claim 10 for the pixel block setting processing for an image block is shifted by one pixel in the downward direction with respect to the reference image block and the new reference image block Generator. 13. An image block setting process in which said pixel position specifying means sets said generated image block as a new difference image block and sets an image block shifted by one pixel from said reference image block as a new reference image block. 2
The image generation apparatus according to claim 10 , wherein the direction of the shift of the reference image block is different for each image block setting process in the image block setting process repeated twice or more times. 14. When the pixel position specifying means performs the image block setting processing two or more times, an image block shifted one pixel to the right with respect to the reference image block is defined as the new reference image block. And the case where an image block shifted downward by one pixel with respect to the reference image block is set as the new reference image block. claim and a case where the block with該新reference image block 1
3. The image generation device according to 3 . 15. At least a motion compensation process or a technique
A command signal that instructs processing including the posture mapping processing
And indicate the corresponding first pixel location
Signal, signal indicating second pixel position, and third pixel position
Output a signal indicating the position, and the command signal
When you are instructed, the pixels of the reference image blanking the lock
A position is defined as the first pixel position, and an image in the generated image block is
The elementary position is defined as the second pixel position, and the
The position of the pixel is designated as the third pixel position, and the command signal
Is instructing the texture mapping process,
The position of the pixel within the polygon to be mapped is
The pixel position and the second pixel position in the texture image
Pixel position specifying method for setting the position of the pixel
Receiving a signal indicating the stages, the pixel position of the first, the first pixel
The pixel value at the position is output as first pixel data,
Receives a signal indicating the pixel position of the third pixel position,
A pixel value is output as third pixel data, and the second pixel
Receiving the position signal and the second pixel data;
Image memo for writing the second pixel data at the pixel position of No. 2
Re-accessing means, and calculating between the first pixel data and the third pixel data
And outputs the operation result as the second pixel data
Comprising a pixel calculation unit that, the, after performing block motion compensation process is a motion compensation processing for one image block, the pixel position specifying means, and the texture image to generate image blocks generated by the block motion compensation process set, also performs image block setting processing for setting the newly generated polygons, the image memory access means, said generating polygons within said first pixel said in the texture image reading said first pixel data from the position reading the third the third from the pixel position of the pixel data, the pixel calculation means calculates by using the pixel data of the first pixel data and the third, the second pixel is the result the data subjected to processing for writing to the second pixel position in the generated polygons, sequentially perform these processes the pixel data in the generated polygons, the image generating apparatus. 16. A control means for outputting a command signal indicating the content to be processed by the image generating apparatus, and receiving the command signal, and when the command signal indicates a motion compensation process, a reference image for motion compensation. A pixel position in the block is determined and output as a first pixel position signal, and a pixel position in the generated image block is determined and output as a second pixel position signal, which is difference data between the reference image block and the generated image block. A pixel position in the difference image block is determined and output as a third pixel position signal. When the command signal indicates a texture mapping process, a pixel position in the texture image is determined and the third pixel position signal is determined. And the pixel position in the generated polygon to which the image data at the pixel position indicated by the third pixel position signal is mapped and written. Are obtained as the first pixel position signal and the second pixel position signal, and when the command signal indicates an image block setting process and an image block position, the reference image block and the generated image block Pixel position specifying means for newly setting the positions of the difference image block or the generated polygon and the texture image; the first pixel position signal, the second pixel position signal, and the third pixel position signal.
, The pixel data is read from the pixel position indicated by the first pixel position signal and output as the first pixel data signal, and the pixel data is read out from the pixel position indicated by the third pixel position signal. Image memory access means for reading data and outputting as a third pixel data signal, and writing pixel data input by the second pixel data signal at a pixel position indicated by the second pixel position signal; When the command signal indicates a motion compensation process for the pixel data indicated by the first pixel data signal and the third pixel data signal, the pixel data is converted into signed data in units of pixels. Perform the operation indicated by the command signal, and if the command signal indicates texture mapping processing, the image data is converted to unsigned data. Pixel operation means for performing the operation indicated by the command signal in pixel units, and outputting the operation result as the second pixel data signal, wherein the control means receives a control command for an image generation operation, An image generating apparatus for interpreting a command and outputting the command signal for instructing the pixel position designating means and the pixel calculating means of processing contents. 17. The image memory access means according to claim 1, wherein :
And a second pixel position signal, and the first
Pixel data is read from the pixel position indicated by the pixel position signal, and is output as the first pixel data signal;
A first image memory access unit for writing pixel data input by the second pixel data signal at a pixel position indicated by the second pixel position signal; and a third pixel position signal; A second image memory access unit that reads pixel data from a pixel position indicated by a pixel position signal of No. 3 and outputs the pixel data as a third pixel data signal, and stores a texture image or a difference image block; 17. The image generation apparatus according to claim 16 , further comprising a block image memory that outputs pixel data from a pixel position specified by the second image memory access unit. 18. The image memory access means according to claim 1, wherein :
A pixel position signal, a second pixel position signal, and a fourth pixel position signal, read pixel data from a pixel position indicated by the first pixel position signal, and output the read pixel data as a first pixel data signal; The pixel data is read from the pixel position indicated by the fourth pixel position signal and output as a fourth pixel data signal, and is input to the pixel position indicated by the second pixel position signal by the second pixel data signal. A first image memory access unit for writing the pixel data, a third pixel position signal and a fifth pixel position signal, and reading out pixel data from a pixel position indicated by the third pixel position signal. And writes the pixel data input by the fifth pixel data signal at the pixel position indicated by the fifth pixel position signal.
And a block image memory for storing a texture image or a difference image block and inputting / outputting pixel data from a pixel position designated by the second image memory access unit. Receiving the command signal, and when the command signal indicates an image block setting process and an image block position, sequentially sets the pixel read position in the difference image block or the texture image as the fourth pixel position signal. Outputting the pixel writing position in the block image memory sequentially as the fifth pixel position signal, and outputting the pixel data input as the fourth pixel data signal as the fifth pixel data signal. Moves the difference image block or the texture image to the block image memory Claim comprising an image moving means for
17. The image generation device according to 16 . 19. The image memory access means according to claim 1, wherein :
A pixel position signal and a second pixel position signal, read pixel data from a pixel position indicated by the first pixel position signal, and output the pixel data as the first pixel data signal;
A first image memory access unit for writing pixel data input by the second pixel data signal into a pixel position indicated by the second pixel position signal; The pixel data is read from the pixel position indicated by the pixel position signal, and is output as a third pixel data signal. The second pixel position signal is input, and the pixel position in the block image memory is determined by the second pixel position signal. In the case shown, the second pixel data which is input by the second pixel data signal is written into the block image memory.
And a block image memory for storing a texture image or a difference image block and inputting / outputting pixel data from a pixel position designated by the second image memory access unit. The image generating apparatus according to claim 16 . 20. A pixel position specifying means for obtaining a pixel position in a reference image block for motion compensation when a simultaneous execution of a motion compensation process and a texture mapping process is instructed by a control device as a first pixel position signal. 17. The image generation apparatus according to claim 16 , wherein the image generation device outputs the pixel position in the generated polygon, outputs the obtained pixel position as a second pixel position signal, and obtains the pixel position in the difference image block as a third pixel position signal. 21. A pixel generating means for generating a multiplier in accordance with an operation indicated by said control means,
A multiplication unit for multiplying the pixel data input by the first pixel data signal by a multiplier generated by the multiplier generation unit; and a calculation result of the pixel data input by the third pixel data signal and the multiplication unit claim 16 comprising an addition unit for outputting the added result of addition as the second pixel data signal
An image generation device according to claim 1. 22. A pixel generating means for generating a first multiplier and a second multiplier in accordance with the calculation indicated by the control means, and a pixel data input by the first pixel data signal. To the first number generated by the multiplier generation unit.
A first multiplier for multiplying the pixel data input by the third pixel data signal by a second multiplier generated by the multiplier generator; and a second multiplier for multiplying the pixel data input by the third pixel data signal by the second multiplier generated by the multiplier generator. 17. The image generation apparatus according to claim 16 , further comprising: an addition unit that adds an operation result of the multiplication unit of 1 and an operation result of the second multiplication unit and outputs an addition result as a second pixel data signal. 23. The multiplication section or said first multiplier unit is 1-fold, 1/2-fold, generating images according to claim 21 or claim 22 perform 1/4 times and 1/8 times of operation apparatus. 24. The number of pixels of the image data that can be stored in the block image memory, a reference image block, generates image blocks, according to claim 1 which is the same size as the differential image blocks
The image generation device according to any one of claims 7 to 19 . 25. An image generating apparatus according to claim 19 wherein the block image according to claim 17 memory capable of storing image data is rectangular image data of each 16 pixels vertically and horizontally. 26. The method of claim 25, wherein the block image memory, at least, the image generating apparatus according to claim 19 claim 17 which is implemented on the same semiconductor element and pixel calculation unit. 27. A compressed image block, which is a result of subjecting a difference image block before encoding obtained by performing a difference operation to a reference image block before encoding by motion prediction processing to an orthogonal transform and variable length encoding, And a variable-length inverse encoding means for performing a variable-length inverse encoding process on the compressed image block, and a reference image block and a difference image block by inputting the result of the variable-length inverse encoding process and performing orthogonal transformation.
And orthogonal transform means for outputting a result der of the orthogonal transform means
That of the reference image block and the differential image blocks enter the motion compensation processing performed-out further animal compensation processing results live
By providing an image generating apparatus according to any one of claims 26 claim 1 for performing texture mapping using the image data of the formed image block, the expansion processing and texture mapping processing on compressed video data Simultaneous video expansion mapping device. And 28. A CPU, a main memory, according to claim 2
7. A moving image decompression mapping device according to item 7 , and a frame memory, the frame memory being connected to the moving image decompression mapping device, storing at least a reference image and a generated image, and decompressing the compressed moving image data while performing decompression processing. Multimedia device that performs texture mapping on the surface of a three-dimensional object at the same time.