JPH09102047A - Picture generation device moving picture expanding/ mapping device and multi-media unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a processing circuit scale and to reduce data transfer quantity, which are required for executing a movement compensation processing in expanding a compressed picture and a texture mapping processing in generating three-dimensional CG(computer graphics). SOLUTION: The position of a picture element in a differential picture in a block picture memory 205, the position of the picture element in a reference picture in a frame memory 109 and the position of the picture element in a generated picture are designated from a picture element position designation circuit 202. Differential picture data and reference picture data are calculated in a picture element calculation circuit 203 and a result is written into the generated picture. Thus, a movement compensation processing is executed and the generated picture is moved to the block picture memory 205. The position of the picture element in the texture picture in the block picture memory 205 and the position of the picture element in a generated polygon in the frame memory 109 are designated from the picture element position designation circuit 202. Texture picture data and picture data of the generated polygon are calculated in the picture element calculation circuit 203 and the result is written into the generated polygon. Thus, a texture mapping processing is executed. Thus, the compressed picture is expanded and texture mapping can be executed. Consequently, the processing circuit scale can be reduced and data transfer quantity can be reduced.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention generates computer graphics (hereinafter referred to as "CG") and expands a highly efficient encoded digital moving image (hereinafter referred to as "compressed moving image expansion"). The present invention relates to an image generation device. More specifically, an image generation apparatus that realizes real-time generation of CG by a texture mapping method and decompression of a high-efficiency-encoded digital moving image using inter-frame motion compensation with an apparatus having a single configuration and an image generation apparatus The present invention relates to a moving image expansion mapping device and multimedia equipment used.

[0002]

2. Description of the Related Art Recently, in home TV game machines, personal computers, workstations, etc., C
Image data (hereinafter referred to as "texture image") prepared in advance in order to generate CG to be displayed and output on an RT display or the like is converted into a polygonal area (hereinafter referred to as "polygon").
A method of pasting), that is, a “texture mapping method” may be used. When the texture mapping method is used, when drawing a complicated three-dimensional solid shape, a relatively high-quality image can be obtained even if approximate drawing is performed with a few polygons.

On the other hand, in the case of displaying and outputting moving image data on a CRT display or the like in a personal computer, an audiovisual apparatus, etc., moving image data that has been compressed in advance is displayed while being expanded in real time. By using the compressed moving image data, there is an effect that the capacity of the storage medium and the data transmission amount can be reduced.

As one of the methods for efficiently compressing moving image data, there is a so-called inter-frame predictive coding method for coding only differential data of image data over a plurality of frames. For inter-frame predictive coding, “forward predictive coding” is performed by combining the images that take the difference,
There are "reverse predictive coding" and "bidirectional predictive coding".

In the input image sequence, an image to be compressed is an "input image", an image temporally earlier than the input image is a "past image", and an image later than the input image is a "future image". Then, the forward predictive coding encodes the difference data between the input image and the past image, and the backward predictive coding:
The difference data between the input image and the future image is coded, and the bidirectional predictive coding is performed by using both the past image and the future image. The past image and the future image are collectively referred to as a reference image.

Further, motion estimation processing is performed to increase the compression rate. The motion prediction process divides the input image into image blocks (for example, 16 × 16 pixels) of a certain size, not only the difference at the same position in the image when obtaining the difference data, but also corresponds to each block. This is a method in which the position where the difference is minimized is obtained by shifting in the horizontal and vertical directions in the reference image, and the block position and the difference data at that time are encoded. In this method, "forward prediction", "reverse prediction", "bidirectional prediction", or "no predictive coding" is adaptively applied to each image block so that the compression efficiency is highest. It is switched and encoded. Further, the position of the reference image block may be obtained with not only one pixel accuracy but also half pixel accuracy.

In recent years, there is a method of using a compression-encoded moving image as a texture image prepared in advance in the texture mapping method. That is, this is a method of mapping the expanded moving image on a polygon. An example of a conventional image generation apparatus that realizes this method is disclosed in Japanese Patent Laid-Open No. 6-162165.

In this conventional example, an image processing method is provided in which a moving image can be used as a texture image instead of a geometric pattern which has been conventionally used as a texture image or a still image such as a photograph or a handwritten pattern. By doing so, it is possible to generate a CG image rich in changes. The configuration of this conventional example is such that data reproducing means for reproducing compressed moving picture data from an information recording medium, image data expanding means for expanding compressed moving picture data reproduced by the data reproducing means, and expansion from the image data expanding means. And an image synthesizing means for synthesizing the image data by receiving the image data thus generated via a bus line.

[0009]

However, the above-mentioned conventional configuration has the following problems.

First, the configuration of a conventional image generating apparatus will be described with reference to FIG. In FIG. 27, the system bus 2
401 serves as a data transmission medium between blocks. CPU2
402 controls the entire system. The main memory 2403 stores programs and data. The image expansion circuit 2404 expands the compressed moving image data. The CD-ROM decoder 2405 reproduces data from the CD-ROM 2409 which is an information recording medium. The image synthesizing circuit 2406 synthesizes the decompressed moving image data on the frame memory 2410 by the texture mapping process. The coordinate conversion circuit 2407 obtains coordinates for image composition. The DMA controller 2408 transfers data on the system bus. The CD-ROM 2409 is an information storage medium that stores image data and the like. The frame memory 2410 is a work memory for image synthesis processing. The DA conversion circuit 2412 DA-converts the signal in order to display the image synthesis result.

Next, the operation of the image generating apparatus will be described. CD
-Compressed video data stored in ROM2409 is CD-R
It is read and reproduced by the OM decoder 2405. The reproduced compressed moving image data is transferred to the image expansion circuit 2404 via the system bus 2401. The image expansion circuit 2404 expands the compressed moving image data. The expanded moving image data is transferred again to the image combining circuit 2406 via the system bus 2401. On the other hand, the drawing coordinates are the CPU 2402 and the coordinate conversion circuit 2407.
Image synthesis circuit 24 via the system bus 2401
Instructed by 06. In the image synthesis circuit 2406, the image expansion circuit
The moving image data transferred from 2404 is combined with the coordinates on the frame memory 2410 designated by the CPU 2402 by texture mapping. The image combined on the frame memory 2410 is read by the image combining circuit 2406 and is DA conversion circuit.
Transferred to 2412. The DA conversion circuit 2412 converts the digital signal into an analog signal and outputs it.

FIG. 40 shows the image expansion circuit shown in FIG.
2404 shows the internal configuration. The operation of the motion compensation process will be described below with reference to FIG.

The compressed moving image data read from the CD-ROM decoder is input to the image decompression circuit 2404 via the system bus 2401 in image block units of, for example, 16 pixels × 16 pixels. The compressed image block data input to the image decompression circuit 2404 is first converted into the orthogonal transform coefficient image 3002 by the variable length inverse encoding circuit 3001. Then, it is converted into a difference image block by the orthogonal transformation circuit 3003 and stored in the difference image memory 3004. The compressed image block data input to the image decompression circuit 2404 includes not only the difference image component but also motion compensation data necessary for decoding this image block. Motion compensation data refers to the type of motion prediction processing used when this image block is coded (that is, "forward prediction", "reverse prediction", "bidirectional prediction", or "prediction coding". Data) indicating the position of the reference image block used for prediction. Based on the content of the motion compensation data, the subsequent motion compensation circuit 3013 performs motion compensation processing. From the reference image memory 3012, the image block at the position corresponding to the data indicating the position of the reference image block is read by the image reading circuit 300.
7, read through 3008. For example, when the image block to be converted is a block coded by forward prediction, the averaging / selecting circuit 3009 causes the past image memory
The image block read from 3005 is selected and input to the addition circuit 3010. The addition circuit 3010 adds the past image block data and the difference image block data,
It is written in the generated image memory 3011. Future image memory 3006 for image blocks encoded with backward prediction
The image block read from is selected and added to the difference image block. If the image block to be converted is a block encoded by bidirectional prediction, the averaging / selecting circuit 3009 determines whether the image block read from the past image memory 3005 and the image block read from the future image memory 3006. The average value for each pixel is calculated, and the result is sent to the addition circuit 3010, added with the difference image block, and stored in the generated image memory 3011. When the data indicating the position of the reference image block is given with 1/2 pixel precision, the image block read circuits 3007 and 3008 are adjacent to each other.
It has a reading function corresponding to 1/2 pixel accuracy, such as reading an image block while calculating the average value of one pixel. The expanded moving image data stored in the generated image memory 3011 is transferred to the image combining circuit via the system bus 2401. The content of the generated image memory 3011 is also transferred to the past image memory 3005 or the future image memory 3006 in the reference image memory 3012 for use in the subsequent moving image expansion. The above is the internal configuration of the conventional image decompression circuit and the operation of the motion compensation process.

Next, the operation of the conventional texture mapping process performed in the image synthesizing circuit 2406 and the frame memory 2410 shown in FIG. 27 will be described with reference to FIG.

The moving image data transferred to the image synthesizing circuit 2406 is temporarily stored in the frame memory 2410 as the texture image data 3101 when the image synthesizing circuit performs the texture mapping operation. Then, the texture image 3101 is read by the image synthesizing circuit, the image is transformed, and stored (mapped) in the generated image 3102 area. One pixel 3104 in the texture image 3101 is input to the first multiplication circuit 3107 in the texture mapping circuit 3106. At the same time, the pixel 3105 in the generated image area 3102 to which this pixel is mapped is input to the second multiplication circuit 3108. The first multiplication circuit 3107 multiplies a multiplier α (where α is a numerical value of 0 or more and 1 or less), and the second multiplication circuit 3108 multiplies a multiplier (1-α), and the respective multiplication results are added. It is input to the circuit 3109 and added. The result is stored in the position of the pixel 3105 in the generated image 3102. By sequentially performing the above operation for all pixels of the texture image, a texture mapping process involving translucent synthesis of the original image (which is generally a background image) in the generated image and the texture image is realized. It The above is the operation of the texture mapping process performed by the conventional image synthesizing circuit.

In the image generating apparatus as described above, the compressed moving image data is transferred to the image decompression circuit 2404 via the system bus 2401 in order to decompress the moving image, and the moving image data of the decompression result is re-imaged via the system bus 2401. Transferred to the synthesis circuit 2406. The amount of decompressed moving image data is generally very large (for a moving image with 320 x 240 pixels, about 7M is required).
(B / sec), the data transfer capacity of the system bus 2401 will be heavily loaded. In addition, an inter-frame motion prediction technique is often used when moving images are compressed with high efficiency, and a frame memory is required to decompress a moving image compressed based on this technique. That is, if the image decompression circuit 2404 and the image composition circuit 2406 are separately provided, the frame memory for image decompression processing and the frame memory 2410 for drawing an image must be separately provided. Become. Further, since the image decompression circuit 2404 and the image synthesis circuit 2406 are connected via the system bus 2401, the control such as the timing control of data transfer is complicated and the system configuration is complicated.

The conventional image generating apparatus has many problems as described above.

The present invention has been made in view of the above circumstances, and an object of the present invention is to expand a digital moving image encoded by using inter-frame motion prediction.
Simplification of the device configuration by realizing CG image generation by a texture mapping method with a single configuration,
An object of the present invention is to provide an image generation apparatus that can reduce the circuit scale of a processing device, reduce the amount of memory required for processing, and reduce the amount of data transfer by eliminating moving image data transfer between memories.

[0019]

An image generating apparatus of the present invention receives a command signal instructing processing including at least motion compensation processing or texture mapping processing,
According to the processing, the positions of at least two images in the memory are respectively designated, the image data of the at least two images are received, an operation is performed using the image data of the at least two images, and the operation result is newly generated. By outputting the generated image to a predetermined position on the memory, the common processing of the motion compensation processing and the texture mapping processing is performed by a single means, thereby achieving the above object.

Another image generation apparatus of the present invention receives a command signal instructing at least a process including a motion compensation process or a texture mapping process, and outputs a signal indicating a corresponding first pixel position and a second pixel position, respectively. And a signal indicating the third pixel position, and when the command signal indicates the motion compensation process, the position of the pixel in the reference image block is set as the first pixel position, and the generated image is generated. The position of the pixel in the block is the second pixel position, the position of the pixel in the difference image block is the third pixel position, and when the command signal indicates texture mapping processing, the polygon to be mapped A pixel position in which the positions of pixels in the texture image are defined as the first pixel position and the second pixel position, and the positions of pixels in the texture image are defined as the third pixel position. Designating means receives a signal indicating a pixel position of the first, the pixel value of the pixel position of the first first
Of the second pixel position, the signal indicating the third pixel position is received, the pixel value of the third pixel position is output as third pixel data, and the signal indicating the second pixel position and the second signal are output. Image memory access means for receiving the second pixel data and writing the second pixel data to the second pixel position,
The above-described object is achieved by including a pixel operation unit that performs an operation between the first pixel data and the third pixel data and outputs the operation result as the second pixel data. To be done.

When the command indicates a motion compensation process, the second pixel position and the third pixel position may be the same.

When the command indicates a motion compensation process, the second pixel position and the third pixel position may be different.

Preferably, the pixel position designating means sets the position of the pixel in the reference image block as the first pixel position when the command signal indicates the simultaneous processing of the motion compensation processing and the texture mapping processing. The pixel position in the generated polygon to be texture-mapped is output as the second pixel position, and the pixel position in the difference image block is output as the third pixel position.

Preferably, the pixel calculation means receives the command signal, and when the command signal indicates a motion compensation process, or when a motion compensation process and a texture mapping process are simultaneously instructed, If the first pixel data and the third pixel data are arithmetically processed as signed precision data, and the command signal indicates texture mapping processing, the first pixel data
The pixel data and the third pixel data are processed as data with accuracy without a code.

Preferably, the pixel position specifying means receives the command signal, and when the command signal indicates a motion compensation process, the relative position of the first pixel position in the reference image block. And a pixel position at which the relative position of the third pixel position in the difference image block and the relative position of the second pixel position in the generated image block are the same.

Preferably, the pixel position designating means receives the command signal, and when the command signal indicates a texture mapping process, the pixel position of each pixel on a horizontal line in the generated polygon. Are sequentially calculated, and the pixel position is set to the first pixel position and the second pixel position.
Pixel position in the texture image corresponding to the first pixel position and the second pixel position, and the pixel position is specified as the third pixel position. When the signal indicates the motion compensation process, the pixel position of each pixel on the horizontal line in the generated image block is sequentially calculated, and the pixel position is designated as the second pixel position. The relative position of the third pixel position in the difference image block and the relative position of the first pixel position in the reference image block are the same as the relative position of the second pixel position in the generated image block. The third pixel position and the first pixel position such that

The pixel position designating means receives the command signal, and when the command signal indicates a texture mapping process, sequentially calculates the pixel position of each pixel on a horizontal line in the texture image. Then, the pixel position is designated as the third pixel position, the pixel position in the generated polygon corresponding to the third pixel position is obtained, and this pixel position is determined as the first pixel position and the second pixel position.
When the command signal indicates the motion compensation process, the pixel position of each pixel on the horizontal line in the difference image block is sequentially calculated, and the pixel position is calculated as 3 pixel position, and the first
So that the relative position of the pixel position in the reference image block and the relative position of the second pixel position in the generated image block are the same as the relative position of the third pixel position in the difference image block. The first pixel position and the second pixel position may be designated.

Preferably, the pixel position designating means receives the command signal, and when the command signal indicates to execute the motion compensation process and the texture mapping process at the same time, within the generated polygon. The pixel position of each pixel on the horizontal line is sequentially calculated, and this pixel position is designated as the second pixel position.
The pixel position mapped to the pixel data of the pixel position is determined in the difference image block, this pixel position is designated as the third pixel position, and the relative position of the third pixel position in the difference image block. And the relative position of the first pixel position in the reference image block is the same, the first pixel position is designated.

The pixel position specifying means receives the command signal, and when the command signal indicates to simultaneously execute the motion compensation processing and the texture mapping processing, the horizontal position in the difference image block is set. The pixel position of each pixel on the line is sequentially calculated, this pixel position is designated as the third pixel position, the pixel position in the generated polygon corresponding to the third pixel position is obtained, and this pixel position is calculated. The first pixel position is designated as the second pixel position, and the relative position of the third pixel position in the difference image block is the same as the relative position of the first pixel position in the reference image block. The pixel position may be designated.

Preferably, the pixel calculation means receives the command signal, and when the command signal indicates a motion compensation process, the first pixel position from the first pixel position in the reference image block. Of the pixel data, read the third pixel data from the third pixel position in the difference image block, calculate using the first pixel data and the third pixel data, and calculate the calculation result. After performing a block motion compensation process for writing all the pixels in one image block, the procedure of writing the second pixel data that is the second pixel position indicating the pixel position in the generated image block, The pixel position designating unit uses the generated image block as a new difference image block, and an image block at a position displaced by one pixel from the reference image block as a new reference image. Performing image block setting process for specifying a pixel position as a lock.

The pixel position designating means may perform a pixel block setting process in which an image block shifted by one pixel to the right with respect to the reference image block is set as the new reference image block.

The pixel position designating means may perform a pixel block setting process in which an image block shifted by one pixel in the downward direction from the reference image block is used as the new reference image block.

An image block setting process in which the pixel position specifying means sets the generated image block as a new difference image block, and sets an image block at a position shifted by one pixel from the reference image block as a new reference image block In the image block setting process that is repeated twice or more and twice or more, the direction of deviation of the reference image block may be different for each image block setting process.

In the case where the pixel position designating means performs the image block setting process two or more times, the image block shifted by one pixel to the right from the reference image block is set as the new reference image block. And a case where an image block shifted downward by one pixel with respect to the reference image block is used as the new reference image block, and an image block shifted downward by one pixel in the lower right direction with respect to the reference image block. The new reference image block may be used.

After performing the block motion compensation process which is the motion compensation process for one image block, the pixel position designating means sets the generated image block generated by the block motion compensation process as a texture image, and newly. Image block setting processing for setting a generation polygon is performed, and the image memory access unit reads the first pixel data from the first pixel position in the generation polygon and the third pixel position in the texture image. The third pixel data is read out, and the pixel calculation means calculates using the first pixel data and the third pixel data, and the resulting second pixel data is stored in the generated polygon. The writing processing may be performed at the second pixel position of, and the processing may be sequentially performed on the pixel data in the generated polygon.

Still another image generating apparatus of the present invention includes a control means for outputting a command signal for instructing the contents to be processed by the image generating apparatus, the command signal for receiving the command signal, and the command signal for instructing the motion compensation processing. In this case, the pixel position in the reference image block for motion compensation is obtained and output as the first pixel position signal, and the pixel position in the generated image block is obtained and output as the second pixel position signal. If the pixel position in the difference image block, which is the difference data of the generated image block, is obtained and output as the third pixel position signal, and the command signal indicates texture mapping processing, the pixel position in the texture image And output as the third pixel position signal, and the image data at the pixel position indicated by the third pixel position signal is mapped and written. When the pixel position in the generated polygon to be generated is obtained and is output as the first pixel position signal and the second pixel position signal, and the command signal indicates the image block setting process and the image block position, Pixel position specifying means for newly setting the position of the reference image block, the generated image block, the difference image block, or the generated polygon, the texture image, the first pixel position signal, and the second pixel position A signal and the third pixel position signal are input, pixel data is read from the pixel position indicated by the first pixel position signal, and is output as a first pixel data signal, which is indicated by the third pixel position signal. Pixel data is read from the pixel position to be output and output as a third pixel data signal, and is input to the pixel position indicated by the second pixel position signal by the second pixel data signal The image memory access means for writing the stored pixel data, and the command signal for the pixel data indicated by the input first pixel data signal and the input third pixel data signal. If the command signal indicates a texture mapping process, the pixel data is processed as the data with a sign in pixel units, and if the command signal indicates a texture mapping process, the image data is processed as a data without a sign in the pixel unit. And perform the operation instructed by the command signal,
Pixel calculation means for outputting a calculation result as the second pixel data signal, the control means receiving a control command for an image generation operation, interpreting the control command, the pixel position designation means, the pixel calculation The command signal for instructing the processing contents is output to the means, whereby the above object is achieved.

Preferably, the image memory access means inputs a first pixel position signal and a second pixel position signal, reads pixel data from a pixel position indicated by the first pixel position signal, and outputs the pixel data. A first image memory access unit for outputting the pixel data signal of No. 1 and writing the pixel data input by the second pixel data signal at the pixel position indicated by the second pixel position signal; A second image memory access unit for inputting a pixel position signal, reading pixel data from a pixel position indicated by the third pixel position signal, and outputting the pixel data as a third pixel data signal. A differential image block is stored, and a block image memory for outputting pixel data from the pixel position designated by the second image memory access unit is provided. I have.

Preferably, the image memory access means inputs the first pixel position signal, the second pixel position signal and the fourth pixel position signal, and outputs the pixel position indicated by the first pixel position signal. Pixel data is read out and output as a first pixel data signal, pixel data is read out from a pixel position indicated by the fourth pixel position signal, and is output as a fourth pixel data signal, and the second pixel position signal is output. The first image memory access unit for writing the pixel data input by the second pixel data signal to the indicated pixel position, the third pixel position signal and the fifth pixel position signal are input, and the third pixel position signal is input. Pixel data is read from the pixel position indicated by the pixel position signal and output as a third pixel data signal, and the fifth pixel data signal is output to the pixel position indicated by the fifth pixel position signal. A second image memory access unit for writing the pixel data input by the second image memory access unit, stores a texture image or a difference image block, and stores a pixel from a pixel position designated by the second image memory access unit. A block image memory for inputting / outputting data is provided, and when the command signal is received and the command signal indicates an image block setting process and an image block position, a pixel read position in the difference image block or the texture image Are sequentially output as the fourth pixel position signal, pixel writing positions in the block image memory are sequentially output as the fifth pixel position signal, and the pixel data input as the fourth pixel data signal is output. By outputting as a fifth pixel data signal, the difference image block or the tech And an image moving means for moving the tea image on the block image memory.

Preferably, the image memory access means inputs the first pixel position signal and the second pixel position signal, reads pixel data from a pixel position indicated by the first pixel position signal, and outputs the pixel data. A first image memory access unit for outputting the pixel data signal of No. 1 and writing the pixel data input by the second pixel data signal at the pixel position indicated by the second pixel position signal; The pixel position signal is input, the pixel data is read from the pixel position indicated by the third pixel position signal and is output as a third pixel data signal, the second pixel position signal is input, and the second pixel position signal is input. A second image memory for writing the pixel data input by the second pixel data signal to the block image memory when the pixel position in the block image memory is indicated. A block image memory that stores a texture image or a difference image block and that inputs and outputs pixel data from a pixel position designated by the second image memory access unit. .

Preferably, the pixel position specifying means obtains a pixel position in a reference image block for motion compensation when the control means instructs the simultaneous execution of the motion compensation process and the texture mapping process to obtain a first pixel position signal. As the second pixel position signal, the pixel position in the generated polygon is obtained, and the pixel position in the difference image block is obtained and output as the third pixel position signal.

Preferably, the pixel calculation means generates a multiplier according to the calculation indicated by the control means, and a multiplier generation section for the pixel data input by the first pixel data signal. A multiplication unit for multiplying the calculated multiplier, the pixel data input by the third pixel data signal, and the calculation result of the multiplication unit, and the addition result is a second addition result.
And an addition unit that outputs the pixel data signal of.

Preferably, the pixel calculation means is a multiplier generation section for generating a first multiplier and a second multiplier according to the calculation indicated by the control means, and a pixel input by the first pixel data signal. A first multiplication unit that multiplies data by a first multiplier generated by the multiplier generation unit, and a second multiplier generated by the multiplier generation unit for pixel data input by the third pixel data signal. And a second multiplication unit that multiplies by, and an addition unit that adds the calculation result of the first multiplication unit and the calculation result of the second multiplication unit and outputs the addition result as a second pixel data signal.

The multiplying unit or the first multiplying unit is 1
It may be possible to perform only double, 1/2 times, 1/4 times and 1/8 times operations.

The number of pixels of image data that can be stored in the block image memory may be the same size as the reference image block, the generated image block, and the difference image block.

The image data that can be stored in the block image memory may be rectangular image data having 16 pixels in each length and width.

Preferably, the block image memory is
At least it is realized on the same semiconductor element as the pixel calculation means.

The moving picture expansion mapping apparatus of the present invention performs the difference calculation with the reference image block by the motion prediction process to perform the orthogonal transformation and the variable length coding on the difference image block, and outputs the compressed image block. As input,
Variable length inverse encoding means for performing variable length inverse encoding processing on the compressed image block, orthogonal transformation means for inputting the result of the variable length inverse encoding processing, and orthogonal transformation, and results of the orthogonal transformation means. And the image generation apparatus according to any one of claims 1 to 26, which performs a motion compensation process and further performs a texture mapping process using the image data of the result of the motion compensation process. The decompression process and the texture mapping process are simultaneously performed on the moving image data, thereby achieving the above object.

The multimedia device of the present invention comprises a CPU
28. A main memory, a moving picture expansion mapping device according to claim 27, and a frame memory, the frame memory being connected to the moving picture expansion mapping device, storing at least a reference image and a generated image, and compressing a moving picture. While the data is being expanded, the image resulting from the expansion is simultaneously texture-mapped on the surface of the three-dimensional object, thereby achieving the above object.

According to the present invention, the compressed moving picture decompression processing using the motion compensation processing and the texture mapping processing are processed by a single memory and a single processing device, so that many of the processings which are the problems of the conventional technique are performed. Points that require memory,
This is done by focusing on the point that it is possible to solve the problem that a large amount of data needs to be transferred from the motion compensation processing device to the texture mapping processing device.

By newly devising the processing procedure of the motion compensation processing, the procedure of memory access, and the data arrangement method in the memory, the motion compensation processing is performed by the apparatus for performing the texture mapping processing and the apparatus having a configuration similar to the memory configuration. It is possible to do. As a result, both the texture mapping process and the motion compensation process can be processed by a single image generation device and a single image memory.

A basic texture mapping processing apparatus uses a texture image that is rectangular image data and a generated image that is an image that is being generated, and uses the texture image in the texture image that corresponds to the image data that constitutes the polygon in the generated image. The processing is performed by reading the image data and the image data already drawn in the generated image from the image memory, weighting and adding the already drawn image data and the texture image data, and writing them again in the image memory as the generated image. .

Considering the operation procedure of such texture mapping processing, the processing procedure of the motion compensation processing described below was devised.

The reference image and the difference image for motion compensation processing are stored in the image memory, and the image data in the reference image is read at the same time as the image data in the difference image is read out.
After multiplying the reference image data by a coefficient as needed, the reference image data is added to the difference image data and written in the generated image in the image memory. By doing so, it becomes possible to perform motion compensation processing by a device having a configuration similar to the texture mapping processing device.

Further, in the motion compensation processing, bidirectional motion compensation processing may be performed using two future and past reference images, or the position of the reference image may be specified with half-pixel accuracy.

In the present invention, the bidirectional motion compensation process is performed by performing the motion compensation process by using the reference image and the difference image of either one of the two as described above and writing the result back to the image memory. This is realized by regarding the rewritten image as a difference image and performing similar motion compensation processing between this and another reference image. At this time, an accurate bidirectional motion compensation process can be performed by setting the coefficient by which the reference image data is multiplied to 0.5.

When the reference image position with half-pixel precision is designated, the reference image is read from a position shifted by half a pixel from the designated reference image position with half-pixel precision, and this image data is read. On the other hand, the motion compensation processing is performed by adding the difference image data and writing it back to the image memory, and the motion compensation processing result is newly set as the difference image, and the position of the reference image is shifted by one pixel, and the image data And the motion compensation process is repeated.
Repeat the motion compensation processing for a reference image that is shifted by half a pixel vertically and horizontally (or vertically or horizontally) with respect to a reference image with half-pixel precision position specification, and at the same time control the coefficient by which the reference image data is multiplied. By doing so, it is possible to average the reference image data of the upper, lower, left, and right sides to obtain the reference image for specifying the half-pixel position, and to perform the motion compensation process with the reference image and the difference image at the same time.

By devising a new processing method in which the motion compensation processing is realized by the weighted addition of the reference image and the difference image, the write-back processing to the memory, and the repetition of this processing as described above, the texture mapping processing is performed. It has become possible to implement by a similar processing procedure, memory access procedure, and data allocation method in the memory.

As a result, the motion compensation process and the texture mapping process can be realized by a single image processing device and a single memory configuration, which is necessary when both the motion compensation process and the texture mapping process are performed. Compared with each dedicated processing unit and dedicated memory,
It is possible to reduce the circuit amount and the memory amount of the processing device and at the same time greatly reduce the data transfer amount between the memories.

Further, the motion compensation processing and the texture mapping processing, which have conventionally been sequentially performed, can be simultaneously processed.

The operation will be described below.

According to the present invention, with the above configuration, it is possible to realize the decompression process of the digital moving image compressed using the motion compensation prediction and the texture mapping process to the polygon in the CG generation with a single configuration. is there. Furthermore, by simultaneously performing the compressed moving image decompression processing and the texture mapping processing, it is possible to map the moving image decompressed by the compressed moving picture decompression processing onto a polygon as a texture image of the texture mapping processing.

In image compression by motion compensation prediction, a similar block is searched for from a block in the past image (hereinafter referred to as a past image block) for a block area in the image being compressed, and this block is referred to as the past image block. The compression is performed by taking the difference from the image block being compressed. Further, not only the past image but also the future image is searched for a similar block (hereinafter, this block is referred to as a future image block), and the difference from this is also taken. Further, the positions of the past image block and the future image block in the image are also designated with an accuracy of half a pixel.

The decompression of the image thus compressed takes out a reference image block (a past image block or a future image block) which is a reference for motion prediction from the past image or the future image, and the image in the image currently being reproduced. Processing is performed by adding difference image data (hereinafter, referred to as a difference image block) between a block (hereinafter, referred to as a generated image block) and a reference image block to the reference image block. The decompression process is performed by performing the same process for all the blocks forming the entire image. Such decompression processing is called motion compensation processing. The motion prediction process during compression encoding may be performed using a past image, a future image, or both a past image and a future image as the reference image, and may not use the reference image. The optimum prediction method may be selected for each block. In order to deal with this, it is necessary to switch the motion compensation method for each processing of the image block even in the motion compensation processing.

In the following description, decompression processing from the past image block is forward motion compensation, decompression processing from the future image block is backward motion compensation, and decompression processing from the future or past reference image block is decompressed. The processing is referred to as unidirectional motion compensation, and the decompression processing using both the past image block and the future image block is referred to as bidirectional motion compensation. Also, the pixel data in the past image block is P (x, y), the pixel data in the future image block is F (x, y), the pixel data in the reference image block is R (x, y), and the difference image block is Pixel data in D (x,
y), the pixel data in the generated image block will be expressed as G (x, y). Here, x and y represent pixel positions in the image block.

On the other hand, in the texture mapping process, the drawing pixel position in the generated image and the reference pixel position in the texture image are obtained based on the vertex coordinate information of the polygon, and the pixel data at the reference pixel position in the texture image is read out. The pixel data is processed by writing the pixel data at the drawing pixel position in the generated image. An image of the entire screen is generated by similarly performing texture mapping processing on all the polygons forming the screen.

Further, the motion compensation process is performed on one image block, and the image block generated subsequently is used as the texture image to perform the texture mapping process, so that the moving image decompression process and the texture mapping process are simultaneously performed in parallel. Is possible. Further, in the present invention, it is possible to simultaneously perform the motion compensation processing and the texture mapping processing.

The image generating apparatus of the present invention has a pixel calculating means for calculating using two pieces of image data. The pixel calculation means has different types of image data to be calculated according to the contents to be processed. That is, when performing the motion compensation process, the motion compensation process is performed using the image data of the reference image and the image data of the difference image, and when performing the texture mapping process, the image data of the texture image is used. , Texture data is calculated using the image data in the polygon. As a result, the image generating apparatus of the present invention can perform the motion compensation processing and the texture mapping processing with the same apparatus, and as a result, the circuit scale can be reduced. Also, in conventional clothing,
It is possible to reduce the amount of memory required for processing by eliminating the need for data transfer.

In the following, the respective processes of the basic image generating device will be described in detail, and then the processes and features of another image generating device of the present invention, a moving image expansion mapping device using these image generating devices, and the like. The operation of the system will be described.

First, the basic motion compensation processing of the image generating apparatus will be described.

The forward motion compensation process or the backward motion compensation process, that is, the unidirectional motion compensation process is processed as follows.

The image data G (x, y) of the image block generated by the one-way motion compensation is the same as the image data D (x, y) of the differential image block as shown in (Equation 1). Image data R of the reference image block in the reference image that became the reference
It is calculated by adding (x, y).

[0072]

## EQU1 ## G (x, y) = R (x, y) + D (x, y) The reference image and the difference image are stored in the image memory. Pixel data in the difference image block of the difference image is sequentially designated by the pixel position designating means, read by the image memory access means, and input to the pixel computing means. From the reference image, a reference image block that is a reference for motion prediction of the currently generated image block is selected, pixel data in the reference image block is sequentially designated by the pixel position designating means, and read by the image memory access means. And input to the pixel calculation means. In the pixel calculation means, two input pixel data are added, and the calculation result is sequentially written via the image memory access means to the pixel position designated by the pixel position designation means in the generated image block in the image memory. By performing the calculation shown in (Equation 1) in this way, the unidirectional motion compensation process, that is, the forward motion compensation process and the backward motion compensation process are performed.

The bidirectional motion compensation process will be described below.

For bidirectional motion compensation, the average of the results of the forward motion compensation and the result of the backward motion compensation may be obtained, and the bidirectional motion compensation is processed by performing the calculation shown in (Equation 2).

[0075]

[Equation 2] G (x, y) = ((P (x, y) + D (x, y)) + (F (x, y) +
D (x, y)) / 2 In the present invention, the operation of (Equation 2) is transformed into (Equation 3), and the processing is performed according to this.

[0076]

[Equation 3] G (x, y) = (P (x, y) / 2 + D (x, y)) + F (x, y) / 2 That is, for the difference image block, The bidirectional motion compensation process is realized by adding 0.5 times the pixel data and further adding 0.5 times the pixel data of the future image block to this result.

The image memory stores past images, future images and difference images as motion compensation reference images. Pixel data in the difference image block of the difference image is sequentially designated by the pixel position designating means, read by the image memory access means, and input to the pixel computing means. This input pixel data is D (x, y). From the past image, a past image block which is a reference for motion prediction of the currently generated image block is selected, and pixel data in the past image block is sequentially designated by the pixel position designating means and read by the image memory accessing means. And input to the pixel calculation means. This input pixel data is P (x, y). In the pixel calculation means, P (x, y) / 2 + D (x, y) is obtained, and the calculation result is the image memory access to the pixel position designated by the pixel position designation means in the generated image block in the image memory. Sequentially written through the means. When the processing in one generated image block is completed, the image block just generated is regarded as a difference image block, and the pixel data in this difference image block is sequentially instructed by the pixel position specifying means and read out by the image memory access means. It is input to the calculation means. This input pixel data is P (x, y) / 2 + D (x, y). From the future image, the future image block that is the reference for motion prediction of the currently generated image block is selected, and the pixel data in the future image block is sequentially designated by the pixel position designating means and read by the image memory access means. And input to the pixel calculation means. This input pixel data is F (x, y). In the pixel calculation means, {P (x, y) / 2 + D (x, y)} + F (x, y) / 2
Is calculated, and the calculation results are sequentially written to the pixel positions designated by the pixel position designating means in the generated image block in the image memory via the image memory access means. The bidirectional motion compensation process is realized by performing the calculation shown in (Equation 3) as described above.

Next, the half-pixel precision motion compensation processing will be described. In the motion compensation with half-pixel accuracy, the position designation of the reference image block may deviate by half a pixel from the position where the pixel exists. In this case, the specified reference image block is obtained by performing an interpolation operation based on the image blocks located up and down and to the left and right by half a pixel from the specified reference image block position, and the motion compensation processing is performed using this as a reference. To do.

The operation of the one-way motion compensation processing with half pixel accuracy will be described below. When the position of the reference image block R (x, y) is specified by being shifted by half a pixel in the horizontal direction from the position where the pixel exists, the reference image block (R0 (x, y)) and the reference image block (R1 (x, y)) that is "right" by half a pixel from the specified position, the reference image block R (x, y) is obtained according to (Equation 4).

[0080]

[Equation 4] R (x, y) = (R0 (x, y) + R1 (x, y)) / 2 The position of the reference image block R (x, y) is half the vertical position than the pixel position. When specified with a pixel offset, the reference image block (R0 (x, y) that is "upper" than the specified position by half a pixel
Then, the reference image block R (x, y) is obtained according to (Equation 5) from the reference image block (R2 (x, y)) that is “down” by half a pixel from the specified position.

[0081]

[Equation 5] R (x, y) = (R0 (x, y) + R2 (x, y)) / 2 The position of the reference image block R (x, y) is half the vertical and horizontal directions than the pixel position. When specified with a pixel offset, the reference image block (R0
(x1, y)), and a reference image block that is "upper right" by half a pixel than the specified position (assumed to be R1 (x, y)), and a reference image block that is "lower left" by half a pixel than the specified position (R2
(x, y)) and the reference image block (R3 (x, y)) that is "lower right" by half a pixel from the specified position, according to (Equation 6), the reference image block R (x, y) ).

[0082]

(6) R (x, y) = (R0 (x, y) + R1 (x, y) + R2 (x, y) +
R3 (x, y)) / 4 half-pixel precision motion compensation is based on the image data of the reference image block and the image data of the motion compensation difference image block obtained as described above, and Processing is performed according to the bidirectional motion compensation processing procedure.

That is, the one-way motion compensation is calculated according to (Equation 7).

[0084]

[Formula 7] When half pixel is shifted in the horizontal direction G (x, y) = (R0 (x, y) + R1 (x, y)) / 2 + D (x, y) Half pixel is shifted in the vertical direction G (x, y) = (R0 (x, y) + R2 (x, y)) / 2 + D (x, y) G / x, y) G + (x, y) = (R0 (x, y) + R1 (x, y) + R2 (x, y) + R3 (x, y)) / 4
+ D (x, y) In another configuration of the present invention, processing is performed according to (Equation 8) which is a modification of (Equation 7).

[0085]

[Equation 8] When half pixel is shifted in the horizontal direction G (x, y) = (R0 (x, y) / 2 + D (x, y)) + R1 (x, y) / 2 Half in the vertical direction Pixel shift G (x, y) = (R0 (x, y) / 2 + D (x, y)) + R2 (x, y) / 2 Half pixel shift in both horizontal and vertical directions G (x , y) = [{(R0 (x, y) / 4 + D (x, y)) + R1 (x, y) / 4} + R2
(x, y) / 4] + R3 (x, y) / 4 If the pixel is shifted by half a pixel only in the horizontal or vertical direction,
As can be seen by comparing with (Equation 3), the same operation procedure as the bidirectional motion compensation is used.

The image memory stores a motion compensation reference image and a difference image. Pixel data in the difference image block of the difference image is sequentially designated by the pixel position designating means, read by the image memory access means, and input to the pixel computing means. This input pixel data is D (x, y).
From the reference image block, pixel data R0 (x, y) in the reference image block which is deviated by a half pixel from the specified position is sequentially instructed by the pixel position specifying means, read out by the image memory access means and input to the pixel calculation means. It In the pixel calculation means, R0 (x, y) / 2 + D (x, y) is obtained, and the calculation result is the image memory access to the pixel position designated by the pixel position designation means in the generated image block in the image memory. Sequentially written through the means. When the processing in one generated image block is completed, the image block just generated is regarded as a difference image block, and the pixel data in this difference image block is sequentially instructed by the pixel position specifying means and read out by the image memory access means. It is input to the calculation means.
This input pixel data is R0 (x, y) / 2 + D (x, y). Again, from the reference image, R1 (x, y) or R2 (x, y) pixel data is sequentially designated by the pixel position designating means, read by the image memory accessing means, and input to the pixel computing means. In the pixel calculation means, {R0 (x, y) / 2 + D (x, y)} + R1 (x, y) / 2 or
{R0 (x, y) / 2 + D (x, y)} + R2 (x, y) / 2 was obtained, and the operation result was instructed by the pixel position specifying means in the generated image block in the image memory. The data is sequentially written in the pixel positions via the image memory access means. The pixel data G (x, y) can be obtained as described above.

If there is a half pixel shift in both the vertical and horizontal directions, the following procedure is taken.

The image memory stores a motion compensation reference image and a difference image. The pixel data D (x, y) in the difference image block of the difference image is sequentially read and input to the pixel calculation means. Pixel data R0 (x, in the reference image block deviated from the reference image block by half a pixel from the specified position
y) is input to the pixel calculation means. In the pixel calculation means, R0
(x, y) / 4 + D (x, y) is obtained, and the calculation result is written in the generated image block in the image memory. When the processing in one generated image block is completed, the image block just generated is regarded as a difference image block, and the pixel data R0 (x, y) / 4 + D (x, y) in this difference image block is read sequentially. It is output and input to the pixel calculation means. From the reference image, the pixel data of R1 (x, y) is sequentially read and input to the pixel calculation means. In the pixel calculation means, (R0 (x, y) / 4 + D (x, y)) + R1 (x, y)
/ 4 is obtained, and the calculation result is written in the generated image block in the image memory. The generated image block is regarded as the difference image block, and the pixel data (R0 (x, y) / 4 + D (x, y)) + R1 (x, y) / 4 in this difference image block are read sequentially. It is output and input to the pixel calculation means. From the reference image, R2
Pixel data of (x, y) is sequentially read and input to the pixel calculation means. In the pixel calculation means, ((R0 (x, y) / 4 + D (x, y)) +
R1 (x, y) / 4} + R2 (x, y) / 4 is obtained, and the calculation result is written in the generated image block in the image memory. Furthermore, the image block just generated is regarded as a difference image block,
Pixel data in this difference image block {(R0 (x, y) / 4 + D
(x, y)) + R1 (x, y) / 4} + R2 (x, y) / 4 are sequentially read and input to the pixel calculation means. From the reference image, the pixel data of R3 (x, y) are sequentially read and input to the pixel calculation means. In the pixel calculation means, [((R0 (x, y) / 4 + D (x, y)) + R1 (x,
y) / 4} + R2 (x, y) / 4] + R3 (x, y) / 4 is obtained, and the calculation result is written in the generated image block in the image memory.
In this way, the pixel data G (x, y) of the generated image block
Can be obtained.

In order to perform bidirectional motion compensation processing with half-pixel precision, half-pixel precision forward motion compensation and half-pixel precision backward motion compensation are obtained by the method described above and averaged. Although there are many cases corresponding to the designation of the image block positions of the past image block and the future image block, basically the same process as the one-way motion compensation may be performed. After the difference image data is added with the image data of the image block that is half a pixel shifted from the specified pixel position of the reference image block multiplied by the multiplication coefficient according to the ratio of the interpolation operation, and the result is written as the generated image block. It is only necessary to regard this generated image block as a new difference image block, add another reference image block to it, and write it to the generated image block.

The texture mapping processing of the basic image generating apparatus of the present invention will be described below.

A texture image is stored in the image memory. Based on the vertex coordinates of the polygon to be texture-mapped, the pixel position designating unit determines the reference pixel position in the texture image and the drawing pixel position in the generated polygon in the generated image. The pixel data of the pixel position is read out through the image memory access means by the reference pixel position in the texture image and given to the pixel calculation means. The pixel calculation means outputs the input pixel data as it is as the calculation result. The calculation result is written to the drawing pixel position in the generated polygon specified by the pixel position specifying means via the image memory access means. The texture mapping process is performed by performing this process for all the pixels in the generated polygon.

Further, the pixel data in the generated polygon read from the image memory is input to the pixel calculation means, and the pixel calculation is performed with the texture pixel data read from the texture image to obtain a semi-transparent texture. It is possible to perform texture mapping processing of images and anti-aliasing processing of polygon edges in the texture mapping processing.

The texture mapping process for the semi-transparent texture image will be described in detail below. The translucency of the texture image is α, the value of the pixel data already written in the drawing pixel position of the generated polygon is A, and the pixel data of the texture image to be mapped is B. The pixel data A already written in the drawing pixel position has its pixel position specified by the pixel position specifying means and is read out via the image memory access means. The read pixel data A is input to the pixel calculation means. Pixel data B of texture image
The pixel position is specified by the pixel position specifying means and is read out through the image memory access means. The read pixel data B is input to the pixel calculation means. The pixel calculation means calculates α × A + (1−α) × B. The pixel data of the calculation result is written to the drawing pixel position in the generated polygon designated by the pixel position designating means through the image memory access means. In this way, the texture mapping of the semi-transparent texture image becomes possible. Also,
For the anti-aliasing of the polygon edge, the same processing as that of the semi-transparent texture may be performed on only the pixels forming the polygon edge. By the above processing, the semi-transparent texture mapping and the edge anti-aliasing processing can be performed.

Simultaneous execution of motion compensation processing and texture mapping processing, that is, texture mapping of a compressed image will be described below.

The compressed image which is the target of the motion compensation process is a collection of a plurality of block images which are units of the motion compensation process. These blocks are designated as B0 to B99.
The polygon to be texture-mapped is defined for each block image which is a unit of motion compensation. These polygons are represented as P0 to P99. Here, 100 from B0 to B99
The image made of individual block images is P0 to P99 of 10
It is assumed that an image that is texture-mapped to 0 polygons is generated.

First, the motion compensation process is performed on the B0 image block according to the above-described motion compensation process method. Next, the decompressed image block is used as a texture image, and this block image is subjected to texture mapping according to the above-described texture mapping processing method for the polygon P0. In this way, the motion compensation process and the texture mapping process are repeated in block units, and all blocks (B0 ~
By performing processing on B99), decompression processing from the reference image and difference image consisting of multiple image blocks and texture mapping processing using this decompression result as a texture image are performed simultaneously in a single configuration. Can be done in.

Further, by specifying the pixel position in the pixel position specifying means as described below, it is possible to simultaneously execute the motion compensation process and the texture mapping process of the block image. That is, the pixel position in the reference image block is designated as the first pixel position, the pixel position in the generated polygon is designated as the second pixel position, and the third pixel position is designated.
The pixel position in the difference image block is designated as the pixel position of.

Pixel data is read from the first pixel position in the reference image block, and pixel data in the difference image block corresponding to this pixel data is read from the third pixel position in the difference image block. These pixel data are calculated by the pixel calculation means, and the resulting pixel data is written in the pixel position in the generated polygon designated by the second pixel position. This process is sequentially performed on all the pixels in the generated polygon. By performing the processing in this way, it is possible to perform the decompression processing by the motion compensation processing based on the difference image block compressed by the motion prediction and the reference image, and at the same time, deform and draw the polygon of the generated polygon.

As described above, with the image generating apparatus of the present invention, it is possible to realize both texture mapping processing and compressed moving picture decompression processing by motion compensation with a single configuration, and simultaneously execute these processings. Is possible.

Further, the image data of the difference image block in the case of the motion compensation process or the texture image data in the case of the texture mapping process may be stored in the block image memory.

By separating the memory storing the generated image and the reference image from the block image memory, the access to the pixel data in the reference image block conflicts with the access to the pixel data in the difference image block. Or
It is possible to avoid a collision between access to the pixel data in the texture image and access to the pixel data in the generated polygon. The size of the difference image block is generally about 8 × 8 pixels to 16 × 16 pixels, and a memory of this size constitutes an image generating device.
Since it can be built in the LSI, the circuit can be downsized and the processing speed (block image memory access speed) can be increased.

Furthermore, it is also possible to adopt a configuration that enables the image block of the generated image to be moved to the block image memory. As a result, the bidirectional motion compensation process and the half-pixel precision motion compensation process of the above-described basic image generation device are performed in the following procedure.

The processing procedure of bidirectional motion compensation is performed according to the above (Equation 3). That is, bidirectional motion compensation is performed by adding 0.5 times the pixel data of the past image block to the difference image block, and further adding 0.5 times the pixel data of the future image block to this result. To be realized. The bidirectional motion compensation process will be described in more detail below.

First, forward motion compensation processing is performed. Difference image blocks are stored in the block image memory. Pixel data D (x, y) designated by the third pixel position signal is sequentially read from the block image memory via the second image memory access unit. On the other hand, the pixel data P (x, y) of the past image block designated as the first pixel position signal is sequentially read from the past image block via the first image memory access unit. These pixel data are input to the pixel calculation means, and P (x, y) / 2 + D (x, y) is obtained. The pixel position designating unit designates a pixel position in the generated image block by the second pixel position signal, and the pixel data obtained by the pixel computing unit is stored in the designated pixel position in the generated image block as the first pixel position. Written via the image memory access unit. This process is sequentially performed on the pixel data of the entire image block.

Next, the image moving means sequentially designates the pixel position in the generated image block obtained by performing the forward motion compensation processing as the fourth pixel position signal, and sets the first image memory access unit. Pixel data is read out via. On the other hand, the image moving means sequentially specifies the pixel position in the block image memory as the fifth pixel position signal, and writes the pixel data read from the generated image block into the block image memory via the second image memory access unit. In this way, the block image resulting from the forward motion compensation processing is moved to the block image memory.

Next, reverse motion compensation processing is performed. Third
Pixel data specified by the pixel position signal of P (x, y) / 2 + D
(x, y) are sequentially read out via the second image memory access unit. On the other hand, from the future image block, the pixel data of the future image block designated as the first pixel position signal
F (x, y) is sequentially read out via the first image memory access unit. These pixel data are input to the pixel calculation means, and {P (x, y) / 2 + D (x, y)} + F (x, y) / 2 are obtained.
From the pixel position specifying means, the pixel position in the generated image block of the generated image is specified as the second pixel position signal,
The obtained pixel data is written via the first image memory access unit. This process is sequentially performed on the pixel data of the entire image block.

The calculation shown in (Equation 3) is performed as described above. The bidirectional motion compensation process is performed by performing the above process for all the image blocks forming the generated image.

The half-pixel precision motion compensation processing will be described below.

First, the differential image block is stored in the block image memory, and the pixel data designated by the third pixel position signal is sequentially read out via the second image memory access section. On the other hand, the pixel data designated as the first pixel position signal is sequentially read from the reference image block via the first image memory access unit. These pixel data are input to the pixel calculation means and pixel calculation is performed. From the pixel position designating means, the pixel position in the generated image block is designated by the second pixel position signal,
The pixel data obtained by the pixel calculation means is written at the designated pixel position in the generated image block via the first image memory access unit. This process is sequentially performed on the pixel data of the entire image block, and the block motion compensation process is performed.

Next, the image moving means sequentially specifies the pixel positions in the generated image block obtained by performing the one-way motion compensation process as the fourth pixel position signal, and sets the first image memory access unit. Pixel data is read out via. On the other hand, the image moving means sequentially specifies the pixel position in the block image memory as the fifth pixel position signal, and writes the pixel data read from the generated image block into the block image memory via the second image memory access unit. In this way, the block image resulting from the one-way motion compensation process is moved to the block image memory.

Half-pixel precision motion compensation can be performed by repeating the above block motion compensation processing and moving the image block to the block image memory as many times as necessary for the half-pixel precision motion compensation processing. Becomes

By adopting the above-mentioned configuration, it is possible to achieve both the realization of the texture mapping process and the realization of the compressed moving image decompression process by the half-pixel precision unidirectional / bidirectional motion compensation with a single configuration. Becomes

In still another configuration, the pixel data output from the pixel calculation means can be written in the block image memory via the second image memory access section. Thus, the bidirectional motion compensation processing and the half-pixel precision motion compensation processing described in the basic configuration are performed in the following procedure.

The processing procedure of bidirectional motion compensation is performed according to the above (Equation 3).

That is, in the bidirectional motion compensation, 0.5 times the pixel data of the past image block is first added to the motion-compensated difference image block, and the result is further reduced to 0.5 of the pixel data of the future image block. It is realized by adding 5 times.

First, forward motion compensation processing is performed. A difference image block is stored in the block image memory, and the pixel data D (x, y) designated by the third pixel position signal is stored.
Are sequentially read out via the second image memory access unit. On the other hand, from the past image block, the pixel data P (x, y) of the past image block designated as the first pixel position signal
Are sequentially read out via the first image memory access unit. These pixel data are input to the pixel calculation means,
P (x, y) / 2 + D (x, y) is obtained. The pixel position designating means designates a pixel position in the block image memory as a second pixel position signal, and the pixel data obtained by the pixel computing means is stored in the designated pixel position in the block image memory as the second pixel position signal. Written via the image memory access unit.
This process is sequentially performed on the pixel data of the entire image block.

Next, reverse motion compensation processing is performed. Third
Pixel data specified by the pixel position signal of P (x, y) / 2 + D
(x, y) are sequentially read out via the second image memory access unit. On the other hand, from the future image block, the pixel data of the future image block designated as the first pixel position signal
F (x, y) is sequentially read out via the first image memory access unit. These pixel data are input to the pixel calculation means, and {P (x, y) / 2 + D (x, y)} + F (x, y) / 2 are obtained.
From the pixel position specifying means, the pixel position in the generated image block of the generated image is specified as the second pixel position signal,
The obtained pixel data is written via the first image memory access unit. This process is sequentially performed on the pixel data of the entire image block.

The calculation shown in (Equation 3) is performed as described above. The bidirectional motion compensation process is performed by performing the above process for all the image blocks forming the generated image.

Furthermore, the half-pixel precision motion compensation process can be performed as follows. That is, the pixel data of the difference image block stored in the block image memory and the pixel data of the reference image block are used for calculation by the pixel calculation means, and the calculation result is written back to the block image memory. Only the block motion compensation process performed last among the plurality of repeated processes of the half-pixel precision motion compensation process is written in the generated image via the first image memory access unit.

By performing the processing in this way, it is not necessary to move the image block data, which was necessary in the above processing, and it is possible to greatly reduce the number of memory accesses.

The image data in the block image memory is referred to only once for each pixel in the block motion compensation process.
Since the calculation result is written back to the referenced pixel position, there is no problem in processing the image data to be calculated even if the calculation result is overwritten.

As described above, with a single configuration, it is possible to achieve both the realization of texture mapping processing and the realization of compressed moving picture expansion processing by half-pixel precision unidirectional / bidirectional motion compensation. Further, as compared with the case described above, the number of accesses to the block image memory is reduced, so that the processing speed can be increased.

As described above, the compressed moving image decompression processing by the motion compensation processing and the texture mapping processing are realized with a single structure, thereby simplifying the structure of the image generating apparatus.
It is possible to reduce the circuit scale, reduce the amount of memory required for processing, and reduce the amount of data transfer by eliminating moving image data transfer between memories.

In the moving picture expansion mapping apparatus using the above-mentioned image generating apparatus, moving picture compression using motion prediction of image blocks, orthogonal transformation such as DCT, variable length coding such as Huffman coding, represented by MPEG, is used. We perform video decompression processing and texture mapping processing using the decompressed video as a texture image for the digital video compressed and encoded by the method. The details of the processing will be described below.

In the compressed moving image, the image block of the difference result between the compressed reference image data obtained by orthogonally transforming and variable-length coding the reference image and the image block forming the reference image is orthogonally transformed,
It is composed of variable length encoded compressed difference image data.

In the moving picture expansion mapping apparatus of the present invention, the compression reference image data is first subjected to variable length inverse encoding, and orthogonal conversion is performed to obtain an expansion reference image. If the texture mapping process is necessary, the texture mapping process is performed using this image as a texture image. Next, the compressed difference image data is subjected to variable length inverse encoding, and orthogonal transformation is performed to obtain a difference image. Further, a motion compensation process is performed by the image generating apparatus of the present invention to obtain an expanded image (generated image). If texture mapping processing is required, this generated image is used as a texture image and texture mapping processing is performed. Alternatively, as in the operation of the image generation apparatus described above, the motion compensation process and the texture mapping process are simultaneously executed on the difference image.

As described above, the moving picture decompression mapping device of the present invention is capable of decompressing a compressed moving picture and performing texture mapping processing using this moving picture as a texture image. And can be realized with a small-scale device.

In a multimedia device using the above-described moving picture expansion mapping device, moving picture expansion processing and texture mapping processing are performed as follows.

The CPU controls the moving picture decompression mapping device and other parts of the system and executes the application program according to the program stored in the main memory. In the moving image expansion mapping device, an image is generated and displayed on the frame memory based on the instruction and data from the CPU.

In this system, the operation of the process of texture-compressing and displaying a compressed moving image on a three-dimensional polygon will be described. It is assumed that compressed moving image data and a three-dimensional polygon shape are stored in the storage device of the multimedia device.

The CPU reads the compressed moving image data and the polygon shape data from the storage device into the main memory. CPU
Then, the three-dimensional polygon shape is converted into a two-dimensional polygon shape on the screen. Next, the CPU transmits the compressed moving image data to the moving image expansion mapping device, and instructs the image block setting process. The image generation device in the moving image expansion mapping device sets a generated image area and a reference image in the frame memory. Next, the CPU gives an instruction for decompression processing.
The moving picture expansion mapping device performs variable length inverse encoding processing and orthogonal transformation processing on given compressed moving picture data.
Further, a motion compensation process is performed by an image generation device in the moving image expansion mapping device to generate an expanded image in a generated image area in the frame memory. Next, the CPU instructs the moving image expansion mapping device to perform the image block setting process, and the generated image data is set as a texture image and a newly generated polygon is set. Then, the CPU gives a two-dimensional polygon shape to instruct the texture mapping process. Based on the set texture image and the generated polygon, the image generation device performs texture mapping processing to generate a polygon image in which the moving image data is texture-mapped. With respect to one piece of decompressed image data, not only one polygon shape but also a plurality of polygon shapes are given and it is possible to perform mapping to a complicated shape.

Further, in the above description, the texture mapping process is performed after performing the image decompression process to obtain the decompressed image, but it is also possible to alternately perform the moving image decompression process and the texture mapping process in image block units. . Furthermore, it is possible to simultaneously execute the motion compensation process and the texture mapping process in moving image expansion.

Since it is possible to configure and operate a multimedia device in this way, decompressed image data is not transferred to the system bus, and the load of data transfer on the system bus is greatly reduced. . Further, since the moving image decompression process and the texture mapping process are processed using the same frame memory, it is possible to reduce the memory amount in the system.

As described above, according to the present invention, the decompression process of the digital moving image compressed using the inter-frame motion prediction, the texture mapping process to the polygon in the CG generation, and the decompression process and the texture mapping process are performed simultaneously. It becomes possible to realize an image generation device, a moving image expansion mapping device, and a multimedia device that can be performed. As a result, it is possible to simplify the configuration of the device and equipment, reduce the circuit scale, reduce the amount of memory required for processing, and reduce the amount of data transfer by eliminating the need for moving image data transfer between memories. .

[0135]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of an image processing device, a moving image expansion mapping device, and a multimedia device according to the present invention will be described below with reference to the drawings.

By using the present invention, it is possible to decompress moving image data compressed based on motion estimation and generate an image by texture mapping using the decompressed moving image data as a texture image.

Motion estimation is to reduce the data amount by obtaining the most similar block from a past or future image for a certain block image in the image and obtaining the difference. In a moving image compression encoding method such as MPEG, as shown in FIG.
For 182 (16 × 16 pixel image block, etc.), the difference is calculated from the similar image block 181 in the reference image to perform motion prediction, and further orthogonal transformation (DCT, etc.) and variable length coding (Huffman coding, etc.) And is encoded.

Decompression of a moving image encoded in this way is
As shown in FIG. 22, the compressed moving image data is first subjected to variable-length inverse coding and converted into an orthogonal transform coefficient image 191, and then orthogonal transform is performed on this image to transform it into a difference image 192. Difference image 192
Is divided into image blocks, and motion compensation processing is performed for each block, that is, an image block 193 serving as a motion prediction reference is taken out from the reference image, and this image data and differential image data are calculated to obtain an image of the motion compensation result. The reference image for the motion prediction process during compression encoding is a past image (forward motion prediction), a future image (reverse motion prediction), or both past and future images (bidirectional motion prediction). In some cases, the reference image is not used (no motion prediction), and the optimum prediction method is selected for each block of motion prediction. In order to deal with this, it is necessary to switch the motion compensation method for each processing of the image block even in the motion compensation processing.

In the image generation apparatus of the present invention, not only the motion compensation processing shown in FIG. 22 but also the image transformation by the texture mapping processing shown in FIG. 23, and the motion compensation processing and texture mapping shown in FIG. Simultaneous execution of processing is possible with a single device.

Texture mapping of a moving image is performed by, for example,
When a three-dimensional effect is added to a moving image in video editing or multimedia equipment (see FIG. 25 (A)), or when a moving image is pasted into an image generated by three-dimensional CG with a sense of perspective (Fig. 25 (see (B)) and the like. By using the present invention, the texture mapping process for compressed moving image data can be efficiently processed.

In the following, regarding the multimedia device of the present invention using the above-mentioned image generation device, the system configuration, the image generation device configuration and operation, the operation of the moving picture expansion mapping device using the image generation device, and the operation of the multimedia device. Will be explained.

(Embodiment 1) FIG. 1 shows a block diagram of a first embodiment of a multimedia device according to the present invention.

The multimedia device 100 of this embodiment is configured as follows. The CPU 101 controls the entire system, the moving image expansion mapping device, the image generation device, the disk device, and the like. Main memory
Reference numeral 102 stores a program for controlling the image generating apparatus executed by the CPU, a program describing the operation of each part of the system, an image or other data read from the disk device, and data used by the program. .
The disk device 103 stores compressed image data, texture image data, a drawing program, and the like. The moving picture decompression mapping device 108 performs variable length inverse encoding processing on the compressed moving picture data written by the CPU 101 and outputs a variable length inverse encoding unit 105 for outputting orthogonal transform coefficient image data. An orthogonal transformation unit 106 that performs orthogonal transformation processing to output difference image data, an image generation device 104, and a local bus 107 for connecting these to the frame memory 109 are provided. The frame memory 109 stores the reference image, the difference image, and the generated image used by the image generation device 104. The DA converter 110 stores the image generated by the image generator 104 in the frame memory 109.
Further, DA conversion is performed for display on the readout display device 111. The display device 111 displays the image output from the DA converter 110. The system bus 112 connects the CPU 101, the main memory 102, the disk device 103, and the moving image expansion mapping device 108 to exchange data.

Next, regarding the multimedia device 100,
The operations of decompressing a compressed moving image, texture mapping processing, and texture mapping using a compressed moving image as texture data will be described.

Here, a description will be given of the operation of compressing moving image data by MPEG stored in the disk device 103, performing the moving image decompressing process while reading this, and displaying it as a moving image. This operation procedure is stored in the main memory as a program.

The compressed moving image data sequence based on MPEG is subjected to the compression coding by performing the motion prediction based on the I picture in which the intra-frame coding process of one image is performed without the motion prediction and the past image. It is composed of a P picture and a B picture that has been subjected to a compression coding process by performing motion prediction from both the past image and the future image. Here, it is assumed that the moving image is composed of image 1, image 2, image 3, and image 4 in order. Image 1 is compressed as an I picture,
The image 4 is compressed as a P-picture in which the forward motion prediction is performed using the image 1 as the reference image, and the images 2 and 3 are the images 1 and 1.
It is assumed that the image 4 is compressed as a B picture that has been subjected to bidirectional motion prediction.

First, the CPU 101 reads the compressed data of the image 1 from the disk device 103. Since this data is an I picture, motion compensation processing is unnecessary. The compressed image is divided into image blocks (written as compressed image blocks), and each block is sent to the moving image expansion mapping device 108. Since the compressed image block has been Huffman coded, the variable length inverse encoding unit 105 performs inverse transformation and outputs the result to the orthogonal transformation unit 106. The orthogonal transform unit 106 performs the inverse DCT transform and writes the result in the generated image in the frame memory 109. The decompressed image 1 is generated in the frame memory 109 by similarly processing all the compressed image blocks.

Next, the image 1 generated in the frame memory 109 is set as the reference image. The CPU 101 reads the compressed image data of image 4 from the disk device 103. This data is a P picture and requires forward motion compensation processing. The compressed image is divided into compressed image block units, and each block is sent to the moving image expansion mapping device 108. Since the compressed image block is Huffman encoded, the variable length inverse encoding unit 105 performs inverse transformation, and the result is obtained by the orthogonal transformation unit 106.
Output to The orthogonal transform unit 106 performs the inverse DCT transform and writes the result in the difference image in the frame memory 109.
The image generation apparatus 104 performs forward motion compensation processing based on the reference image block in the reference image (image 1) and the difference image block in the difference image, and the image 4 is stored in the frame memory 109.
To be generated.

From this point in time, the DA converter 110 reads the image 1 in the frame memory 109 and displays it on the display device 111.

Next, the already generated images 1 and 4 are set as reference images. The CPU 101 reads the compressed image data of image 2 from the disk device 103. This data is a B picture, and bidirectional motion compensation processing is required. The compressed image is divided into compressed image block units, and each block is sent to the moving image expansion mapping device 108. Since the compressed image block is Huffman coded, the variable length inverse coding unit 105
The inverse transform is performed in and output to the orthogonal transform unit 106. The orthogonal transform unit 106 performs the inverse DCT transform and writes the result in the difference image in the frame memory 109. Past image (Image 1)
The image generation apparatus 104 performs bidirectional motion compensation processing to generate image 2 based on the reference image block in the image, the reference image block in the future image (image 4), and the difference image block in the difference image.

From this point in time, the DA converter 110 reads out the image 2 in the frame memory 109 and displays it on the display device 111.

Next, the CPU 101 reads the compressed image data of the image 3 from the disk device 103. This data is a B picture, and bidirectional motion compensation processing is required. The compressed image is divided into compressed image block units, and each block is sent to the moving image expansion mapping device 108. Since the compressed image block is Huffman coded, the variable length inverse coding unit 105
The inverse transform is performed in and output to the orthogonal transform unit 106. The orthogonal transform unit 106 performs the inverse DCT transform and writes the result in the difference image in the frame memory 109. Past image (Image 1)
The image generation apparatus 104 performs bidirectional motion compensation processing to generate image 3 based on the reference image block in the image, the reference image block in the future image (image 4), and the difference image block in the difference image.

From this point, the DA converter reads the image 3 in the frame memory 109 and displays it on the display device. When the display of the image 3 is finished, the image 4 already generated
Is read by the DA converter 110 and displayed on the display device 111.

As described above, in the multimedia device 100, it is possible to decompress and display the image data compressed and encoded by the MPEG. This procedure is shown in the timing chart of FIG.

In the above description, the motion prediction method in one image is fixed to one, but in MPEG, an image compressed without motion prediction is included in an image block forming a P picture. The presence of an image block that has been subjected to forward motion prediction or compression without motion prediction is allowed in the block and the image block forming the B picture.
Even in such a case, in the present embodiment, it is possible to perform the decompression process by performing the above process while resetting the motion compensation method for each image block.

Here, the texture image is a disk device.
The method of reading the image stored in 103, performing texture mapping on the polygon, and displaying the generated image will be described. This operation procedure is stored in the main memory as a program.

First, the CPU 101 reads a texture image from the disk device 103 and writes it in the frame memory 109 via the moving image expansion mapping device 108. The CPU 101 designates the polygon position (vertex coordinates) in the generated image generated in the frame memory 109 to the image generating device 104, and designates the position of the texture image in the frame memory to the image generating device 104 to perform texture mapping. Give instructions. The image generation device 104 draws the generated polygon in the generated image from the texture image in the frame memory 109.
When the texture mapping processing for all polygons in the generated image is completed, the completed generated image is stored in the frame memory 109.
The DA converter 110 reads out the data from, and displays it on the display device 111. In order to generate the next image, the CPU 101 writes the newly required texture data in the frame memory 109, and then the image generation device 104 is instructed to the polygon position to perform texture mapping.

Here, processing for displaying moving images mapped to polygons by storing moving image data compressed by MPEG is stored in the disk device 103, while performing moving image decompression processing while reading this, and simultaneously performing texture mapping processing. . This operation procedure is stored in the main memory 102 as a program. It is assumed that the moving image is composed of an image 1 and an image 2 in order, the image 1 is compressed as an I picture, and the image 2 is compressed as a P picture in which forward motion prediction is performed using the image 1 as a reference image. It is assumed that the image 1 is a reference image for the motion compensation process of the image 2, but the image 2 is not a reference image for another motion compensation process.

First, the CPU 101 reads the compressed data of the image 1 from the disk device 103. Since this data is an I picture, motion compensation processing is unnecessary. Divide the compressed image into image blocks (hereinafter referred to as compressed image blocks),
Each block is sent to the moving picture expansion mapping device 108. Since the compressed image block has been Huffman coded, the variable length inverse encoding unit 105 performs inverse transformation and outputs the result to the orthogonal transformation unit 106. The orthogonal transform unit 106 performs the inverse DCT transform and writes the result in the texture image in the frame memory 109. Here, the CPU 101 causes the image generation device 104 to specify the image block expanded in the above description as a texture image, and instructs the position of the polygon to be generated to perform texture mapping. By processing all the compressed image blocks in the same manner, the decompressed image 1 and the generated image obtained by texture mapping the image 1 and mapping it to polygons are obtained in the frame memory 109.

At this point, the DA converter 110 reads the image 1 in the frame memory 109 and displays it on the display device 111.

Next, the CPU 101 reads the compressed data of the image 2 from the disk device 103. This data is a P picture, and it is necessary to perform forward motion compensation processing using the image 1 as a reference image. The compressed image is divided into image blocks (written as compressed image blocks), and each block is sent to the moving image expansion mapping device 108. Since the compressed image block has been Huffman coded, the variable length inverse encoding unit 105 performs inverse transformation and outputs the result to the orthogonal transformation unit 106. The orthogonal transform unit 106 performs the inverse DCT transform and writes the result to the differential image block in the differential image in the frame memory 109. Here, the CPU 101 designates this difference image block as a difference image block, designates the reference image block of the reference image (image 1) already written in the frame memory 109, and the position of the generated polygon in the generated image. Is specified, and the image generation apparatus 104 is instructed to execute the moving image decompression process and the texture mapping simultaneously. Image generator
In 104, while performing the forward motion compensation process from the difference image and the reference image, the pixel data subjected to the motion compensation process is texture-mapped in the generated polygon. By processing all compressed image blocks in the same manner, a generated image in which the decompressed image 2 is texture-mapped to polygons is obtained in the frame memory 109. The result of this processing is only the image that has been texture-mapped and deformed, but as described above, image 2 does not become a reference image for motion compensation of the next image, so no problem occurs.

At this time, the DA converter 110 reads the image 2 in the frame memory 109 and displays it on the display device 111.

As described above, in the multimedia device shown in FIG. 1, it is possible to realize the expansion process of compressed moving images such as MPEG, the texture mapping process, and the process of texture mapping using the compressed moving images as texture data.

The configuration and operation of the image generating apparatus 104 used in the above multimedia equipment will be described below in detail. The image generation device 104 is configured as shown in FIG.

The control circuit 201 interprets the command specified by the command input signal 20a, and the pixel position specifying circuit 202,
A command signal for controlling the operations of the image moving circuit 206 and the pixel arithmetic circuit 203 is output. The pixel position specifying circuit 202 is
According to an instruction (command signal) from the control circuit 201, a pixel position (first pixel position) in the reference image block or the generation polygon is obtained, and this is output as a first pixel position signal 20b. Pixel position (second pixel position) is obtained and output as the second pixel position signal 20c, and the pixel position (third pixel position) in the difference image block or texture image is obtained as the third pixel position signal 20d. Output. The image memory access circuit 204 receives the first pixel position signal 20b and reads the pixel data at the specified pixel position in the frame memory 109 to obtain the first pixel data signal 20e.
And outputs the second pixel position signal 20c to the specified pixel position in the frame memory 109 to output the second pixel data signal 2
The first image memory access unit 212 that writes the pixel data input as 0f, receives the fourth pixel position signal 20h, reads the pixel data at the specified pixel position in the frame memory 109, and outputs the read pixel data as the fourth pixel data signal 20j. Then, the third pixel position signal 20d is received and the pixel data at the specified pixel position in the block image memory 205 is read out and output as a third pixel data signal 20g, and the fifth pixel position signal 20i is received in the block image memory. And a second image memory access unit 211 for writing the pixel data input as the fifth pixel data signal 20k at the designated pixel position. Pixel calculation circuit 2
The 03 inputs the first pixel data signal 20e and the third pixel data signal 20g and outputs the result of calculating these data as a second pixel data signal 20f. In this embodiment,
The pixel arithmetic circuit 203 generates a first multiplier and a second multiplier according to an instruction from the control circuit 201, and outputs the multiplier to the first multiplier and the second multiplier, and a multiplier generator 207. A first multiplication unit 209 that multiplies the first pixel data specified by from the first pixel data, and a second multiplication unit that multiplies the second multiplier specified by the multiplier generation unit 207 by the third pixel data. Part 208
And an addition unit 210 that adds the calculation result of the first multiplication unit 209 and the calculation result of the second multiplication unit 208. If the instruction (command signal) from the control circuit 201 indicates a motion compensation process, or if the pixel calculation circuit 203 indicates a simultaneous process of the motion compensation process and the texture mapping process, the pixel calculation circuit 203 outputs the data with the code. When the texture mapping process is instructed, the calculation is performed with accuracy, and the data is calculated with unsigned data precision.

The image moving circuit 206 generates the fourth pixel position and the fifth pixel position for moving the image block in the frame memory 109 to the block image memory 205 according to the instruction from the control circuit 201. Fourth pixel position signal 20
The pixel data designated by h is read from the frame memory 109 via the image memory access circuit 204, and this image data is placed at the pixel position in the block image memory 205 designated by the fifth pixel position signal 20i. Write through.

The image generating device 104 configured as described above
The operation of will be described below. Image generation device 104 is CP
Operation is instructed by command by U101, control circuit 2
It operates by interpreting the command at 01 and outputting a command signal for controlling each circuit of the image generating apparatus 104 accordingly. In the following description, a control procedure by issuing a command to the control circuit 201 by the CPU 101 and an operation procedure of each circuit in the image generating apparatus 104 which receives this control will be described.

The operation of the motion compensation processing in the image generating apparatus 104 will be described below. The control procedure of the image generation device 104 by the CPU 101 is shown in FIGS.
5 and 6 show the operation procedure of each circuit. In FIGS. 5 and 6, broken line arrows represent the flow of data / control signals, and solid line arrows represent the flow of processing.

First, the CPU 10 for performing motion compensation processing
The control procedure of the image generation device 104 according to 1 will be described with reference to FIGS. 3 and 4.

In STEP 301, a reference image (past image or future image) serving as a reference for motion compensation is set in the frame memory 109, and a difference image is written. The reference image is the disk device 10
There are a case where the image is read out from the memory 3 and written in the frame memory 109 by the CPU 101, and a case where the image is already generated by the image generating apparatus 104.

At STEP 311, it is judged whether or not the motion compensation process is to be performed. If there is no motion compensation process, the process proceeds to STEP 312. If not, the process proceeds to STEP 321.

In STEP 312, the image block setting process is instructed to the image generating apparatus 104. By this instruction, the position of the generated image block in the generated image and the position of the difference image block in the difference image are specified.

In STEP 313, the block motion compensation processing is controlled according to FIG.

At STEP 321, it is judged whether or not it is the forward motion compensation processing, and if it is the forward motion compensation processing, STEP 3
22. If not, proceed to STEP 331.

In STEP 322, the image generation device 104 is instructed to perform the image block setting process. By this instruction, the position of the past image block in the past image, the position of the generated image block in the generated image, and the position of the differential image block in the differential image are designated.

In STEP 323, block motion compensation processing is controlled according to FIG.

In STEP 331, it is judged whether or not the backward motion compensation processing is performed, and if it is the backward motion compensation processing, STEP 3
Proceed to 32, otherwise proceed to STEP 341.

In STEP 332, the image generation device 104 is instructed to perform the image block setting process. By this instruction, the position of the future image block in the future image, the position of the generated image block in the generated image, and the position of the difference image block in the difference image are specified.

In STEP 333, the motion compensation processing of the block is controlled according to FIG.

In STEP 341, the image block setting process is instructed to the image generating apparatus 104. By this instruction, the position of the past image block in the past image, the position of the generated image block in the generated image, and the position of the differential image block in the differential image are designated.

In STEP 342, the block motion compensation processing is controlled according to FIG.

In STEP 343, the image generation device 104 is instructed to perform the image block setting process. By this instruction, the position of the future image block in the future image, the position of the generated image block in the generated image, and the position of the difference image block in the difference image are specified.

In STEP 344, block motion compensation processing is controlled according to FIG.

In STEP 302, it is determined whether or not the motion compensation processing has been completed for all the image blocks in the generated image. If it is finished, proceed to STEP 303. If it is not finished,
The process proceeds to STEP 311 to process the next image block.

In STEP 306, it is judged whether or not the motion compensation processing for all the images is completed. If not completed, STEP 306 is executed.
The process proceeds to step 301 and the next image is processed.

Next, the control procedure of the image generation device 104 by the CPU 101 for performing the motion compensation processing of the image block will be described in detail with reference to FIG.

At STEP 351, the image generation apparatus 104 is instructed to perform image block setting processing. As a result, the difference image block to be processed next is moved from the difference image in the frame memory to the block image memory.

In STEP 361, it is determined whether or not motion compensation with half-pixel precision is to be performed. If motion compensation with half-pixel precision is not performed, the process proceeds to STEP 362, and if motion compensation with half-pixel precision is performed, the process proceeds to STEP 371.

At STEP 362, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
Is instructed.

In STEP 371, it is determined whether or not the half-pixel precision position designation of the reference image block is only in the horizontal direction. If only in the horizontal direction, the process proceeds to STEP 372. If not, the process proceeds to STEP 381.

In STEP 372, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
Is instructed. The position of this reference image block indicates a position which is shifted to the left by 0.5 pixel from the position of half pixel precision designated for motion compensation.

In STEP 373, the image generation apparatus 104 is instructed to perform image block setting processing. As a result, it is instructed to move the generated image block to the block image memory 205 in order to use it as a difference image block for the next motion compensation process.

In STEP 374, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
Is instructed. The position of this reference image block indicates a position that is shifted to the right by 0.5 pixel from the half-pixel precision position specified for motion compensation.

In STEP 381, it is determined whether the half-pixel precision position designation of the reference image block is in the vertical direction only. If only in the vertical direction, the process proceeds to STEP 382. If not, the process proceeds to STEP 391.

In STEP 382, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
Is instructed. The position of this reference image block indicates a position shifted by 0.5 pixel from the half-pixel precision position designated for motion compensation.

In STEP 383, the image generation apparatus 104 is instructed to perform image block setting processing. As a result, it is instructed to move the generated image block to the block image memory 205 in order to use it as a difference image block for the next motion compensation process.

In STEP 384, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
Is instructed. The position of this reference image block indicates a position shifted by 0.5 pixel from the half-pixel precision position designated for motion compensation.

In STEP 391, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
Is instructed. The position of this reference image block indicates a position shifted by 0.5 pixel on the left side by 0.5 pixel from the position of half pixel precision designated for motion compensation.

In STEP 392, the image generation apparatus 104 is instructed to perform image block setting processing. As a result, it is instructed to move the generated image block to the block image memory 205 in order to use it as a difference image block for the next motion compensation process.

In STEP 393, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
Is instructed. The position of this reference image block indicates a position shifted by 0.5 pixel on the right by 0.5 pixel from the position of half-pixel precision designated for motion compensation.

In STEP 394, the image generation apparatus 104 is instructed to perform image block setting processing. As a result, it is instructed to move the generated image block to the block image memory 205 in order to use it as a difference image block for the next motion compensation process.

In STEP 395, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
Is instructed. The position of this reference image block indicates a position shifted by 0.5 pixel to the left by 0.5 pixel from the half-pixel precision position designated for motion compensation.

In STEP 396, the image generation apparatus 104 is instructed to perform image block setting processing. As a result, it is instructed to move the generated image block to the block image memory 205 in order to use it as a difference image block for the next motion compensation process.

At STEP 397, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
Is instructed. The position of this reference image block indicates a position shifted by 0.5 pixel to the right by 0.5 pixel from the position of half pixel precision designated for motion compensation.

The operation procedure of the image generating apparatus 104 controlled by the CPU 101 as described above to perform the motion compensation process will be described with reference to FIGS. Here, the CPU 101
The operation procedure will be described for each type of operation instruction by.

FIG. 5 shows an operation procedure when the CPU 101 instructs the image generation apparatus 104 to perform block motion compensation processing. In this processing, the instruction from the CPU 101 is analyzed by the control circuit 201 of the image generating apparatus 104, and the pixel position specifying circuit
202, a command signal for controlling the operation of the multiplier generation unit 207 of the pixel calculation circuit 203 is output.

The pixel position specifying circuit 202 operates in the following procedure.

At STEP 401, the pixel position (first pixel position) in the reference image block is obtained, and this pixel position is output to the image memory access circuit 204 as the first pixel position signal 20b.

At the same time, in STEP 402, the pixel position (third pixel position) in the difference image block is obtained, and this pixel position is used as the third pixel position signal 20d.
Output to 04.

At the same time, in STEP 403, the pixel position (second pixel position) in the generated image block is obtained, and this pixel position is used as the second pixel position signal 20c, and the image memory access circuit 2
Output to 04.

In STEP 404, after the second image data is written in the second pixel position output in STEP 403, it is judged whether or not the processing of all the pixels in the generated image block is completed, and the processing is completed. Otherwise STEP 401, STEP 402, STEP 403
Processing proceeds to

The order in which the pixel position designating circuit 202 designates pixels is normally focused on one image block in the generated image block, the difference image block or the reference image block, and the horizontal line of the block, that is, the scan line. Pixels are specified in order. When the processing for one horizontal line is completed, the pixels on the next horizontal line are designated in order. When the processing of the pixels on all the horizontal lines is completed, the processing on all the pixels in the image block is completed.

The image memory access circuit 204 operates in the following procedure.

In STEP 411, the first pixel position signal 20b is input, the image data of the pixel position in the frame memory 109 is read, and this image data is used as the first image data signal.
Output as 20e.

In STEP 412, the second pixel position signal 20c and the second image data signal 20f are input and the frame memory 109
The image data is written in the pixel position in question.

In STEP 413, the third pixel position signal 20d is input, the image data of the pixel position of the block image memory 205 is read, and this image data is output as the third image data signal 20g.

The pixel calculation circuit 203 operates in the following procedure.

In STEP 421, the CPU 101 controls the control circuit 201.
According to the type of motion compensation processing instructed via, the multiplier generator 207 outputs the first multiplier and the second multiplier as shown in (Table 1).

[0219]

[Table 1]

In STEP 422, the first image data signal 20e
The first multiplying unit 209 multiplies the image data input as and the first multiplier.

In STEP 423, the third image data signal 20g
The second multiplying unit 208 multiplies the image data input as and the second multiplier.

In STEP 424, the addition unit 210 adds the results of the first multiplication unit 209 and the second multiplication unit 208, and outputs the addition result as the second image data signal 20f.

When the CPU 101 issues an image block setting processing instruction to the image generating apparatus 104, the instruction from the CPU 101 is analyzed by the control circuit 201 of the image generating apparatus 104, and the operations of the pixel position specifying circuit 202 and the pixel moving circuit 206 are controlled. Command signal is output.

The pixel position specifying circuit has a difference image block, a reference image block, a position of a generated image block, or a texture image for which a pixel position must be generated for the next motion compensation process and texture mapping process described later. , The position of the generated polygon is set. The pixel position designation circuit generates pixel positions according to the set image block positions during block motion compensation processing and texture mapping processing.

In the image moving circuit, the operation shown in FIG. 6 is performed to move the image block from the frame memory to the block image memory.

In STEP 501, a pixel position (fourth pixel position) in the generated image block is obtained, and this pixel position is output to the image memory access circuit 204 as a fourth pixel position signal 20h.

In STEP 502, the pixel position (fifth pixel position) in the block image memory is obtained, and this pixel position is set to the fifth pixel position.
It is output to the image memory access circuit 204 as the pixel position signal 20i.

In STEP 503, after the fifth image data is written in the fifth pixel position output in STEP 502, it is judged whether or not the processing of all the pixels in the generated image block is completed, and the processing is completed. If not, proceed to STEP 501 and STEP 502.

In STEP 511, the fourth pixel position signal 20h is input, the image data of the pixel position of the frame memory 109 is read, and this image data is used as the fourth image data signal.
Output as 20j. The fourth image data signal 20j is output as a fifth image data signal 20k via the image moving circuit.

In STEP 512, the fifth pixel position signal 20i and the fifth image data signal 20k are input and the image data is written in the pixel position in the block image memory 205.

By the above procedure, the image generating apparatus 104 of this embodiment realizes the motion compensation processing.

According to the operation procedure at the time of the motion compensation processing described above,
The operation of the multimedia device 100 during the forward motion compensation process will be specifically described with reference to FIG. Here, for simplification, the image size is 8 × 8 pixels, and the image block size for motion compensation is 4 × 4 pixels.

In FIG. 7, a past image 601 is a past image serving as a reference image for forward motion compensation. The past image block 602 is a block in which a reference of motion compensation in the past image is extracted. Difference image block 603
Is a difference image block for generating the 4 × 4 pixel generated image block on the upper left of the generated image by motion compensation. The generated image block 604 is a generated image block resulting from the forward motion compensation process. The generated image 605 shows an image to be generated by the expansion processing.

In the following description, the pixel position and the image block position are represented by the pixel position numbers given to the past image 601 and the generated image 605. For example, an image block of 4 × 4 pixels (block indicated by a thick line in the figure) located one pixel below the upper left corner of the past image is an image block (0,1)-
Notated as (3,4). Pixels in a block are represented by coordinates within the block. That is, the past image block (0,1)-
The pixel in the upper left corner of (3,4) is expressed as pixel (0,0).

The operations of the multimedia device 100 and the image generation device 104 will be described in detail below.

The past image 601 is stored in the frame memory 109. A storage area in which the generated image 605 should be stored is also secured in the frame memory 109. CPU1
01 writes the difference image to the frame memory 109 (STEP 30
1).

[0237] Since the forward motion compensation processing is performed, the processing is ST
Proceed to EP 322 (STEP 311 → STEP 321).

The CPU 101 instructs the image generation device 104 to perform the image block setting process, and the image block in the past image 601 is
(0,1)-(3,4) is designated as the past image block 602, image block (0,0)-(3,3) in the generated image 605 is designated as the generated image block 604, and the difference image block A position within the frame memory 109 of the 603 is designated (STEP 322).

The processing shown in FIG. 4 is performed on the designated image block.

In STEP 351, the CPU 101 gives an instruction to move the difference image block 603 to the block image memory as an image block setting process. Upon receiving this instruction, the control circuit 201 controls the image moving circuit 206 to move the difference image block 603 in the frame memory 109 to the block image memory 205 (FIG. 6).

Since the position designation of the past image block 602 (0,1)-(3,4) in the past image 601 does not cause a half pixel shift from the pixel existing position, half-pixel precision motion compensation is not performed (S
TEP361).

The CPU 101 instructs the image generation device 104 to perform block motion compensation processing. The image generation device 104 receives this instruction and performs processing according to FIG.

The pixel position specifying circuit 202 calculates the pixel position (first pixel position) (0,0) in the past image block 602 and outputs this pixel position as the first pixel position signal 20b. On the other hand, the pixel position in the difference image block 603 (the third position
Pixel position) (0,0) is calculated, this pixel position is output as the third pixel position signal 20d, and the pixel position (second pixel position) (0,0) in the generated image block 604 is calculated. Then, this pixel position is output as the second pixel position signal 20c (ST
EP 401, STEP 402, STEP 40
3).

The first image memory access section 212 in the image memory access circuit 204 receives the first pixel position signal 20b and inputs the pixel at the pixel position (0,0) from the past image block 602 in the frame memory 109. The data is read and this pixel data is output as the first pixel data signal 20e (STEP 4
11). The second image memory access unit 211 inputs the third pixel position signal 20d, reads the pixel data at the pixel position (0,0) from the difference image block 603 in the block image memory 205, and uses this pixel data as the first pixel data. It is output as the pixel data signal 20g of 3 (STEP 413).

In the multiplier generation unit 207 in the pixel calculation circuit 203,
According to (Table 1), 1.0 is generated as the first multiplier and the second multiplier is generated.
Generates 1.0 as a multiplier of (STEP 421). First multiplier
In 209, the pixel data input by the first pixel data signal 20e is multiplied by the first multiplier, and the result is output to the adder 210 (STEP 422). The second multiplication unit 208 multiplies the pixel data input by the third pixel data signal 20g by the second multiplier and outputs the result to the addition unit 210 (STEP 423). The addition unit 210 adds the multiplication results of the first multiplication unit 209 and the second multiplication unit 208 and outputs the result as the second pixel data signal 20f.
(STEP 424).

Next, in the first image memory access section 212, the second pixel position signal 20c and the second pixel data signal 20f
Is input, and this pixel data is written at the pixel position (0,0) in the generated image block 604 (STEP 412).

The processing of FIG. 5 is repeated for each pixel in the generated image block 604 until all the pixels in the generated image block 604 have been generated (STEP 404).

By performing the processing in this way, a generated image block 604 is generated from the past image block 602 and the motion compensation difference image 603.

The operation of the backward motion compensation process is different from the above-described forward motion compensation in that the future image is used as the reference image instead of the past image.

Next, the operations of the multimedia device 100 and the image generation device 104 during the bidirectional motion compensation process will be specifically described with reference to FIG.

In FIG. 8, the past image 701 is a reference image for forward motion compensation. The future image 702 becomes a reference image for backward motion compensation. The past image block 703 is obtained by extracting a block serving as a reference for motion compensation in the past image. The future image block 704 is a block that is a reference for motion compensation in the future image. The difference image block 705 is the upper left 4 × 4 of the generated image.
This is a block for generating a generated image block of pixels by motion compensation processing. The generated image block 706 is a block resulting from the bidirectional motion compensation process. Generated image
707 indicates the image to be generated.

In the case of bidirectional motion compensation processing, first, the same operation as in STEP 322 and STEP 323 of the forward motion compensation processing described above is performed by the CPU 101, and an operation instruction is given to the image generation device 104 (STEP 341). → STEP 342). Upon receiving this instruction, the image generation apparatus 104 operates in the same processing procedure as the above-described forward motion compensation processing. However, as shown in (Table 1), the multiplier generator
The first multiplier generated in 207 is "0.5", and the second multiplier is "1.0". As a result of the forward motion compensation processing, the result of adding 0.5 times the pixel data of the past image block 703 and the pixel data of the difference image block 705 to the generated image block is obtained.

After this, STEP 332 of the backward motion compensation processing,
An operation similar to STEP 333 is controlled by the CPU 101, and an operation instruction is given to the image generation device 104 (STEP 343 → STEP 3
44).

Upon receipt of this instruction, the image generating apparatus 104 operates in the same processing procedure as the backward motion compensation processing. First, the generated image block generated by the forward motion compensation processing is moved to the block image memory as a difference image block (FIG. 6). Next, motion compensation processing is performed according to the procedure of FIG. However, as shown in (Table 1), the first value generated by the multiplier generation unit 207
The multiplier is "0.5", and the second multiplier is "1.0". As a result of this backward motion compensation processing, a generated image block 706 is obtained.

Next, the half-pixel precision motion compensation processing will be described with reference to FIG.

The overall operation procedure of the forward motion compensation and the bidirectional motion compensation is as described above. However, when the designation of the position of the reference image block in the reference image is deviated from the existing position of the pixel by half a pixel, Interpolation processing must be performed between a plurality of reference image blocks to obtain an image block. This processing procedure is shown in FIG. Here, it is assumed that the position is specified with half-pixel accuracy in both the vertical and horizontal directions.

[0257] The image that is the reference of the motion compensation process is the reference image.
801 and the block to be generated at this time is the generated image block 802, the position designation of the reference image block is indicated by a thick line in the reference image 801 in order to make the data of the motion compensation difference image block as close to 0 as possible. The enclosed rectangular area is (0.5,0.5)-(3.5,3.5). However, since the position designation of the reference image block is deviated from the existing position of the pixel by a half pixel, such a reference image block is set to the reference image 801.
Cannot be read from. Therefore, an image block shifted by half a pixel from the designated position of the reference image block, that is, image block 803 (0,0)-(3,3), image block 804 (1,
0)-(4,3), image block 805 (0,1)-(3,4), image block
806 (1,1)-(4,4) are taken out and the four image blocks are interpolated to obtain the reference image block. Then, if the forward or bidirectional motion compensation process is performed on the reference image block and the motion compensation difference image block obtained in this way as described above, the half-pixel precision motion compensation process can be realized.

In this embodiment, the interpolation and motion compensation processing of the four image blocks 803, 804, 805 and 806 are simultaneously performed by the following procedure.

According to the procedure already described (FIG. 4), the process proceeds to STEP 391 in the motion compensation process with half-pixel precision both vertically and horizontally. Here, STEP 391 and subsequent steps will be described in detail.

In STEP 391, the image generation device 104 is instructed to perform the motion compensation process for the reference image block 803. In the pixel calculation circuit 203 of the image generation device 104, “0.25” is set as the first multiplier (“0.2” in the case of bidirectional motion compensation).
125 ”) is generated, the pixel data in the reference image 803 is multiplied by the first multiplier, and the pixel data of the difference image block is added to this, and written in the generated image block.

In STEP 392, the generated image block is moved to the block image memory 205 as a difference image block by the procedure shown in FIG.

In STEP 393, the image generation device 104 is instructed to perform the motion compensation process for the reference image block 804. In the pixel calculation circuit 203 of the image generation device 104, “0.25” is set as the first multiplier (“0.2” in the case of bidirectional motion compensation).
125 ”) is generated, the pixel data in the reference image 804 is multiplied by a first multiplier, and the pixel data of the difference image block is added to this, and written in the generated image block.

In STEP 394, the generated image block is moved to the block image memory 205 as a difference image block by the procedure shown in FIG.

In STEP 395, the image generation device 104 is instructed to perform the motion compensation process for the reference image block 805. In the pixel calculation circuit 203 of the image generation device 104, “0.25” is set as the first multiplier (“0.2” in the case of bidirectional motion compensation).
125 ”) is generated, the pixel data in the reference image 805 is multiplied by the first multiplier, the pixel data of the difference image block is added to this, and the result is written in the generated image block.

In STEP 396, the generated image block is moved to the block image memory 205 as a difference image block by the procedure shown in FIG.

In STEP 397, the image generation device 104 is instructed to perform the motion compensation process for the reference image block 806. In the pixel calculation circuit 203 of the image generation device 104, “0.25” is set as the first multiplier (“0.2” in the case of bidirectional motion compensation).
125 ”) is generated, the pixel data in the reference image 806 is multiplied by a first multiplier, the pixel data of the difference image block is added to this, and the result is written in the generated image block 802.

Motion compensation processing with half-pixel precision only in the vertical or horizontal direction can also be realized by performing the same processing as the above processing as shown in FIG.

As described above, in the present embodiment, the interpolation processing for obtaining the reference image block and the motion compensation processing can be performed at the same time, and the motion compensation processing with half pixel accuracy can be realized.

The operation at the time of texture mapping processing in the image generating apparatus 104 will be described below. CPU101
FIG. 10 shows a control procedure of the image generating apparatus 104 according to the above, and FIG. 11 shows an operation procedure of each circuit of the image generating apparatus 104. FIG.
Then, the dashed arrows represent the flow of data / control signals, and the solid arrows represent the flow of processing.

First, FIG. 1 shows a control procedure of the image generating apparatus 104 by the CPU 101 when performing the texture mapping process.
Explanation will be made using 0.

At STEP 901, the previously generated image in the frame memory 109 is erased.

At STEP 902, the texture image is written in the frame memory 109.

At STEP 903, the image generation device 104 is instructed to perform the image block setting process. By this instruction, the position of the generated polygon (vertex position) and the position of the texture image mapped to this generated polygon are specified. Further, the specified texture image is stored in the block image memory 205.
Instruct to move to.

In STEP 904, the texture mapping processing is instructed to the image generating apparatus 104.

At STEP 905, it is determined whether the texture mapping processing of all polygons in the generated image is completed. If it is completed, the process proceeds to STEP 906, and if it is not completed, the process proceeds to STEP 903 to process the next image block.

At STEP 906, it is judged whether or not the texture mapping processing for all the images is completed. If not completed, the processing proceeds to STEP 901 and the next image is processed.

The operation procedure of the image generating apparatus 104, which is controlled by the CPU 101 as described above to perform the texture mapping process, will be described with reference to FIGS. 11 and 6. Here, the operation procedure will be described for each type of operation instruction from the CPU 101.

FIG. 11 shows an operation procedure when the CPU 101 instructs the image generation apparatus 104 to perform texture mapping processing. In this processing, the instruction from the CPU 101 is analyzed by the control circuit 201 of the image generation device 104, and a command signal for controlling the operation of the pixel position designation circuit 202 and the multiplier generation unit 207 of the pixel calculation circuit 203 is output.

The pixel position specifying circuit 202 operates in the following procedure.

At STEP 1001, the pixel position (first pixel position) in the generated polygon is obtained, and this pixel position is output to the image memory access circuit 204 as the first pixel position signal 20b.

At the same time, in STEP 1002, the pixel position (third pixel position) of the texture to be mapped to the first pixel position is obtained, and this pixel position is output to the image memory access circuit 204 as the third pixel position signal 20d. To do.

At the same time, STEP 1003 sets the second pixel position signal 20 to the same pixel position (second pixel position) as the first pixel position.
It is output to the image memory access circuit 204 as c.

In STEP 1004, the second output in STEP 1003
After the second image data is written in the pixel position of, it is judged whether the processing of all the pixels in the generated image block is completed, and if not completed, STEP 1001, STEP 1002, STEP
Processing proceeds to 1003.

Regarding the order in which the pixel position specifying circuit 202 specifies pixels, usually, paying attention to either the generated polygon or the texture image block, the pixels on the horizontal line of the block or polygon, that is, the scan line are specified in order. When the processing for one horizontal line is completed,
The pixels on the next horizontal line are designated in order. When the processing of the pixels on all the horizontal lines is completed, the processing on all the pixels in the image block is completed.

The image memory access circuit 204 operates in the following procedure.

At STEP 1011, it is judged whether the texture image of the semi-transparent texture image is subjected to texture mapping processing or the pixel to which anti-aliasing is to be performed. , When not using anti-aliasing, proceed to STEP 1013.

In STEP 1012, the first pixel position signal 20b is input, the image data of the pixel position of the frame memory 109 is read, and this image data is used as the first image data signal.
Output as 20e.

In STEP 1013, the second pixel position signal 20c and the second image data signal 20f are input, and the frame memory 109
The image data is written in the pixel position in question.

In STEP 1014, the third pixel position signal 20d is input, the image data of the pixel position of the block image memory 205 is read, and this image data is output as the third image data signal 20g.

The pixel calculation circuit 203 operates in the following procedure.

In STEP 1021, the CPU 101 controls the control circuit 20.
Transparency α (1 for opaque, 0 for transparent) in the texture mapping process specified via 1 and a translucency coefficient β for antialiasing of the edge (the pixel being processed constitutes the edge of the polygon. If not, β = 1) is calculated, αβ is output as a second multiplier to the second multiplier, and (1-αβ) is output as a first multiplier to the first multiplier.

In STEP 1022, the first image data signal 20e
The first multiplying unit 209 multiplies the image data input as and the first multiplier.

In STEP 1023, the third image data signal 20g
The second multiplying unit 208 multiplies the image data input as and the second multiplier.

In STEP 1024, the addition unit 210 adds the results of the first multiplication unit 209 and the second multiplication unit 208, and outputs the addition result as the second image data signal 20f.

When the CPU 101 instructs the image generation apparatus 104 to perform the image block setting process, it is as described in the motion compensation process. When the movement of the texture image is instructed in the image block setting processing, the image movement circuit performs the operation shown in FIG. 6 to move the texture image from the frame memory to the block image memory.

At STEP 501, a pixel position (fourth pixel position) in the texture image is obtained, and this pixel position is output to the image memory access circuit 204 as a fourth pixel position signal 20h.

In STEP 502, the pixel position (fifth pixel position) in the block image memory is obtained, and this pixel position is set to the fifth pixel position.
It is output to the image memory access circuit 204 as the pixel position signal 20i.

At STEP 503, after the fifth image data is written at the fifth pixel position outputted at STEP 502, it is judged whether or not the processing of all the pixels in the generated image block is completed, and the processing is completed. If not, proceed to STEP 501 and STEP 502.

In STEP 511, the fourth pixel position signal 20h is input, the image data of the pixel position in the frame memory 109 is read, and this image data is used as the fourth image data signal.
Output as 20j. The fourth image data signal 20j is output as a fifth image data signal 20k via the image moving circuit.

In STEP 512, the fifth pixel position signal 20i and the fifth image data signal 20k are input and the image data is written in the pixel position in the block image memory 205.

By the above procedure, the image generating apparatus of this embodiment realizes texture mapping processing.

In accordance with the above-mentioned texture mapping operation procedure, the image generation device 104 at the time of texture mapping processing
The operation will be specifically described with reference to FIG. 12 in association with FIGS. 10 and 11. In FIG. 12, an image 1101 represents a texture image. Polygon 1102 is a texture image
1101 represents a polygon to be pasted. The pixel 1103 is a pixel forming the polygon 1102. Pixel 1104 is pixel 11
These are the pixels that make up the texture image that is pasted on 03. A pixel 1105 represents a pixel forming an edge of the polygon 1102. Pixel 1106 represents a pixel of the texture image corresponding to pixel 1105.

The operation is as follows.

The texture image 1101 is stored in the frame memory 109.
Stored in (STEP 902).

The CPU 101 instructs the image generation device 104 to perform image block setting processing, and the polygon vertex coordinates and the semitransparency of the texture (here, the translucency of the entire texture is 0.6).
Are set in the pixel position designation circuit 202 and the multiplier generation unit 207 of the pixel calculation circuit 203 via the control circuit 201. Also,
Instruct the image moving circuit 206 to move the texture image (S
TEP 903).

The image moving circuit 206 sequentially outputs the pixel position of the texture image in the frame memory 109 as the fourth pixel position to the first image memory access section 212 of the image memory access circuit 204 (STEP 501). The first image memory access unit 212 reads the texture image data from the frame memory 109 and outputs it as the fourth pixel data (STEP 511). The fourth pixel data is input to the image moving circuit, and this data is output as the fifth pixel data. At the same time, the storage pixel position of the texture image in the block image memory is sequentially output from the image moving circuit as the fifth pixel position (STEP 502). The fifth pixel position and the fifth pixel data are input to the second image memory access section of the image memory access circuit, and the fifth image data is written in the fifth pixel position of the block image memory (STEP 512). .
By repeating this operation for all pixels in the texture image (STEP 503), the texture image is moved to the block image memory.

Next, the CPU 101 instructs the image generation device 104 to perform texture mapping processing. The image generation device receives this instruction and performs the texture mapping process according to the procedure of FIG.

In the pixel position specifying circuit 202, the generated polygon 1
Pixel positions (first pixel positions) in 102 are sequentially calculated, and this pixel position is output as a first pixel position signal 20b.
Now, assume that the pixel 1103 is generated. The pixel position of the pixel 1103 is output as the first pixel position. On the other hand, the pixel position (third pixel position) of the pixel 1104 in the texture image 1101 mapped to the pixel 1103 is calculated, and this pixel position is set to the third pixel position.
The pixel position signal 20d is output. Also, the same pixel position as the first pixel position is output as the second pixel position signal 20c. (STEP 1001, STEP 1002 and STEP 1003) In order to perform semi-transparent texture mapping, the first image memory access unit 212 in the image memory access circuit 204
The first pixel position signal 20b is input, pixel data is read from the generation polygon 1102 in the frame memory 109, and this pixel data is output as the first pixel data signal 20e.
This pixel data becomes the background pixel data (STEP 1012).
The second image memory access unit 211 inputs the third pixel position signal 20d, reads pixel data from the texture image 1101 in the block image memory 205, and outputs this pixel data as a third pixel data signal 20g. (STEP101
Four).

The multiplier generation unit 207 in the pixel calculation circuit 203
According to the translucency of 0.6, 0.4 is generated as the first multiplier and 0.6 is generated as the second multiplier (STEP 1021). The first multiplication unit 209 multiplies the pixel data input by the first pixel data signal 20e by the first multiplier and adds the result to the addition unit.
Output to 210 (STEP 1022). The second multiplication unit 208 multiplies the pixel data input by the third pixel data signal 20g by the second multiplier and outputs the result to the addition unit 210 (STEP 1
023). In the addition unit 210, the first multiplication unit 209 and the second multiplication unit 209
The multiplication result of 208 is added and output as the second pixel data signal 20f (STEP 1024). As a result, 0.6 × (texture pixel data) + 0.4 × (background pixel data) is output as the second pixel data.

Next, the first image memory access section 212 receives the second pixel position signal 20c and the second pixel data signal 20f.
Is input, and this pixel data is written in the pixel position of the pixel 1103 in the generated polygon 1102 (STEP 1013). In this way, the pixel data 1104 of the texture image is texture-mapped.

Next, consider the case where the pixel 1105 forming the edge of the generation polygon 1102 is generated. Here, 70% of the pixel area of the pixel 1105 is inside the polygon 1102, and
It is assumed that 0% is outside the polygon. In this case the multiplier generator
In 207, the first multiplier is (1-0.6 × 0.7) = 0.
58 is output as a second multiplier of 0.6 × 0.7 = 0.42, and 0.42 × (texture pixel data) + as a second pixel data signal as the calculation result in the pixel calculation circuit.
0.58 × (background pixel data) is output, and this pixel data is written in the pixel 1105 in the generation polygon 1102.

The processing of FIG. 11 is repeated for each pixel in the generation polygon 1102 until all the pixels in the generation polygon 1102 are generated (STEP 1004).

By performing the processing as described above, it is possible to perform the texture mapping processing on the generated polygon 1102 from the texture image 1101.

As described above, according to this embodiment, the texture mapping of the semi-transparent texture image can be processed including the anti-aliasing processing of the polygon edge portion.

Next, the operation of the image generating apparatus 104 when the motion compensation processing and the texture mapping processing are executed simultaneously will be described. 13 and 14 show the control procedure of the image generation apparatus 104 by the CPU 101, and FIG. 15 shows the operation procedure of each circuit of the image generation apparatus 104. In FIG. 15, broken line arrows represent data / control signal flows, and solid line arrows represent processing flows. In FIGS. 13, 14 and 15, when the same processing as the above-described motion compensation processing (described with reference to FIGS. 3, 4 and 5) is performed, the same processing procedure number is assigned.

The control procedure of the image generation device 104 by the CPU 101 when the motion compensation process and the texture mapping process are simultaneously executed will be described with reference to FIGS. 13 and 14. The description of the steps that take the same procedure as in the case of independently executing the motion compensation process will be omitted.

When the processing advances to STEP 1201, it is determined whether or not the image to be motion-compensated, that is, the generated image, is used as a reference image in the subsequent motion-compensation processing of the image.

When used as a reference image, STEP 1
Follow the steps from 211 to STEP 1213. After the motion compensation processing (FIG. 4) of the image block currently being processed is completed (STEP 1211) and the generated image block is generated, the generated image block is moved to the block image memory 205 for use as a texture image. (STEP 1212), instructs the image generation device 104 to execute texture mapping processing (STEP 12112).
3). When it is necessary to generate the generated image for the subsequent motion compensation processing, the generated image block is generated as described above, and then the texture mapping processing is performed. When compressed and encoded by MPEG, the picture resulting from the expansion of an I picture is used for motion compensation of a P picture or B picture, and the expanded picture of a P picture is used for motion compensation of another P picture or B picture. In some cases, the processing procedure described above is applied to such an image. The operation procedure of the image generation device 104 corresponding to this processing procedure is the same as that of FIG. 5 (motion compensation processing) and FIG. 11 (texture mapping processing).

On the other hand, when the image of the motion compensation processing result is not used for the subsequent motion compensation processing like the B picture of MPEG, the motion compensation processing and the texture mapping processing are executed by the procedure of STEP 1202 (FIG. 14). To do. Also in FIG. 14, when the same procedure as in FIG. In the processing procedure of FIG. 14, when the block motion compensation processing is required a plurality of times, it is the same as that of FIG. 4 except the last one motion compensation processing. In FIG. 14, the texture mapping process is performed at the same time as the final block motion compensation process. STEP 1251, STEP 1261, STEP 1
In 271 and STEP1281, the CPU 101 instructs the image block setting process. In this step, the position of the generated polygon is specified, and also STEP 1261, STEP 1271 and STEP
In 1281, the generated image block is stored in the block image memory 205.
Move to After that, the block motion compensation process and the texture mapping are simultaneously instructed. FIG. 6 shows the operation procedure of the image generation device 104 when the CPU 101 issues an image block setting processing instruction to move the generated image block to the block image memory 205 to the image generation device 104.
As explained in.

The operation procedure of the image generating apparatus 104 when the CPU 101 instructs the image generating apparatus 104 to simultaneously execute the block motion compensation process and the texture mapping will be described with reference to FIG. In the operation procedure of FIG. 15, the same numbers are given to the same operation steps as those in the description of FIG.

In this processing, the instruction from the CPU 101 is analyzed by the control circuit 201 of the image generating apparatus 104, and a command signal for controlling the operation of the pixel position specifying circuit 202 and the multiplier generating unit 207 of the pixel arithmetic circuit 203 is output.

The pixel position specifying circuit 202 operates in the following procedure.

In STEP 401, the pixel position (first pixel position) in the reference image block is obtained, and this pixel position is output to the image memory access circuit 204 as the first pixel position signal 20b.

At the same time, in STEP 402, the pixel position (third pixel position) in the difference image block is obtained, and this pixel position is set as the third pixel position signal 20d to the image memory access circuit 2
Output to 04.

At the same time, in STEP 1301, the pixel position (second pixel position) in the generated polygon is obtained, and this pixel position is used as the second pixel position signal 20c.
Output to

In STEP 404, the second output in STEP 1301
After the second image data is written in the pixel position of, the process determines whether or not the processing of all the pixels in the generated polygon is completed, and if not completed, STEP 401, STEP 402, STEP 130
Proceed to 1.

The image memory access circuit 204 operates according to the following procedure.

In STEP 411, the first pixel position signal 20b is input, the image data of the pixel position of the frame memory 109 is read, and this image data is used as the first image data signal.
Output as 20e.

In STEP 412, the second pixel position signal 20c and the second image data signal 20f are input and the frame memory 109
The image data is written in the pixel position in question.

In STEP 413, the third pixel position signal 20d is input, the image data of the pixel position of the block image memory 205 is read, and this image data is output as the third image data signal 20g.

The pixel calculation circuit 203 operates in the following procedure.

In STEP 421, the CPU 101 controls the control circuit 201.
According to the type of motion compensation processing instructed via, the multiplier generator 207 outputs the first multiplier and the second multiplier as shown in (Table 1).

In STEP 422, the first image data signal 20e
The first multiplication unit 209 performs multiplication using the image data input as and the first multiplier.

In STEP 423, the third image data signal 20g
The second multiplication unit 208 performs multiplication using the image data input as and the second multiplier.

At STEP 424, the addition unit 210 adds the result of the first multiplication unit 209 and the result of the second multiplication unit 208,
The addition result is output as the second image data signal 20f.

With the above procedure, the image generating apparatus of this embodiment realizes the simultaneous execution of motion compensation processing and texture mapping.

In the following, the motion compensation processing and the texture mapping processing are executed simultaneously using FIG. 16 (STEP 125 in FIG. 14).
2, STEP 1262, STEP 1272, STEP 1282, FIG. 15) will be described.

Reference image 1401 is stored in frame memory 109.
Is stored in the pixel position specifying circuit 202 as a reference image block 1402. On the other hand, the difference image 1403 in the frame memory 109
The differential image block 1404 is set, and the image data of this image block is moved to the block image memory 205. (STEP 1251, STEP 1261, STEP 1271, STEP 1281, FIG. 6) The pixel position designation circuit 202 maps the second pixel position signal 20c indicating the pixel position in the generated polygon 1405 and the second pixel position. The pixel position in the reference image block 1402 and the pixel position in the difference image block 1404 are output as the first pixel position signal 20b and the third pixel position signal 20d. The pixel data at the first pixel position is read from the frame memory 109 via the first image memory access section 212 (first pixel data), while the pixel data at the third pixel position is the second image memory. It is read from the block image memory 205 via the access unit 211 (third pixel data). The first pixel data and the third pixel data are multiplied by the multipliers shown in (Table 1) in the pixel calculation circuit 203, the respective multiplication results are added, and the addition result is output as the second pixel data. By this calculation, the pixel data resulting from the motion compensation processing is obtained as the second pixel data. The second pixel data is the generated polygon 1405.
It is written in the pixel position in the frame memory 109 indicated by the second pixel position signal indicating the pixel position inside. The second pixel position indicates a pixel position in the generated polygon 1405 to be texture-mapped, and the texture mapping process is achieved by writing the second pixel data. As described above, the motion compensation process and the texture mapping process can be simultaneously executed.

As described above, according to this embodiment,
A block image memory 205 for storing a texture image, a motion compensation difference image, etc., and this block image memory 205,
An image memory access circuit 204 for accessing the pixel data of the frame memory 109 in which the reference image, the generated polygon and the generated image are stored, the third pixel data read from the block image memory 205 and the frame memory 109. The pixel calculation circuit 203 that calculates the calculated first pixel data between pixels, and the pixel data to be calculated according to the processing, such as motion compensation processing, texture mapping processing, and simultaneous execution of motion compensation processing and texture mapping processing. By providing a pixel position specifying circuit 202 for generating and specifying a pixel position and a pixel position for writing pixel data of a calculation result, and an image moving circuit 206 for moving an image block in the frame memory 109 to the block image memory 205, Compressed using motion estimation, such as directional motion compensation, bidirectional motion compensation, and half-pixel precision motion compensation. And expansion processing of the video,
It is possible to realize texture mapping processing including texture mapping of a semi-transparent texture image and anti-aliasing processing of polygon edges, and simultaneous execution of motion compensation processing and texture mapping processing.

Note that the size of the image block for motion compensation is set to 4 × 4 pixels for simplification of description, but the size is not limited to this, and 16 × 16 pixels or 8
It does not exclude the use of other sizes such as × 8 pixels. Although the size of the image is set to 8 × 8 pixels, it is possible to select any size. Furthermore, the processing order of the image blocks described in the procedure of the motion compensation processing of the present embodiment is not limited to this, and the processing orders can be interchanged. Furthermore, in the description of the texture mapping process of the present embodiment, the translucency of the translucent texture is described as one type, but it is also possible to give different translucency to each pixel forming the texture.

(Second Embodiment) With respect to the configuration and operation of the second embodiment of the multimedia device according to the present invention, the description of the same parts as those of the first embodiment will be omitted.

The configuration of the image generating apparatus 104 in this embodiment will be described with reference to FIG. The same components as those of the image generating apparatus according to the first embodiment are designated by the same numbers.

The control circuit 201 interprets the command designated by the command input signal 20a, and the pixel position designation circuit 1501,
The control circuit outputs a command signal for controlling the operations of the image moving circuit 206 and the pixel arithmetic circuit 203. The pixel position designation circuit 1501 follows the instruction of the control circuit 201, and the pixel position in the reference image block or the generated polygon (first pixel position).
And outputs this as the first pixel position signal 20b, and calculates the pixel position (second pixel position) in the generated image block or generated polygon and outputs it as the second pixel position signal 150a,
Pixel position in the difference image block or texture image (3rd
Pixel position) is obtained and output as a third pixel position signal 20d. The image memory access circuit 1502 receives the first pixel position signal 20b, reads the pixel data at the specified pixel position in the frame memory 109, and outputs it as the first pixel data signal 20e, and receives the second pixel position signal 150a. When the second pixel position signal 150a indicates a pixel position in the frame memory 109, the pixel data input as the second pixel data signal 150b is written to the designated pixel position in the frame memory 109, and the fourth pixel position The first image memory access unit 1504 for receiving the signal 20h and reading the pixel data at the specified pixel position in the frame memory 109 and outputting it as the fourth pixel data signal 20j, and the third pixel position signal 20d for receiving the block image memory The pixel data at the specified pixel position in 205 is read out and output as the third pixel data signal 20g, and the second pixel position signal 150a is received and the second pixel position signal 150a When pointing to a pixel position in the image memory 205, the pixel data input as the second pixel data signal 150b is written to a designated pixel position in the block image memory 205, and the fifth pixel position signal 20i is received to receive the block image memory. A second image memory access unit 1503 for writing the pixel data input as the fifth pixel data signal 20k at a designated pixel position therein.

The pixel calculation circuit 203 inputs the first pixel data signal 20e and the third pixel data signal 20g and outputs the result of calculating these data as the second pixel data signal 150b. In the present embodiment, the pixel arithmetic circuit 203 generates a first multiplier and a second multiplier according to an instruction (command signal) from the control circuit 201, and outputs a multiplier to a multiplier, and a multiplier generator 207,
A first multiplication unit 209 that multiplies the first multiplier specified by the multiplier generation unit 207 and the first pixel data, and a second multiplier specified by the multiplier generation unit 207 and the third pixel data. The second multiplication unit 208, the calculation result of the first multiplication unit 209, and the second
And an adder 210 that adds the calculation result of the multiplier 208.

If the instruction (command signal) from the control circuit 201 indicates motion compensation processing, the pixel calculation circuit 203
Alternatively, when the motion compensation process and the texture mapping process are instructed simultaneously, the operation is performed with signed data precision, and when the texture mapping process is instructed, the operation is performed with unsigned data precision. To do.

The image moving circuit 206 generates the fourth pixel position and the fifth pixel position for moving the image block in the frame memory 109 to the block image memory 205 in accordance with the instruction (command signal) from the control circuit 201. Let Pixel data designated by the fourth pixel position signal 20h is read from the frame memory 109 via the image memory access circuit 1502, and this image data is designated by the fifth pixel position signal 20i at the pixel position in the block image memory 205. To the image memory access circuit 1502.

The image generation device 104 configured as described above.
The operation of will be described below. Image generation device 104 is CP
Operation is instructed by command by U101, control circuit 2
It operates by interpreting the command at 01 and outputting a command for controlling each circuit of the image generating apparatus 104 accordingly. In the following description, a control procedure by issuing a command to the control circuit 201 by the CPU 101 and an operation procedure of each circuit in the image generating apparatus 104 which receives this control will be described.

First, the image generating apparatus 104 according to the present embodiment.
The operation at the time of the motion compensation processing will be described. 18 and 19 show the control procedure of the image generation device 104 by the CPU 101.
FIG. 20 shows the operation procedure of each circuit of the image generation device 104.
18, 19, and 20, respectively, in FIG.
The same processing steps as those in FIGS. 4 and 5 are given the same numbers.

First, the CPU 10 for performing motion compensation processing.
18 and 1 show the control procedure of the image generation device 104 according to the first embodiment.
9 will be described.

In STEP 301, a reference image (past image or future image) serving as a reference for motion compensation is set in the frame memory 109 and a difference image is written. The reference image is the disk device 10
There are a case where the image is read out from the memory 3 and written in the frame memory 109 by the CPU 101, and a case where the image is already generated by the image generating apparatus 104.

At STEP 311, it is determined whether or not the motion compensation process is performed. If no motion compensation process is performed, the process proceeds to STEP 1601. If not, the process proceeds to STEP 321.

In STEP 1601, the image generation device 104 is instructed to perform the image block setting process. Thereby, the position of the generated image block in the block image memory and the position of the difference image block in the difference image are designated.

In STEP 1602, block motion compensation processing is controlled according to FIG.

In STEP 321, it is judged whether or not the forward motion compensation processing is performed, and if it is the forward motion compensation processing, STEP 1
Proceed to 611, otherwise proceed to STEP 331.

In STEP 1611, the image generation device 104 is instructed to perform the image block setting process. By this instruction, the position of the past image block in the past image, the position of the generated image block in the block image memory, and the position of the differential image block in the differential image are designated.

In STEP 1612, the block motion compensation processing is controlled according to FIG.

In STEP 331, it is judged whether or not the backward motion compensation processing is performed, and if it is the backward motion compensation processing, STEP 1
The process proceeds to 621, and if not, the process proceeds to STEP 341.

In STEP 1621, the image generation device 104 is instructed to perform the image block setting process. By this instruction, the position of the future image block in the future image, the position of the generated image block in the block image memory, and the position of the difference image block in the difference image are specified.

In STEP 1622, block motion compensation processing is controlled according to FIG.

In STEP 1631, the image generation device 104 is instructed to perform the image block setting process. By this instruction, the position of the past image block in the past image, the position of the generated image block in the block image memory, and the position of the differential image block in the differential image are designated.

In STEP 1632, the block motion compensation processing is controlled according to FIG.

In STEP 1633, the image generation device 104 is instructed to perform the image block setting process. By this instruction, the position of the future image block in the future image, the position of the generated image block in the block image memory, and the position of the difference image block in the difference image are specified.

In STEP 1634, block motion compensation processing is controlled according to FIG.

In STEP 302, it is determined whether or not the motion compensation processing has been completed for all the image blocks in the generated image. If it is finished, proceed to STEP 303. If it is not finished,
The process proceeds to STEP 311 to process the next image block.

At STEP 303, it is judged whether or not the motion compensation processing for all the images is completed.
The process proceeds to step 301 and the next image is processed.

Next, referring to FIG. 19, the image generation device 104 by the CPU 101 for performing the motion compensation processing of the image block.
Details of the control procedure will be described.

At STEP 351 the image generation apparatus 104 is instructed to perform image block setting processing. As a result, the difference image block to be processed next is moved from the difference image in the frame memory 109 to the block image memory 205.

In STEP 361, it is determined whether or not motion compensation with half-pixel precision is to be performed. If motion compensation with half-pixel precision is not performed, the process proceeds to STEP 1651, and if motion compensation with half-pixel precision is performed, the process proceeds to STEP 371.

In STEP 1651, the image block setting process is instructed, and the inside of the generated image in the frame memory 109 is designated as the generation position of the generated image block.

In STEP 362, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
Is instructed.

In STEP 371, it is determined whether the half-pixel precision position designation of the reference image block is in the horizontal direction only. If only in the horizontal direction, the process proceeds to STEP 372. If not, the process proceeds to STEP 381.

In STEP 372, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
Is instructed. The position of this reference image block indicates a position which is shifted to the left by 0.5 pixel from the position of half pixel precision designated for motion compensation.

At STEP 1661, the image generation apparatus 104 is instructed to perform the image block setting process. As a result, the inside of the generated image in the frame memory 109 is designated as the generation position of the generated image block.

At STEP 374, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
Is instructed. The position of this reference image block indicates a position that is shifted to the right by 0.5 pixel from the half-pixel precision position specified for motion compensation.

In STEP 381, it is determined whether or not the half-pixel precision position designation of the reference image block is only in the vertical direction. If only in the vertical direction, the process proceeds to STEP 382. If not, the process proceeds to STEP 391.

At STEP 382, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
To instruct. The position of this reference image block indicates a position shifted by 0.5 pixel from the half-pixel precision position designated for motion compensation.

In STEP 1671, the image generation apparatus 104 is instructed to perform image block setting processing. As a result, the inside of the generated image in the frame memory 109 is designated as the generation position of the generated image block.

At STEP 384, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
To instruct. The position of this reference image block indicates a position shifted by 0.5 pixel from the half-pixel precision position designated for motion compensation.

In STEP 391, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
Is instructed. The position of this reference image block indicates a position shifted by 0.5 pixel on the left side by 0.5 pixel from the position of half pixel precision designated for motion compensation.

[0380] In STEP 393, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
To instruct. The position of this reference image block indicates a position shifted by 0.5 pixel on the right by 0.5 pixel from the position of half-pixel precision designated for motion compensation.

At STEP 395, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
To instruct. The position of this reference image block indicates a position shifted by 0.5 pixel to the left by 0.5 pixel from the half-pixel precision position designated for motion compensation.

In STEP 1681, the image generation apparatus 104 is instructed to perform image block setting processing. As a result, the inside of the generated image in the frame memory 109 is designated as the generation position of the generated image block.

At STEP 397, the image generation apparatus 104 is instructed to perform block motion compensation processing. As a result, the position of the reference image block in the reference image is determined by the image generation device 104.
To instruct. The position of this reference image block indicates a position shifted by 0.5 pixel to the right by 0.5 pixel from the position of half pixel precision designated for motion compensation.

The operation procedure of the image generating apparatus 104 controlled by the CPU 101 as described above to perform the motion compensation process will be described with reference to FIG.

FIG. 20 shows the operation procedure when the CPU 101 instructs the image generation apparatus 104 to perform block motion compensation processing.
Shown in In this processing, the instruction from the CPU 101 is analyzed by the control circuit 201 of the image generation device 104, and a command signal for controlling the operation of the pixel position designation circuit 1501 and the multiplier generation unit 207 of the pixel calculation circuit 203 is output.

The pixel position specifying circuit 1501 operates according to the following procedure.

In STEP 401, the pixel position (first pixel position) in the reference image block is obtained, and this pixel position is output to the image memory access circuit 1502 as the first pixel position signal 20b.

At the same time, in STEP 402, the pixel position (third pixel position) in the difference image block is obtained, and this pixel position is set as the third pixel position signal 20d.
Output to 502.

At the same time, in STEP 1701, the pixel position (second pixel position) in the generated image block is obtained, and this pixel position is output to the image memory access circuit 1502 as the second pixel position signal 150a. The second pixel position indicates a pixel position in the frame memory 109 or a pixel position in the block image memory, as instructed by the pixel block setting process.

In STEP 404, after the second image data is written in the second pixel position output in STEP 403, it is judged whether or not the processing of all the pixels in the generated image block is finished, and the processing is finished. Otherwise STEP 401, STEP 402, STEP 170
Proceed to 1.

The image memory access circuit 1502 operates in the following procedure.

First, the first image memory access section operates as follows.

In STEP 411, the first pixel position signal 20b is input, the image data of the pixel position of the frame memory 109 is read, and this image data is used as the first image data signal.
Output as 20e.

In STEP 1711, the second pixel position signal 150a
Is input, and it is determined whether or not this pixel position indicates the inside of the frame memory 109. If the second pixel position indicates the inside of the frame memory 109, proceed to STEP 412. If not, proceed to ST 412.
Proceed to EP 411.

In STEP 412, the second pixel position signal 150a and the second image data signal 150b are input, and the frame memory 10
The image data is written in the relevant pixel position within 9.

The second image memory access section operates as follows.

In STEP 413, the third pixel position signal 20d is input, the image data of the pixel position of the block image memory 205 is read, and this image data is output as the third image data signal 20g.

In STEP 1712, the second pixel position signal 150a
Is input and it is determined whether or not this pixel position indicates the inside of the block image memory 205. Second pixel position is block image memory
If it is 205, the procedure proceeds to STEP 1713, and if not, the procedure proceeds to STEP 413.

In STEP 1713, the second pixel position signal 150a
And the second image data signal 150b are input, and the image data is written in the pixel position in the block image memory 205.

The pixel calculation circuit 203 operates in the following procedure.

In STEP 421, the CPU 101 controls the control circuit 201.
According to the type of motion compensation processing instructed via, the multiplier generator 207 outputs the first multiplier and the second multiplier as shown in (Table 1).

In STEP 422, the first image data signal 20e
The first multiplying unit 209 multiplies the image data input as and the first multiplier.

In STEP 423, the third image data signal 20g
The second multiplying unit 208 multiplies the image data input as and the second multiplier.

In STEP 424, the addition unit 210 adds the results of the first multiplication unit 209 and the second multiplication unit 208, and outputs the addition result as the second image data signal 150b.

When the CPU 101 issues an image block setting processing instruction to the image generating apparatus 104, the instruction from the CPU 101 is analyzed by the control circuit 201 of the image generating apparatus 104, and the operations of the pixel position specifying circuit 1501 and the pixel moving circuit 206 are controlled. To be done.

The pixel position specifying circuit 1501 has a position of a difference image block, a reference image block and a position of a generated image block for which pixel positions must be generated for the next motion compensation process and texture mapping process (procedures will be described later),
Alternatively, the positions of the texture image and the generated polygon are set. The pixel position specifying circuit 1501 generates a pixel position according to the set image block position at the time of block motion compensation processing or texture mapping processing. In this embodiment, not only the frame memory 109 but also the block image memory 205 may be designated as the position of the generated image block, and accordingly, the pixel position in the frame memory 109 is set as the second pixel position. Whether to generate or to generate a pixel position in the block image memory 205 is controlled.

In the image moving circuit 206, the operation shown in FIG. 6 is performed to move the image block from the frame memory 109 to the block image memory 205. The details of the operation are as described in the first embodiment, and are omitted here.

By the above procedure, the image generating apparatus 104 according to the present embodiment realizes the motion compensation processing.

The operation during the forward motion compensation processing will be specifically described with reference to FIG. 7 in accordance with the operation procedure of the motion compensation processing described above. Here, the size of the image is set to 8x8 for simplification.
Pixels, size of image block for motion compensation 4x4
Pixels.

The operation is as follows.

The past image 601 is stored in the frame memory 109. A storage area in which the generated image 605 should be stored is also secured in the frame memory 109. CPU1
01 writes the difference image to the frame memory 109 (STEP 30
1).

[0412] Since the forward motion compensation processing is performed, the processing is ST
Proceed to EP 1611 (STEP 311 → STEP 321).

[0413] The CPU 101 instructs the image generation device 104 to perform the image block setting process, and the image blocks in the past image 601.
(0,1)-(3,4) is designated as the past image block 602, the block image memory is designated as the generated image block, and the position of the difference image block 603 in the frame memory 109 is designated (STEP 322). .

The processing of FIG. 19 is performed on the designated image block.

In STEP 351, the CPU 101 gives an instruction to move the difference image block 603 to the block image memory as an image block setting process. Upon receiving this instruction, the control circuit 201 controls the image moving circuit 206 to move the difference image block 603 in the frame memory 109 to the block image memory 205 (FIG. 6).

Since the position designation of the past image block 602 (0,1)-(3,4) in the past image 601 does not cause a half-pixel deviation from the pixel existing position, half-pixel precision motion compensation is not performed (S
TEP361).

The CPU 101 instructs the image generation apparatus 104 to perform the image block setting process (STEP 1651). By this instruction, the image generation device 104 specifies the position of the generated image block 604 in the generated image 605.

The CPU 101 instructs the image generation apparatus 104 to perform block motion compensation processing (STEP 1652). Image generator 10
4 receives this instruction and performs the process according to FIG.

The pixel position designation circuit 1501 calculates the pixel position (first pixel position) (0,0) in the past image block 602, and outputs this pixel position as the first pixel position signal 20b. On the other hand, the pixel position in the difference image block 603 (the third position
Pixel position) (0,0) is calculated, this pixel position is output as the third pixel position signal 20d, and the pixel position (second pixel position) (0,0) in the generated image block 604 is calculated. Then, this pixel position is output as the second pixel position signal 150a (STEP 40
1, STEP 402, STEP 1701).

In the first image memory access section 1504 in the image memory access circuit 1502, the first pixel position signal 20b
And past image block 602 in the frame memory 109.
The pixel data at the pixel position (0,0) is read from and is output as the first pixel data signal 20e (STEP
411). In the second image memory access unit 1503, the third pixel position signal 20d is input, the pixel data at the pixel position (0, 0) is read from the difference image block 603 in the block image memory 205, and this pixel data is It is output as the pixel data signal 20g of 3 (STEP 413).

In the multiplier generation unit 207 in the pixel calculation circuit 203,
According to (Table 1), 1.0 is generated as the first multiplier and the second multiplier is generated.
Generates 1.0 as a multiplier of (STEP 421). First multiplication unit 2
In 09, the pixel data input by the first pixel data signal 20e is multiplied by the first multiplier, and the result is output to the addition unit 210 (STEP 422). The second multiplication unit 208 multiplies the pixel data input by the third pixel data signal 20g by the second multiplier and outputs the result to the addition unit 210 (STEP 423). The addition unit 210 adds the multiplication results of the first multiplication unit 209 and the second multiplication unit 208 and outputs the result as the second pixel data signal 150b.
(STEP 424).

Next, in the first image memory access section 1504, the second pixel position signal 150a and the second pixel data signal 15
0b is input, it is determined that the second pixel position indicates the inside of the frame memory (STEP 1711), and the second pixel data is written to the pixel position (0,0) in the generated image block 604 (STEP 4111).
2). On the other hand, in the second image memory access unit 1503,
It is determined that the pixel position of does not point in the block image memory (STEP 1711).

The processing of FIG. 20 is repeated for each pixel in the generated image block 604 until all pixels in the generated image block 604 have been generated (STEP 404).

By performing the processing as described above, the generated image block 604 is generated from the past image block 602 and the motion compensation difference image 603.

The operation of the backward motion compensation processing is different from the above-described forward motion compensation only in that the future image is used as the reference image instead of the past image.

Next, the operation during the bidirectional motion compensation processing will be described. Here, only the points different from the bidirectional motion compensation processing in the first embodiment will be described.

For the bidirectional motion compensation processing, the backward motion compensation processing is performed after the forward motion compensation processing. ST
In the image block setting process of EP 1631, the inside of the block image memory 205 is designated as the generation place of the generated image block, and in the forward motion compensation process of STEP 1632, the generated image block is generated in the block image memory 205.
That is, the image block resulting from the forward motion compensation process is stored in the block image memory 205. This image data is the image data of the difference image block in the backward motion compensation process performed after the forward motion compensation process, and is already stored in the block image memory 205 at the time when the forward motion compensation process ends. The process of moving the generated image block to the block image memory for the backward motion compensation process, which is necessary in the first embodiment, is unnecessary.
In the backward motion compensation process, the position of the generated image block is specified in the frame memory 109, so the result of the bidirectional motion compensation process is generated in the generated image in the frame memory 109.

Next, the operation during the half-pixel precision motion compensation processing will be described with reference to FIG. Here, as in the first embodiment,
The description will be made assuming that the position is specified with half-pixel accuracy in both the vertical and horizontal directions.

According to this embodiment, 4
Interpolation interpolation and motion compensation processing of two image blocks 803, 804, 805 and 806 are performed at the same time.

The image block setting process instructs the generated image block to be written in the block image memory 205.

In STEP 391, the image data of the reference image block 803 is multiplied by 0.25 according to (Table 1), the difference image block read from the block image memory 205 is added, and the result is written in the block image memory 205.

In STEP 393, the image data of the reference image block 804 is multiplied by 0.25 according to (Table 1), the image block read from the block image memory 205 is added, and the result is written in the block image memory 205.

In STEP 395, the image data of the reference image block 805 is multiplied by 0.25 according to (Table 1), the image block read from the block image memory 205 is added, and the result is written in the block image memory 205.

Here, the position of the generated image block is changed within the generated image in the frame memory 109 (STEP 1681).

In STEP 397, the image data of the reference image block 804 is multiplied by 0.25 according to (Table 1), the image block read from the block image memory 205 is added, and the generated image block 205 in the frame memory 109 is added. Written in.

The unidirectional motion compensation process has been described above. In the case of the bidirectional motion compensation process, the block motion compensation process similar to the unidirectional motion compensation process is repeated eight times. In the case of the bidirectional motion compensation process, the first multiplier multiplied by the reference image data becomes 0.125, and the image blocks generated in the first seven block motion compensation processes are written in the block image memory 205, and finally. The result is written in the generated image block in the generated image of the frame memory 109 in the block motion compensation processing of.

As described above, in this embodiment, it is possible to simultaneously perform the interpolation processing and the motion compensation processing for obtaining the reference image block.

As described above, according to this embodiment, it is possible to realize the motion compensation processing with half-pixel precision.

The texture mapping processing in this embodiment can be realized by the same processing procedure as in the first embodiment. Of the components of the image generation apparatus in the first embodiment and the components in the second embodiment, the pixel position designation circuit 202 is the pixel position designation circuit 1501, the image memory access circuit 204 is the image memory access circuit 1502, and the first image memory The access unit 212 is replaced by the first image memory access unit 1504.
By associating the image memory access unit 211 of 1. with the first image memory access unit 1503, the operation of the texture mapping processing of the second embodiment becomes the same as that of the first embodiment. Therefore, the description of the operation of the texture mapping process is omitted here.

According to the image generating apparatus of this embodiment, the texture mapping of the semitransparent texture image can be processed including the antialiasing processing of the polygon edge portion.

Simultaneous execution of the motion compensation processing and the texture mapping processing in the present embodiment is similar to the feature of the motion compensation processing of the present embodiment described above, except for the points described below, except for the points described below. It is the same as the operation procedure for simultaneous execution.

The difference from the operation procedure of the first embodiment is the following two points.

(I) In one or more block motion compensation processes required for the motion compensation process, in the last block motion compensation process (or the block motion compensation and texture mapping process are simultaneously executed), the reference image block is used. And the calculation result of the image data of the difference image block is written in the pixel position forming the generated polygon in the frame.

(Ii) In the block motion compensation process before the final block motion compensation process (or simultaneous execution of block motion compensation and texture mapping process), the calculation result of the image data of the reference image block and the difference image block is stored in the block image memory 205. Write in.

By operating in this way, the motion compensation processing and the texture mapping processing can be executed simultaneously,
When the block motion compensation processing is performed a plurality of times, it is not necessary to move the image block from the frame memory 109 to the block image memory 205, and the processing speed is increased as compared with the image generation apparatus of the first embodiment.

The operation timing and processing time of the motion compensation process and the process of simultaneously executing the motion compensation process and the texture mapping process in the image generating apparatus 104 will be described below. Here, the operation of the image generation apparatus 104 will be considered when expanding a moving image compressed by the MPEG-1 standard, and when expanding a compressed moving image and performing texture mapping at the same time.

An image (frame) of the MPEG-1 standard is 320 ×
It is composed of 240 pixels. 16 blocks for motion estimation
It is composed of x16 pixels. As a result, one frame is composed of 20 × 15 = 300 image blocks. In addition, an image of 30 frames is included in one second. Here, it is assumed that the number of pixels of the polygon generated by the texture mapping process is 256, and that the reference image and the difference image are written in the frame memory 109 before the motion compensation process.

The performance of each block of the image generating apparatus according to this embodiment is assumed as follows.

The first image memory access section 1504 is configured to read / write two pixels simultaneously in parallel from / to the frame memory 109, and is further configured to execute a read or write cycle in one clock. Suppose

The block image memory 205 is composed of a dual port memory, and has a second image memory access section.
It is assumed that the 1503 can simultaneously process the reading of data from the block image memory 205 and the writing of data to the block image memory 205. The first multiplying unit 209, the second multiplying unit 208, and the adding unit 210 are assumed to be able to perform operations in one clock. Each circuit of the image generation device 104 is configured to be capable of pipeline operation. That is,
Memory access, multiplication, addition, and pixel location generation are processed in parallel. The image generation device 104 is composed of an LSI that operates at a clock of 50 MHz. In addition,
These performances are performances that can be sufficiently realized by the current technology.

The following two types of image processing will be described based on the above assumptions.

(I) When decompressing an MPEG-1 P picture, and when decompressing, texture mapping is performed. It is assumed that the image blocks forming the P picture are all compressed using the forward motion prediction, and the motion compensation of all the image blocks has half-pixel accuracy in both the vertical and horizontal directions. This will be considered when the throughput is the maximum.

(Ii) The texture mapping process is performed when the B-picture of MPEG-1 is expanded and when the B-picture is expanded. Consider a case where all the image blocks forming the B picture are compressed using bidirectional motion prediction, and the motion compensation of all the image blocks is half-pixel accurate in both the vertical and horizontal directions. This will be considered when the throughput is the maximum.

The processing procedure of the image motion compensation processing of (i) is as follows:
It is as follows.

1: Move the differential image block from the frame memory 109 to the block image memory 205 2: Read one pixel of pixel data from within the differential image block (block image memory 205) 3: Within the reference image block (frame memory 109) Read one pixel data from the pixel 4: Multiply the difference image data by a multiplier 5: Multiply the reference image data by a multiplier 6: Add the multiplication result 7: Write the pixel data of the operation result to the block image memory 205 8: Repeat processes 2 to 7 for each pixel in the block (total 256 times (= 16x16)) 9: Shift the reference image block position one pixel to the right and downward, and repeat processes 2 to 8 twice 10: Reference image Shift block position to lower right 11: Read 1 pixel of pixel data from difference image block 12: Read 1 pixel of pixel data from reference image block 13: Multiply the difference image data by the multiplier 14: Multiply the reference image data by the multiplier 15: Add the multiplication result 16: Write the pixel data of the calculation result to the frame memory 109 17: Process 11 to 16 in the block Repeat for each pixel 18: Repeat processing 1 to 17 for each block forming an image (300 times) The operation timing of processing 1 to 17 is shown in the timing chart of FIG.

From FIG. 28, the operation time of each step is as follows.

1 ... 16 clocks 2 to 8 ... 259 clocks 2 to 9 ... 777 clocks (= 259 × 3) 11 to 17 ... 259 clocks 18 ... 315600 clocks (= (16 + 259 × 4) ×
300) One clock cycle of an LSI operating at 50MHz is
It is 20ns, and the processing time is 6.312ms.
Becomes

A processing speed of 30 frames / sec is sufficient for decompressing an MPEG-1 image.

That is, the processing time for one frame is 33 ms.
If it is below, it means that the required performance is satisfied. From this, it is understood that the motion compensation process of the image generation apparatus 104 of the present embodiment can perform the motion compensation process at a sufficient speed.

Next, the case where the motion compensation process and the texture mapping process are simultaneously executed for the image (i) will be described.

In the P picture, since it is necessary to leave the generated image for motion compensation of the succeeding B picture, the processing procedure is as follows.

1: Perform processing 1 to 17 of motion compensation processing 2: Move the generated image block to the block image memory 205 3: Read one pixel of pixel data from within the texture image (block image memory 205) 4: Generated polygon One pixel data is read from the inside (frame memory 109) 5: Texture image data is multiplied by a semi-transparent multiplier 6: Generated polygon image data is multiplied by a semi-transparent multiplier 7: Multiplication result is added 8: Calculation result Pixel data is written in the generated polygon of the frame memory 109 9: Processes 3 to 8 are repeated for each pixel in the block (256 times) 10: 1 to 9 are repeated for each block forming the image (300 times) Process The operation timings of 1 to 9 are shown in the timing chart of FIG.

From FIG. 29, it is understood that the operation time of each step is as follows.

1 ... 1052 clocks (777 + 259) 2 ... 16 clocks 3-9 ... 259 clocks 1-10 ... 398100 clocks = (1052 + 16 + 259)
× 300 Processing time per frame is 7.962ms,
It can be seen that the processing speed required for expanding MPEG-1 is satisfied.

Next, the image motion compensation process (ii) will be described.

The processing procedure is as follows.

1: Move the differential image block from the frame memory 109 to the block image memory 205 2: Read one pixel of pixel data from within the differential image block (block image memory 205) 3: Within the reference image block (frame memory 109) Read one pixel data from the pixel 4: Multiply the difference image data by a multiplier 5: Multiply the reference image data by a multiplier 6: Add the multiplication result 7: Write the pixel data of the operation result to the block image memory 205 8: Repeat processing 2 to 7 for each pixel in the block (total 256 times (= 16x16)) 9: Shift the reference image block position by 1 pixel to the right, down, and lower right, and repeat processing 2 to 3 three times 10 : Set the future image as the reference image 11: Read one pixel of pixel data from the difference image block (block image memory 205) 12: In the reference image block (frame One pixel data is read from the memory memory 109) 13: Difference image data is multiplied by a multiplier 14: Reference image data is multiplied by a multiplier 15: Multiplication result is added 16: Pixel data of the calculation result is stored in the block image memory 205 Write 17: Repeat steps 11 to 16 for each pixel in the block (256 times (= 16x16) in total) 18: Shift the reference image block position one pixel to the right and below, and repeat steps 11 to 17 twice 19 : Shift the reference image block position to the lower right 20: Read one pixel of pixel data from the difference image block (block image memory 205) 21: Read one pixel data from the reference image block (frame memory 109) 22: Difference Multiply image data by multiplier 23: Multiply reference image data by multiplier 24: Add multiplication result 25: Write pixel data of operation result to frame memory 109 26: Process 2 0 to 25 are repeated for each pixel in the block 27: Processes 1 to 26 are repeated for each block forming the image (300 times) The operation timing of the processes 1 to 17 is shown in the timing chart of FIG.

From FIG. 30, the operation time of each step is as follows.

1 ... 16 clocks 2-8 ... 259 clocks 2-9 ... 1036 clocks (= 259 × 4) 11-17 ... 259 clocks 11-18 ... 777 clocks (= 259 × 3) 20-26 ... 259 clocks 27 … 626400 clocks (= (16 + 259 × 4 + 25
9 × 4) × 300) One clock cycle of an LSI operating at 50 MHz is
20ns, and the processing time is 12.528m.
s.

A processing speed of 30 frames / sec is sufficient for expanding an MPEG-1 image.

That is, it is sufficient if the processing time for one image is 33 ms or less. From this, it is understood that the motion compensation process of the image generating apparatus of the present invention can perform the motion compensation process at a sufficient speed.

Next, the simultaneous execution of the motion compensation process and the texture mapping process for the image (ii) will be described.

In the B picture, it is not necessary to leave the generated image for the subsequent motion compensation, and therefore the following processing procedure is taken.

1: Perform processing 1 to 19 of motion compensation processing 2: Set a generated polygon 3: Read one pixel of pixel data from the block image memory 205 4: Read pixel data from the reference image block (frame memory 109) 1 pixel is read 5: Difference image data (= texture image data) is multiplied by a multiplier 6: Reference image data is multiplied by a multiplier 7: Multiplication result is added 8: Calculation result pixel data is generated in the frame memory 109 9: Repeat processing 3 to 8 for each pixel in the block (256 times) 10: Repeat processing 1 to 9 for each block forming the image. (300 times) The difference from the case where only the motion compensation process is performed is that the writing destination of the generated pixel data in the processes 3 to 8 is a polygon, and the operation timing of the processes 1 to 9 is as shown in FIG.
It becomes the same as the timing chart of 0.

The operation time of each step is as follows.

1 ... 1829 clocks (= 16 + 259 × 4 + 259 × 3) 3-9 ... 259 clocks 1-10 ... 626400 clocks (= (1829 + 259) × 300) 1 clock cycle of an LSI operating at 50 MHz is
20ns, and the processing time is 12.528m.
s, which indicates that the processing speed is sufficient.

As described above, the image generating apparatus of the present invention
When implemented as a circuit operating at 0 MHz, in the expansion processing of an image compressed by MPEG-1, not only the motion compensation processing can be processed in real time, but also the expanded image is regarded as a texture image and the motion compensation processing and the texture mapping processing are performed. Simultaneous execution of can be processed in real time.

When the operation clock is set to 50 MHz, the motion compensation processing of the compressed image and the texture mapping of the image of the motion compensation result can be executed in a maximum of about 13 ms, so that the processing time per frame is 33 ms (1
Compared to (s / 30 frames), there is a margin of 20 ms, and this time can be used for texture mapping processing for other texture images and motion compensation processing for higher resolution images.

As described above, according to this embodiment,
A block image memory 205 that stores a texture image, a difference image, a motion compensation processing intermediate image, and the like, a block image memory 205, and pixel data of a reference image, a generated polygon, and a frame memory 109 that stores a generated image. An image memory access circuit 1502 to access, a pixel operation circuit 203 for performing an operation between pixels on the third pixel data read from the block image memory 205 and the first pixel data read from the frame memory 109, Pixel position specifying circuit that generates and specifies the pixel position of the pixel data of the calculation target and the pixel position where the pixel data of the calculation result is written according to processing such as motion compensation processing, texture mapping processing and simultaneous execution of motion compensation processing and texture mapping processing. 1501 and the image block in the frame memory 109 are moved to the block image memory 205. By providing the image moving circuit 206, forward motion compensation, bidirectional motion compensation, half-pixel precision motion compensation, and other decompression processing of moving images compressed using motion prediction and texture mapping of semi-transparent texture images and It is possible to realize texture mapping processing including anti-aliasing processing of polygon edges, and also simultaneous execution of motion compensation processing and texture mapping processing.

Further, according to the present embodiment, since the intermediate result of the motion compensation processing is stored in the block image memory 205, the number of memory accesses is significantly reduced as compared with the image generating apparatus of the first embodiment, and therefore the operation speed is high. Processing can be realized. Further, since the block image memory 205 only needs to store one difference image block, the block image memory 205 has a small capacity and can be integrated on the LSI together with a circuit for motion compensation processing / texture mapping processing. This means that the access speed of the block image memory 205 can be increased, which means that processing that is remarkably faster than the case of the first embodiment can be realized.

In the present embodiment, the size of the image block for motion compensation is set to 4 × 4 pixels for simplification of description, but the size is not limited to this, and 16 × 16 pixels or 8 × 8 pixels are used. It does not preclude the use of other sizes such as pixels. Although the size of the image is set to 8 × 8 pixels, it is possible to select any size. Furthermore, the processing order of the image blocks described in the procedure of the motion compensation processing of the present embodiment is not limited to this, and the processing orders can be interchanged. Furthermore, in the description of the texture mapping processing according to the present embodiment, the translucency of the translucent texture has been described as one type, but it is also possible to give different translucency to each pixel forming the texture. Further, although the backward motion compensation process is not described in the present embodiment, the backward motion compensation process has the same processing procedure as the forward motion compensation process, and thus the configuration of the image generation device described in the present embodiment is used. It is feasible.

(Third Embodiment) FIG. 31 shows the configuration of an image generation system 3100 according to the present invention. Hereinafter, the configuration and operation of the image generation system 3100 will be described in the order shown below.

1. Configuration of image generation system 3100 2. Configuration of image generation device 3114 3. 3. Texture mapping processing in the image generation device 3114 4. Video expansion processing in the image generation device 3114 Summary 1. Image Generation System 3100 Configuration The image generation system 3100 includes a CPU 3111 and a main memory.
3112, external storage device 3113, image generation device 3114, digital-analog converter (DAC) 3115, and display device
It includes a 3116 and a system bus 3117.

The CPU 3111 controls the entire system, the image generating device 3114, the external storage device 3113, and the like.

[0485] The main memory 3112 includes a program for controlling the image generating apparatus executed by the CPU 3111,
It stores a program describing the operation of each part of the system, images and other data read from the external storage device 3113, and data used by the program.

The external storage device 3113 stores compressed image data, texture image data, a drawing program and the like.

The image generating device 3114 will be described later.

The DAC 3115 converts the image data output from the image generation apparatus 3114 into a format that can be displayed on the display apparatus 3116.
A conversion is performed.

The display device 3116 displays the image output from the DAC 3115.

The system bus 3117 is used for connecting the CPU 3111, the main memory 3112, the external storage device 3113, and the image generating device 3114 to each other, and exchanging data between them.

2. Configuration of Image Generating Device 3114 FIG. 32 shows the configuration of the image generating device 3114 shown in FIG.

The first image memory 3121 has a region for storing image data for forward motion compensation prediction, a region for storing image data for backward motion compensation prediction and texture image data, And an area for storing currently generated image data.

The pixel position specifying circuit 3127 uses the first image address signal 3120a to specify the position of one of the plurality of pixels stored in the first image memory 3121. When the position of the pixel stored in the first image memory 3121 is specified by the pixel position specifying circuit 3127, the first image memory 31
21 reads the data corresponding to the specified position and outputs the data to the second image memory 3122 as the first image data signal 3120d.

Also, the pixel position specifying circuit 3127 uses the second image address signal 3120b to output the first image memory 3121.
The position of one of the plurality of pixels stored in is designated. The first image memory 31 by the pixel position specifying circuit 3127
When the position of the pixel stored in 21 is designated, the first image memory 3121 reads the data corresponding to the designated position, and the data is used as the second image data signal 3120e by the first multiplication unit 3123. Then, the generated image data signal 3120g output from the adder 3125 is written in the designated position.

The second image memory 3122 stores the image data written by the first image data signal 3120d read from the first image memory 3121. The pixel position specifying circuit 3127 specifies the position of one of the plurality of pixels stored in the second image memory 3122 using the third image address signal 3120c. The second image memory 3122 is
The data corresponding to the designated position is read out, and the data is used as the third image data signal 3120f in the second multiplication unit 312.
Output to 4.

The image data input circuit 3126 uses the image data input signal 3120h to generate the texture image data and the motion compensation difference image data designated by the CPU 3111 as the first image data.
Of image memory 3121.

The first multiplying unit 3123 operates the first image memory 31.
The image data read from 21 as the second image data signal 3120e is multiplied by the first multiplication coefficient designated by the multiplier generation unit 3128, and the multiplication result is output to the addition unit 3125.

The second multiplying unit 3124 uses the second image memory 31.
The image data read out from 22 as the third image data signal 3120f is multiplied by the second multiplication coefficient designated by the multiplier generation unit 3128, and the multiplication result is output to the addition unit 3125.

The addition unit 3125 adds the first multiplied image data signal output from the first multiplication unit 3123 and the second multiplied image data signal output from the second multiplication unit 3124, and the addition is performed. The result is output to the first image memory 3121 as a generated image data signal 3120g.

The pixel position designation circuit 3127 outputs the first image address signal 3120a, the second image address signal 3120b and the third image address signal 3120c to the first image memory 3121. The first image address signal 3120a corresponds to the first pixel address of the texture image to be texture-mapped or the pixel position of the reference image block in the motion compensation process.
It is used to indicate the read pixel position of the image memory 3121 of. The second image address signal 3120b indicates the read / write pixel position of the first image memory 3121 according to the pixel position inside the polygon to be texture-mapped or the pixel position inside the motion compensation difference image data in the moving image expansion processing. Used for. Third image address signal 31
20c is used to indicate the read pixel position of the second image memory 3122 according to the pixel position inside the texture image to be texture-mapped and the pixel position inside the reference image block at the time of motion compensation.

The multiplier generation unit 3128 determines the first multiplier signal indicating the first multiplier coefficient and the second multiplier signal according to the translucency of the texture image in the texture mapping, the translucency in the anti-aliasing process, and the processing procedure in the motion compensation process. And a second multiplier signal indicative of a multiplier coefficient of. The first multiplier signal is output to the first multiplier 3123, and the second multiplier signal is output to the second multiplier 3124.

The image data output circuit 3129 sequentially reads the image data stored in the first image memory 3121 for image display, and outputs the read image data to the DAC 3115.

Hereinafter, an image generating apparatus having the above-mentioned configuration
The operation of 3114 will be described.

3. Texture Mapping Process in Image Generating Device 3114 Hereinafter, the texture mapping process in the image generating device 3114 will be described.

FIG. 34 shows a procedure of texture mapping processing when a texture image is pasted on one polygon.

When a texture mapping start command is issued from the CPU 3111 (STEP 3401), the image data input circuit 3
126 stores the texture image data in the first image memory 3121.
Store in (STEP 3402).

The pixel position designation circuit 3127 calculates the position of each pixel forming the polygon to which the texture is attached based on the vertex coordinates of the polygon, and outputs a signal indicating that position as the second image address signal 3120b. To prepare for this (STEP 3403).

Further, the pixel position designating circuit 3127 finds the pixel position of the texture image to be attached to the polygon, and outputs a signal indicating the position as the first image address signal 3120a (STEP 3404).

The multiplier generation unit 3128 generates the first multiplier and the second multiplier according to whether the texture image is translucent or the generated pixels constitute the edge of the polygon (STEP 3405). ). The translucency of the texture image is α
In the case of (opaque: 1; transparent: 0), a second multiplier having the value α is generated and input to the second multiplication unit 3124, and the first multiplier having the value (1-α) is generated. It is generated and input to the first multiplication unit 3123. If the pixel currently being processed constitutes an edge of the polygon, a semitransparent coefficient β for antialiasing the edge is calculated, and the multiplication coefficient having the value αβ is calculated by the second multiplication unit 3124. And outputs the multiplication coefficient having the value (1-αβ) to the first multiplication unit.
Output to 3123. When the texture image is neither semi-transparent nor anti-aliasing processing of the edge of the polygon is performed, the multiplier “0” is input to the first multiplier 3123 and the multiplier “1” is input to the second multiplier 3124. Is entered.

Next, the image data (that is, the texture image) at the pixel position designated by the first image address signal 3120a is read from the first image memory 3121 and written in the second image memory 3122 (STEP 3406). .

[0511] According to the second image address signal 3120b indicating each pixel position in the polygon obtained in STEP 3403, the pixel data of one corresponding pixel is read from the first image memory 3121, and this is read as the second image data. The signal 3120e is input to the first multiplication unit 3123 (STEP 3407). This image data is image data existing at the position where the texture image is pasted before the texture image is pasted. Hereinafter, in this specification, such image data will be referred to as “background image data”.

At the same time, the pixel position specifying circuit 3127 outputs the address in the second image memory 3122 of the texture image to be attached to the pixel position indicated by the second image address signal 3120b, as the third image address signal 3120c. Then, one pixel of the corresponding texture image is read from the second image memory 3122 (STEP 3408). Second image memory 312
The image data read from 2 is input to the second multiplication unit 3124 as a third image data signal 3120f.

[0513] The first multiplication unit 3123 uses the first image memory 31
Multiplier generator for background image data read from 21
Multiply the first multiplier specified by 3128 (STEP 34
09).

[0514] The second multiplication section 3124 uses the second image memory 31
Multiplier generator 3128 for the image data read from 22
Multiply the second multiplier specified by (STEP 341
0).

[0515] The addition unit 3125 adds the multiplication result output from the first multiplication unit 3123 and the multiplication result output from the second multiplication unit 3124 (STEP 3411).

[0516] The addition result of the addition unit 3125 is written in the generated image pixel position of the first image memory 3121 as the generated image data signal 3120g (STEP 3412). The generated image pixel position is the pixel indicated by the second image address signal 3120b. This completes the mapping process for one pixel in the polygon.

[0517] In STEP 3413, it is determined whether all the pixels in the polygon have been processed. If all pixels have not been processed, the processes of STEP 3407 to STEP 3412 are repeated.

This completes the texture mapping process for one polygon (STEP 341).
Four).

The image generating apparatus 3414 implements the texture mapping process by performing the above operation for all the polygons forming the generated image.

FIG. 35A shows a concrete flow of image data in the texture mapping processing described above.
In FIG. 35 (a), the same components as those of FIG. 32 are designated by the same reference numerals.

That is, reference numeral 3121 indicates the first image memory, reference numeral 3122 indicates the second image memory, reference numeral 3123 indicates the first multiplication unit, and reference numeral 3124 indicates the second.
, And reference numeral 3125 indicates an adder.

Reference numbers 3151 and 3153 indicate texture images. Reference numeral 3152 indicates a generated image (polygon) to which the texture image 3151 is pasted. It is assumed that the texture image 3151 has been read into the first image memory 3121 in the above-mentioned operation STEP 3402.

[0523] Operation In STEP 3406, the first image memory 312
The texture image 3151 in 1 is the second image memory 3122.
Is forwarded to

When processing a semi-transparent polygon of transparency α, in operation STEP 3407 to STEP 3412, the background images in the polygon 3152 are sequentially read out, multiplied by (1-α) by the first multiplication unit 3123, and The pixel to be attached to this pixel position is read from the texture image 3153 in the second image memory 3122, multiplied by α by the second multiplication unit 3124, and input to the addition unit 3125. The generated image data output from the addition unit 3125 is written in the same pixel position in the polygon 3152.

FIG. 35B is a drawing in which horizontal lines indicating the order of read / write pixels are overlaid on the polygon 3152. The pixels in the polygon 3152 are read out in the order of the horizontal lines (1) to (8), and the generated image data is written at the positions of the read pixels. In addition, FIG.
Corresponding to the order shown in (b), the texture image 3153
It shows the order in which the pixels inside are read out. Straight line (1) ~
Pixels in the texture image 3153 are read out in the order of (8).

[0526] Second output from pixel position designation circuit 3127
Image address signal 3120a and third image address signal 312
The order of pixel positions specified by 0b and
This corresponds to the pixel access order shown in FIGS. 35B and 35C.

FIG. 35D shows the state of the polygon 3152 after the texture image 3153 is pasted. FIG.
In (d), the background image is omitted for simplicity of explanation.

Note that the size of the texture image transferred to the second image memory 3122 is, in the above description, the size necessary and sufficient for pasting on one polygon. However, the size of the texture image transferred to the second image memory 3122 can be set to an arbitrary size regardless of the size of the polygon.

[0529] When the size of the second image memory 3122 is smaller than the size of the texture image attached to one polygon, the size of the texture image that can be transferred to the second image memory 3122 at one time is one polygon. Not enough to stick on. Therefore, the texture image to be attached to one polygon is divided into a plurality of times and the second image memory is divided.
Need to transfer to 3122. In this case, the transfer operation from the first image memory 3121 to the second image memory 3122 STEP
Incorporate 3406 into the operation of STEP 3407 ~ STEP 3412,
The transfer operation may be performed a plurality of times as needed.

[0530] When the size of the second image memory 3122 is larger than the size of the texture image to be attached to one polygon, the operation STEP 3406 is performed once,
It is possible to transfer all the texture images needed to attach to one polygon. Thereby, the first
Since the number of times the image memory 3121 is accessed is reduced, the processing speed is improved.

[0531] Moving Image Decompression Processing in Image Generating Device 3114 Hereinafter, moving image decompressing processing in the image generating device 3114 will be described. Decompression of a moving image compressed by the inter-frame motion compensation predictive coding method is sequentially performed for each image block forming the image.

FIG. 36 shows the procedure of the moving picture decompression processing for one image block.

When the CPU 3111 issues an image block expansion command (STEP 3601), the image data input circuit 3126
The motion compensation difference data of this image block is stored in the first image memory 3121 (STEP 3602). It should be noted that the first image memory 3121 stores a forward prediction image or a backward prediction image for an image that is expanded as necessary. These predicted images are read out from the external storage device 3113 and C
There are a case where the image is written in the image generating apparatus 3114 by the PU 3111 and a case where the image is already generated by the image generating apparatus 3114.

The pixel position specifying circuit 3127 calculates the position of each pixel forming the image block to be expanded (that is, the pixel position where the motion compensation difference data is written), and outputs a signal indicating the position as the second image address. First as signal 3120b
To output to the image memory 3121 of the
03).

Next, the pixel position specifying circuit 3127 obtains the pixel position of the image of the predicted image data block from the first predicted image according to the value of the motion vector, and outputs a signal indicating that position as the first image address signal 3120a. As the first image memory 3
Output to 121 (STEP 3604).

It is calculated how many times the predicted image data block is read out and added to the generated image block according to the type of the predicted image used for the motion prediction and the value of the motion vector, and the calculated number of times is calculated. A second multiplier is generated based on this (STEP 3605). The detailed calculation method of the second multiplier will be described later. Note that the first multiplier is always set to "1" during the moving image expansion process.

Next, the image data of the pixel position designated by the first image address signal 3120a (that is, the image data of the predicted image data block) is transferred to the first image memory.
Read from 3121 and write to second image memory 3122 (S
TEP 3606).

[0538] According to the second image address signal 3120b indicating the position of each pixel forming the image block obtained in STEP 3603, the pixel data of one corresponding pixel (that is,
(Motion compensation difference data) is read from the first image memory 3121 and is input to the first multiplication unit 3123 as a second image data signal 3120e (STEP 3607).

At the same time, the pixel position specifying circuit 3127 determines the address in the second image memory 3122 of the predicted image corresponding to the motion compensation difference data of the pixel position indicated by the second image address signal 3120b as the third image address. The signal 3120c is output, and one pixel of the corresponding predicted image is read from the second image memory 3122 (STEP 3608). Second image memory 312
The image data read from 2 is input to the second multiplication unit 3124 as a third image data signal 3120f.

The first multiplying unit 3123 has the first image memory 31
Multiplier generator 3128 for the difference data read from 21
Multiply the first multiplier specified by (STEP 360
9). As set in STEP 3605, the first multiplier is always "1".

The second multiplying unit 3124 uses the second image memory 31
Multiplier generator 3128 for the image data read from 22
Multiply the second multiplier specified by (STEP 361
0).

The addition unit 3125 adds the multiplication result output from the first multiplication unit 3123 and the multiplication result output from the second multiplication unit 3124 (STEP 3611).

The result of addition by the adder 3125 is written as a generated image data signal 3120g at the generated image pixel position in the first image memory 3121 (STEP 3612). The generated image pixel position is the pixel indicated by the second image address signal 3120b. This completes the process for one predicted image of one pixel in the image block.

[0544] In STEP 3613, it is determined whether or not all the pixels in the image block have been processed. If all pixels have not been processed, the processes of STEP 3607 to STEP 3612 are repeated.

[0545] In STEP 3614, it is determined whether or not all necessary predicted images have been processed. If the image block being processed still needs to be added to another predicted image, the processes of STEP 3604 to STEP 3613 are repeated.

This completes the decompression process for one image block (STEP 3615).

By performing the above operation for all the image blocks that form the generated image, the image generating apparatus 34
14 realizes the moving picture expansion processing.

FIG. 37 (a) shows a concrete flow of image data in the moving picture expansion processing when the image block to be expanded is compressed using two prediction images in the forward direction and the backward direction. In FIG. 37 (a), the same components as those in FIG. 35 (a) are designated by the same reference numerals.

That is, reference numeral 3121 indicates the first image memory, reference numeral 3122 indicates the second image memory, reference numeral 3123 indicates the first multiplication unit, and reference numeral 3124 indicates the second.
, And reference numeral 3125 indicates an adder.

Reference numeral 3171 indicates forward prediction image data, and reference numeral 3172 indicates backward prediction image data. Reference numeral 3173 indicates an area in which generated image data is stored. Reference number 3174 indicates a forward prediction image data block, reference number 3175 indicates a backward prediction image data block, reference number 3176 indicates a prediction image data block,
Reference numeral 3177 indicates a generated image data block.

FIG. 37 (a) shows an image generating apparatus in a state where the processing of STEP 3602 is finished, that is, a state in which the motion compensation difference data block is loaded in the first image memory 3121.
The state of the image data in 3114 is shown. That is, the content of the generated image data block 3177 is motion compensation difference data.

FIG. 37B shows the result of processing using the forward prediction image block. That is, it is the result of performing the processes STEP 3604 to STEP 3613 shown in FIG. 36 on the forward prediction image block. The pixel data in the motion compensation difference data block stored in the generated image data block 3177 is represented as D (x, y), and the pixel data stored in the forward prediction image data block 3174 is F (x, y). When expressed as, the result of processing using the forward prediction image block,
The image data G1 (x, y) as shown in (Equation 9) is written to the generated image data block 3177.

Here, γ is a multiplier given to the second multiplication section 3124, which is 0.5 in this case.

[0554]

## EQU9 ## G1 (x, y) = D (x, y) + F (x, y) × γ FIG. 37 (c) shows the result of further processing using the backward prediction image block. Show. That is, FIG.
It is a result of performing the processing STEP 3604 to STEP 3613 shown in FIG. The pixel data stored in the backward prediction image data block 3175 is converted to R (x, y)
Then, as a result of processing using the backward prediction image block, image data G2 (x, y) as shown in (Equation 10) is written in the generated image data block 3177.

[0555]

## EQU10 ## G2 (x, y) = G1 (x, y) + R (x, y) × γ where γ is a multiplier given to the second multiplication unit 3124, which is 0.5 in this case.

By transforming (Equation 10) into (Equation 11), the pixel value of the generated image data block becomes 2
It can be seen that it is a value obtained by adding the motion compensation difference data to the average of the two predicted image blocks.

[0557]

[Equation 11] G2 (x, y) = G1 (x, y) + R (x, y) × γ = {D (x, y) + F (x, y) × γ} + R (x, y ) × γ = {F (x, y) + R (x, y)} × γ + D (x, y) γ = 0,5, so G2 (x, y) = {F (x, y) ) + R (x, y)} / 2 + D (x, y) In FIGS. 37 (a) to (c), the image block to be decompressed is compressed using two prediction images in the forward direction and the backward direction. The operation of the decompression process in the case of the above has been described. Next, the operation of the decompression process when the image block to be decompressed is compressed using one forward or backward predicted image will be described.

Processing shown in FIG. 36 STEP 3604 to STEP 36
13 is performed for one prediction image block in the forward direction or the backward direction. The result G1 (x, y) is as shown in (Equation 12).

[0559]

## EQU12 ## G1 (x, y) = D (x, y) + F (x, y) × γ where γ is 1.

Next, the decompression process when the value of the motion vector is given in units of half pixels will be described. In this case as well, the processing STEP 3604 to STE is performed for the forward and / or backward prediction images used for compression.
Repeating P 3613 is exactly the same, but for the prediction image in which the value of the motion vector is given in units of half pixels, the prediction image block shifted by 1 pixel is used twice or four times.
Repeat steps 3604 to 3613. And
The multiplier γ given to the second multiplication unit 3124 is changed according to the number of times of repetition and whether the predicted image is in the forward direction or the backward direction or in the forward direction and the backward direction.

FIGS. 38 (a) to 38 (c) show predicted image blocks when the value of the motion vector is given in half pixel units. 38A to 38C, it is assumed that one image block is composed of 4 × 4 pixels. The horizontal direction is indicated by x and the vertical direction is indicated by y. The ∘ mark represents one pixel forming the predicted image block. Written on the image block
The numbers 16 to 20 and the numbers 21 to 25 written on the right side of the image block mean the address indicating the pixel position.

As shown in FIG. 38 (a), for a predictive image block whose motion vector is specified in the horizontal direction in units of half pixels, first process the predictive block Q0 (x, y) and then predict the predictive block Q0 (x, y). The block Q1 (x, y) is processed.
In this way, the processing is performed twice in total.

As shown in FIG. 38 (b), with respect to a prediction image block whose motion vector is designated in the vertical direction in half pixel units, the prediction block Q0 (x, y) is processed first, and then the prediction block Q0 (x, y) is processed. The block Q2 (x, y) is processed.
In this way, the processing is performed twice in total.

As shown in FIG. 38 (c), with respect to the predictive image block in which the motion vector is specified in the vertical and horizontal directions in half pixel units, the predictive blocks Q0 (x, y), Q1 (x, y), Q2
(x, y) and Q3 (x, y) are processed four times in total. that time,
The value of γ set in the second multiplication unit 3124 is shown in (Table 2).

[0565]

[Table 2] As shown in (Table 2), the value of γ changes according to the number of prediction images used when the original image block is compressed.

In (Equation 13), an image block compressed using both forward and backward predicted images, the forward predicted image block is designated in units of half pixels in the vertical and horizontal directions.
The progress and the result of the decompression process in the case where the backward predicted image block is specified only in the horizontal direction in units of half pixels are shown. G1 (x, y), G2 (x, y), G3 (x, y), G4 (x, y), G5 (x,
y) represents the pixel value of the generated image block during the decompression process, and G6 (x, y) represents the final result of the decompression process. Also, F0 (x,
As can be inferred from FIG. 38 (c), y), F1 (x, y), F2 (x, y), and F3 (x, y) are shifted by half a pixel in the prediction block of the forward prediction image. Represents the pixel value of one image block, R0
(x, y) and R1 (x, y) represent pixel values of two image blocks that are shifted by half a pixel in the prediction block of the backward prediction image.

[0567]

[Equation 13] G1 (x, y) = D (x, y) + F0 (x, y) x 0.125 G2 (x, y) = G1 (x, y) + F1 (x, y) x 0.125 G3 ( x, y) = G2 (x, y) + F2 (x, y) x 0.125 G4 (x, y) = G3 (x, y) + F3 (x, y) x 0.125 G5 (x, y) = G4 (x, y) + R0 (x, y) × 0.25 G6 (x, y) = G5 (x, y) + R1 (x, y) × 0.25 The result G6 (x, y) is transformed into (Equation 14 )become that way.
According to this, the pixel value of the generated generated image data block is an average of a plurality of predicted image blocks that are shifted by one pixel according to the value of the motion vector for each of the forward and backward predicted images. It can be seen that it is a value obtained by adding the average value of both of the predicted image blocks and the motion compensation difference data.

[0568]

[Equation 14] G6 (x, y) = [{F0 (x, y) ＋ F1 (x, y) ＋ F2 (x, y)
＋ F3 (x, y)} / 4+ {R0 (x, y) ＋ R1 (x, y)} / 2] / 2 +
By performing similar processing on combinations other than D (x, y) (Equation 13), it is possible to realize the expansion processing of the image block.

5. Summary As described above, according to the present invention, a decompression process of a moving image compressed using inter-frame motion compensation predictive coding,
It becomes possible to perform texture mapping of a semi-transparent texture image and texture mapping processing including anti-aliasing processing of polygon edges using a single image generation device 3114.

[0570] Note that the above-described image generation device 3114 is capable of performing the anti-aliasing process of the semi-transparent texture image and the polygon edge, and the decompression process using a plurality of predicted images, in order to enable the second process. The multiplier 3124 was provided.

However, in the texture mapping process, if the texture image is always opaque (that is, the translucency is always 1), the second multiplication unit 3124 may be omitted. In this case, the multiplier generator 3128 is the first multiplier
The multiplier “0” is output to 3123. As a result, the generated image data signal output from the addition unit 3125 does not include the background image component at all, and includes only the texture image component.

Also in the decompression processing, the image block is compressed using only either the forward prediction image data or the backward prediction image data, and the value of the motion vector is specified in pixel units. If the second
The multiplication unit 3124 of may be omitted. In this case, the multiplier generator
The 3128 outputs the multiplier “1” to the first multiplication unit 3123. As a result, the generated image data signal output from the adder 3125 includes the result of adding one piece of prediction image data and motion compensation difference image data.

As described above, even in the image generating apparatus having the configuration in which the second multiplication unit 3124 is removed, both the texture mapping process and the moving image decompression process can be realized under a specific condition. All the processes that can be realized are basic processes.

(Embodiment 4) According to the following order,
The configuration and operation of another image generation system 3100 'according to the present invention will be described.

1. Configuration of image generation system 3100 '2. Configuration of image generation device 3114 '3. 3. Texture mapping processing in the image generation device 3114 ' 4. Video expansion processing in the image generation device 3114 ' Summary 1. Configuration of Image Generation System 3100 ′ The configuration of the image generation system 3100 ′ is the same as the configuration of the image generation system 3100 shown in FIG. 31, except for the image generation device 3114 ′. Therefore, the same components are designated by the same reference numerals, and the description thereof will be omitted here.

2. Configuration of Image Generation Device 3114 ′ FIG. 33 shows the configuration of the image generation device 3114 ′. The configuration of the image generation device 3114 ′ is the same as the configuration of the image generation device 3114 shown in FIG. 32 except that a FIFO memory 3130 is added. Therefore, the same components are designated by the same reference numerals, and the description thereof will be omitted here.

[0577] The second image data signal 31 read from the position designated by the second image address signal 3120b.
20e is stored in the FIFO before being input to the first multiplication unit 3123.
It is input to the memory 3130 and held there. Further, the image generation device 3114 ′ is configured so that the image data input signal input from the image data input circuit 3126 can be directly written in the FIFO memory 3130. Therefore, the image data input circuit 3126 writes the texture image data or the motion compensation difference image data in the first image memory 3121 or the FIFO memory 3130 by the image data input signal. The FIFO memory 3130 is an image data input circuit.
The image data written by the image data input signal output from 3126 or the image data input from the first image memory 3121 as the second image data signal 3120e is stored and output as the fourth image data signal. .
Hereinafter, the operation of the image generating apparatus 3114 ′ having the above configuration will be described. The operation of the image generating apparatus 3114 ′ is the same as the operation of the image generating apparatus 3114 except for a part. Therefore, in the following, a part different from the operation of the image generating device 3114 will be described in detail.

[0578] 3. Texture Mapping Process in Image Generating Device 3114 ′ The texture mapping process in the image generating device 3114 ′ will be described below.

The procedure of the texture mapping process when a texture image is pasted on one polygon is shown in FIG.
As shown in FIG.

The FIFO memory 3130 stores the second image data signal read from the position designated by the second image address signal 3120b (the image data in the polygon existing at the generated image position before the texture is pasted). That is, before inputting (background image data) to the first multiplication unit 3123, it serves as a buffer memory through which the data is once passed.

The operation of the FIFO memory 3130 is included in the processing STEP 3407 in FIG. In STEP 3407, according to the second image address signal 3120b indicating each pixel position in the polygon obtained in STEP 3403, the image data of one corresponding pixel is output as the second image data signal 3120e from the first image memory 3121. The data is read out and is once input to the first multiplication unit 3123 as a fourth image data signal via the FIFO memory 3130.

The other operations in the texture mapping processing are the same as the operations of the image generating apparatus 3114 shown in FIG. As a result, it is possible to realize texture mapping processing including texture mapping of a semi-transparent texture image and polygon-edge anti-aliasing processing.

[0583] 4. Video Decompression Process in Image Generation Device 3114 ′ Next, the video decompression process in the image generation device 3114 ′ will be described.

FIG. 39 shows the operation procedure of the decompression process for one image block. The operation shown in FIG. 39 is ST
The operation is the same as that shown in FIG. 36 except for EP 3902, STEP 3093, STEP 3907-1 and STEP 3907-2. In FIG. 39, the same operations as those in FIG. 36 are designated by the same reference numerals.

When the CPU 3111 issues an image block expansion command (STEP 3601), the image data input circuit 3126
The motion compensation difference data of this image block is stored in the FIFO memory 3130 (STEP 3902). It should be noted that the first image memory 3121 stores a forward prediction image or a backward prediction image for an image that is expanded as necessary. These predicted images are read from the external storage device 3113 and the CP
The image may be an image written in the image generating device 3114 ′ by the U 3111 or may be an image that has already been generated by the image generating device 3114 ′.

The pixel position specifying circuit 3127 calculates the position of each pixel forming the image block to be expanded (that is, the pixel position where the motion compensation difference data is written), and outputs a signal indicating the position as the second image address. First as signal 3120b
To output to the image memory 3121 of the
03). The order of the pixel positions indicated by the second image address signal 3120b corresponds to the order of reading the motion compensation difference data of the image block to be expanded from the FIFO memory 3130.

Next, the pixel position specifying circuit 3127 obtains the pixel position of the image of the predicted image data block from the first predicted image according to the value of the motion vector, and outputs a signal indicating that position as the first image address signal 3120a. As the first image memory 3
Output to 121 (STEP 3604).

[0588] The number of times a predicted image data block is read out and added to the generated image block is calculated according to the type of the predicted image used for motion prediction and the value of the motion vector, and the calculated number of times is calculated. A second multiplier is generated based on this (STEP 3605). The detailed calculation method of the second multiplier is as described in the third embodiment. The first
The multiplier of is always set to "1" at the time of moving image expansion processing.

Next, the image data at the pixel position designated by the first image address signal 3120a (that is, the image data of the predicted image data block) is transferred to the first image memory.
Read from 3121 and write to second image memory 3122 (S
TEP 3606).

[0590] According to the second image address signal 3120b indicating the position of each pixel forming the image block obtained in STEP 3603, the pixel data of one corresponding pixel (that is,
(Motion compensation difference data) is read from the FIFO 3130 and is input to the first multiplication unit 3123 as the fourth image data signal (STEP 3907-2).

The processing of STEP 3608 to STEP 3613 is the same as that of the third embodiment.

[0592] In STEP 3614, it is determined whether or not all necessary predicted images have been processed. If the image block being processed still needs to be added to another predicted image, the processes of STEP 3604 to STEP 3613 are repeated.

In this embodiment, in the processing of STEP 3907 for the second and subsequent predicted images, the first image memory 3131 is used.
The image data is read from the pixel position in the generated image area of, and is once input to the first multiplication unit 3123 as the fourth image data signal via the FIFO memory 3130 (STEP.
3907-1 ~ STEP 3907-2).

5. Summary As described above, in the present embodiment, the motion compensation difference image data block first written from the image data input circuit 3126 is not stored in the first image memory 3121, but instead, it is directly stored in the FIFO memory 3130. By writing to the second image data signal from the first image memory 3121.
The operation of reading out as 3120e can be omitted. As a result, the number of accesses to the first image memory 3132 can be reduced and the processing speed can be increased. This effect remarkably appears when decompressing an image data block compressed using only one of the forward prediction image and the backward prediction image in motion compensation prediction. That is,
In the case of the configuration in which the FIFO memory 3130 is not provided, the number of accesses to the first image memory 3121 required to decompress the image data block compressed by using one predicted image is read or write. Two times, a total of four times. On the other hand, in the case of the configuration in which the FIFO memory 3130 is provided, the number of accesses to the first image memory 3121 required to decompress the image data block compressed by using one predicted image is read or write. It is a total of 2 times, once each. By providing the FIFO memory 3130 in this way, the number of accesses to the first image memory 3121 can be reduced by half.

The image generating device 3114 ′ described above is configured to be able to perform the anti-aliasing process for the semi-transparent texture image and the polygon edge, and the decompression process using the plurality of predicted images. Was equipped with a multiplication unit 3124.

However, in the texture mapping process, if the texture image is always opaque (that is, the semi-transparency is always 1), the second multiplication unit 3124 may be omitted. In this case, the multiplier generator 3128 is the first multiplier
The multiplier “0” is output to 3123. As a result, the generated image data signal output from the addition unit 3125 does not include the background image component at all, and includes only the texture image component.

Also in the decompression processing, the image block is compressed using only either the forward prediction image data or the backward prediction image data, and the value of the motion vector is specified in units of one pixel. If the second
The multiplication unit 3124 of may be omitted. In this case, the multiplier generator
The 3128 outputs the multiplier “1” to the first multiplication unit 3123. As a result, the generated image data signal output from the adder 3125 includes the result of adding one piece of prediction image data and motion compensation difference image data.

As described above, even in the image generating apparatus having the configuration in which the second multiplication unit 3124 is removed, both the texture mapping process and the moving image decompression process can be realized under a specific condition. All the processes that can be realized are basic processes.

(Fifth Embodiment) An image generating apparatus 4110 according to the present invention will be described below.

The image generation device 4110 includes a pixel calculation circuit 4120 and an image memory 4130. The image generation device realizes texture mapping processing and motion compensation processing with a single configuration.

Hereinafter, with reference to FIG. 42, the image generation device 41
The procedure of texture mapping processing in 10 will be described. The numbers 1 to 3 in FIG. 42 correspond to the numbers indicating the processing procedure described below.

0: Write the texture image in the image memory 4130.

The generated image area is set in the image memory 4130.

1: Read pixel data from the texture image.

[0605] The pixel data of the texture mapping target coordinates of the texture mapping target polygon in the generated image is read.

2: The texture pixel data and the polygon pixel data are each multiplied by a coefficient, and the multiplication results are added.

3: Write the pixel data of the addition result to the texture mapping target coordinates in the generated image.

4: Processings 1 to 3 are repeated until all pixels in the texture mapping target polygon are processed.

Hereinafter, with reference to FIG. 43, the image generation device 41
The procedure of motion compensation processing in 10 will be described. The numbers 1 to 4 and 7 in FIG. 43 correspond to the numbers indicating the processing procedure described below.

0: The difference image is written in the image memory 4130.

The reference image is written in the image memory 4130.

A generation image area is set in the image memory 4130.

1: Extract an image block to be subject to motion compensation from the difference image.

(In FIG. 43, the image block is copied. Instead of copying the image block, the location of the image block may be designated.) The reference image block in the reference image is set.

Set the generated image block in the generated image.

2: Read pixel data from the difference image block.

Pixel data is read from the reference image block.

3: The pixel data read from the reference image block is multiplied by the coefficient.

(The coefficient is whether the motion compensation processing is bidirectional,
It depends on whether or not the position designation of the reference image block has half-pixel precision. ) Add the multiplication result and the pixel data read from the difference image block.

4: Write the pixel data of the addition result into the generated image block.

5: Processes 2 to 4 are repeated until all pixels in the image block have been processed.

6: When the motion compensation processing of the image block is completed, the processing is repeated for all the image blocks in the generated image.

7: In the case of bidirectional motion compensation processing or when the position of the reference image block is designated with half-pixel precision, a new reference image block is set.

The generated image block is extracted as a difference image block.

(In FIG. 43, the image block is copied. Instead of copying the image block, the location of the image block may be designated.) 8: Bidirectional motion compensation processing and reference image block The processes 2 to 7 are repeated according to the position designation with half-pixel precision.

Hereinafter, with reference to FIG. 44, the image generation device 41
The procedure of another motion compensation process in 10 will be described. FIG.
The numbers 1 to 4 and 7 in the above correspond to the numbers indicating the processing procedure described below.

0: The difference image is written in the image memory 4130.

[0628] The reference image is written in the image memory 4130.

A generated image area (the same area as the difference image) is set in the image memory 4130.

1: Extract an image block serving as a reference for motion compensation from the reference image.

(In FIG. 44, the image block is copied. Instead of copying the image block, the location of the image block may be designated.) The difference image block in the difference image is set.

The same image block as the difference image block is set as the generated image block.

2: Read pixel data from the difference image block.

Pixel data is read from the reference image block.

3: The pixel data read from the reference image block is multiplied by the coefficient.

(The coefficient is whether the motion compensation processing is bidirectional,
It depends on whether or not the position designation of the reference image block has half-pixel precision. ) Add the multiplication result and the pixel data read from the difference image block.

4: Write the pixel data of the addition result at the same coordinates as the differential pixel data read coordinates in the generated image block.

5: Processes 2 to 4 are repeated until all pixels in the image block have been processed.

6: When the motion compensation processing of the image block is completed, the processing is repeated for all the image blocks in the generated image.

7: In the case of bidirectional motion compensation processing, or when the reference image block is designated with half-pixel accuracy, a new reference image block is extracted.

(In FIG. 44, the image block is copied. Instead of copying the image block, the location of the image block may be designated.) The generated image block is set as the difference image block.

8: The processes 2 to 7 are repeated according to the bidirectional motion compensation process and the half-pixel precision position designation of the reference image block.

Hereinafter, with reference to FIG. 45, the image generation device 41
The processing procedure in the case of simultaneously performing the texture mapping processing and the motion compensation processing in 10 will be described. Numbers 1 to 4 in FIG. 45 correspond to numbers indicating the processing procedure described below.

0: The difference image is written in the image memory 4130.

A reference image is written in the image memory 4130.

A generation image area is set in the image memory 4130.

1: Extracts an image block to be motion-compensated from the difference image.

(In FIG. 45, the image block is copied. Instead of copying the image block, the location of the image block may be designated.) The reference image block in the reference image is set.

A polygon to be texture-mapped is set in the generated image.

2: Read pixel data from the difference image block.

Pixel data is read from the reference image block.

3: Multiplies the pixel data read from the reference image block by a coefficient.

(The coefficient is whether the motion compensation processing is bidirectional,
It depends on whether or not the position designation of the reference image block has half-pixel precision. ) Add the multiplication result and the pixel data read from the difference image block.

4: The pixel data of the addition result is written in the generated polygon.

5: Processes 2 to 4 are repeated until all pixels in the generated polygon have been processed.

6: When the generation polygon processing is completed, the processing is repeated for all the polygons in the generation image.

[0657]

According to the image generating apparatus of the present invention, not only forward or backward motion compensation processing but also bidirectional motion compensation processing, half-pixel precision motion compensation processing, and texture mapping processing are realized by a single apparatus. it can.

Furthermore, the simultaneous parallel processing of the motion compensation processing and the texture mapping processing can be realized by a device having a single structure.

By storing the intermediate result of the motion compensation processing in the block image memory, it is possible to greatly reduce the number of memory accesses required for the motion compensation processing.

Furthermore, by integrating the block image memory and the pixel calculation means on the LSI, it is possible to significantly speed up the memory access required for the motion compensation processing and the texture mapping processing.

As described above, by realizing the image generation apparatus capable of performing the compressed moving picture decompression processing by the motion compensation processing, the texture mapping processing, and the simultaneous parallel processing of these processing with a single configuration, this image Reduction of data transfer amount by simplifying configuration of moving picture expansion mapping device and multimedia device using generator, reduction of circuit scale, reduction of memory amount required for processing, and elimination of moving image data transfer between memories Can be realized.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a multimedia device according to a first embodiment of the present invention.

FIG. 2 is a configuration diagram of an image generation apparatus according to the first embodiment.

FIG. 3 is a control flowchart for controlling the operation of the image generating apparatus by the CPU when performing the motion compensation process in the first embodiment.

FIG. 4 is a part of a control flowchart for controlling the operation of the image generating apparatus by the CPU when performing motion compensation processing in the first embodiment.

FIG. 5 is an operation flowchart of each circuit in the image generation apparatus when performing motion compensation processing in the first embodiment.

FIG. 6 is an operation flowchart of each circuit in the image generation apparatus when performing an image block setting process in the first embodiment.

FIG. 7 is an explanatory diagram of forward motion compensation processing according to the present invention.

FIG. 8 is an explanatory diagram of bidirectional motion compensation processing according to the present invention.

FIG. 9 is an explanatory diagram of half-pixel precision motion compensation processing according to the present invention.

FIG. 10 is a control flowchart for controlling the operation of the image generating apparatus by the CPU when performing texture mapping processing in the present invention.

FIG. 11 is an operation flowchart of each circuit in the image generating apparatus when performing texture mapping processing in the present invention.

FIG. 12 is an explanatory diagram of texture mapping processing according to the present invention.

FIG. 13 is a control flowchart for controlling the operation of the image generation apparatus by the CPU when simultaneously performing the motion compensation process and the texture mapping process in the first embodiment.

FIG. 14 is a part of a control flowchart for controlling the operation of the image generation apparatus by the CPU when simultaneously performing the motion compensation process and the texture mapping process in the first embodiment.

FIG. 15 is an operation flowchart of each circuit in the image generation apparatus when the motion compensation process and the texture mapping process are simultaneously executed in the first embodiment.

FIG. 16 is an explanatory diagram of simultaneous execution of motion compensation processing and texture mapping processing according to the present invention.

FIG. 17 is a configuration diagram of an image generation apparatus according to a second embodiment.

FIG. 18 is a control flowchart for controlling the operation of the image generating apparatus by the CPU when performing motion compensation processing in the second embodiment.

FIG. 19 is a part of a control flowchart for controlling the operation of the image generating apparatus by the CPU when performing motion compensation processing in the second embodiment.

FIG. 20 is an operation flowchart of each circuit in the image generation apparatus when performing motion compensation processing in the second embodiment.

FIG. 21 is an explanatory diagram of a procedure of image compression processing using motion prediction processing.

[Fig. 22] Fig. 22 is an explanatory diagram of a decompression processing procedure of a compressed moving image that has been compression-encoded by motion prediction.

FIG. 23 is an explanatory diagram of texture mapping processing.

FIG. 24 is a conceptual explanatory diagram of simultaneous execution of motion compensation processing and texture mapping processing.

FIG. 25 is an explanatory diagram of an application of texture mapping using moving image data as a texture image.

FIG. 26 is an operation timing chart of the multimedia device of the second embodiment.

FIG. 27 is a configuration diagram of an image generation system in a conventional example.

[Fig. 28] Fig. 28 is a diagram illustrating a timing chart when motion compensation processing is performed on an MPEG-1 P picture.

[Fig. 29] Fig. 29 is a diagram illustrating a timing chart when the motion compensation process and the texture mapping process are simultaneously performed on an MPEG-1 P picture.

[Fig. 30] Fig. 30 is a diagram illustrating a timing chart when motion compensation processing or motion compensation processing and texture mapping processing are simultaneously performed on an MPEG-1 B picture.

FIG. 31 is a diagram showing a configuration of an image generation system 3100 according to the present invention.

FIG. 32 is a diagram showing a configuration of an image generation device 3114 included in the image generation system 3100.

FIG. 33 is a diagram showing the configuration of an image generation device 3114 ′ included in the image generation system 3100 ′.

FIG. 34 is a flowchart showing a procedure of texture mapping processing when a texture image is attached to one polygon in the image generation device 3114.

FIG. 35A is a diagram showing a specific flow of image data in the texture mapping process, FIG. 35B is a diagram showing the order of read / written pixels, and FIG. 35C is a sequence of reading pixels in the texture image. FIG. 3D is a diagram showing a state of the polygon after the texture image is pasted.

[Fig. 36] Fig. 36 is a flowchart illustrating a procedure of moving image decompression processing for one image block in the image generation device 3114.

FIG. 37 (a) is a diagram showing a specific flow of image data in the moving image decompression process, FIG. 37 (b) is a diagram showing a result of processing using forward prediction image blocks, and FIG. 37 (c) is a diagram (b). It is a figure which shows the result of having further processed using the backward prediction image block to the result of this.

38A to 38C are diagrams showing predicted image blocks when the value of a motion vector is given in a unit of a half pixel.

[Fig. 39] Fig. 39 is a flowchart illustrating a procedure of moving image decompression processing for one image block in the image generation device 3114 '.

40 is a diagram showing an internal configuration of the image decompression circuit 2404 shown in FIG. 27. FIG.

41 is a diagram for explaining the operation of a conventional texture mapping process performed in the image synthesizing circuit 2406 and the frame memory 2410 shown in FIG. 27.

[Fig. 42] Fig. 42 is a diagram for describing the procedure of texture mapping processing in the image generation device 4110.

[Fig. 43] Fig. 43 is a diagram for describing the procedure of motion compensation processing in the image generation device 4110.

[Fig. 44] Fig. 44 is a diagram for describing the procedure of another motion compensation process in the image generation device 4110.

[Fig. 45] Fig. 45 is a diagram for describing a processing procedure in the case where the texture mapping process and the motion compensation process are simultaneously performed in the image generation device 4110.

[Explanation of symbols]

 101 CPU 104 Image generation device 105 Variable length inverse encoding unit 106 Orthogonal transformation unit 108 Video expansion mapping device 109 Frame memory 201 Control circuit 202 Pixel position designation circuit 203 Pixel operation circuit 204 Image memory access circuit 205 Block image memory 206 Image movement circuit 207 Multiplier generation unit 208 Second multiplication unit 209 First multiplication unit 210 Addition unit 211 Second image memory access unit 212 First image memory access unit 1501 Pixel position specification circuit 1502 Image memory access circuit 1503 Second image Memory access unit 1504 First image memory access unit

Claims (30)

[Claims] 1. A pixel calculation means for performing calculation using two pieces of image data, wherein the pixel calculation means, when performing motion compensation processing, includes image data of a reference image and image data of a difference image. An image generation device that performs a texture mapping calculation using image data of a mapping image and image data in a polygon when performing a motion compensation processing calculation using the. 2. A command signal for instructing a process including at least a motion compensation process or a texture mapping process is received, and a signal indicating a corresponding first pixel position, a signal indicating a second pixel position, and a third signal position, respectively. When the signal indicating the pixel position is output and the command signal indicates the motion compensation process, the position of the pixel in the reference image block is set as the first pixel position, and the position of the pixel in the generated image block is set as the pixel position. The second pixel position is set, the position of the pixel in the difference image block is set as the third pixel position, and when the command signal indicates the texture mapping process, the position of the pixel in the polygon to be mapped is set to the third pixel position. Pixel position specifying means for setting the pixel position of 1 and the second pixel position, and the position of the pixel in the texture image as the third pixel position; and the first pixel Receiving a signal indicative of the location, and outputs the pixel value of the pixel position of the first as the first pixel data, said third
Signal indicating the pixel position of the second pixel position, the pixel value of the third pixel position is output as third pixel data, the signal indicating the second pixel position and the second pixel data are received, and Image memory access means for writing the second pixel data at the pixel position of the second pixel data and the first pixel data and the third pixel data, and the calculation result is used as the second pixel data. An image generation apparatus comprising: a pixel calculation unit for outputting. 3. The image generating apparatus according to claim 2, wherein the second pixel position and the third pixel position are the same when the command instructs a motion compensation process. 4. The image generating apparatus according to claim 2, wherein the second pixel position and the third pixel position are different from each other when the command instructs a motion compensation process. 5. The pixel position specifying means sets the position of a pixel in a reference image block as the first pixel position when the command signal indicates simultaneous processing of motion compensation processing and texture mapping processing, The position of the pixel in the generated polygon to be texture-mapped is determined by the second
The image generation apparatus according to claim 2, wherein the pixel position of the third pixel is output, and the pixel position in the difference image block is output as the position of the third pixel. 6. The pixel calculating means receives the command signal, and when the command signal indicates a motion compensation process or a simultaneous process of a motion compensation process and a texture mapping process, When the first pixel data and the third pixel data are arithmetically processed as signed precision data, and the command signal indicates texture mapping processing, the first pixel data and the third pixel data are processed. The image generation apparatus according to claim 2 or 5, wherein the pixel data of 1 is subjected to arithmetic processing as data of accuracy without a code. 7. The pixel position specifying means receives the command signal, and when the command signal indicates a motion compensation process, the pixel position specifying means determines the relative position of the first pixel position in the reference image block. , The relative position of the third pixel position in the difference image block, and the second position
The image generation apparatus according to claim 2, wherein a pixel position at which the relative position of the pixel position of the above is the same as the relative position within the generated image block is specified. 8. The pixel position designating means receives the command signal, and when the command signal indicates a texture mapping process, a pixel position of each pixel on a horizontal line in the polygon is generated. The pixel positions are sequentially calculated, the pixel positions are designated as the first pixel position and the second pixel position, and the pixel positions in the texture image corresponding to the first pixel position and the second pixel position are calculated. And the pixel position is specified as the third pixel position,
When the command signal indicates the motion compensation process, the pixel position of each pixel on the horizontal line in the generated image block is sequentially calculated, and the pixel position is designated as the second pixel position, Further, the relative position of the third pixel position in the difference image block and the relative position of the first pixel position in the reference image block are the relative position of the second pixel position in the generated image block. The image generation device according to claim 7, wherein the third pixel position and the first pixel position that are the same are designated. 9. The pixel position designating means receives the command signal, and when the command signal indicates a texture mapping process, the pixel position designating means determines a pixel position of each pixel on a horizontal line in the texture image. The pixel position is sequentially calculated, the pixel position is designated as the third pixel position, the pixel position in the generated polygon corresponding to the third pixel position is obtained, and this pixel position is determined as the first pixel position and the first pixel position. If the pixel signal is designated as the second pixel position and the command signal indicates the motion compensation processing, the pixel position of each pixel on the horizontal line in the difference image block is sequentially calculated, and the pixel position is calculated. The third pixel position is designated, and the relative position of the first pixel position in the reference image block and the relative position of the second pixel position in the generated image block are the third pixel position. The image generating apparatus according to claim 7, wherein the first pixel position and the second pixel position are designated so as to be the same as the relative position of the third pixel position in the difference image block. 10. The pixel position designating means receives the command signal, and when the command signal indicates to simultaneously execute a motion compensation process and a texture mapping process, The pixel position of each pixel on the horizontal line is sequentially calculated, this pixel position is designated as the second pixel position, and the pixel position mapped to the pixel data of the second pixel position is stored in the difference image block. This pixel position is designated as the third pixel position, and the relative position of the third pixel position in the difference image block and the relative position of the first pixel position in the reference image block are The image generation apparatus according to claim 5, wherein the first pixel positions that are the same are designated. 11. The pixel position specifying means receives the command signal, and when the command signal indicates to simultaneously execute a motion compensation process and a texture mapping process, the pixel position specifying means The pixel position of each pixel on the horizontal line is sequentially calculated, the pixel position is designated as the third pixel position, and the pixel position in the generated polygon corresponding to the third pixel position is obtained. The position is designated as the second pixel position, and the third pixel position
Relative position in the difference image block of the pixel position of the
6. The image generation apparatus according to claim 5, wherein the first pixel position is designated such that the relative position of the pixel position in the reference image block is the same. 12. The pixel calculation means receives the command signal, and when the command signal indicates a motion compensation process, the first pixel position from the first pixel position in the reference image block is used. Pixel data is read, the third pixel data is read from the third pixel position in the difference image block, calculation is performed using the first pixel data and the third pixel data, and the calculation result is obtained. There is the second
After performing a block motion compensation process for writing all the pixels in one image block, the procedure of writing the pixel data of 1 to the second pixel position indicating the pixel position in the generated image block, and then performing the pixel position specifying means. 3. An image block setting process is performed in which the generated image block is set as a new difference image block, and an image block located at a position shifted by one pixel from the reference image block is set as a new reference image block to specify a pixel position. Alternatively, the image generation apparatus according to claim 5. 13. The image according to claim 12, wherein the pixel position designating unit performs a pixel block setting process in which an image block displaced by one pixel to the right from the reference image block is set as the new reference image block. Generator. 14. The image according to claim 12, wherein the pixel position designating unit performs a pixel block setting process in which an image block shifted downward by one pixel from the reference image block is used as the new reference image block. Generator. 15. An image block setting process in which the pixel position designating unit sets the generated image block as a new difference image block, and sets an image block at a position shifted by one pixel from the reference image block as a new reference image block. 2
The image generating apparatus according to claim 12, wherein the reference image block shift direction is different for each image block setting process in the image block setting process repeated twice or more and twice or more. 16. The pixel block designating means, when performing the image block setting process two or more times, an image block which is shifted by one pixel to the right from the reference image block is defined as the new reference image block. And a case where an image block shifted by one pixel in the downward direction with respect to the reference image block is used as the new reference image block, and an image shifted by one pixel in the lower right direction with respect to the reference image block. The block may be the new reference image block in some cases.
5. The image generation device according to item 5. 17. After performing a block motion compensation process which is a motion compensation process for one image block, the pixel position specifying means sets the generated image block generated by the block motion compensation process as a texture image, and An image block setting process for newly setting a generated polygon is performed, and the image memory access means reads the first pixel data from the first pixel position in the generated polygon and the third pixel in the texture image. The third pixel data is read from the pixel position, and the pixel calculation means is
A process is performed using the first pixel data and the third pixel data, and the resulting second pixel data is written to the second pixel position in the generated polygon, and these processes are performed. The image generation apparatus according to claim 2, wherein the step S is sequentially performed on the pixel data in the generation polygon. 18. A control means for outputting a command signal for instructing contents to be processed by the image generating device, and a reference image for motion compensation when the command signal is received and the command signal indicates motion compensation processing. The pixel position in the block is obtained and output as a first pixel position signal, the pixel position in the generated image block is obtained and output as a second pixel position signal, and is the difference data between the reference image block and the generated image block. The pixel position in the difference image block is obtained and output as a third pixel position signal, and when the command signal indicates texture mapping processing, the pixel position in the texture image is obtained and the third pixel position signal is obtained. And the pixel position in the generated polygon to which the image data at the pixel position indicated by the third pixel position signal is mapped and written. Position, and outputs as the first pixel position signal and the second pixel position signal, and when the command signal indicates the image block setting process and the image block position, the reference image block and the generated image A pixel position designating unit for newly setting the position of the block, the difference image block, the generated polygon, or the texture image, the first pixel position signal, the second pixel position signal, and the third pixel position signal.
Pixel position signal of the first pixel position signal is input, pixel data is read from the pixel position indicated by the first pixel position signal and is output as a first pixel data signal, and the pixel position is changed from the pixel position indicated by the third pixel position signal. An image memory access unit for reading out data, outputting it as a third pixel data signal, and writing the pixel data input by the second pixel data signal at the pixel position indicated by the second pixel position signal, When the command signal indicates motion compensation processing for the pixel data indicated by the first pixel data signal and the third pixel data signal, the pixel data is treated as signed data in pixel units. When the command signal indicates a texture mapping process, the image data is set as unsigned data. A pixel operation unit for performing an operation instructed by the command signal on a pixel-by-pixel basis and outputting the operation result as the second pixel data signal, wherein the control unit receives a control command for an image generation operation and performs the control. An image generation apparatus which interprets a command and outputs the command signal for instructing the pixel position designating means and the pixel computing means of processing contents. 19. The image memory access means comprises a first
Of the first pixel position signal and the second pixel position signal of
Reading pixel data from the pixel position indicated by the pixel position signal of, and outputting as the first pixel data signal,
A first image memory access unit for writing the pixel data input by the second pixel data signal to a pixel position indicated by the second pixel position signal, and a third pixel position signal are input, A second image memory access unit for reading pixel data from a pixel position indicated by the pixel position signal of No. 3 and outputting it as a third pixel data signal, storing a texture image or a difference image block, The image generation apparatus according to claim 18, further comprising a block image memory that outputs pixel data from a pixel position designated by the second image memory access unit. 20. The image memory access means comprises a first
The pixel position signal, the second pixel position signal, and the fourth pixel position signal are input, pixel data is read from the pixel position indicated by the first pixel position signal, and output as a first pixel data signal, Pixel data is read from the pixel position indicated by the fourth pixel position signal and output as a fourth pixel data signal, which is input to the pixel position indicated by the second pixel position signal by the second pixel data signal. A first image memory access unit for writing the pixel data, the third pixel position signal and the fifth pixel position signal are input, and the pixel data is read out from the pixel position indicated by the third pixel position signal. Second pixel data signal, and writes the pixel data input by the fifth pixel data signal at the pixel position indicated by the fifth pixel position signal.
And a block image memory that stores a texture image or a difference image block and that inputs / outputs pixel data from a pixel position designated by the second image memory access unit. When the command signal is received and the command signal indicates an image block setting process and an image block position, the pixel read position in the difference image block or the texture image is sequentially set as the fourth pixel position signal. Outputting, sequentially outputting pixel writing positions in the block image memory as the fifth pixel position signal, and outputting pixel data input as the fourth pixel data signal as the fifth pixel data signal. Move the difference image block or the texture image to the block image memory by The image generating apparatus according to claim 18 having an image moving means. 21. The image memory access means comprises:
The pixel position signal and the second pixel position signal are input, pixel data is read from the pixel position indicated by the first pixel position signal, and output as the first pixel data signal,
A first image memory access unit for writing the pixel data input by the second pixel data signal at a pixel position indicated by the second pixel position signal, and a third image pixel access signal for inputting a third pixel position signal. Pixel data from the pixel position indicated by the pixel position signal is output as a third pixel data signal, the second pixel position signal is input, and the pixel position in the block image memory is changed by the second pixel position signal. Writing the pixel data input by the second pixel data signal in the case shown to the block image memory;
And a block image memory that stores a texture image or a difference image block and that inputs / outputs pixel data from a pixel position designated by the second image memory access unit. The image generation device according to claim 18. 22. The pixel position specifying means obtains a pixel position in a reference image block for motion compensation as a first pixel position signal when the control means instructs simultaneous execution of motion compensation processing and texture mapping processing. 19. The image generating apparatus according to claim 18, which outputs the pixel position in the generated polygon, outputs the pixel position as a second pixel position signal, and outputs the pixel position in the difference image block as a third pixel position signal. 23. The pixel calculation means, a multiplier generation section for generating a multiplier according to the calculation indicated by the control means,
A multiplication unit that multiplies the pixel data input by the first pixel data signal by the multiplier generated by the multiplier generation unit, the pixel data input by the third pixel data signal, and the calculation result of the multiplication unit. 19. An addition unit that adds and outputs the addition result as a second pixel data signal.
The image generation device according to 1. 24. The multiplier unit for generating the first multiplier and the second multiplier according to the calculation indicated by the control unit, and the pixel data input by the first pixel data signal. The first generated by the multiplier generation unit
A first multiplication unit that multiplies the pixel data input by the third pixel data signal by a second multiplier that is generated by the multiplier generation unit; The image generation apparatus according to claim 18, further comprising: an addition unit configured to add the calculation result of the first multiplication unit and the calculation result of the second multiplication unit and output the addition result as a second pixel data signal. 25. The image generation according to claim 23 or 24, wherein the multiplication unit or the first multiplication unit performs 1 ×, 1/2 ×, 1/4 × and 1/8 × operations. apparatus. 26. The number of pixels of image data that can be stored in the block image memory is the same size as the reference image block, the generated image block, and the difference image block.
The image generation device according to any one of claims 9 to 21. 27. The image generation apparatus according to claim 19, wherein the image data that can be stored in the block image memory is rectangular image data having 16 pixels in each of vertical and horizontal directions. 28. The image generation apparatus according to claim 19, wherein the block image memory is realized at least on the same semiconductor element as the pixel calculation means. 29. A compressed image block, which is a result of performing orthogonal transformation and variable length coding on a difference image block obtained by performing a difference calculation with a reference image block by motion prediction processing, and inputs the compressed image block. To the variable length inverse encoding means, the result of the variable length inverse encoding processing is input, the orthogonal transformation means for performing orthogonal transformation, and the result of the orthogonal transformation means are input. 29. The compensation process is performed, and the texture mapping process is performed using the image data resulting from the motion compensation process.
The image expansion device according to any one of claims 1 to 3, which simultaneously performs expansion processing and texture mapping processing on compressed moving image data. 30. A CPU, a main memory, and
9. The moving picture expansion mapping device according to 9 and a frame memory, which is connected to the moving picture expansion mapping device, stores at least a reference image and a generated image, and expands the compressed moving picture data while expanding the result. A multimedia device that simultaneously performs texture mapping on the surface of a three-dimensional object.