JPH08163560A - Picture information generation method, picture information processing method and recording medium - Google Patents

Abstract

PURPOSE: To provide the high quality picture information of a short data length by orthogonally transforming the picture information of the spatial axis of a macroblock unit, quantizing coefficient data based on the transformation, run length encoding them and generating bit streams. CONSTITUTION: The picture information of the spatial axis of the macroblock unit read from the CD-ROM buffer 83 of an optical disk control part 80 through the CPU 51 of a control system 50 is arithmetically processed at a high speed and orthogonally transformed corresponding to the instruction of the CPU 51 in the geometric transfer engine(GTE) 61 of a graphic system 60. Then, the coefficient data obtained by the transformation are quantized, the quantized coefficient data are run length encoded and the bit streams of the format of 16 bits of a run-level pair are generated and stored in the main memory 53 of the control system. By this method, the high quality picture information of the short data length capable of storing quantization step information in a first run area is generated.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image information generating method for generating image information by information processing, an image information processing method for processing the image information, and a recording medium on which the image information is recorded.

[0002]

2. Description of the Related Art For example, in a home game machine, a personal computer device, a graphic computer device, or the like, a television receiver, a monitor receiver, or a C receiver.
Most of the images output and displayed on the RT display device or the like are two-dimensional. Basically, basically, a two-dimensional character or the like is appropriately arranged on a two-dimensional planar background and moved or changed. The image is displayed in such a form that the image is displayed.

However, in the above-described two-dimensional image display and video, there are restrictions on the background and characters that can be expressed, and their movements, and it is difficult to enhance the sense of presence in a game, for example.

Therefore, for example, a pseudo three-dimensional image or video is created by the following method. That is, as the character, images viewed from several directions are prepared, and one of these images is selected and displayed in accordance with a change in the viewpoint on the display screen, or a two-dimensional image is displayed. There is a method of expressing a pseudo three-dimensional image by superimposing the images in the depth direction.
Also, when generating or creating image data, a texture mapping method that attaches an image of a so-called texture (texture, background pattern) to a desired surface such as a polyhedron, or a so-called color lookup table for the color data of the image is used. A method of changing the display color by converting is adopted.

FIG. 27 shows an example of a schematic structure of a conventional home-use game machine. In FIG. 27, a CPU 391 including a microprocessor or the like takes out operation information of an input device 394 such as an input pad or a joystick via an interface 393 and a main bus 399. Simultaneously with the extraction of this operation information, the data of the three-dimensional image stored in the main memory 392 is transferred to and stored in the source video memory 395 by the video processor 396.

Further, the CPU 391 sends to the video processor 396 the read order of the image data for superimposing and displaying the images stored in the source video memory 395. The video processor 396 reads the image data from the source video memory 395 according to the order of reading the image data, and displays the images in a superimposed manner.

At the same time as displaying the image as described above, the audio processor 397 outputs the audio data corresponding to the displayed image stored in the audio memory 398 according to the audio information in the extracted operation information. To do.

FIG. 28 is a diagram showing a procedure for outputting a three-dimensional image by using the data of the two-dimensional image in the home-use game machine having the configuration shown in FIG. In FIG. 28,
A case where a cylindrical object is displayed as a three-dimensional image on a checkered background image will be described.

In the source video memory 395 of FIG. 28, a checkered background image 200 and this background image 200 are shown.
A rectangular image representing the cross-section of the upper cylindrical object in the depth direction,
Data of so-called sprites 201, 202, 203, 204 are stored. This rectangular image 201, 20
Portions other than the image of the cylindrical cross section on 2, 203, and 204 are drawn in a transparent color.

The sync generator 400 in the video processor 396 generates a read address signal that matches the sync signal of the image to be displayed. Further, the sync generator 400 sends the read address signal to the read address table 401 provided from the CPU 391 of FIG. 27 via the main bus 399. Further, the sync generator 400 reads the image data in the source video memory 395 according to the information from the read address table 401.

The read image data is the CP
Based on the superposition order of the images in the priority table 402 given by the U 391 via the main bus 399, the superposition processing unit 403 sequentially superposes the images. In this case, the background image 200 has the lowest rank, and the rectangular images 201, 202, 203, and 20.
Since the rank is higher in the order of 4, the images are sequentially superposed from the background image 200.

Next, in the transparent color processing unit 404, the images 201, 202, 203, 20 of the cross-sections of the cylinders that have been superimposed are displayed in order to display a portion other than the cylinders as a background image.
A process for making a portion other than the cylinder displayed by 4 transparent is performed.

By the above-described processing, the data of the two-dimensional image of the cylindrical object is output as the image data of the three-dimensional image VD ₀ shown in FIG.

[0014]

In the home game machine, the image data supplied to the main memory 392 is generally supplied from a large-capacity external or internal auxiliary storage device. In recent years, data compression has been proposed in order to store more image data in the auxiliary storage device.

However, in recent years, more realistic and high quality images have been demanded. However, such a high-quality image has a large amount of data, and even if the above compression is performed, the amount of data becomes enormous.

From the above, it is desired that the compressed image data stored in the auxiliary storage device be of a format in which the data length becomes shorter and the quality of the image obtained after decompression is high.

The present invention is directed to an image information generation method for generating image information that requires a short data length and high image quality, an image information processing method for processing the image information, and image information. To provide a recording medium on which is recorded.

[0018]

According to the image information generation method of the present invention, orthogonal transformation is applied to image information on a spatial axis of a predetermined macroblock unit, the obtained coefficient data is quantized, and the quantization is performed. It is characterized in that coefficient data is run-length encoded to generate a bit stream. Here, the quantization step information at the time of the above-mentioned quantization can be stored in the first run area of the macroblock which has been orthogonally transformed and run-length encoded.

Further, according to the image information processing method of the present invention, the orthogonal transformation is performed on the image information of the spatial axis of a predetermined macroblock unit, the obtained coefficient data is quantized, and the quantized coefficient data is run. Run-length decoding is performed on the length-encoded bit stream, the obtained coefficient data is inversely quantized, the inversely quantized coefficient data is inversely orthogonally transformed, and the spatial axis in macroblock units is used. Is generated. Here, the inverse quantization can be performed using the quantization step information stored in the first run area of the macroblock of the bitstream.

Further, the recording medium of the present invention performs orthogonal transform on the image information of the spatial axis of a predetermined macroblock unit, quantizes the obtained coefficient data, and quantizes the quantized coefficient data with a run length code. It is characterized in that the bit stream generated by the conversion is recorded.
Furthermore, it is also possible to store the quantization step information at the time of the above quantization in the first run area of the macroblock that has been subjected to the orthogonal transform and run-length encoded.

[0021]

In the present invention, the image information on the spatial axis of a predetermined macroblock unit is orthogonally transformed, and the coefficient data obtained by quantizing the image information is run-length encoded.
The data length is getting shorter. Further, since the quantization step information at the time of quantization is stored in the first run area of the macroblock that has been orthogonally transformed and run-length encoded, the quantization step can be set for each macroblock.

[0022]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A preferred embodiment of the present invention will be described below with reference to the drawings.

First, prior to the description of the image information generating method of the embodiment of the present invention, an image information processing apparatus to which the image information processing method of the present invention is applied, that is, image data generated by the image information generating method of the present invention is used. An image processing system that generates and displays three-dimensional graphics data will be described. The image processing system of this embodiment is applied to, for example, a home game machine, a personal computer device, a graphic computer device, or the like, and the example of FIG. 1 shows an example applied to a home game device.

The image processing system of the present embodiment is a recording medium of the present invention in which the data of the format of the present invention described later is recorded, for example, an optical disc (for example, CD-R).
Data from OM disc) (eg game program)
By reading and executing, the game is played according to the instruction from the user, and has a configuration as shown in FIG.

That is, the image processing system includes a control system 50 including a central processing unit (CPU 51) and its peripheral devices (peripheral device controller 52, etc.) and a graphics processing unit (GPU 62) for drawing on a frame buffer 63. ) And the like, a sound system 70 including a sound processing unit (SPU 71) that generates a musical sound, a sound effect, and the like, and an optical disk (CD-ROM disk) drive 81 that is an auxiliary storage device. An optical disc control unit 80 for decoding reproduction information,
A communication control unit 90 for controlling input / output of an instruction input from a controller 92 for inputting an instruction from a user and input / output from an auxiliary memory (memory card 93) for storing game settings and the like
And a main bus B to which the control system 50 to the communication control unit 90 are connected.

The control system 50 includes a CPU 51, a peripheral device controller 52 for controlling interrupt control, time control, memory control, direct memory access (DMA) transfer, and the like, and a main storage device (RAM) of, for example, 2 megabytes ( Main memory) 53,
The main memory 53 and the graphic system 6
0, a ROM 54 of, for example, 512 kilobytes in which programs such as a so-called operating system for managing the sound system 70 and the like are stored.

The CPU 51 is, for example, a 32-bit RIS.
C (reduced instruction set computor) CPU,
The entire device is controlled by executing the operating system stored in the ROM 54. The CP
The U51 is equipped with an instruction cache and a scratchpad memory, and also manages the real memory.

The graphic system 60 includes a geometry transfer engine (GTE) 61, which is a co-processor for coordinate calculation for performing processing such as coordinate conversion, and a CPU.
A graphics processing unit (GPU) 62 for drawing in accordance with a drawing instruction from the GPU 51;
For example, a 1-megabyte frame buffer 63 for storing the image drawn by 2 and an image decoder (hereinafter referred to as MDEC) 6 for decoding image data compressed and encoded by orthogonal transform such as so-called discrete cosine transform
4 and.

The GTE 61 has, for example, a parallel operation mechanism for executing a plurality of operations in parallel, and as a coprocessor of the CPU 51, coordinate conversion, light source calculation, for example, fixed point format matrix or vector conversion in response to an operation request from the CPU 51. The operation can be performed at high speed.

More specifically, the GTE 61 can perform coordinate calculation of up to about 1.5 million polygons per second in the case of performing flat shading in which the same color is drawn on one triangular polygon. Therefore, in this image processing system, the CP
It is possible to reduce the load on the U51 and perform high-speed coordinate calculation.

The GPU 62 operates according to a polygon drawing command from the CPU 51, and draws a polygon (polygon) on the frame buffer 63. This GPU62
Is capable of drawing up to about 360,000 polygons per second. In addition, this GPU62
Has a two-dimensional address space independent of the CPU 51, and the frame buffer 63 is mapped therein.

The frame buffer 63 is composed of a so-called dual port RAM, and is capable of simultaneously performing drawing from the GPU 62 or transfer from the main memory 53 and reading for display.

The frame buffer 63 has a capacity of, for example, 1 megabyte, and each has a 16-bit width 102.
4 is treated as a matrix of 512 vertical pixels.

Video output means 65 such as a display device is provided in an arbitrary area of the frame buffer 63.
It can be output to.

In addition to the display area output as video output, the frame buffer 63 also has a GPU 62.
CL that stores a color lookup table (hereinafter referred to as CLUT) that is referred to when the polygon or the like is drawn
A UT area and a texture area for storing a material (texture) to be inserted (mapped) in a polygon or the like that is coordinate-converted at the time of drawing and drawn by the GPU 62 are provided. These CLUT area and texture area are dynamically changed according to the change of the display area and the like. That is, the frame buffer 63 can execute drawing access to the area being displayed, and can also perform high-speed DMA transfer with the main memory 53.

In addition to the above flat shading, the GPU 62 pastes the Gouraud shading for interpolating from the colors of the vertices of the polygon to determine the color within the polygon and the texture stored in the texture area to the polygon. Texture mapping can be performed.

When performing these Gouraud shading or texture mapping, the GTE 61 is
A maximum of about 500,000 polygon coordinate operations can be performed per second.

Under the control of the CPU 51, the MDEC 64 decodes the image data of the still image or the moving image read from the CD-ROM disk and stored in the main memory 53, and stores it again in the main memory 53. Specifically, the MDEC64 can perform an inverse discrete cosine transform (inverse DCT) operation at a high speed, and a color still image compression standard (so-called JPEG) read from a disk or a storage media type moving image coding standard (so-called MPEG,
However, in the present embodiment, it is possible to decompress the compressed data (intra-frame compression only).

By the way, DCT (Discrete Cosine Transform) is used when compressing the image data recorded on the disk. The DCT will be described here.

The DCT is included in a linear transform generally called an orthogonal transform and can be considered as a kind of frequency transform. When DCT processing is performed on a rectangular image of N × N pixels on the spatial axis, low frequency components of the image are concentrated in one place. Data compression is established by Huffman encoding the result. That is, the DCT converts data into a form that can be easily compressed, and does not itself reduce the data size. It is Huffman coding that actually performs data compression. When the DCT transform is performed, since the frequency components of a general image are concentrated in the low frequency range, most of the components after the conversion become 0, which is a much more efficient compression than the direct Huffman coding of the image. realizable. Huffman coding here is VLC (Variable Length Coding)
Call. The byte / word boundary of VLC-processed data is logically meaningless, and is represented by a simple bit sequence. This is called a bitstream. Then, this series of operations are all performed in units of 16 × 16 rectangular areas. This unit is called a macro block.
Therefore, DCT compression can also be said to take a macroblock as input, compress it, and convert it into a bitstream format. Further, the image is quantized in a certain unit after being DCT-transformed. By controlling this quantization step, the compression rate can be controlled.
Generally, the coarser the quantization step, the better the compression rate.

On the other hand, the expansion of the DCT is performed in the reverse procedure of the compression. That is, once the captured bitstream is VLC decoded, it is IDCT (inverse D
CT) to restore the original image. Therefore, the bitstream is expanded by the IDC after the VLC decoding process.
It is composed of two paths of T processing.

The compressed image data recorded on the recording medium (CD-ROM disc) of the present invention is, for example, 24-bit mode full color data (direct color data) as described later. The 24-bit full-color data is once DCT-converted and then converted into intermediate data in a run-length compressed format (run level pair described later). Thereafter, this intermediate data is subjected to VLC to form a bit stream, which is recorded on the recording medium (disk) of the present invention. The compression rate is controlled by designating the quantization step in the process of generating the run level. During compression in this embodiment, the run level which is the intermediate data is not output. More specifically, first, the data of the 16 × 16 macroblock in the 24-bit mode is quantized, and at this time, the compression step is set by designating the quantization step. Next, this quantized data is run-length coded to a run level (intermediate data), and then is made into a bit stream by VLC.

Therefore, when the compressed image data recorded on the recording medium of the present invention is expanded in the MDEC 64, the work opposite to the compression work is performed. That is, the bit stream of the compressed image data reproduced from the disc is VLC-decoded to be the run level. This run level is returned to the DCT coefficient data, which is IDCT. After that, it is further quantized by using the quantization step at the time of compression to obtain 16 ×
Data of 16 macroblocks can be obtained.
It should be noted that the image data to be subjected to DCT in this embodiment is full-color data in 24-bit mode (direct color data), as will be described later, but a bit stream created by compressing this is a 16-bit mode. /
It can be expanded in either 24-bit mode. The mode setting can be selected at the time of deployment. In the case of 16-bit mode data,
It is also possible to select ON / OFF of the first 1 bit (STP bit) described later at the time of expansion.

Since the inverse DCT requires a long processing time, the MDEC 64 performs the processing in parallel with the CPU 51. Therefore, a function for transferring data to the MDEC 64 and a function for receiving the expanded data are prepared.

The MDEC 64 and the CPU 51 operate in parallel while sharing the main memory 53 with each other. The function for transferring data from the CPU 51 to the MDEC 64 sews while the CPU 51 is releasing the bus right, and the run level described later is MDEC in the background.
To 64. Similarly, the function for receiving the expanded data also transfers the expanded macroblock described later to the main memory 53 in the background.

The data expanded by MDEC64 is
It is always transferred to the frame buffer 63 via the main memory 53. At this time, transmission / reception between the MDEC 64 and the main memory 53 can be performed asynchronously. Therefore, when developing an image for one frame (640 × 240), it is possible to develop the image without constructing a buffer for the frame size in the main memory 53. As an example of this, an image can be divided into vertically elongated strip-shaped regions of 16 × 240 (15 macroblocks), and data can be received and transferred for each strip-shaped region.

From the above, in the system of this embodiment, the moving picture can be reproduced by continuously reading the bit stream from the recording medium (disk) of the present invention and reproducing it.

The resolution of the moving image and the number of frames are determined by the expansion speed and the transfer speed from the disk. MD
The expansion speed of EC64 is up to 9000 macroblocks /
Seconds. This corresponds to developing 320 × 240 images at a speed of 30 sheets per second. This expansion speed is, of course, inversely proportional to the image resolution and the number of reproduction frames, regardless of the compression rate. That is, 320 × 24
If the image is 0, 30 frames / second, 640 × 240
If so, 15 frames / second, and if 60 × 480, 7.
A movie of 5 frames / second can be realized.

On the other hand, CD-ROM has a transfer rate of
150 kilobytes / second (standard speed), 300 kilobytes /
You can select either second (double speed).

At the time of double speed reproduction, the bit stream constituting one frame is 10 kilobytes (= 300 kilobytes /
30) If it is compressed below and recorded on a CD-ROM,
-It will be possible to read 30 frames / second of data from the ROM. Therefore, for example, data of 10 frames / second is read at 20 kilobytes and data of 7.5 frames / second is read at 30 kilobytes.

The moving image reproduction rate is determined by these two conditions. For example, during double speed reproduction, one frame (320 × 2)
40) is compressed into a bit stream of 10 kilobytes (= 300 kilobytes / 30), and the CD-ROM
You can record it in. The number of frames, the image resolution, and the compression rate can be arbitrarily selected as long as this range is satisfied.

Returning to FIG. 1, the image data expanded as described above is stored in the frame buffer 63 via the GPU 62 so that it can be used as the background of the image drawn by the GPU 62. It is also becoming.

The sound system 70 includes a CPU 51.
A sound reproduction processor (SPU) 71 for generating a musical sound, a sound effect, etc. based on an instruction from the CD-RO
A sound buffer 72 of, for example, 512 kilobytes in which data such as voices and musical tones read from M and sound source data are stored, and a speaker 73 as a sound output means for outputting musical tones and sound effects generated by the SPU 71.
It has and.

The SPU 71 uses 16-bit audio data as a 4-bit differential signal by adaptive differential encoding (ADP).
CM) ADPCM decoding function for reproducing the voice data, a reproduction function for generating a sound effect by reproducing the sound source data stored in the sound buffer 72, and the voice data stored in the sound buffer 72. It has a modulation function and the like for modulating and reproducing the same. That is, the SPU 71 is an ADPC having functions such as looping and automatic change of operating parameters with time as a coefficient.
It has 24 voices of M sound source built in and operates by an operation from the CPU 51. Also, the SPU 71 is a sound buffer 7.
2 manages its own address space mapped to C,
The ADPCM data is transferred from the PU 51 to the sound buffer 72, and the data is reproduced by directly passing the key-on / key-off and the modulation information.

By having such a function, the sound system 70 can be used as a so-called sampling sound source for generating a musical sound, a sound effect, etc. based on the audio data recorded in the sound buffer 72 according to an instruction from the CPU 51. You can do it.

The optical disc controller 80 is a CD-R.
A disc drive device 81 for reproducing programs, data, etc. recorded on an optical disc which is an OM disc;
For example, a decoder 82 for decoding a program, data, etc. recorded with an error correction (ECC) code added
And a buffer 83 of, for example, 32 kilobytes for storing the read data from the CD-ROM disc by temporarily storing the reproduced data from the disc drive device 81. That is, the optical disc control unit 80 includes the drive device 81 and the decoder 8
It is composed of parts necessary for reading a disc such as a second disc. It should be noted that here, it is possible to support CD-DA and CD-ROM XA as the disc format. The decoder 82 also constitutes part of the sound system 70.

As the audio data recorded on the disc reproduced by the disc drive device 81, the above-mentioned ADPCM data (ADPC of the CD-ROM XA is used.
In addition to M data), there is so-called PCM data obtained by analog / digital converting a voice signal.

As ADPCM data, for example, audio data recorded by expressing the difference of 16-bit digital data by 4 bits is error-corrected and decoded by the decoder 82, and then supplied to the above-mentioned SPU 71, where it is sent to the SPU.
After processing such as digital / analog conversion at 71,
It is used to drive the speaker 73.

Audio data recorded as PCM data, for example, 16-bit digital data, is decoded by the decoder 82 and then used to drive the speaker 73. The audio output of the decoder 82 once enters the SPU 71, is mixed with the SPU output, and becomes the final audio output via the reverb unit.

Further, the communication control unit 90 controls the communication with the CPU 51 via the main bus B, and the controller 9 for inputting an instruction from the user.
2 and a memory card 93 for storing game settings and the like.

The controller 92 is an interface for transmitting the intention of the user to the application, and has, for example, 16 instruction keys for inputting an instruction from the user, and according to the instruction from the communication control device 91, The state of this instruction key is transmitted to the communication control device 91 about 60 times per second by synchronous communication. Then, the communication control device 91 transmits the state of the instruction key of the controller 92 to the CPU 51. The controller 92 has two connectors on the main body, and in addition, it is possible to connect a large number of controllers using a multi-tap.

As a result, an instruction from the user is sent to the CPU 5
1 is input to the CPU 51, and the CPU 51 performs processing according to an instruction from the user based on the game program or the like being executed.

When it is necessary to store the settings of the game being executed, the CPU 51 sends the stored data to the communication control device 91, and the communication control device 91 sends the data from the CPU 51 to the memory card. Store in 93.

Since this memory card 93 is separated from the main bus B, it can be attached and detached while the power is on. As a result, game settings and the like can be stored in the plurality of memory cards 93.

Further, the system of this embodiment has a main bus B.
A 16-bit parallel input / output (I / O) port 101 and an asynchronous serial input / output (I / O) port 102 which are connected to each other.

The parallel I / O port 101 can be connected to peripheral devices, and the serial I / O port 102 can be used to communicate with other video game devices. To be able to do.

By the way, the main memory 53, the GPU
A large amount of image data needs to be transferred at high speed between the 62, the MDEC 64, the decoder 82, and the like when reading a program, displaying an image, drawing, or the like.

Therefore, in this image processing system, the main memory 53, GP is controlled by the control from the peripheral device controller 52 without the intervention of the CPU 51 as described above.
It is possible to perform so-called DMA transfer for directly transferring data between the U62, MDEC 64, the decoder 82 and the like.

This allows the CPU 51 to transfer data.
It is possible to reduce the load and to perform high-speed data transfer.

In this video game device, when the power is turned on, the CPU 51 executes the operating system stored in the ROM 54.

When executed by this operating system, the CPU 51 controls the graphic system 60, the sound system 70 and the like.

When the operating system is executed, the CPU 51 controls the optical disc control unit 80 after initialization of the entire device such as operation confirmation and controls the game etc. recorded on the optical disc. Run the program.

By executing the program such as this game,
The CPU 51 controls the graphic system 60, the sound system 70, etc. according to an input from the user to control the display of images, the generation of sound effects, and the generation of musical sounds.

Next, the display on the display in the image processing system of this embodiment will be described.

The GPU 62 has a frame buffer 63.
The content of the arbitrary rectangular area in 4 is displayed as it is on the display of the video output means 65, such as a CRT. This area is hereinafter referred to as a display area. The relationship between the rectangular area and the display on the display screen is as shown in FIG.

Further, the GPU 62 supports the following 10 screen modes.

[Mode] [Standard resolution] Remarks Mode 0 256 (H) × 240 (V) Non-interlace mode 1 320 (H) × 240 (V) Non-interlace mode 2 512 (H) × 240 (V) Non-interlace mode 3 640 (H) × 240 (V) Non-interlace mode 4 256 (H) × 480 (V) Interless mode 5 320 (H) × 480 (V) Interless mode 6 512 (H) × 480 (V) Interless mode 7 640 (H) × 480 (V) Interless mode 8 384 (H) × 240 (V) Non-interlace mode 9 384 (H) × 480 (V) Interless screen size or display The number of pixels on the screen is variable, and as shown in FIG. 3, the display start position (coordinates (DTX, DTY)) and the display end position (coordinates (DBX, DBY)) can be specified independently in the horizontal and vertical directions. You can

The relationship between the value that can be designated for each coordinate and the screen mode is as follows. The coordinate values DTX and DBX must be set to be a multiple of 4. Therefore, the minimum screen size is 4 pixels wide,
2 pixels vertically (when non-interlaced) or 4 pixels (when interlaced).

[Specifiable Range of X Coordinate] [Mode] [DTX] [DBX] 0,40 to 276 4 to 280 1,5 0 to 348 4 to 352 2,6 0 to 556 4 to 560 3,7 0 to 700 4 to 704 8,9 0 to 396 4 to 400 [Specifiable range of Y coordinate] [Mode] [DTY] [DBY] 0 to 3, 8 0 to 241 2 to 243 4 to 7, 90 0 480 4 to 484 Next, the GPU 62 sets, as a mode relating to the number of display colors,
16-bit direct mode (32768 colors), 24
It supports two bit direct modes (full color). 16-bit direct mode (1 below)
The 6-bit mode) is a 32768 color display mode. In the 16-bit mode, the number of display colors is limited as compared with the 24-bit direct mode (hereinafter referred to as the 24-bit mode), but the color calculation inside the GPU 62 at the time of drawing is performed by 24 bits, and the so-called dither function is also used. Since it is also equipped with, it is possible to display pseudo full color (24-bit color). The 24-bit mode is a mode for displaying 16777216 colors (full color). However, only the image data transferred in the frame buffer 63 can be displayed (bitmap display),
The drawing function of the GPU 62 cannot be executed. Here, the bit length of one pixel is 24 bits, but the values of coordinates and display position on the frame buffer 63 must be specified with 16 bits as a reference. That is, 640
The x480 24-bit image data is handled as 960x480 in the frame buffer 63. Further, it is necessary to set the coordinate value DBX to be a multiple of 8. Therefore, the minimum screen size in this 24-bit mode is 8 horizontal pixels × 2 vertical pixels.

The GPU 62 has the following drawing function.

First, for a sprite of 1 × 1 dot to 256 × 256 dots, a 4-bit CLUT (4-bit mode, 16 colors / sprite) or an 8-bit CLUT (8
Bit mode, 256 colors / sprite), 16-bit C
A sprite drawing function capable of LUT (16-bit mode, 32768 colors / sprite), etc. and drawing is performed by designating the coordinates of each vertex of a polygon (polygon such as triangle or quadrangle) on the screen, and Polygon drawing function that performs flat shading that fills with the same color, Gouraud shading that specifies different colors for each vertex and gradation inside, and texture mapping that prepares and pastes two-dimensional image data (texture pattern) on the polygon surface. And a linear drawing function capable of gradation, and the CPU 51 to the frame buffer 6
3, transfer from the frame buffer 63 to the CPU 51, transfer from the frame buffer 63 to the frame buffer 63
Image transfer function such as transfer to the other, and other functions that take the average of each pixel and make it semi-transparent (called the α blending function because the pixel data of each pixel is mixed at a predetermined ratio α), color There is a dither function that blurs noise on the boundary of, a drawing clipping function that does not display the portion beyond the drawing area, an offset specification function that moves the drawing origin according to the drawing area, and so on.

The coordinate system for drawing is 11 with a sign.
Bit is used as a unit, and X- and Y-1024
It takes a value of +1023. Further, as shown in FIG. 4, the size of the frame buffer 63 in this embodiment is 1024 ×
Since it is 512, the protruding portion is folded back. The origin of the drawing coordinates can be set to the frame buffer 6 by the function of arbitrarily setting the offset value of the coordinate value.
It can be changed freely within 3. Drawing is performed only on an arbitrary rectangular area in the frame buffer 63 by the drawing clipping function.

Further, the GPU 62 has a maximum of 256 × 25.
It supports 6-dot sprites, and the vertical and horizontal values can be set freely.

The image data (sprite pattern) attached to the sprite is arranged in the non-display area of the frame buffer 63, as shown in FIG. The sprite pattern is transferred to the frame buffer 63 prior to executing the drawing command. The sprite pattern is 256
As many as 256 pixels can be placed on the frame buffer 63 as one page as long as the memory allows, and this 256 × 256 area is called a texture page. The location of one texture page is determined by designating the page number as a parameter for designating the position (address) of the texture page called TSB of the drawing command.

The sprite pattern has a 4-bit CLU.
T (4 bit mode), 8 bit CLUT (8 bit mode), 16 bit CLUT (16 bit mode) 3
There are different color modes. In the 4-bit CLUT and 8-bit CLUT color modes, the CLUT is used.

As shown in FIG. 6, the CLUT means that the R, G, and B values of the three primary colors representing the finally displayed color are 16
256 to 256 are arranged on the frame buffer 63. Each R, G, B value is numbered in order from the left on the frame buffer 63, and the sprite pattern represents the color of each pixel by this number. Further, the CLUT can be selected for each sprite, and it is possible to have an independent CLUT for all sprites. Further, in FIG. 6, one entry has the same structure as one pixel in 16-bit mode, and therefore one set of C
LUT is 1 × 16 (in 4-bit mode), 1 × 255
It becomes equal to the image data (in 8-bit mode).
The storage position of the CLUT in the frame buffer 63 is determined by specifying the coordinates of the left end of the CLUT to be used as a parameter for specifying the position (address) of the CLUT called CBA in the drawing command.

Further, the concept of sprite drawing can be schematically shown as shown in FIG. Note that U and V of the drawing command in FIG. 7 are parameters that specify which position on the texture page in horizontal (U) and vertical (V),
X and Y are parameters for designating the position of the drawing area.

Further, the GPU 62 uses a method called frame double buffering as a moving image display method. This frame double buffering means, as shown in FIG. 8, that two rectangular areas are prepared on the frame buffer 63, the other side is displayed while drawing in one area, and 2 is displayed when drawing is completed. The two areas are exchanged for each other. This makes it possible to avoid displaying the rewriting state. The buffer switching operation is performed within the vertical blanking period. Further, in the GPU 62, since the rectangular area to be drawn and the origin of the coordinate system can be freely set, it is possible to realize a plurality of buffers by moving these two.

Next, a specific structure of the MDEC 64 will be described with reference to FIG.

The MDEC 64 is connected to the main bus B by the terminal 170. The compression-coded image data supplied via the main bus B is sent to the command decoder 17 via the input buffer 171 composed of a RAM.
Sent to 2. The command decoder 172 analyzes a command to be described later, and subsequent processing is performed according to the command.

The output data of the command decoder 172 is sent to the run length decoder 173. The run-length decoder 173 performs expansion (decoding) of VLC (variable length code) and conversion into the run-level DCT coefficient, and sends the decoded data to the FIFO memory 174. The data read from the FIFO memory 174 is subjected to inverse quantization processing corresponding to the quantization performed at the time of encoding by the inverse quantizer 175. At this time, the step size for inverse quantization in the inverse quantizer 175 is
It is determined based on an inverse quantization matrix (IQ matrix) table, which will be described later, arranged in the step size generation circuit 191.

The inversely quantized DCT coefficient data from the inverse quantizer 175 is sent to the IDCT circuit 176. The IDCT circuit 176 performs an inverse discrete cosine transform process corresponding to the discrete cosine transform performed at the time of encoding. At this time, the IDCT matrix circuit 192 stores the IDCT matrix table described later for the IDCT in the IDCT circuit 176.

The IDCT circuit 176 outputs image data in RGB mode or RAW mode, which will be described later. The image data in the RGB mode is stored in the FIFO memory 17
8 to the converter 179, where (Y,
The conversion from (Cb, Cr) to (R, G, B) is performed, and then the data is sent to the packing logic 180, where the data of a predetermined pack unit is converted. Also, the above R
AW mode image data can be directly packed by logic 1
Sent to 80. The output of the packing logic 180 is temporarily stored in the output buffer 190, and then the terminal 17
It is output to the main bus B via 0.

Next, the data format of the present invention handled by the image processing system of this embodiment will be described.

The MDEC 64 is a host system (C
The commands written from the PU 51) to the general-purpose port are sequentially executed. The commands are roughly divided into IDCT (inverse discrete cosine transform) command and IQ (inverse quantization) table setting command (IQ mattix cofficients setting-up com
mand), IDCT matrix setting command (IDCT mattix
cofficients setting-up command).

The general-purpose port has 32 inputs and 32 outputs at a time.
Up to word (= 128 bytes) words can be written. Both general read / write from the CPU 51 (I / O transfer) and DMA transfer are possible. In the case of a long command, a part of the command can be transferred by I / O and the rest can be transferred by DMA. D
MA transfer can be performed in units of 32 words.

IDCT command (command code = 01
h) gives a block data string after the command code. The command code is as shown in FIG. In the figure, OFM is an output format. When 00, the decoding result is output in 4-bit RAW mode, when 01, the decoding result is output in 8-bit RAW mode, and when it is 10, the decoding result is 24-bit RGB mode. , And when it is 11, the decoding result is output in 16-bit RGB mode. S
GN is a sign bit, and when it is 0, the output pixel is clipped from 0 to 255 and output without a sign, and when it is 1, the output pixel is clipped from -127 to 127 and output with a sign. STP specifies the most significant bit of a 16-bit RGB pixel. SIZE is the number of data words following the command.

The command code is followed by block data B1 to Bn as shown in FIG.

In the case of the RGB mode in which the size of each block is different for each block, each luminance block (Y0, Y1, Y2, Y which constitutes a macro block as shown in FIG. 12 is used.
3) and the color difference blocks (Cb, Cr) are 0 to 0 in FIG.
The data is transferred to MDEC 64 in the order shown by 5. That is, the n-th input block following the command is judged as n = 6k + 0: Cb block n = 6k + 1: Cr block n = 6k + 2: Y0 block n = 6k + 3: Y1 block n = 6k + 4: Y2 block n = 6k + 5: Y3 block To be done. Note that the luminance block and the color difference block have the same input format. When SGN = 0, unsigned 4 to 8 bits are output, and when SGN = 1, the output is output as signed 4 to 8 bits.

It is also possible to transfer a plurality of blocks successively after the command, and the number of words to be transferred is specified by SIZE. However, it is not possible to specify a transfer size that ends in the middle of a block. Furthermore, in RGB mode,
The number of blocks transferred must be a multiple of 6.

The IQ table setting command (command code = 02h) gives the inverse quantization table data after the command code. You can set 2 types of IQ table,
0 for RAW mode and inverse quantization of Y luminance block
No. 1 table is used for quantization of the Cr / Cb color difference block. The command code is as shown in FIG. Note that TSL in the figure is the IQ matrix table number (0/1).

Table data as shown in FIG. 14 follows the command code of the IQ table setting command. The table size is 16 words in total, and the IQ table coefficients are specified in the zig-zag order. Note that ZQ (x, y) in the figure is a zigzag order dequantization matrix coefficient (unsigned 8 bits).

The IDCT matrix setting command (command code = 03h) is IDCT followed by IDCT.
Gives the coefficients of the matrix. The command code is shown in Figure 15.
As shown in. After this command code,
Coefficient data follows, as shown in FIG. The table size is 32 words in total. Note that C (x, y) in FIG.
Is an IDCT coefficient (signed 16 bits).

Next, the specifications of the input block format to the MDEC 64 will be described.

The input block format is common to each block type (luminance block / color difference block), and ID
The CT coefficients are run-length encoded (run, level) pairs (r
transfer at un-level). In the MDEC 64, this coefficient (run-level) is processed in the order of run length decoding 173 → inverse quantizer 175 → IDCT circuit 176.

One coefficient (run-level) pair has 16 bits and its format is as shown in FIG. Note that RUN in FIG. 17 is the number of zero coefficients preceding the nonzero coefficient (unsigned 6 bits), and LEVEL is the value of the nonzero coefficient (signed 10 bits).

However, a quantization step is included in the RUN field of the first coefficient (DC coefficient) of the block.
That is, it becomes as shown in FIG. In addition, QUANT in FIG. 18 indicates a quantization step (unsigned 6 bits) by L
EVEL indicates a DC coefficient (signed 10 bits). QU
If ANT = 0, then dequantization is not performed. Also,
The coefficient (run-level) pair is, for example, 0xfe00 (QUA
In the case of NT = 3xf and LEVEL = 0x100),
The pair is considered a NOP and is skipped.

Two pairs of coefficient (run-level) are packed and transferred to 32 bits as shown in FIG. RL (n) in the figure is a (run-level) pair. At this time, the number N of coefficient (run-level) pairs per block differs for each block. However, 2 ≦ N ≦ 63. Further, the NOP designated at the beginning of the block is simply skipped. If there is a NOP in the middle of the block, M
The DEC 64 interrupts the decoding of the coefficient (run-level) pair of the block (0 is filled in the unset field) and starts IQ decoding.

Next, the output block formats of MDEC 64 are RAW mode and RGB mode.

In the RAW mode, there is a one-to-one correspondence between input blocks and output blocks. In the RGB mode, the six blocks following the command are regarded as one macroblock, and a 16 × 16 block is reconstructed and output. This block is called an RGB block.
Therefore, in the RGB mode, when 6 blocks (= 1 macro block) are input, 1 RGB block is output. Therefore, the number of input blocks and the number of output blocks are different in the RGB mode.

Next, the RAW mode includes an 8-bit RAW mode and a 4-bit RAW mode. In the 8-bit RAW mode, the IDCT decoding result that does not pass RGB conversion is output in block units. This mode has the configuration of FIG. Note that F (x, y) in the figure indicates data (8 bits) decoded by IDCT. SG of IDCT command
When N is 0, the decoding result is clipped so as to satisfy 0 ≦ F (x, y) ≦ 255, and is output as unsigned 8 bits.
When SGN is 1, the decoding result is -127≤F (x, y).
Clipped so that ≦ 127, and output with a signed 8 bits.

On the other hand, in the 4-bit RAW mode, the pixel value of 5 output in the 8-bit RAW mode is 5
The bits are rounded and the upper 4 bits are packed and output. This mode has the configuration of FIG. Note that F _xy in the figure indicates data (upper 4 bits) decoded by IDCT.

The RGB modes include 24-bit RGB mode, 16-bit RGB mode and 8-bit RG.
There is a B mode, and in the 24-bit RGB mode, the decoding result is converted into RGB, packed into 24 bits, and output. The format of the decode block is as shown in FIGS. 22 and 23. FIG. 22 shows the I / O mode format during I / O transfer in this mode.
FIG. 23 shows the format of the DMA mode during DMA transfer. As described above, the order of pixels is different between the I / O transfer and the DMA transfer. Note that FIG.
2, R (x, y), G (x, y), B (x, in FIG.
y) is the RGB luminance value (8 bits × 3). Further, when the SNG of the IDCT command is 0, the decoding result is clipped so that 0 ≦ [RGB] (x, y) ≦ 255, and is output as an unsigned 8-bit. When SGN is 1, the decoding result is -127≤ [RGB] (x, y) ≤ + 127.
It is clipped so as to be output as 8-bit signed code.

On the other hand, in the 16-bit RGB mode, as shown in FIG.
As shown in FIG. 4 and FIG. 25, the 6th bit of the pixel value output in the 8-bit RGB mode is rounded off to obtain the upper 5
Bits are packed into 16 bits and output. The order of pixels is different between I / O transfer and DMA transfer. FIG. 24 shows the format of the I / O mode during I / O transfer in this mode, and FIG. 25 shows the format of the DMA mode during DMA transfer. Note that FIG.
4, PIX (x, y) in FIG. 25 indicates a 16-bit pixel.

The 16-bit pixel format is as shown in FIG. The first 1 bit is filled with the value of the STP field of the IDCT command. In addition, RGB in the drawing indicates each luminance value (5 bits).

As described above, in the present embodiment, the input block (coefficient) of the inverse orthogonal transform is in the run-length encoded form, and the inverse quantization process performed before the inverse orthogonal transform of the image block. The coefficient can be set at the beginning of the block, and the block after the inverse orthogonal transform can be used in addition to the normal video signal (Y, Cb, Cr) and CSC (Colo).
r Space Converter), it is also possible to select the output as R, G, B brightness values.
When output as G and B luminance values, the color resolution thereof must be a format that can be selected for each block at the time of decoding, and the code of the output block can be selected with or without a code in block units. It is a feature.

That is, in this embodiment, since the input block (coefficient) of the inverse orthogonal transform is run-length coded, the data length can be short. Moreover, since the coefficient of the inverse quantization process performed before the orthogonal transformation of the image block can be set at the beginning of the block, the quantization step can be set finely for each block. Furthermore, the block after the inverse orthogonal transform is CS in addition to the normal video signal (Y, Cb, Cr).
R, G, B by passing through C (Color Space Converter)
Since it can be selected to output as a brightness value, it can be used as it is as a background screen or a texture source on the frame buffer. Further, when the R, G, and B brightness values are output, the color resolution can be selected for each block at the time of decoding, so that the pixel resolution can be selected according to the pixel resolution of the drawing device, the quality of the requested image, and the frame buffer size. . Further, since the code of the output block can be selected with or without a code in block units, it is possible to handle not only a normal image but also a differential image used in motion compensation using frame correlation. .

[0118]

As is apparent from the above description, in the present invention, the image information on the spatial axis of a predetermined macroblock unit is orthogonally transformed, and the quantized coefficient data is run-length encoded. Therefore, the data length is short. Further, since the quantization step information at the time of quantization is stored in the first run area of the macroblock that has been orthogonally transformed and run-length encoded, the quantization step can be set for each macroblock. That is,
According to the present invention, the data length can be short, and the image quality can be high.

[Brief description of drawings]

FIG. 1 is a block circuit diagram showing a schematic configuration of an image information processing apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram for explaining display on a display.

FIG. 3 is a diagram for explaining display settings on a display.

FIG. 4 is a diagram for explaining a function of drawing clipping.

FIG. 5 is a diagram illustrating a texture page.

FIG. 6 is a diagram for explaining a CLUT structure.

FIG. 7 is a diagram for explaining the concept of sprite drawing.

FIG. 8 is a diagram for explaining frame double buffering.

FIG. 9 is a block circuit diagram for explaining a specific configuration of MDEC.

FIG. 10 is a diagram showing a structure of an IDCT command.

FIG. 11 is a diagram showing a command code of an IDCT command and macro block data following it.

FIG. 12 is a diagram showing a configuration of a macro block.

FIG. 13 is a diagram showing a configuration of an IQ table setting command.

FIG. 14 is a diagram showing table data following a command code of an IQ table setting command.

FIG. 15 is a diagram showing the structure of an IDCT matrix setting command.

FIG. 16 is a diagram showing a command code of an IDCT matrix setting command and coefficient data following it.

FIG. 17 is a diagram showing an input block format.

FIG. 18 is a diagram showing a state in which a quantization step is included in a run field of the first coefficient in the input block format.

FIG. 19 is a diagram showing a pack of a run level pair.

FIG. 20 is a diagram showing data decoded by IDCT.

FIG. 21 is a diagram used to describe a 4-bit RAW mode.

FIG. 22 is a diagram showing each pixel data in the I / O mode of the 24-bit RGB mode.

FIG. 23 is a diagram showing each pixel data in the DMA mode of the 24-bit RGB mode.

FIG. 24 is a diagram showing each pixel data in the 16-bit RGB mode I / O mode.

FIG. 25 is a diagram showing each pixel data in a 16-bit RGB mode DMA mode.

FIG. 26 is a diagram showing a format of 16-bit pixels in 16-bit RGB mode.

FIG. 27 is a block circuit diagram showing a configuration example of a conventional image creating device (home game machine).

FIG. 28 is a diagram used to describe an image creating method by a conventional image creating apparatus.

[Explanation of symbols]

 51 CPU 52 Peripheral Device Controller 53 Main Memory 54 ROM 60 Graphic System 61 Geometry Transfer Engine (GTE) 62 Graphics Processing Unit 63 Frame Buffer 64 Image Decoder (MDEC) 65 Video Output Means (Display Device) 70 Sound System 71 Sound Processing Unit (SPU) 72 Sound buffer 73 Speaker 80 Optical disk controller 81 Disk drive device 82 Decoder 83 Buffer 90 Communication controller 91 Communication controller 92 Controller 93 Memory card 101 Parallel I / O port 102 Serial I / O port

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification number Office reference number FI Technical indication location H03M 7/46 9382-5K H04N 1/41 B 5/92 H04N 5/92 H

Claims (6)

[Claims] 1. The orthogonal transformation is performed on the image information on the spatial axis of a predetermined macroblock unit, the obtained coefficient data is quantized, and the quantized coefficient data is run-length encoded to generate a bitstream. An image information generation method characterized by the above. 2. The image information generating method according to claim 1, wherein the quantization step information at the time of the quantization is stored in the first run area of the macroblock subjected to the orthogonal transform and run-length encoded. . 3. The orthogonal transformation is applied to the image information of the spatial axis of a predetermined macroblock unit, the obtained coefficient data is quantized, and the quantized coefficient data is run-length encoded with respect to the bit stream. Run-length decoding is performed, the obtained coefficient data is inversely quantized, and the inversely quantized coefficient data is inversely orthogonally transformed to generate image information on the spatial axis in macroblock units. Image processing method. 4. The image information processing method according to claim 3, wherein the inverse quantization is performed using the quantization step information stored in the first run area of the macroblock of the bitstream. 5. A bitstream generated by orthogonally transforming image information on a spatial axis of a predetermined macroblock unit, quantizing the obtained coefficient data, and run-length coding the quantized coefficient data. A recording medium on which is recorded. 6. The recording medium according to claim 5, wherein quantization step information at the time of the quantization is stored in a first run area of the macroblock which has been orthogonally transformed and run-length encoded. .