CA2105840A1 - Image processing apparatus - Google Patents
Image processing apparatusInfo
- Publication number
- CA2105840A1 CA2105840A1 CA 2105840 CA2105840A CA2105840A1 CA 2105840 A1 CA2105840 A1 CA 2105840A1 CA 2105840 CA2105840 CA 2105840 CA 2105840 A CA2105840 A CA 2105840A CA 2105840 A1 CA2105840 A1 CA 2105840A1
- Authority
- CA
- Canada
- Prior art keywords
- data
- image
- image data
- block
- color
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0007—Image acquisition
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
ABSTRACT OF THE DISCLOSURE
Image data composed of plural blocks are compressed in accordance with the coefficient of a Q-table. The coefficient may be changed for each block using a Q-factor so that display quality of the image may be changed block-by-block easily.
Image data composed of plural blocks are compressed in accordance with the coefficient of a Q-table. The coefficient may be changed for each block using a Q-factor so that display quality of the image may be changed block-by-block easily.
Description
21~g~
IMAGl~ PROCESSING APPARAl~JS
BA(~RGROUND OP TEI~ INV~TION
The present invention relates to an image processing apparatus, and more particularly to a computer graphic apparatus in which image data are compressed for effective transmission.
In an image proces~ing apparatus, an image ~uch as a natural picture taken by a video camera, an image scanner or the like, is digitalized to be processed by a computer. Such a process i~ called "Digitizing" or "Quantizing".
In order to realize the quantizing process, analog image data are sampled at predetermined intervals. For example, analog image data are sampled for each time "dt", where the image data are indicated by a function "f(t)", as ~hown in ~ig.
1. The quantizing process i8 performed using the sampled data, that is, the sampled data are integrated for each time d(t), f(t) / t. In this process, integrated values are approximated to be even numbers by rounding or the like.
The analog data must be sampled and quantized at as 2~ short intervals a~ possible in order to realize a high quality di~play. However, the memory capacity and processing time of computer~ are limited, and as a result, the sampling interval is limited as well.
Ater that, when the quantized data (digital data) are reproduced, the data are transformed into analog data, that is, ~nvers~ quantizing processing is performed.
2i~8~
Under a condition that the function f(t) varies on a cycle ~T~, cycle ~T~ may be indicated by ~IN~ when the analog image data are sampled at N points in the period of "0~ to "N-1"
in the function f(t).
The function f(t) may be tran~formed by Fourier transform, where "j~' and ~E~' represent the square root of and the sum total of "k = O to N~ re~pectively.
f(n) = E F (k/N) exp (; 2~n k/N) F(k/N) = ( l/N) E f (n) exp (-; 2~n k/N) The above formulas 4how that the "n" numbers of "f n are given as the sum of amplitude~ of waves having a variety of frequencies. In these formula~, "exp" may be indicated a~
follows by ~uler method, and therefore, "f(n)" and "P(k/N)" may be indicated using "cos" and "sin".
exp (~x) = c08(x) + ~ sin(x) exp (-~x) = c08(x) - ~ sin(x) It is a DCT (Discrete Cosine Tran~form) to indicate "P(k/N)" by a co~ine function, and the inverse i~ an IDCT
(Inverse DCT). That is, "F(k/N) " and the original image signal ~f(n)" may be given by transforming "f(n)" according to DCT and IDCT, re~pectively.
(n) = E P (k/N) cos (2~n k/N) P(k/N) = (1/N) E f (n) cos (-; 2~n k/N) DCT and IDCT are especially effective for encoding operation~ of image data.
In computer image processing, "f" and "F" must be 21~84~
indicated in a secondary dimension as follows:
F(u,v) = {2C(u)C(v)/N} ~f(x,y)cos{~u(2x+1)/2N} co8{nv(2y+1)/2N}
F(x,y) = ~ S C(U)C(v)F(u,v)cos{~u(2x+1)/2N} cos{~v(2y+1)J2N}
C(N) = 1 / 8qrt(N) for N = 0 C(N) = 1 for N ~ 0 'Isqrt(N)ll : square root of IIN
u, V, X and Y = 0 to N-l llg(x, y)11 represents concentration information of the image, and is developed by using a DCT coefficient F(u, v) when it is transformed by DCT. The DCT coefficient F(u, v~ is the function of a space frequency llu, vll.
A (i, ~) = (1 / 8) C (i) cos {~i (2~ + 1) /N}
C (i) = 1 / 8 for N = 0 5 C (i) z 1 for N ~ 0 : square root of ~N"
In this case, "F" and "f" components are given by the following matrix arithmetic operations.
tF (u, v)] = [A (u, x)] [f(x, y)] [At (y~ v)]
[P (x, y)] = [At (x, u)] [F(u, v)] [A (v, y)]
[At] : replace matrix of [A]
u, v, x and y = 0 to N-l "Fl' for DCT, and "A" and "C" for IDCT are, hereinafter, called a DC coefficient and an AC coefficient, respectively.
A period for the matrix arithmetic operation is 2~
increased in proportion to the square of dimension N~. In game computers, the arithmetic operations are carried out for each block divided from one image. In general, a game computer uses a screen of 256 x 240 dots, which is divided into blocks of 16 x 16 dots, as shown in Fig. 2.
In such a game computer, RGB and YCrCb systems are used to display colors. According to the YCrCb system, each color may be treated independently, and the system is used in game computers. In the YCrCb system, each color is defined by a brightness (Y) and a color difference (U and V).
Accordlng to the YCrCb sy~tem, each component needR 8 bits to display a natural picture having 16,700,000 colors Therefore, in the screen of 256 x 240 dots, 180k bytes (256 x 240 x 3) capacity is required to display the natural picture.
Generally, in natural pictures, successive dots are not very different in color from each other, and therefore the next dots may be distinguished in color by adjusting the brightness thereof only. In such natural pictures, the brightnes~ i8 sampled for each dot, and the color difference is sampled at one dot interval, as shown in Fig. 3. When color data (Yl, Ul, Vl) and (Y2, U2, V2) are arranged at next each other, "U2~ and "V2" are not sampled, whereby (Yl, Ul, Vl) and (Y2, Ul, Vl) only are reproduced.
In the case as in Fig. 3, each block is defined by 8 x 8 dots, ~o that "N" become~ 8, and therefore, DCT and IDCT
formula~ are g~ven a~ follows:
8 ~ ~
F(u~v) = (C(u)C(v)/4} ~Ef (x,y)cos{~u(2x+1)/16} cos{~v(2y+1~/16}
F(x,y) = ~ ~ (C(u)C(v)/4F(u,v)cos{~u(2x+1)/16} cos{~v(2y+1)/16}
u, v, x and y = 0 to 7 In these formulas, the AC coefficient is not different from that in the formulas described before; however, a range of (u~ v, x and y) is 0 to 7, and as a result, the arithmetic time is shortened remarkably.
Image data which have been transformed by DCT are quantized in linear fashion by a Mid-treat type quantizing device (odd level quantizing device). "Y" and "C (U, V) n components are quantized independently, and 64 (8 x 8) DCT
coefficients are also quantized (divided) by their own quantizat$on steps, respectively. Each quantized coefficient is defined by 9 bits.
In order to decode the image data, the DCT
coefficients are reproduced by multiplying by the quantization ~teps. The quantizing process has an important effect on both of tne quality and compression rate of the decoded image, and the DCT coefficients are most important parameters for the encoding. These parameters are stored in quantizing matrix table~, which are called "Q-tables". The contents of the Q-table~ are different from each other depending on the display quality to be ~elected for the image.
~g. 4 shows a conventional system for encoding and decoding i~age data. The sy~tem includes a plurality of Q-21~8~
tables, and the data stored in the Q-tables are supplied to a decoder when a quantizing level is changed. Each Q-table has its own number to be used for being selected by the decoder.
The contents of the selected Q-tables are transmitted with the table numbers to the decoder.
According to the conventional system, however, there are disadvantages in that the system needs many Q-tables to realize a variety of display quality of image, and therefore, a large area in a memory is occupied for the Q-tables. Further, the ~ystem needs a buffer having a large capacity for buffering the Q-tables. The compression rate of the image data iB
remarkably changed in accordance with the size~ of the Q-tables.
In Fig. 4, "encoding" repre~ents a process for compressing an image data, that is, binary data of the image are transformed to the number of "1~ and "0".
For example, the following binary data, "00000000001111111111~00000111111110000000011100"
are transformed to the following number data, ~'10 11 5 8 8 3 2"
The number data are again transformed in the computer as follows.
"1010 1011 101 1000 1000 11 10"
In these data, a predetermined mark iB inserted in the spaces as ollows:
"1010*1011*101*1000*1000*11*10~
In general, a Huffman code iB used as the insert mark, 21~8~
and the codes are determined in the order of the highest frequency, that is, a numeral that is the most frequently used has the shortest code. As a result, the compression rate of the image is increased when the Yame binary numerals are arranged many times continuously.
In the conventional system, the encoding and decoding operations are not required when all the image blockg have the same color. When a moving picture is treated, data at a region where plural frames have the same information thereat is encoded in order to usad the code data in common. However, all regions in the same picture are not encoded. The conventional system is not suitable for treating still picture~.
8UHNARY OP TH~ INV~NTION
It is an ob~ect of the present invention to provide a high performance image processing apparatus in which the quality of an image to be displayed may be changed using a gingle Q-table.
It is another ob~ect of the invention to provide a high performance image proce~sing apparatus in which image data, e~peeially image data having square blocks of the same color, may be compre~ed at a high rate.
According to a first feature of the invention, in an ~mage proces~$ng apparatus, image data are compre~sed in accordance w$th a coeff~cient of a quantizing matrix table. The co~ic$ent $~ changed $n accordance with a predetermined 58~
factor.
According to a second feature of the invention, in an image processing apparatus, the image data are encoded using a null block code and null block length data when the image data include plural blocks having the same color, the blocks being arranged in succession continuously. The null block code and null block length represent the color and the number of the continuously arranged blocks, respectively.
Specifically, when an image data transmission is initialized, or when the content of the current Q-table is changed, the following Q-table is transmitted to an arithmetic circuit. The Q-factor is transmitted with the image blocks to the arithmetic circuit 80 that the display quality o the image may be changed for each block.
On the other hand, in a decoding process, when the following Q-table is transmitted to the arithmetic circuit, the Q-table is changed to the current Q-table. When the Q-factor is tran~mitted to the arithmetic circuit, the content of the current Q-table is changed in accordance with the Q-factor.
BRIPP D~SCRIPTION OP TH~ DRAWINGS
Pig. 1 is a graph used for explaining quantizing and sampling operations of image data.
Pig. 2 is a diagram ~howing the structure of a screen.
Pig 3 i~ a diagram showing the sampling operation.
~lg. 4 i8 a block diagram showing the operation o a 21~8~
conventional image processing apparatus.
Fig. 5 is a block diagram showing the operation of an image processing apparatu~ according to the invention.
Fig. 6 is a diagram showing the operation for sampling image data in accordance with the invention.
Fig. 7 is a diagram showing the operation for defining colors of the image data in accordance with the invention.
Fig. 8 is a diagram showing an example of an image to be displayed in accordance with the invention.
Fig. 9 is a block diagram showing a computer system of a preferred embodiment according to the invention.
Fig. 10 is a block diagram showing a control unit ~hown in Fig. 9.
Fig. 11 is a block diagram showing an image data extension unit shown in Fig. 9.
Pig. 12 is a diagram showing the operation for deining colors of the image according to the preferred embodiment.
Fig. 13 is a diagram showing the format of the image data compressed in accordance with the preferred embodiment.
Fig. 14 is a diagram showing the format of data including IDCT data according to the preferred embodiment.
Pig. 15 is a diagram showing the configuration of th~
IDCT data according to t~e preferred embodiment.
~5 Pig. 16 is a diagram showing the configuration of a part of the IDCT data according to the preferred embodiment.
210~8~0 Fig. 17 is a diagram showing color data for one image block in accordance with the preferred embodiment.
Fig. 18 i8 a diagram showing examples o~ a Q-table according to the preferred embodiment.
5Pig. 19 i8 the format of image data compressed by using a null block according to the preferred embodiment.
Fig. 20 is the format of image data according to the conventional system.
D~TAILED DFSCRIPTION OF THP INVENTION
Before describing a preferred embodiment of the invention, the fundamental theory of the invention is fir~t explained in con~unction with Pigs. 5 to 8.
Fig. 5 shows the process of transmission of image data in accordance with the invention In the invention, only one Q-table is used for each transmission. Quantizing and inver~e quantizing operations are carried out in accordance with a Q-factor and the Q-table. The Q-factor is established for each image block whereby the display quality of the image may be changed block-by-block. When the current Q-table is not suitable for the image, the Q-table is renewed by another Q-table. In this embodiment, a YCrCb system is used to display color~ of the image. According to the YCrCb ~y~tem, each color i~ defined by a brightness (Y) and a color difference (U and V).
In natural picture~, ~ucces~ive dots are not very 21~38~0 different in color from each other, 80 that the next dots may be distinguished by ad~usting the brightness thereof. For that reason, the brightness is sampled for each dot, and the color difference i~ sampled at one dot interval, a~ shown in Fig. 6 When the color component~ Y and C (U, V) are indicated by Y1~ and C1~ (Ui~, V1~), picture elements G1~ are indicated by "Yi~
+ U1~ + V1~", in which "+'1 represents synthesizing.
The Cmn components meet the following condition, because the Cmn components, "m" or "n~' being an odd number, are not sampled.
Cmn ~ C~n l (m : 0 or even number, n : odd number) = Cm l~ (m : odd number, n : 0 or even number) = Cmlnl ( both "m" and "n" : odd numbers) In this case, each of C10, C01 and Cll i9 equivalent to C00.
When a screen is divided by image blocks each defined by rectangular regions of 2k x 21 dots, as shown in Fig. 7, the ~ data need 2k x 21 dots and each of U and V data need k x 1 dot~.
In order to display a natural picture clearly, 16M
color~ must be u~ed in the system. In a natural picture, shown in Pig. 8, the ~ky varies in the color blue to light blue by the effect of white ground covered with snow. In ~uch a picture, the same color is continued substantially throughout, that is, the ~ame block~ are arranged in succession continuously. These block~ ar~ called "null block~". In this example, the image 21~8~0 data are indicated by ~null block code~ + ~null block length~, where ~+~ doe~ not represent summing. The null block length represents the number of the null blocks, and is also called a run-length. Such blocks indicated by the null block system are directly decoded without calculation using a Q-table. A number ~Z~ of dots to be decoded is given by the following formula.
Z = (the number of dots in the block) x (the number of null blocks) = 2k x 21 x (the number of null blocks) Next, a preferred embodiment according to the present invention will be explained in con~unction with Figs. 9 to 20.
Fig. 9 shows an image processing apparatu~ of the preferred embodiment. The apparatus includes a game-software 15recording medium 100 such as a CD-ROM, a CPU 102 of the 32-bit type, a control unit 104 for mainly controlling transmi~sion of image and sound data and interfacing most devices to each other, an image data extension unit 106, an image data output unit, a sound data output unit 110, a video encoder unit 112, a VDP unit 20114 and a TV display 116. CPU 102, control unit 104, image data exten~ion unit 106 and VDP unit 114 are provided with their own memories K-RAM, M-RAM, R-RAM and V-RAM, respectively.
CPU 102 directly control~ a DRAM via a memory support, and perorm~ communication through an I/O port to peripheral device~, that i~ called an I/O conkrol function. CPU 102 ~nclude~ a t~mer, a parallel I/O port and an interruption 2~ ~8~0 control sy~tem. VDP unit 114 reads display data which have been written in the VRAM by CPU 102. The diRplay data are transmitted to video encoder unit 112 whereby the data are displayed on the TV display 116. VDP unit 114 ha~ at most two screens each composed of background and sprite images, which are of an external block sequence type of 8 x 8 blocks.
Fig. 10 shows control unit 104. Control unit 104 includes an SCSI controller to which image and sound data are supplied through an SCSI interface from CD-ROM 100. Data supplied to the SCSI controller are buffered in the K-RAM.
Control unit 104 also includes a DRAM controller for reading data which have been buffered in the K-RAM at a predetermined timing In control unit 104, priority ~udyement is carried out for each dot of natural background image data, and an output signal i8 transmitted to video encoder unit 112.
Control unit 104 transmits moving image data (full color, pallet), which have been compressed, to image data exten~ion unit 106 whereby the compressed data are extended.
Pig. 11 ~hows image data extension unit 106. The image data extension unit includes an inverse DCT converter, an inver~e ~uantizing means, a Huffman coding and decoding means and a run-length coding and decoding means. That is, the image data exten~ion unit 106 performs a DCT transformation for a natural moving picture, and treats compressed data encoded by the Huffmsn coding method and run-length compressed data for a moYing animation ~mage and the like.
21~8~
Video encoder unit 112 superimposes VDP image data, natural background image data and moving image data (full color, pallet) transmitted from VDP unit 114, control unit 104 and image data extension unit 108. Video encoder unit 112 performs color pallet reproducing, special effect processing, D/A
converting and the like. Output data of video encoder unit 112 are encoded into a YCrCb signal by an external circuit.
ADPCM sound data recorded in CD-ROM 100 are buffered in the K-RAN and then transmitted to sound data output unit 110 by control unit 104. The sound data are reproduced by sound data output unit 110.
Fig. 12 shows an image data format according to the preferred embodiment. In this embodiment, each block is defined by 16 x 16 dots, and the image data are transmitted for 16 rasters.
Fig. 13 shows the configuration of compressed data to be transmitted. In this configuration, "A," B," C" and ~D~
represent the type of the image data, the first and la~t halves byte~ of the data length of a compre~ed data region, and a region storing data for forming boundaries for each two byte~ in the c~mpressed data, respectively. When image data with pallet information ~uch as an animation image are transmitted to the ~y~tem, region "A" is set at an F3H, F2H, FlH or FOH, the~e data being ind~ca~ed by hexadecimal numbers.
Pig. 14 show~ the coniguration of compressed data to be tran~mitt~d when the Q-table is initialized or the current Q-table is changed. The Q-table is located at the front of the IDCT data.
Fig. 15 shows the configuration of the IDCT data. In this configuration, "QCx" represents the coefficient of an inverse Q-table, and "NRL" represents that the following data is the null block.
Fig. 16 shows the arrangement of a "BLOCK SET" in the IDCT data. In this arrangement, each of "ZRL" and "EOB" i~ code data representing that 16 zero data are continued therein and that all data are zero data thereafter, respectively.
Fig. 17 shows the structure of a block forming a part of the image. The block is defined by the brightness Yl, Y2, Y3 and Y4 and the color difference U1 and Vl. The block is transmitted with a Q-factor (QCx) at the front thereof. In lS accordance with the Q-factor and Q-table, the following formulas are given.
Qy' (i,j) z QCx * Qy (i,j) / 4 i,j : 0 to 7 Qc' (i,~) = QCx * Qc (i,j) / 4 i,j : 0 to 7 i = j ~ O
Qc' (i,~) = Qc (i,~) / 4 i = j = 0 Qy' : Q-table for Y for quantizing / inverse-quantizing Qc' : Q-table for C for quantizing / inverse-quantizing Qy : down loaded Q-table for Y
Qc : down roaded Q-table for C
QCx : constant Q-factor, QCx = x (ex. QC7 - 7) 2~ ~84~
~ Down load~ means to transfer data from an external memory to the RAM.
Fig. 18 shows Q-tables to be down loaded.
Referring Fig. 13 again, ~A~ is ~et at FFH or F8H when a natural picture i8 transmitted. When IDCT inverse quantizing table data or compressed image data with IDCT inverse quantizing data are transmitted, ~A~ is set at FFH. When compressed image data for IDCT are transmitted, ~A~ is set at F8H.
When 16 image blocks of blue are included in 16 raster~ in the picture shown in Fig. 8, the null block process is available. That is, each null block code has a null block length (null block run-length) of 16.
Fig. 19 shows the arrangement of compressed null block data of 19 bits. The null block data are composed of a null block code region of 7 bits and a null block length region of 12 bit~. Therefore, "A" is ~et at F8H in the compressed image data format, and data of "(null block code) + (16)" are set in the null block region, shown in Fig. 19.
On the other hand, if the image data are transmitted ~n accordance with the conventional method, color data must be transmitted for each dot, as ~hown in Fig. 20. That is, data of ~Yl + Y2 + Y3 + Y4 + Ul + Vl" must be set for each dot. In Fig.
20, n BOB" repre~ent~ the end of a block.
A~ de~cribed above, according to the invention, the natural image, a~ ~hown in Fig. 8, may be transmitted using 19 bit~ o data only. On the other hand, the conventional system 2 1 ~
needs 1184 bits of data to transmit it.
IMAGl~ PROCESSING APPARAl~JS
BA(~RGROUND OP TEI~ INV~TION
The present invention relates to an image processing apparatus, and more particularly to a computer graphic apparatus in which image data are compressed for effective transmission.
In an image proces~ing apparatus, an image ~uch as a natural picture taken by a video camera, an image scanner or the like, is digitalized to be processed by a computer. Such a process i~ called "Digitizing" or "Quantizing".
In order to realize the quantizing process, analog image data are sampled at predetermined intervals. For example, analog image data are sampled for each time "dt", where the image data are indicated by a function "f(t)", as ~hown in ~ig.
1. The quantizing process i8 performed using the sampled data, that is, the sampled data are integrated for each time d(t), f(t) / t. In this process, integrated values are approximated to be even numbers by rounding or the like.
The analog data must be sampled and quantized at as 2~ short intervals a~ possible in order to realize a high quality di~play. However, the memory capacity and processing time of computer~ are limited, and as a result, the sampling interval is limited as well.
Ater that, when the quantized data (digital data) are reproduced, the data are transformed into analog data, that is, ~nvers~ quantizing processing is performed.
2i~8~
Under a condition that the function f(t) varies on a cycle ~T~, cycle ~T~ may be indicated by ~IN~ when the analog image data are sampled at N points in the period of "0~ to "N-1"
in the function f(t).
The function f(t) may be tran~formed by Fourier transform, where "j~' and ~E~' represent the square root of and the sum total of "k = O to N~ re~pectively.
f(n) = E F (k/N) exp (; 2~n k/N) F(k/N) = ( l/N) E f (n) exp (-; 2~n k/N) The above formulas 4how that the "n" numbers of "f n are given as the sum of amplitude~ of waves having a variety of frequencies. In these formula~, "exp" may be indicated a~
follows by ~uler method, and therefore, "f(n)" and "P(k/N)" may be indicated using "cos" and "sin".
exp (~x) = c08(x) + ~ sin(x) exp (-~x) = c08(x) - ~ sin(x) It is a DCT (Discrete Cosine Tran~form) to indicate "P(k/N)" by a co~ine function, and the inverse i~ an IDCT
(Inverse DCT). That is, "F(k/N) " and the original image signal ~f(n)" may be given by transforming "f(n)" according to DCT and IDCT, re~pectively.
(n) = E P (k/N) cos (2~n k/N) P(k/N) = (1/N) E f (n) cos (-; 2~n k/N) DCT and IDCT are especially effective for encoding operation~ of image data.
In computer image processing, "f" and "F" must be 21~84~
indicated in a secondary dimension as follows:
F(u,v) = {2C(u)C(v)/N} ~f(x,y)cos{~u(2x+1)/2N} co8{nv(2y+1)/2N}
F(x,y) = ~ S C(U)C(v)F(u,v)cos{~u(2x+1)/2N} cos{~v(2y+1)J2N}
C(N) = 1 / 8qrt(N) for N = 0 C(N) = 1 for N ~ 0 'Isqrt(N)ll : square root of IIN
u, V, X and Y = 0 to N-l llg(x, y)11 represents concentration information of the image, and is developed by using a DCT coefficient F(u, v) when it is transformed by DCT. The DCT coefficient F(u, v~ is the function of a space frequency llu, vll.
A (i, ~) = (1 / 8) C (i) cos {~i (2~ + 1) /N}
C (i) = 1 / 8 for N = 0 5 C (i) z 1 for N ~ 0 : square root of ~N"
In this case, "F" and "f" components are given by the following matrix arithmetic operations.
tF (u, v)] = [A (u, x)] [f(x, y)] [At (y~ v)]
[P (x, y)] = [At (x, u)] [F(u, v)] [A (v, y)]
[At] : replace matrix of [A]
u, v, x and y = 0 to N-l "Fl' for DCT, and "A" and "C" for IDCT are, hereinafter, called a DC coefficient and an AC coefficient, respectively.
A period for the matrix arithmetic operation is 2~
increased in proportion to the square of dimension N~. In game computers, the arithmetic operations are carried out for each block divided from one image. In general, a game computer uses a screen of 256 x 240 dots, which is divided into blocks of 16 x 16 dots, as shown in Fig. 2.
In such a game computer, RGB and YCrCb systems are used to display colors. According to the YCrCb system, each color may be treated independently, and the system is used in game computers. In the YCrCb system, each color is defined by a brightness (Y) and a color difference (U and V).
Accordlng to the YCrCb sy~tem, each component needR 8 bits to display a natural picture having 16,700,000 colors Therefore, in the screen of 256 x 240 dots, 180k bytes (256 x 240 x 3) capacity is required to display the natural picture.
Generally, in natural pictures, successive dots are not very different in color from each other, and therefore the next dots may be distinguished in color by adjusting the brightness thereof only. In such natural pictures, the brightnes~ i8 sampled for each dot, and the color difference is sampled at one dot interval, as shown in Fig. 3. When color data (Yl, Ul, Vl) and (Y2, U2, V2) are arranged at next each other, "U2~ and "V2" are not sampled, whereby (Yl, Ul, Vl) and (Y2, Ul, Vl) only are reproduced.
In the case as in Fig. 3, each block is defined by 8 x 8 dots, ~o that "N" become~ 8, and therefore, DCT and IDCT
formula~ are g~ven a~ follows:
8 ~ ~
F(u~v) = (C(u)C(v)/4} ~Ef (x,y)cos{~u(2x+1)/16} cos{~v(2y+1~/16}
F(x,y) = ~ ~ (C(u)C(v)/4F(u,v)cos{~u(2x+1)/16} cos{~v(2y+1)/16}
u, v, x and y = 0 to 7 In these formulas, the AC coefficient is not different from that in the formulas described before; however, a range of (u~ v, x and y) is 0 to 7, and as a result, the arithmetic time is shortened remarkably.
Image data which have been transformed by DCT are quantized in linear fashion by a Mid-treat type quantizing device (odd level quantizing device). "Y" and "C (U, V) n components are quantized independently, and 64 (8 x 8) DCT
coefficients are also quantized (divided) by their own quantizat$on steps, respectively. Each quantized coefficient is defined by 9 bits.
In order to decode the image data, the DCT
coefficients are reproduced by multiplying by the quantization ~teps. The quantizing process has an important effect on both of tne quality and compression rate of the decoded image, and the DCT coefficients are most important parameters for the encoding. These parameters are stored in quantizing matrix table~, which are called "Q-tables". The contents of the Q-table~ are different from each other depending on the display quality to be ~elected for the image.
~g. 4 shows a conventional system for encoding and decoding i~age data. The sy~tem includes a plurality of Q-21~8~
tables, and the data stored in the Q-tables are supplied to a decoder when a quantizing level is changed. Each Q-table has its own number to be used for being selected by the decoder.
The contents of the selected Q-tables are transmitted with the table numbers to the decoder.
According to the conventional system, however, there are disadvantages in that the system needs many Q-tables to realize a variety of display quality of image, and therefore, a large area in a memory is occupied for the Q-tables. Further, the ~ystem needs a buffer having a large capacity for buffering the Q-tables. The compression rate of the image data iB
remarkably changed in accordance with the size~ of the Q-tables.
In Fig. 4, "encoding" repre~ents a process for compressing an image data, that is, binary data of the image are transformed to the number of "1~ and "0".
For example, the following binary data, "00000000001111111111~00000111111110000000011100"
are transformed to the following number data, ~'10 11 5 8 8 3 2"
The number data are again transformed in the computer as follows.
"1010 1011 101 1000 1000 11 10"
In these data, a predetermined mark iB inserted in the spaces as ollows:
"1010*1011*101*1000*1000*11*10~
In general, a Huffman code iB used as the insert mark, 21~8~
and the codes are determined in the order of the highest frequency, that is, a numeral that is the most frequently used has the shortest code. As a result, the compression rate of the image is increased when the Yame binary numerals are arranged many times continuously.
In the conventional system, the encoding and decoding operations are not required when all the image blockg have the same color. When a moving picture is treated, data at a region where plural frames have the same information thereat is encoded in order to usad the code data in common. However, all regions in the same picture are not encoded. The conventional system is not suitable for treating still picture~.
8UHNARY OP TH~ INV~NTION
It is an ob~ect of the present invention to provide a high performance image processing apparatus in which the quality of an image to be displayed may be changed using a gingle Q-table.
It is another ob~ect of the invention to provide a high performance image proce~sing apparatus in which image data, e~peeially image data having square blocks of the same color, may be compre~ed at a high rate.
According to a first feature of the invention, in an ~mage proces~$ng apparatus, image data are compre~sed in accordance w$th a coeff~cient of a quantizing matrix table. The co~ic$ent $~ changed $n accordance with a predetermined 58~
factor.
According to a second feature of the invention, in an image processing apparatus, the image data are encoded using a null block code and null block length data when the image data include plural blocks having the same color, the blocks being arranged in succession continuously. The null block code and null block length represent the color and the number of the continuously arranged blocks, respectively.
Specifically, when an image data transmission is initialized, or when the content of the current Q-table is changed, the following Q-table is transmitted to an arithmetic circuit. The Q-factor is transmitted with the image blocks to the arithmetic circuit 80 that the display quality o the image may be changed for each block.
On the other hand, in a decoding process, when the following Q-table is transmitted to the arithmetic circuit, the Q-table is changed to the current Q-table. When the Q-factor is tran~mitted to the arithmetic circuit, the content of the current Q-table is changed in accordance with the Q-factor.
BRIPP D~SCRIPTION OP TH~ DRAWINGS
Pig. 1 is a graph used for explaining quantizing and sampling operations of image data.
Pig. 2 is a diagram ~howing the structure of a screen.
Pig 3 i~ a diagram showing the sampling operation.
~lg. 4 i8 a block diagram showing the operation o a 21~8~
conventional image processing apparatus.
Fig. 5 is a block diagram showing the operation of an image processing apparatu~ according to the invention.
Fig. 6 is a diagram showing the operation for sampling image data in accordance with the invention.
Fig. 7 is a diagram showing the operation for defining colors of the image data in accordance with the invention.
Fig. 8 is a diagram showing an example of an image to be displayed in accordance with the invention.
Fig. 9 is a block diagram showing a computer system of a preferred embodiment according to the invention.
Fig. 10 is a block diagram showing a control unit ~hown in Fig. 9.
Fig. 11 is a block diagram showing an image data extension unit shown in Fig. 9.
Pig. 12 is a diagram showing the operation for deining colors of the image according to the preferred embodiment.
Fig. 13 is a diagram showing the format of the image data compressed in accordance with the preferred embodiment.
Fig. 14 is a diagram showing the format of data including IDCT data according to the preferred embodiment.
Pig. 15 is a diagram showing the configuration of th~
IDCT data according to t~e preferred embodiment.
~5 Pig. 16 is a diagram showing the configuration of a part of the IDCT data according to the preferred embodiment.
210~8~0 Fig. 17 is a diagram showing color data for one image block in accordance with the preferred embodiment.
Fig. 18 i8 a diagram showing examples o~ a Q-table according to the preferred embodiment.
5Pig. 19 i8 the format of image data compressed by using a null block according to the preferred embodiment.
Fig. 20 is the format of image data according to the conventional system.
D~TAILED DFSCRIPTION OF THP INVENTION
Before describing a preferred embodiment of the invention, the fundamental theory of the invention is fir~t explained in con~unction with Pigs. 5 to 8.
Fig. 5 shows the process of transmission of image data in accordance with the invention In the invention, only one Q-table is used for each transmission. Quantizing and inver~e quantizing operations are carried out in accordance with a Q-factor and the Q-table. The Q-factor is established for each image block whereby the display quality of the image may be changed block-by-block. When the current Q-table is not suitable for the image, the Q-table is renewed by another Q-table. In this embodiment, a YCrCb system is used to display color~ of the image. According to the YCrCb ~y~tem, each color i~ defined by a brightness (Y) and a color difference (U and V).
In natural picture~, ~ucces~ive dots are not very 21~38~0 different in color from each other, 80 that the next dots may be distinguished by ad~usting the brightness thereof. For that reason, the brightness is sampled for each dot, and the color difference i~ sampled at one dot interval, a~ shown in Fig. 6 When the color component~ Y and C (U, V) are indicated by Y1~ and C1~ (Ui~, V1~), picture elements G1~ are indicated by "Yi~
+ U1~ + V1~", in which "+'1 represents synthesizing.
The Cmn components meet the following condition, because the Cmn components, "m" or "n~' being an odd number, are not sampled.
Cmn ~ C~n l (m : 0 or even number, n : odd number) = Cm l~ (m : odd number, n : 0 or even number) = Cmlnl ( both "m" and "n" : odd numbers) In this case, each of C10, C01 and Cll i9 equivalent to C00.
When a screen is divided by image blocks each defined by rectangular regions of 2k x 21 dots, as shown in Fig. 7, the ~ data need 2k x 21 dots and each of U and V data need k x 1 dot~.
In order to display a natural picture clearly, 16M
color~ must be u~ed in the system. In a natural picture, shown in Pig. 8, the ~ky varies in the color blue to light blue by the effect of white ground covered with snow. In ~uch a picture, the same color is continued substantially throughout, that is, the ~ame block~ are arranged in succession continuously. These block~ ar~ called "null block~". In this example, the image 21~8~0 data are indicated by ~null block code~ + ~null block length~, where ~+~ doe~ not represent summing. The null block length represents the number of the null blocks, and is also called a run-length. Such blocks indicated by the null block system are directly decoded without calculation using a Q-table. A number ~Z~ of dots to be decoded is given by the following formula.
Z = (the number of dots in the block) x (the number of null blocks) = 2k x 21 x (the number of null blocks) Next, a preferred embodiment according to the present invention will be explained in con~unction with Figs. 9 to 20.
Fig. 9 shows an image processing apparatu~ of the preferred embodiment. The apparatus includes a game-software 15recording medium 100 such as a CD-ROM, a CPU 102 of the 32-bit type, a control unit 104 for mainly controlling transmi~sion of image and sound data and interfacing most devices to each other, an image data extension unit 106, an image data output unit, a sound data output unit 110, a video encoder unit 112, a VDP unit 20114 and a TV display 116. CPU 102, control unit 104, image data exten~ion unit 106 and VDP unit 114 are provided with their own memories K-RAM, M-RAM, R-RAM and V-RAM, respectively.
CPU 102 directly control~ a DRAM via a memory support, and perorm~ communication through an I/O port to peripheral device~, that i~ called an I/O conkrol function. CPU 102 ~nclude~ a t~mer, a parallel I/O port and an interruption 2~ ~8~0 control sy~tem. VDP unit 114 reads display data which have been written in the VRAM by CPU 102. The diRplay data are transmitted to video encoder unit 112 whereby the data are displayed on the TV display 116. VDP unit 114 ha~ at most two screens each composed of background and sprite images, which are of an external block sequence type of 8 x 8 blocks.
Fig. 10 shows control unit 104. Control unit 104 includes an SCSI controller to which image and sound data are supplied through an SCSI interface from CD-ROM 100. Data supplied to the SCSI controller are buffered in the K-RAM.
Control unit 104 also includes a DRAM controller for reading data which have been buffered in the K-RAM at a predetermined timing In control unit 104, priority ~udyement is carried out for each dot of natural background image data, and an output signal i8 transmitted to video encoder unit 112.
Control unit 104 transmits moving image data (full color, pallet), which have been compressed, to image data exten~ion unit 106 whereby the compressed data are extended.
Pig. 11 ~hows image data extension unit 106. The image data extension unit includes an inverse DCT converter, an inver~e ~uantizing means, a Huffman coding and decoding means and a run-length coding and decoding means. That is, the image data exten~ion unit 106 performs a DCT transformation for a natural moving picture, and treats compressed data encoded by the Huffmsn coding method and run-length compressed data for a moYing animation ~mage and the like.
21~8~
Video encoder unit 112 superimposes VDP image data, natural background image data and moving image data (full color, pallet) transmitted from VDP unit 114, control unit 104 and image data extension unit 108. Video encoder unit 112 performs color pallet reproducing, special effect processing, D/A
converting and the like. Output data of video encoder unit 112 are encoded into a YCrCb signal by an external circuit.
ADPCM sound data recorded in CD-ROM 100 are buffered in the K-RAN and then transmitted to sound data output unit 110 by control unit 104. The sound data are reproduced by sound data output unit 110.
Fig. 12 shows an image data format according to the preferred embodiment. In this embodiment, each block is defined by 16 x 16 dots, and the image data are transmitted for 16 rasters.
Fig. 13 shows the configuration of compressed data to be transmitted. In this configuration, "A," B," C" and ~D~
represent the type of the image data, the first and la~t halves byte~ of the data length of a compre~ed data region, and a region storing data for forming boundaries for each two byte~ in the c~mpressed data, respectively. When image data with pallet information ~uch as an animation image are transmitted to the ~y~tem, region "A" is set at an F3H, F2H, FlH or FOH, the~e data being ind~ca~ed by hexadecimal numbers.
Pig. 14 show~ the coniguration of compressed data to be tran~mitt~d when the Q-table is initialized or the current Q-table is changed. The Q-table is located at the front of the IDCT data.
Fig. 15 shows the configuration of the IDCT data. In this configuration, "QCx" represents the coefficient of an inverse Q-table, and "NRL" represents that the following data is the null block.
Fig. 16 shows the arrangement of a "BLOCK SET" in the IDCT data. In this arrangement, each of "ZRL" and "EOB" i~ code data representing that 16 zero data are continued therein and that all data are zero data thereafter, respectively.
Fig. 17 shows the structure of a block forming a part of the image. The block is defined by the brightness Yl, Y2, Y3 and Y4 and the color difference U1 and Vl. The block is transmitted with a Q-factor (QCx) at the front thereof. In lS accordance with the Q-factor and Q-table, the following formulas are given.
Qy' (i,j) z QCx * Qy (i,j) / 4 i,j : 0 to 7 Qc' (i,~) = QCx * Qc (i,j) / 4 i,j : 0 to 7 i = j ~ O
Qc' (i,~) = Qc (i,~) / 4 i = j = 0 Qy' : Q-table for Y for quantizing / inverse-quantizing Qc' : Q-table for C for quantizing / inverse-quantizing Qy : down loaded Q-table for Y
Qc : down roaded Q-table for C
QCx : constant Q-factor, QCx = x (ex. QC7 - 7) 2~ ~84~
~ Down load~ means to transfer data from an external memory to the RAM.
Fig. 18 shows Q-tables to be down loaded.
Referring Fig. 13 again, ~A~ is ~et at FFH or F8H when a natural picture i8 transmitted. When IDCT inverse quantizing table data or compressed image data with IDCT inverse quantizing data are transmitted, ~A~ is set at FFH. When compressed image data for IDCT are transmitted, ~A~ is set at F8H.
When 16 image blocks of blue are included in 16 raster~ in the picture shown in Fig. 8, the null block process is available. That is, each null block code has a null block length (null block run-length) of 16.
Fig. 19 shows the arrangement of compressed null block data of 19 bits. The null block data are composed of a null block code region of 7 bits and a null block length region of 12 bit~. Therefore, "A" is ~et at F8H in the compressed image data format, and data of "(null block code) + (16)" are set in the null block region, shown in Fig. 19.
On the other hand, if the image data are transmitted ~n accordance with the conventional method, color data must be transmitted for each dot, as ~hown in Fig. 20. That is, data of ~Yl + Y2 + Y3 + Y4 + Ul + Vl" must be set for each dot. In Fig.
20, n BOB" repre~ent~ the end of a block.
A~ de~cribed above, according to the invention, the natural image, a~ ~hown in Fig. 8, may be transmitted using 19 bit~ o data only. On the other hand, the conventional system 2 1 ~
needs 1184 bits of data to transmit it.
Claims (2)
1. An image processing apparatus, in which image data composed of plural blocks are compressed to be well managed, comprising;
means for storing a matrix table;
means for storing a predetermined factor;
means for quantizing the image data for each block in accordance with the matrix table; and means responsive to the factor storing means for changing the matrix table.
means for storing a matrix table;
means for storing a predetermined factor;
means for quantizing the image data for each block in accordance with the matrix table; and means responsive to the factor storing means for changing the matrix table.
2. An image processing apparatus, in which image data composed of plural blocks are compressed to be well managed, comprising;
means for detecting the color blocks which have the same color and are arranged continuously; and means responsive to the detecting means for encoding the image data by using a color code representing the same color and a length code representing the number of the same color blocks.
means for detecting the color blocks which have the same color and are arranged continuously; and means responsive to the detecting means for encoding the image data by using a color code representing the same color and a length code representing the number of the same color blocks.
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP29807392A JPH06181441A (en) | 1992-10-09 | 1992-10-09 | Picture processing method and equipment therefor |
| JP29807492A JPH06181522A (en) | 1992-10-09 | 1992-10-09 | Method and device for picture processing |
| JP4-298073 | 1992-10-09 | ||
| JP4-298074 | 1992-10-09 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA2105840A1 true CA2105840A1 (en) | 1994-04-10 |
Family
ID=26561366
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA 2105840 Abandoned CA2105840A1 (en) | 1992-10-09 | 1993-09-09 | Image processing apparatus |
Country Status (2)
| Country | Link |
|---|---|
| EP (1) | EP0593159A3 (en) |
| CA (1) | CA2105840A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3703195B2 (en) * | 1995-03-27 | 2005-10-05 | ヒューレット・パッカード・カンパニー | Color image transmission device |
| PL1709801T3 (en) * | 2004-01-30 | 2013-02-28 | Panasonic Ip Corp America | Video Decoding Method Using Adaptive Quantization Matrices |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0810935B2 (en) * | 1988-12-09 | 1996-01-31 | 富士写真フイルム株式会社 | Image signal compression encoding device and decompression reproducing device |
| JP3085465B2 (en) * | 1989-10-31 | 2000-09-11 | オリンパス光学工業株式会社 | Image data encoding apparatus and encoding method |
| US5046119A (en) * | 1990-03-16 | 1991-09-03 | Apple Computer, Inc. | Method and apparatus for compressing and decompressing color video data with an anti-aliasing mode |
| EP0469835B1 (en) * | 1990-07-31 | 1998-09-30 | Canon Kabushiki Kaisha | Image processing apparatus and method |
| JPH04323960A (en) * | 1991-04-23 | 1992-11-13 | Canon Inc | Picture coding method and device |
-
1993
- 1993-09-03 EP EP19930306987 patent/EP0593159A3/en not_active Withdrawn
- 1993-09-09 CA CA 2105840 patent/CA2105840A1/en not_active Abandoned
Also Published As
| Publication number | Publication date |
|---|---|
| EP0593159A2 (en) | 1994-04-20 |
| EP0593159A3 (en) | 1994-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6687407B2 (en) | Quantization table adjustment | |
| US6947592B2 (en) | Encoding method of a color image and its encoding device and a decoding method of the color image and its decoding device | |
| JP3069455B2 (en) | Quantization and dequantization circuits in image data compression and decompression equipment | |
| KR100287624B1 (en) | Image data compression, decompression method and device | |
| US5619591A (en) | Encoding and decoding color image data based on mean luminance and an upper and a lower color value | |
| JPH0662262A (en) | Method and apparatus for control of block for digital image codec (compression and expansion) processor | |
| US6353685B1 (en) | Method and apparatus for image compression | |
| JP3469438B2 (en) | Image signal processing method and apparatus, recording medium | |
| JP3442111B2 (en) | Image compression device, image reproduction device and drawing device | |
| CA2105840A1 (en) | Image processing apparatus | |
| JPH11308465A (en) | Encoding method for color image, encoder therefor, decoding method for color image and decoder therefor | |
| JP3023215B2 (en) | Image processing device | |
| JP2646503B2 (en) | data format | |
| US8068681B2 (en) | Method and system for pipelined processing in an integrated embedded image and video accelerator | |
| JP2003348355A (en) | Image processing apparatus and control method therefor | |
| JP3981651B2 (en) | Image processing device | |
| JPH1130978A (en) | Color image encoding method and its encoder and color image decoding method and its decoder | |
| JPH06181522A (en) | Method and device for picture processing | |
| JP3170313B2 (en) | Image processing device | |
| JP2646504B2 (en) | Image processing method | |
| Razavi et al. | Low-cost still-picture compression chip set | |
| JPH0746407A (en) | Picture data compressing device and picture data restoring device | |
| JPH06315143A (en) | Image processor | |
| JP3532961B2 (en) | Character font synthesizer | |
| JPH06181441A (en) | Picture processing method and equipment therefor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| FZDE | Dead |