JP2016072826A - Image decoding device, graphic processing device, image decoding method, and graphic processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a graphic processing technology for efficiently decompressing a compressed texture.SOLUTION: An image decoding device includes a main memory 300 and a graphic processing unit (GPU) 200. The GPU 200 includes a variable length decoding part 30 and an inverse discrete cosine transformation (IDCT) part 40. The variable length decoding part 30 executes variable length decoding on a compressed texture on the basis of an encoding table for allocating a code corresponding to a pair of a range of a run number and a range of a level value together with an immediate field indicating at least either an immediate value of the run number or an immediate value of the level value, and the IDCT part 40 restores the texture by performing inverse discrete cosine transformation on the texture on which the variable length decoding has been performed. The main memory 300 includes a texture pool for partially caching the restored texture.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for decoding an image, and more particularly to a graphics processing technique for decompressing a compressed texture.

  High-quality images such as playing games and simulations using high-quality 3D computer graphics and playing video content that combines live-action and computer graphics on personal computers and game machines The use of graphics is spreading.

  In general, graphics processing is executed by cooperation of a CPU and a graphics processing unit (GPU). While the CPU is a general-purpose processor that performs general-purpose operations, the GPU is a dedicated processor for performing advanced graphics operations. The CPU performs a geometry calculation such as projection conversion based on the three-dimensional model of the object, and the GPU receives vertex data from the CPU and executes rendering. The GPU is composed of dedicated hardware such as a rasterizer and a pixel shader, and executes graphics processing by pipeline processing. Some recent GPUs have programmable shader functions called programmable shaders, and graphics libraries are generally provided to support shader programming.

  In the graphics processing, texture mapping is performed in which a texture is pasted on the surface of the object in order to express the texture of the surface of the object. As the images used in applications such as games become higher in definition, higher resolution data is used for textures, and the volume of texture data is increasing. For example, the texture used in the game is in the order of GiB (Gibibyte), and it is difficult to store all necessary texture data on the memory.

  In view of this, uncompressed textures or low-compressed textures that can be directly handled by the GPU are stored in a storage device such as a hard disk, and loaded into a texture buffer on a memory and used for drawing as necessary. The time it takes to load a texture from the hard disk is usually tens of milliseconds to sometimes several seconds and is not stable. Therefore, when the texture loading from the hard disk is not in time, there arises a problem that the texture to be originally displayed cannot be used.

  On the other hand, in the case of a highly compressed texture, even a texture exceeding the main memory capacity can be held in the memory, and the texture can be handled without loading from the hard disk. However, in this case, since the high-compression texture is generally not directly handled by the GPU, dedicated hardware for decompressing the high-compression texture in real time is required. If dedicated hardware is not available, the CPU will decompress the compressed texture and expand it in the texture buffer. In this case, however, it takes time to decompress and it becomes difficult to perform drawing in real time.

  The present invention has been made in view of these problems, and an object thereof is to provide a graphics processing technique capable of efficiently decompressing a compressed texture.

  In order to solve the above problems, an image decoding apparatus according to an aspect of the present invention indicates a code corresponding to a pair of a range of run numbers and a range of level values, and indicates at least one of an immediate value of run numbers and an immediate value of level values. A variable length decoding unit that performs variable length decoding of a compressed image based on an encoding table that is assigned together with an immediate field, and an inverse spatial frequency conversion unit that restores an image by performing inverse spatial frequency conversion on the variable length decoded image Including.

  Another aspect of the present invention is a graphics processing apparatus. This device is a graphics processing device including a main memory and a graphics processing unit, wherein the graphics processing unit assigns a code corresponding to a pair of a run number range and a level value range to an immediate value of the run number. And a variable-length decoding unit that performs variable-length decoding of the compressed texture based on an encoding table that is assigned together with an immediate field indicating at least one of the immediate values of the level value, and texture by performing inverse spatial frequency conversion on the variable-length decoded texture And an inverse spatial frequency transforming unit for restoring. The main memory includes a texture pool that partially caches the restored texture.

  Yet another aspect of the present invention is an image decoding method. In this method, a variable length of a compressed image is assigned based on a coding table in which a code corresponding to a pair of a run number range and a level value range is assigned together with an immediate field indicating at least one of an immediate value of a run number and an immediate value of a level value. Performing decoding, and restoring the image by performing inverse spatial frequency transform on the variable length decoded image.

  Yet another embodiment of the present invention is a graphics processing method. This method is a graphics processing method in a graphics processing device including a main memory and a graphics processing unit, and the graphics processing unit supports a pair of a range of run numbers and a range of level values by a compute shader. Variable-length decoding of the compressed texture is performed based on an encoding table that assigns the code to be executed together with an immediate field indicating at least one of the immediate value of the run number and the immediate value of the level value, and the variable-length decoded texture is subjected to inverse spatial frequency conversion. The texture is restored, and the restored texture is stored in the texture pool in the main memory where the texture is partially cached.

  It should be noted that any combination of the above-described constituent elements and the expression of the present invention converted between a method, an apparatus, a system, a computer program, a data structure, a recording medium, and the like are also effective as an aspect of the present invention.

  According to the present invention, an encoded image, particularly a compressed texture can be efficiently decompressed.

It is a block diagram of the graphics processing apparatus which concerns on embodiment. Fig.2 (a)-FIG.2 (c) are the figures explaining a mipmap texture. It is a figure explaining the mechanism of PRT of this Embodiment. It is a figure which shows an example of the encoding table with an immediate field of FIG. It is a figure which shows another example of the encoding table with an immediate field of FIG. It is a figure which shows another example of the encoding table with an immediate field of FIG. It is a figure explaining the execution process of the thread | thread when a branch destination has no bias for a comparison. It is a figure explaining the execution process of the thread | thread when a branch destination has a bias. FIG. 7 is a diagram for explaining a branch when searching for which row of the encoding table with an immediate field in FIG. 6 corresponds to the encoded data. It is a figure which shows the program code which has the branch demonstrated in FIG. It is a figure which shows the Example of the encoding table with an immediate field of FIG.

  FIG. 1 is a configuration diagram of a graphics processing apparatus according to an embodiment. The graphics processing device includes a main processor 100, a graphics processing unit (GPU) 200, and a main memory 300.

  The main processor 100 may be a single main processor, a multiprocessor system including a plurality of processors, or a multicore processor in which a plurality of processor cores are integrated in one package. Good. The main processor 100 can read / write data from / to the main memory 300 via the bus.

  The GPU 200 is a graphic chip equipped with a graphic processor core, and can read / write data from / to the main memory 300 via a bus.

  The main processor 100 and the GPU 200 are connected by a bus, and the main processor 100 and the GPU 200 can exchange data with each other via the bus.

  FIG. 2 shows a configuration related to texture processing in the graphics processing, and the configuration related to other processing is omitted.

  The memory area of the main memory 300 is memory-mapped in an address space referred to by the GPU 200 so that the GPU 200 can access the GPU 200, and the GPU 200 can read texture data from the main memory 300. Texture data is partially cached in the main memory 300 using a method called PRT (Partially Resident Textures).

  The main processor 100 includes a graphics calculation unit 20 and a PRT control unit 10. The graphics calculation unit 20 receives an LOD (level of detail) value indicating the level of detail of the texture from the graphics processing unit 50 of the GPU 200 and passes the LOD value to the PRT control unit 10. Based on the LOD value received from the graphics processing unit 50, the PRT control unit 10 calculates a mipmap texture that will be required in the future, instructs the PRT cache 320, which is a texture pool, and uses it. PRT mapping is updated by removing the missing page.

  Fig.2 (a)-FIG.2 (c) are the figures explaining a mipmap texture. The mipmap texture is a plurality of textures having different resolutions according to the level of detail (LOD). The mipmap texture 340 in FIG. 2A is a high-resolution texture. The mipmap texture 342 in FIG. 2B is a medium resolution texture in which the vertical and horizontal sizes of the high resolution mipmap texture 340 in FIG. The mipmap texture 344 in FIG. 2C is a low resolution texture in which the vertical and horizontal sizes of the medium resolution mipmap texture 342 in FIG.

  Returning to FIG. 1, the PRT control unit 10 instructs the GPU 200 to read the mipmap texture of the level of detail specified in the graphics calculation unit 20. More specifically, the PRT control unit 10 controls the variable length decoding unit 30 and the inverse discrete cosine transform (IDCT) unit 40 of the GPU 200, and also swaps in and swaps the PRT cache 320 stored in the main memory 300. Control out.

  The GPU 200 includes a variable length decoding unit 30, an IDCT unit 40, and a graphics processing unit 50.

  The variable length decoding unit 30 reads out the compressed texture 310 corresponding to the level of detail specified from the PRT control unit 10 from the main memory 300, and calls the encoding table 60 with an immediate value field (hereinafter referred to as “coding table 60” for short). The compressed texture 310 is subjected to variable length decoding with reference to the data, and stored in the DCT block ring buffer 80.

  The IDCT unit 40 performs inverse discrete cosine transform on the textured DCT block after variable length decoding stored in the DCT block ring buffer 80 and stores it in the PRT cache 320.

  The graphics processing unit 50 reads a necessary mipmap texture from the PRT cache 320. The PRT cache 320 is a texture tile pool that partially caches textures, swaps in necessary textures, and swaps out unnecessary ones.

  FIG. 3 is a diagram for explaining the mechanism of the PRT according to the present embodiment.

  On the virtual memory, areas of mipmap textures 340, 342, and 344 are arranged. The texture area is divided into chunks of a certain size, and only the necessary texture area is stored in the texture tile pool 360 using the page table 330. Here, since the texture exists in the main memory 300 as the compressed texture 310, when the texture area is cached in the texture tile pool 360, a process for expanding the compressed texture 310 is required. The PRT control unit 10 controls the variable length decoding unit 30 and the IDCT unit 40 in accordance with a request from the graphics processing unit 50, and expands the compressed texture 310 as necessary.

  In the example shown in the figure, the chunk 352 of the high resolution mipmap texture 340 and the chunk 358 of the medium resolution mipmap texture 342 are associated with the pages 332 and 338 of the page table 330, respectively, and the physical memory is a texture tile. Mapped from pool 360.

  On the other hand, the chunk 354 of the high-resolution mipmap texture 340 and the chunk 356 of the medium-resolution mipmap texture 342 are associated with the pages 334 and 336 of the page table 330, respectively. Not mapped from pool 360. In this case, as described above, the PRT control unit 10 controls the texture tile pool 360 so that the necessary texture is in the texture tile pool 360 based on the LOD value received from the graphics processing unit 50, and the physical memory of the texture tile pool 360 is The necessary texture data is expanded from the compressed texture 310 and stored in the texture tile pool 360. On the other hand, the graphics processing unit 50 reads the mipmap texture from the texture tile pool 360 using the LOD value calculated by itself without going through the main processor 100. At this time, if the mipmap texture corresponding to the calculated LOD value does not exist in the texture tile pool 360, the graphics processing unit 50 falls back to reduce the requested detail level, Read from the texture tile pool 360 and draw.

  Here, the data format of the texture will be described. The original texture data before being compressed is given in RGB 32-bit format, for example. As a texture format that can be directly handled by the GPU 200, there is a texture compressed by a texture compression method called BC5 or BC7. According to this, compared with the original texture data, the quality is kept relatively good. Data volume can be reduced with a compression ratio of 1/4. If the quality may be relatively low, a texture compressed by a texture compression method called BC1 or DXT1 can be used. In this case, the compression ratio is about 1/8 compared to the original texture data. Can reduce the amount of data.

  On the other hand, the GPU 200 cannot directly handle the data, but if a texture compressed by JPEG is used, the data amount can be reduced at a compression ratio of about 1/20 compared to the original texture data. In this case, it is inefficient to execute a complicated algorithm such as JPEG decompression in the compute shader of the GPU 200, and if there is no dedicated hardware capable of performing JPEG decompression, the compressed texture is decompressed in real time. It is difficult to use for processing.

  In the present embodiment, the amount of data can be reduced with a compression rate of approximately 1/20 by performing variable length encoding using a DCT and an encoding table with an immediate field. When high compression is performed up to this point, the compressed texture 310 can be made resident in the main memory 300. The GPU 200 reads the compressed texture 310 from the main memory 300, and performs variable length decoding and inverse discrete cosine transform (IDCT) using an encoding table with an immediate field in real time by a compute shader to restore the texture. Is possible.

  Since textures compressed with JPEG cannot be directly used by the GPU 200, they need to be decoded once by a JPEG decoder. A graphics device equipped with a JPEG codec can handle JPEG-compressed textures, but generally cannot use a JPEG codec. JPEG compression performs Huffman coding after subjecting an image to discrete cosine transform and quantization. Since Huffman coding is a complicated compression algorithm, if the compute shader of the GPU 200 performs Huffman decoding of a texture that has been JPEG-compressed, the amount of calculation will be enormous.

  On the other hand, the variable-length coding using the coding table 60 with the immediate field can reduce the coding table by using the immediate field, so that unlike the normal Huffman coding, It can be efficiently executed by a compute shader.

  In normal Huffman coding, one Huffman code is assigned to a combination of “Run” indicating the number of consecutive “0” and “Level” which is a value other than “0”. Assign and encode. Minimize the average code length of data by assigning short codes to combinations of run numbers and level values with high appearance frequency and long codes to combinations of run numbers and level values with low appearance frequency be able to.

  On the other hand, variable-length coding using the coding table 60 with an immediate field is made by combining a “run number” and “level value” pair with an exponent Golomb-like “immediate field” to create a code. The number of rows in the encoding table is reduced. The number of rows in the coding table is about 12 at most, and each row of the coding table represents a pair of “run number range” and “level value range” determined for each row, and an actual run Numbers and level values are given by "immediate" given in the immediate field of each row. Here, for the "run number range" and "level value range" pairs, a pair with a high application frequency is expressed with a code with a short bit length, and a pair with a low application frequency is expressed with a code with a long bit length. .

  At the time of variable-length decoding using the encoding table 60 with an immediate value field, first, a search is made as to which row of the encoding table 60 is applicable, and a pair of “run number range” and “level value range” is determined from the applied row. And the immediate value of the run number and the immediate value of the level value may be obtained from the immediate field of the row. In the case of a normal Huffman coding table, the number of rows is large and the table search is complicated and difficult to execute on the GPU 200. However, since the coding table 60 with an immediate value field has a small number of rows, conditional branches may be reduced. The GPU 200 can execute variable length decoding efficiently by executing a plurality of threads in parallel.

  FIG. 4 is a diagram illustrating an example of the encoding table 60 with an immediate field. The encoding table 60 has four rows, and codes (codes) having different bit lengths are assigned to pairs of the range of run numbers and the range of level values. Here, the DCT block is 16 × 16, and the value of the 12-bit DCT coefficient is divided and coded every 256. Therefore, the number of runs takes a value of 0 to 255, and the level value is a value of 0 to 4095. I take the.

  Code 1 “1RRsLL” corresponds to a pair of run numbers ranging from 0 to 3 (2 bits) and level values ranging from 0 to 3 (2 bits), and has a bit length of 6. The leading “1” is a code for identifying the code 1. “RR” is an immediate value of the number of runs and takes any value from 0 to 3. “LL” is an immediate value of the level value and takes any value from 0 to 3. “S” is a sign bit and indicates whether the level value is positive or negative (the same applies hereinafter).

  Code 2 “01RRRRRRsLLLLLL” corresponds to a pair of run number range 0 to 31 (5 bits) and level value range 0 to 31 (5 bits), and has a bit length 13. The leading “01” is a code for identifying the code 2. “RRRRRR” is an immediate value of the number of runs and takes any value from 0 to 31. “LLLLLL” is an immediate value of the level value and takes any value from 0 to 31.

  Code 3 “001RRRRRRRRsLLLLLLLLLLLLLL” corresponds to a pair of run numbers ranging from 0 to 255 (8 bits) and level values ranging from 0 to 4095 (12 bits), and has a bit length of 24. The leading “001” is a code for identifying the code 3. “RRRRRRRR” is an immediate value of the number of runs and takes any value from 0 to 255. “LLLLLLLLLLLLLL” is an immediate value of the level value and takes any value from 0 to 4095.

  The code 4 “0001” corresponds to a code EOB (End of Block) indicating the end of the block, indicating that it is all 0 thereafter, and has a bit length of 4. “0001” is a code for identifying the code 4.

  As described above, each row of the encoding table 60 with the immediate value field includes the code identification code corresponding to the pair of the range of the run number and the range of the level value, the immediate value of the run number, the immediate value of the level value, and the sign of the level value. And an immediate field representing the sign bit to be indicated.

  In texture compression according to the present embodiment, discrete cosine transform (DCT) is performed on a block of an image, and then quantization and variable length coding are performed. When a natural image is subjected to discrete cosine transform, most of the frequency components are concentrated in the low frequency region, and the high frequency components become so small that they can be ignored. In particular, due to quantization, the DCT coefficient of the high frequency component becomes almost zero. For this reason, there are many cases in which a lot of zeros continue in the variable length coding input data.

  When variable length coding is performed on a quantized DCT coefficient of a texture image based on the coding table 60 of FIG. 4, the number of appearances of each code is 7,200 for code 1, 810 for code 2, and code 3 There were 62 and code 4 was 260. When the number of appearances is multiplied by the bit length of each code and totaled, the code amount of the entire compressed texture is 56,258 bits.

  FIG. 5 is a diagram showing another example of the encoding table 60 with an immediate field. In the encoding table 60 of FIG. 4, since the number of occurrences of code 1 corresponding to the pairs of the run number range 0 to 3 and the level value range 0 to 3 is very large, the encoding table 60 of FIG. 4 is added to the 4-row coding table 60 in FIG. 4 by adding 3-bit code 1 corresponding to the range of run numbers 0 to 1 and level value 1 to obtain a 5-row table.

  In the coding table 60 of FIG. 5, code 1 “10s” (3 bits), code 2 “01RRRRLL” (7 bits), code 3 “001RRRRRRsLLLLLL” (14 bits), code 4 “0001RRRRRRRRsLLLLLLLLLLLL” (25 bits), code It is encoded with 5 “00001” (5 bits).

  In the encoding table 60 of FIG. 4, the number of occurrences of code 1 (6 bits) is 7,200, and the total number of bits of code 1 is 43,200 bits, whereas the encoding of FIG. In the table 60, this is divided into 3,900 code 1 (3 bits) and 3,300 code 2 (7 bits), and the total number of bits of code 1 and code 2 is 11,700 + 23, 100 = 34, Decrease to 800 bits. When the encoding table 60 of FIG. 5 is used, the code amount of the entire compressed texture is 48,990 bits, and the code amount can be reduced as compared with the case of using the encoding table 60 of FIG.

  FIG. 6 is a diagram showing still another example of the encoding table 60 with an immediate field. Compared with the encoding table 60 of FIG. 5, the encoding table 60 of FIG. 6 further increases the number of rows, that is, the number of combinations of the pairs of the range of run numbers and the range of the level value, so that 10 rows, that is, 10 codes. The table.

  Code 1 “1Rs” corresponds to a pair of run numbers ranging from 0 to 1 (1 bit) and level value 1, and has a bit length of 3. “R” takes a value of 0 or 1, and represents the immediate value of the run number as it is. The code 1 is a code for encoding (Run, Level) = (0, 1), (1, 1).

  Code 2 “010RsL” corresponds to a pair of run numbers ranging from 0 to 1 (1 bit) and level values ranging from 2 to 3 (1 bit), and has a bit length of 6. “R” takes a value of 0 or 1, and represents the immediate value of the run number as it is. “L” takes a value of 0 or 1 and adds an offset 2 to represent an immediate value of the level value.

  Code 3 “011RRsLL” corresponds to a pair of run number range 2 to 5 (2 bits) and level value range 1 to 4 (2 bits), and has a bit length of 8. “RR” takes any value from 0 to 3 and adds an offset 2 to represent an immediate value of the number of runs. “LL” takes any value from 0 to 3 and adds an offset 1 to represent an immediate value of the level value.

  Code 4 “0010RsLL” corresponds to a pair of run numbers ranging from 0 to 1 (1 bit) and level values ranging from 4 to 7 (2 bits), and has a bit length of 8. “R” takes a value of 0 or 1, and represents the immediate value of the run number as it is. “LL” takes any value from 0 to 3, and an offset 4 is added to represent the immediate value of the level value.

  Code 5 “0011RRsLL” corresponds to a pair of run numbers ranging from 6 to 9 (2 bits) and level values ranging from 1 to 4 (2 bits), and has a bit length of 9. “RR” takes any value from 0 to 3 and adds an offset 6 to represent an immediate value of the number of runs. “LL” takes any value from 0 to 3 and adds an offset 1 to represent an immediate value of the level value.

  Code 6 “00010RRRRRRs” corresponds to a pair of run numbers ranging from 10 to 73 (6 bits), level value 1, and has a bit length of 12. “RRRRRR” takes any value from 0 to 63 and adds an offset of 10 to represent the immediate value of the number of runs.

  Code 7 “00011RRRRRRsLLLLLL” corresponds to a pair of run number range 0 to 31 (5 bits) and level value range 0 to 31 (5 bits), and has a bit length of 16. “RRRRRR” takes any value from 0 to 31 and represents the immediate value of the run number as it is. “LLLLLL” takes any value from 0 to 31 and represents the immediate value of the level value as it is.

  A code 8 “00001sLLLLLLLLLLLLLL” corresponds to a pair of 0 in the number of runs and a level value range of 0 to 4095 (12 bits), and has a bit length of 18. “LLLLLLLLLLLLLL” takes any value from 0 to 4095 and represents the immediate value of the level value as it is.

  The code 9 “000001” corresponds to a code EOB (End of Block) indicating the end of the block, indicating that it is all 0 thereafter, and has a bit length of 6.

  Code 10 “0000001RRRRRRRRsLLLLLLLLLLLLLL” corresponds to a pair of run number range 0-255 (8 bits) and level value range 0-4095 (12 bits), and has a bit length of 28. “RRRRRRRR” takes any value from 0 to 255 and represents the immediate value of the run number as it is. “LLLLLLLLLLLLLL” takes any value from 0 to 4095 and represents the immediate value of the level value as it is.

  The number of occurrences and the total number of bits of codes 1 to 10 are as shown in the figure. When the encoding table 60 of FIG. 6 is used, the number of bits with a short bit length is increased, so that the total number of bits in each row can be reduced. As a result, the code amount of the entire compressed texture is 43,536 bits. Yes, the amount of codes can be further reduced as compared with the case of using the encoding table 60 of FIG.

  In any of the encoding tables 60 with an immediate value field according to the present embodiment, the range of run numbers and the range of level values are allowed to overlap between different codes. In the case where the run number range and the level value range of two or more codes are applicable, a code having a shorter code length is preferentially used.

  With reference to the encoding table 60 with an immediate field in FIG. 4, variable length decoding using the encoding table 60 with an immediate field will be described. In order to search which of the codes 1 to 4 in the encoding table 60 of FIG. 4 corresponds to the encoded data, it is examined what bit the 1 appears first in the bit string of the encoded data.

  If 1 appears first in the first bit (referred to as “branch A”), it is code 1 and the immediate value of the run number (2 bits), the sign bit, and the immediate value of the level value from the remaining 5-bit immediate field Read (2 bits) sequentially.

  If 1 appears first in the second bit (referred to as “branch B”), it is code 2 and the immediate value of the run number (5 bits), the sign bit, and the immediate value of the level value from the remaining 11-bit immediate field Read (5 bits) sequentially.

  If 1 appears first in the third bit (referred to as “branch C”), it is code 3, and the immediate value of the run number (8 bits), the sign bit, and the immediate value of the level value from the remaining 21-bit immediate field Read (12 bits) in order.

  When 1 appears first in the fourth bit (referred to as “branch D”), it is code 4 and is EOB.

  From the example of the number of appearances of each code shown in FIG. 4, when variable length decoding is performed on the compressed texture data that has been subjected to variable length encoding using the encoding table 60 of FIG. Recognize. Due to the nature of variable length coding using the coding table 60 as shown in FIG. 4, the compute shader of the GPU 200 can efficiently perform variable length decoding. This is because the GPU 200 has a SIMD (Single Instruction Multiple Data) architecture, and a plurality of threads execute the same instruction simultaneously on different data, so if the branch condition is biased, the degree of parallelism increases and the execution efficiency increases. .

  In the GPU 200, one program counter (PC) refers to an instruction stored in the instruction cache, and, for example, 16 ALUs (Arithmetic Logic Units) execute instructions referred to by the PC at the same time. A different instruction is set in 16 threads for each branch of the loop of the if-then-else statement and the switch-case statement, and is executed simultaneously. For the 16 threads, in the conditional branch by the if-then-else statement, the thread responsible for the pixel when the if condition is true (True) is executed and executed in parallel. In the else branch, the else condition is If it is established (False), the thread responsible for the pixel is enabled and executed in parallel. In the conditional branch by the switch-case statement, the thread in charge of the pixel of the case where the condition is satisfied is enabled and executed in parallel.

  In conditional branching with if-then-else statements, if the if condition is satisfied and the else condition is approximately equal, the threads that are enabled in the case of True and False are frequently replaced. However, if the if condition is satisfied, such as 80%, and the else condition is satisfied, such as 20%, the set of threads to be enabled in the case of True can be used repeatedly, and the execution efficiency increases. In the conditional branch by the switch-case statement, if the number of cases established is almost the same, the threads to be activated are frequently replaced for each case, but the frequency of establishment of each case is biased. For example, since a set of threads that are activated in a case having a high establishment frequency can be used repeatedly, execution efficiency is increased. This point will be described in more detail with reference to FIGS.

  FIG. 7 is a diagram for explaining the thread execution process when there is no bias in the branch destination for comparison.

  The GPU 200 includes a plurality of computing units. The number of threads simultaneously executed by one calculation unit of the GPU 200 is determined by the number of arithmetic units in the calculation unit. A group of up to 16 threads that can be simultaneously input to one computing unit is called a “thread set”. Each thread included in the thread set executes the same shader program, but the data to be processed is different. If there is a branch in the program, each thread may have a different branch destination. One calculation unit executes one thread set (here, a maximum of 16 threads) in parallel in a certain cycle.

  For example, even if the number of instructions required at each branch destination is only a few, the program counter is one, and all the arithmetic units in the calculation unit have a SIMD structure that executes the same instruction. Each instruction of each branch is executed while changing the thread to be executed.

  As an example, assume that branch A when executing variable length decoding using the encoding table 60 of FIG. 4 is executed with 4 instructions, branch B with 4 instructions, branch C with 4 instructions, and branch D with 2 instructions. In the example of FIG. 7, branch destinations of 16 threads in the thread set 450 are A, A, B, A, A, A, B, C, B, A, B, A, B, A, B, The case of D is described.

  In cycle 1, only the thread that executes branch A (in this case, eight threads) is enabled, and the four instructions A-1, A-2, A-3, A-4 is executed.

  In cycle 5, only the thread that executes branch B (in this case, six threads) is enabled, and the four instructions B-1, B-2, B-3, B-4 is executed.

  In cycle 9, only the thread that executes branch C (in this case, one thread) is enabled, and while the program counter is advanced by one, four instructions C-1, C-2, C-3, Execute C-4.

  In cycle 13, only the thread that executes branch D (in this case, one thread) is enabled, and the two instructions D-1 and D-2 of branch D are executed while the program counter is advanced one by one.

  Thus, in the example of FIG. 7, 14 cycles are required for the 16 threads included in the thread set to execute all the instructions of the four branches A to D.

  FIG. 8 is a diagram illustrating a thread execution process when there is a bias in the branch destination. In the example of FIG. 8, the branch destinations of the 16 threads in the thread set 452 are A, A, B, A, A, A, B, B, B, A, B, A, A, A, A, The case of A is described. In this example, there are four types of branch destinations in the shader program, but the pixels that satisfy the branch condition are biased, and there are only two types of branch destinations, branch A and branch B. The 16 threads included in the thread set need only execute these two types of branches.

  In cycle 1, only the thread that executes branch A (in this case, 11 threads) is enabled and the four instructions A-1, A-2, A-3, A-4 is executed.

  In cycle 5, only the thread that executes branch B (in this case, five threads) is enabled, and while the program counter is advanced by one, four instructions B-1, B-2, B-3, B-4 is executed.

  As described above, in the example of FIG. 8, the 16 threads included in the thread set only need to execute all the instructions of the two branches A and B, and the necessary number of cycles is reduced to 8.

  If there is a bias in the branch destination of the program due to the nature of the input data, the same thread mask can be used as it is to execute the instruction repeatedly, improving the execution efficiency. If there are variations in the branch destinations, the thread mask is switched for each branch, and the execution efficiency decreases.

  Due to the nature of DCT coefficients derived from natural images, values other than 0 are concentrated on the upper left low frequency component of the DCT coefficient matrix, and 0 continues to the lower right high frequency component of the DCT coefficient matrix. Therefore, when an image block after the discrete cosine transform is made into a one-dimensional array by a zigzag pattern, the DCT coefficient of any block tends to be a data string in which a non-zero value continues first and 0 continues in the latter half.

  Based on the tendency of the DCT coefficient, variable length encoded data is allocated to each thread of the thread set so as to process the DCT coefficient of a different DCT block, and the DCTs in which the threads are relatively in the same position in the DCT block are allocated. Configure the thread set to perform variable length decoding of the coefficients. In the case of the encoding table 60 in FIG. 4, the branch destination is one of the branches A to D. From the configuration of the thread set, the tendency of the DCT coefficient is similar at the relatively same position in the DCT block, so that the branch destination of each thread in the thread set is biased to the same. Accordingly, the branch destinations do not vary as shown in FIG. 7, but the branch destinations become biased as shown in FIG. 8, and the efficient execution state of the thread set can be continued for a long time. As a result, variable length decoding is efficiently performed by the thread set.

  The procedure of variable length decoding using the encoding table 60 with immediate field in FIG. 6 will be described in detail. FIG. 9 is a diagram for explaining a branch when searching for which row of the encoding table 60 in FIG. 6 corresponds to the encoded data. It is examined what number bit the 1 appears first in the bit string of the encoded data.

  If 1 appears first in the first bit, it is case 0; if 1 appears first in the second bit, it is case 1; if 1 appears first in the third bit, it is case 2 If 1 appears first in the 4th bit, it is case 3; if 1 appears first in the 5th bit, it is case 4; if 1 appears first in the 6th bit, it is in case 5 Yes, if 1 appears first in the seventh bit, it is case6.

  Case 0 corresponds to code 1, case 4 corresponds to code 8, case 5 corresponds to code 9, and case 6 corresponds to code 10. Therefore, the immediate value of the run number and the immediate value of the level value are appropriately set from the remaining immediate field. Read it out.

  Case 1 corresponds to code 2 and code 3, and if the third bit is 0, it is determined to be code 2 and if the third bit is 1, it is determined to be code 3, and then the number of runs from the remaining immediate field The immediate value of level and the immediate value of the level value may be read out.

  Similarly, case2 corresponds to code 4 and code 5, code 4 if the fourth bit is 0, and code 5 if the fourth bit is 1. Case 3 corresponds to code 6 and code 7, and is code 6 if the 5th bit is 0, and code 7 if the 4th bit is 1. If any code is specified, the immediate value of the run number and the immediate value of the level value are appropriately read from the remaining immediate value fields.

  FIG. 10 shows program source code having the branch described in FIG. clz = FirstSetBit_Hi_MSB (code) is an arithmetic expression for obtaining a column number clz in which 1 first appears in the bit string of the encoded data. Since the column numbers are counted from 0, case 0 to case 6 of the switch statement of the program code correspond to case 0 to case 6 in FIG. The function BITAT (code, n−1, m) is an operation for reading an m-bit bit string forward from the n-th column of the bit string of the encoded data.

  Describing the source code of case 1 of the switch statement, if (BITAT (code, 2,1) == 0) is the case where the third bit of the bit string of the encoded data is 0, which is shown in FIG. Code 2. In code 2, run = BITAT (code, 3,1) is executed because the immediate value of the run number may be read from the fourth bit. Next, the immediate value of the level value is read from the 6th bit, but since it is necessary to add 2 as an offset, level = BITAT (code, 5,1) +2 is executed. Since the sign bit may be read from the fifth bit, sign = BITAT (code, 4, 1) is executed.

  If if (BITAT (code, 2,1) == 0) does not hold, since the third bit of the bit string of the encoded data is 1, this is code 3 in FIG. In this case, the else statement is executed. In code 3, the immediate value of the number of runs is read from the 4th and 5th bits, but since 2 must be added as an offset, run = BITAT (code, 4,2) +2 is executed. Here, since BITAT (code, 4, 2) is an operation for reading a bit string of 2 bits forward from the 5th bit, it is noted that the 4th and 5th bits are read as a result. Next, the immediate value of the level value is read from the 7th bit and the 8th bit, but since it is necessary to add 1 as an offset, level = BITAT (code, 7, 2) +1 is executed. Since the sign bit may be read from the sixth bit, sign = BITAT (code, 5, 1) is executed.

  Similarly, in case 2 to case 6 of the switch statement, the immediate value of the run number and the immediate value of the level value are read from the immediate field according to the range of the number of runs and the level value range determined in each line, and an appropriate offset is added. What is necessary is just to perform an operation.

  FIG. 11 is a diagram illustrating an example of the encoding table 60 with an immediate field. The coding table 60 is further increased by two lines from the coding table 60 of FIG. The range of code run numbers, the range of level values, the bit length, and the number of appearances of each line are as shown in the figure.

The code properties of the encoding table 60 of FIG. 11 can be summarized as follows.
(1) A code of 3 to 12 bits is assigned to a level value 1 following 0 to 73 consecutive zeros.
(2) A code of 6 to 9 bits is assigned to level values 2 to 4 following 0 to 9 consecutive zeros.
(3) An 8-bit code is assigned to level values 4 to 7 following zero within one.
(4) A 16-bit code is assigned to level values 0 to 31 following 0 to 31 consecutive zeros.
(5) An 18-bit code is assigned to 32 or more consecutive level values.
(6) A 29-bit code is assigned to any other level value following any other consecutive zero.

  In Huffman coding, for a given image, a short code is assigned to a combination of run number and level value with high appearance frequency, and a long code is assigned to a combination of run number and level value with low appearance frequency. The encoding table is dynamically generated. On the other hand, in the variable length coding using the coding table 60 with the immediate field according to the present embodiment, the coding table 60 with the immediate field is not dynamically generated, but a predetermined one is used. It is done. Of course, a plurality of different encoding tables 60 with immediate fields may be prepared and used by switching to one of the tables under some condition. A given image is selected from the plurality of encoding tables 60 with immediate fields. A table having the smallest code amount when variable length coding is actually performed may be selected as the optimum table.

  According to the graphics processing apparatus of the present embodiment, the texture capacity that has been subjected to variable length coding using the coding table with an immediate field after discrete cosine transform is used, so that the texture capacity can be greatly reduced. Since the compute shader of the GPU 200 performs variable-length decoding of the compressed texture using the encoding table with an immediate field and performs inverse discrete cosine transform, the compressed texture can be decompressed at high speed and input to the graphics processing. Since the highly compressed texture can be resident in the memory, it is not necessary to read out a large-capacity texture from a storage device such as a hard disk, and the PRT can be executed on-memory. Since the compressed texture is on-memory, the latency is short and texture processing can be executed in real time even if the compressed texture is read out as needed, decompressed, and swapped into the PRT cache.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  In the above embodiment, the compressed texture is stored in the memory. However, the compressed texture may be stored in a recording medium such as a hard disk or an optical disk. Since the texture is highly compressed, the storage capacity can be suppressed, and the latency in reading from the recording medium can be suppressed to some extent although it does not meet the latency in the case of on-memory.

  In the above embodiment, the discrete cosine transform is used as an example of the spatial frequency transform for transforming the spatial region of the image into the spatial frequency region. However, other spatial frequency transforms such as a discrete Fourier transform may be used.

  In the above embodiment, the procedure for expanding the compressed texture in the configuration in which the GPU 200 includes the variable length decoding unit 30 and the IDCT unit 40 has been described. However, the variable length using the encoding table 60 with the immediate value field according to the present embodiment is described. Decoding can be used not only for decompressing compressed textures in a graphics processing apparatus, but also when decoding a variable-length encoded image in a general image processing apparatus.

  10 PRT control unit, 20 graphics operation unit, 30 variable length decoding unit, 40 inverse discrete cosine transform unit, 50 graphics processing unit, 60 encoding table with immediate field, 80 DCT block ring buffer, 100 main processor, 200 GPU 300 main memory, 310 compressed textures, 320 PRT cache, 330 page tables, 340 mipmap textures, 360 texture tile pool.

Claims (7)

Variable length decoding of a compressed image is executed based on an encoding table that assigns a code corresponding to a pair of run number range and level value range together with an immediate field indicating at least one of the immediate value of the run number and the immediate value of the level value. A variable length decoding unit;
An image decoding apparatus comprising: an inverse spatial frequency conversion unit that restores an image by performing inverse spatial frequency conversion on the variable length decoded image. A graphics processing device including a main memory and a graphics processing unit,
The graphics processing unit compresses textures based on an encoding table that assigns a code corresponding to a run number range and level value range pair together with an immediate field indicating at least one of an immediate value of a run number and an immediate value of a level value. A variable length decoding unit that performs variable length decoding, and an inverse spatial frequency conversion unit that restores texture by performing inverse spatial frequency conversion on the variable length decoded texture,
The graphics processing apparatus according to claim 1, wherein the main memory includes a texture pool that partially caches the restored texture.   The graphics processing apparatus according to claim 2, wherein the variable length decoding unit is executed by a plurality of threads of a compute shader.   4. The graphics processing apparatus according to claim 2, wherein the compressed texture is stored in the main memory, and the variable length decoding unit reads the compressed texture from the main memory. Variable length decoding of a compressed image is executed based on an encoding table that assigns a code corresponding to a pair of run number range and level value range together with an immediate field indicating at least one of the immediate value of the run number and the immediate value of the level value. Steps,
And a step of restoring the image by performing inverse spatial frequency conversion on the variable length decoded image. A graphics processing method in a graphics processing apparatus including a main memory and a graphics processing unit,
Graphics processing unit
A variable length of compressed texture based on a coding table that is assigned by the compute shader to a code corresponding to a run number range and level value range pair with an immediate field indicating at least one of the immediate value of the run number and the immediate value of the level value. Perform decryption,
A graphics processing method comprising: restoring a texture by performing inverse spatial frequency conversion on a variable length decoded texture; and storing the restored texture in the texture pool in the main memory that partially caches the texture . Variable length decoding of compressed texture is performed based on an encoding table that assigns codes corresponding to a pair of run number ranges and level value ranges together with an immediate field indicating at least one of the immediate value of the run number and the immediate value of the level value. Steps,
Restoring the texture by inverse spatial frequency transforming the variable length decoded texture and storing the restored texture in a texture pool that partially caches the texture, and causing the compute shader of the graphics processing unit to perform A program characterized by

Images