TW201941615A - Image processing device and method for operating image processing device - Google Patents

Abstract

An image processing device and method for operating image processing device are provided. The image processing device includes a multimedia intellectual property (IP) block which processes image data; a memory; and a frame buffer compressor (FBC) which compresses the image data to generate compressed data and stores the compressed data in the memory. The frame buffer compressor includes a logic circuit which determines a combination of a quantization parameter (QP) table and an entropy table and controls compression of the image data based on the determined combination of the QP table and the entropy table.

Description

Image processing device and method for operating image processing device

This disclosure relates to an image processing device and a method of operating an image processing device.

More and more applications require high-resolution video images and high frame rate images. Therefore, the amount of data accessed from the memory (ie, bandwidth) that stores these images through the various Intellectual Property (IP) blocks of the image processing device has greatly increased.

Each image processing device has limited processing capabilities. When the bandwidth increases, the processing capacity of the image processing device may reach this limit. As a result, users of image processing devices may experience slower speeds when recording or playing video images.

At least one embodiment of the inventive concept provides an image processing device that performs optimized image data compression.

At least one embodiment of the inventive concept provides a method for operating an image processing apparatus that performs optimized image data compression.

According to an exemplary embodiment of the inventive concept, an image processing device is provided, including: a multimedia intellectual property (IP) block that processes image data; a memory; and a frame buffer compressor (FBC) to compress an image Data to generate compressed data and store the compressed data in memory. The frame buffer compressor includes a logic circuit that determines a combination of a quantization parameter (QP) table and an entropy table, and controls compression of image data based on the determined combination of the QP table and the entropy table.

According to an exemplary embodiment of the inventive concept, an image processing device is provided, including: a multimedia IP block that processes image data; a memory; and a frame buffer compressor (FBC) that compresses the image data to generate compressed data, and The compressed data is stored in memory. The frame buffer compressor includes a logic circuit that determines a quantization parameter (QP) table containing up to 16 entries and an entropy table determined by a k value of up to 4 for entropy coding, and is based on the determined QP The combination of tables and entropy tables controls the compression of image data.

According to an exemplary embodiment of the inventive concept, a method for operating an image processing apparatus is provided, including: converting image data into predicted image data including prediction data and residual data; and determining a quantization parameter including a predetermined number of entries ( QP) table; using the determined QP table to quantify the predicted image data; using a predetermined number of k values for entropy coding to determine the entropy table; and using the determined entropy table to perform entropy coding on the quantized image data to Generate compressed data.

1 to 3 are block diagrams for explaining an image processing apparatus according to an exemplary embodiment of the inventive concept.

Referring to FIG. 1, an image processing apparatus according to an exemplary embodiment of the inventive concept includes a multimedia IP (Intellectual Property Right) 100 (eg, IP blocks and IP cores, circuits, etc.), a frame buffer compressor (FBC) 200 (eg, , Circuits, digital signal processors, etc.), memory 300 and system bus 400.

In the embodiment, the multimedia IP 100 is a part of the image processing device, which directly performs the image processing of the image processing device. The multimedia IP 100 may include a plurality of modules for recording and reproducing images such as photographic coding and playback of video images.

The multimedia IP 100 receives first data (eg, video data) from an external source such as a camera, and converts the first data into second data. For example, the first data may be moving image data or original image data. The second data is data generated by the multimedia IP 100, and may include data generated by the multimedia IP 100 that processes the first data. The multimedia IP 100 can repeatedly store the second data in the memory 300 and update the second data through multiple steps. The second information may contain all the information used in these steps. The second data may be stored in the memory 300 in the form of the third data. Therefore, the second data may be data before being stored in the memory 300 or after being read from the memory 300. This will be explained in more detail below.

In the exemplary embodiment, the multimedia IP 100 includes an image signal processor ISP 110, an oscillation correction module G2D 120, a multi-format codec MFC 130, a GPU 140, and a display 150. However, the inventive concept is not limited thereto. That is, the multimedia IP 100 may include at least one of an image signal processor 110, an oscillation correction module 120, a multi-format codec 130, a GPU 140, and a display 150. The multimedia IP 100 may be implemented by a processing module, such as a processor, which accesses the memory 300 to process data representing a moving image or a still image.

The image signal processor 110 receives the first data and pre-processes the first data to convert the first data into the second data. In an exemplary embodiment, the first data is RGB-type image source data. For example, the image signal processor 110 may convert the first data of the RGB type into the second data of the YUV type.

In the embodiment, the RGB type data means a data format representing colors based on the three primary colors of light. That is, it uses three types of colors (ie, red (RED), green (GREEN), and blue (BLUE)) to indicate the type of image. In contrast, the YUV type means a data format representing brightness separately, that is, a luma signal and a chroma signal. That is, Y means a luma signal, and U (Cb) and V (Cr) respectively mean a chroma signal. U means the difference between the lightness signal and the blue signal component, and V means the difference between the lightness signal and the red signal component.

YUV type data can be obtained by converting RGB type data using a conversion formula. For example, conversion formulas such as Y = 0.3R + 0.59G + 0.11B, U = (B-Y) × 0.493, and V = (R-Y) × 0.877 can be used to convert RGB type data into YUV type data.

Because human eyes are sensitive to lightness signals and less sensitive to color signals, it is easier to compress YUV-type data than RGB-type data. Therefore, the image signal processor 110 can convert the first data of the RGB type into the second data of the YUV type.

The image signal processor 110 converts the first data into the second data, and then stores the second data in the memory 300.

The oscillation correction module 120 may perform oscillation correction of still image data or moving image data. The oscillation correction module 120 may perform the oscillation correction by reading the first data or the second data stored in the memory 300. In an embodiment, the oscillation correction means detecting the oscillation from the camera of the moving image data and removing the oscillation from the moving image data.

The oscillation correction module 120 may correct the oscillation of the first data or the second data to update the first data or the second data and store the updated data in the memory 300.

The multi-format codec 130 may be a codec for compressing moving image data. Generally speaking, since the size of moving image data is very large, a compression module for reducing its size is necessary. The moving image data may be compressed through association between multiple frames, and this compression may be performed by the multi-format codec 130. The multi-format codec 130 can read and compress the first data or the second data stored in the memory 300.

The multi-format codec 130 may compress the first data or the second data to generate the updated first data or the updated second data, and store the updated data in the memory 300.

A graphics processing unit (GPU) 140 may perform arithmetic processing to generate two-dimensional or three-dimensional graphics. The GPU 140 may perform arithmetic processing on the first data or the second data stored in the memory 300. The GPU 140 may be specifically used for graphics data processing to process portions of the graphics data in parallel.

The GPU 140 may compress the first data or the second data to generate updated first data or updated second data, and store the updated data in the memory 300.

The display 150 can display the second data stored in the memory 300 on the screen. The display 150 can display image data processed by the components of the multimedia IP 100, which are an image signal processor 110, an oscillation correction module 120, a multi-format codec 130, and a GPU 140. However, the inventive concept is not limited to these examples.

The video signal processor 110, the oscillation correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 can be separately operated. That is, the image signal processor 110, the oscillation correction module 120, the multi-format codec 130, the GPU 140, and the display 150 can individually access the memory 300 to write or read data.

In an embodiment, the frame buffer compressor 200 compresses the second data to convert the second data into the third data before the multimedia IP 100 separately accesses the memory 300. The frame buffer compressor 200 transmits the third data to the multimedia IP 100, and the multimedia IP 100 transmits the third data to the memory 300.

Therefore, the third data compressed by the frame buffer compressor 200 is stored in the memory 300. On the contrary, the third data stored in the memory 300 can be loaded by the multimedia IP 100 and transmitted to the frame buffer compressor 200. In an embodiment, the frame buffer compressor 200 decompresses the third data to convert the third data into the second data. The frame buffer compressor 200 may transmit the second data (ie, decompressed data) to the multimedia IP 100.

In the embodiment, whenever the image signal processor 110, the oscillation correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 separately access the memory 300, the frame buffer compressor 200 will The second data is compressed into third data and the third data is transmitted to the memory 300. For example, after one of the components of the multimedia IP 100 generates the second data and stores the second data in the memory 300, the frame buffer compressor 200 can compress the stored data and store the compressed data to the memory 300 in. In the embodiment, whenever a data request is transmitted from the memory 300 to the image signal processor 110, the oscillation correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP, the frame buffer compressor 200 decompresses the third data into the second data, and transmits the second data to the image data processor 110, the oscillation correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100, respectively.

The memory 300 stores the third data generated by the frame buffer compressor 200, and the stored third data can be provided to the frame buffer compressor 200 so that the frame buffer compressor 200 can decompress the third data.

In the embodiment, the multimedia IP 100 and the memory 300 are connected to the system bus 400. Specifically, the image signal processor 110, the oscillation correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 may be separately connected to the system bus 400. The system bus 400 may be a path. The image signal processor 110, the oscillation correction module 120, the multi-format codec 130, the GPU 140, the display 150, and the memory 300 of the multimedia IP 100 transmit data to each other through the path. .

The frame buffer compressor 200 is not connected to the system bus 400, and when the video signal processor 110, the oscillation correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 respectively access the memory , Performing operations of converting the second data into third data (for example, image compression) and converting the third data into second data (for example, image decompression).

Next, referring to FIG. 2, the frame buffer compressor 200 of the image processing apparatus according to an exemplary embodiment of the inventive concept is directly connected to the system bus 400.

The frame buffer compressor 200 is not directly connected to the multimedia IP 100 and is connected to the multimedia IP 100 via the system bus 400. Specifically, each of the image signal processor 110, the oscillation correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 can transmit data to the frame buffer via the system bus 400. The compressor 200 and the frame buffer compressor 200 transmit data, and thus the data can be transmitted to the memory 300.

That is, during the compression process, each of the image signal processor 110, the oscillation correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 may transfer the second through the system bus 400. The data is transmitted to the frame buffer compressor 200. Subsequently, the frame buffer compressor 200 can compress the second data into the third data, and transmit the third data to the memory 300 via the system bus 400.

Similarly, even during the decompression process, the frame buffer compressor 200 can receive the third data stored in the memory 300 via the system bus 400 and can decompress it into the second data. Subsequently, the frame buffer compressor 200 can transmit the second data to the image signal processor 110, the oscillation correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 via the system bus 400.

Referring to FIG. 3, in an image processing apparatus according to an exemplary embodiment of the inventive concept, a memory 300 and a system bus 400 are connected to each other via a frame buffer compressor 200.

That is, the memory 300 is not directly connected to the system bus 400, but is only connected to the system bus 400 via the frame buffer compressor 200. In addition, the image signal processor 110, the oscillation correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 are directly connected to the system bus 400. Therefore, the image signal processor 110, the oscillation correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 access the memory 300 only through the frame buffer compressor 200.

In this specification, the second data is referred to as video data 10 and the third data is referred to as compressed data 20.

FIG. 4 is a block diagram for explaining the frame buffer compressor of FIGS. 1 to 3 in detail.

Referring to FIG. 4, the frame buffer compressor 200 includes an encoder 210 (eg, an encoding circuit) and a decoder 220 (eg, a decoding circuit).

The encoder 210 may receive the image data 10 from the multimedia IP 100 to generate the compressed data 20. The image data 10 can be transmitted from each of the image signal processor 110, the oscillation correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100. The compressed data 20 can be transmitted to the memory 300 via the multimedia IP 100 and the system bus 400.

Instead, the decoder 220 can decompress the compressed data 20 stored in the memory 300 into image data 10. The image data 10 can be transmitted to the multimedia IP 100. The image data 10 can be transmitted to each of the image signal processor 110, the oscillation correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100.

FIG. 5 is a block diagram for explaining the encoder of FIG. 4 in detail.

Referring to FIG. 5, the encoder 210 includes a first mode selector 219 (for example, a logic circuit), a prediction module 211 (for example, a logic circuit), a quantization module 213 (for example, a logic circuit), and an entropy encoding module 215 (for example, , Logic circuit) and padding module 217 (eg, logic circuit).

In an embodiment, the first mode selector 219 determines whether the encoder 210 operates in a lossless mode or a lossy mode. When the encoder 210 operates in the lossless mode according to the first mode selector 219, the image data 10 is compressed along the lossless path (Lossless) of FIG. 5, and when the encoder 210 operates in the lossy mode, it follows the lossy path ( Lossy) compressed image data10.

The first mode selector 219 may receive a signal from the multimedia IP 100, which is used to determine whether to perform lossless compression or perform lossy compression. Lossless compression means compression without loss of data. The compression ratio can be changed depending on the data losslessly compressed. Unlike lossless compression, lossy compression is one in which data is partially lost. Lossy compression has a higher compression ratio than lossless compression, and may have a fixed compression ratio set in advance.

In the case of the lossless mode, the first mode selector 219 enables the image data 10 to flow along the lossless path to the prediction module 211, the entropy encoding module 215, and the padding module 217. In contrast, in the lossy mode, the first mode selector 219 enables the image data 10 to flow along the lossy path to the prediction module 211, the quantization module 213, and the entropy coding module 215.

The prediction module 211 can compress the image data 10 by dividing the image data 10 into prediction data and residual data. The prediction data and residual data together occupy less space than the image data10. In the embodiment, the prediction data is image data of one pixel of the image data, and the residual data is generated from the difference between the prediction data of the pixels of the image data adjacent to one pixel and the image data. For example, if the image data of a pixel has a difference between 0 and 255, 8 bits may be required to represent this value. When the neighboring pixels have a value similar to that of one pixel, the residual data of each of the neighboring pixels is much smaller than the predicted data, and thus the number of data bits representing the image data 10 can be greatly reduced. For example, when pixels with values of 253, 254, and 255 are continuous, if the prediction data is set to 253, it means that (253 (forecast), 1 (residual), and 2 (residual)) residual data is sufficient , And the number of bits per pixel representing these residual data can be reduced from 8 bits to 2 bits. For example, the 24-bit data of 253, 254, and 255 can be due to the 8-bit prediction data of 253 (11111101), the 2-bit residual data of 254-251 = 1 (01), and 255-253 = 2 (10 ) 'S 2-bit residual data is reduced to 12 bits.

Therefore, the prediction module 211 can compress the total size of the image data 10 by dividing the image data 10 into prediction data and residual data. Various methods can be used to set the type of prediction data.

The prediction module 211 may perform prediction based on pixels, or may perform prediction based on blocks. In this case, a block may mean an area formed by a plurality of adjacent pixels. For example, pixel-based prediction may mean that all residual data is generated from one of the pixels, and block-based prediction may mean that residual data is generated for each block from a pixel corresponding to the block.

The quantization module 213 can further compress the image data 10 compressed by the prediction module 211. In the exemplary embodiment, the quantization module 213 removes the lower bits of the image data 10 via a preset quantization coefficient. Specifically, the representative value is selected by multiplying the data by the quantization coefficient, but a loss may occur by truncating the decimal part. If the value of the pixel data is between 0 and 2 ^8-1 (= 255), the quantization coefficient can be defined as / (2 ⁿ -1) (where n is an integer less than or equal to 8). However, the embodiments of the present invention are not limited thereto. For example, if the prediction data is 253 (11111101), the prediction data can be reduced from 8 bits to 6 bits by removing the lower 2 bits, which results in (111111) 252 prediction data.

However, the removed lower bits were not recovered later, and were therefore lost. Therefore, the quantization module 213 is used only in the lossy mode. However, since the lossy mode has a relatively higher compression ratio than the lossless mode and may have a fixed compression ratio set in advance, information on the compression ratio is not required separately later.

The entropy coding module 215 can compress the image data 10 compressed by the quantization module 213 in a lossy mode or the image data 10 compressed by the prediction module 211 through the entropy coding in a lossless mode. In an embodiment, the entropy coding uses a method of determining the number of allocated bits based on the video rate.

In the exemplary embodiment, the entropy encoding module 215 uses Huffman encoding to compress the image data 10. In an alternative embodiment, the entropy coding module 215 compresses the image data 10 via exponential golomb coding or golomb rice coding. In the embodiment, the entropy encoding module 215 determines the entropy encoding value (for example, the k value) according to the data to be compressed, generates a graph according to the k value, and uses the graph to compress the image data 10.

The padding module 217 can perform padding on the image data 10 compressed by the entropy encoding module 215 in a lossless mode. In this article, padding can mean adding meaningless data to match a particular size. This will be explained in more detail below.

The padding module 217 can be enabled not only in a lossless mode but also in a lossy mode. In the lossy mode, when compressed by the quantization module 213, the image data 10 can be further compressed than the required compression ratio. In this case, even in a lossy mode, the image data 10 can be converted into compressed data 20 via the padding module 217 and transmitted to the memory 300. In the exemplary embodiment, the padding module 217 is omitted so that padding is not performed.

The compression management module 218 determines a combination of a quantization parameter (QP) table and an entropy table for quantization and entropy coding, and controls the compression of the image data 10 according to the determined combination of the QP table and the entropy table.

In this case, the first mode selector 219 determines that the encoder 210 operates in a lossy mode, and thus the image data 10 is compressed along a lossy path (Lossy) of FIG. 5. That is, the compression management module 218 determines the combination of the QP table and the entropy table and compresses the configuration of the image data 10 according to the determined combination of the QP table and the entropy table. The frame buffer compressor 200 uses a lossy compression algorithm. It is assumed that the image data 10 is compressed.

Specifically, the QP table includes one or more entries, and each entry may include a quantization coefficient for quantizing the image data 10.

In an embodiment, the entropy table includes multiple encoding tables that use k-value discrimination to perform an entropy encoding algorithm. An entropy table that can be used in some embodiments of the inventive concept includes at least one of exponential Columbus coding and Columbus coding.

The compression management module 218 determines a QP table containing a predetermined number of entries, and the frame buffer compressor 200 performs the quantization of the predicted image data 10 using the determined QP table. In addition, the compression management module 218 uses a predetermined number of k values to determine the entropy table, and the frame buffer compressor 200 uses the determined entropy table to perform entropy encoding on the quantized image data 10. That is, the frame buffer compressor 200 generates the compressed data 20 based on the combination of the QP table and the entropy table determined by the compression management module 218.

Thereafter, the frame buffer compressor 200 may write the generated compressed data 20 into the memory 300. Similarly, the frame buffer compressor 200 can read the compressed data 20 from the memory 300, and decompress the read compressed data 20 to generate decompressed data to provide the decompressed data to the multimedia IP 100.

More details of the compression management module 218 for performing such operations will be described later with reference to FIGS. 7 to 16.

FIG. 6 is a block diagram for explaining the encoder of FIG. 4 in detail.

Referring to FIG. 6, the decoder 220 includes a second mode selector 229 (for example, a logic circuit), an unfilled module 227 (for example, a logic circuit), an entropy decoding module 225 (for example, a logic circuit), and an inverse quantization module 223. (Eg, a logic circuit) and the prediction compensation module 221 (eg, a logic circuit).

The second mode selector 229 determines whether the compressed data 20 stored in the memory 300 has been compressed in a lossless manner or in a lossy manner. In an exemplary embodiment, the second mode selector 229 determines whether the compressed data 20 has been compressed by a lossless mode or a lossy mode via the presence or absence of a header. This will be explained in more detail below.

In the case of the lossless mode, the second mode selector 229 enables the compressed data 20 to flow along the lossless path to the unfilled module 227, the entropy decoding module 225, and the prediction compensation module 221. In contrast, in the case of the lossy mode, the second mode selector 229 enables the compressed data 20 to flow along the lossy path to the entropy decoding module 225, the inverse quantization module 223, and the prediction compensation module 221.

The unfilled module 227 removes the padded portion of the data filled by the padded module 217 of the encoder 210. When the filling module 217 is omitted, the unfilled module 227 may be omitted.

The entropy decoding module 225 can decompress the data compressed by the entropy encoding module 215. The entropy decoding module 225 may perform decompression via Huffman coding, exponential Columbus coding, or Columbus coding. Since the compressed data 20 includes a k value, the entropy decoding module 225 may use the k value to perform decoding.

The inverse quantization module 223 can decompress the data compressed by the quantization module 213. The inverse quantization module 223 can recover the compressed data 20 compressed using the quantization coefficient determined by the quantization module 213, but it is impossible to completely recover the part lost during the compression process. Therefore, the inverse quantization module 223 is used only in the lossy mode.

The prediction compensation module 221 can recover the data represented by the prediction data and the residual data generated by the prediction module 211. The prediction compensation module 221 may, for example, convert residual data representations (253 (forecast), 1 (residual), and 2 (residual)) into 253, 254, and 255. For example, the prediction compensation module 221 may recover data by adding residual data to the prediction data.

The prediction compensation module 221 may recover the prediction performed in units of pixels or blocks according to the prediction module 211. Therefore, the compressed data 20 can be restored or decompressed, and can be transmitted to the multimedia IP 100.

When decompressing the compressed data 20, the decompression management module 228 may perform a combination of the QP table and the entropy table that can appropriately reflect the determination of the QP table and the entropy table determined by the compression management module 218 described above with reference to FIG. jobs.

The operation of the image processing apparatus described above will now be described with reference to FIGS. 7 to 16.

7 to 10 are diagrams for explaining an operation of an image processing apparatus according to an exemplary embodiment of the inventive concept.

Referring to FIG. 7, the compression management module 218 determines a QP table 230 containing a predetermined number of entries. In this embodiment, the compression management module 218 determines a QP table 230 including eight entries that achieves eight levels of quantization.

The QP table 230 includes a first entry and a second entry. Here, the first entry is a quantization coefficient corresponding to the index 0, and the second entry is a quantization coefficient corresponding to the index 7. In addition, the QP table 230 contains one or more third entries between the first entry and the second entry. In an embodiment of the present invention, the one or more third entries correspond to a quantization coefficient, which corresponds to an index 1 to an index 6.

The first entry is determined to be a predetermined first value. In the embodiment of the present invention, the predetermined first value is represented by "MaxShiftValue". The first value is a constant value arbitrarily determined in an image processing device or an actual implementation of a method for operating an image processing device according to various embodiments of the inventive concept, as required.

After the first entry is determined as the first value, the second entry is determined by the following equation:

MaxShiftValue ＞＞ BitDepth * (1-CompressionRatio)

Here, MaxShiftValue represents the aforementioned first value, BitDepth represents the bit depth of the image data 10, and CompressionRatio represents the compression ratio.

That is, when the first values are equal and the target compression ratio is equal, the compression management module 218 may set the final value of the QP table 230 differently according to the bit depth of the image data 10.

In this manner, after determining the first entry as the first value and the second entry as the second value, the third entry may be determined via sampling. That is, in the QP table 230 containing a predetermined number of entries, for the remaining third entries other than the first and second entries, the third entry may be sampled so that the quantization coefficients are appropriately distributed.

In another aspect, the compression management module 218 may use a predetermined number of k values to determine the entropy table. In some embodiments of the present disclosure, the entropy table is determined by a k value of 4 or less.

Specifically, the entropy table may be defined by the k values of n, n + 1, n + 2, and n + 3 (where n is 0 or an integer greater than 0) of the image data 10 having the first bit depth, and may be N + a, n + a + 1, n + a + 2, and n + a + 3 (where a is 1 or greater than 1) by image data 10 having a second bit depth greater than the first bit depth Integer).

For example, the entropy table can be determined by the k values of 0, 1, 2, and 3 of the image data with a first bit depth of 8 bits, and can be determined by a second bit depth of 10 bits. Determine the k value of 1, 2, 3 and 4 of the image data.

That is, the compression management module 218 may set, for example, four consecutive k values differently according to the bit depth of the image data 10.

When performing block-based lossy compression of the image data 10, prediction, quantization, and entropy encoding steps are performed as described above. In the case of quantization, since the QP table containing quantization factors has more entries, it is possible to perform compression more precisely. However, memory resources are consumed to maintain a QP table with multiple entries, and the bit of the stream header 20 of the compressed data 20 transmitted and received between the encoder 210 and the decoder 220 of the frame buffer controller 200 can be increased Yuan number. In particular, in an environment in which the image data 10 is compressed in units of blocks, when considering the overall image size, even if the stream header of the compressed data 20 is increased by only one bit, a large amount of bandwidth can still be consumed.

In addition, in the case of entropy coding, since the distribution of the residual signal (residue) is different between the case of image data having a bit depth of 8 bits and the case of image data having a bit depth of 10 bits, for example, The entropy table used for entropy coding also needs to be determined differently. For example, since the tendency for the values of the entropy table entries to converge to 0 becomes lower in the case of image data with a bit depth of 10 bits than in the case of image data with a bit depth of 8 bits, It is necessary to determine the value of k differently depending on the bit depth.

Therefore, by determining a QP table of a suitable size with good compression quality and selecting an appropriate k value in consideration of the distribution of the residual signal to determine the entropy table, the bandwidth between the frame buffer compressor 200 and the memory 300 can be reduced. Small, while enhancing the compression performance of the image data10.

In some embodiments of the inventive concept, the compression management module 218 determines a QP table containing a maximum of 16 entries. Similarly, in some embodiments of the present disclosure, the compression management module 218 determines that the QP table contains eight or more entries.

Next, referring to FIG. 8, the compression management module 218 determines a QP table 232 including eight entries.

As an example, assume that the first entry of QP table 232 is determined to be 4096. The compression management module 218 performs a bit shift operation on 4096 corresponding to the first value by BitDepth * (1-CompressionRatio) as follows.

4096 ＞＞ 8 × (1-0.5) = 256

Here, 8 is a value corresponding to the bit depth of the image data 10, and 0.5 (for example, 50%) is a value corresponding to the target compression ratio.

In this manner, after determining the first entry as 4096 and the second entry as 256, the third entry may be determined via sampling. That is, in the QP table 232 containing eight entries, the third entry may be determined to be, for example, 1365, 819, 585, 455, 372, and 315, so that for the corresponding entries except the first entry 4096 and the second entry The third entry of Remaining Index 1 to Remaining Index 6 other than 256 distributes quantization coefficients appropriately. The entries of the QP table 232 thus determined can also be used as quantization coefficients by dividing by 4096, which is, for example, the value of MaxShiftValue.

In an embodiment, the compression management module 218 determines that the k value is 0, 1, 2, and 3 when the image data 10 has a bit depth of 8 bits, and determines that the image data 10 has a bit depth of 10 bits. The k values are 1, 2, 3, and 4.

In this way, since the compression management module 218 determines the combination of the QP table and the entropy table respectively used for quantization and entropy coding, and controls the compression of the image data 10 according to the determined combination of the QP table and the entropy table, it is possible to reduce The frame buffers the bandwidth between the compressor 200 and the memory 300 while enhancing the compression performance of the image data 10.

Next, referring to FIG. 9, the compression management module 218 determines a QP table 234 containing eight entries.

For example, suppose the first entry of QP table 234 is determined to be 4096. The compression management module 218 performs a bit shift operation on 4096 corresponding to the first value by BitDepth * (1-CompressionRatio) as follows.

4096 ＞＞ 10 × (1-0.5) = 128

Here, 10 is a value corresponding to the bit depth of the image data 10, and 0.5 is a value corresponding to the target compression ratio.

In this way, after determining the first entry as 4096 and the second entry as 128, the third entry may be determined via sampling. That is, in the QP table 234 containing eight entries, the third entry may be determined to be, for example, 819, 455, 293, 216, 171, and 146, so that for the corresponding entries except the first entry 4096 and the second entry The third entry of the remaining indices 1 to 6 other than 128, appropriately distributes the quantization coefficients. The entries of the QP table 234 judged in this way can also be used as quantization coefficients by dividing by 4096 which is, for example, the value of MaxShiftValue.

In an embodiment, the compression management module 218 determines that the k value is 0, 1, 2, and 3 when the image data 10 has a bit depth of 8 bits, and determines that the image data 10 has a bit depth of 10 bits. The k values are 1, 2, 3, and 4.

In this way, the compression management module 218 determines the combination of the QP table and the entropy table, which are respectively used for quantization and entropy coding, and controls the compression of the image data 10 according to the determined combination of the QP table and the entropy table. Therefore, it is possible to reduce the bandwidth between the frame buffer compressor 200 and the memory 300 while enhancing the compression performance of the image data 10.

Next, referring to FIG. 10, the compressed bit stream 236 generated by the encoder 210 of the frame buffer compressor 200 through the aforementioned processing as described above includes a header, which includes 4-bit QP table information and 2-bit k-value information and contains compressed data 20 as compressed binary after the header.

11 to 14 are diagrams for explaining an operation of an image processing apparatus according to an exemplary embodiment of the present disclosure.

Referring to FIG. 11, the compression management module 218 determines a QP table 240 containing a predetermined number of entries. In this embodiment, the compression management module 218 determines a QP table 240 with 16 entries that achieves 16-level quantization.

The QP table 240 includes a first entry and a second entry. Here, the first entry is a quantization coefficient corresponding to the index 0, and the second entry is a quantization coefficient corresponding to the index 15. In addition, the QP table 240 contains one or more third entries between the first entry and the second entry. In an embodiment of the present invention, the one or more third entries correspond to a quantization coefficient, which corresponds to an index of 1 to 14.

After determining the first entry as the first value and the second entry as the second value, the third entry may be determined via sampling. That is, in the QP table 240 containing a predetermined number of entries, the third entry may be sampled so that the quantization coefficients are appropriately distributed for the remaining third entries other than the first entry and the second entry.

In an embodiment, the compression management module 218 may use a predetermined number of k values to determine the entropy table. In some embodiments of the inventive concept, the entropy table may be determined by a value of k that is at most 4 or less.

For example, the entropy table can be determined by k values of 0, 1, 2, and 3 with image data having a first bit depth of 8 bits, and can be determined by having a second bit depth of 10 bits Determine the k value of 1, 2, 3 and 4 of the image data.

That is, the compression management module 218 may set, for example, four consecutive k values differently according to the bit depth of the image data 10.

Next, referring to FIG. 12, the compression management module 218 determines a QP table 242 containing 16 entries.

For example, suppose the first entry in QP table 242 is determined to be 4096. The compression management module 218 performs a bit shift operation on 4096 corresponding to the first value by BitDepth * (1-CompressionRatio) as follows.

4096 ＞＞ 8 × (1-0.5) = 256

Here, 8 is a value corresponding to the bit depth of the image data 10, and 0.5 is a value corresponding to the target compression ratio.

In this manner, after determining the first entry as 4096 and the second entry as 256, the third entry may be determined via sampling. That is, in the QP table 242 containing 16 entries, the third entry may be determined to be, for example, 2048, 1365, 1024, 819, 683, 585, 512, 455, 410, 372, 341, 315, 293, and 273 , So that for the third entry corresponding to the remaining index 1 to the remaining index 14 other than the first entry 4096 and the second entry 256, the quantization coefficients are appropriately distributed. The entries of the QP table 242 thus determined can also be used as quantization coefficients by dividing by 4096, which is, for example, the value of MaxShiftValue.

In an embodiment, the compression management module 218 may determine that the k values are 0, 1, 2, and 3 when the image data 10 has a bit depth of 8 bits, and may determine that the image data 10 has a bit depth of 10 bits At that time, it is determined that k values are 1, 2, 3, and 4.

In this way, since the compression management module 218 determines the combination of the QP table and the entropy table respectively used for quantization and entropy coding, and controls the compression of the image data 10 according to the determined combination of the QP table and the entropy table, it is possible to reduce The small frame buffers the bandwidth between the compressor 200 and the memory 300 while enhancing the compression performance of the image data 10.

Next, referring to FIG. 13, the compression management module 218 determines a QP table 244 containing 16 entries.

Assume that the first entry of QP table 244 is determined to be 4096. The compression management module 218 performs a bit shift operation on 4096 corresponding to the first value by BitDepth * (1-CompressionRatio) as follows.

4096 ＞＞ 10 × (1-0.5) = 128

Here, 10 is a value corresponding to the bit depth of the image data 10, and 0.5 is a value corresponding to the target compression ratio.

In this way, after determining the first entry as 4096 and the second entry as 128, the third entry may be determined via sampling. That is, in the QP table 244 containing 16 entries, the third entry may be determined to be, for example, 1365, 819, 585, 455, 372, 315, 273, 228, 205, 186, 171, 158, 146, and 137 , So that for the third entry corresponding to the remaining index 1 to the remaining index 14 other than the first entry 4096 and the second entry 128, the quantization coefficients are appropriately distributed. The entries of the QP table 244 thus determined can also be used as quantization coefficients by dividing by 4096, which is, for example, the value of MaxShiftValue.

In an embodiment, the compression management module 218 may determine that the k values are 0, 1, 2, and 3 when the image data 10 has a bit depth of 8 bits, and may determine that the image data 10 has a bit depth of 10 bits At that time, it is determined that k values are 1, 2, 3, and 4.

In this way, since the compression management module 218 determines the combination of the QP table and the entropy table respectively used for quantization and entropy coding, and controls the compression of the image data according to the determined combination of the QP table and the entropy table, it is possible to reduce The frame buffers the bandwidth between the compressor 200 and the memory 300 while enhancing the compression performance of the image data 10.

Next, referring to FIG. 14, the compressed bit stream 246 generated by the encoder 210 of the frame buffer compressor 200 through the aforementioned processing as described above includes a header, which includes 4-bit QP table information and 2-bit k-value information and contains compressed data 20 as compressed binary after the header.

FIG. 15 is a diagram for explaining an advantageous effect of an image processing apparatus and a method of operating the image processing apparatus according to some exemplary embodiments of the inventive concept.

Referring to FIG. 15, when the number of entries of the QP table is 12 and the number of k values is 4, the best PSNR gain is shown.

That is, by determining a QP table of a suitable size with good compression quality, and determining an entropy table by considering the distribution of the residual signal to select an appropriate k value, the frame buffer compressor 200 and the memory 300 can be reduced. Bandwidth, while enhancing the compression performance of the image data10.

FIG. 16 is a flowchart illustrating a method for operating an image processing apparatus according to an exemplary embodiment of the inventive concept.

Referring to FIG. 16, a method of operating an image processing apparatus according to an exemplary embodiment of the inventive concept includes determining a QP table containing a predetermined number of entries (S1601).

In an embodiment of the inventive concept, a QP table determined to contain a predetermined number of entries includes a QP table determined to contain a maximum of 16 entries. In another embodiment of the inventive concept, a QP table determined to contain a predetermined number of entries includes a QP table determined to contain eight or more entries.

In addition, the above method includes performing quantization on the predicted image data using the determined QP table (S1603).

In addition, the method includes determining an entropy table using a predetermined number of k values (S1605).

In an embodiment of the inventive concept, determining the entropy table includes determining the entropy table using a value of k that is at most 4 or less.

In addition, the above method includes performing entropy encoding on the quantized image data using the determined entropy table to generate compressed data (S1607).

Those of ordinary skill in the art will appreciate that many variations and modifications can be made to the exemplary embodiments without substantially departing from the principles of the inventive concept.

10‧‧‧Image data

20‧‧‧ Compressed data

100‧‧‧Multimedia intellectual property rights

110‧‧‧Image Signal Processor

120‧‧‧Oscillation Correction Module

130‧‧‧Multi-format codec

140‧‧‧Graphics Processing Unit

150‧‧‧ Display

200‧‧‧Frame buffer compressor

210‧‧‧ Encoder

211‧‧‧ Forecast Module

213‧‧‧Quantitative module

215‧‧‧Entropy coding module

217‧‧‧ Fill module

218‧‧‧Compression Management Module

219‧‧‧Mode selector

220‧‧‧ decoder

221‧‧‧ Forecast Compensation Module

223‧‧‧Inverse quantization module

225‧‧‧ Entropy Decoding Module

227‧‧‧Unfilled Module

228‧‧‧Unzip Management Module

229‧‧‧Mode selector

230, 232, 234, 240, 242, 244‧‧‧

236, 246‧‧‧compressed bit stream

300‧‧‧Memory

400‧‧‧System Bus

S1601, S1603, S1605, S1607‧‧‧ steps

The present invention will become more apparent by describing its exemplary embodiments in detail with reference to the accompanying drawings, in which: FIGS. 1 to 3 are block diagrams for explaining an image processing apparatus according to some embodiments of the inventive concept. FIG. 4 is a block diagram for explaining the frame buffer compressor of FIGS. 1 to 3 in detail. FIG. 5 is a block diagram for explaining the encoder of FIG. 4 in detail. FIG. 6 is a block diagram for explaining the decoder of FIG. 4 in detail. 7 to 10 are diagrams for explaining an operation of an image processing apparatus according to an exemplary embodiment of the inventive concept. 11 to 14 are diagrams for explaining an operation of an image processing apparatus according to an exemplary embodiment of the inventive concept. FIG. 15 is a diagram for explaining an advantageous effect of an image processing apparatus and a method of operating the image processing apparatus according to an exemplary embodiment of the inventive concept. FIG. 16 is a flowchart illustrating a method for operating an image processing apparatus according to an exemplary embodiment of the inventive concept.

Claims (20)

An image processing device includes: a multimedia intellectual property (IP) block configured to process image data; a memory; and a frame buffer compressor (FBC) configured to compress the image data to generate compression Data and storing the compressed data in the memory, wherein the frame buffer compressor includes a logic circuit configured to determine a combination of a quantization parameter (QP) table and an entropy table and based on the determined said A combination of a QP table and the entropy table controls the compression of the image data. The image processing device according to item 1 of the scope of patent application, wherein the frame buffer compressor writes the compressed data generated based on a combination of the QP table and the entropy table into the memory. The image processing device according to item 1 of the scope of patent application, wherein the frame buffer compressor reads the compressed data from the memory, decompresses the read compressed data to generate decompressed data, And providing the decompressed data to the multimedia IP block. The image processing device according to item 1 of the scope of patent application, wherein the frame buffer compressor compresses the image data using a lossy compression algorithm. The image processing device according to item 1 of the scope of patent application, wherein the QP table includes a first entry and a second entry, the first entry is a predetermined first value, and the second entry is borrowed Determined by the following equation: MaxShiftValue >> BitDepth * (1-CompressionRatio) where MaxShiftValue is the predetermined first value, BitDepth is the bit depth of the image data, and CompressionRatio is the compression ratio. The image processing apparatus according to item 5 of the scope of patent application, wherein the QP table includes one or more third entries between the first entry and the second entry, and the third Entries are determined by sampling. The image processing device according to item 1 of the patent application range, wherein the entropy table is determined by a k value of 4 or less for entropy coding. The image processing apparatus according to item 7 of the scope of patent application, wherein when the image data has a first bit depth, the k value includes values of n, n + 1, n + 2, and n + 3, and When the image data has a second bit depth greater than the first bit depth, the k value includes one of n + a, n + a + 1, n + a + 2, and n + a + 3. Value, where n is an integer> = 0 and a is an integer> = 1. An image processing device includes: a multimedia intellectual property (IP) block configured to process image data; a memory; and a frame buffer compressor (FBC) configured to compress the image data to generate compression Data and storing the compressed data in the memory, wherein the frame buffer compressor includes a logic circuit configured to determine a quantization parameter (QP) table containing up to 16 entries and borrow An entropy table determined by a maximum k value of 4 used for entropy coding, and the compression of the image data is controlled according to the determined combination of the QP table and the entropy table. The image processing apparatus according to item 9 of the scope of patent application, wherein the QP table includes a first entry and a second entry, the first entry is a predetermined first value, and the second entry is borrowed Determined by the following equation: MaxShiftValue >> BitDepth * (1-CompressionRatio) where MaxShiftValue is the predetermined first value, BitDepth is the bit depth of the image data, and CompressionRatio is the compression ratio. The image processing apparatus according to claim 10, wherein the QP table includes one or more third entries between the first entry and the second entry, wherein the third entry Entries are determined by sampling. The image processing device according to item 9 of the scope of patent application, wherein when the image data has a first bit depth, the k value includes values of n, n + 1, n + 2, and n + 3, and When the image data has a second bit depth greater than the first bit depth, the k value includes one of n + a, n + a + 1, n + a + 2, and n + a + 3. Value, where n is an integer> = 0 and a is an integer> = 1. A method for operating an image processing device, the method comprising: converting image data into predicted image data including prediction data and residual data; determining a quantization parameter (QP) table containing a predetermined number of entries; using the determined The QP table to quantify the predicted image data to generate quantized image data; use a predetermined number of k values for entropy encoding to determine an entropy table; and use the determined entropy table to quantize the quantized image The data performs the entropy coding to generate compressed data. The method according to item 13 of the scope of patent application, wherein determining the QP table including the predetermined number of entries includes: determining the QP table including a maximum of 16 entries. The method according to item 14 of the scope of patent application, wherein determining the QP table containing the predetermined number of entries comprises: determining a QP table containing eight or more entries. The method according to item 13 of the scope of patent application, wherein the QP table includes a first entry and a second entry, and the QP table judged to include the predetermined number of entries includes: determining the first entry As the predetermined first value, and the second entry is determined by the following equation: MaxShiftValue >> BitDepth * (1-CompressionRatio) where MaxShiftValue is the predetermined first value and BitDepth is the position of the image data Element depth, and CompressionRatio is the compression ratio. The method according to item 16 of the scope of patent application, wherein the QP table includes one or more third entries between the first entry and the second entry, and it is determined to include the predetermined number The QP table of entries includes: determining the third entry via sampling. The method of claim 13, wherein determining the entropy table includes: determining the entropy table using the maximum k or less than 4 for the entropy coding. The method according to item 13 of the patent application scope, wherein determining the entropy table includes: when the image data has a first bit depth, using the n, n + 1, n + 2, and n + 3 The k value determines the entropy table, and when the image data has a second bit depth greater than the first bit depth, n + a, n + a + 1, n + a + 2, and n + The k value of a + 3 determines the entropy table, where n is an integer> = 0, and a is an integer> = 1. The method of claim 13, wherein the entropy table includes at least one of an exponential golomb coding and a golomb rice coding.