KR20190091178A - Image processing device and method for operating image processing device - Google Patents

Abstract

An image processing apparatus and a method for operating the image processing apparatus are provided. The image processing apparatus includes a multimedia IP for processing image data; a memory accessed by the multimedia IP; and a frame buffer compressor (FBC) for compressing the image data to generate compressed data and storing the compressed data in the memory. The frame buffer compressor includes a compression management module that determines a combination of a QP table and an entropy table and controls the compression of the image data according to the determined combination of the QP table and the entropy table. It is possible to perform optimized image data compression.

Description

IMAGE PROCESSING DEVICE AND METHOD FOR OPERATING IMAGE PROCESSING DEVICE}

The present invention relates to an image processing apparatus and an operating method of the image processing apparatus.

As the need for video high-resolution images and high-frame rate images has emerged, the amount of access to memory, i.e., bandwidth, of various multimedia IPs of the image processing apparatus has been greatly increased. It became.

When the band whistle is increased, the processing power of the image processing apparatus reaches a limit, and thus, a problem may occur that the speed decreases during recording and playback of a video image.

Accordingly, a method of compressing the size of data when the multimedia IP accesses the memory is considered. For example, the data may be compressed before writing data to the memory, and the compressed data may be decompressed before the data is read from the memory.

SUMMARY OF THE INVENTION The present invention has been made in an effort to provide an image processing apparatus that performs optimized image data compression.

Another technical problem to be solved by the present invention is to provide an operating method of an image processing apparatus that performs optimized image data compression.

Technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided an image processing apparatus including: a multimedia IP for processing image data; Memory accessed by the multimedia IP; Compression is performed on the image data to generate compressed data, and includes a frame buffer compressor (FBC) for storing the compressed data in the memory, wherein the frame buffer compressor is a combination of a QP table and an entropy table. And a compression management module for controlling compression of the image data according to the determined combination of the QP table and the entropy table.

According to another aspect of the present invention, there is provided an image processing apparatus including a multimedia IP for processing image data; Memory accessed by the multimedia IP; Compressing the image data to generate compressed data and storing the compressed data in the memory, the frame buffer compressor comprising: a QP table including up to 16 entries and up to 4 k And a compression management module that determines an entropy table determined by a value and controls compression of the image data according to the determined combination of the QP table and the entropy table.

According to another aspect of the present invention, there is provided a method of operating an image processing apparatus, including performing prediction on image data, determining a QP table including a predetermined number of entries, and determining the determined QP. Quantization is performed on the image data on which the prediction is completed using a table, an entropy table is determined using a predetermined number of k values, and entropy coding is performed on the quantized image data using the determined entropy table. Performing compression to generate compressed data.

Specific details of other embodiments are included in the detailed description and the drawings.

1 to 3 are block diagrams illustrating an image processing apparatus according to some embodiments of the present invention.
4 is a block diagram illustrating in detail the frame buffer compressor of FIGS. 1 to 3.
FIG. 5 is a block diagram illustrating in detail the encoder of FIG. 4.
FIG. 6 is a block diagram for describing the decoder of FIG. 4 in detail.
7 to 10 are schematic diagrams for describing an operation of an image processing apparatus according to some embodiments of the present disclosure.
11 through 14 are schematic diagrams for describing an operation of an image processing apparatus in some embodiments of the present disclosure.
FIG. 15 is a view for explaining an advantageous effect by an image processing apparatus and an operating method of the image processing apparatus according to some embodiments of the present disclosure.
16 is a flowchart illustrating a method of operating an image processing apparatus according to some embodiments of the present disclosure.

1 to 3 are block diagrams illustrating an image processing apparatus according to some embodiments of the present invention.

Referring to FIG. 1, an image processing apparatus according to some embodiments of the present disclosure may include a multimedia IP (100), a frame buffer compressor (FBC) 200, a memory 300, and a system bus. 400).

The multimedia IP 100 may be a part that directly performs image processing of the image processing apparatus. In other words, the multimedia IP 100 may refer to various modules for recording and playing back images such as camcoding and playing back video images.

The multimedia IP 100 may receive first data from an external device such as a camera and convert the first data into second data. In this case, the first data may be moving image or image raw data. The second data is data generated by the multimedia IP 100 and may also include data being processed by the multimedia IP 100. That is, the multimedia IP 100 may repeat the step of storing the second data in the memory 300 and updating again through various steps. The second data may include all data in this step. However, since the second data may be stored in the form of third data when the second data is stored in the memory 300, the second data in the memory 300 may be stored before or in the memory 300. It may mean data after being read. This will be explained in more detail later.

Specifically, the multimedia IP 100 includes an image signal processor (ISP) 110, a shake correction module (G2D) 120, a multi-format codec (MFC) 130, a GPU 140, and a display 150. can do. However, the present embodiment is not limited thereto. That is, the multimedia IP 100 may include only at least some of the image signal processor 110, the shake correction module 120, the multi-format codec 130, the GPU 140, and the display 150 described above. That is, the multimedia IP 100 may refer to a processing module that needs to access the memory 300 in order to process a video or an image.

The image signal processor 110 may receive the first data and preprocess it to convert the second data into the second data. In this case, the first data may be RGB image source data. For example, the image signal processor 110 may convert the first data of the RGB method into the second data of the YUV method.

In this case, the RGB data refers to a data format in which colors are expressed based on three primary colors of light. That is, the image is expressed by using three kinds of colors, RED, GREEN, and BLUE. In contrast, the YUV method refers to a data format that expresses brightness, that is, a luminance signal and a chroma signal separately. That is, Y denotes a luminance signal, and U (Cb) and V (Cr) each mean a color difference signal. U means the difference between the luminance signal and the blue signal component, and V means the difference between the luminance signal and the red signal component.

The YUV data is converted from RGB data using a conversion equation such as Y = 0.3R + 0.59G + 0.11B, U = (BY) x0.493, V = (RY) x0.877. Can be obtained.

Since the human eye is sensitive to luminance signals but less sensitive to color signals, YUV data may be easier to compress than RGB data. Accordingly, the image signal processor 110 may convert the first data of the RGB method into the second data of the YUV method.

The image signal processor 110 may convert the first data into the second data and then store the first data in the memory 300.

The shake correction module 120 may perform shake correction of image or video data. The shake correction module 120 may perform shake correction by reading the first data or the second data stored in the memory 300. In this case, the shake correction means detecting and removing the shake of the camera from the video data.

The shake correction module 120 may generate or update new second data by correcting shake of the first data or the second data, and store the new second data in the memory 300.

The multi-format codec 130 may be a codec that compresses video data. In general, video data is very large, so a compression module is needed to reduce its size. The video data may be compressed through an association between a plurality of frames, and the multi-format codec 130 may perform the same. The multi-format codec 130 may read and compress the first data or the second data stored in the memory 300.

The multi-format codec 130 may compress the first data or the second data to generate new second data or update the second data, and store the same in the memory 300.

The graphics processing unit (GPU) 140 may perform calculation and generation of two-dimensional or three-dimensional graphics. The GPU 140 may process the first data or the second data stored in the memory 300. The GPU 140 may be specialized in processing graphic data, and may process graphic data in parallel.

The GPU 140 may compress the first data or the second data to generate new second data or update the second data, and store the second data in the memory 300.

The display 150 may display the second data stored in the memory 300 on the screen. The display 150 displays image data processed by another multimedia IP 100, that is, the image signal processor 110, the shake correction module 120, the multi-format codec 130, and the GPU 140, that is, the second data. Can be displayed on the screen. However, the present embodiment is not limited thereto.

The image signal processor 110, the shake correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 may operate individually. That is, the image signal processor 110, the shake correction module 120, the multi-format codec 130, the GPU 140, and the display 150 may each individually access the memory 300 to write or read data. have.

The frame buffer compressor 200 compresses the second data and converts the second data into the third data before the multimedia IP 100 individually accesses the memory 300. The frame buffer compressor 200 may transmit the third data to the multimedia IP 100 again, and the multimedia IP 100 may transmit the third data to the memory 300.

Accordingly, the third data compressed by the frame buffer compressor 200 may be stored in the memory 300. On the contrary, the third data stored in the memory 300 may be loaded by the multimedia IP 100 and transmitted to the frame buffer compressor 200. The frame buffer compressor 200 may decompress the third data and convert the third data into the second data. The frame buffer compressor 200 may transmit the second data to the multimedia IP 100 again.

That is, each time the image signal processor 110, the shake correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 access the memory 300 individually. The frame buffer compressor 200 may compress the second data into the third data and transmit the compressed data to the memory 300. Conversely, whenever there is a data request from the memory 300 to the image signal processor 110, the shake correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100. The frame buffer compressor 200 decompresses the third data into the second data to decode the image signal processor 110, the shake correction module 120, the multi-format codec 130, and the GPU 140 of the multimedia IP 100. ) And the display 150, respectively.

The memory 300 may store the third data generated by the frame buffer compressor 200 and provide the stored third data to the frame buffer compressor 200 so that the frame buffer compressor 200 can decompress the stored third data. have.

The system bus 400 may be connected to the multimedia IP 100 and the memory 300, respectively. In detail, the image signal processor 110, the shake correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 may be individually connected to the system bus 400. have. The system bus 400 is the image signal processor 110, the shake correction module 120, the multi-format codec 130, the GPU 140 and the display 150 and the memory 300 of the multimedia IP (100) data each other It can be a path for transmitting.

The frame buffer compressor 200 is not connected to the system bus 400, but the image signal processor 110, the shake correction module 120, the multi-format codec 130, the GPU 140 and the display of the multimedia IP 100. When the 150 accesses the memory, the second data may be converted into the third data, and the third data may be converted into the second data.

Next, referring to FIG. 2, the frame buffer compressor 200 of the image processing apparatus according to some embodiments of the present disclosure may be directly connected to the system bus 400.

The frame buffer compressors 200 may not be directly connected to the multimedia IP 100 but may be connected to each other through the system bus 400. In detail, the image signal processor 110, the shake correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 are frame buffered through the system bus 400, respectively. The compressor 200 and the data may be transmitted to each other, and data may be transmitted to the memory 300 through this.

That is, in the compression process, the image signal processor 110, the shake correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 are respectively connected through the system bus 400. The second data may be transmitted to the frame buffer compressor 200. Subsequently, the frame buffer compressor 200 may compress the second data into the third data and transmit the same to the memory 300 through the system bus 400.

Similarly, in the decompression process, the frame buffer compressor 200 receives the third data stored in the memory 300 through the system bus 400, and decompresses the third data into the second data. Subsequently, the frame buffer compressor 200 transmits the second data through the system bus 400 to the image signal processor 110, the shake correction module 120, the multi-format codec 130, and the GPU of the multimedia IP 100. 140 and the display 150, respectively.

Next, referring to FIG. 3, in the image processing apparatus according to some embodiments of the present disclosure, the memory 300 and the system bus 400 may be connected to each other through the frame buffer compressor 200.

That is, the memory 300 may not be directly connected to the system bus 400, but may be connected to the system bus 400 only through the frame buffer compressor 200. In addition, the image signal processor 110, the shake correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 may be directly connected to the system bus 400. Accordingly, the image signal processor 110, the shake correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100 may pass through the frame buffer compressor 200 only through the memory ( 300).

In the present specification, the second data is referred to as the image data 10 and the third data as the compressed data 20.

4 is a block diagram illustrating in detail the frame buffer compressor of FIGS. 1 to 3.

Referring to FIG. 4, the frame buffer compressor 200 may include an encoder 210 and a decoder 220.

The encoder 210 may generate the compressed data 20 by receiving the image data 10 from the multimedia IP 100. In this case, the image data 10 may be transmitted from the image signal processor 110, the shake correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100, respectively. have. The compressed data 20 may be transmitted to the memory 300 through the multimedia IP 100 and the system bus 400.

In contrast, the decoder 220 may decompress the compressed data 20 stored in the memory 300 into the image data 10. Image data 10 may be transferred to the multimedia IP. In this case, the image data 10 may be transferred to the image signal processor 110, the shake correction module 120, the multi-format codec 130, the GPU 140, and the display 150 of the multimedia IP 100, respectively. have.

FIG. 5 is a block diagram illustrating in detail the encoder of FIG. 4.

Referring to FIG. 5, the encoder 210 includes a first mode selector 219, a prediction module 211, a quantization module 213, an entropy encoding module 215, and a padding module 217.

The first mode selector 219 may determine whether the encoder 210 operates in a lossless mode or a lossy mode. According to the first mode selector 219, when the encoder 210 operates in the lossless mode, the image data 10 is compressed along the lossless path of FIG. 3, and the encoder 210 is in the lossy mode. In operation, the image data 10 may be compressed along a lossy path Losssy.

The first mode selector 219 may receive a signal from the multimedia IP 100 to determine whether to perform lossless compression or lossy compression. In this case, lossless compression means compression without loss of data, and means a method in which a compression ratio varies depending on data. On the other hand, lossy compression is a compression in which some data is lost and may have a higher compression ratio than a lossless compression, and may have a predetermined fixed compression ratio.

In the case of the lossless mode, the first mode selector 219 may induce the image data 10 to the prediction module 211, the entropy encoding module 215, and the padding module 217 along a lossless path. . In contrast, the first mode selector 219 may induce the image data 10 to the prediction module 211, the quantization module 213, and the entropy encoding module 215 along the lossy path in the lossy mode. Can be.

The prediction module 211 may express the image data 10 by dividing the prediction data and the residual data. For example, if one pixel has a value of 0 to 255, 8 bits of data per pixel may be required to express it. On the other hand, when the adjacent pixels have similar values, there is no loss of data even when only the difference with the adjacent pixels, that is, the residual, is expressed, and the number of data bits to be represented can be greatly reduced. For example, if the prediction data is 253 when the pixels having the value of (253, 254, 255) are continuous, the residual data representation of (253 (prediction), 1 (residual), 2 (residual)) is sufficient. In addition, the number of bits per pixel for representing the residual data may be very small, 2 bits.

Therefore, the prediction module 211 may compress the size of the overall image data 10 by dividing the image data 10 into the prediction data and the residual data. Of course, various methods may be possible regarding what to use prediction data.

The prediction module 211 may perform prediction in units of pixels or may perform prediction in units of blocks. In this case, the block may mean an area formed by a plurality of adjacent pixels.

The quantization module 213 may further compress the image data 10 compressed by the prediction module 211. The quantization module 213 may remove the lower bits of the image data 10 through preset quantization coefficients. Specifically, the data may be multiplied by a quantization coefficient to select its representative value, but loss may occur by rounding off the decimal point. If the value of the pixel data is 0 to 28-1 (= 255), the quantization coefficient may be defined as 1 / (2n-1), where n is an integer of 8 or less. However, the present embodiment is not limited thereto.

At this time, the removed lower bits may not be recovered later and may be lost. Thus, quantization module 213 may be utilized only in lossy mode. However, the lossy mode may have a relatively higher compression rate than the lossless mode, and may have a predetermined fixed compression rate, so that information on the compression rate may not be necessary later.

The entropy encoding module 215 compresses the image data 10 compressed by the quantization module 213 in lossy mode or the image data 10 compressed by the prediction module 211 in lossless mode through entropy coding. can do. At this time, entropy coding may utilize a method of allocating the number of bits according to the frequency.

The entropy encoding module 215 may compress the image data 10 using Huffman coding. Alternatively, the entropy encoding module 215 may compress the image data 10 through exponential golomb coding or golomb rice coding. At this time, the entropy encoding module 215 may generate a table through the k value, thereby simply compressing the image data 10.

The padding module 217 may pad the image data 10 compressed by the entropy encoding module 215 in the lossless mode. Here, padding may mean adding meaningless data to fit a specific size. This will be explained in more detail later.

The padding module 217 can be activated in a lossy mode as well as a lossless mode. In lossy mode, the image data 10 may be compressed more than the intended compression rate when compressed by the quantization module 213. In this case, the data may be converted into the compressed data 20 through the padding module 217 and transmitted to the memory 300 even in the loss mode.

The compression management module 218 determines a combination of the QP table and the entropy table used for quantization and entropy coding, respectively, and controls the compression of the image data 10 according to the determined combination of the QP table and the entropy table.

In this case, the first mode selector 219 determines that the encoder 210 operates in a lossy mode, so that the image data 10 is compressed along the lossy path of FIG. 5. That is, the compression management module 218 determines the combination of the QP table and the entropy table and compresses the image data 10 according to the determined combination of the QP table and the entropy table, so that the frame buffer compressor 200 uses the lossy compression algorithm ( It is assumed that compression is performed on the image data 10 using a lossy compression algorithm.

In particular, the QP table may include one or more entries, and each entry may include quantization coefficients used to quantize the image data 10. Concepts such as the QP table and the quantization coefficients correspond to contents already known as image compression techniques, and thus detailed description thereof will be omitted.

Meanwhile, the entropy table refers to a plurality of code tables identified through k values for performing an entropy coding algorithm, and entropy tables that can be used in some embodiments of the present invention include exponential golomb codes and golombs. It may include at least one of the rice rice (golomb rice code). Concepts related to entropy coding, exponential Golomb coding algorithm, and Golomb rice coding algorithm, etc. correspond to the contents already known as data compression techniques, and thus, detailed description thereof will be omitted.

The compression management module 218 determines a QP table including a predetermined number of entries, and the frame buffer compressor 200 performs quantization on the predicted image data 10 using the determined QP table. In addition, the compression management module 218 determines the entropy table by using a predetermined number of k values, and the frame buffer compressor 200 performs entropy coding on the quantized image data 10 by using the determined entropy table. To perform. That is, the frame buffer compressor 200 generates the compressed data 20 based on the combination of the QP table and the entropy table determined by the compression management module 218.

Thereafter, the frame buffer compressor 200 may write the generated compressed data 20 to the memory 300. The frame buffer compressor 200 may read the compressed data 20 from the memory 300, decompress the read compressed data 20, and provide the compressed data 20 to the multimedia IP 100.

More details of the compression management module 218 performing such an operation will be described later with reference to FIGS. 7 to 16.

FIG. 6 is a block diagram for describing the decoder of FIG. 4 in detail.

Referring to FIG. 6, the decoder 220 includes a second mode selector 229, an unpadding module 227, an entropy decoding module 225, an inverse quantization module 223, and a prediction compensation module 221.

The second mode selector 229 may determine whether the compressed data 20 stored in the memory 300 is losslessly compressed or lossy compressed. In this case, the second mode selector 229 may determine whether the compressed data 20 is compressed by a lossless mode or a lossy mode based on the presence or absence of a header. This will be explained in more detail later.

In the case of the lossless mode, the second mode selector 229 may induce the compressed data 20 to the unpadding module 227, the entropy decoding module 225, and the prediction compensation module 221 along a lossless path. Can be. On the contrary, in the loss mode, the second mode selector 229 transmits the compressed data 20 to the entropy decoding module 225, the dequantization module 223, and the prediction compensation module 221 along the loss path Lossy. Can be induced.

The unpadding module 227 may remove the padded portion of the data padded by the padding module 217 of the encoder 210.

The entropy decoding module 225 may decompress the data compressed by the entropy encoding module 215. The entropy decoding module 225 may perform decompression through Huffman coding, expansive Gollum coding, or Gollum rice coding. Since the compressed data 20 includes a k value, the entropy decoding module 225 may perform decoding using the k value.

The inverse quantization module 223 may decompress data compressed by the quantization module 213. The inverse quantization module 223 may restore the compressed data 20 compressed by using the quantization coefficients defined by the quantization module 213, but may not completely restore the portion lost in the compression process. Thus, inverse quantization module 223 can only be utilized in lossy mode.

The prediction compensation module 221 may restore the data represented by the prediction data and the residual data by the prediction module 211. The prediction compensation module 221 may convert, for example, the residual data representation of (253 (prediction), 1 (residual), 2 (residual)) to (253, 254, 255).

The prediction compensation module 221 may reconstruct the prediction performed in the pixel unit or the block unit according to the prediction module 211. Accordingly, the compressed data 20 may be restored or decompressed and transmitted to the multimedia IP 100.

The decompression management module 228 is a combination of the QP table and the entropy table that the compression management module 218 described above with respect to FIG. When decompressing, you can perform tasks that can be properly reflected.

7 to 16, the operation of the image processing apparatus described above will be described.

7 to 10 are schematic diagrams for describing an operation of an image processing apparatus according to some embodiments of the present disclosure.

Referring to FIG. 7, the compression management module 218 may determine a QP table 230 including a predetermined number of entries. In this embodiment, the compression management module 218 may determine a QP table 230 having eight entries that enable eight levels of quantization.

QP table 230 includes a first entry and a second entry. Here, the first entry is a quantization coefficient corresponding to index 0, and the second entry is a quantization coefficient corresponding to index 7. And QP table 230 includes one or more third entries between the first entry and the second entry. In the present embodiment, one or more third entries correspond to quantization coefficients corresponding to indices 1 to 6.

The first entry is determined to be a first predetermined value. In the present embodiment, the first predetermined value is expressed as "MaxShiftValue". The first value is a constant value that may be arbitrarily determined as needed in actual implementation of the image processing apparatus or the method of operating the image processing apparatus according to various embodiments of the present disclosure.

After the first entry is determined to be the first value, the second entry may be determined by the following equation.

MaxShiftValue >> BitDepth * (1-CompressionRatio)

Here, MaxShiftValue is the first value described above, BitDepth is a bit depth of the image data 10, and CompressionRatio is a compression rate.

That is, when the first value is the same and the target compression rate is the same, the compression management module 218 may set the final value of the QP table 230 differently according to the bit depth of the image data 10.

As such, after the first entry is determined as the first value and the second entry is determined as the second value, the third entry may be determined through sampling. That is, the third entry may be sampled in the QP table 230 including a predetermined number of entries so that, for the remaining third entries except the first entry and the second entry, the quantization coefficients are properly distributed.

Meanwhile, the compression management module 218 may determine the entropy table by using a predetermined number of k values. In some embodiments of the invention, the entropy table may be determined by up to four k values.

Specifically, the entropy table is determined by k values of n, n + 1, n + 2, n + 3 (where n is an integer of 0 or more) with respect to the image data 10 having the first bit depth, N + a, n + a + 1, n + a + 2, and n + a + 3 for image data 10 having a second bit depth greater than the first bit depth, where a is an integer of 1 or more Can be determined by the value of k.

For example, the entropy table may be determined by k values of 0, 1, 2, 3 for image data having a first bit depth of 8 bits, and for image data having a second bit depth of 10 bits. It can be determined by k values of 1, 2, 3, and 4.

That is, the compression management module 218 may set, for example, four consecutive k values differently according to the bit depth of the image data 10.

When performing block based lossy compression of the image data 10, as described above, the steps of prediction, quantization and entropy coding are performed. In the case of quantization, the more detailed the QP table containing the quantization coefficients can perform compression, but on the other hand, memory resources are consumed to maintain the QP table with multiple entries, and the frame The number of bits of the header of the stream of compressed data 20 exchanged between the encoder 210 and the decoder 220 of the buffer controller 200 may be increased. In particular, in an environment in which image data 10 is compressed in block units, a significant amount of bandwidth may be consumed considering the total image size even if the header of the stream of compressed data 20 increases only one bit. .

Further, in the case of entropy coding, the distribution of the residual signal is used for entropy coding because, for example, the image data having a bit depth of 8 bits is different from the case of image data having a bit depth of 10 bits, for example. The entropy table also needs to be determined differently accordingly. For example, the value of the entropy table entry tends to be reduced to zero in the case of image data having the same bit depth of 10 bits than in the case of image data having a bit depth of 8 bits, so that the k value may be determined differently according to the bit depth. There is a need.

Accordingly, by determining a QP table having a good compression quality and having an appropriate size and determining an entropy table by selecting an appropriate k value in consideration of the distribution of the residual signal, the frame buffer compressor can be increased while increasing the compression performance of the image data 10. Bandwidth between the 200 and the memory 300 can be saved.

In some embodiments of the invention, compression management module 218 may determine a QP table that includes up to 16 entries. In addition, in some embodiments of the present invention, compression management module 218 may determine a QP table that includes eight or more entries.

Referring next to FIG. 8, the compression management module 218 determines a QP table 232 containing eight entries.

Assume that the first entry of the QP table 232 is determined to be 4096. The compression management module 218 performs a bit shift operation on the 4096 corresponding to the first value by BitDepth * (1-CompressionRatio) as follows.

4096 >> 8 X (1-0.5) = 256

8 is a value corresponding to a bit depth of the image data 10, and 0.5 is a value corresponding to a target compression ratio.

As such, after the first entry is determined as 4096 and the second entry is determined as 256, the third entry may be determined through sampling. That is, the third entry may be appropriately distributed in the QP table 232 including eight entries for the third entry corresponding to the indexes 1 to 6 except for the first entry 4096 and the second entry 256. For example, 1365, 819, 585, 455, 372, 315. The entries of the QP table 242 thus determined may be used as quantization coefficients by dividing, for example, by 4096, which is a MaxShiftValue value.

Meanwhile, the compression management module 218 determines k values as 0, 1, 2, and 3 when the image data 10 has an 8-bit bit depth, and the image data 10 has a bit depth of 10 bits. In this case, we can determine k as 1, 2, 3, or 4.

In this way, the compression management module 218 determines the combination of the QP table and the entropy table used for quantization and entropy coding, respectively, and controls the compression of the image data 10 according to the determined combination of the QP table and the entropy table. While improving the compression performance of the data 10, it is possible to save the band whistle between the frame buffer compressor 200 and the memory 300.

Referring next to FIG. 9, the compression management module 218 determines a QP table 234 comprising eight entries.

Assume that the first entry of the QP table 234 is determined to be 4096. The compression management module 218 performs a bit shift operation on the 4096 corresponding to the first value by BitDepth * (1-CompressionRatio) as follows.

4096 >> 10 X (1-0.5) = 128

10 is a value corresponding to a bit depth of the image data 10, and 0.5 is a value corresponding to a target compression ratio.

As such, after the first entry is determined as 4096 and the second entry is determined as 128, the third entry may be determined through sampling. That is, the third entry is a QP table 234 including eight entries so that the quantization coefficients are properly distributed to the third entries corresponding to the indices 1 to 6 except for the first entry 4096 and the second entry 128. For example, 819, 455, 293, 216, 171, 146. The entries of the QP table 242 thus determined may be used as quantization coefficients by dividing, for example, by 4096, which is a MaxShiftValue value.

Meanwhile, the compression management module 218 determines k values as 0, 1, 2, and 3 when the image data 10 has an 8-bit bit depth, and the image data 10 has a bit depth of 10 bits. In this case, we can determine k as 1, 2, 3, or 4.

In this way, the compression management module 218 determines the combination of the QP table and the entropy table used for quantization and entropy coding, respectively, and controls the compression of the image data 10 according to the determined combination of the QP table and the entropy table. While improving the compression performance of the data 10, it is possible to save the band whistle between the frame buffer compressor 200 and the memory 300.

10, the compressed bit stream 236 generated from the encoder 210 of the frame buffer compressor 200 through the above process includes four bits of QP table information and two bits of k value information. A header, followed by the compressed data 20 as compressed binary.

11 through 14 are schematic diagrams for describing an operation of an image processing apparatus in some embodiments of the present disclosure.

Referring to FIG. 11, the compression management module 218 may determine a QP table 240 including a predetermined number of entries. In this embodiment, the compression management module 218 can determine the QP table 240 with 16 entries to enable 16 levels of quantization.

QP table 240 includes a first entry and a second entry. Here, the first entry is a quantization coefficient corresponding to index 0, and the second entry is a quantization coefficient corresponding to index 15. And QP table 240 includes one or more third entries between the first entry and the second entry. In this embodiment, the one or more third entries correspond to quantization coefficients corresponding to the indices 1 to 14.

After the first entry is determined to be the first value and the second entry is determined to be the second value, the third entry may be determined through sampling. That is, the third entry may be sampled in the QP table 240 including the predetermined number of entries so that the quantization coefficients are properly distributed for the third entry except the first entry and the second entry.

Meanwhile, the compression management module 218 may determine the entropy table by using a predetermined number of k values. In some embodiments of the invention, the entropy table may be determined by up to four k values.

For example, the entropy table may be determined by k values of 0, 1, 2, 3 for image data having a first bit depth of 8 bits, and for image data having a second bit depth of 10 bits. It can be determined by k values of 1, 2, 3, and 4.

That is, the compression management module 218 may set, for example, four consecutive k values differently according to the bit depth of the image data 10.

Next, referring to FIG. 12, the compression management module 218 determines a QP table 242 including 16 entries.

Assume that the first entry of the QP table 242 is determined to be 4096. The compression management module 218 performs a bit shift operation on the 4096 corresponding to the first value by BitDepth * (1-CompressionRatio) as follows.

4096 >> 8 X (1-0.5) = 256

8 is a value corresponding to a bit depth of the image data 10, and 0.5 is a value corresponding to a target compression ratio.

As such, after the first entry is determined as 4096 and the second entry is determined as 256, the third entry may be determined through sampling. That is, the third entry is a QP table 232 including eight entries so that the quantization coefficients are properly distributed to the third entries corresponding to the indices 1 to 14 except for the first entry 4096 and the second entry 256. For example, 2048, 1365, 1024, 819, 683, 585, 512, 455, 410, 372, 341, 315, 293, 273. The entries of the QP table 242 thus determined may be used as quantization coefficients by dividing, for example, by 4096, which is a MaxShiftValue value.

Meanwhile, the compression management module 218 determines k values as 0, 1, 2, and 3 when the image data 10 has an 8-bit bit depth, and the image data 10 has a bit depth of 10 bits. In this case, we can determine k as 1, 2, 3, or 4.

In this way, the compression management module 218 determines the combination of the QP table and the entropy table used for quantization and entropy coding, respectively, and controls the compression of the image data 10 according to the determined combination of the QP table and the entropy table. While improving the compression performance of the data 10, it is possible to save the band whistle between the frame buffer compressor 200 and the memory 300.

Referring next to FIG. 13, the compression management module 218 determines a QP table 244 including eight entries.

Assume that the first entry of the QP table 244 is determined to be 4096. The compression management module 218 performs a bit shift operation on the 4096 corresponding to the first value by BitDepth * (1-CompressionRatio) as follows.

4096 >> 10 X (1-0.5) = 128

10 is a value corresponding to a bit depth of the image data 10, and 0.5 is a value corresponding to a target compression ratio.

As such, after the first entry is determined as 4096 and the second entry is determined as 128, the third entry may be determined through sampling. That is, the third entry is a QP table 234 including eight entries so that, for the third entry corresponding to the indices 1 to 14 except for the first entry 4096 and the second entry 128, the quantization coefficients are properly distributed. For example, 1365, 819, 585, 455, 372, 315, 273, 228, 205, 186, 171, 158, 146, 137. The entries of the QP table 242 thus determined may be used as quantization coefficients by dividing, for example, by 4096, which is a MaxShiftValue value.

Meanwhile, the compression management module 218 determines k values as 0, 1, 2, and 3 when the image data 10 has an 8-bit bit depth, and the image data 10 has a bit depth of 10 bits. In this case, we can determine k as 1, 2, 3, or 4.

In this way, the compression management module 218 determines the combination of the QP table and the entropy table used for quantization and entropy coding, respectively, and controls the compression of the image data 10 according to the determined combination of the QP table and the entropy table. While improving the compression performance of the data 10, it is possible to save the band whistle between the frame buffer compressor 200 and the memory 300.

14, the compressed bit stream 246 generated from the encoder 210 of the frame buffer compressor 200 through the above process includes four bits of QP table information and two bits of k value information. A header, followed by the compressed data 20 as compressed binary.

FIG. 15 is a view for explaining an advantageous effect by an image processing apparatus and an operating method of the image processing apparatus according to some embodiments of the present disclosure.

Referring to FIG. 15, the best PSNR gain is shown when the number of entries in the QP table is 12 and the number of k values is four.

That is, by determining a QP table having a good compression quality and having an appropriate size, and determining an entropy table by selecting an appropriate k value in consideration of the distribution of the residual signal, the frame buffer compressor can be increased while increasing the compression performance of the image data 10. Bandwidth between the 200 and the memory 300 can be saved.

16 is a flowchart illustrating a method of operating an image processing apparatus according to some embodiments of the present disclosure.

Referring to FIG. 16, a method of operating an image processing apparatus according to some embodiments of the present disclosure includes determining a QP table including a predetermined number of entries (S1601).

In some embodiments of the invention, determining the QP table comprising a predetermined number of entries may include determining a QP table comprising up to 16 entries. Further, in some embodiments of the present invention, determining the QP table that includes a predetermined number of entries may include determining the QP table that includes eight or more entries.

In addition, the method may include performing quantization on the predicted image data using the determined QP table (S1603).

The method also includes determining an entropy table (S1605) using a predetermined number of k values.

In some embodiments of the invention, determining the entropy table may include determining the entropy table using up to four k values or less.

The method may further include generating compressed data by performing entropy coding on the quantized image data using the determined entropy table (S1607).

Although the embodiments of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to the above embodiments but may be manufactured in various forms, and having ordinary skill in the art to which the present invention pertains. It will be understood by those skilled in the art that the present invention may be embodied in other specific forms without changing the technical spirit or essential features of the present invention. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

100: Multimedia IP
200: frame buffer compressor
300: memory

Claims (20)

Multimedia IP for processing image data;
Memory accessed by the multimedia IP;
A frame buffer compressor (FBC) for compressing the image data to generate compressed data and storing the compressed data in the memory;
The frame buffer compressor includes a compression management module that determines a combination of a QP table and an entropy table and controls the compression of the image data according to the determined combination of the QP table and the entropy table. The method of claim 1,
The QP table includes a first entry and a second entry,
The first entry is determined to be a predetermined first value,
The second entry is an image processing apparatus determined by the following equation:
MaxShiftValue >> BitDepth * (1-CompressionRatio)
Where MaxShiftValue is the first value, BitDepth is the bit depth of the image data, and CompressionRatio represents the compression rate. The method of claim 2,
The QP table includes one or more third entries between the first entry and the second entry,
And the third entry is determined through sampling. The method of claim 1,
And the QP table includes up to 16 entries. The method of claim 4, wherein
And the QP table comprises eight or more entries. The method of claim 1,
And the entropy table is determined by a maximum of four k values or less. The method of claim 6,
The entropy table is,
For image data having a first bit depth, it is determined by k values of n, n + 1, n + 2, n + 3 (where n is an integer of 0 or more),
K values of n + a, n + a + 1, n + a + 2, n + a + 3 (where a is an integer of 1 or more) for image data having a second bit depth greater than the first bit depth Image processing apparatus determined by. The method of claim 1,
The entropy table includes at least one of an exponential golomb code and a golomb rice code. Multimedia IP for processing image data;
Memory accessed by the multimedia IP;
A frame buffer compressor (FBC) for compressing the image data to generate compressed data and storing the compressed data in the memory;
The frame buffer compressor determines an entropy table determined by a QP table including up to 16 entries and up to 4 k values, and controls compression of the image data according to the determined combination of the QP table and the entropy table. An image processing apparatus comprising a compression management module. 10. The method of claim 9,
The QP table includes a first entry and a second entry,
The first entry is determined to be a predetermined first value,
The second entry is an image processing apparatus determined by the following equation:
MaxShiftValue >> BitDepth * (1-CompressionRatio)
Where MaxShiftValue is the first value, BitDepth is the bit depth of the image data, and CompressionRatio represents the compression rate. The method of claim 10,
The QP table includes one or more third entries between the first entry and the second entry,
And the third entry is determined through sampling. 10. The method of claim 9,
And the QP table comprises eight or more entries. 10. The method of claim 9,
The entropy table is,
For image data having a first bit depth, it is determined by k values of n, n + 1, n + 2, n + 3 (where n is an integer of 0 or more),
K values of n + a, n + a + 1, n + a + 2, n + a + 3 (where a is an integer of 1 or more) for image data having a second bit depth greater than the first bit depth Image processing apparatus determined by. Perform predictions on image data,
Determine a QP table that includes a predetermined number of entries,
Perform quantization on the image data on which the prediction is completed using the determined QP table,
The entropy table is determined using a predetermined number of k values,
And generating compressed data by performing entropy coding on the quantized image data using the determined entropy table. The method of claim 14,
Determining a QP table comprising the predetermined number of entries,
A method of operating an image processing apparatus comprising determining a QP table comprising a maximum of 16 entries. 16. The method of claim 15,
Determining a QP table comprising the predetermined number of entries,
And determining a QP table comprising eight or more entries. The method of claim 14,
The QP table includes a first entry and a second entry,
Determining a QP table comprising the predetermined number of entries,
Determine the first entry as a first predetermined value,
A method of operating an image processing apparatus comprising determining the second entry by the following equation:
MaxShiftValue >> BitDepth * (1-CompressionRatio)
Where MaxShiftValue is the first value, BitDepth is the bit depth of the image data, and CompressionRatio represents the compression rate. The method of claim 17,
The QP table includes one or more third entries between the first entry and the second entry,
Determining a QP table comprising the predetermined number of entries,
And determining the third entry by sampling. The method of claim 14,
Determining the entropy table,
And determining the entropy table using up to four k values. The method of claim 14,
Determining the entropy table,
The entropy table is determined using k values of n, n + 1, n + 2, and n + 3 (where n is an integer of 0 or more) for image data having a first bit depth,
K values of n + a, n + a + 1, n + a + 2, n + a + 3 (where a is an integer of 1 or more) for image data having a second bit depth greater than the first bit depth And determining the entropy table by using.

Images