CN110933442B

CN110933442B - 16x4 level reference frame compression method and device

Info

Publication number: CN110933442B
Application number: CN201811100165.0A
Authority: CN
Inventors: 曲道远
Original assignee: Beijing Ingenic Semiconductor Co Ltd
Current assignee: Beijing Ingenic Semiconductor Co Ltd
Priority date: 2018-09-20
Filing date: 2018-09-20
Publication date: 2022-05-13
Anticipated expiration: 2038-09-20
Also published as: CN110933442A

Abstract

The invention provides a 16x 4-level reference frame compression method and a device, wherein the method comprises the following steps: acquiring 16x4 levels of reference frames, wherein the reference frames are used for performing motion compensation in the encoding and decoding process; dividing the reference frame into 4 by 4 pixel blocks; sequentially obtaining the minimum value of each pixel block in the 4 pixel blocks by 4x 4; according to the minimum value of each pixel block, solving a minimum value tree and a bit width tree of each pixel block in 4 pixel blocks of 4x 4; and solving a minimum value tree and a bit width tree of the 16x 4-level reference frame according to the minimum value of each pixel block. By the scheme, the technical problem of overlarge bandwidth requirement in the existing coding and decoding process is solved, and the technical effect of reducing the bandwidth is achieved.

Description

16x4 level reference frame compression method and device

Technical Field

The invention relates to the technical field of data processing, in particular to a 16x 4-level reference frame compression method and a device.

Background

As shown in fig. 1, in the video codec, the steps shown in fig. 1 need to be performed: entropy coding, inverse quantization, inverse transformation, motion compensation, reference frames, external storage, etc. These require constant data exchange with off-chip memory, including: writing of a reconstructed frame, acquisition of a reference frame, and the like. While data is being written or retrieved, a significant amount of bandwidth consumption is incurred.

In the encoding and decoding process, the DDR needs to be accessed in real time to obtain the required data, and due to the huge bandwidth requirement, the DDR consumes a large part of power generated by writing and reading data, so that the DDR bandwidth reduction becomes particularly important in video encoding and decoding.

No effective solution has been proposed at present for the DDR bandwidth if reduced.

Disclosure of Invention

The embodiment of the invention provides a 16x 4-level reference frame compression method and a device, which are used for achieving the purpose of reducing the bandwidth requirement of DDR in the video coding and decoding process.

The embodiment of the invention provides a 16x 4-level reference frame compression method, which comprises the following steps:

acquiring 16x4 levels of reference frames, wherein the reference frames are used for performing motion compensation in the encoding and decoding process;

dividing the reference frame into 4 by 4 pixel blocks;

sequentially obtaining the minimum value of each pixel block in the 4 pixel blocks by 4x 4;

according to the minimum value of each pixel block, solving a minimum value tree and a bit width tree of each pixel block in 4 pixel blocks of 4x 4;

and solving a minimum value tree and a bit width tree of the 16x 4-level reference frame according to the minimum value of each pixel block.

In one embodiment, sequentially obtaining a minimum value of each of the 4 × 4 pixel blocks includes:

the minimum value in each 4 × 4 pixel block is taken as the minimum value of the 4 × 4 pixel block.

In one embodiment, the obtaining the minimum value tree and the bit width tree of each pixel block of the 4 × 4 pixel blocks according to the minimum value of each pixel block includes:

dividing the current 4x4 pixel blocks into 4 2x2 pixel blocks;

and subtracting the minimum value in each 2 × 2 pixel block from the value in each 2 × 2 pixel block to obtain the minimum value tree of the 4 × 4 pixel blocks.

determining the binary digit number corresponding to the maximum value in each 2x2 pixel block in the minimum value tree and the binary digit number corresponding to the minimum value in the 4x4 pixel block;

and taking the binary digit number corresponding to the maximum value in each 2x2 pixel block in the minimum value tree and the binary digit number corresponding to the minimum value in the 4x4 pixel block as the bit width tree corresponding to the 4x4 pixel block.

In one embodiment, after the minimum value tree and the bit width tree of each pixel block in 4 × 4 pixel blocks are obtained according to the minimum value of each pixel block, the minimum value tree and the bit width tree are stored in the TEXTBUFFER.

The embodiment of the present invention further provides a 16x 4-level reference frame compression apparatus, including:

an obtaining module, configured to obtain 16 × 4 levels of reference frames, where the reference frames are used for performing motion compensation in an encoding and decoding process;

a dividing module for dividing the reference frame into 4 × 4 pixel blocks;

the first calculating module is used for calculating the minimum value of each pixel block in the 4 pixel blocks by 4x4 in sequence;

the second obtaining module is used for obtaining a minimum value tree and a bit width tree of each pixel block in 4 pixel blocks by 4x4 according to the minimum value of each pixel block;

and the third calculating module is used for calculating the minimum value tree and the bit width tree of the 16x 4-level reference frame according to the minimum value of each pixel block.

In one embodiment, the first obtaining module is specifically configured to use a minimum value in each 4 × 4 pixel block as the minimum value of the 4 × 4 pixel block.

The embodiment of the invention also provides terminal equipment which comprises a processor and a memory for storing the executable instructions of the processor, wherein the steps of the method are realized when the processor executes the instructions.

Embodiments of the present invention also provide a computer-readable storage medium having stored thereon computer instructions, which when executed, implement the steps of the above-described method.

In the embodiment of the invention, 16x4 levels of reference frames are obtained, wherein the reference frames are used for performing motion compensation in the encoding and decoding process; by compressing the 16x 4-level reference frames, the requirements on bandwidth for reading and storing the reference frames in the encoding and decoding process are lower, so that the technical problem of overlarge bandwidth requirement in the existing encoding and decoding process can be solved, and the technical effect of reducing the bandwidth is achieved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention. In the drawings:

fig. 1 is a video coding method architecture diagram according to the present application;

FIG. 2 is a schematic diagram of a reconstructed frame storage method according to the application;

FIG. 3 is a schematic representation of a reconstructed frame according to the present application;

FIG. 4 is a schematic mb diagram according to the present application;

FIG. 5 is a schematic view of a transverse 4x4 run in series according to the present application;

FIG. 6 is a diagram of a specific example of specific pixel values of 4x4(sbk0) according to the present application;

fig. 7 is a schematic of the minimization of 2x2 according to the present application;

FIG. 8 is a schematic diagram of minimum tree formation according to the present application;

FIG. 9 is a schematic diagram of binary scaling according to the present application;

FIG. 10 is a diagram illustrating a bit width tree corresponding to a minimum tree according to the present application;

FIG. 11 is a schematic diagram of a constrained bit-width tree according to the present application;

FIG. 12 is a schematic of zero _ idx according to the present application;

FIG. 13 is a minimum value schematic of 4x4 blocks according to the present application;

FIG. 14 is a method flow diagram of a 16x4 stage reference frame compression method according to the present application;

fig. 15 is a block diagram of a 16x 4-stage reference frame compression apparatus according to the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the following embodiments and accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.

In this example, in order to reduce the bandwidth of the DDR, it is considered that data can be compressed in the process of video encoding, and the compressed data is stored.

Specifically, as shown in fig. 2, in the process of video encoding and decoding, the compressed data (TEXT) is stored in the TEXT BUFFER, and the LIST identifying each mb information is stored in the LIST BUFFER. Where mb denotes a 16x16 block, i.e., a block of size 16 pixels by 16 pixels. TEXT represents data after the reconstructed frame is compressed, and LIST is used to identify data of the TEXT address size.

To better illustrate the present application, some of the words and concepts involved in FIG. 2 are explained below:

1) original frame, one frame in the original video image.

2) Reconstructed frame, recon frame, reconstructed image frame after encoding and decoding the original frame.

3) A group, 16 × 4 block, i.e., a block of size 16 pixels by 4 pixels, is the basic compression unit.

4) sbk, 4x4 blocks, i.e., blocks of 4 pixels by 4 pixels in size.

5) mbk, 2x2 tiles, i.e., 2 pixels by 2 pixels in size, with 4 mbk in one sbk.

6) pxl, 1 pixel.

Reconstructed frames may be as shown in fig. 3, one reconstructed frame may contain some number of mbs as shown in fig. 4, and one mb contains 16 4 × 4 (sbk). The compression unit is 16x4, i.e. as shown in fig. 5, 4x4 consecutive in the transverse direction, for example: sbk0, sbk1, sbk2, sbk 3. Each square block represents 4x4, and the minimum value is selected at the 4x4 level to generate a minimum value tree and a bit width tree.

Taking a specific pixel value of 4 × 4(sbk0) as an example, the compression process is explained as follows: as shown in fig. 6, for a specific pixel value of 4 × 4(sbk0), the following steps may be included:

s1: a minimum tree is calculated.

As shown in fig. 7, the minimum value is taken for each 2 × 2, and the minimum value is taken again for 4 minimum values.

As shown in fig. 8, the new minimum tree is formed by subtracting the respective minimum values from the 4 2 × 2 values, and subtracting the minimum value (5 of 5, 11, 12, 11) from the 4 minimum value groups. That is, the minimum square 5 is the minimum value of the 4 × 4 blocks, and the remaining values are the difference values.

S2: and calculating a bit width tree.

The correspondence between values and bit widths can be as shown in table 1 below:

TABLE 1

Decimal number	Conversion to binary	Bit width
				0	0	1
1	1	1
			2	10	2
3	11	2
			4	100	3
5	101	3
			6	110	3
7	111	3
			8	1000	4
18	10010	5
			34	100010	6
31	11111	5
			54	110110	6

The maximum value of the 4 numbers is taken and the bit width is calculated.

For example: as shown in fig. 9, the maximum value of the 1 st 2 × 2 is 18, which is 10010 converted to binary, and therefore 5 bits.

Therefore, the bit width tree corresponding to the minimum value tree is shown in fig. 10.

When implemented, the following constraints can be set:

1) when 4 values of bc _ f4 are all 0, bc _ f3 is equal to 15.

2) bc _ f3 is not less than the maximum value in bc _ f4 minus 1.

3) Each value in bc _ f4 is not less than bc _ f3 minus 2.

After being constrained by the above constraint conditions, the bit width tree becomes as shown in fig. 11.

The 4 values of bc _ f4 are respectively subtracted from bc _ f3 to obtain four difference values: bc _ f4_ diff0, bc _ f4_ diff1, bc _ f4_ diff2 and bc _ f4_ diff 3.

Where a diff value of 0 represents a difference of-2, 1 represents a difference of-1, 2 represents a difference of 0, and 3 represents a difference of 1, then 4 diff values are written in the TEXT.

Wherein bc _ f3 equals 15 if all pixels within the 4x4 block are the same; if min _ f3 are all equal to 0, bc _ f3 is equal to 0; if min _ f3 equals 0 or 1, bc _ f3 equals 1, only 4 1-bit values bin _ f3_ mbk of min _ f3 need to be stored; if min _ f4 equals 0 or 1, bc _ f4 equals 1, only 4 1-bit values bin _ f4_ pxl of min _ f4 need to be stored; if 4 of the values bc _ f3 and bc _ f4 are greater than 1, zero _ idx is calculated for each of the 4 node minimum trees, where zero _ idx may be as shown in fig. 12.

Specifically, as shown in FIG. 13, the minimum value min _ f2 of 4x4 blocks (sbk0, sbk1, sbk2, sbk3) can be obtained in sequence,

sbk0_ min _ f2, sbk1_ min _ f2, sbk2_ min _ f2, sbk3_ min _ f 2. Then, as shown in fig. 13, a new minimum value tree and a new bit width tree are generated according to the above method, resulting in min _ f1 and 4 min _ f 2.

Meanwhile, bc _ f2 is calculated according to min _ f2, bc _ f2 is not restricted and has a value range of 0-8 and 15, wherein 0 represents that 4 values of min _ f2 are all 0, and 15 represents that pixels in 16x4 are all equal to min _ f 1. When bc _ f2 is equal to 1, 4 1-bit values bin _ f2_ sbk are also written, and when bc _ f2 is greater than 1 and less than 15, zero _ idx _ f2 and three bins _ f2_ idx are also written.

S3: TEXT write format

After compression, the format of TEXT write to memory can be as shown in table 2 below:

TABLE 2

S4: LIST write format:

LIST information of each mb occupies 64bits, and the format is shown in table 3 below:

TABLE 3

In this specification, adjectives such as first and second may only be used to distinguish one element or action from another, without necessarily requiring or implying any actual such relationship or order. References to an element or component or step (etc.) should not be construed as limited to only one of the element, component, or step, but rather to one or more of the element, component, or step, etc., where the context permits.

Fig. 14 is a 16 × 4 reference frame compression method according to an embodiment of the present disclosure. Although the flow described below includes operations that occur in a particular order, it should be appreciated that the processes may include more or less operations that are performed sequentially or in parallel (e.g., using parallel processors or a multi-threaded environment). As shown in fig. 14, the method includes:

step 1401: acquiring 16x4 levels of reference frames, wherein the reference frames are used for performing motion compensation in the encoding and decoding process;

step 1402: dividing the reference frame into 4 by 4 pixel blocks;

step 1403: sequentially obtaining the minimum value of each pixel block in the 4 pixel blocks by 4x 4;

step 1404: according to the minimum value of each pixel block, solving a minimum value tree and a bit width tree of each pixel block in 4 pixel blocks of 4x 4;

step 1405: and solving a minimum value tree and a bit width tree of the 16x 4-level reference frame according to the minimum value of each pixel block.

Specifically, the minimum value of each of the 4 pixel blocks by 4 may be sequentially obtained by taking the minimum value of each of the 4 pixel blocks by 4 as the minimum value of the 4 pixel blocks by 4.

In the step 104, obtaining the minimum tree and the bit width tree of each pixel block of the 4 × 4 pixel blocks according to the minimum value of each pixel block may include:

s1: dividing the current 4x4 pixel blocks into 4 2x2 pixel blocks;

s2: and subtracting the minimum value in each 2 × 2 pixel block from the value in each 2 × 2 pixel block to obtain the minimum value tree of the 4 × 4 pixel blocks.

S3: determining the binary digit number corresponding to the maximum value in each 2x2 pixel block in the minimum value tree and the binary digit number corresponding to the minimum value in the 4x4 pixel block;

s4: and taking the binary digit number corresponding to the maximum value in each 2x2 pixel block in the minimum value tree and the binary digit number corresponding to the minimum value in the 4x4 pixel block as the bit width tree corresponding to the 4x4 pixel block.

After the minimum value tree and the bit width tree of each pixel block in the 4 by 4 pixel blocks are obtained according to the minimum value of each pixel block, the minimum value tree and the bit width tree can be stored in the TEXT BUFFER, and because the minimum value tree and the bit width tree are compressed frames, the bandwidth required in the video coding and decoding process can be reduced.

The following constraint conditions may also be set for the bit width of the minimum value:

1) the bit width of the minimum value is not less than the bit width minus 1 of the maximum value in each 2x2 pixel block;

2) the bit width of the maximum value in each 2x2 pixel block is not less than the bit width of the minimum value minus 2.

The reconstructed frame may be a 16 × 4 pixel frame, and certainly, in an actual implementation, the reconstructed frame may also be a pixel frame of other sizes.

Based on the same inventive concept, embodiments of the present invention further provide a 16 × 4 level reference frame compression apparatus, as described in the following embodiments. Since the principle of the 16 × 4 reference frame compression apparatus for solving the problem is similar to the 16 × 4 reference frame compression method, the implementation of the 16 × 4 reference frame compression apparatus may refer to the implementation of the 16 × 4 reference frame compression method, and repeated details are not repeated. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated. Fig. 15 is a block diagram of a structure of a 16 × 4-level reference frame compression apparatus according to an embodiment of the present invention, and as shown in fig. 15, the apparatus may include: the following describes the configuration of the acquisition module 1501, the division module 1502, the first obtaining module 1503, the second obtaining module 1504, and the third obtaining module 1505.

An obtaining module 1501, configured to obtain 16 × 4 levels of reference frames, where the reference frames are used for performing motion compensation in an encoding and decoding process;

a dividing module 1502 for dividing the reference frame into 4 × 4 pixel blocks;

a first obtaining module 1503, configured to sequentially obtain a minimum value of each pixel block of the 4 × 4 pixel blocks;

a second obtaining module 1504, configured to obtain a minimum tree and a bit width tree of each pixel block of the 4 × 4 pixel blocks according to the minimum value of each pixel block;

the third calculating module 1505 is configured to calculate a minimum value tree and a bit width tree of the 16 × 4 level reference frame according to the minimum value of each pixel block.

In one embodiment, the first obtaining module 1503 may specifically use the minimum value in each 4 × 4 pixel block as the minimum value of the 4 × 4 pixel block.

In one embodiment, the second obtaining module 1504 may include: a dividing unit, configured to divide a current 4 × 4 pixel block into 4 2 × 2 pixel blocks; and the calculating unit is used for subtracting the minimum value in each 2 × 2 pixel block from the minimum value in the 2 × 2 pixel block to obtain the minimum value tree of the 4 × 4 pixel block.

In one embodiment, the second obtaining module 1504 may include: a determining unit, configured to determine a binary digit number corresponding to a maximum value in each 2 × 2 pixel block in the minimum tree, and a binary digit number corresponding to a minimum value in the 4 × 4 pixel block; and a generating unit, configured to use the binary digit number corresponding to the maximum value in each 2 × 2 pixel block in the minimum value tree and the binary digit number corresponding to the minimum value in the 4 × 4 pixel block as the bit width tree corresponding to the 4 × 4 pixel block.

In one embodiment, after the minimum value tree and the bit width tree of each pixel block of the 4 × 4 pixel blocks are obtained according to the minimum value of each pixel block, the minimum value tree and the bit width tree may be stored in the TEXT BUFFER.

In another embodiment, a software is provided, which is used to execute the technical solutions described in the above embodiments and preferred embodiments.

In another embodiment, a storage medium is provided, in which the software is stored, and the storage medium includes but is not limited to: optical disks, floppy disks, hard disks, erasable memory, etc.

From the above description, it can be seen that the embodiments of the present invention achieve the following technical effects: acquiring 16x4 levels of reference frames, wherein the reference frames are used for performing motion compensation in the encoding and decoding process; by compressing the 16x 4-level reference frames, the requirements on bandwidth for reading and storing the reference frames in the encoding and decoding process are lower, so that the technical problem of overlarge bandwidth requirement in the existing encoding and decoding process can be solved, and the technical effect of reducing the bandwidth is achieved.

Although the present application provides method steps as described in an embodiment or flowchart, additional or fewer steps may be included based on conventional or non-inventive efforts. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. When an actual apparatus or client product executes, it may execute sequentially or in parallel (e.g., in the context of parallel processors or multi-threaded processing) according to the embodiments or methods shown in the figures.

The apparatuses or modules illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. The functionality of the modules may be implemented in the same one or more software and/or hardware implementations of the present application. Of course, a module that implements a certain function may be implemented by a plurality of sub-modules or sub-units in combination.

The methods, apparatus or modules described herein may be implemented in computer readable program code to a controller implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, Application Specific Integrated Circuits (ASICs), programmable logic controllers and embedded microcontrollers, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic for the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may therefore be considered as a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.

Some of the modules in the apparatus described herein may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary hardware. Based on such understanding, the technical solutions of the present application may be embodied in the form of software products or in the implementation process of data migration, which essentially or partially contributes to the prior art. The computer software product may be stored in a storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to perform the methods described in the various embodiments or portions of the embodiments of the present application.

The embodiments in the present specification are described in a progressive manner, and the same or similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. All or portions of the present application are operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

While the present application has been described with examples, those of ordinary skill in the art will appreciate that there are numerous variations and permutations of the present application without departing from the spirit of the application, and it is intended that the appended claims encompass such variations and permutations without departing from the spirit of the application.

Claims

1. A 16x 4-level reference frame compression method, comprising:

dividing the reference frame into 4 by 4 pixel blocks;

according to the minimum value of each pixel block, solving a minimum value tree and a bit width tree of each pixel block in 4 × 4 pixel blocks;

according to the minimum value of each pixel block, solving a minimum value tree and a bit width tree of the 16x 4-level reference frame;

sequentially calculating the minimum value of each pixel block in the 4 × 4 pixel blocks, including:

taking the minimum value in each 4 × 4 pixel block as the minimum value of the 4 × 4 pixel block;

according to the minimum value of each pixel block, solving a minimum value tree and a bit width tree of each pixel block in 4 pixel blocks by 4, wherein the minimum value tree and the bit width tree comprise:

dividing the current 4x4 pixel blocks into 4 2x2 pixel blocks;

respectively taking the minimum value of the 4 pixel blocks of 2x2 to form a minimum value group, taking the minimum value of the 4 minimum values in the minimum value group again to obtain the minimum value of the minimum value group, respectively subtracting the respective minimum value from the 4 pixel blocks of 2x2, and simultaneously respectively subtracting the minimum value from the 4 minimum values in the minimum value group to form a minimum value tree;

and obtaining a bit width tree corresponding to the 4x4 pixel block according to the binary digit number corresponding to the maximum value in each 2x2 pixel block in the minimum value tree and the binary digit number corresponding to the minimum value in the 4x4 pixel block, and storing the minimum value tree and the bit width tree into a TEXT BUFFER.

2. A 16x 4-level reference frame compression apparatus, comprising:

a dividing module for dividing the reference frame into 4 × 4 pixel blocks;

the third calculating module is used for dividing the current 4x4 pixel blocks into 4 2x2 pixel blocks; respectively taking the minimum value of the 4 pixel blocks of 2x2 to form a minimum value group, taking the minimum value of the 4 minimum values in the minimum value group again to obtain the minimum value of the minimum value group, respectively subtracting the respective minimum value from the 4 pixel blocks of 2x2, and simultaneously respectively subtracting the minimum value from the 4 minimum values in the minimum value group to form a minimum value tree;