CN110933443A

CN110933443A - Video reconstruction frame storage method and device

Info

Publication number: CN110933443A
Application number: CN201811100642.3A
Authority: CN
Inventors: 曲道远
Original assignee: Beijing Ingenic Semiconductor Co Ltd
Current assignee: Beijing Ingenic Semiconductor Co Ltd
Priority date: 2018-09-20
Filing date: 2018-09-20
Publication date: 2020-03-27

Abstract

The invention provides a video reconstruction frame storage method and a video reconstruction frame storage device, wherein the method comprises the following steps: acquiring a reconstructed frame; dividing the reconstructed frame into a plurality of 4 × 4 pixel blocks in sequence; calculating a minimum value tree and a bit width tree of each 4 × 4 pixel block in the plurality of 4 × 4 pixel blocks; calculating to obtain a minimum value tree and a bit width tree of the reconstructed frame according to the minimum value tree and the bit width tree of each 4-by-4 pixel block; and writing the minimum value tree and the bit width tree of the reconstructed frame into a memory. The scheme realizes the compression processing of the reconstructed frame, and because the reconstructed frame is the compressed data, the bandwidth requirement can be reduced in the encoding and decoding process.

Description

Video reconstruction frame storage method and device

Technical Field

The present invention relates to the field of data processing technologies, and in particular, to a method and an apparatus for storing video reconstruction frames.

Background

As shown in fig. 1, in the video codec, the steps shown in fig. 1 need to be performed: entropy coding, inverse quantization, inverse transformation, motion compensation, reference frames, external storage, etc. These require constant data exchange with off-chip memory, including: writing of a reconstructed frame, acquisition of a reference frame, and the like. While data is being written or retrieved, a significant amount of bandwidth consumption is incurred.

In the encoding and decoding process, the DDR needs to be accessed in real time to obtain the required data, and due to the huge bandwidth requirement, the DDR consumes a large part of power generated by writing and reading data, so that the DDR bandwidth reduction becomes particularly important in video encoding and decoding.

No effective solution has been proposed at present for the DDR bandwidth if reduced.

Disclosure of Invention

The embodiment of the invention provides a video reconstruction frame storage method and a video reconstruction frame storage device, which aim to reduce the bandwidth requirement of DDR in the video coding and decoding process.

The embodiment of the invention provides a video reconstruction frame storage method, which comprises the following steps:

acquiring a reconstructed frame;

dividing the reconstructed frame into a plurality of 4 × 4 pixel blocks in sequence;

calculating a minimum value tree and a bit width tree of each 4 × 4 pixel block in the plurality of 4 × 4 pixel blocks;

calculating to obtain a minimum value tree and a bit width tree of the reconstructed frame according to the minimum value tree and the bit width tree of each 4-by-4 pixel block;

and writing the minimum value tree and the bit width tree of the reconstructed frame into a memory.

In one embodiment, calculating a minimum tree for each 4 × 4 pixel block of the plurality of 4 × 4 pixel blocks includes:

dividing the current 4x4 pixel blocks into 4 2x2 pixel blocks;

determining the minimum pixel value in each 2x2 pixel block to obtain 4 pixel values;

taking the minimum value in the 4 pixel values as the minimum value;

and respectively subtracting the minimum value of the corresponding 2 × 2 pixel blocks from the 4 2 × 2 pixel blocks, and determining the minimum pixel value in each 2 × 2 pixel block after the minimum value is subtracted to obtain the minimum value tree.

In one embodiment, calculating the bit width tree for each 4 × 4 pixel block of the plurality of 4 × 4 pixel blocks includes:

determining the maximum value of each 2x2 pixel block after the 4 2x2 pixel blocks are respectively subtracted by the minimum value;

converting the maximum value in each 2x2 pixel block into a binary value;

and converting the maximum value in each 2x2 pixel block into binary digits as bit width to obtain a bit width tree.

In one embodiment, after converting the maximum value in each 2 × 2 pixel block into a binary number of bits as a bit width to obtain a bit width tree, the method further includes:

determining the bit width of the minimum value according to the following constraint conditions:

the bit width of the minimum value is not less than the bit width minus 1 of the maximum value in the maximum values in each 2x2 pixel block;

the bit width of the maximum value in each 2x2 pixel block is not less than the bit width of the minimum value minus 2.

In one embodiment, the reconstructed frame is a 16x4 frame of pixels.

The present application further provides a video reconstruction frame storage device, including:

an obtaining module, configured to obtain a reconstructed frame;

a dividing module, configured to divide the reconstructed frame into a plurality of 4 × 4 pixel blocks in sequence;

a first calculation module, configured to calculate a minimum tree and a bit width tree of each 4 × 4 pixel block of the plurality of 4 × 4 pixel blocks;

the second calculation module is used for calculating the minimum value tree and the bit width tree of the reconstructed frame according to the minimum value tree and the bit width tree of each 4x4 pixel block;

and the writing module is used for writing the minimum value tree and the bit width tree of the reconstructed frame into a memory.

In one embodiment, the first calculation module comprises:

a dividing unit, configured to divide a current 4 × 4 pixel block into 4 2 × 2 pixel blocks;

a first determining unit, configured to determine a minimum pixel value in each 2 × 2 pixel block, to obtain 4 pixel values;

a generating unit, configured to take a minimum value of the 4 pixel values as a minimum value;

and the second determining unit is used for subtracting the minimum value of the corresponding 2 × 2 pixel blocks from the 4 2 × 2 pixel blocks respectively, and determining the minimum pixel value in each 2 × 2 pixel block after subtracting the minimum value to obtain the minimum value tree.

In one embodiment, the first computing module further comprises:

a third determining unit, configured to determine a maximum value in each 2 × 2 pixel block after subtracting the minimum value from each of the 4 2 × 2 pixel blocks;

a conversion unit for converting the maximum value in each 2 × 2 pixel block into a binary value;

and the generating unit is used for converting the maximum value in each 2x2 pixel block into binary digits as bit width to obtain a bit width tree.

The application also provides a terminal device, which comprises a processor and a memory for storing processor executable instructions, wherein the processor executes the instructions to realize the steps of the method.

The present application also provides a computer readable storage medium having stored thereon computer instructions which, when executed, implement the steps of the above-described method.

In the embodiment of the invention, the reconstructed frame is divided into a plurality of 4-by-4 pixel blocks in sequence, then the minimum value tree and the bit width tree of each 4-by-4 pixel block in the plurality of 4-by-4 pixel blocks are obtained through calculation, and then the minimum value tree and the bit width tree are obtained through calculation; and reconstructing the minimum value tree and the bit width tree of the frame and writing the minimum value tree and the bit width tree into the memory, thereby realizing the compression processing of the reconstructed frame.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention. In the drawings:

fig. 1 is a video coding method architecture diagram according to the present application;

FIG. 2 is a schematic diagram of a reconstructed frame storage method according to the application;

FIG. 3 is a schematic representation of a reconstructed frame according to the present application;

FIG. 4 is a schematic mb diagram according to the present application;

FIG. 5 is a schematic view of a transverse 4x4 run in series according to the present application;

FIG. 6 is a diagram of a specific example of specific pixel values of 4x4(sbk0) according to the present application;

fig. 7 is a schematic of the minimization of 2x2 according to the present application;

FIG. 8 is a schematic diagram of minimum tree formation according to the present application;

FIG. 9 is a schematic diagram of binary scaling according to the present application;

FIG. 10 is a diagram illustrating a bit width tree corresponding to a minimum tree according to the present application;

FIG. 11 is a schematic diagram of a constrained bit-width tree according to the present application;

FIG. 12 is a schematic of zero _ idx according to the present application;

FIG. 13 is a minimum value schematic of 4x4 blocks according to the present application;

FIG. 14 is a method flow diagram of a video reconstruction frame storage method according to the present application;

fig. 15 is a block diagram of a video reconstruction frame storage apparatus according to the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the following embodiments and accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.

In this example, in order to reduce the bandwidth of the DDR, it is considered that data can be compressed in the process of video encoding, and the compressed data is stored.

Specifically, as shown in fig. 2, in the process of video encoding and decoding, the compressed data (TEXT) is stored in the TEXT BUFFER, and the LIST identifying each mb information is stored in the LIST BUFFER. Where mb denotes a 16x16 block, i.e., a block of size 16 pixels by 16 pixels. TEXT represents data after the reconstructed frame is compressed, and LIST is used to identify data of the TEXT address size.

To better illustrate the present application, some of the words and concepts involved in FIG. 2 are explained below:

1) original frame, one frame in the original video image.

2) Reconstructed frame, recon frame, reconstructed image frame after encoding and decoding the original frame.

3) A group, 16 × 4 block, i.e., a block of size 16 pixels by 4 pixels, is the basic compression unit.

4) sbk, 4x4 blocks, i.e., blocks of 4 pixels by 4 pixels in size.

5) mbk, 2x2 blocks, i.e., blocks of 2 pixels by 2 pixels in size, one sbk contains 4 mbk.

6) pxl, 1 pixel.

The reconstructed frame may be as shown in fig. 3, one reconstructed frame may contain some number of mb as shown in fig. 4, and one mb contains 16 4 × 4 (sbk). The compression unit is 16x4, i.e. as shown in fig. 5, 4x4 consecutive in the transverse direction, for example: sbk0, sbk1, sbk2, sbk 3. Each square block represents 4x4, and the minimum value is selected at the 4x4 level to generate a minimum value tree and a bit width tree.

Taking a specific pixel value of 4 × 4(sbk0) as an example, the compression process is explained as follows: as shown in fig. 6, for a specific pixel value of 4 × 4(sbk0), the following steps may be included:

s1: a minimum tree is calculated.

As shown in fig. 7, the minimum value is taken for each 2 × 2, and the minimum value is taken again for 4 minimum values.

As shown in fig. 8, the new minimum tree is formed by subtracting the respective minimum values from the 4 2 × 2 values, and subtracting the minimum value (5 of 5, 11, 12, 11) from the 4 minimum value groups. That is, the minimum square 5 is the minimum value of the 4 × 4 blocks, and the remaining values are the difference values.

S2: and calculating a bit width tree.

The correspondence between values and bit widths can be as shown in table 1 below:

TABLE 1

Decimal number	Conversion to binary	Bit width
				0	0	1
1	1	1
			2	10	2
3	11	2
			4	100	3
5	101	3
			6	110	3
7	111	3
			8	1000	4
18	10010	5
			34	100010	6
31	11111	5
			54	110110	6

The maximum value of the 4 numbers is taken and the bit width is calculated.

For example: as shown in fig. 9, the maximum value of the 1 st 2 × 2 is 18, which is 10010 converted to binary, and therefore 5 bits.

Therefore, the bit width tree corresponding to the minimum value tree is shown in fig. 10.

When implemented, the following constraints can be set:

1) when 4 values of bc _ f4 are all 0, bc _ f3 is equal to 15.

2) bc _ f3 is not less than the maximum value in bc _ f4 minus 1.

3) Each value in bc _ f4 is not less than bc _ f3 minus 2.

After being constrained by the above constraint conditions, the bit width tree becomes as shown in fig. 11.

The 4 values of bc _ f4 are respectively subtracted from bc _ f3 to obtain four difference values: bc _ f4_ diff0, bc _ f4_ diff1, bc _ f4_ diff2, and bc _ f4_ diff 3.

Where a diff value of 0 represents a difference of-2, 1 represents a difference of-1, 2 represents a difference of 0, and 3 represents a difference of 1, then 4 diff values are written in the TEXT.

Wherein bc _ f3 equals 15 if all pixels within the 4x4 block are the same; if min _ f3 are all equal to 0, bc _ f3 is equal to 0; if min _ f3 equals 0 or 1, bc _ f3 equals 1, only 4 1-bit values bin _ f3_ mbk of min _ f3 need to be stored; if min _ f4 equals 0 or 1, bc _ f4 equals 1, only 4 1-bit values bin _ f4_ pxl of min _ f4 need to be stored; if 4 of the values bc _ f3 and bc _ f4 are greater than 1, zero _ idx is calculated for each of the 4 node minimum trees, where zero _ idx may be as shown in fig. 12.

Specifically, as shown in FIG. 13, the minimum value min _ f2 of 4 × 4 blocks (sbk0, sbk1, sbk2, sbk3) can be obtained in sequence,

sbk0_ min _ f2, sbk1_ min _ f2, sbk2_ min _ f2, sbk3_ min _ f 2. Then, as shown in fig. 13, a new minimum value tree and a new bit width tree are generated according to the above method, resulting in min _ f1 and 4 min _ f 2.

Meanwhile, bc _ f2 is calculated according to min _ f2, the bc _ f2 is not restricted, and has the value range of 0-8 and 15, wherein 0 represents that 4 values of min _ f2 are all 0, and 15 represents that the 16x4 pixels are all equal to min _ f 1. When bc _ f2 is equal to 1, 4 1-bit values bin _ f2_ sbk are also written, and when bc _ f2 is greater than 1 and less than 15, zero _ idx _ f2 and three bins _ f2_ idx are also written.

S3: TEXT write format

After compression, the format of TEXT write to memory can be as shown in table 2 below:

TABLE 2

S4: LIST write format:

LIST information of each mb occupies 64bits, and the format is shown in table 3 below:

TABLE 3

In this specification, adjectives such as first and second may only be used to distinguish one element or action from another, without necessarily requiring or implying any actual such relationship or order. References to an element or component or step (etc.) should not be construed as limited to only one of the element, component, or step, but rather to one or more of the element, component, or step, etc., where the context permits.

Fig. 14 is a storage method of video reconstruction frames according to an embodiment of the present application. Although the flow described below includes operations that occur in a particular order, it should be appreciated that the processes may include more or less operations that are performed sequentially or in parallel (e.g., using parallel processors or a multi-threaded environment). As shown in fig. 14, the method includes:

step 1401: acquiring a reconstructed frame;

step 1402: dividing the reconstructed frame into a plurality of 4 × 4 pixel blocks in sequence;

step 1403: calculating a minimum value tree and a bit width tree of each 4 × 4 pixel block in the plurality of 4 × 4 pixel blocks;

step 1404: calculating to obtain a minimum value tree and a bit width tree of the reconstructed frame according to the minimum value tree and the bit width tree of each 4-by-4 pixel block;

step 1405: and writing the minimum value tree and the bit width tree of the reconstructed frame into a memory.

In the above example, the reconstructed frame is divided into a plurality of 4 × 4 pixel blocks in sequence, and then the minimum value tree and the bit width tree of each 4 × 4 pixel block in the plurality of 4 × 4 pixel blocks are obtained through calculation; and reconstructing the minimum value tree and the bit width tree of the frame and writing the minimum value tree and the bit width tree into the memory, thereby realizing the compression processing of the reconstructed frame.

Specifically, in step 1403, the calculating a minimum tree of each 4 × 4 pixel block in the plurality of 4 × 4 pixel blocks may include:

s1: dividing the current 4x4 pixel blocks into 4 2x2 pixel blocks;

s2: determining the minimum pixel value in each 2x2 pixel block to obtain 4 pixel values;

s3: taking the minimum value in the 4 pixel values as the minimum value of a 4x4 pixel block;

s4: and respectively subtracting the minimum value of the corresponding 2 × 2 pixel blocks from the 4 2 × 2 pixel blocks, and determining the minimum pixel value in each 2 × 2 pixel block after the minimum value is subtracted to obtain the minimum value tree.

S5: determining the maximum value of each 2x2 pixel block after the 4 2x2 pixel blocks are respectively subtracted by the minimum value;

s6: converting the maximum value in each 2x2 pixel block into a binary value;

s7: and converting the maximum value in each 2x2 pixel block into binary digits as bit width to obtain a bit width tree.

The following constraint conditions may also be set for the bit width of the minimum value:

1) the bit width of the minimum value is not less than the bit width minus 1 of the maximum value in the maximum values in each 2x2 pixel block;

2) the bit width of the maximum value in each 2x2 pixel block is not less than the bit width of the minimum value minus 2.

The reconstructed frame may be a 16 × 4 pixel frame, and certainly, in an actual implementation, the reconstructed frame may also be a pixel frame of other sizes.

Based on the same inventive concept, embodiments of the present invention further provide a video reconstruction frame storage apparatus, as described in the following embodiments. The principle of the video reconstruction frame storage device for solving the problems is similar to that of the video reconstruction frame storage method, so the implementation of the video reconstruction frame storage device can refer to the implementation of the video reconstruction frame storage method, and repeated parts are not repeated. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated. Fig. 15 is a block diagram of a video reconstruction frame storage apparatus according to an embodiment of the present invention, and as shown in fig. 15, the video reconstruction frame storage apparatus may include: the structure of the acquisition module 1501, the division module 1502, the first calculation module 1503, the second calculation module 1504, and the write module 1505 will be described below.

An obtaining module 1501, configured to obtain a reconstructed frame;

a dividing module 1502, configured to divide the reconstructed frame into a plurality of 4 × 4 pixel blocks in sequence;

a first calculating module 1503, configured to calculate a minimum value tree and a bit width tree of each 4 × 4 pixel block of the plurality of 4 × 4 pixel blocks;

a second calculating module 1504, configured to calculate a minimum tree and a bit width tree of the reconstructed frame according to the minimum tree and the bit width tree of each 4 × 4 pixel block;

a writing module 1505 is used for writing the minimum value tree and the bit width tree of the reconstructed frame into a memory.

In one embodiment, the first calculation module 1503 may include: a dividing unit, configured to divide a current 4 × 4 pixel block into 4 2 × 2 pixel blocks; a first determining unit, configured to determine a minimum pixel value in each 2 × 2 pixel block, to obtain 4 pixel values; a generating unit, configured to take a minimum value of the 4 pixel values as a minimum value; and the second determining unit is used for subtracting the minimum value of the corresponding 2 × 2 pixel blocks from the 4 2 × 2 pixel blocks respectively, and determining the minimum pixel value in each 2 × 2 pixel block after subtracting the minimum value to obtain the minimum value tree.

In one embodiment, the first calculation module 1503 may further include: a third determining unit, configured to determine a maximum value in each 2 × 2 pixel block after subtracting the minimum value from each of the 4 2 × 2 pixel blocks; a conversion unit for converting the maximum value in each 2 × 2 pixel block into a binary value; and the generating unit is used for converting the maximum value in each 2x2 pixel block into binary digits as bit width to obtain a bit width tree.

In another embodiment, a software is provided, which is used to execute the technical solutions described in the above embodiments and preferred embodiments.

In another embodiment, a storage medium is provided, in which the software is stored, and the storage medium includes but is not limited to: optical disks, floppy disks, hard disks, erasable memory, etc.

From the above description, it can be seen that the embodiments of the present invention achieve the following technical effects: dividing a reconstructed frame into a plurality of 4-by-4 pixel blocks in sequence, then calculating to obtain a minimum value tree and a bit width tree of each 4-by-4 pixel block in the plurality of 4-by-4 pixel blocks, and then calculating to obtain the minimum value tree and the bit width tree; and reconstructing the minimum value tree and the bit width tree of the frame and writing the minimum value tree and the bit width tree into the memory, thereby realizing the compression processing of the reconstructed frame.

Although the present application provides method steps as described in an embodiment or flowchart, additional or fewer steps may be included based on conventional or non-inventive efforts. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. When an actual apparatus or client product executes, it may execute sequentially or in parallel (e.g., in the context of parallel processors or multi-threaded processing) according to the embodiments or methods shown in the figures.

The apparatuses or modules illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. The functionality of the modules may be implemented in the same one or more software and/or hardware implementations of the present application. Of course, a module that implements a certain function may be implemented by a plurality of sub-modules or sub-units in combination.

The methods, apparatus or modules described herein may be implemented in computer readable program code to a controller implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, Application Specific Integrated Circuits (ASICs), programmable logic controllers and embedded microcontrollers, examples of which include, but are not limited to, the following microcontrollers: the ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic for the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may therefore be considered as a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.

Some of the modules in the apparatus described herein may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary hardware. Based on such understanding, the technical solutions of the present application may be embodied in the form of software products or in the implementation process of data migration, which essentially or partially contributes to the prior art. The computer software product may be stored in a storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to perform the methods described in the various embodiments or portions of the embodiments of the present application.

The embodiments in the present specification are described in a progressive manner, and the same or similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. All or portions of the present application are operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

While the present application has been described with examples, those of ordinary skill in the art will appreciate that there are numerous variations and permutations of the present application without departing from the spirit of the application, and it is intended that the appended claims encompass such variations and permutations without departing from the spirit of the application.

Claims

1. A method for storing video reconstruction frames, comprising:

acquiring a reconstructed frame;

2. The method of claim 1, wherein computing a minimum tree for each 4x4 pixel block of the plurality of 4x4 pixel blocks comprises:

dividing the current 4x4 pixel blocks into 4 2x2 pixel blocks;

taking the minimum value in the 4 pixel values as the minimum value of the 4x4 pixel blocks;

3. The method of claim 3, wherein computing the bit width tree for each 4x4 pixel block of the plurality of 4x4 pixel blocks comprises:

converting the maximum value in each 2x2 pixel block into a binary value;

4. The method according to claim 3, wherein after converting the maximum value in each 2x2 pixel block into a binary number of bits as a bit width to obtain a bit width tree, the method further comprises:

5. The method according to any one of claims 1 to 4, wherein the reconstructed frame is a 16x4 frame of pixels.

6. A video reconstruction frame storage apparatus, comprising:

an obtaining module, configured to obtain a reconstructed frame;

7. The apparatus of claim 6, wherein the first computing module comprises:

and the second determining unit is used for subtracting the corresponding 2 × 2 pixel blocks from the 4 2 × 2 pixel blocks respectively, and determining the minimum pixel value in each 2 × 2 pixel block after the minimum value is subtracted to obtain the minimum value tree.

8. The apparatus of claim 7, wherein the first computing module further comprises:

9. A terminal device comprising a processor and a memory for storing processor-executable instructions which, when executed by the processor, implement the steps of the method of any one of claims 1 to 5.

10. A computer readable storage medium having stored thereon computer instructions which, when executed, implement the steps of the method of any one of claims 1 to 5.