CN116527895A - Space-borne heterogeneous H.264 video compression coding system and coding method - Google Patents
Space-borne heterogeneous H.264 video compression coding system and coding method Download PDFInfo
- Publication number
- CN116527895A CN116527895A CN202310374018.7A CN202310374018A CN116527895A CN 116527895 A CN116527895 A CN 116527895A CN 202310374018 A CN202310374018 A CN 202310374018A CN 116527895 A CN116527895 A CN 116527895A
- Authority
- CN
- China
- Prior art keywords
- parallel
- block
- macroblock
- module
- level
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000007906 compression Methods 0.000 title claims abstract description 33
- 230000006835 compression Effects 0.000 title claims abstract description 33
- 238000013139 quantization Methods 0.000 claims abstract description 65
- 238000001914 filtration Methods 0.000 claims abstract description 27
- 230000001133 acceleration Effects 0.000 claims abstract description 25
- 238000006243 chemical reaction Methods 0.000 claims abstract description 24
- 238000003384 imaging method Methods 0.000 claims abstract description 6
- 238000012545 processing Methods 0.000 claims description 23
- 230000008569 process Effects 0.000 claims description 12
- 238000004458 analytical method Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000013461 design Methods 0.000 claims description 6
- 238000013144 data compression Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 4
- 241000023320 Luma <angiosperm> Species 0.000 claims description 2
- 238000006073 displacement reaction Methods 0.000 claims description 2
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 claims description 2
- 230000001131 transforming effect Effects 0.000 claims description 2
- 230000000007 visual effect Effects 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 230000006872 improvement Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000009466 transformation Effects 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000010219 correlation analysis Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000006317 isomerization reaction Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/117—Filters, e.g. for pre-processing or post-processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/129—Scanning of coding units, e.g. zig-zag scan of transform coefficients or flexible macroblock ordering [FMO]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/625—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using discrete cosine transform [DCT]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
- H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/20—Adaptations for transmission via a GHz frequency band, e.g. via satellite
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Discrete Mathematics (AREA)
- Astronomy & Astrophysics (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention belongs to the technical field of spaceborne heterogeneous video compression, and particularly relates to a spaceborne heterogeneous H.264 video compression coding system and a coding method, wherein the system is realized based on a domestic CPU and a GPU and comprises a spaceborne heterogeneous parallel forward coding subsystem and a parallel acceleration backward reconstruction subsystem; the system comprises a space-borne heterogeneous parallel forward coding subsystem, a parallel acceleration backward reconstruction subsystem, a macro-block-level parallel intra-frame prediction and inter-frame prediction, a macro-block-level parallel DCT conversion, a macro-block-level parallel quantization and coding, wherein the space-borne heterogeneous parallel forward coding subsystem is used for combining a reconstructed image output by the parallel acceleration backward reconstruction subsystem, performing macro-block-level parallel intra-frame prediction and inter-frame prediction on a video image sequence acquired by space-borne high-resolution imaging equipment in real time according to a new Z-shaped scanning sequence, and obtaining a compressed code stream; and the parallel acceleration backward reconstruction subsystem is used for carrying out parallel inverse quantization on the coefficient data blocks after the macroblock-level parallel quantization and carrying out parallel inverse DCT on the macroblock-level to obtain residual data blocks, and then obtaining a reconstructed image through macroblock-level parallel loop filtering.
Description
Technical Field
The application relates to the technical field of satellite-borne heterogeneous video compression, in particular to a satellite-borne heterogeneous H.264 video compression coding system and a coding method.
Background
With the increasing complexity and diversification of Chinese aerospace tasks, the number and the precision of the satellite-borne imaging type load are continuously improved, and the video image data volume is increased in geometric progression. The satellite mostly adopts a mechanism of firstly storing and then descending under the limit of geographic distribution of ground receiving stations. I.e. the acquired data is first put into the on-board memory and then transferred to the ground when passing the station. Because satellites are limited in aspects of volume, weight, power consumption and the like, satellite data storage and data downlink transmission bandwidth are limited, and huge amounts of video image data cause great pressure on satellite data management. In-planet video compression is a key technology to solve this problem. Processing of on-board tasks and on-board payload data using high performance heterogeneous systems is a current research hotspot.
The following FIG. 1 is a block diagram of a conventional H.264 encoding, typically implemented on an FPGA or CPU. The system comprises a left-to-right forward coding subsystem and a right-to-left backward reconstruction subsystem; wherein,,
the forward coding subsystem comprises an intra-frame prediction module, an inter-frame prediction module, a transformation module, a quantization module and an entropy coding module. The modules are controlled by the same controller and are used for encoding the input video image information to obtain a compressed code stream.
The backward reconstruction subsystem comprises an inverse quantization module, an inverse transformation module and a filtering module. The above modules are all controlled by the same controller for providing reconstructed frames for the inter prediction module.
The existing compression algorithms mainly comprise H.26X series, MPEG series, AVS series and HEVC. Wherein HEVC/H.265/H.266/AVS2/AVS3 mainly faces to high definition and ultra-high definition videos, and has higher complexity; in the MPEG series, MPEG-1 and MPEG-2 have serious blocking effect, can not be edited and played back, and under the same code rate, MPEG-4 and H.264 have miscellaneous points at the edges of the image quality words, and the network transmission speed is slow, namely the MPEG series has the characteristics of poor image quality and slow speed; in comparison, the H.264 has moderate complexity, is suitable for video transmission in a wireless channel with serious interference, can obtain better image quality in real time, and is suitable for aerospace application.
The existing heterogeneous video compression technology is mainly realized based on FPGA isomerism, DSP isomerism and CPU+GPU isomerism. However, based on FPGA heterogeneous implementation, various timing and FPGA resource limitations need to be considered, and it is difficult to implement a complex image processing algorithm. Based on DSP heterogeneous implementations, processing speeds are relatively slow. Based on the heterogeneous realization of CPU and GPU, the research is almost all heterogeneous realization based on Inter CPU and Nvidia GPU. The special application of aerospace necessitates the autonomous isomerization technology in developing China, which is urgent.
Disclosure of Invention
Aiming at the problems that an H.264 serial encoder in the technology is limited by time sequence resources, has low processing speed, is self-controlled and domesticated, and the like, the invention aims to overcome the defects in the prior art and provides a satellite-borne heterogeneous H.264 video compression coding system and a satellite-borne heterogeneous H.264 video compression coding method.
In order to achieve the above purpose, the invention provides a space-borne heterogeneous H.264 video compression coding system, which is realized based on a domestic CPU and a GPU and comprises a space-borne heterogeneous parallel forward coding subsystem and a parallel acceleration backward reconstruction subsystem; wherein,,
the space-borne heterogeneous parallel forward coding subsystem is used for combining the reconstructed images output by the parallel acceleration backward reconstruction subsystem, carrying out macro-block-level parallel intra-frame prediction and inter-frame prediction on the video image sequences acquired by the space-borne high-resolution imaging equipment in real time according to a new Z-shaped scanning sequence, and obtaining compressed code streams through macro-block-level parallel DCT conversion and macro-block-level parallel quantization and coding;
the parallel acceleration backward reconstruction subsystem is used for carrying out parallel inverse quantization on the coefficient data blocks after the parallel quantization on the macro block level, carrying out parallel inverse DCT on the macro block level to obtain residual data blocks, and carrying out parallel loop filtering on the macro block level to obtain a reconstructed image.
As an improvement of the above system, the space-borne heterogeneous parallel forward coding subsystem comprises: the system comprises a reading analysis module and an entropy coding module which are deployed in a CPU, and a macro-block-level parallel intra-frame prediction module, a macro-block-level parallel inter-frame prediction module, a macro-block-level parallel DCT conversion module and a macro-block-level parallel quantization module which are deployed in a GPU; wherein,,
the reading analysis module is used for reading the video image sequence, analyzing all the macro block information which can be processed in parallel frame by frame, storing the macro block information into an intermediate variable, and sending the macro block information to the macro block level parallel intra-frame prediction module;
the macro-block-level parallel intra-frame prediction module is used for predicting the pixels of the current block by adopting a new Z-shaped scanning sequence through adjacent pixels decoded by the parallel acceleration backward reconstruction subsystem;
the macroblock-level parallel inter-frame prediction module is used for performing motion compensation according to the calculated motion vector by taking a block which is most similar to the current block in the adjacent reference frame reconstructed by the parallel acceleration backward reconstruction subsystem as a prediction block to obtain a predicted image block;
the macro-block-level parallel DCT conversion module is used for redistributing the predicted residual data blocks in a conversion domain through macro-block-level parallel DCT conversion so as to reduce the correlation among pixels; the prediction residual data block is obtained by subtracting an original image block and a prediction image block which are stored in an intermediate variable;
the macroblock-level parallel quantization module is used for obtaining quantized coefficient data blocks through macroblock-level parallel quantization processing and realizing data compression;
the entropy coding module is used for entropy coding the quantized coefficient data blocks by combining intra-frame prediction information of the macroblock-level parallel intra-frame prediction module and motion information of the macroblock-level parallel inter-frame prediction module, reducing video coding information quantity by removing information entropy redundancy, and outputting a code stream to a network abstraction layer.
As an improvement of the above system, the macroblock-level parallel intra prediction module includes luminance 4×4 sub-blocks, the sub-block numbers being from 0 to 15;
the scanning sequence corresponding to sub-block numbers 0- >1- >2- >4- >3- >8- >6- >5- >9- >7- >10- >12- >11- >13- >14- >15 is adopted.
As an improvement of the above system, the processing procedure of the macroblock-level parallel DCT transform module and the macroblock-level parallel quantization module includes: the DCT conversion and quantization processes are combined into a whole, and on the basis of realizing the integer operation through multiplication and displacement, the GPU is utilized to process the multiplication operation in parallel, so that the real-time performance of encoding compression is improved; coarse quantization is performed on the high frequency portion and fine quantization is performed on the low frequency portion by adjusting the quantization step QP value to reduce visual redundancy and quantization error.
As an improvement of the above system, the parallel accelerated backward reconstruction subsystem includes a macroblock-level parallel inverse quantization module, a macroblock-level parallel inverse DCT module, and a macroblock-level parallel loop filter module deployed in the GPU, wherein,
the macroblock-level parallel inverse quantization module is used for carrying out parallel inverse quantization on the quantized coefficient data blocks through the macroblock level to obtain inverse quantized coefficient data blocks;
the macroblock-level parallel inverse DCT module is used for transforming the coefficient data blocks after inverse quantization into residual data blocks through macroblock-level parallel inverse DCT;
the macro-block-level parallel loop filtering module is used for carrying out loop filtering processing on the reconstructed data block, removing block effect and obtaining a reconstructed pixel block, wherein the reconstructed data block is obtained by adding a residual data block and a predicted image block.
As an improvement of the system, the macro-block-level parallel loop filtering module performs parallel optimization design according to the parameter characteristics for determining the filtering strength BS, takes 16×16 blocks as a calculation unit, takes 8 edges of one frame of image and 128 boundary points, gives one frame of image to 1 block, and uses 128 threads for parallel processing.
As an improvement of the system, the domestic CPU is a Loongson CPU, and the domestic GPU is a Wei-solid GPU.
On the other hand, the invention provides a satellite-borne heterogeneous H.264 video compression coding method which is realized based on the system, and comprises a forward coding flow and a backward reconstruction flow:
the reading analysis module reads the video image sequence, analyzes all the macroblock information which can be processed in parallel frame by frame, stores the macroblock information into an intermediate variable, sends the intermediate variable to the macroblock-level parallel intra-frame prediction module, and enters a forward coding flow:
the macro-block-level parallel intra-frame prediction module adopts a new Z-shaped scanning sequence to predict the pixels of the current block through parallel acceleration backward reconstruction subsystem decoded adjacent pixels;
the macroblock-level parallel inter-frame prediction module performs motion compensation according to the calculated motion vector by taking a block which is most similar to the current block in an adjacent reference frame reconstructed by the parallel acceleration backward reconstruction subsystem as a prediction block, so as to obtain a predicted image block;
the macroblock-level parallel DCT conversion module redistributes the predicted residual data blocks in a conversion domain through macroblock-level parallel DCT conversion so as to reduce the correlation among pixels; the prediction residual data block is obtained by subtracting an original image block and a prediction image block which are stored in an intermediate variable;
the macroblock-level parallel quantization module obtains quantized coefficient data blocks through macroblock-level parallel quantization processing, and data compression is achieved;
the entropy coding module performs entropy coding on the quantized coefficient data blocks by combining intra-frame prediction information of the macroblock-level parallel intra-frame prediction module and motion information of the macroblock-level parallel inter-frame prediction module, reduces video coding information amount by removing information entropy redundancy, and outputs a code stream to a network abstraction layer. It should be noted that, h.264 is divided into a video coding layer (Video Coding Layer, VCL for short) and a network abstraction layer (Network Abstraction Layer, NAL for short) at the system level, and compressed and coded video data (VCL data) is packaged into NAL units for transmission through a network.
The backward reconstruction process comprises the following steps:
the macroblock-level parallel inverse quantization module performs parallel inverse quantization on the quantized coefficient data blocks through the macroblock level to obtain inverse quantized coefficient data blocks;
the macroblock-level parallel inverse DCT module carries out macroblock-level parallel inverse DCT on the coefficient data blocks after inverse quantization to obtain residual data blocks;
and the macroblock-level parallel loop filtering module carries out loop filtering processing on the reconstructed data block, removes the block effect and obtains a reconstructed pixel block, and the reconstructed data block is obtained by adding a residual data block and a predicted image block.
Compared with the prior art, the invention has the advantages that:
1. the method has the advantages of parallel acceleration, effective reduction of calculation amount and independent intellectual property. The invention is beneficial to solving the limitation of the advanced aerospace country on the Chinese aerospace technology and high-performance aerospace devices, guaranteeing the aerospace safety of China and reducing the overall engineering cost; the development of aerospace computer application technology with independent intellectual property rights in China is facilitated;
2. the invention discloses a novel Z-shaped scanning sequence applied to a space-borne heterogeneous parallel forward coding subsystem for in-orbit video compression. The scanning sequence can reduce strong data correlation, reduce the waiting times required by 4×4 prediction in one macro block, and is beneficial to parallel realization, and the specific scanning sequence is as follows: 0- >1- >2- >4- >3- >8- >6- >5- >9- >7- >10- >12- >11- >13- >14- >15;
3. the macroblock-level parallel loop filtering module in the in-orbit video compression parallel acceleration backward reconstruction subsystem performs parallel optimization design according to the parameter characteristics of the determined filtering strength BS, and can accelerate the filtering speed under the condition of almost not affecting the image quality;
4. the method is oriented to the on-orbit video compression and the autonomous controllable requirement of key devices, can solve the contradiction between the downloading of on-orbit massive video data and the limited downlink transmission bandwidth of satellites, has the advantages of being free from the limitation of FPGA time sequence and resources and being capable of accelerating in parallel, and can provide a technical foundation for the on-orbit application of video compression in the subsequent model tasks.
Drawings
FIG. 1 is a flow diagram of a conventional H.264 video compression coding system;
FIG. 2 is a block diagram of a H.264 video compression coding system based on domestic CPU+GPU space-borne isomerism;
FIG. 3 is a prior art "zig-zag" scanning sequence; wherein (a) is a traditional intra luminance 4 x 4 sub-block Zig-Zag scanning order, and (b), (c), (d), (e), (f) are various existing modified scanning orders;
FIG. 4 is a new "zig-zag" scanning sequence in the on-track video compression of the present invention for the on-track heterogeneous parallel forward encoding subsystem;
FIG. 5 is a flow chart of a macroblock-level parallel DCT transform and macroblock-level parallel quantization module in the on-track video compression on-board heterogeneous parallel forward coding subsystem of the present invention;
FIG. 6 is a flow chart of a macroblock-level parallel loop filter module in the parallel accelerated backward reconstruction subsystem for in-orbit video compression according to the present invention;
fig. 7 is a schematic diagram of the present invention for compression encoding of a satellite borne heterogeneous h.264 video.
Detailed Description
The technical scheme of the invention is described in detail below with reference to the accompanying drawings and examples.
Example 1
As shown in fig. 2, embodiment 1 of the present invention proposes a space-borne heterogeneous h.264 video compression coding system, which is implemented based on a domestic CPU and GPU and is mounted on an in-orbit satellite, and includes a space-borne heterogeneous parallel forward coding subsystem and a parallel accelerated backward reconstruction subsystem; wherein,,
the space-borne heterogeneous parallel forward coding subsystem is used for combining the reconstructed images output by the parallel acceleration backward reconstruction subsystem, carrying out macro-block-level parallel intra-frame prediction and inter-frame prediction on the video image sequences acquired by the space-borne high-resolution imaging equipment in real time according to a new Z-shaped scanning sequence, and obtaining compressed code streams through macro-block-level parallel DCT conversion and macro-block-level parallel quantization and coding;
the parallel acceleration backward reconstruction subsystem is used for carrying out parallel inverse quantization on the coefficient data blocks after the parallel quantization on the macro block level, carrying out parallel inverse DCT on the macro block level to obtain residual data blocks, and carrying out parallel loop filtering on the macro block level to obtain a reconstructed image.
The space-borne heterogeneous parallel forward coding subsystem comprises a reading analysis module, a new Z-shaped scanning sequence intra-frame prediction module, a macro-block level parallel inter-frame prediction module, a macro-block level parallel DCT conversion module, a macro-block level parallel quantization module and an entropy coding module, and is used for carrying out on-track data coding on video image data acquired by space-borne high-resolution imaging equipment in real time to obtain a compressed code stream. In particular the number of the elements,
the reading analysis module is used for reading the video image sequence, analyzing all the macro block information which can be processed in parallel frame by frame, storing the macro block information into an intermediate variable, and sending the intermediate variable to the macro block level parallel intra-frame prediction module;
the macro-block-level parallel intra-frame prediction module is used for predicting the pixels of the current block by adopting a new Z-shaped scanning sequence through adjacent pixels decoded by the parallel acceleration backward reconstruction subsystem;
the macroblock-level parallel inter-frame prediction module is used for performing motion compensation according to the calculated motion vector by taking a block which is the most similar to the current block in the adjacent reference frame reconstructed by the parallel acceleration backward reconstruction subsystem as a prediction block to obtain a predicted image block;
the macroblock-level parallel DCT conversion module is used for redistributing the predicted residual data blocks in a conversion domain through macroblock-level parallel DCT conversion so as to reduce the correlation among pixels; the prediction residual data block is obtained by subtracting an original image block and a prediction image block which are stored in an intermediate variable;
the macroblock-level parallel quantization module is used for obtaining quantized coefficient data blocks through macroblock-level parallel quantization processing and realizing data compression;
the entropy coding module is used for entropy coding the quantized coefficient data blocks by combining intra-frame prediction information of the macroblock-level parallel intra-frame prediction module and motion information of the macroblock-level parallel inter-frame prediction module, reducing video coding information quantity by removing information entropy redundancy, and outputting a code stream to a network abstraction layer.
The parallel acceleration backward reconstruction subsystem comprises a GPU macro-block level parallel inverse quantization module, a macro-block level parallel inverse DCT module and a macro-block level parallel loop filtering module, wherein the GPU macro-block level parallel inverse quantization module, the macro-block level parallel inverse DCT module and the macro-block level parallel loop filtering module are used for carrying out inverse quantization and inverse DCT on quantized coefficient data blocks to obtain residual data blocks, and further obtaining a reconstructed image through the loop filtering module to provide a reference frame for an intra/inter prediction module. In particular the number of the elements,
the macroblock-level parallel inverse quantization module is used for carrying out parallel inverse quantization on the quantized coefficient data blocks through the macroblock level to obtain inverse quantized coefficient data blocks;
the macroblock-level parallel inverse DCT module is used for carrying out macroblock-level parallel inverse DCT on the coefficient data blocks after inverse quantization to obtain residual data blocks;
and the macro-block-level parallel loop filtering module is used for carrying out loop filtering processing on the reconstructed data block, removing the block effect and obtaining a reconstructed pixel block, wherein the reconstructed data block is obtained by adding a residual data block and a predicted image block.
FIG. 3 shows a conventional scanning sequence. Wherein,,
(a) Is a conventional intra luma 4 x 4 sub-block Zig-Zag scan order. The adoption of this scan sequence brings about a large amount of data correlation, which is extremely disadvantageous for parallel design. For example, when processing the 3 rd sub-block, the prediction and mode discrimination can be performed after waiting for the 2 nd sub-block to finish reconstruction, and when the 4 th sub-block can be processed, no correlation exists. If the scanning method of the (a) diagram is adopted, the intra-frame prediction path is extremely long, and a lot of unnecessary waiting time is wasted, 326 clock cycles are needed in the scanning method.
(b) (c), (d), (e) and (f) are currently available scan orders proposed by quantitatively analyzing the path costs brought by the scan orders. Wherein (f) the correlation is minimized by removing mode 3 and mode 7 in some 4 x 4 blocks, but this results in a loss of image quality. (b) The same performance can be achieved by adjusting the reference pixel pattern for which the block of weak correlation sequence numbers exists, as with the strong correlation numbers of (c), (d), and (e). (d) only 163 clock cycles are required to complete a 4 x 4 sub-block.
Fig. 4 is a new "zig-zag" scanning sequence in the on-track video compression of the present invention for the on-track heterogeneous parallel forward coding subsystem. The number of strong correlations is 2, and the number of wait times required to complete a 4 x 4 prediction is the same as (b), (c), (d), and (e) in fig. 3. The strong correlation means that the next block needs to be performed after the reconstruction of the previous block is completed, and the reason for the weak correlation is the correlation due to several modes of the current 4 x 4 sub-block reference pixels, and the weak correlation related to the present invention is generated by mode 3, mode 7 and mode 8. I.e. in performing mode 3, mode 7 and mode 8 predictions, the reference pixel point needs to use the reconstructed pixel values of the upper right or lower left 4 x 4 sub-block of the current 4 x 4 sub-block, which may be in the process. The weak correlation in these methods can be eliminated by adopting a mode order reordering method, such as putting mode 3, mode 7, and mode 8 of the block where the weak correlation exists at the end of the processing. For example, the 4 x 4 sub-blocks with weak correlation in fig. 4 are numbered 2, 3, 5, 8, 11, 14, and these sub-blocks still have 6 modes available for prediction and calculation before processing mode 3, mode 7, and mode 8, which can be eliminated as long as the upper right 4 x 4 reference pixel point of the current 4 x 4 sub-block is guaranteed to be reconstructed before the 6 modes are processed.
Table 1 correlation analysis of various scanning methods
| Scanning method | Number of strong correlations | Number of weak correlations |
| Method a | 8 | 4 |
| Method b | 2 | 4 |
| Method c | 2 | 3 |
| Method d | 2 | 7 |
| Method e | 2 | 5 |
| Method f | 2 | 0 |
| New method | 2 | 6 |
Fig. 5 is a flow chart of macroblock-level parallel DCT transform and macroblock-level parallel quantization module in the on-track video compression on-board heterogeneous parallel forward coding subsystem of the present invention. The conventional transform has K-L transform in addition to DCT transform, but its complexity is high so that it is not considered. H.264 coding is typically implemented by multiplying and shifting by combining quantization division and transform normalization. There are also designs that reduce the amount of computation in the process by butterfly operations. The invention not only combines quantization division and transformation normalization into a whole and realizes the division by multiplication and shift, but also realizes butterfly operation by a parallel mode, thereby greatly reducing the operation amount of the process. The multiplication is computationally intensive and therefore is accelerated in parallel with the GPU.
Fig. 6 is a flow chart of a macroblock-level parallel loop filter module in the parallel accelerated backward reconstruction subsystem for in-orbit video compression according to the present invention. And carrying out parallel optimization design according to the parameter characteristics of the determined filter strength BS, wherein the process takes 16 multiplied by 16 blocks as a calculation unit, the number of edges of one frame of image is 8, and the number of boundary points is 128. The parameter values required by the boundary point calculation are stored in the global memory, one frame of image can be directly submitted to 1 block, and 128 threads are used for parallel processing. The filtering speed is increased with little impact on image quality.
Example 2
As shown in fig. 7, embodiment 2 of the present invention proposes a method for compressing and encoding a satellite-borne heterogeneous h.264 video, including:
firstly, a video sequence to be coded is read on a Loongson CPU and enters a forward coding flow, and then all macro block information which can be processed in parallel in one frame of image is analyzed and stored in an intermediate variable. The data is then transferred to a Widget GPU for macroblock-level parallel intra/inter prediction, DCT transformation, and quantization. The quantized data are transmitted back to the Loongson CPU for entropy coding. The backward reconstruction loop comprises an inverse quantization, inverse DCT transformation and loop filtering module, and the reconstructed image after loop filtering can be used as a reference frame to continue to participate in the processing of the subsequent frames. The encoded data is output to the network abstraction layer and transmitted to the ground receiving station through the data transmission system when the satellite passes through the station. The on-orbit real-time video compression function is realized.
Finally, it should be noted that the above embodiments are only for illustrating the technical solution of the present invention and are not limiting. Although the present invention has been described in detail with reference to the embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made thereto without departing from the spirit and scope of the present invention, which is intended to be covered by the appended claims.
Claims (8)
1. The space-borne heterogeneous H.264 video compression coding system is characterized by being realized based on a domestic CPU and a GPU and comprising a space-borne heterogeneous parallel forward coding subsystem and a parallel acceleration backward reconstruction subsystem; wherein,,
the space-borne heterogeneous parallel forward coding subsystem is used for combining the reconstructed images output by the parallel acceleration backward reconstruction subsystem, carrying out macro-block-level parallel intra-frame prediction and inter-frame prediction on the video image sequences acquired by the space-borne high-resolution imaging equipment in real time according to a new Z-shaped scanning sequence, and obtaining compressed code streams through macro-block-level parallel DCT conversion and macro-block-level parallel quantization and coding;
the parallel acceleration backward reconstruction subsystem is used for carrying out parallel inverse quantization on the coefficient data blocks after the parallel quantization on the macro block level, carrying out parallel inverse DCT on the macro block level to obtain residual data blocks, and carrying out parallel loop filtering on the macro block level to obtain a reconstructed image.
2. The on-board heterogeneous h.264 video compression coding system of claim 1, wherein the on-board heterogeneous parallel forward coding subsystem comprises: the system comprises a reading analysis module and an entropy coding module which are deployed in a CPU, and a macro-block-level parallel intra-frame prediction module, a macro-block-level parallel inter-frame prediction module, a macro-block-level parallel DCT conversion module and a macro-block-level parallel quantization module which are deployed in a GPU; wherein,,
the reading analysis module is used for reading the video image sequence, analyzing all the macro block information which can be processed in parallel frame by frame, storing the macro block information into an intermediate variable, and sending the macro block information to the macro block level parallel intra-frame prediction module;
the macro-block-level parallel intra-frame prediction module is used for predicting the pixels of the current block by adopting a new Z-shaped scanning sequence through adjacent pixels decoded by the parallel acceleration backward reconstruction subsystem;
the macroblock-level parallel inter-frame prediction module is used for performing motion compensation according to the calculated motion vector by taking a block which is most similar to the current block in the adjacent reference frame reconstructed by the parallel acceleration backward reconstruction subsystem as a prediction block to obtain a predicted image block;
the macro-block-level parallel DCT conversion module is used for redistributing the predicted residual data blocks in a conversion domain through macro-block-level parallel DCT conversion so as to reduce the correlation among pixels; the prediction residual data block is obtained by subtracting an original image block and a prediction image block which are stored in an intermediate variable;
the macroblock-level parallel quantization module is used for obtaining quantized coefficient data blocks through macroblock-level parallel quantization processing and realizing data compression;
the entropy coding module is used for entropy coding the quantized coefficient data blocks by combining intra-frame prediction information of the macroblock-level parallel intra-frame prediction module and motion information of the macroblock-level parallel inter-frame prediction module, reducing video coding information quantity by removing information entropy redundancy, and outputting a code stream to a network abstraction layer.
3. The system of claim 2, wherein the macroblock-level parallel intra prediction module comprises luma 4 x 4 sub-blocks, sub-block numbers from 0 to 15;
the scanning sequence corresponding to sub-block numbers 0- >1- >2- >4- >3- >8- >6- >5- >9- >7- >10- >12- >11- >13- >14- >15 is adopted.
4. The system of claim 2, wherein the processing of the macroblock-level parallel DCT transform module and macroblock-level parallel quantization module comprises: the DCT conversion and quantization processes are combined into a whole, and on the basis of realizing the integer operation through multiplication and displacement, the GPU is utilized to process the multiplication operation in parallel, so that the real-time performance of encoding compression is improved; coarse quantization is performed on the high frequency portion and fine quantization is performed on the low frequency portion by adjusting the quantization step QP value to reduce visual redundancy and quantization error.
5. The system of claim 2, wherein the parallel accelerated backward reconstruction subsystem comprises a macroblock-level parallel inverse quantization module, a macroblock-level parallel inverse DCT module, and a macroblock-level parallel loop filter module deployed on a GPU, wherein,
the macroblock-level parallel inverse quantization module is used for carrying out parallel inverse quantization on the quantized coefficient data blocks through the macroblock level to obtain inverse quantized coefficient data blocks;
the macroblock-level parallel inverse DCT module is used for transforming the coefficient data blocks after inverse quantization into residual data blocks through macroblock-level parallel inverse DCT;
the macro-block-level parallel loop filtering module is used for carrying out loop filtering processing on the reconstructed data block, removing block effect and obtaining a reconstructed pixel block, wherein the reconstructed data block is obtained by adding a residual data block and a predicted image block.
6. The system of claim 5, wherein the macroblock-level parallel loop filter module performs parallel optimization design according to the parameter characteristics for determining the filtering strength BS, takes 16×16 blocks as a calculation unit, takes 8 edges of a frame of image as a calculation unit, takes 128 boundary points as a calculation unit, gives a frame of image to 1 block, and uses 128 threads for parallel processing.
7. The system of claim 1, wherein the native CPU is a loongson CPU and the native GPU is a wiry GPU.
8. A method for compression encoding of a space-borne heterogeneous h.264 video, implemented on the basis of the system of claim 5, the method comprising a forward encoding process and a backward reconstruction process:
the reading analysis module reads the video image sequence, analyzes all the macroblock information which can be processed in parallel frame by frame, stores the macroblock information into an intermediate variable, sends the intermediate variable to the macroblock-level parallel intra-frame prediction module, and enters a forward coding flow:
the macro-block-level parallel intra-frame prediction module adopts a new Z-shaped scanning sequence to predict the pixels of the current block through parallel acceleration backward reconstruction subsystem decoded adjacent pixels;
the macroblock-level parallel inter-frame prediction module performs motion compensation according to the calculated motion vector by taking a block which is most similar to the current block in an adjacent reference frame reconstructed by the parallel acceleration backward reconstruction subsystem as a prediction block, so as to obtain a predicted image block;
the macroblock-level parallel DCT conversion module redistributes the predicted residual data blocks in a conversion domain through macroblock-level parallel DCT conversion so as to reduce the correlation among pixels; the prediction residual data block is obtained by subtracting an original image block and a prediction image block which are stored in an intermediate variable;
the macroblock-level parallel quantization module obtains quantized coefficient data blocks through macroblock-level parallel quantization processing, and data compression is achieved;
the entropy coding module performs entropy coding on the quantized coefficient data blocks by combining intra-frame prediction information of the macroblock-level parallel intra-frame prediction module and motion information of the macroblock-level parallel inter-frame prediction module, reduces video coding information amount by removing information entropy redundancy, and outputs a code stream to a network abstraction layer;
the backward reconstruction process comprises the following steps:
the macroblock-level parallel inverse quantization module performs parallel inverse quantization on the quantized coefficient data blocks through the macroblock level to obtain inverse quantized coefficient data blocks;
the macroblock-level parallel inverse DCT module carries out macroblock-level parallel inverse DCT on the coefficient data blocks after inverse quantization to obtain residual data blocks;
and the macroblock-level parallel loop filtering module carries out loop filtering processing on the reconstructed data block, removes the block effect and obtains a reconstructed pixel block, and the reconstructed data block is obtained by adding a residual data block and a predicted image block.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310374018.7A CN116527895A (en) | 2023-04-10 | 2023-04-10 | Space-borne heterogeneous H.264 video compression coding system and coding method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310374018.7A CN116527895A (en) | 2023-04-10 | 2023-04-10 | Space-borne heterogeneous H.264 video compression coding system and coding method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116527895A true CN116527895A (en) | 2023-08-01 |
Family
ID=87395011
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310374018.7A Pending CN116527895A (en) | 2023-04-10 | 2023-04-10 | Space-borne heterogeneous H.264 video compression coding system and coding method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116527895A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117354529A (en) * | 2023-11-28 | 2024-01-05 | 广东匠芯创科技有限公司 | Image processing method based on video coding system, electronic equipment and medium |
-
2023
- 2023-04-10 CN CN202310374018.7A patent/CN116527895A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117354529A (en) * | 2023-11-28 | 2024-01-05 | 广东匠芯创科技有限公司 | Image processing method based on video coding system, electronic equipment and medium |
| CN117354529B (en) * | 2023-11-28 | 2024-03-12 | 广东匠芯创科技有限公司 | Image processing method based on video coding system, electronic equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US5299019A (en) | Image signal band compressing system for digital video tape recorder | |
| CN102150429B (en) | System and method for video encoding using constructed reference frame | |
| CN100579224C (en) | Method and apparatus for lossless video encoding and decoding | |
| CN102065298B (en) | High-performance macroblock coding implementation method | |
| CN102740077B (en) | H.264/AVC standard-based intra-frame prediction mode selection method | |
| CN101394560B (en) | Mixed production line apparatus used for video encoding | |
| KR20070026317A (en) | Bi-directional predicting method for video coding/decoding | |
| CN101621696B (en) | Allow selectivity using method and the encoder of scores videos estimation and two-way video estimation | |
| CN101273638A (en) | Image processing device, image processing method, and program | |
| CN103327325A (en) | Intra-frame prediction mode rapid self-adaptation selection method based on HEVC standard | |
| CN103347185B (en) | The comprehensive compaction coding method of unmanned plane reconnaissance image based on the conversion of selectivity block | |
| KR20090096143A (en) | Encoding system using motion estimation and encoding method using motion estimation | |
| CN101742288B (en) | Video noise reduction encoding method and video noise reduction encoding device | |
| CN101742290B (en) | Methods and devices for denoising in video coding and decoding, | |
| CN116527895A (en) | Space-borne heterogeneous H.264 video compression coding system and coding method | |
| CN104754337B (en) | Method for video coding | |
| CN102291582A (en) | Distributed video encoding method based on motion compensation refinement | |
| CN102196272B (en) | P frame encoding method and device | |
| CN102769753B (en) | H264 encoder and coding method | |
| CN104113761A (en) | Code rate control method for video encoding and encoder | |
| US20100040143A1 (en) | Video coding apparatus and method | |
| CN106060555A (en) | Encoder based on multi-core processor | |
| CN101715124A (en) | Single-input and multi-output video encoding system and video encoding method | |
| CN102595137B (en) | Fast mode judging device and method based on image pixel block row/column pipelining | |
| CN110351552A (en) | A kind of fast encoding method in Video coding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |