WO2019126929A1 - 编码器、图像处理系统、无人机及编码方法 - Google Patents

编码器、图像处理系统、无人机及编码方法 Download PDF

Info

Publication number
WO2019126929A1
WO2019126929A1 PCT/CN2017/118265 CN2017118265W WO2019126929A1 WO 2019126929 A1 WO2019126929 A1 WO 2019126929A1 CN 2017118265 W CN2017118265 W CN 2017118265W WO 2019126929 A1 WO2019126929 A1 WO 2019126929A1
Authority
WO
WIPO (PCT)
Prior art keywords
sub
image block
image
block
intra prediction
Prior art date
Application number
PCT/CN2017/118265
Other languages
English (en)
French (fr)
Inventor
王亚强
秦东
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2017/118265 priority Critical patent/WO2019126929A1/zh
Priority to CN201780017210.3A priority patent/CN108781298B/zh
Publication of WO2019126929A1 publication Critical patent/WO2019126929A1/zh
Priority to US16/910,827 priority patent/US20200320274A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C39/00—Aircraft not otherwise provided for
    • B64C39/02—Aircraft not otherwise provided for characterised by special use
    • B64C39/024—Aircraft not otherwise provided for characterised by special use of the remote controlled vehicle type, i.e. RPV
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2101/00—UAVs specially adapted for particular uses or applications
    • B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography

Definitions

  • the present application relates to the field of image coding and decoding, and more particularly to an encoder, an image processing system, a drone, and an encoding method.
  • Image coding and decoding techniques can be used to achieve image compression.
  • Common image compression methods include intraframe compression and interframe compression.
  • the process of intraframe compression generally includes steps of intra prediction, residual coding, and reconstruction.
  • intra prediction is to select the optimal intra prediction direction from a plurality of preset intra prediction directions.
  • Intra prediction of a certain sub-image block (or block) usually needs to be performed based on adjacent pixels of the sub-image block.
  • One type of prior art performs intra prediction based on reconstructed pixels of adjacent pixels of a sub-image block.
  • Such intra prediction mode has high prediction accuracy, but introduces data dependency between sub-image blocks. Due to the data dependency between the sub-image blocks, the prior art generally performs serial processing on each sub-image block based on the numbering order of the sub-image blocks, resulting in an in-frame compression process that cannot be pipelined.
  • the present application provides an encoder, an image processing system, a drone, and an encoding method to improve the efficiency of intraframe compression.
  • an encoder comprising: a first processing module configured to perform intra prediction on the first sub-image block according to reconstructed pixels of adjacent pixels of the first sub-image block to determine An optimal intra prediction direction of the first sub-image block, wherein the first sub-image block is a sub-image block of the image block to be encoded in the first division mode; and the second processing module is configured to Generating, according to the optimal intra prediction direction of the first sub-image block, the quantized data of the first sub-image, and reconstructing the first sub-image block according to the quantized data of the first sub-image;
  • a processing module is further configured to: in response to receiving the first indication signal indicating completion of the reconstruction of the first sub-image block, to the second sub-pixel according to reconstructed pixels of adjacent pixels of the second sub-image block The image block performs intra prediction to determine an optimal intra prediction direction of the second sub image block; and in response to receiving the first indication signal, according to the reconstructed pixel of the adjacent pixel of the third sub image block,
  • an image processing system comprising the encoder of the first aspect.
  • a drone comprising the image processing system of the second aspect.
  • a fourth aspect provides an encoding method, including: performing intra prediction on the first sub-image block according to a reconstructed pixel of an adjacent pixel of a first sub-image block to determine the first sub-image block An optimal intra prediction direction, wherein the first sub-image block is one sub-image block of the image block to be encoded in the first division mode; and the generating is performed according to an optimal intra prediction direction of the first sub-image block Quantizing data of the first sub-image, and reconstructing the first sub-image block according to the quantized data of the first sub-image; in response to receiving a first indication indicating completion of reconstruction of the first sub-image block Signaling, performing intra prediction on the second sub-image block according to reconstructed pixels of neighboring pixels of the second sub-image block to determine an optimal intra prediction direction of the second sub-image block; in response to receiving Up to the first indication signal, performing intra prediction on the third sub image block according to the reconstructed pixels of the adjacent pixels of the third sub image block to determine an optimal intra frame of the third sub image block Prediction
  • the present application does not serially process each sub-image block in the order of sub-picture block numbers as in the conventional technology, but changes the processing order of the sub-image block to: after the first sub-image block is reconstructed, it will be located at the same time.
  • the sub-image block on the right side of the first sub-image block, and the sub-image block located at the lower left of the first sub-image block are determined as sub-image blocks to be processed.
  • the present application introduces a first processing module to perform intra prediction on a sub-image block, and introduces a second processing module to quantize and reconstruct the sub-image block.
  • the first processing module is equivalent to the prediction pipeline level of the sub-image block, and is used for intra prediction of the sub-image block
  • the second processing module is equivalent to the reconstruction pipeline level of the sub-image block, and is used for quantizing and weighting the sub-image block.
  • the two pipeline stages exchange information with each other through the indication signals.
  • the introduction of the prediction pipeline level and the reconstruction pipeline level realizes the pipeline design of the intraframe compression process and improves the efficiency of intraframe compression.
  • FIG. 1 is a schematic flow chart of an intraframe compression mode.
  • FIG. 2 is an exemplary diagram of an image block to be encoded.
  • FIG. 3 is a schematic structural diagram of an encoder provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of an adjusted coding sequence provided by an embodiment of the present application.
  • FIG. 5 is a diagram showing an example of an operation mode of a pipeline designed in an embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of an encoder according to another embodiment of the present application.
  • FIG. 7 is a schematic diagram of a manner of setting a suspension threshold provided by an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of an image processing system according to an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a drone provided by an embodiment of the present application.
  • FIG. 10 is a schematic flowchart of an encoding method provided by an embodiment of the present application.
  • the embodiments of the present application can be applied to various image or video codec technologies, such as H.264, H.265, and the like.
  • the embodiments of the present application can be applied to various application fields that require image coding and decoding and/or video transmission, such as the field of image transmission of a drone, the field of mobile photography, and the field of high-definition surveillance.
  • the intraframe compression process generally includes: intra prediction (intra prediction can be further subdivided into intra-prediction direction search and matching cost calculation sub-processes), residual calculation, transform (such as discrete cosine transform). , quantization, entropy coding and other processes. Further, the intra-frame compression process may further include: performing inverse quantization, inverse transform, reconstruction, and the like on the quantization result of the block to determine a reconstructed pixel of the block.
  • Intra prediction needs to be performed based on the neighboring pixels of the block.
  • the reconstructed pixels of adjacent pixels of a certain block may be used as a reference for intra prediction of the block, so that good image compression quality can be obtained, but a block between the blocks is introduced.
  • Data dependency results in low intraframe compression of the image.
  • the data dependency mentioned here is particularly evident in the partition mode of block 4 ⁇ 4. 2, the block 4 ⁇ 4 partition mode will be described in detail as an example.
  • Figure 2 shows a 16 x 16 macroblock.
  • This macroblock is decomposed into 16 blocks (sub-image blocks) according to the division mode of block 4 ⁇ 4.
  • the next block is The operation needs to wait for the current block operation to end before it can be started. Taking block 6 in FIG.
  • the protocol stipulates that the intra prediction of block 6 needs to depend on the adjacent pixels of the left, upper left, upper, and upper right directions of blcok 6 as a prediction reference, and therefore, it is necessary to wait for block 1
  • the intraframe compression process of block 6 can be started. Therefore, the 16 4 ⁇ 4 sub-image blocks in Figure 2 are usually serially processed in the order of numbering.
  • the hardware resources in the encoder cannot be pipelined, and the limited hardware resources cannot be fully utilized. High performance processing.
  • the original pixels of neighboring pixels of a certain block may be used as a reference for intra prediction of the block. Since the original pixels of each block naturally exist, the start of intraframe compression of one block has nothing to do with whether or not the adjacent block is reconstructed. Therefore, this implementation can release the data dependency between the blocks, so that the frame can be targeted.
  • the internal compression process is pipelined and can even be processed in parallel. But the original pixel of the block is an approximation of the reconstructed pixel, and the difference between the two will reduce the compression quality of the image.
  • the low-end products usually use the original pixels of the adjacent pixels of the block as the intra prediction reference of the block, which is easy to achieve high hardware performance, but at the expense of image compression quality.
  • High-end products usually use reconstructed pixels of adjacent pixels of the block as intra prediction bases, so that better image compression quality can be obtained, but the efficiency of intraframe compression is reduced.
  • the embodiment of the present application provides an encoder that can improve intraframe compression efficiency while maintaining image compression quality.
  • the encoder provided in the embodiment of the present application will be described in detail below by taking FIG. 3 as an example.
  • the encoder 300 may be, for example, an on-chip encoding system that performs an image encoding function.
  • the encoder 300 can include a first processing module 310 and a second processing module 320.
  • the first processing module 310 and the second processing module 320 may be, for example, two mutually independent hardware modules in the encoder 300, and may communicate with each other (eg, using a semaphore mechanism for communication).
  • the processing object of the processing module (such as the first processing module, the second processing module, etc., or the third processing module, the fourth processing module, etc. mentioned later) is explicitly defined as a luminance block or a chromaticity.
  • the type of the sub-image block processed by the processing module is not limited, and may be a luma block or a chroma block.
  • first processing module 310 and the second processing module 320 are described in detail below by taking the processing procedure of the first sub-image block as an example. It should be understood that the first sub-image block may be one sub-image block of the image block to be encoded in the first division mode.
  • the size of the coded image block and the first division mode are not specifically limited in this embodiment of the present application, which is related to the coding standard on which the encoder 300 is based.
  • the image block to be encoded may be a 16 ⁇ 16 image block (or macroblock), and the first partition mode may be a 4 ⁇ 4 partition mode.
  • the first sub-image block may be any one of the sub-image blocks having the following numbers in the 16 ⁇ 16 image block: 1, 3, 4, 6, 9, 12 (image block and The correspondence of numbers can be seen in Figure 2).
  • the image block to be coded may be a 64 ⁇ 64 image block
  • the first division mode may be a 16 ⁇ 16, 8 ⁇ 8 or 4 ⁇ 4 division mode.
  • the first partition mode as a 16 ⁇ 16 partitioning mode
  • the first sub-image block may be any one of the sub-image blocks corresponding to the following numbers in the 64 ⁇ 64 image block: 1, 3, 4, 6,9,12.
  • the first processing module 310 can be configured to perform intra prediction on the first sub-image block based on the reconstructed pixels of the neighboring pixels of the first sub-image block to determine an optimal intra prediction direction of the first sub-image block.
  • the first processing module 310 can perform the operation corresponding to the intra prediction shown in FIG. 1. For example, the matching cost of the first sub-image block in each intra prediction direction may be calculated, and then the intra prediction direction with the smallest matching cost is selected as the optimal intra prediction direction of the first sub-image block.
  • the second processing module 320 may be configured to generate the quantized data of the first sub-image according to the optimal intra prediction direction of the first sub-image block, and reconstruct the first sub-image block according to the quantized data of the first sub-image.
  • the second processing module 320 can perform operations such as residual calculation, transform, quantization, inverse quantization, inverse transform, reconstruction, and the like as shown in FIG. 1.
  • the second processing module 320 may acquire information indicating the optimal intra prediction direction of the first sub-module from the first processing module 310, and based on the optimal intra-frame of the first sub-module The prediction direction sequentially performs the following operations: performing residual calculation on the first sub-module based on the optimal intra prediction direction of the first sub-module to obtain residual data; transforming the residual data to obtain transformed data; The data is quantized to obtain the quantized data of the first sub-module; the quantized data of the first sub-module is inverse quantized to obtain the inverse quantized data of the first sub-module; the inverse quantized data of the first sub-module is inversely transformed and heavily Constructed to obtain reconstructed pixels of the first sub-module.
  • the second processing module 320 may acquire information indicating the optimal intra prediction direction of the first sub-module from
  • the first processing module 310 may be further configured to: in response to receiving the first indication signal indicating completion of the reconstruction of the first sub-image block, the second sub-image according to the reconstructed pixels of the adjacent pixels of the second sub-image block The block performs intra prediction to determine an optimal intra prediction direction of the second sub image block; in response to receiving the first indication signal, the third sub image according to the reconstructed pixels of the adjacent pixels of the third sub image block The block performs intra prediction to determine an optimal intra prediction direction of the third sub-image block.
  • the second sub-image block and the third sub-image block are both sub-image blocks of the image block to be encoded in the first division mode, and the second sub-image block is the right sub-image block of the first sub-image block, and the third sub-image The block is the lower left sub-image block of the first sub-image block.
  • the first sub-image block is the sub-image block 1
  • the second sub-image block and the third sub-image block may be the sub-image block 2 and the sub-image block 4.
  • the first sub-image block is the sub-image block 12
  • the second sub-image block and the third sub-image block may be the sub-image block 11 and the sub-image block 13.
  • the intra prediction of the second sub-image block and the third sub-image block may be performed in parallel or in sequence, which is not limited in this embodiment of the present application. Processing the second sub-image block and the third sub-image block in parallel can improve intraframe compression efficiency.
  • the specific processing manner of the first sub-image block and the third sub-image block of the first processing module 310 is similar to the processing manner of the first sub-image block by the first processing module 310.
  • the specific processing manner of the first sub-image block and the third sub-image block of the first processing module 310 is similar to the processing manner of the first sub-image block by the first processing module 310.
  • the second processing module 320 is further configured to: generate, according to the optimal intra prediction direction of the second sub image block, in response to receiving the second indication signal indicating completion of intra prediction of the second sub image block The quantized data of the two sub-images is reconstructed from the quantized data of the second sub-image. And generating, according to the third indication signal indicating completion of the intra prediction of the third sub image block, the quantization data of the third sub image according to the optimal intra prediction direction of the third sub image block, and according to the third sub image The quantized data reconstructs the third sub-image block.
  • the specific processing manner of the second sub-image block and the third sub-image block of the second processing module 320 is similar to the processing manner of the first sub-image block by the second processing module 320. For details, refer to the description above. Detailed.
  • the data is dependent on the data dependency between the sub-image blocks, and the conventional technique generally adopts a serial processing method.
  • Each sub-image block is processed, that is, after the reconstruction of one sub-image block, only the intra prediction and reconstruction process of the next image block of the sub-image block is turned on.
  • the first sub-image block (such as the sub-image block 1, 3, 4, 6, 9, or 12 in FIG. 2) has a right sub-image block and a lower left sub-image block
  • the reconstructed pixels of the adjacent sub-image block of the first sub-image block and the adjacent pixels of the lower left sub-image block have all been obtained.
  • the present application does not serially process each sub-image block in the order of sub-picture block numbers as in the conventional technology, but changes the processing order of the sub-image block to: after the first sub-image block is reconstructed, simultaneously A sub-image block located to the right of the first sub-image block and a sub-image block located at the lower left of the first sub-image block are determined as sub-image blocks to be processed.
  • the present application introduces a first processing module to perform intra prediction on a sub-image block, and introduces a second processing module to quantize and reconstruct the sub-image block.
  • the first processing module is equivalent to the prediction pipeline level of the sub-image block, and is used for intra prediction of the sub-image block
  • the second processing module is equivalent to the reconstruction pipeline level of the sub-image block, and is used for quantizing and weighting the sub-image block.
  • the two pipeline stages exchange information with each other through the indication signals.
  • the introduction of the prediction pipeline level and the reconstruction pipeline level realizes the pipeline design of the intraframe compression process and improves the efficiency of intraframe compression.
  • the image block to be encoded is a 16 ⁇ 16 image block
  • the first partitioning mode is a block 4 ⁇ 4 partitioning manner
  • the processing process of the predicted pipeline level and the reconstructed pipeline level is detailed. for example.
  • FIG. 4 shows block 0-15.
  • the conventional technique serially processes block 0-15 in accordance with the block numbering order. By carefully observing the data dependency between blocks 0-15, it can be found that after the reconstructed pixels of block 1 are determined, the adjacent sub-image block block 2 and the lower left sub-image block block 4 of block 1 are required for intra prediction. The reconstructed pixels have been obtained; after the reconstructed pixels of block 4 are determined, the reconstructed pixels of the adjacent pixels required for intra prediction of the right sub-image block block 5 and the lower left sub-image block block 3 of block 4 have been obtained.
  • the reconstructed pixels of block 6 After the reconstructed pixel of block 6 is determined, the reconstructed pixels of the adjacent pixels required for intra prediction of the right sub-image block block 7 and the lower left sub-image block block 9 of block 6 have been obtained; the reconstructed pixels of block 9 After the determination, the reconstructed pixels of the adjacent pixels required for the intra prediction of the right sub-image block block 12 and the lower left sub-image block block 10 of the block 9 have been obtained; after the reconstructed pixels of the block 12 are determined, the right of the block 12 The reconstructed pixels of the adjacent pixels required for the intra prediction by the sub-picture block block 13 and the lower left sub-picture block block 11 have been obtained.
  • the embodiment of the present application adjusts the operation order of the 16 blocks in (a) of FIG. 4 to the operation order shown in (b) of FIG. 4, that is, block 0 ⁇ block 1 ⁇ block 2,4 ⁇ block3. , 5 ⁇ block 8, 6 ⁇ block 9, 7 ⁇ block 10, 12 ⁇ block 11, 13 ⁇ block 14 ⁇ block 15.
  • This embodiment of the present application refers to this operation sequence as a wavefront-based operation sequence.
  • the embodiment of the present application performs a pipeline design on the operation process of the 16 blocks, and decomposes the operation process of the 16 blocks into the predicted pipeline level that is the responsibility of the first processing module 310, and the reconstruction pipeline that is responsible for the second processing module 320. level.
  • the prediction pipeline level may calculate an optimal intra prediction direction of each block based on reconstructed pixels of adjacent pixels of each block; the reconstruction pipeline level may perform residual calculation based on an optimal intra prediction direction of each block, Transformation, quantization, inverse quantization, inverse transformation, reconstruction and other operations.
  • each block can be processed in sequence using a certain mechanism, such as a state machine mechanism.
  • a certain mechanism such as a state machine mechanism.
  • the predicted pipeline level also needs to rely on the information of the reconstructed pipeline level output. Therefore, the two pipeline stages can communicate through a certain communication mechanism and interact with each other. Information needed.
  • the two flow levels can be interactively controlled by semaphores.
  • the reconstructed flow level depends on the semaphore output of the predicted flow level.
  • the predicted flow level depends on the semaphore output of the reconstructed flow level, and the control circuit of the two flow stages.
  • the respective startup semaphores are read in a predetermined order and filled in with their respective output semaphores.
  • FIG. 5 is a schematic diagram of a processing procedure of a pipeline according to an embodiment of the present application.
  • FIG. 5 includes a predicted flow level 52 and a reconstructed flow level 54.
  • the operation of predicting the pipeline stage 52 includes: P0-P15, where Pi represents prediction of the block numbered i.
  • the operation of reconstructing the pipeline stage 54 includes: R0-R15, Rj represents quantization and reconstruction of the block numbered j.
  • P4, P5, P8, P6, P7, P10, P12, and P13 all achieve sufficient water flow, compressing the bubbles in the pipeline, and achieving The purpose of parallel processing.
  • the embodiment of the present application pipelines the processing of the block image of the block 4 ⁇ 4 based on the operation order of the wavefront, realizes the parallel processing of the block 4 ⁇ 4, and improves the calculation efficiency of the encoder.
  • the image block to be encoded generally needs to adopt multiple partition modes for intra prediction, and selects the optimal partition mode (such as the partition mode with the smallest matching cost) as the final partition mode of the image block to be encoded.
  • the image block to be encoded is a 16 ⁇ 16 image block, and the 16 ⁇ 16 image block needs to perform intra prediction in three partition modes of block 4 ⁇ 4, block 8 ⁇ 8, and block 16 ⁇ 16, and encoding.
  • the device needs to select the optimal division mode from the above three division modes as the final division mode of the image block to be encoded.
  • some related technologies pre-configure the abort threshold. If the accumulated value of the partitioning cost of the block in a certain partitioning mode is greater than the abort threshold, it is considered that the partitioning mode is not the optimal partitioning mode. In this case, the encoder stops subsequent operations in the partition mode, thereby reducing the power consumption of the encoder.
  • the suspension thresholds of such related technologies are mostly empirical values or statistical values, and cannot be adaptively adjusted according to different coding scenarios.
  • the embodiment of the present application further provides an encoder 300, which can adaptively adjust the suspension threshold according to different coding scenarios.
  • the structure and function of the encoder 300 will be described in detail below.
  • the encoder 300 may include a first processing module 310, a second processing module 320, a third processing module 330, and a fourth processing module 340.
  • the first processing module 310 can be configured to perform intra prediction on each sub-image block of the coded image block in the first split mode.
  • the second processing module 320 can be configured to quantize and reconstruct each sub-image block of the coded image block in the first partition mode.
  • the third processing module 330 can be configured to perform intra prediction on each sub-image block of the coded image block in the second split mode.
  • the fourth processing module 340 can be configured to quantize and reconstruct each sub-image block of the coded image block in the second partition mode.
  • the first processing module 310 and the second processing module 320 are corresponding to the hardware processing channel corresponding to the first partition mode, and are used for processing each sub-image block of the coded image block in the first split mode; the third processing module 330 and the The four processing module 340 is equivalent to a hardware processing channel corresponding to the second division mode, and is configured to process each sub-image block of the to-be-coded image block in the second division mode. Therefore, the embodiment of the present application is equivalent to introducing different hardware processing channels to respectively process the sub-image blocks corresponding to the first dividing mode and the second dividing mode.
  • the size of the sub-image block obtained by dividing the second division mode in the embodiment of the present application is larger than the size of the sub-image block obtained by the first division mode division.
  • the image block to be coded is a 16 ⁇ 16 image block
  • the first partition mode may be a block 4 ⁇ 4 partition mode
  • the second partition mode may be a block 8 ⁇ 8 partition mode, or may be block16 ⁇ 16 division mode.
  • the image block to be encoded is a 64 ⁇ 64 image block
  • the first partition mode may be a block 4 ⁇ 4 partition mode
  • the second partition mode may be a block 64 ⁇ 64 partition mode; or, the first partition The mode may be a partition mode of block 16 ⁇ 16, and the second split mode may be a partition mode of block 32 ⁇ 32 or block 64 ⁇ 64.
  • the first processing module 310 and/or the second processing module 320 may also be configured as a current partitioning cost corresponding to the first partitioning mode of the image block to be encoded (can be understood as a currently calculated sub-routine in the first partitioning mode)
  • the sum of the matching costs of the image blocks is greater than or equal to the total partitioning cost of the image block to be encoded corresponding to the second partitioning mode (which can be understood as the sum of the matching costs of the sub-image blocks in the second partitioning mode), and the encoding is stopped. Processing of image blocks.
  • the size of the sub-image block obtained by the second division mode division is larger than the size of the sub-image block obtained by the first division mode division, and the sub-image obtained by the second division mode division is compared with the sub-image block obtained by the first division mode division.
  • the number of blocks is small, and therefore, the operation process of the sub-image block in the second division mode ends early.
  • the embodiment of the present application controls the processing process of the sub-image block in the first division mode by using the total division cost corresponding to the second division mode as the suspension threshold.
  • the coding scheme or the image to be coded is different, and the total division cost corresponding to the first division mode is different. Therefore, the embodiment of the present application can adaptively change the suspension threshold according to the coding scenario or the image to be encoded, so that the configuration of the suspension threshold is further improved. accurate.
  • the image block to be encoded is a 16 ⁇ 16 image block, and the luminance component of the image block to be encoded needs to be divided into three types: block 4 ⁇ 4, block 8 ⁇ 8, and block 16 ⁇ 16.
  • the sub-image block obtained by dividing the luminance component of the image block to be encoded in the block 4 ⁇ 4 partition mode is referred to as a 4 ⁇ 4 luma block; the luminance component of the image block to be encoded is in the block 8 ⁇ 8 partition mode.
  • the divided sub-image block is referred to as an 8 ⁇ 8 luma block; the sub-image block obtained by dividing the luma component of the image block to be encoded in the block 16 ⁇ 16 partition mode is referred to as a 16 ⁇ 16 luma block; the image block to be encoded is to be encoded.
  • the sub-image block corresponding to the chroma component is called a chroma block.
  • the embodiment of the present application configures a corresponding hardware processing module (or hardware processing path) for a 4 ⁇ 4 luma block, an 8 ⁇ 8 luma block, and a 16 ⁇ 16 luma block, wherein the first partition described above
  • the mode may correspond to block 4 ⁇ 4 in the present embodiment
  • the second division mode described above may correspond to block 16 ⁇ 16 in the present embodiment.
  • the calculation process of the 16 ⁇ 16 luma block can be quickly ended.
  • the total partition cost of the 16 ⁇ 16 luma block can be obtained (in this example, since the image block to be encoded contains only one 16 ⁇ 16 luma block, The total partitioning cost of the 16 ⁇ 16 luma block is equal to the matching cost of the 16 ⁇ 16 luma block itself as the abort threshold, and the operation process of controlling other hardware processing paths is controlled.
  • the calculation of the processing path of the 8 ⁇ 8 luma block can be stopped to save power; similarly, when 4 ⁇ When the current cumulative cost of the luminance block of 4 exceeds the total division cost of the 16 ⁇ 16 luminance block, the calculation of the processing path of the 4 ⁇ 4 luminance block is stopped to save power.
  • the third processing module 330 can also be used for intra prediction of chroma components of the image block to be encoded.
  • the fourth processing module 340 can also be used to quantize and reconstruct the chroma components of the block of code to be encoded.
  • the number of sub-image blocks that the third processing module 330 needs to process is small, and the calculation pressure is low.
  • the number of chroma blocks to be processed is generally smaller than that of the luminance block to be processed.
  • the chroma block calculation is usually performed only for block 8 ⁇ 8. Therefore, the embodiment of the present application multiplexes hardware resources to implement operations of the luma block and the luma block in the second partition mode.
  • the hardware processing resources corresponding to the 16 ⁇ 16 luma block can continue to process the chroma block, which can improve the multiplexing rate of the hardware and reduce the chip. volume.
  • the architecture of the conventional intraframe compression has been improved from various angles.
  • the processing effect of 4K@30fps can be easily realized in the TSMChpc 28nm process and the 260MHz frequency. Therefore, the embodiment of the present application can perform intra-frame compression of high quality and high hardware performance on the image with low power consumption without increasing the hardware frequency and increasing the hardware area.
  • An embodiment of the present application also provides an image processing system.
  • the image processing system 800 can include the encoder 300 described in any of the above embodiments.
  • the embodiment of the present application also provides a drone.
  • the drone 900 can include an image processing system 800 as shown in FIG.
  • the device embodiment of the present application is described in detail above with reference to FIG. 1 to FIG. 9, and the method embodiment of the present application is described in detail below with reference to FIG. It should be understood that the description of the method embodiments corresponds to the description of the device embodiments, and therefore, portions that are not described in detail may be referred to the previous device embodiments.
  • FIG. 10 is a schematic flowchart of an encoding method according to an embodiment of the present application.
  • the encoding method of FIG. 10 includes steps S1010-S1060.
  • the first sub-image block is intra-predicted according to the reconstructed pixels of the adjacent pixels of the first sub-image block to determine an optimal intra prediction direction of the first sub-image block.
  • the first sub-image block is a sub-image block of the image block to be encoded in the first division mode.
  • the quantized data of the first sub-image is generated according to the optimal intra prediction direction of the first sub-image block, and the first sub-image block is reconstructed according to the quantized data of the first sub-image.
  • the second sub-image block and the third sub-image block are both sub-image blocks of the image block to be encoded in the first division mode, and the second sub-image block is the right sub-image block of the first sub-image block, and the third sub-image The block is the lower left sub-image block of the first sub-image block.
  • the intra prediction processes of the second sub-image block and the third sub-image block are performed in parallel.
  • the image block to be encoded is a 16 ⁇ 16 image block
  • the first division mode is a 4 ⁇ 4 division mode
  • the first sub image block is a sub image block corresponding to the following number in the 16 ⁇ 16 image block. Any image block: 1, 3, 4, 6, 9, 12.
  • the encoding method may further include: performing intra prediction on each sub-image block in the second split mode of the image block to be encoded; and performing quantization and reconstruction on each sub-image block in the second split mode of the image block to be encoded.
  • the size of the sub-image block obtained by dividing the second division mode is larger than the size of the sub-image block obtained by the first division mode division; and the current division cost corresponding to the first division mode of the image block to be encoded is greater than or equal to the image to be encoded
  • the total division cost of the block corresponding to the second division mode stops the processing of the image block to be coded.
  • the image block to be encoded is a 16 ⁇ 16 image block
  • the first division mode is a 4 ⁇ 4 division mode
  • the second division mode is a 16 ⁇ 16 division mode.
  • the computer program product includes one or more computer instructions.
  • the computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable device.
  • the computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be from a website site, computer, server or data center Transmission to another website site, computer, server or data center via wired (eg coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg infrared, wireless, microwave, etc.).
  • the computer readable storage medium can be any available media that can be accessed by a computer or a data storage device such as a server, data center, or the like that includes one or more available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a digital video disc (DVD)), or a semiconductor medium (such as a solid state disk (SSD)).
  • a magnetic medium for example, a floppy disk, a hard disk, a magnetic tape
  • an optical medium such as a digital video disc (DVD)
  • a semiconductor medium such as a solid state disk (SSD)
  • the disclosed systems, devices, and methods may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Aviation & Aerospace Engineering (AREA)

Abstract

提供一种编码器、图像处理系统、无人机及编码方法。该编码器包括：第一处理模块，用于为根据第一子图像块的重构相邻像素对第一子图像块进行帧内预测；第二处理模块，用于对第一子图像进行量化和重构；第一处理模块还响应于第一子图像块重构完成，对第二子图像块和第三子图像进行帧内预测，第二子图像块和第三子图像分别为第一子图像块的右子图像块和左下子图像块；第二处理模块还响应于第二子图像块帧内预测完成，对第二子图像进行量化和重构；响应于第三子图像块帧内预测完成，对第三子图像块进行量化和重构。该技术方案对子图像块的处理过程进行了流水线设计，提高了帧内压缩的效率。

Description

编码器、图像处理系统、无人机及编码方法
版权申明
本专利文件披露的内容包含受版权保护的材料。该版权为版权所有人所有。版权所有人不反对任何人复制专利与商标局的官方记录和档案中所存在的该专利文件或者该专利披露。
技术领域
本申请涉及图像编解码领域，并且更为具体地，涉及一种编码器、图像处理系统、无人机及编码方法。
背景技术
图像编解码技术可用于实现图像的压缩。常见的图像压缩方式包括帧内压缩和帧间压缩。帧内压缩的过程大致包括帧内预测，残差编码以及重构等步骤。
帧内预测的目的是从预设的多种帧内预测方向中选取最优帧内预测方向。某个子图像块(或称block)的帧内预测通常需要以该子图像块的相邻像素为基准进行。
一类现有技术以子图像块的相邻像素的重构像素为基准进行帧内预测，这类帧内预测方式的预测精度高，但会引入子图像块之间的数据依赖性。由于子图像块之间存在数据依赖性，因此，现有技术通常会基于子图像块的编号顺序对各子图像块进行串行处理，导致帧内压缩过程无法进行流水线设计。
发明内容
本申请提供一种编码器、图像处理系统、无人机及编码方法，以提高帧内压缩的效率。
第一方面，提供一种编码器，包括：第一处理模块，被配置为根据第一子图像块的相邻像素的重构像素，对所述第一子图像块进行帧内预测，以确定所述第一子图像块的最优帧内预测方向，其中所述第一子图像块为待编码图像块在第一划分模式下的一个子图像块；第二处理模块，被配置为根据所 述第一子图像块的最优帧内预测方向生成所述第一子图像的量化数据，并根据所述第一子图像的量化数据对所述第一子图像块进行重构；所述第一处理模块还被配置为：响应于接收到指示所述第一子图像块重构完成的第一指示信号，根据第二子图像块的相邻像素的重构像素，对所述第二子图像块进行帧内预测，以确定所述第二子图像块的最优帧内预测方向；响应于接收到所述第一指示信号，根据第三子图像块的相邻像素的重构像素，对所述第三子图像块进行帧内预测，以确定所述第三子图像块的最优帧内预测方向，其中所述第二子图像块和所述第三子图像块均为所述待编码图像块在所述第一划分模式下的子图像块，且所述第二子图像块为所述第一子图像块的右子图像块，所述第三子图像块为所述第一子图像块的左下子图像块；所述第二处理模块还被配置为：响应于接收到指示所述第二子图像块的帧内预测完成的第二指示信号，根据所述第二子图像块的最优帧内预测方向生成所述第二子图像的量化数据，并根据所述第二子图像的量化数据对所述第二子图像块进行重构；响应于接收到指示所述第三子图像块的帧内预测完成的第三指示信号，根据所述第三子图像块的最优帧内预测方向生成所述第三子图像的量化数据，并根据所述第三子图像的量化数据对所述第三子图像块进行重构。
第二方面，提供一种图像处理系统，包括如第一方面所述的编码器。
第三方面，提供一种无人机，包括如第二方面所述的图像处理系统。
第四方面，提供一种编码方法，包括：根据第一子图像块的相邻像素的重构像素，对所述第一子图像块进行帧内预测，以确定所述第一子图像块的最优帧内预测方向，其中所述第一子图像块为待编码图像块在第一划分模式下的一个子图像块；根据所述第一子图像块的最优帧内预测方向生成所述第一子图像的量化数据，并根据所述第一子图像的量化数据对所述第一子图像块进行重构；响应于接收到指示所述第一子图像块重构完成的第一指示信号，根据第二子图像块的相邻像素的重构像素，对所述第二子图像块进行帧内预测，以确定所述第二子图像块的最优帧内预测方向；响应于接收到所述第一指示信号，根据第三子图像块的相邻像素的重构像素，对所述第三子图像块进行帧内预测，以确定所述第三子图像块的最优帧内预测方向，其中所述第二子图像块和所述第三子图像块均为所述待编码图像块在所述第一划分模式下的子图像块，且所述第二子图像块为所述第一子图像块的右子图像块，所述第三子图像块为所述第一子图像块的左下子图像块；响应于接收到 指示所述第二子图像块的帧内预测完成的第二指示信号，根据所述第二子图像块的最优帧内预测方向生成所述第二子图像的量化数据，并根据所述第二子图像的量化数据对所述第二子图像块进行重构；响应于接收到指示所述第三子图像块的帧内预测完成的第三指示信号，根据所述第三子图像块的最优帧内预测方向生成所述第三子图像的量化数据，并根据所述第三子图像的量化数据对所述第三子图像块进行重构。
本申请更并非像传统技术那样按照子图像块编号顺序对各子图像块进行串行处理，而是将子图像块的处理顺序更改为：在第一子图像块重构完毕之后，同时将位于该第一子图像块右侧的子图像块，以及位于该第一子图像块左下的子图像块确定为待处理的子图像块。此外，本申请引入第一处理模块对子图像块进行帧内预测，并引入第二处理模块对子图像块进行量化和重构。第一处理模块相当于子图像块的预测流水级，用于对子图像块进行帧内预测，第二处理模块相当于子图像块的重构流水级，用于对子图像块进行量化和重构，两个流水级通过指示信号交互彼此需要的信息。预测流水级和重构流水级的引入实现了帧内压缩过程的流水线设计，提高了帧内压缩的效率。
附图说明
图1是帧内压缩方式的示意性流程图。
图2是待编码图像块的示例图。
图3是本申请一个实施例提供的编码器的示意性结构图。
图4是本申请实施例提供的调整后的编码顺序的示意图。
图5是本申请实施例设计的流水线的运行方式的示例图。
图6是本申请另一实施例提供的编码器的示意性结构图。
图7是本申请实施例提供的中止阈值的设置方式的示意图。
图8是本申请实施例提供的图像处理系统的示意性结构图。
图9是本申请实施例提供的无人机的示意性结构图。
图10是本申请实施例提供的编码方法的示意性流程图。
具体实施方式
本申请实施例可应用于各种图像或视频编解码技术，如H.264，H.265 等。此外，本申请实施例可应用于需要进行图像编解码和/或视频传输的各种应用领域，如无人机的图传领域，移动摄影领域，高清监控领域等。
为了便于理解，下面结合图1，以H.264为例，对传统的帧内压缩过程进行简单介绍。
H.264以16x16的宏块为单位对图像进行帧内压缩，宏块内部还会分别按照如下三种划分模式进行更为精细的帧内预测和/或压缩处理：block4x4、block8x8和block16x16。如图1所示，帧内压缩过程大致包括：帧内预测(帧内预测可以进一步细分为帧内预测方向搜索和匹配代价计算等子过程)，残差计算，变换(如离散余弦变换)，量化，熵编码等过程。进一步地，帧内压缩过程还可以包括：对block的量化结果进行反量化、反变换、重构等逆过程，以确定block的重构像素。
帧内预测需要以block的相邻像素为基准进行。可选地，在一些实现方式中，可以使用某个block的相邻像素的重构像素作为该block的帧内预测的基准，这样可以得到很好的图像压缩质量，但会引入block之间的数据依赖性，导致图像的帧内压缩效率低。这里提到的数据依赖性在block4×4这种划分模式下体现的尤为明显。下面结合图2，以block4×4的划分模式为例进行详细说明。
图2示出的是16×16的宏块。该宏块按照block4×4的划分模式分解成16个block(子图像块)。在以block的相邻像素的重构像素作为帧内预测的基准的实现方式中，由于每个block的下一block的帧内预测需要以当前block的重构像素为基准，因此，下一block的运算需要等当前block的运算结束之后才能开启。以图2中的block 6为例进行说明，协议规定，block 6的帧内预测需要依赖blcok 6的左侧、左上、上方、右上等方向的相邻像素作为预测基准，因此，需要等block 1、block 3、block 4、block 5的重构过程均完成之后，才能启动block 6的帧内压缩过程。所以，图2中的16个4×4的子图像块通常会按照编号顺序进行串行处理，编码器中的硬件资源无法进行流水线(pipeline)设计，也就无法充分利用有限的硬件资源来达到高性能处理。
可选地，在另一些实现方式中，可以使用某个block的相邻像素的原始像素作为该block的帧内预测的基准。由于每个block的原始像素天然存在，因此，一个block的帧内压缩的启动与相邻block是否重构完成无关，所以， 这种实现方式可以解除block之间的数据依赖性，从而可以针对帧内压缩过程进行流水线设计，甚至可以并行处理。但block的原始像素是重构像素的近似，二者之间的差异会降低图像的压缩质量。
目前，中低端产品通常采用block的相邻像素的原始像素作为该block的帧内预测基准，这样容易达到很高的硬件性能，却牺牲了图像压缩质量。高端产品通常采用block的相邻像素的重构像素作为帧内预测基准，这样可以获得更好的图像压缩质量，但帧内压缩的效率降低。
本申请实施例提供一种编码器，可以在保持图像压缩质量的前提下，提高帧内压缩效率。下面以图3为例对本申请实施例提供的编码器进行详细说明。
如图3所示，编码器300例如可以是执行图像编码功能的片上编码系统。编码器300可以包括第一处理模块310和第二处理模块320。第一处理模块310和第二处理模块320例如可以是编码器300中的两个相互独立的硬件模块，且二者之间可以相互通信(如利用信号量机制进行通信)。
本申请中，除明确限定处理模块(如上文的第一处理模块、第二处理模块等，或后文提及的第三处理模块、第四处理模块等)的处理对象为亮度块或色度块之外，本申请对处理模块处理的子图像块的类型并不限定，可以是亮度块，也可以是色度块。
下面以第一子图像块的处理过程为例，对第一处理模块310和第二处理模块320的功能进行详细描述。应理解，第一子图像块可以为待编码图像块在第一划分模式下的一个子图像块。本申请实施例对待编码图像块的尺寸以及第一划分模式不做具体限定，这与编码器300所基于的编码标准有关。
以H.264为例，待编码图像块可以为16×16的图像块(或称宏块)，第一划分模式可以为4×4的划分模式。在这种情况下，第一子图像块可以为16×16的图像块中的具有如下编号的子图像块中的任一图像块：1，3，4，6，9，12(图像块与编号的对应关系可以参见图2)。
以H.265为例，待编码图像块可以为64×64的图像块，第一划分模式可以为16×16、8×8或4×4的划分模式。以第一划分模式为16×16的划分模式为例，第一子图像块可以为64×64的图像块中的如下编号对应的子图像块中的任一图像块：1，3，4，6，9，12。
第一处理模块310可被配置为根据第一子图像块的相邻像素的重构像 素，对第一子图像块进行帧内预测，以确定第一子图像块的最优帧内预测方向。
以图1为例，第一处理模块310可以执行图1所示的帧内预测对应的操作。例如，可以计算第一子图像块在各帧内预测方向下的匹配代价，然后从中选取匹配代价最小的帧内预测方向作为第一子图像块的最优帧内预测方向。
第二处理模块320可被配置为根据第一子图像块的最优帧内预测方向生成第一子图像的量化数据，并根据第一子图像的量化数据对第一子图像块进行重构。
以图1为例，第二处理模块320可以执行如图1所示的残差计算、变换、量化、反量化、反变换、重构等操作。以第一子图像的处理过程为例，第二处理模块320可以从第一处理模块310获取指示第一子模块的最优帧内预测方向的信息，并基于第一子模块的最优帧内预测方向依次执行以下操作：基于第一子模块的最优帧内预测方向对第一子模块进行残差计算，得到残差数据；对残差数据进行变换，得到变换后的数据；对变换后的数据进行量化，得到第一子模块的量化数据；对第一子模块的量化数据进行反量化，得到第一子模块的反量化数据；对第一子模块的反量化数据进行反变换和重构，得到第一子模块的重构像素。可选地，在一些实施例中，第二处理模块320还可执行图1所示的熵编码操作。
第一处理模块310还可以被配置为：响应于接收到指示第一子图像块重构完成的第一指示信号，根据第二子图像块的相邻像素的重构像素，对第二子图像块进行帧内预测，以确定第二子图像块的最优帧内预测方向；响应于接收到第一指示信号，根据第三子图像块的相邻像素的重构像素，对第三子图像块进行帧内预测，以确定第三子图像块的最优帧内预测方向。
第二子图像块和第三子图像块均为待编码图像块在第一划分模式下的子图像块，且第二子图像块为第一子图像块的右子图像块，第三子图像块为第一子图像块的左下子图像块。以H.264为例，如图2所示，假设第一子图像块为子图像块1，则第二子图像块和第三子图像块可以是子图像块2和子图像块4。又如，假设第一子图像块为子图像块12，则第二子图像块和第三子图像块可以是子图像块11和子图像块13。第二子图像块和第三子图像块的帧内预测可以并行进行，也可以先后处理，本申请实施例对此并不限定。 并行处理第二子图像块和第三子图像块可以提高帧内压缩效率。
第一处理模块310对第二子图像块和第三子图像块的具体处理方式与第一处理模块310对第一子图像块的处理方式类似，具体可以参见上文的描述，此处不再详述。
进一步地，第二处理模块320还可被配置为：响应于接收到指示第二子图像块的帧内预测完成的第二指示信号，根据第二子图像块的最优帧内预测方向生成第二子图像的量化数据，并根据第二子图像的量化数据对第二子图像块进行重构。响应于接收到指示第三子图像块的帧内预测完成的第三指示信号，根据第三子图像块的最优帧内预测方向生成第三子图像的量化数据，并根据第三子图像的量化数据对第三子图像块进行重构。
第二处理模块320对第二子图像块和第三子图像块的具体处理方式与第二处理模块320对第一子图像块的处理方式类似，具体可以参见上文的描述，此处不再详述。
如前文所述，在以子图像块的相邻像素的重构像素为基准的帧内压缩实现方式中，受到子图像块之间的数据依赖性的约束，传统技术通常采用串行的处理方式对各子图像块进行处理，即当一个子图像块重构完毕之后，仅会开启该子图像块的下一图像块的帧内预测和重构过程。
实际上，在待编码图像块中，如果第一子图像块(如图2中的子图像块1，3，4，6，9或12)存在右子图像块和左下子图像块，则第一子图像块被重构之后，该第一子图像块的右子图像块和左下子图像块的相邻像素的重构像素就已经全部获得。因此，本申请更并非像传统技术那样按照子图像块编号顺序对各子图像块进行串行处理，而是将子图像块的处理顺序更改为：在第一子图像块重构完毕之后，同时将位于该第一子图像块右侧的子图像块，以及位于该第一子图像块左下的子图像块确定为待处理的子图像块。此外，本申请引入第一处理模块对子图像块进行帧内预测，并引入第二处理模块对子图像块进行量化和重构。第一处理模块相当于子图像块的预测流水级，用于对子图像块进行帧内预测，第二处理模块相当于子图像块的重构流水级，用于对子图像块进行量化和重构，两个流水级通过指示信号交互彼此需要的信息。预测流水级和重构流水级的引入实现了帧内压缩过程的流水线设计，提高了帧内压缩的效率。
下面结合图4-图6，以待编码图像块为16×16的图像块，第一划分方 式为block4×4的划分方式为例，对预测流水级和重构流水级的处理过程进行详细地举例说明。
首先，图4中的(a)示出了block 0-15。传统技术依照block的编号顺序对block 0-15串行处理。仔细观察block 0-15之间的数据依赖关系可以发现，block 1的重构像素确定之后，block 1的右子图像块block 2和左下子图像块block 4进行帧内预测所需的相邻像素的重构像素均已获得；block 4的重构像素确定之后，block 4的右子图像块block 5和左下子图像块block 3进行帧内预测所需的相邻像素的重构像素均已获得；block 6的重构像素确定之后，block 6的右子图像块block 7和左下子图像块block 9进行帧内预测所需的相邻像素的重构像素均已获得；block 9的重构像素确定之后，block 9的右子图像块block 12和左下子图像块block 10进行帧内预测所需的相邻像素的重构像素均已获得；block 12的重构像素确定之后，block 12的右子图像块block 13和左下子图像块block 11进行帧内预测所需的相邻像素的重构像素均已获得。
基于以上分析，本申请实施例将图4的(a)中的16个block的运算顺序调整为图4的(b)所示的运算顺序，即block 0→block 1→block 2,4→block3,5→block 8,6→block 9,7→block 10,12→block 11,13→block 14→block 15。本申请实施例将这种运算顺序称为基于wavefront的运算顺序。
接着，本申请实施例对16个block的运算过程进行流水线设计，将16个block的运算过程分解成由第一处理模块310负责的预测流水级，以及由第二处理模块320负责的重构流水级。预测流水级可基于每个block的相邻像素的重构像素，计算每个block的最优帧内预测方向；重构流水级可基于每个block的最优帧内预测方向进行残差计算、变换、量化、反量化、反变换、重构等运算。
两个流水级内部可以采用一定的机制按序处理各block，如状态机机制。进一步地，由于重构流水级需要依赖预测流水级输出的信息，预测流水级也需要依赖重构流水级输出的信息，因此，两个流水级之间可以通过一定的通信机制进行通信，交互彼此需要的信息。例如，两个流水级之间可以通过信号量进行交互控制，重构流水级依赖预测流水级输出的信号量，预测流水级则依赖重构流水级输出的信号量，两个流水级的控制电路依赖预定的顺序去读取各自的启动信号量，并填写各自的输出信号量。
图5为根据本申请实施例的流水线的处理过程的示意图。图5包括预测流水级52和重构流水级54。预测流水级52的运算过程包含：P0-P15，其中Pi表示对编号为i的block进行预测。重构流水级54的运算过程包含：R0-R15，Rj表示对编号为j的block进行量化和重构。从图5可以看出，在不增加硬件资源的情况下，P4、P5、P8、P6、P7、P10、P12、P13处都实现了充分流水，压缩了流水线中的气泡(bubble)，达到了并行处理的目的。综上所述，本申请实施例基于wavefront的运算顺序对block4×4的子图像块的处理进行了流水线设计，实现了block4×4的并行处理，提高了编码器的计算效率。
前文已经指出，待编码图像块一般需要采用多种划分模式进行帧内预测，并从中选取出最优的划分模式(如匹配代价最小的划分模式)作为待编码图像块的最终划分模式。以H.264为例，待编码图像块为16×16的图像块，该16×16的图像块需要在block4×4、block8×8以及block16×16三种划分模式下进行帧内预测，编码器需要从上述三种划分模式中选取出最优的划分模式作为待编码图像块的最终划分模式。
为了降低功耗，有些相关技术会预先配置中止阈值，如果某种划分模式下的block的划分代价的累加值大于该中止阈值，则认为此种划分模式不会是最优划分模式。在这种情况下，编码器会停止该划分模式下的后续运算，从而降低编码器的功耗。但是，此类相关技术的中止阈值大多为经验值或统计值，无法根据编码场景的不同而自适应调整。
如图6所示，本申请实施例还提供一种编码器300，能够根据编码场景的不同自适应调整中止阈值。下面对编码器300的结构和功能进行详细介绍。
编码器300可以包括第一处理模块310、第二处理模块320、第三处理模块330和第四处理模块340。
第一处理模块310可被配置为对待编码图像块在第一划分模式下的各子图像块进行帧内预测。第二处理模块320可被配置为对待编码图像块在第一划分模式下的各子图像块进行量化和重构。第一处理模块310和第二处理模块320的详细描述可以参见前文，此处不再详述。
第三处理模块330可被配置为对待编码图像块在第二划分模式下的各子图像块进行帧内预测。第四处理模块340可被配置为对待编码图像块在第二划分模式下的各子图像块进行量化和重构。
第一处理模块310和第二处理模块320相当于第一划分模式对应的硬件处理通道，用于对待编码图像块在第一划分模式下的各子图像块进行处理；第三处理模块330和第四处理模块340相当于第二划分模式对应的硬件处理通道，用于对待编码图像块在第二划分模式下的各子图像块进行处理。因此，本申请实施例相当于引入不同的硬件处理通道分别处理第一划分模式和第二划分模式对应的子图像块。
本申请实施例中的第二划分模式划分得到的子图像块的尺寸大于第一划分模式划分得到的子图像块的尺寸。
以H.264为例，待编码图像块为16×16的图像块，第一划分模式可以是block4×4的划分模式，第二划分模式可以是block8×8的划分模式，也可以是block16×16的划分模式。
以H.265为例，待编码图像块为64×64的图像块，第一划分模式可以是block4×4的划分模式，第二划分模式可以是block64×64的划分模式；或者，第一划分模式可以是block16×16的划分模式，第二划分模式可以是block32×32或block64×64的划分模式。
第一处理模块310和/或第二处理模块320还可被配置为当待编码图像块的与第一划分模式对应的当前划分代价(可以理解为第一划分模式下的当前已计算完成的子图像块的匹配代价之和)大于或等于待编码图像块的与第二划分模式对应的总划分代价(可以理解为第二划分模式下的各子图像块的匹配代价之和)，停止对待编码图像块的处理。
由于第二划分模式划分得到的子图像块的尺寸大于第一划分模式划分得到的子图像块的尺寸，与第一划分模式划分得到的子图像块相比，第二划分模式划分得到的子图像块的数量较少，因此，第二划分模式下的子图像块的运算过程会提前结束。在第二划分模式下的子图像块的运算过程结束之后，本申请实施例将第二划分模式对应的总划分代价作为中止阈值，控制第一划分模式下的子图像块的处理过程。编码场景或待编码的图像不同，第一划分模式对应的总划分代价相应不同，因此，本申请实施例能够根据编码场景或待编码图像的不同，自适应更改中止阈值，使得中止阈值的配置更加准确。
下面结合图7，以待编码图像块为16×16的图像块，该待编码图像块的亮度分量需要进行block4×4，block8×8以及block16×16三种形式的划分为 例进行说明。为了便于区分，下文将待编码图像块的亮度分量在block4×4划分模式下划分得到的子图像块称为4×4的亮度块；将待编码图像块的亮度分量在block8×8划分模式下划分得到的子图像块称为8×8的亮度块；待编码图像块的亮度分量在block16×16划分模式下划分得到的子图像块称为16×16的亮度块；将待编码图像块的色度分量对应的子图像块称为色度块。
首先，本申请实施例为4×4的亮度块、8×8的亮度块以及16×16的亮度块配置各自对应的硬件处理模块(或称硬件处理通路)，其中上文描述的第一划分模式可对应于本实施例中的block4×4，上文描述的第二划分模式可对应于本实施例中的block16×16。
由于待编码图像块仅包含一个16×16的亮度块，因此，该16×16的亮度块的计算过程很快就可以结束。参见图7，当16×16的亮度块计算结束之后，可以将该16×16的亮度块的总划分代价(在本例中，由于待编码图像块仅包含一个16×16的亮度块，因此，该16×16的亮度块的总划分代价等于该16×16的亮度块自身的匹配代价)作为中止阈值，控制其他硬件处理通路的运算过程。例如，当8×8的亮度块的当前累计代价超过16×16的亮度块的总划分代价时，可以停止8×8的亮度块的处理通路的计算，以节省功率；同理，当4×4的亮度块的当前累计代价超过16×16的亮度块的总划分代价时，停止4×4的亮度块的处理通路的计算，以节省功率。
进一步地，在一些实施例中，第三处理模块330还可用于对待编码图像块的色度分量进行帧内预测。同理，第四处理模块340还可用于对待编码图像块的色度分量进行量化和重构。
相比第一处理模块310，第三处理模块330需要处理的子图像块的数量较少，计算压力较低。此外，在图像编码标准中，与待处理的亮度块相比，待处理的色度块的数量一般较少，以H.264为例，通常只会针对block8×8进行色度块计算。因此，本申请实施例复用硬件资源实现色度块和第二划分模式下的亮度块的运算。以图6为例，在对16×16的亮度块处理完毕之后，16×16的亮度块对应的硬件处理资源可以继续对色度块进行处理，这样可以提高硬件的复用率，降低芯片的体积。
上文从多个角度对传统帧内压缩的架构进行了改进，利用该改进后的帧内压缩架构，可以在TSMChpc28nm工艺、260MHz频率下轻松实现4K@30fps的处理效果。因此，本申请实施例可以在不提高硬件频率、不增 大硬件面积的前提下，以较低的功耗对图像进行高质量、高硬件性能的帧内压缩。
本申请实施例还提供一种图像处理系统。如图8所示，该图像处理系统800可以包含上文任意实施例描述的编码器300。
本申请实施例还提供一种无人机。如图9所示，该无人机900可以包括如图8所示的图像处理系统800。
上文结合图1至图9，详细描述了本申请的装置实施例，下面结合图10，详细描述本申请的方法实施例。应理解，方法实施例的描述与装置实施例的描述相互对应，因此，未详细描述的部分可以参见前面装置实施例。
图10是根据本申请实施例的编码方法的示意性流程图。图10的编码方法包括步骤S1010-S1060。
在S1010中，根据第一子图像块的相邻像素的重构像素，对第一子图像块进行帧内预测，以确定第一子图像块的最优帧内预测方向。第一子图像块为待编码图像块在第一划分模式下的一个子图像块。
在S1020中，根据第一子图像块的最优帧内预测方向生成第一子图像的量化数据，并根据第一子图像的量化数据对第一子图像块进行重构。
在S1030中，响应于接收到指示第一子图像块重构完成的第一指示信号，根据第二子图像块的相邻像素的重构像素，对第二子图像块进行帧内预测，以确定第二子图像块的最优帧内预测方向。
在S1040中，响应于接收到第一指示信号，根据第三子图像块的相邻像素的重构像素，对第三子图像块进行帧内预测，以确定第三子图像块的最优帧内预测方向。
第二子图像块和第三子图像块均为待编码图像块在第一划分模式下的子图像块，且第二子图像块为第一子图像块的右子图像块，第三子图像块为第一子图像块的左下子图像块。
在S1050中，响应于接收到指示第二子图像块的帧内预测完成的第二指示信号，根据第二子图像块的最优帧内预测方向生成第二子图像的量化数据，并根据第二子图像的量化数据对第二子图像块进行重构。
在S1060中，响应于接收到指示第三子图像块的帧内预测完成的第三指示信号，根据第三子图像块的最优帧内预测方向生成第三子图像的量化数据，并根据第三子图像的量化数据对第三子图像块进行重构。
可选地，第二子图像块和第三子图像块的帧内预测过程是并行进行的。
可选地，待编码图像块为16×16的图像块，第一划分模式为4×4的划分模式，第一子图像块为16×16的图像块中的如下编号对应的子图像块中的任一图像块：1，3，4，6，9，12。
可选地，编码方法还可包括：对待编码图像块在第二划分模式下的各子图像块进行帧内预测；对待编码图像块在第二划分模式下的各子图像块进行量化和重构，其中第二划分模式划分得到的子图像块的尺寸大于第一划分模式划分得到的子图像块的尺寸；当待编码图像块的与第一划分模式对应的当前划分代价大于或等于待编码图像块的与第二划分模式对应的总划分代价，停止对待编码图像块的处理。
可选地，待编码图像块为16×16的图像块，第一划分模式为4×4的划分模式，第二划分模式为16×16的划分模式。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其他任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如数字视频光盘(digital video disc，DVD))、或者半导体介质(例如固态硬盘(solid state disk，SSD))等。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方 法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (13)

  1. 一种编码器，其特征在于，包括：
    第一处理模块，被配置为根据第一子图像块的相邻像素的重构像素，对所述第一子图像块进行帧内预测，以确定所述第一子图像块的最优帧内预测方向，其中所述第一子图像块为待编码图像块在第一划分模式下的一个子图像块；
    第二处理模块，被配置为根据所述第一子图像块的最优帧内预测方向生成所述第一子图像的量化数据，并根据所述第一子图像的量化数据对所述第一子图像块进行重构；
    所述第一处理模块还被配置为：响应于接收到指示所述第一子图像块重构完成的第一指示信号，根据第二子图像块的相邻像素的重构像素，对所述第二子图像块进行帧内预测，以确定所述第二子图像块的最优帧内预测方向；响应于接收到所述第一指示信号，根据第三子图像块的相邻像素的重构像素，对所述第三子图像块进行帧内预测，以确定所述第三子图像块的最优帧内预测方向，其中所述第二子图像块和所述第三子图像块均为所述待编码图像块在所述第一划分模式下的子图像块，且所述第二子图像块为所述第一子图像块的右子图像块，所述第三子图像块为所述第一子图像块的左下子图像块；
    所述第二处理模块还被配置为：响应于接收到指示所述第二子图像块的帧内预测完成的第二指示信号，根据所述第二子图像块的最优帧内预测方向生成所述第二子图像的量化数据，并根据所述第二子图像的量化数据对所述第二子图像块进行重构；响应于接收到指示所述第三子图像块的帧内预测完成的第三指示信号，根据所述第三子图像块的最优帧内预测方向生成所述第三子图像的量化数据，并根据所述第三子图像的量化数据对所述第三子图像块进行重构。
  2. 如权利要求1所述的编码器，其特征在于，所述第一处理模块被配置为：响应于接收到所述第一指示信号，并行地对所述第二子图像块和所述第三子图像块进行帧内预测。
  3. 如权利要求1或2所述的编码器，其特征在于，所述待编码图像块为16×16的图像块，所述第一划分模式为4×4的划分模式，所述第一子图像块为所述16×16的图像块中的如下编号对应的子图像块中的任一图像块： 1，3，4，6，9，12。
  4. 如权利要求1-3中任一项所述的编码器，其特征在于，所述第一处理模块被配置为对所述待编码图像块在所述第一划分模式下的各子图像块进行帧内预测，所述第二处理模块被配置为对所述待编码图像块在所述第一划分模式下的各子图像块进行量化和重构；
    所述编码器还包括：
    第三处理模块，被配置为对所述待编码图像块在第二划分模式下的各子图像块进行帧内预测；
    第四处理模块，被配置为对所述待编码图像块在第二划分模式下的各子图像块进行量化和重构，其中所述第二划分模式划分得到的子图像块的尺寸大于所述第一划分模式划分得到的子图像块的尺寸；
    所述第一处理模块和/或所述第二处理模块还被配置为当所述待编码图像块的与所述第一划分模式对应的当前划分代价大于或等于所述待编码图像块的与所述第二划分模式对应的总划分代价，停止对所述待编码图像块的处理。
  5. 如权利要求4所述的编码器，其特征在于，所述待编码图像块为16×16的图像块，所述第一划分模式为4×4的划分模式，所述第二划分模式为16×16的划分模式。
  6. 如权利要求5所述的编码器，其特征在于，所述第一处理模块和所述第三处理模块均用于对所述待编码图像块的亮度分量进行帧内预测，所述第三处理模块还用于对所述待编码图像块的色度分量进行帧内预测。
  7. 一种图像处理系统，其特征在于，包括如权利要求1-6中任一项所述的编码器。
  8. 一种无人机，其特征在于，包括如权利要求7所述的图像处理系统。
  9. 一种编码方法，其特征在于，包括：
    根据第一子图像块的相邻像素的重构像素，对所述第一子图像块进行帧内预测，以确定所述第一子图像块的最优帧内预测方向，其中所述第一子图像块为待编码图像块在第一划分模式下的一个子图像块；
    根据所述第一子图像块的最优帧内预测方向生成所述第一子图像的量化数据，并根据所述第一子图像的量化数据对所述第一子图像块进行重构；
    响应于接收到指示所述第一子图像块重构完成的第一指示信号，根据第 二子图像块的相邻像素的重构像素，对所述第二子图像块进行帧内预测，以确定所述第二子图像块的最优帧内预测方向；
    响应于接收到所述第一指示信号，根据第三子图像块的相邻像素的重构像素，对所述第三子图像块进行帧内预测，以确定所述第三子图像块的最优帧内预测方向，其中所述第二子图像块和所述第三子图像块均为所述待编码图像块在所述第一划分模式下的子图像块，且所述第二子图像块为所述第一子图像块的右子图像块，所述第三子图像块为所述第一子图像块的左下子图像块；
    响应于接收到指示所述第二子图像块的帧内预测完成的第二指示信号，根据所述第二子图像块的最优帧内预测方向生成所述第二子图像的量化数据，并根据所述第二子图像的量化数据对所述第二子图像块进行重构；
    响应于接收到指示所述第三子图像块的帧内预测完成的第三指示信号，根据所述第三子图像块的最优帧内预测方向生成所述第三子图像的量化数据，并根据所述第三子图像的量化数据对所述第三子图像块进行重构。
  10. 如权利要求9所述的编码方法，其特征在于，所述第二子图像块和所述第三子图像块的帧内预测过程是并行进行的。
  11. 如权利要求9或10所述的编码方法，其特征在于，所述待编码图像块为16×16的图像块，所述第一划分模式为4×4的划分模式，所述第一子图像块为所述16×16的图像块中的如下编号对应的子图像块中的任一图像块：1，3，4，6，9，12。
  12. 如权利要求9-11中任一项所述的编码方法，其特征在于，所述编码方法还包括：
    对所述待编码图像块在第二划分模式下的各子图像块进行帧内预测；
    对所述待编码图像块在第二划分模式下的各子图像块进行量化和重构，其中所述第二划分模式划分得到的子图像块的尺寸大于所述第一划分模式划分得到的子图像块的尺寸；
    当所述待编码图像块的与所述第一划分模式对应的当前划分代价大于或等于所述待编码图像块的与所述第二划分模式对应的总划分代价，停止对所述待编码图像块的处理。
  13. 如权利要求12所述的编码方法，其特征在于，所述待编码图像块为16×16的图像块，所述第一划分模式为4×4的划分模式，所述第二划分 模式为16×16的划分模式。
PCT/CN2017/118265 2017-12-25 2017-12-25 编码器、图像处理系统、无人机及编码方法 WO2019126929A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/CN2017/118265 WO2019126929A1 (zh) 2017-12-25 2017-12-25 编码器、图像处理系统、无人机及编码方法
CN201780017210.3A CN108781298B (zh) 2017-12-25 2017-12-25 编码器、图像处理系统、无人机及编码方法
US16/910,827 US20200320274A1 (en) 2017-12-25 2020-06-24 Encoder, image processing system, unmanned aerial vehicle and encoding method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/118265 WO2019126929A1 (zh) 2017-12-25 2017-12-25 编码器、图像处理系统、无人机及编码方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/910,827 Continuation US20200320274A1 (en) 2017-12-25 2020-06-24 Encoder, image processing system, unmanned aerial vehicle and encoding method

Publications (1)

Publication Number Publication Date
WO2019126929A1 true WO2019126929A1 (zh) 2019-07-04

Family

ID=64033813

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/118265 WO2019126929A1 (zh) 2017-12-25 2017-12-25 编码器、图像处理系统、无人机及编码方法

Country Status (3)

Country Link
US (1) US20200320274A1 (zh)
CN (1) CN108781298B (zh)
WO (1) WO2019126929A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111263156B (zh) * 2019-02-20 2022-03-25 北京达佳互联信息技术有限公司 视频解码方法、视频编码方法及装置
CN112204974A (zh) * 2019-10-31 2021-01-08 深圳市大疆创新科技有限公司 图像预测和视频编码方法、装置、可移动平台及存储介质
CN112954438B (zh) * 2021-02-05 2022-02-11 摩尔线程智能科技(北京)有限责任公司 一种图像处理方法以及装置
CN115481702B (zh) * 2022-10-28 2023-02-17 中国人民解放军国防科技大学 面向多元时序数据处理的预见式对照表征方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102318346A (zh) * 2009-02-20 2012-01-11 索尼公司 图像处理设备和方法
US20120014441A1 (en) * 2010-07-15 2012-01-19 Sharp Laboratories Of America, Inc. Parallel video coding based on boundaries
US20130343457A1 (en) * 2010-12-23 2013-12-26 British Broadcasting Corporation Compression of pictures
CN106688233A (zh) * 2016-12-08 2017-05-17 深圳市大疆创新科技有限公司 用于传输图像的方法、设备和无人机

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102318346A (zh) * 2009-02-20 2012-01-11 索尼公司 图像处理设备和方法
US20120014441A1 (en) * 2010-07-15 2012-01-19 Sharp Laboratories Of America, Inc. Parallel video coding based on boundaries
US20130343457A1 (en) * 2010-12-23 2013-12-26 British Broadcasting Corporation Compression of pictures
CN106688233A (zh) * 2016-12-08 2017-05-17 深圳市大疆创新科技有限公司 用于传输图像的方法、设备和无人机

Also Published As

Publication number Publication date
CN108781298B (zh) 2021-05-25
US20200320274A1 (en) 2020-10-08
CN108781298A (zh) 2018-11-09

Similar Documents

Publication Publication Date Title
CN107566848B (zh) 编解码的方法及装置
CN106170092B (zh) 用于无损编码的快速编码方法
US11863746B2 (en) Method and apparatus for processing image signal
US11589051B2 (en) Method and apparatus for processing image signal
WO2015096822A1 (zh) 图像编码、解码方法及装置
WO2019126929A1 (zh) 编码器、图像处理系统、无人机及编码方法
KR20210134402A (ko) 인코더, 디코더 및 대응하는 인트라 예측을 위한 방법
US10931945B2 (en) Method and device for processing prediction information for encoding or decoding an image
KR20210125088A (ko) 매트릭스 기반 인트라 예측 및 2차 변환 코어 선택을 조화시키는 인코더, 디코더 및 대응 방법
US20190387227A1 (en) Method and apparatus for selecting a coding mode used for encoding/decoding a residual block
EP4075800A1 (en) Encoding method and encoder
US20240214562A1 (en) Video coding with dynamic groups of pictures
AU2020376669B2 (en) Image encoding/decoding method and apparatus using adaptive transform, and method for transmitting bitstream
WO2020002117A2 (en) Methods and devices for performing sample adaptive offset (sao) filtering
KR20220065880A (ko) 아핀 모션 보상에서 dct 기반 보간 필터 및 향상된 쌍선형 보간 필터의 사용
EP4387224A1 (en) Picture partitioning method and apparatus
US20220337835A1 (en) Image encoding/decoding method and device using adaptive color transform, and method for transmitting bitstream
US10764577B2 (en) Non-MPM mode coding for intra prediction in video coding
CN115550666A (zh) 用于视频数据的编码方法、解码方法、计算设备和介质
US10412383B2 (en) Compressing groups of video frames using reversed ordering
KR20210113399A (ko) 인트라 예측을 위해 인트라 모드 코딩을 사용하는 인코더, 디코더 및 대응 방법
JP7492027B2 (ja) Ｐｔｌ関連情報をシグナリングする画像符号化／復号方法および装置、ならびにビットストリームを記憶したコンピュータ読み取り可能記録媒体
US20230379479A1 (en) Systems and methods for determining chroma samples in an intra prediction mode of video coding
CN118283283A (zh) 视频编码器、方法、系统和图像处理设备、车辆
KR20240089756A (ko) 다층 코딩 및 디코딩을 위한 alf aps

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17936287

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17936287

Country of ref document: EP

Kind code of ref document: A1