WO2019011011A1 - 图像处理方法、设备及系统 - Google Patents

图像处理方法、设备及系统 Download PDF

Info

Publication number
WO2019011011A1
WO2019011011A1 PCT/CN2018/081678 CN2018081678W WO2019011011A1 WO 2019011011 A1 WO2019011011 A1 WO 2019011011A1 CN 2018081678 W CN2018081678 W CN 2018081678W WO 2019011011 A1 WO2019011011 A1 WO 2019011011A1
Authority
WO
WIPO (PCT)
Prior art keywords
image block
reconstructed
current
reconstructed image
encoded
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/081678
Other languages
English (en)
French (fr)
Inventor
张红
杨海涛
刘杉
吴枫
李跃
刘�东
李厚强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
University of Science and Technology of China USTC
Original Assignee
Huawei Technologies Co Ltd
University of Science and Technology of China USTC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd, University of Science and Technology of China USTC filed Critical Huawei Technologies Co Ltd
Publication of WO2019011011A1 publication Critical patent/WO2019011011A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding

Definitions

  • the present application relates to image processing technologies, and in particular, to an image processing method, device, and system.
  • FIG. 1 is a schematic diagram of a digital video provided by the present application.
  • the digital video is composed of a multi-frame digital image.
  • 2 is a schematic diagram of a digital image provided by the present application.
  • the image is composed of 12*16 pixels, wherein each pixel is referred to as one pixel, and 12*16 represents image resolution.
  • the image resolution of 2K video is 1920*1080
  • the image resolution of 4K video is 3840*2160.
  • the original video contains a large amount of data, which is not suitable for storage and transmission, and requires efficient video compression coding techniques to compress the original data.
  • FIG. 3 is a schematic diagram of encoding of an encoding end provided by the present application.
  • the encoding process of the encoding end includes: after receiving the video by the encoding end, dividing the image into multiple to-bes for each frame image constituting the video. Encode image blocks.
  • the image block is first reconstructed by reference (the reference reconstructed image block is used to provide a reference pixel required for the current image block to be encoded, and the reference pixel is used to predict the current image block to be encoded).
  • the coded image block is predicted to obtain a prediction signal of the current image block to be coded; the prediction signal is subtracted from the original signal of the current image block to be coded to obtain a residual signal. After prediction, the magnitude of the residual signal is much smaller than the original signal.
  • the residual signal is transformed and quantized. After transform quantization, the transform quantized coefficients are obtained, and the quantized coefficients and other indication information in the encoding are encoded by the entropy coding technique to obtain a code stream. Further, the encoding end further needs to reconstruct the current image block to be encoded, so as to provide reference pixels for encoding the subsequent image block to be encoded.
  • the encoding end needs to perform inverse quantization and inverse transform on the transform quantized coefficients of the current image block to be encoded to obtain a reconstructed residual signal, and the reconstructed residual signal is
  • the prediction signals corresponding to the current image block to be encoded are added to obtain a reconstructed signal of the current image block to be encoded, and the reconstructed image block is obtained according to the reconstructed signal.
  • the reconstructed image block can predict a subsequent image block to be encoded.
  • the residual signal is transformed to obtain a transform coefficient, and the transform coefficient has information loss after being quantized, and the information loss is irreversible.
  • the inversely quantized transform coefficients are distorted, so that the reconstructed signal is inconsistent with the original signal.
  • This compression method is lossy compression. Therefore, for lossy compression, after the reconstructed image block is obtained, the reconstructed image block needs to be filtered to remove some distortion introduced by lossy compression, such as blockiness, ringing effect, and the like.
  • a DBK filter in the H.264, H.265 standard can be used.
  • the SAO filter in H.265, and the ALF filter in the next generation standard, etc. can be used.
  • the residual signal is obtained by using a lossless transform operation to obtain transform coefficients, and the transform coefficients are subjected to entropy coding without performing quantization operations.
  • lossless compression filtering operations are generally not performed. Further, after each image block of the current image is reconstructed, a reconstructed image is obtained, wherein the reconstructed image can predict other subsequent frame images.
  • FIG. 4 is a schematic diagram of decoding of a decoding end provided by the present application. As shown in FIG. 4, after acquiring a code stream, the decoding end first performs entropy decoding on the code stream to obtain a transform quantized coefficient of the current image block to be reconstructed, and then transforms the quantized coefficient. Performing inverse quantization and inverse transform to obtain a reconstructed residual signal of the current image block to be reconstructed.
  • FIG. 5 is a schematic diagram of encoding of the encoding end provided by the present application. As shown in FIG. 5, the encoding end uses downsampling processing on the entire image, and then Each image block to be encoded in the downsampled image is encoded to obtain a code stream. The resolution of the reconstructed image block corresponding to each image block to be encoded is a downsampling resolution.
  • the decoding end parses the code stream, and the resolution of each image block to be reconstructed is the downsampling resolution, and the resolution of the corresponding reconstructed image block is also the downsampling resolution, and the decoding end needs to adopt the reconstructed image block. Sampling processing to obtain a reconstructed image block of the original resolution.
  • the encoding end uses downsampling processing for the entire image.
  • the characteristics of each image block included in the entire image may be different.
  • some image blocks may be relatively flat, and the encoding end is suitable for downsampling processing.
  • Some image blocks may have more details, in which case the downsampling process will lose these details. This will result in poor coding on the encoding side.
  • this will result in some reconstructed image blocks obtained by the decoding end being relatively ambiguous, that is, the reconstructed image blocks obtained by the decoding end are not effective.
  • the application provides an image processing method, device and system, so that the reconstructed image block obtained by the decoding end has better effect, and the encoding effect of the encoding end is better.
  • the present application provides an image processing method, including: parsing a code stream to obtain an encoding mode of a current image block to be reconstructed of a current image, a residual signal of a current image block to be reconstructed, and a current image block to be reconstructed.
  • Each reference reconstructs a pixel in the image block, wherein the encoding mode is the original resolution encoding mode or the down sampling encoding mode, and the current image block to be reconstructed corresponds to M reference reconstructed image blocks, and M is a positive integer greater than or equal to 1; Determining, by a coding mode of the image block to be reconstructed and a pixel in the M reference reconstructed image blocks, a plurality of reference pixels of the current image block to be reconstructed; generating a prediction signal of the current image block to be reconstructed according to the plurality of reference pixels; The residual signal generates a reconstructed signal of the current image block to be reconstructed, and reconstructs the current image block to be reconstructed according to the reconstructed signal to obtain a current reconstructed image block.
  • the present application considers that image blocks have respective characteristics, and the encoding end uses different encoding methods for them. Based on this, the decoding end encodes the current image block to be reconstructed and M. The two coding factors of the reference reconstructed image block are taken into consideration to reconstruct the current image block to be reconstructed. Thereby, the reconstructed image block obtained by the decoding end is more effective.
  • the method further includes: performing, on the current reconstructed image block, the pixels of the adjacent reconstructed image block required for performing the upsampling process on the current reconstructed image block. Upsampling processing.
  • the upsampling process is performed after all the image blocks of the current image are reconstructed; correspondingly, the code stream includes: a coding mode of each of the reference reconstructed image blocks in the M reference reconstructed image blocks; Determining a plurality of reference pixels of the image block to be reconstructed according to the coding mode of the image block to be reconstructed and the pixels in the M reference reconstructed image blocks, including: coding according to the current image block to be reconstructed, and encoding of the M reference reconstructed image blocks The mode and the M reference reconstructed pixels in the image block determine a plurality of reference pixels of the current image block to be reconstructed.
  • the upsampling process is performed after all the reconstructed image blocks required for performing the upsampling process of the current reconstructed image block are reconstructed; correspondingly, the code stream includes: M reference reconstructed image blocks And determining, by the encoding mode of the image block to be reconstructed and the pixels in the M reference reconstructed image blocks, determining a plurality of reference pixels of the current image block to be reconstructed, including: according to the current image block to be reconstructed The coding mode, the coding mode of the M reference reconstructed image blocks, and the pixels in the M reference reconstructed image blocks determine a plurality of reference pixels of the current image block to be reconstructed.
  • the upsampling process of the current reconstructed image block includes: performing upsampling on the current reconstructed image block according to a pixel of a part of the adjacent reconstructed image block that is currently reconstructed in the adjacent reconstructed image block. Processing; if another part of the adjacent reconstructed image block of the currently uncompleted reconstruction in the required adjacent reconstructed image block has been reconstructed, then the partial boundary of the currently reconstructed image block is quadraminated according to another partial adjacent reconstructed image block.
  • a sampling process in which a partial boundary of a currently reconstructed image block is adjacent to another partially adjacent reconstructed image block.
  • the upsampling process of the current reconstructed image block includes: performing up-sampling the current reconstructed image block according to a pixel of a portion of the adjacent reconstructed image block that is currently reconstructed in the required adjacent reconstructed image block. Processing; if all the image blocks of the current image have been reconstructed, performing a second upsampling process on a partial boundary of the currently reconstructed image block according to another partially adjacent reconstructed image block in the desired adjacent reconstructed image block And another partial adjacent reconstructed image block is an image block that is not reconstructed when the current up-sampling image block is subjected to the first upsampling process; a partial boundary of the current reconstructed image block is adjacent to another partially adjacent reconstructed image block.
  • a part of the adjacent reconstructed image block is an upper image block and a left image block of the current reconstructed image block, and another partial adjacent reconstructed image block is a lower image block and a right image block of the current reconstructed image block; or, a part of the adjacent image block
  • the reconstructed image block is the upper left image block, the upper image block, the upper right image block and the left image block of the current reconstructed image block
  • the other adjacent reconstructed image block is the right image block of the current reconstructed image block, the lower left image block, and the lower image block and The bottom right image block.
  • a partial boundary of the currently reconstructed image block is a right boundary and a lower boundary of the currently reconstructed image block.
  • the required adjacent reconstructed image block includes an upper image block, a lower image block, a left image block, and a right image block of the current reconstructed image block; or the desired adjacent reconstructed image block includes the current reconstructed image block.
  • the method further includes: identifying that the current reconstructed image block has completed the upsampling process. It is thus possible to avoid repeated upsampling of the currently reconstructed image block.
  • the reference pixels include: if the encoding mode of the image block to be reconstructed is the original resolution encoding mode, and the M reference reconstructed image blocks include the reference reconstructed image block whose encoding mode is the original resolution encoding mode, then the encoding mode is original Determining the reference pixel of the current image block to be reconstructed in the pixel of the reference reconstructed image block of the resolution coding mode; if the coding mode of the current image block to be reconstructed is the original resolution coding mode, and the M reference reconstructed image blocks include the coding mode
  • the reference reconstructed image block of the sampling and coding mode acquires pixels required for reconstructing the current image block to be reconstructed from the pixels of the reference reconstructed image block whose encoding mode is the down sampling coding mode, and reconstructs the current image block to be reconstructed.
  • the required pixels are subjected to upsampling processing to obtain reference pixels of the current image block to be reconstructed;
  • the coding mode of the image block to be reconstructed is a down-sampling coding mode, and the M reference reconstructed image blocks include a reference reconstructed image block whose coding mode is a down-sampling coding mode, and the pixel of the reference reconstructed image block whose coding mode is the down-sampling coding mode.
  • Determining a reference pixel of the current image block to be reconstructed if the encoding mode of the current image block to be reconstructed is a downsampling coding mode, and the M reference reconstructed image blocks include a reference reconstructed image block whose encoding mode is the original resolution encoding mode, then The pixel required to reconstruct the current image block to be reconstructed is obtained from the pixels of the reference reconstructed image block of the original resolution coding mode, and the pixels required for reconstructing the current image block to be reconstructed are downsampled to The reference pixel of the current image block to be reconstructed is obtained.
  • determining a plurality of reference pixels of the current image block to be reconstructed according to the current coding mode of the image block to be reconstructed and the pixels in the M reference reconstructed image blocks including: if the current to be reconstructed
  • the encoding mode of the image block is the original resolution encoding mode, and the pixels required for reconstructing the current image block to be reconstructed are acquired in the pixels of the M reference reconstructed image blocks, and the current waiting is determined in the M reference reconstructed image blocks.
  • Reconstructing a reference pixel of the image block if the encoding mode of the current image block to be reconstructed is a down sampling coding mode, the pixels required for reconstructing the current image block to be reconstructed are downsampled to obtain a reference of the current image block to be reconstructed. Pixel.
  • the reference pixels of the currently reconstructed image block can be effectively determined, thereby realizing the reconstruction of the currently reconstructed image block.
  • the present application provides an image processing method, including: acquiring an encoding mode of a current image block to be encoded of a current image, and a pixel in each reference reconstructed image block corresponding to a current image block to be encoded, where the encoding mode is In the original resolution coding mode or the down sampling coding mode, the current image block to be coded corresponds to M reference reconstructed image blocks, and M is a positive integer greater than or equal to 1; according to the current coding mode of the image block to be encoded and M reference reconstructed image blocks a pixel in the image, determining a plurality of reference pixels of the current image block to be encoded; generating a prediction signal of the current image block to be encoded according to the plurality of reference pixels; acquiring an encoded signal of the current image block to be encoded, wherein when the image block to be currently encoded is When the encoding mode is the original resolution encoding mode, the encoded signal is the original signal of the current image block to be encoded,
  • the encoded signal is the original signal of the current image block to be encoded. a signal obtained after downsampling processing; generating a current to-be-programmed based on the predicted signal and the encoded signal The residual signal of the code image block; the residual signal pair is encoded.
  • the present application considers that the image blocks to be encoded have respective characteristics, and the encoding manners used by the encoding end are also different. Based on this, the encoding end encodes the current image block to be reconstructed. And the encoding method of the M reference reconstructed image blocks are taken into consideration to encode the current image block to be encoded. Thereby the encoding effect of the encoding end is better.
  • the method further includes: generating a reconstructed signal of the current image block to be encoded, and reconstructing the current image block to be encoded according to the reconstructed signal to obtain a current reconstructed image block; if the encoding mode of the current reconstructed image is a downsampling encoding mode, based on the current The pixels of the adjacent reconstructed image block required for the up-sampling process are reconstructed, and the currently reconstructed image block is subjected to upsampling processing.
  • the upsampling process is performed after all the image blocks of the current image are reconstructed.
  • the plurality of reference pixels of the image block to be reconstructed include: obtaining an encoding manner of each of the reference reconstructed image blocks of the M reference reconstructed image blocks; and encoding the encoding manner of the M reference reconstructed image blocks according to the current encoding mode of the image block to be encoded.
  • the pixels in the M reference reconstructed image blocks determining a plurality of reference pixels of the current image block to be encoded.
  • the upsampling process is performed after all the reconstructed image blocks required for performing the upsampling process of the current reconstructed image block are reconstructed; correspondingly, according to the coding mode of the current image block to be encoded and the And determining, by the pixels in the M reference reconstructed image blocks, the plurality of reference pixels of the current image block to be reconstructed, including: acquiring an encoding manner of each of the reference reconstructed image blocks in the M reference reconstructed image blocks; according to the current image block to be encoded
  • the coding mode, the coding mode of the M reference reconstructed image blocks, and the pixels in the M reference reconstructed image blocks determine a plurality of reference pixels of the current image block to be encoded.
  • the upsampling process of the current reconstructed image block includes: performing upsampling on the current reconstructed image block according to a pixel of a part of the adjacent reconstructed image block that is currently reconstructed in the adjacent reconstructed image block. Processing; if another part of the adjacent reconstructed image block of the currently uncompleted reconstruction in the required adjacent reconstructed image block has been reconstructed, then the partial boundary of the currently reconstructed image block is quadraminated according to another partial adjacent reconstructed image block.
  • a sampling process in which a partial boundary of a currently reconstructed image block is adjacent to another partially adjacent reconstructed image block.
  • the upsampling process of the current reconstructed image block includes: performing up-sampling the current reconstructed image block according to a pixel of a portion of the adjacent reconstructed image block that is currently reconstructed in the required adjacent reconstructed image block. Processing; if all the image blocks of the current image have been reconstructed, performing a second upsampling process on a partial boundary of the currently reconstructed image block according to another partially adjacent reconstructed image block in the desired adjacent reconstructed image block And another partial adjacent reconstructed image block is an image block that is not reconstructed when the current up-sampling image block is subjected to the first upsampling process; a partial boundary of the current reconstructed image block is adjacent to another partially adjacent reconstructed image block.
  • a part of the adjacent reconstructed image block is an upper image block and a left image block of the current reconstructed image block, and another partial adjacent reconstructed image block is a lower image block and a right image block of the current reconstructed image block; or, a part of the adjacent image block
  • the reconstructed image block is the upper left image block, the upper image block, the upper right image block and the left image block of the current reconstructed image block
  • the other adjacent reconstructed image block is the right image block of the current reconstructed image block, the lower left image block, and the lower image block and The bottom right image block.
  • a partial boundary of the currently reconstructed image block is a right boundary and a lower boundary of the currently reconstructed image block.
  • the required adjacent reconstructed image block includes an upper image block, a lower image block, a left image block, and a right image block of the current reconstructed image block; or the desired adjacent reconstructed image block includes the current reconstructed image block.
  • the method further includes: identifying that the current reconstructed image block has completed the upsampling process. It is thus possible to avoid repeated upsampling of the currently reconstructed image block.
  • the reference pixels include: if the encoding mode of the current image block to be encoded is the original resolution encoding mode, and the M reference reconstructed image blocks include the reference reconstructed image block whose encoding mode is the original resolution encoding mode, then the encoding mode is original Determining the reference pixel of the current image block to be encoded in the pixel of the reference reconstructed image block of the resolution coding mode; if the current coding mode of the image block to be encoded is the original resolution coding mode, and the M reference reconstructed image blocks include the coding mode
  • the reference reconstructed image block of the sampling and coding mode acquires pixels required for reconstructing the current reconstructed image block from the pixels of the reference reconstructed image block whose encoding mode is the downsampling coding mode, and reconstructs the current image block to be encoded.
  • the required pixels are subjected to upsampling processing to obtain reference pixels of the current image block to be encoded;
  • the encoding mode of the coded image block is a down-sampling coding mode
  • the M reference reconstructed image blocks include a reference reconstructed image block whose coding mode is a down-sampling coding mode, and is in a pixel of a reference reconstructed image block whose coding mode is a down-sampling coding mode.
  • Determining a reference pixel of the current image block to be encoded if the encoding mode of the current image block to be encoded is a downsampling coding mode, and the M reference reconstructed image blocks include a reference reconstructed image block whose encoding mode is the original resolution encoding mode, the encoding is performed.
  • the method is: acquiring pixels required for reconstructing the current image block to be encoded in the pixels of the reference reconstructed image block of the original resolution coding mode, and performing downsampling processing on the pixels required for reconstructing the current image block to be encoded, to obtain The reference pixel of the current image block to be encoded.
  • determining, according to the current coding mode of the image block to be encoded and the pixels in the M reference reconstructed image blocks, multiple reference pixels of the current image block to be encoded including: if currently coded If the encoding mode of the image block is the original resolution encoding mode, the reference pixels of the current image block to be encoded are determined in the pixels of the M reference reconstructed image blocks; if the encoding mode of the current image block to be encoded is the down sampling encoding mode, then The pixels required for reconstructing the current image block to be encoded are acquired in the pixels of the M reference reconstructed image blocks, and the pixels required for reconstructing the current image block to be encoded are subjected to down sampling processing to obtain the current image block to be encoded. Reference pixel.
  • the reference pixels of the currently reconstructed image block can be effectively determined by the above two optional methods, thereby realizing reconstruction of the currently reconstructed image block.
  • the obtaining the encoding mode of the current image block to be encoded includes: determining a first encoding cost when the current image block to be encoded adopts the original resolution encoding mode; determining a second when the current image block to be encoded adopts the down sampling encoding mode Encoding cost; the coding mode corresponding to the smaller coding cost of the first coding cost and the second coding cost is used as the coding mode of the current image block to be coded.
  • the coding mode used by the image block to be coded is a coding mode with a small coding cost, thereby reducing the coding complexity of the coding end, thereby improving the coding efficiency of the coding end.
  • the present application provides an image processing apparatus, including: a parsing module, configured to parse a code stream, to obtain an encoding mode of a current image block to be reconstructed of a current image, a residual signal of a current image block to be reconstructed, and a current waiting Reconstructing pixels in each reference reconstructed image block corresponding to the image block, wherein the encoding mode is the original resolution encoding mode or the down sampling encoding mode, and the current image block to be reconstructed corresponds to M reference reconstructed image blocks, where M is greater than or equal to 1 a positive integer; a determining module, configured to determine, according to an encoding manner of the current image block to be reconstructed and pixels in the M reference reconstructed image blocks, a plurality of reference pixels of the current image block to be reconstructed; and a generating module, configured to a pixel is used to generate a prediction signal of the current image block to be reconstructed; a reconstruction module is configured to generate a
  • the present application provides an image processing apparatus, including: an obtaining module, configured to acquire an encoding manner of a current image block to be encoded of a current image and a pixel in each reference reconstructed image block corresponding to a current image block to be encoded,
  • the encoding mode is the original resolution encoding mode or the down sampling encoding mode.
  • the current image block to be encoded corresponds to M reference reconstructed image blocks, and M is a positive integer greater than or equal to 1.
  • the determining module is configured to use the current image block to be encoded.
  • Encoding mode and pixels in the M reference reconstructed image blocks determining a plurality of reference pixels of the current image block to be encoded; generating a module, configured to generate a prediction signal of the current image block to be encoded according to the plurality of reference pixels;
  • An encoded signal for acquiring a current image block to be encoded wherein when the encoding mode of the current image block to be encoded is the original resolution encoding mode, the encoded signal is the original signal of the current image block to be encoded, when the image block to be currently encoded is When the coding mode is the downsampling coding mode, the coded signal is the original signal of the current image block to be coded.
  • the signal obtained after processing the samples; generating module is further configured to generate a residual signal of a current image block to be encoded and a prediction signal based on the encoded signal; an encoding module for encoding the residual signal.
  • the present application provides an image processing system, comprising: the image processing device according to the third aspect, and the image processing device according to the fourth aspect.
  • the present application provides an image processing apparatus including a decoder configured to:
  • a pixel in the image determining a plurality of reference pixels of the image block to be reconstructed; generating a prediction signal of the current image block to be reconstructed according to the plurality of reference pixels; generating a reconstruction signal of the current image block to be reconstructed according to the prediction signal and the residual signal, and according to The reconstructed signal reconstructs the current image block to be reconstructed to obtain a current reconstructed image block.
  • the present application provides an image processing apparatus including an encoder configured to:
  • the encoding mode of the current image block to be encoded of the current image and the pixel in each reference reconstructed image block corresponding to the current image block to be encoded, where the encoding mode is the original resolution encoding mode or the down sampling encoding mode, and the current image to be encoded
  • the block corresponds to M reference reconstructed image blocks, and M is a positive integer greater than or equal to 1; determining a plurality of reference pixels of the current image block to be encoded according to the encoding mode of the current image block to be encoded and the pixels in the M reference reconstructed image blocks Generating a prediction signal of the current image block to be encoded according to the plurality of reference pixels; acquiring an encoded signal of the current image block to be encoded, wherein when the encoding mode of the current image block to be encoded is the original resolution encoding mode, the encoded signal is currently The original signal of the image block is encoded.
  • the encoded signal is a signal obtained by down-sampling the original signal of the current image block to be encoded; and is generated according to the prediction signal and the encoded signal.
  • the residual signal of the current image block to be encoded encoding the residual signal.
  • the present application provides a computer storage medium for storing computer software instructions for use in the image processing apparatus of the third aspect or the sixth aspect, which is configured to perform the third aspect or the sixth aspect described above. Designed program.
  • the present application provides a computer program product comprising instructions which, when executed by a computer, cause the computer to perform the functions performed by the image processing apparatus of the third aspect or the sixth aspect.
  • the present application provides a computer storage medium for storing computer software instructions for use in the image processing device of the fourth aspect or the seventh aspect, which is configured to perform the fourth aspect or the seventh aspect described above. Designed program.
  • the present application provides a computer program product comprising instructions which, when executed by a computer, cause the computer to perform the functions performed by the image processing apparatus of the fourth aspect or the seventh aspect.
  • the present application provides an image processing method, device, and system. Since the present application considers that image blocks have respective characteristics, the encoding end uses different encoding methods for them, and based on this, the decoding end will image the current image to be reconstructed.
  • the encoding method of the block and the encoding mode of the M reference reconstructed image blocks are taken into consideration to reconstruct the current image block to be reconstructed. Thereby, the reconstructed image block obtained by the decoding end is more effective.
  • the encoding end takes into account two factors of the encoding mode of the current image block to be reconstructed and the encoding mode of the M reference reconstructed image blocks to encode the current image block to be encoded. Thereby the encoding effect of the encoding end is better.
  • FIG. 1 is a schematic diagram of a digital video provided by the present application.
  • FIG. 2 is a schematic diagram of a digital image provided by the present application.
  • FIG. 3 is a schematic diagram of coding of an encoding end provided by the present application.
  • FIG. 4 is a schematic diagram of decoding of a decoding end provided by the present application.
  • FIG. 5 is a schematic diagram of coding of an encoding end provided by the present application.
  • FIG. 6 is a schematic diagram of an image being encoded according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a reference pixel template according to an embodiment of the present application.
  • FIGS. 8A and 8B are schematic diagrams of a Planar mode according to an embodiment of the present application.
  • FIG. 9 is a schematic diagram of specific directions of 33 kinds of angle prediction modes according to an embodiment of the present application.
  • FIG. 10 is a schematic diagram of image down sampling according to an embodiment of the present application.
  • FIG. 11 is a schematic diagram of image up sampling according to an embodiment of the present application.
  • FIG. 12 is a schematic diagram of image up sampling according to an embodiment of the present application.
  • FIG. 13 is a flowchart of an image processing method according to an embodiment of the present application.
  • FIG. 14 is a schematic diagram of a 4-neighbor pixel provided by an embodiment of the present application.
  • FIG. 15 is a schematic diagram of an 8-neighbor pixel provided by an embodiment of the present application.
  • FIG. 16 is a schematic diagram of image upsampling according to an embodiment of the present disclosure.
  • FIG. 17 is a schematic diagram of image upsampling according to another embodiment of the present disclosure.
  • FIG. 18 is a schematic diagram of image upsampling according to an embodiment of the present disclosure.
  • FIG. 19 is a schematic diagram of image upsampling according to another embodiment of the present application.
  • FIG. 20 is a schematic diagram of a current image block to be reconstructed and a reference reconstructed image block according to an embodiment of the present disclosure
  • FIG. 21 is a schematic diagram of a current image block to be reconstructed and a reference reconstructed image block according to an embodiment of the present disclosure
  • FIG. 22 is a schematic diagram of a current image block to be reconstructed and a reference reconstructed image block according to an embodiment of the present disclosure
  • FIG. 23 is a schematic diagram of a current image block to be reconstructed and a reference reconstructed image block according to an embodiment of the present disclosure
  • FIG. 24 is a flowchart of an image processing method according to another embodiment of the present application.
  • FIG. 25 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present disclosure.
  • FIG. 26 is a schematic structural diagram of an image processing apparatus according to another embodiment of the present disclosure.
  • FIG. 27 is a schematic structural diagram of an image processing system provided by the present application.
  • Digital video is a video recorded in digital form.
  • Digital video consists of multi-frame digital images. Often the original video contains a large amount of data, which is not suitable for storage and transmission, and requires efficient video compression coding techniques to compress the original data.
  • Video compression technology achieves compression by eliminating video redundancy.
  • Video redundancy mainly includes the following items: spatial redundancy, temporal redundancy, visual redundancy, and information entropy redundancy.
  • Spatial redundancy It is the most important data redundancy for static images. It means that the amplitudes of adjacent pixels are similar in an image. This spatial coherence is called spatial correlation or spatial redundancy.
  • the spatial redundancy is mainly eliminated by the intra prediction method.
  • the intra prediction method refers to using the correlation of the video spatial domain to predict the pixels of the current image block by using the pixels of the reference reconstructed image block to achieve the purpose of removing the video spatial redundancy. .
  • Temporal redundancy It is the redundancy that is often included in video sequences. Since adjacent images of video often contain the same or similar background and moving objects, only the spatial position of the moving object is slightly different. The high correlation of data is called time redundancy. Temporal redundancy is mainly eliminated by inter-frame prediction techniques, which refer to the use of pixels of temporally adjacent images to predict the current pixel.
  • Visual redundancy The human visual system is insensitive to changes in image detail, and even if these subtle changes are lost, the human eye does not feel it.
  • the visual redundancy is mainly eliminated by transforming and transforming technology, which refers to transforming the image signal into the frequency domain for processing, and performing data expression and bit redistribution according to the contribution of the different frequency signals to the visual quality, so that the space can be corrected. Unreasonable expression of uniform sampling over a domain.
  • the need to remove visual redundancy is taken into consideration, and the quantization operation is omitted, and the excessively high-frequency component expression is omitted to achieve effective compression.
  • Information entropy redundancy It can be known from information theory that as a pixel representing image data, it is only necessary to allocate the corresponding number of bits according to the size of its information entropy, and for each pixel of image data, it is difficult to obtain its information at the time of image acquisition. Entropy, therefore, is generally expressed by the same number of bits per pixel, so there must be redundancy. Information entropy redundancy is mainly solved by entropy coding technology. Entropy coding technology distributes different bit numbers for data with different information entropy through the information entropy distribution of statistical coefficients.
  • the current mainstream video compression coding architecture is a hybrid coding architecture.
  • different technologies are adopted to eliminate redundancy, and these technologies are combined to form a hybrid architecture of video coding.
  • the encoding side divides the image into image blocks to be encoded for each frame of the image constituting the video.
  • the current image block to be encoded is first predicted by referring to the reconstructed image block to obtain a prediction signal of the current image block to be encoded; and the residual signal is obtained by subtracting the prediction signal from the original signal of the current image block to be encoded. .
  • the magnitude of the residual signal is much smaller than the original signal.
  • the residual signal is transformed and quantized.
  • the transform quantized coefficients are obtained, and the quantized coefficients and other indication information in the encoding are encoded by the entropy coding technique to obtain a code stream. Further, the encoding end further needs to reconstruct the current image block to be encoded, so as to provide reference pixels for encoding the subsequent image block to be encoded.
  • the encoding end needs to perform inverse quantization and inverse transform on the transform quantized coefficients of the current image block to be encoded to obtain a reconstructed residual signal, and the reconstructed residual signal is
  • the prediction signals corresponding to the current image block to be encoded are added to obtain a reconstructed signal of the current image block to be encoded, and the reconstructed image block is obtained according to the reconstructed signal.
  • the decoding end after acquiring the code stream, the decoding end first performs entropy decoding on the code stream to obtain a transformed quantized coefficient of the current image block to be reconstructed, and then inverse quantizes and inverse transforms the transformed quantized coefficient to obtain an image to be reconstructed.
  • the residual signal of the reconstruction of the block Predicting the current image block to be reconstructed by referring to the reconstructed image block, obtaining a prediction signal of the current image block to be reconstructed, and then adding the prediction signal and the reconstructed residual signal to obtain a reconstruction signal of the current image block to be reconstructed, and then according to The reconstructed signal obtains a current reconstructed image block corresponding to the current image block to be reconstructed.
  • the encoding end first downsamples the image of each frame. As shown in FIG. 5, the encoding end uses downsampling processing on the entire image, and then each of the images subjected to the downsampling process.
  • the coded image block is encoded to obtain a code stream.
  • the resolution of the reconstructed image block corresponding to each image block to be encoded is a downsampling resolution.
  • the decoding end parses the code stream, and the resolution of each image block to be reconstructed is the downsampling resolution, and the resolution of the corresponding reconstructed image block is also the downsampling resolution, and the decoding end needs to adopt the reconstructed image block. Sampling processing to obtain a reconstructed image block of the original resolution.
  • the encoding end and the decoding end both involve predicting the current image block (the current image block to be encoded or the current image block to be reconstructed) by referring to the reconstructed image block, and obtaining a prediction signal of the current image block.
  • the prediction mode (mainly the intra prediction method) of the current image block may adopt the prior art, as follows:
  • FIG. 6 is a schematic diagram of an image being encoded according to an embodiment of the present application.
  • the image includes a plurality of image blocks, where the encoding order of the image is: from top to bottom, from left to right. right.
  • image blocks C, B, D, E, and A represent reconstructed image blocks that have been reconstructed
  • image block F is the current image block to be encoded
  • other regions in the image are uncoded image regions.
  • H.265 supports dividing the current image block to be encoded into smaller sub-image blocks for prediction operations.
  • the partition structure of the sub-image block is a quadtree structure, that is, one image block can be divided into four sub-image blocks, and each sub-image block can be further divided into four sub-image blocks.
  • FIG. 6 it is assumed that the current image block to be encoded is divided into 7 sub-image blocks for prediction operation, and the current image block to be encoded can also be divided into more sub-image blocks for prediction operations.
  • FIG. 7 is a schematic diagram of a reference pixel template according to an embodiment of the present application. As shown in FIG. 7, P 1, 1 , P 2 , 1 ...
  • the sub-image block to be encoded may be the sub-image in FIG. Block 1.
  • other reference pixels R 0,0 , R 1,0 ... R 2N+1,0 ... R 0,2N constitute a reference pixel template, assuming that the image is to be encoded.
  • the sub-image block is the sub-image block 1 in FIG. 6, in which case some of the reference pixels are the pixels of the last row of the reference reconstructed image block B, and the other portion of the pixels are the rightmost side of the reference reconstructed image block A A column of pixels.
  • some of the reference pixels are pixels of the lower plurality of rows included in the reference reconstructed image block B, and the other partial pixels are pixels of the right plurality of columns included in the reference reconstructed image block A. That is, the present application does not limit the reference pixel template.
  • FIG. 8A and FIG. 8B are schematic diagrams of the Planar mode provided by an embodiment of the present application. As shown in FIG. 8, two linear filters in the horizontal and vertical directions are used, respectively. Two predicted values with And with The average value is used as a prediction signal for the pixel (x, y).
  • the DC mode is applicable to a large area flat area, and the prediction signal of the current sub-image block to be encoded can be obtained from the average value of the reference pixels on the left and the top thereof, as shown in FIG. 7, the prediction of each pixel in the sub-image block to be encoded.
  • the signal can be obtained by averaging R 0,1 ,..., R 0,N , R 1,0 ,..., R N,0 .
  • FIG. 9 is a schematic diagram of specific directions of 33 kinds of angle prediction modes according to an embodiment of the present invention. As shown in FIG. 9 , the 33 angle prediction modes are classified into a horizontal type mode (2 to 17) and a vertical type mode (18 to 34). ). Where V0 (mode 26) and H0 (mode 10) represent the vertical and horizontal directions, respectively, and the prediction directions of the remaining angle prediction modes can be regarded as an angular offset in the vertical or horizontal direction.
  • the angle prediction process is illustrated by taking the vertical direction V0 (26) as an example.
  • the vertical direction prediction is to predict the current sub-image block to be encoded using a row of reference pixels adjacent to the sub-image block to be currently encoded, which is currently to be encoded.
  • the position of the reference pixel may be the position between two adjacent reference pixels. If this is the case, a reference pixel needs to be interpolated between the two reference pixels according to the calculated position.
  • a prediction signal is generated from the obtained reference pixels.
  • the present application also relates to image downsampling processing and image upsampling processing.
  • the image downsampling process involves three aspects of information: 1. Downsampling ratio; 2. Downsampling position; 3. Filter used for downsampling.
  • the downsampling ratio refers to the ratio of the original image to the downsampled image, which can be described in the horizontal direction and the vertical direction, respectively.
  • the image signal can be downsampled in the horizontal direction by 2:1, the vertical direction is downsampled by 4:1; or the horizontal direction is not downsampled, the vertical direction is downsampled by 2:1; or both horizontal and vertical directions are performed under 2:1. Sampling, etc.
  • the down sampling position refers to the positional relationship between the down sampling point and the original sampling point.
  • the position of the down sampling point may be the same as the position of some of the original sampling points, or the down sampling point falls between several original sampling points.
  • the downsampling filter may be a 3-lobe Lanczos filter, a Bilinear filter, a Bicubic, a Gauss filter, or the like.
  • FIG. 10 is a schematic diagram of image down sampling according to an embodiment of the present invention. It is assumed that the sampling ratios in the horizontal direction and the vertical direction are both 2:1. In the horizontal direction, the position of the down sampling point falls to the left of the two original sampling points. The original sampling point position, in the vertical direction, the down sampling point falls on the original sampling point position above the two original sampling points. As shown in Figure 10, the circled circle indicates the position of the downsampling point.
  • the downsampling filter is as follows:
  • the filter is a simple low-pass filter that can be thought of as a two-dimensional filter or as two one-dimensional filters. If used as a two-dimensional filter, downsampling in both horizontal and vertical directions can be done simultaneously in one filtering operation. As shown in FIG. 10, when downsampling point A is downsampled, eight adjacent original sampling points (circles circled by a triangle) are used, and the pixel value of the down sampling point A is calculated according to the above filter. If it is treated as two one-dimensional filters, it is necessary to first perform down-sampling in the horizontal or vertical direction, and then down-sample the results of the horizontal or vertical down-sampling in the completed horizontal or vertical direction. As shown in FIG.
  • downsampling point A when downsampling point A is down-sampled, firstly, one original sampling point is used to perform horizontal down sampling, and then the down-sampling point A is used to sample one original sample. The point is downsampled in the vertical direction, and the pixel value of the down sample point A is calculated according to the above filter. Using the same method, the entire 16*16 image block is downsampled. The final downsampling result is shown in Figure 10. The position of each downsampled point is shown by the circle in the box, and the pixel value of the downsampled point is The value after passing the filter operation. As shown in FIG. 10, the resolution of the downsampled image block is 8*8.
  • the upsampling process involves three aspects of information: 1. Upsampling ratio; 2. Upsampling position; 3 Filter used for upsampling.
  • the upsampling ratio refers to the ratio of the image before the upsampling to the image after the upsampling, which can be described in the horizontal direction and the vertical direction, respectively.
  • the upsampled image signal can be upsampled in the horizontal direction by 1:2, the vertical direction is 1:4 upsampled; or the horizontal direction is not upsampled, the vertical direction is 1:2 upsampled; or both horizontal and vertical directions are Perform 1:2 upsampling, etc.
  • the upsampling position is a positional relationship between the sampled point after the upsampling and the sampled point before the upsampling.
  • FIG. 11 is a schematic diagram of image upsampling provided in an embodiment of the present application, as shown in FIG.
  • the horizontal 1:2 upsampling ratio is used, and the position of the upsampled sample point can be on the right side of the sample point before the upsampling, where ⁇ represents the position of the sampled point after upsampling, and the circle represents the position of the sampled point before the upsampling.
  • the horizontal 1:2 upsampling ratio is used, and the position of the upsampled sample point can be to the left of the sample point before the upsampling, where ⁇ represents the position of the sampled point after upsampling, and the circle represents the position before the upsampling.
  • the sampling point location It should be noted that the position of the sampled point after the upsampling should correspond to the position selection of the above-mentioned down sampling point. For example, when performing downsampling, the position of the down sampling point is selected as the position of the original sampling point on the left side thereof. Then, when performing upsampling, the position of the sampled point after upsampling is selected as the position of the sampling point (downsampling point) before the upper right sampling.
  • the upsampling filter may be a DCTIF filter, a bilinear interpolation filter, a sinc filter, or the like.
  • the upsampling process will be described below by taking an image block with a resolution of 8*8 (i.e., the image block after downsampling described above) as an example. Assume that the upsampling ratios in the horizontal direction and the vertical direction are both 1:2.
  • the position of the sampled point after upsampling is the position of the sampling point before the right upsampling, and in the vertical direction, the upsampling
  • the position of the subsequent sampling point is the position of the sampling point before the upper upsampling, where the horizontal and vertical directions are respectively sampled as an example, and the DCTIF filter is taken as an example to illustrate the upsampling process.
  • the DCTIF filter is (-1, 4, -11, 40, 40, -11, 4, -1), assuming that the horizontal direction is currently being sampled.
  • Figure 11 assuming that the B3 sample point needs to be inserted, the following is used.
  • the formula determines the pixel value of B3:
  • FIG. 12 is a schematic diagram of image upsampling according to an embodiment of the present invention. As shown in FIG. 12, x represents a sampled point after upsampling, and a circle represents a sample point before upsampling.
  • the encoding end uses downsampling processing for the entire image.
  • the characteristics of each image block included in the entire image may be different.
  • some image blocks may be relatively flat, and the encoding end is suitable for downsampling processing.
  • Some image blocks may have more details, in which case the downsampling process will lose these details. This will result in poor coding on the encoding side.
  • this will result in some reconstructed image blocks obtained by the decoding end being relatively ambiguous, that is, the reconstructed image blocks obtained by the decoding end are not effective.
  • the present application provides an image processing method, device, and system.
  • the present application is based on the coding diagrams of FIG. 3 and FIG. 5.
  • the coding mode of the image block to be encoded included in one image may be the original resolution coding mode as shown in FIG.
  • the original resolution encoding method refers to directly performing an encoding operation on the current image block to be encoded.
  • the downsampling coding mode refers to performing downsampling processing on the current image block to be encoded, and then performing encoding operation on the currently sampled image block to be coded.
  • the texture image block adopts the original resolution coding mode
  • the smooth image block adopts the down sampling coding mode.
  • the encoding side needs to mark the encoding method used by each image block to be encoded, and write the label into the code stream.
  • the decoding end is caused to perform corresponding operations on the image block to be reconstructed according to the mark.
  • the main idea of the present application is that the decoding end determines the current to be reconstructed according to the coding mode of the current image block to be reconstructed, the coding mode of the M reference reconstructed image blocks corresponding to the current image block to be reconstructed, and the pixels in the M reference reconstructed image blocks.
  • a plurality of reference pixels of the image block generate a prediction signal according to the plurality of reference pixels, thereby reconstructing the image block to be reconstructed.
  • the encoding end determines the current image block to be encoded according to the encoding mode of the current image block to be encoded, the encoding mode of the M reference reconstructed image blocks corresponding to the current image block to be encoded, and the pixels in the M reference reconstructed image blocks.
  • the reference pixels generate a prediction signal according to the plurality of reference pixels, thereby obtaining a residual signal, and encoding the residual signal.
  • FIG. 13 is a flowchart of an image processing method according to an embodiment of the present disclosure. As shown in FIG. 13, the method includes:
  • Step S1301 Parsing the code stream to obtain an encoding mode of the current image block to be reconstructed of the current image, a residual signal of the current image block to be reconstructed, and a pixel in each reference reconstructed image block corresponding to the current image block to be reconstructed;
  • the decoding end parses the code stream, where the code stream carries the label corresponding to the encoding mode of the current image block to be reconstructed, and the transform quantization coefficient of the current image block to be reconstructed.
  • the decoding end may perform inverse quantization and inverse transform on the transform quantized coefficients to obtain a residual signal of the current image block to be reconstructed.
  • the current image block to be reconstructed corresponds to M reference reconstructed image blocks, and M is a positive integer greater than or equal to 1.
  • the reference reconstructed image block is used to determine a plurality of reference pixels of the image block to be reconstructed, wherein the plurality of reference pixels are used to generate a prediction signal of the current reconstructed image block.
  • the reference reconstructed image block is specifically related to which prediction image block and the prediction mode employed by the decoding end. When any of the above-described prediction modes in 35 is employed, reference may be made to the reference pixel template as shown in FIG.
  • Step S1302 Determine a plurality of reference pixels of the current image block to be reconstructed according to the coding mode of the current image block to be reconstructed and the pixels in the M reference reconstructed image blocks.
  • the reference reconstructed image block has the same resolution as the current image block to be reconstructed, at least one reference pixel is directly determined in the reference reconstructed image block; if the current image block to be reconstructed is the original resolution, the resolution of the reference reconstructed image block is determined.
  • Rate the downsampling resolution obtain at least one pixel required for reconstructing the current reconstructed image block from the reference reconstructed image block, and perform upsampling processing on at least one pixel required for reconstructing the current reconstructed image block
  • At least one pixel required for reconstruction of the image block to be reconstructed is currently subjected to downsampling processing to obtain at least one reference pixel of the current image block to be reconstructed.
  • Step S1303 Generate a prediction signal of the current image block to be reconstructed according to the plurality of reference pixels
  • Step S1304 Generate a reconstruction signal of the current image block to be reconstructed according to the prediction signal and the residual signal, and reconstruct the current image block to be reconstructed according to the reconstruction signal to obtain a current reconstructed image block.
  • the prediction mode of the current image block to be reconstructed is generated according to the plurality of reference pixels, and any one of the prediction modes in the foregoing 35 prediction mode may be used, and of course, other prediction modes in the prior art may also be used. Make restrictions. Finally, the reconstructed residual signal is added to the prediction signal to obtain a reconstructed signal of the current image block to be reconstructed, and the current image block to be reconstructed is reconstructed according to the reconstructed signal to obtain a current reconstructed image block.
  • the present application considers that image blocks have their own characteristics, and the encoding end uses different encoding methods for them. Based on this, the decoding end takes the factor of the encoding mode of the image block to be reconstructed into consideration to reconstruct The image block to be reconstructed currently. Thereby, the reconstructed image block obtained by the decoding end is more effective.
  • the image processing method further includes: pixels of the adjacent reconstructed image block required for performing the upsampling process on the current reconstructed image block, and the current reconstructed image block Perform upsampling processing.
  • the pixels of the adjacent reconstructed image blocks are mainly used for upsampling the partial boundaries of the currently reconstructed image block, and the portions of the current reconstructed image block except the above partial boundaries are used. The pixels are upsampled.
  • the decoding end uses the filter to perform upsampling processing on the current reconstructed image block, the adjacent reconstructed image blocks required for the current reconstructed image block are also different for different filters.
  • FIG. 14 is a A schematic diagram of a 4-neighbor pixel provided by an embodiment, as shown in FIG. 14, the adjacent reconstructed image block required for the current reconstructed image block includes: an upper image block, a lower image block, a left image block, and a right image of the current reconstructed image block. Piece. It is assumed that the filter is a Convolutional Neural Network (CNN) filter. In this case, the adjacent reconstructed image block required for reconstructing the image block is as follows: FIG.
  • CNN Convolutional Neural Network
  • the adjacent reconstructed image block includes: an upper image block, a lower image block, a left image block, a right image block, an upper left image block, a lower left image block, an upper right image block, and a lower right image block of the current reconstructed image block.
  • the current coding order from top to bottom, from left to right coding order
  • the lower image block, the right image block, the lower left image block, and the lower right image block of the current reconstructed image block have not yet been reconstructed.
  • the technique is to achieve upsampling processing by copying the pixels of the current reconstructed image block itself, but this way will cause the current reconstructed image block after the upsampling process to have a problem that the right and lower boundaries are discontinuous.
  • the present application provides the following four options:
  • the upsampling process is performed after all the reconstructed image blocks required for the upsampling process of the current reconstructed image block are reconstructed; correspondingly, the code stream includes: each of the M reference reconstructed image blocks. Referring to the coding mode of the reconstructed image block; determining, according to the current coding mode of the image block to be reconstructed and the pixels in the M reference reconstructed image blocks, the plurality of reference pixels of the current image block to be reconstructed, including: coding according to the current image block to be reconstructed The manner, the coding manner of the M reference reconstructed image blocks, and the pixels in the M reference reconstructed image blocks determine a plurality of reference pixels of the current image block to be reconstructed.
  • the upsampling process is performed after all the image blocks of the current image are reconstructed.
  • the code stream includes: a coding mode of each reference reconstructed image block in the M reference reconstructed image blocks;
  • the coding mode of the image block and the pixels in the M reference reconstructed image blocks determine a plurality of reference pixels of the current image block to be reconstructed, including: coding manner according to the current image block to be reconstructed, coding mode of the M reference reconstructed image blocks, and
  • the M reference reconstructed pixels in the image block determine a plurality of reference pixels of the current image block to be reconstructed.
  • Manner 3 performing a upsampling process on the currently reconstructed image block according to a pixel of a part of the adjacent reconstructed image block that is currently reconstructed in the required adjacent reconstructed image block; if the required adjacent reconstructed image block is in the If another part of the adjacent reconstructed image block that has not been reconstructed has been reconstructed, the partial boundary of the currently reconstructed image block is subjected to a second upsampling process according to another part of the adjacent reconstructed image block, wherein a part of the boundary of the currently reconstructed image block is Another portion of the adjacent reconstructed image blocks are contiguous.
  • the method of performing the upsampling process on the current reconstructed image block includes: performing an upsampling process on the currently reconstructed image block according to the pixels of the part of the adjacent reconstructed image block that are currently reconstructed in the required adjacent reconstructed image block; If all the image blocks of the current image have been reconstructed, the partial boundary of the currently reconstructed image block is subjected to secondary upsampling according to another partially adjacent reconstructed image block in the desired adjacent reconstructed image block, wherein A portion of the adjacent reconstructed image block is an image block that is not reconstructed when the first upsampling process is performed on the current reconstructed image block; a partial boundary of the current reconstructed image block is adjacent to another partially adjacent reconstructed image block.
  • the partial boundary of the current reconstructed image block satisfies the condition that, in the first upsampling process performed on the current reconstructed image block, another part of the adjacent reconstructed image block required for the partial boundary is not reconstructed.
  • a partial boundary of the currently reconstructed image block is a right boundary and a lower boundary of the currently reconstructed image block.
  • the part of the adjacent reconstructed image block is an upper image block and a left image block of the current reconstructed image block
  • the another partial adjacent reconstructed image block is a lower image block and a right side of the current reconstructed image block.
  • the part of the adjacent reconstructed image block is an upper left image block, an upper image block, an upper right image block and a left image block of the current reconstructed image block, and the other partially adjacent reconstructed image block is a right side of the current reconstructed image block.
  • the adjacent reconstructed image blocks required to reconstruct the image block are also different for different filters.
  • the adjacent reconstructed image block required for the current reconstructed image block includes: an upper image block, a lower image block, a left image block, and a right image block of the current reconstructed image block.
  • the adjacent reconstructed image block required for the current reconstructed image block includes: an upper image block, a lower image block, a left image block, a right image block, an upper left image block, a lower left image block, and an upper right of the currently reconstructed image block.
  • Image block and bottom right image block is also different for different filters.
  • FIG. 16 is a schematic diagram of image upsampling provided by an embodiment of the present application.
  • the adjacent reconstructed image block 1, the adjacent reconstructed image block 2, and the adjacent reconstructed image block required for reconstructing the image block B are as shown in FIG. 3 and the adjacent reconstructed image block 4 have all been reconstructed.
  • the current reconstructed image block B is subjected to upsampling processing, as shown in FIG. 16, wherein a circle in B indicates a sampling point before upsampling, and ⁇ indicates a sampling point after upsampling.
  • the currently reconstructed image block itself is a boundary image block of an image, in this case, even if the required adjacent reconstructed image block is reconstructed, it is necessary to copy the current reconstructed image when performing the upsampling process.
  • Pixel For example, as shown in FIG. 14, when the currently reconstructed image block is the rightmost image block of an image, its right image block does not exist, so the rightmost column included in the currently reconstructed image block can be included.
  • the pixels are copied to achieve upsampling processing.
  • other methods may also be used for the upsampling process, which is not limited in this application.
  • each adjacent reconstructed image block required for reconstructing the image block may be a downsampling coding mode or an original resolution coding mode, when performing the upsampling process on the current reconstructed image block, Divided into the following two cases:
  • the current reconstructed image block may be directly upsampled according to the pixels in the adjacent reconstructed image block.
  • the coding mode of an adjacent reconstructed image block is the original resolution coding mode
  • at least one pixel required for the upsampling process in the pixels of the adjacent reconstructed image block may be acquired, and the pixels are downsampled, and The current reconstructed image block is subjected to upsampling processing according to at least one pixel after the downsampling process.
  • the adjacent reconstructed image block is mainly used for performing upsampling processing on a partial boundary of the currently reconstructed image block (the partial boundary is different according to the filter), for example, as shown in FIG. 16, the adjacent reconstructed image block 3
  • the downsampling coding mode is adopted.
  • the right boundary of the current reconstructed image block B can be upsampled directly by using the pixels included in the adjacent reconstructed image block 3.
  • the adjacent reconstructed image block 4 adopts the original resolution encoding mode, the pixels required for the upsampling process included in the adjacent reconstructed image block 4 need to be downsampled, or the adjacent reconstructed image block 4 is subjected to downsampling processing.
  • FIG. 17 is a schematic diagram of image upsampling according to another embodiment of the present application. As shown in FIG. 17, this is a case of 8 neighborhood pixels, and the decoding side samples the current reconstructed image block C by using an upsampling processing method and 4 neighborhoods. The method in the case of a pixel is similar, and will not be described here.
  • the current reconstructed image block may be subjected to upsampling processing, and the current reconstructed image block may be identified as having completed the upsampling process.
  • the current reconstructed image block is subjected to upsampling processing according to a certain rule.
  • the upsampling process is based on the case of 4 neighborhood pixels
  • the current reconstructed image block may be subjected to upsampling processing.
  • the upsampling process is based on the case of 8 neighborhood pixels, once the lower right image block of the currently reconstructed image block is reconstructed, the current reconstructed image block may be subjected to upsampling processing.
  • the adjacent reconstructed image blocks it needs have been reconstructed, based on which, for any reconstructed image using downsampling coding
  • the blocks can be upsampled.
  • the specific upsampling process is similar to the mode of the foregoing mode 1, and the application is not described herein again.
  • the upsampling process performed on the current reconstructed image block includes two upsampling processes.
  • the first upsampling process is: performing a upsampling process on the currently reconstructed image block according to the pixels of a part of the adjacent reconstructed image blocks of the currently completed reconstructed image in the required adjacent reconstructed image block.
  • the second upsampling process is: if another part of the adjacent reconstructed image block of the currently uncompleted reconstruction in the required adjacent reconstructed image block has been reconstructed, the currently reconstructed image block is reconstructed according to another partial adjacent reconstructed image block.
  • the partial boundary is subjected to a second upsampling process in which the partial boundary of the currently reconstructed image block is contiguous with another partially adjacent reconstructed image block.
  • the current reconstructed image block of the downsampling coding mode is saved, and the reference pixels are provided for subsequent prediction of the other image blocks to be reconstructed.
  • the decoding end may copy the pixels of the rightmost column or columns included in the current reconstructed image block to obtain interpolated pixels. Upsampling processing is performed on the right boundary included in the currently reconstructed image block by these interpolated pixels. The decoding end may also copy the pixels of the lowermost row or rows included in the current reconstructed image block to obtain interpolated pixels. Upsampling processing is performed on the lower boundary included in the currently reconstructed image block by these interpolated pixels.
  • FIG. 18 is a schematic diagram of image upsampling according to an embodiment of the present invention.
  • the current reconstructed image block is image block A
  • the current reconstructed image block A is subjected to the second upsampling process
  • the DCTIF filter performs upsampling processing.
  • the current reconstructed image block A has completed the first upsampling process.
  • the right four reference pixels required for each x do not exist. If the adjacent reconstructed image block C is reconstructed, the second upsampling process is performed on the right boundary of the currently reconstructed image block according to the adjacent reconstructed image block C.
  • the upsampling processing method here is the same as the above-described upsampling processing method, and details are not described herein again.
  • the current reconstructed image block is the image block B
  • the DCTIF filter is currently used for the upsampling process
  • the current reconstructed image block B has been completed for the first time.
  • the upsampling process when using the DCTIF filter for upsampling processing as described above, requires four pixels above and below, then in the first upsampling process, the lower four rows of the currently reconstructed image block B ⁇ ( ⁇ indicates The four reference pixels required for the sample point after the first upsampling process are incomplete. For example, for the bottom row x, the lower four reference pixels required for each x do not exist.
  • the second upsampling process is performed on the lower boundary of the currently reconstructed image block according to the adjacent reconstructed image block C.
  • the upsampling processing method here is the same as the above-described upsampling processing method, and details are not described herein again.
  • FIG. 19 is a schematic diagram of image upsampling according to another embodiment of the present application. As shown in FIG. 19, this is the case of 8 neighborhood pixels, and the decoding end is on the right boundary of the currently reconstructed image block A and the current reconstructed image block B.
  • the upsampling processing method used in the lower boundary is similar to the method in the case of 4 neighborhood pixels, and will not be described here.
  • the current reconstructed image block may be identified as having completed the upsampling process.
  • the current reconstructed image block is subjected to upsampling processing according to a certain rule.
  • the upsampling process is based on the case of 4 neighborhood pixels
  • the current upsampling image block may be subjected to the second upsampling process.
  • the upsampling process is based on the case of 8 neighborhood pixels
  • the current upsampling image block may be subjected to a second upsampling process.
  • the processed reconstructed image block can be subjected to the second upsampling process.
  • the specific upsampling process is similar to the mode of the foregoing mode 3, and details are not described herein again.
  • the current reconstructed image block of the downsampling coding mode is saved, and the reference pixels are provided for subsequent prediction of the other image blocks to be reconstructed.
  • step S1302 is described in detail based on the above manners 1 and 2:
  • Step S1302 specifically includes the following four cases:
  • the encoding mode of the image block to be reconstructed is the original resolution encoding mode
  • the M reference reconstructed image blocks include the reference reconstructed image block whose encoding mode is the original resolution encoding mode
  • the encoding mode is the original resolution encoding.
  • the reference pixels of the current image block to be reconstructed are determined in the pixels of the reference reconstructed image block.
  • Case 2 If the encoding mode of the image block to be reconstructed is the original resolution encoding mode, and the M reference reconstructed image blocks include the reference reconstructed image block whose encoding mode is the down sampling encoding mode, the encoding mode is the down sampling encoding mode.
  • the pixels required for reconstructing the current image block to be reconstructed are obtained from the pixels of the reconstructed image block, and the pixels required for reconstructing the current image block to be reconstructed are subjected to upsampling processing to obtain a reference of the current image block to be reconstructed. Pixel.
  • Case 3 If the coding mode of the current image block to be reconstructed is a downsampling coding mode, and the M reference reconstructed image blocks include a reference reconstructed image block whose coding mode is a downsampling coding mode, the coding mode is a reference of the downsampling coding mode. A reference pixel of the current image block to be reconstructed is determined in the pixels of the reconstructed image block.
  • Case 4 If the coding mode of the current image block to be reconstructed is a downsampling coding mode, and the M reference reconstructed image blocks include a reference reconstructed image block whose coding mode is the original resolution coding mode, the coding mode is the original resolution coding mode.
  • the pixels required to reconstruct the current image block to be reconstructed are acquired in the pixels of the reference reconstructed image block, and the pixels required for reconstructing the current image block to be reconstructed are downsampled to obtain a reference of the current image block to be reconstructed. Pixel.
  • FIG. 20 is a schematic diagram of a current image block to be reconstructed and a reference reconstructed image block according to an embodiment of the present application.
  • the M reference reconstructed image blocks corresponding to the image block E to be reconstructed are reference reconstructed image blocks A, B, C, and D.
  • the image block E also needs to use an image.
  • the image block below the block A, but in the current coding order, the image block below the image block A has not been reconstructed, and the prior art realizes the prediction of the image block E by copying the image block A. This application does not elaborate on this.
  • the reference reconstructed image blocks A, B, C and D since their encoding methods are all original resolution encoding methods, and the current encoding mode of the image block E to be reconstructed is also the original resolution encoding mode, it can be reconstructed directly from the reference.
  • the reference pixels of the current image block to be reconstructed are determined in image blocks A, B, C, and D. For example, as shown in FIG. 20, the pixels in the frame are reference pixels.
  • FIG. 21 is a schematic diagram of a current image block to be reconstructed and a reference reconstructed image block according to an embodiment of the present application. As shown in FIG. 21, it is assumed that the reference pixel template shown in FIG. 7 is used in the embodiment of the present application.
  • the M reference reconstructed image blocks corresponding to the image block E to be reconstructed are reference reconstructed image blocks A, B, C, and D, wherein the current image to be reconstructed is encoded in the original resolution encoding mode, and the reference reconstructed image is used.
  • the coding mode of the block B is a downsampling coding mode.
  • the pixel of the lowermost row in the reference reconstructed image block B needs to be acquired, and then the acquired pixel is subjected to upsampling processing to obtain a reference pixel of the current reconstructed image block E.
  • the pixels in the frame are all reference pixels.
  • the reference pixels of the current reconstructed image block E and the like may be determined according to the plurality of rows of pixels in the reference reconstructed image block B, which is not limited in this application.
  • FIG. 22 is a schematic diagram of a current image block to be reconstructed and a reference reconstructed image block according to an embodiment of the present invention.
  • the M reference reconstructed image blocks corresponding to the image block E to be reconstructed are reference reconstructed image blocks A, B, C, and D, wherein the current image to be reconstructed is encoded in the original resolution encoding mode, and the reference reconstructed image is used.
  • the coding modes of blocks A, B and D are all original sampling coding modes.
  • the reference pixels of the current image block to be reconstructed can be directly determined from the reference reconstructed image blocks A, B, C and D.
  • the pixels in the frame are reference pixels.
  • FIG. 23 is a schematic diagram of a current image block to be reconstructed and a reference reconstructed image block according to an embodiment of the present invention. As shown in FIG. 23, it is assumed that the reference pixel template shown in FIG. 7 is used in the embodiment of the present application.
  • the M reference reconstructed image blocks corresponding to the image block E to be reconstructed are reference reconstructed image blocks A, B, C, and D, wherein the current image to be reconstructed is encoded in a downsampling mode, and the reference reconstructed image block is referenced.
  • the encoding mode of B is the original resolution encoding mode.
  • the pixels in the frame are all reference pixels.
  • the downsampling process and the upsampling process involved in the above four cases can be performed by the prior art method, which is not limited in this application.
  • any image block to be reconstructed using the downsampling coding mode after the reconstructed image block is reconstructed, the first upsampling process is performed, so any one The resolution of the reconstructed image block is the original resolution.
  • the step S1302 includes: if the coding mode of the image block to be reconstructed is the original resolution coding mode, determining the reference of the current image block to be reconstructed in the pixels of the M reference reconstructed image blocks. Pixel; if the encoding mode of the image block to be reconstructed is a downsampling coding mode, the pixels required for reconstructing the current image block to be reconstructed are acquired in the pixels of the M reference reconstructed image blocks, and the image block to be reconstructed is currently obtained.
  • the pixels required for reconstruction are subjected to downsampling processing to obtain reference pixels of the current image block to be reconstructed; or, if a neighboring reconstructed image block required for the current image block to be reconstructed is subjected to downsampling coding, the phase is saved. If the encoding mode of the image block to be reconstructed is the down sampling encoding mode, the reference pixel of the current image block to be reconstructed may be directly determined in the previously saved pixel.
  • the reference pixels of the currently reconstructed image block can be effectively determined by the above method, thereby realizing reconstruction of the currently reconstructed image block.
  • the above mainly introduces the image processing method at the decoding end.
  • the image processing method at the encoding end will be described below.
  • FIG. 24 is a flowchart of an image processing method according to another embodiment of the present application. As shown in FIG. 24, the method includes:
  • Step S2401 Obtain an encoding manner of a current image block to be encoded of a current image, and a pixel in each reference reconstructed image block corresponding to the current image block to be encoded;
  • Step S2402 determining, according to an encoding manner of the current image block to be encoded and pixels in the M reference reconstructed image blocks, a plurality of reference pixels of the current image block to be encoded;
  • Step S2403 Generate a prediction signal of a current image block to be encoded according to a plurality of reference pixels
  • Step S2404 Acquire an encoded signal of the current image block to be encoded, wherein when the encoding mode of the current image block to be encoded is the original resolution encoding mode, the encoded signal is the original signal of the current image block to be encoded, and the current image block to be encoded When the coding mode is the downsampling coding mode, the coded signal is a signal obtained by down-sampling the original signal of the current image block to be coded;
  • Step S2405 Generate a residual signal of the current image block to be encoded according to the prediction signal and the encoded signal;
  • Step S2406 Encoding the residual signal.
  • the encoding mode is the original resolution encoding mode or the down sampling encoding mode.
  • the current image block to be encoded corresponds to M reference reconstructed image blocks, and M is a positive integer greater than or equal to 1.
  • the reference reconstructed image block has the same resolution as the current image block to be reconstructed, at least one reference pixel is directly determined in the reference reconstructed image block; if the current image block to be reconstructed is the original resolution, the resolution of the reference reconstructed image block is determined.
  • Rate the downsampling resolution obtain at least one pixel required for reconstructing the current reconstructed image block from the reference reconstructed image block, and perform upsampling processing on at least one pixel required for reconstructing the current reconstructed image block
  • At least one pixel required for reconstruction of the image block to be encoded is currently subjected to downsampling processing to obtain at least one reference pixel of the current image block to be reconstructed.
  • the reference reconstructed image block is specifically related to which prediction image block and the prediction mode employed by the decoding end.
  • any of the above-described prediction modes in 35 reference may be made to the reference pixel template as shown in FIG.
  • the prediction signal of the current image block to be reconstructed is generated according to the plurality of reference pixels, and any one of the prediction modes in the above-mentioned 35 mode may be used. Of course, other prediction modes in the prior art may also be used, which is not limited in this application. .
  • a residual signal of the current image block to be encoded is generated according to the prediction signal and the encoded signal; encoding the residual signal includes: transforming and quantizing the residual signal. After transform quantization, the transform quantized coefficients are obtained, and the quantized coefficients and other indication information in the encoding are encoded by the entropy coding technique to obtain a code stream.
  • the present application considers that the image blocks to be coded have their own characteristics, and the encoding end uses different encoding methods for them. Based on this, the encoding end takes into account the factor of the encoding mode of the image block to be reconstructed. To encode the current image block to be encoded. Thereby the encoding effect of the encoding end is better.
  • the method further includes: generating a reconstructed signal of the current image block to be encoded, and reconstructing the current image block to be encoded according to the reconstructed signal to obtain a current reconstructed image block; if the encoding mode of the current reconstructed image block is a downsampling coding mode, The current reconstructed image block is subjected to upsampling processing based on the pixels of the adjacent reconstructed image block required for the up-sampling process of the current reconstructed image block.
  • the generating the reconstructed signal of the current image block to be encoded includes: transforming and quantizing the residual signal of the current image block to be encoded, obtaining a quantized coefficient of the current image block to be encoded, performing inverse quantization and inverse transform on the quantized coefficient, and reconstructing
  • the residual signal is added to the reconstructed residual signal and the predicted signal of the current image block to be encoded to obtain a reconstructed signal of the current image block to be encoded.
  • the current reconstructed image block is subjected to upsampling processing in order to obtain the current reconstructed image block of the original resolution, thereby preparing for the subsequent interframe prediction technique.
  • Method 1 The upsampling process is performed after all the image blocks of the current image are reconstructed; correspondingly, the current image block to be reconstructed is determined according to the coding mode of the current image block to be encoded and the pixels in the M reference reconstructed image blocks.
  • the plurality of reference pixels include: obtaining an encoding manner of each of the M reference reconstructed image blocks; encoding according to the current image block to be encoded, encoding manners of the M reference reconstructed image blocks, and M references The pixels in the image block are reconstructed to determine a plurality of reference pixels of the current image block to be encoded.
  • the upsampling process is performed after all the reconstructed image blocks required for the upsampling process of the current reconstructed image block are reconstructed; correspondingly, according to the coding mode of the current image block to be encoded and M reference reconstructions.
  • Determining, by a pixel in the image block, a plurality of reference pixels of the image block to be reconstructed comprising: acquiring an encoding mode of each of the reference reconstructed image blocks in the M reference reconstructed image blocks; and encoding according to the current image block to be encoded, M Referring to the coding mode of the reconstructed image block and the pixels in the M reference reconstructed image blocks, a plurality of reference pixels of the current image block to be encoded are determined.
  • the third method of performing the upsampling process on the current reconstructed image block includes: performing an upsampling process on the currently reconstructed image block according to a pixel of a part of the adjacent reconstructed image block that is currently reconstructed in the required adjacent reconstructed image block; If another part of the adjacent reconstructed image block of the currently uncompleted reconstructed image block in the adjacent reconstructed image block has been reconstructed, the partial boundary of the current reconstructed image block is subjected to secondary upsampling processing according to another partial adjacent reconstructed image block. Where a portion of the boundary of the currently reconstructed image block is contiguous with another portion of the adjacent reconstructed image block.
  • the method of performing the upsampling process on the current reconstructed image block includes: performing an upsampling process on the currently reconstructed image block according to the pixels of the part of the adjacent reconstructed image block that are currently reconstructed in the required adjacent reconstructed image block; If all the image blocks of the current image have been reconstructed, the partial boundary of the currently reconstructed image block is subjected to secondary upsampling according to another partially adjacent reconstructed image block in the desired adjacent reconstructed image block, wherein A portion of the adjacent reconstructed image block is an image block that is not reconstructed when the first upsampling process is performed on the current reconstructed image block; a partial boundary of the current reconstructed image block is adjacent to another partially adjacent reconstructed image block.
  • the part of the adjacent reconstructed image block is an upper image block and a left image block of the current reconstructed image block
  • the another partial adjacent reconstructed image block is a lower image block and a right side of the current reconstructed image block.
  • the part of the adjacent reconstructed image block is an upper left image block, an upper image block, an upper right image block and a left image block of the current reconstructed image block, and the other partially adjacent reconstructed image block is a right side of the current reconstructed image block.
  • a partial boundary of the current reconstructed image block is a right boundary and a lower boundary of the current reconstructed image block.
  • the required adjacent reconstructed image block includes an upper image block, a lower image block, a left image block, and a right image block of the current reconstructed image block; or, the desired adjacent reconstructed image block includes the current
  • the upper image block, the lower image block, the left image block, the right image block, the upper left image block, the lower left image block, the upper right image block, and the lower right image block of the image block are reconstructed.
  • step S2402 specifically includes the following four situations:
  • the encoding mode of the current image block to be encoded is the original resolution encoding mode
  • the M reference reconstructed image blocks include the reference reconstructed image block whose encoding mode is the original resolution encoding mode
  • the encoding mode is the original resolution encoding.
  • the reference pixels of the current image block to be encoded are determined in the pixels of the reference reconstructed image block.
  • Case 2 If the encoding mode of the current image block to be encoded is the original resolution encoding mode, and the M reference reconstructed image blocks include the reference reconstructed image block whose encoding mode is the down sampling encoding mode, the encoding mode is the down sampling encoding mode. Obtaining a pixel required for reconstructing the current reconstructed image block in a pixel of the reconstructed image block, and performing upsampling processing on a pixel required for reconstructing the current image block to be encoded to obtain a reference pixel of the current image block to be encoded .
  • Case 3 If the encoding mode of the current image block to be encoded is the downsampling coding mode, and the M reference reconstructed image blocks include the reference reconstructed image block whose coding mode is the downsampling coding mode, the coding mode is the reference of the downsampling coding mode.
  • a reference pixel of the current image block to be encoded is determined in the pixels of the reconstructed image block.
  • Case 4 If the encoding mode of the current image block to be encoded is a downsampling coding mode, and the M reference reconstructed image blocks include a reference reconstructed image block whose encoding mode is the original resolution encoding mode, the encoding mode is the original resolution encoding mode.
  • the pixels required for reconstructing the current image block to be encoded are acquired in the pixels of the reference reconstructed image block, and the pixels required for reconstructing the current image block to be encoded are subjected to down sampling processing to obtain a reference of the current image block to be encoded. Pixel.
  • the step S2402 includes: if the encoding mode of the current image block to be encoded is the original resolution encoding mode, determining the reference pixel of the current image block to be encoded in the pixels of the M reference reconstructed image blocks; If the encoding mode of the current image block to be encoded is the down sampling coding mode, the pixels required for reconstructing the current image block to be encoded are acquired in the pixels of the M reference reconstructed image blocks, and the current image block to be encoded is reconstructed. The required pixels are subjected to downsampling processing to obtain reference pixels of the current image block to be encoded.
  • the method further includes: identifying that the current reconstructed image block has completed the upsampling process.
  • the current reconstructed image block is subjected to upsampling processing according to a certain rule.
  • the upsampling process is based on the case of 4 neighborhood pixels
  • the upsampling process is based on the case of 8 neighborhood pixels
  • the upsampling process is based on the case of 8 neighborhood pixels
  • the upsampling process is based on the case of 8 neighborhood pixels, once the lower right image block of the currently reconstructed image block is reconstructed, the current reconstructed image block may be subjected to upsampling processing.
  • the acquiring the encoding mode of the current image block to be encoded in step S2401 includes: determining a first encoding cost when the current image block to be encoded adopts the original resolution encoding mode; determining when the current image block to be encoded adopts the down sampling encoding mode The second coding cost; the coding mode corresponding to the smaller coding cost of the first coding cost and the second coding cost is used as the coding mode of the current image block to be coded.
  • the coding end may use a Rate Distortion Optimization (RDO) method to calculate the coding cost of the current coded image block.
  • RDO Rate Distortion Optimization
  • the RDO method is an optimized method to improve video compression performance.
  • the principle is to optimize the video's lossy (picture quality) and bit rate (the amount of data required for encoding) to achieve an optimal balance point.
  • this algorithm was originally used in video compression encoders, it can also be used for various multimedia encodings including video, video, audio, etc., as long as the encoding takes into account both quality and file size.
  • the coding mode adopted by the image block to be coded is a coding mode with a small coding cost, thereby reducing the coding complexity of the coding end, thereby improving the coding efficiency of the coding end.
  • FIG. 25 is a schematic structural diagram of an image processing device according to an embodiment of the present disclosure.
  • the device includes: a parsing module 2501, configured to parse a code stream to obtain a current image block to be reconstructed of a current image. a coding mode, a residual signal of the current image block to be reconstructed, and a pixel in each reference reconstructed image block corresponding to the current image block to be reconstructed, wherein the coding mode is a primary resolution coding mode or a down sampling coding
  • the current image block to be reconstructed corresponds to M reference reconstructed image blocks, and M is a positive integer greater than or equal to 1.
  • the determining module 2502 is configured to use the encoding mode of the current image block to be reconstructed and the M reference reconstructed image blocks. a pixel that determines a plurality of reference pixels of the image block to be reconstructed; a generating module 2503, configured to generate a prediction signal of the current image block to be reconstructed according to the plurality of reference pixels; and a reconstruction module 2504, configured to generate the signal according to the prediction signal and the residual signal The reconstructed signal of the image block to be reconstructed is reconstructed, and the current image block to be reconstructed is reconstructed according to the reconstructed signal to obtain a current reconstructed image block.
  • the method further includes: a processing module 2505.
  • the processing module 2505 is configured to: if the encoding mode of the currently reconstructed image block is the down sampling encoding mode, the pixels of the adjacent reconstructed image block required for the upsampling process based on the current reconstructed image block, and the current reconstructed image block Perform upsampling processing.
  • the processing module 2505 is specifically configured to: perform the upsampling process after all image blocks of the current image are reconstructed.
  • the code stream includes: a coding mode of each of the M reference reconstructed image blocks; the determining module 2502 is specifically configured to: according to the coding mode of the current image block to be reconstructed, Determining a plurality of reference pixels of the current image block to be reconstructed by encoding the M reference reconstructed image blocks and pixels in the M reference reconstructed image blocks.
  • the processing module 2505 is specifically configured to perform the upsampling process after all the reconstructed image blocks required for performing the upsampling process of the current reconstructed image block are reconstructed.
  • the code stream includes: a coding mode of each of the M reference reconstructed image blocks; the determining module 2502 is specifically configured to: according to the coding mode of the current image block to be reconstructed, Determining a plurality of reference pixels of the current image block to be reconstructed by encoding the M reference reconstructed image blocks and pixels in the M reference reconstructed image blocks.
  • the determining module 2502 is specifically configured to: if the encoding mode of the image block to be reconstructed is the original resolution encoding mode, and the M reference reconstructed image blocks include the reference reconstructed image block whose encoding mode is the original resolution encoding mode, Determining a reference pixel of the current image block to be reconstructed in a pixel of the reference reconstructed image block whose encoding mode is the original resolution encoding mode; if the current encoding mode of the image block to be reconstructed is the original resolution encoding mode, and M reference reconstructed image blocks Include a reference reconstructed image block whose coding mode is a down-sampling coding mode, and obtain a pixel required for reconstructing the current image block to be reconstructed from the pixels of the reference reconstructed image block whose coding mode is the down-sampling coding mode, and wait for the current pixel
  • the pixels required for reconstructing the image block for performing reconstruction are subjected to upsampling processing to obtain reference
  • the pixels required for reconstructing the current image block to be reconstructed are acquired from the pixels of the reference reconstructed image block whose encoding mode is the original resolution encoding mode, and are required for reconstructing the current image block to be reconstructed.
  • the pixels are downsampled to obtain reference pixels of the current image block to be reconstructed.
  • the processing module 2505 is specifically configured to: perform an upsampling process on the current reconstructed image block according to a pixel of a part of the adjacent reconstructed image block that is currently reconstructed in the required adjacent reconstructed image block; If another part of the adjacent reconstructed image block of the current incomplete reconstruction in the required adjacent reconstructed image block has been reconstructed, the partial boundary of the current reconstructed image block is secondarily upsampled according to another partial adjacent reconstructed image block. Processing, wherein a partial boundary of the currently reconstructed image block is contiguous with the another portion of the adjacent reconstructed image block.
  • the processing module 2505 is specifically configured to: perform a once-sampling process on the currently reconstructed image block according to a pixel of a part of the adjacent reconstructed image block that is currently reconstructed in the required adjacent reconstructed image block; After all the image blocks have been reconstructed, the partial boundary of the current reconstructed image block is subjected to secondary upsampling according to another partially adjacent reconstructed image block in the desired adjacent reconstructed image block, wherein another part is adjacent
  • the reconstructed image block is an image block that is not reconstructed when the first upsampling process is performed on the currently reconstructed image block; a partial boundary of the current reconstructed image block is adjacent to another partially adjacent reconstructed image block.
  • the determining module 2502 is specifically configured to: if the encoding mode of the image block to be reconstructed is the original resolution encoding mode, determine the reference pixels of the current image block to be reconstructed in the pixels of the M reference reconstructed image blocks; If the encoding mode of the image block to be reconstructed is the down sampling encoding mode, acquiring pixels required for reconstructing the current image block to be reconstructed in the pixels of the M reference reconstructed image blocks, and The pixels required for reconstruction of the image block to be reconstructed are subjected to downsampling processing to obtain reference pixels of the current image block to be reconstructed.
  • the part of the adjacent reconstructed image block is an upper image block and a left image block of the current reconstructed image block
  • the another partial adjacent reconstructed image block is a lower image block and a right side of the current reconstructed image block.
  • the part of the adjacent reconstructed image block is an upper left image block, an upper image block, an upper right image block, and a left image block of the current reconstructed image block
  • the another partial adjacent reconstructed image block is the current reconstructed image block.
  • a partial boundary of the current reconstructed image block is a right boundary and a lower boundary of the current reconstructed image block.
  • the required adjacent reconstructed image block includes an upper image block, a lower image block, a left image block, and a right image block of the current reconstructed image block; or the desired adjacent reconstructed image block includes the current reconstructed image.
  • the processing module 2505 is further configured to: after performing the upsampling process on the current reconstructed image block, identify that the current reconstructed image block has completed the upsampling process.
  • the image processing device provided by the present application can perform the image processing method corresponding to FIG. 13 and the optional manner of the method, and the implementation principle and technical effects are similar, and details are not described herein again.
  • FIG. 26 is a schematic structural diagram of an image processing device according to another embodiment of the present disclosure. As shown in FIG. 26, the device includes:
  • the obtaining module 2601 is configured to acquire an encoding manner of a current image block to be encoded of a current image, and a pixel in each reference reconstructed image block corresponding to the current image block to be encoded, where the encoding mode is a primary resolution encoding manner. Or a downsampling coding mode, where the current image block to be encoded corresponds to M reference reconstructed image blocks, and M is a positive integer greater than or equal to 1.
  • the determining module 2602 is configured to determine, according to the encoding manner of the current image block to be encoded and the pixels in the M reference reconstructed image blocks, a plurality of reference pixels of the current image block to be encoded.
  • the generating module 2603 is configured to generate, according to the plurality of reference pixels, a prediction signal of the current image block to be encoded.
  • the acquiring module 2601 is further configured to acquire an encoded signal of the current image block to be encoded, where the encoded signal is a device when the encoding mode of the current image block to be encoded is the original resolution encoding mode.
  • the original signal of the current image block to be encoded when the encoding mode of the current image block to be encoded is the down sampling encoding mode, the encoded signal is after the original signal of the current image block to be encoded is subjected to down sampling processing. The signal obtained.
  • the generating module 2603 is further configured to generate a residual signal of the current image block to be encoded according to the prediction signal and the encoded signal.
  • the encoding module 2604 is configured to encode the residual signal.
  • a processing module 2605 is also included.
  • the generating module 2603 is further configured to: generate a reconstructed signal of the current image block to be encoded, and reconstruct the current image block to be encoded according to the reconstructed signal to obtain a current reconstructed image block; and the processing module 2605 is configured to perform current reconstruction.
  • the coding mode of the image block is the downsampling coding mode, and the current reconstructed image block is subjected to upsampling processing based on the pixels of the adjacent reconstructed image block required for performing the upsampling process on the current reconstructed image block.
  • the processing module 2605 is specifically configured to: after the image blocks of the current image are all reconstructed, perform the upsampling process; correspondingly, the determining module 2602 is specifically configured to: acquire the M Refers to an encoding manner of each reference reconstructed image block in the reconstructed image block; according to the encoding manner of the current image block to be encoded, the encoding manner of the M reference reconstructed image blocks, and the M reference reconstructed image blocks a pixel, determining a plurality of reference pixels of the current image block to be encoded.
  • the processing module 2605 is specifically configured to perform the upsampling process after all the reconstructed image blocks required for performing the upsampling process on the current reconstructed image block are reconstructed.
  • the determining module 2602 is specifically configured to: if the encoding mode of the current image block to be encoded is the original resolution encoding mode, and the M reference reconstructed image blocks include the reference reconstructed image block whose encoding mode is the original resolution encoding mode, Determining a reference pixel of the current image block to be encoded in a pixel of the reference reconstructed image block whose encoding mode is the original resolution encoding mode; if the current encoding mode of the image block to be encoded is the original resolution encoding mode, and M reference reconstructed image blocks Include a reference reconstructed image block whose coding mode is a down-sampling coding mode, and obtain, from a pixel of a reference reconstructed image block whose coding mode is a down-sampling coding mode, a pixel required for reconstructing the current reconstructed image block, and the current to be encoded
  • the pixels required for reconstruction of the image block are subjected to upsampling processing to obtain reference pixels
  • the processing module 2605 is specifically configured to: perform an upsampling process on the currently reconstructed image block according to a pixel of a part of the adjacent reconstructed image block that is currently reconstructed in the required adjacent reconstructed image block; If another part of the adjacent reconstructed image block of the current incomplete reconstruction in the required adjacent reconstructed image block has been reconstructed, the partial boundary of the current reconstructed image block is subjected to the second upsampling process according to another partial adjacent reconstructed image block. Wherein, a partial boundary of the currently reconstructed image block is adjacent to another partially adjacent reconstructed image block.
  • the processing module 2605 is specifically configured to: perform an upsampling process on the currently reconstructed image block according to a pixel of a part of the adjacent reconstructed image block that is currently reconstructed in the required adjacent reconstructed image block; If all the image blocks of the current image have been reconstructed, then the partial boundary of the current reconstructed image block is subjected to secondary upsampling according to another partially adjacent reconstructed image block in the desired adjacent reconstructed image block, wherein the other portion
  • the adjacent reconstructed image block is an image block that is not reconstructed when the first upsampling process is performed on the current reconstructed image block; a partial boundary of the current reconstructed image block is adjacent to another partially adjacent reconstructed image block.
  • the determining module 2502 is specifically configured to: if the encoding mode of the current image block to be encoded is the original resolution encoding mode, determine the reference pixels of the current image block to be encoded in the pixels of the M reference reconstructed image blocks; If the encoding mode of the current image block to be encoded is the down sampling coding mode, acquiring pixels required for reconstructing the current image block to be encoded in the pixels of the M reference reconstructed image blocks, and The pixels required for reconstructing the current image block to be encoded are subjected to down sampling processing to obtain reference pixels of the current image block to be encoded.
  • the part of the adjacent reconstructed image block is an upper image block and a left image block of the current reconstructed image block
  • the another partial adjacent reconstructed image block is a lower image block and a right side of the current reconstructed image block.
  • the part of the adjacent reconstructed image block is an upper left image block, an upper image block, an upper right image block and a left image block of the current reconstructed image block, and the other partially adjacent reconstructed image block is a right side of the current reconstructed image block.
  • a partial boundary of the current reconstructed image block is a right boundary and a lower boundary of the current reconstructed image block.
  • the required adjacent reconstructed image block includes an upper image block, a lower image block, a left image block, and a right image block of the current reconstructed image block; or the desired adjacent reconstructed image block includes the current
  • the upper image block, the lower image block, the left image block, the right image block, the upper left image block, the lower left image block, the upper right image block, and the lower right image block of the image block are reconstructed.
  • the processing module 2605 is further configured to: after performing the upsampling process on the current reconstructed image block, identify that the current reconstructed image block has completed the upsampling process.
  • the acquiring module 2601 is specifically configured to: determine a first encoding cost when the current image block to be encoded adopts the original resolution encoding mode; and determine a second encoding cost when the current image block to be encoded adopts the down sampling encoding mode; The coding mode corresponding to the smaller coding cost of the first coding cost and the second coding cost is used as the coding mode of the current image block to be encoded.
  • the image processing device provided by the present application can perform the image processing method corresponding to FIG. 24 and the optional manner of the method, and the implementation principle and technical effects are similar, and details are not described herein again.
  • FIG. 27 is a schematic structural diagram of an image processing system provided by the present application. As shown in FIG. 27, the system includes: an image processing device 2701 of the above decoding end, and an image processing device 2702 of the encoding end.
  • the present application provides an image processing apparatus including: a processor and a memory for storing executable instructions of the processor; wherein the processor can perform the image processing method corresponding to FIG. 13 and an optional manner of the method .
  • the implementation principle and technical effect are similar, and will not be described here.
  • the present application provides an image processing apparatus including: a processor and a memory for storing executable instructions of the processor; wherein the processor can execute the image processing method corresponding to FIG. 24 and an optional manner of the method .
  • the implementation principle and technical effect are similar, and will not be described here.
  • the image processing device of the decoding end of the image processing system provided by the present application may perform the image processing method corresponding to FIG. 13 and an optional manner of the method, and the image processing device of the encoding end may perform the image processing method corresponding to FIG. 24 described above and The optional method and the technical effect of the method are similar, and are not described here.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请提供一种图像处理方法、设备及系统，包括解析码流，以获取当前图像的当前待重建图像块的编码方式、当前待重建图像块的残差信号、当前待重建图像块对应的每个参考重建图像块中的像素，编码方式为原分辨率编码方式或下采样编码方式，当前待重建图像块对应M个参考重建图像块；根据当前待重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素；根据多个参考像素生成预测信号；根据预测信号和残差信号生成重建信号，根据重建信号重建当前待重建图像块，得到当前重建图像块。从而使得解码端得到的重建图像块效果更佳。

Description

图像处理方法、设备及系统
本申请要求于2017年7月13日提交中国专利局、申请号为201710572139.7、申请名称为“图像处理方法、设备及系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术，尤其涉及一种图像处理方法、设备及系统。
背景技术
数字视频是以数字形式记录的视频。图1为本申请提供的数字视频的示意图，如图1所示，数字视频由多帧数字图像组成。图2为本申请提供的数字图像的示意图，如图2所示，图像由12*16个像素组成，其中，每个像素被称为一个像素，12*16表示图像分辨率。例如2K视频的图像分辨率是1920*1080，4K视频的图像分辨率是3840*2160。通常原始视频包括的数据量较大，不适合存储和传输，需要利用高效的视频压缩编码技术来压缩原始数据。
具体地，图3为本申请提供的编码端的编码示意图，如图3所示，编码端的编码流程包括：编码端接收到视频之后，对于构成视频的每帧图像，将该图像划分成多个待编码图像块。对于当前待编码图像块，首先通过参考重建图像块(该参考重建图像块用于提供当前待编码图像块所需的参考像素，该参考像素用于对当前待编码图像块进行预测)对当前待编码图像块进行预测，得到当前待编码图像块的预测信号；用当前待编码图像块的原始信号减去预测信号，得到残差信号。经过预测后，残差信号的幅值远小于原始信号。将残差信号进行变换和量化操作。经过变换量化后，得到变换量化系数，再通过熵编码技术编码量化系数以及编码中的其他指示信息，得到码流。进一步地，编码端还需要重建当前待编码图像块，以实现对后续待编码图像块的编码提供参考像素。具体地，在得到当前待编码图像块的变换量化系数之后，编码端需要对当前待编码图像块的变换量化系数进行反量化和反变换，得到重建的残差信号，将重建的残差信号与当前待编码图像块对应的预测信号相加，得到当前待编码图像块的重建信号，根据该重建信号得到重建图像块。其中，该重建图像块可以对后续待编码图像块进行预测。可选地，残差信号经过变换后得到变换系数，变换系数通过量化后会有信息损失，该信息损失不可逆。即经过反量化后的变换系数会有失真，从而使得重建信号与原始信号不一致，这种压缩方式为有损压缩。因此，对于有损压缩，在得到重建图像块后，需要对该重建图像块进行滤波，从而去除有损压缩引入的一些失真，例如块效应，振铃效应等。为了去除块效应，可以使用H.264、H.265标准中的DBK滤波器。为了去除振铃效应，可以使用H.265中的SAO滤波器，以及下一代标准中的ALF滤波器等。也有无损压缩方法，即残差信号使用无损的变换操作得到变换系数，不进行量化操作，将变换系数进行熵编。对于无损压缩，一般不再进行滤波操作。进一步地，当前图像的各个图像块都完成重建之后，得到重建图像，其中，该重建图像可以对后续其他帧图像进行预测。
图4为本申请提供的解码端的解码示意图，如图4所示，解码端获取到码流之后，首先对码流进行熵解码，得到当前待重建图像块的变换量化系数，然后对变换量化系数进行反量化和反变换，得到当前待重建图像块的重建的残差信号。通过它的参考重建图像块对当前待重建图像块进行预测，得到当前待重建图像块的预测信号，然后将预测信号和上述重建的残差信号相加，得到当前待重建图像块的重建信号，然后根据该重建信号得到当前待重建图像块对应的当前重建图像块，其中，该当前重建图像块可以对后续其他待重建图像块进行预测。类似于上述编码端的情况，可选地，在解码端需要对当前重建图像块进行滤波。进一步地，当前图像的各个图像块都完成重建之后，得到重建图像，其中，该重建图像可以对后续其他帧图像进行预测。
为了降低编解码复杂度，编码端对每帧图像先进行下采样处理，图5为本申请提供的编码端的编码示意图，如图5所示，编码端对整幅图像采用下采样处理，然后对经过下采样处理后的图像中的每个待编码图像块进行编码，得到码流。其中，每个待编码图像块对应的重建图像块的分辨率为下采样分辨率。相应的，解码端解析码流，每个待重建图像块的分辨率均为下采样分辨率，得到对应的重建图像块的分辨率也为下采样分辨率，解码端需要对重建图像块采用上采样处理，以得到原始分辨率的重建图像块。
现有技术中，编码端对整幅图像都采用下采样处理，然而，整幅图像包括的各个图像块的特性可能不同，例如，有些图像块可能较为平坦，编码端适合对其采用下采样处理；有些图像块可能细节比较多，这种情况下，下采样处理会损失掉这些细节。这将导致编码端的编码效果不佳。相应的，这将导致解码端得到的有些重建图像块比较模糊，即解码端得到的重建图像块效果不佳。
发明内容
本申请提供一种图像处理方法、设备及系统，从而使得解码端得到的重建图像块效果更佳，同时编码端的编码效果更好。
第一方面，本申请提供一种图像处理方法，包括：解析码流，以获取当前图像的当前待重建图像块的编码方式、当前待重建图像块的残差信号、当前待重建图像块对应的每个参考重建图像块中的像素，其中，编码方式为原分辨率编码方式或者下采样编码方式，当前待重建图像块对应M个参考重建图像块，M为大于或者等于1的正整数；根据当前待重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素；根据多个参考像素生成当前待重建图像块的预测信号；根据预测信号和残差信号生成当前待重建图像块的重建信号，并根据重建信号重建所述当前待重建图像块，得到当前重建图像块。
本申请实施例的有益效果是：本申请考虑到图像块具有各自的特性，编码端对它们所采用的编码方式也不尽相同，基于此，解码端将当前待重建图像块的编码方式和M个参考重建图像块的编码方式两个因素考虑在内，以重建当前待重建图像块。从而使得解码端得到的重建图像块效果更佳。
可选地，若当前重建图像块的编码方式为下采样编码方式，则方法还包括：基于当前重建图像块进行上采样处理时所需的相邻重建图像块的像素，对当前重建图像块进行上采样处理。
可选方式一，上采样处理是在当前图像的所有图像块都重建完成后进行的；相应的， 码流包括：M个参考重建图像块中的每个参考重建图像块的编码方式；根据当前待重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素，包括：根据当前待重建图像块的编码方式、M个参考重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素。
可选方式二，上采样处理是在当前重建图像块进行上采样处理时所需的所有相邻重建图像块都重建完成后进行的；相应的，码流包括：M个参考重建图像块中的每个参考重建图像块的编码方式；根据当前待重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素，包括：根据当前待重建图像块的编码方式、M个参考重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素。
可选方式三，对当前重建图像块进行上采样处理包括：根据所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对当前重建图像块进行一次上采样处理；若所需的相邻重建图像块中的当前未完成重建的另一部分相邻重建图像块已完成重建，则根据另一部分相邻重建图像块对当前重建图像块的部分边界进行二次上采样处理，其中，当前重建图像块的部分边界与另一部分相邻重建图像块邻接。
可选方式四，对当前重建图像块进行上采样处理包括：根据所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对当前重建图像块进行一次上采样处理；若当前图像的所有图像块均已完成重建，则根据在所需的相邻重建图像块中的另一部分相邻重建图像块对所述当前重建图像块的部分边界进行二次上采样处理，其中，另一部分相邻重建图像块是在对当前重建图像块进行第一次上采样处理时未完成重建的图像块；当前重建图像块的部分边界与另一部分相邻重建图像块邻接。
综上，通过上述四种方式对当前重建图像块进行上采样处理时，当前重建图像块的部分边界都是通过所需的相邻重建图像块进行上采样处理的，而现有技术中，当前重建图像块的部分边界都是通过复制当前重建图像块的像素进行上采样处理的，因此，本申请提供的方法可以避免当前重建图像块边界不连续的问题。
可选地，一部分相邻重建图像块为当前重建图像块的上边图像块和左边图像块，另一部分相邻重建图像块为当前重建图像块的下边图像块和右边图像块；或者，一部分相邻重建图像块为当前重建图像块的左上图像块，上边图像块，右上图像块和左边图像块，另一部分相邻重建图像块为当前重建图像块的右边图像块，左下图像块，下边图像块和右下图像块。
可选地，当前重建图像块的部分边界为当前重建图像块的右边界和下边界。
可选地，所需的相邻重建图像块包括当前重建图像块的上边图像块，下边图像块，左边图像块和右边图像块；或者，所需的相邻重建图像块包括当前重建图像块的上边图像块，下边图像块，左边图像块，右边图像块，左上图像块，左下图像块，右上图像块和右下图像块。
可选地，对当前重建图像块进行上采样处理后，方法还包括：标识当前重建图像块已完成上采样处理。从而可以避免对当前重建图像块的重复上采样。
可选地，针对上述方式一和方式二，根据当前待重建图像块的编码方式、M个参考重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素，包括：若当前待重建图像块的编码方式为原分辨率编码方式，且M个参考重 建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则在编码方式为原分辨率编码方式的参考重建图像块的像素中确定当前待重建图像块的参考像素；若当前待重建图像块的编码方式为原分辨率编码方式，且M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则从编码方式为下采样编码方式的参考重建图像块的像素中获取对当前待重建图像块进行重建时所需的像素，并对当前待重建图像块进行重建时所需的像素进行上采样处理，以得到当前待重建图像块的参考像素；若当前待重建图像块的编码方式为下采样编码方式，且M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则在编码方式为下采样编码方式的参考重建图像块的像素中确定当前待重建图像块的参考像素；若当前待重建图像块的编码方式为下采样编码方式，且M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则从编码方式为原分辨率编码方式的参考重建图像块的像素中获取对当前待重建图像块进行重建时所需的像素，对当前待重建图像块进行重建时所需的像素进行下采样处理，以得到当前待重建图像块的参考像素。
可选地，针对上述方式三和方式四，根据当前待重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素，包括：若当前待重建图像块的编码方式为原分辨率编码方式，则在M个参考重建图像块的像素中获取对当前待重建图像块进行重建时所需的像素，并在M个参考重建图像块中确定当前待重建图像块的参考像素；若当前待重建图像块的编码方式为下采样编码方式，则对当前待重建图像块进行重建时所需的像素进行下采样处理，以得到当前待重建图像块的参考像素。
通过上述这两种可选方法可以有效的确定当前重建图像块的参考像素，进而实现对当前重建图像块的重建。
第二方面，本申请提供一种图像处理方法，包括：获取当前图像的当前待编码图像块的编码方式、当前待编码图像块对应的每个参考重建图像块中的像素，其中，编码方式为原分辨率编码方式或者下采样编码方式，当前待编码图像块对应M个参考重建图像块，M为大于或者等于1的正整数；根据当前待编码图像块的编码方式和M个参考重建图像块中的像素，确定当前待编码图像块的多个参考像素；根据多个参考像素生成当前待编码图像块的预测信号；获取当前待编码图像块的编码信号，其中，当当前待编码图像块的编码方式是原分辨率编码方式时，编码信号为当前待编码图像块的原始信号，当当前待编码图像块的编码方式是下采样编码方式时，编码信号为当前待编码图像块的原始信号经过下采样处理后得到的信号；根据预测信号和编码信号生成当前待编码图像块的残差信号；对残差信号对进行编码。
本申请实施例的有益效果是：本申请考虑到待编码图像块具有各自的特性，编码端对它们所采用的编码方式也不尽相同，基于此，编码端将当前待重建图像块的编码方式和M个参考重建图像块的编码方式两个因素考虑在内，以对当前待编码图像块进行编码。从而使得编码端的编码效果更佳。
可选地，还包括：生成当前待编码图像块的重建信号，并根据重建信号重建当前待编码图像块，得到当前重建图像块；若当前重建图像的编码方式为下采样编码方式，则基于当前重建图像块进行上采样处理时所需的相邻重建图像块的像素，对当前重建图像块进行上采样处理。
可选方式一，上采样处理是在当前图像的所有图像块都重建完成后进行的；相应的， 根据当前待编码图像块的编码方式和所述M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素，包括：获取M个参考重建图像块中的每个参考重建图像块的编码方式；根据当前待编码图像块的编码方式、M个参考重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待编码图像块的多个参考像素。
可选方式二，上采样处理是在当前重建图像块进行上采样处理时所需的所有相邻重建图像块都重建完成后进行的；相应的，根据当前待编码图像块的编码方式和所述M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素，包括：获取M个参考重建图像块中的每个参考重建图像块的编码方式；根据当前待编码图像块的编码方式、M个参考重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待编码图像块的多个参考像素。
可选方式三，对当前重建图像块进行上采样处理包括：根据所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对当前重建图像块进行一次上采样处理；若所需的相邻重建图像块中的当前未完成重建的另一部分相邻重建图像块已完成重建，则根据另一部分相邻重建图像块对当前重建图像块的部分边界进行二次上采样处理，其中，当前重建图像块的部分边界与另一部分相邻重建图像块邻接。
可选方式四，对当前重建图像块进行上采样处理包括：根据所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对当前重建图像块进行一次上采样处理；若当前图像的所有图像块均已完成重建，则根据在所需的相邻重建图像块中的另一部分相邻重建图像块对所述当前重建图像块的部分边界进行二次上采样处理，其中，另一部分相邻重建图像块是在对当前重建图像块进行第一次上采样处理时未完成重建的图像块；当前重建图像块的部分边界与另一部分相邻重建图像块邻接。
综上，通过上述四种方式对当前重建图像块进行上采样处理时，当前重建图像块的部分边界都是通过所需的相邻重建图像块进行上采样处理的，而现有技术中，当前重建图像块的部分边界都是通过复制当前重建图像块的像素进行上采样处理的，因此，本申请提供的方法可以避免当前重建图像块边界不连续的问题。
可选地，一部分相邻重建图像块为当前重建图像块的上边图像块和左边图像块，另一部分相邻重建图像块为当前重建图像块的下边图像块和右边图像块；或者，一部分相邻重建图像块为当前重建图像块的左上图像块，上边图像块，右上图像块和左边图像块，另一部分相邻重建图像块为当前重建图像块的右边图像块，左下图像块，下边图像块和右下图像块。
可选地，当前重建图像块的部分边界为当前重建图像块的右边界和下边界。
可选地，所需的相邻重建图像块包括当前重建图像块的上边图像块，下边图像块，左边图像块和右边图像块；或者，所需的相邻重建图像块包括当前重建图像块的上边图像块，下边图像块，左边图像块，右边图像块，左上图像块，左下图像块，右上图像块和右下图像块。
可选地，对当前重建图像块进行上采样处理后，方法还包括：标识当前重建图像块已完成上采样处理。从而可以避免对当前重建图像块的重复上采样。
可选地，针对上述方式一和方式二，根据当前待编码图像块的编码方式、M个参考重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待编码图像块的多个参考像素，包括：若当前待编码图像块的编码方式为原分辨率编码方式，且M个参考重 建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则在编码方式为原分辨率编码方式的参考重建图像块的像素中确定当前待编码图像块的参考像素；若当前待编码图像块的编码方式为原分辨率编码方式，且M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则从编码方式为下采样编码方式的参考重建图像块的像素中获取对当前重建图像块进行重建时所需的像素，并对当前待编码图像块进行重建时所需的像素进行上采样处理，以得到当前待编码图像块的参考像素；若当前待编码图像块的编码方式为下采样编码方式，且M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则在编码方式为下采样编码方式的参考重建图像块的像素中确定当前待编码图像块的参考像素；若当前待编码图像块的编码方式为下采样编码方式，且M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则从编码方式为原分辨率编码方式的参考重建图像块的像素中获取对当前待编码图像块进行重建时所需的像素，对当前待编码图像块进行重建时所需的像素进行下采样处理，以得到当前待编码图像块的参考像素。
可选地，针对上述方式三和方式四，根据当前待编码图像块的编码方式和M个参考重建图像块中的像素，确定当前待编码图像块的多个参考像素，包括：若当前待编码图像块的编码方式为原分辨率编码方式，则在M个参考重建图像块的像素中确定当前待编码图像块的参考像素；若当前待编码图像块的编码方式为下采样编码方式，则在M个参考重建图像块的像素中获取对当前待编码图像块进行重建时所需的像素，并对当前待编码图像块进行重建时所需的像素进行下采样处理，以得到当前待编码图像块的参考像素。
通过上述两种可选方法可以有效的确定当前重建图像块的参考像素，进而实现对当前重建图像块的重建。
可选地，获取当前待编码图像块的编码方式，包括：确定当前待编码图像块采用原分辨率编码方式时的第一编码代价；确定当前待编码图像块采用下采样编码方式时的第二编码代价；将第一编码代价和第二编码代价中较小的编码代价对应的编码方式作为当前待编码图像块的编码方式。
本申请中，待编码图像块采用的编码方式为编码代价较小的编码方式，从而降低编码端的编码复杂度，进而提高编码端的编码效率。
下面对图像处理设备及系统进行介绍，其实现原理和技术效果与上述原理和技术效果类似，此处不再赘述。
第三方面，本申请提供一种图像处理设备，包括：解析模块，用于解析码流，以获取当前图像的当前待重建图像块的编码方式、当前待重建图像块的残差信号、当前待重建图像块对应的每个参考重建图像块中的像素，其中，编码方式为原分辨率编码方式或者下采样编码方式，当前待重建图像块对应M个参考重建图像块，M为大于或者等于1的正整数；确定模块，用于根据当前待重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素；生成模块，用于根据多个参考像素生成所述当前待重建图像块的预测信号；重建模块，用于根据预测信号和残差信号生成当前待重建图像块的重建信号，并根据重建信号重建当前待重建图像块，得到当前重建图像块。
第四方面，本申请提供一种图像处理设备，包括：获取模块，用于获取当前图像的当前待编码图像块的编码方式和当前待编码图像块对应的每个参考重建图像块中的像素， 其中，编码方式为原分辨率编码方式或者下采样编码方式，当前待编码图像块对应M个参考重建图像块，M为大于或者等于1的正整数；确定模块，用于根据当前待编码图像块的编码方式和M个参考重建图像块中的像素，确定当前待编码图像块的多个参考像素；生成模块，用于根据多个参考像素生成当前待编码图像块的预测信号；获取模块，还用于获取当前待编码图像块的编码信号，其中，当当前待编码图像块的编码方式是原分辨率编码方式时，编码信号为当前待编码图像块的原始信号，当当前待编码图像块的编码方式是下采样编码方式时，编码信号为当前待编码图像块的原始信号经过下采样处理后得到的信号；生成模块，还用于根据预测信号和编码信号生成当前待编码图像块的残差信号；编码模块，用于对残差信号进行编码。
第五方面，本申请提供一种图像处理系统，包括：如第三方面所述的图像处理设备，以及如第四方面所述的图像处理设备。
第六方面，本申请提供一种图像处理设备，该设备包括经配置以进行以下操作的解码器：
解析码流，以获取当前图像的当前待重建图像块的编码方式、当前待重建图像块的残差信号、当前待重建图像块对应的每个参考重建图像块中的像素，其中，编码方式为原分辨率编码方式或者下采样编码方式，当前待重建图像块对应M个参考重建图像块，M为大于或者等于1的正整数；根据当前待重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素；根据多个参考像素生成当前待重建图像块的预测信号；根据预测信号和残差信号生成当前待重建图像块的重建信号，并根据重建信号重建所述当前待重建图像块，得到当前重建图像块。
第七方面，本申请提供一种图像处理设备，该设备包括经配置以进行以下操作的编码器：
获取当前图像的当前待编码图像块的编码方式、当前待编码图像块对应的每个参考重建图像块中的像素，其中，编码方式为原分辨率编码方式或者下采样编码方式，当前待编码图像块对应M个参考重建图像块，M为大于或者等于1的正整数；根据当前待编码图像块的编码方式和M个参考重建图像块中的像素，确定当前待编码图像块的多个参考像素；根据多个参考像素生成当前待编码图像块的预测信号；获取当前待编码图像块的编码信号，其中，当当前待编码图像块的编码方式是原分辨率编码方式时，编码信号为当前待编码图像块的原始信号，当当前待编码图像块的编码方式是下采样编码方式时，编码信号为当前待编码图像块的原始信号经过下采样处理后得到的信号；根据预测信号和编码信号生成当前待编码图像块的残差信号；对残差信号进行编码。
第八方面，本申请提供一种计算机存储介质，用于储存为上述第三方面或者第六方面涉及的图像处理设备所用的计算机软件指令，其包含用于执行上述第三方面或者第六方面所设计的程序。
第九方面，本申请提供一种计算机程序产品，其包含指令，当所述计算机程序被计算机所执行时，该指令使得计算机执行第三方面或者第六方面中图像处理设备所执行的功能。
第十方面，本申请提供一种计算机存储介质，用于储存为上述第四方面或者第七方面涉及的图像处理设备所用的计算机软件指令，其包含用于执行上述第四方面或者第七方面所设计的程序。
第十一方面，本申请提供一种计算机程序产品，其包含指令，当所述计算机程序被计算机所执行时，该指令使得计算机执行第四方面或者第七方面中图像处理设备所执行的功能。
本申请提供一种图像处理方法、设备及系统，其中由于本申请考虑到图像块具有各自的特性，编码端对它们所采用的编码方式也不尽相同，基于此，解码端将当前待重建图像块的编码方式和M个参考重建图像块的编码方式两个因素考虑在内，以重建当前待重建图像块。从而使得解码端得到的重建图像块效果更佳。同样的，编码端将当前待重建图像块的编码方式和M个参考重建图像块的编码方式两个因素考虑在内，以对当前待编码图像块进行编码。从而使得编码端的编码效果更佳。
附图说明
图1为本申请提供的数字视频的示意图；
图2为本申请提供的数字图像的示意图；
图3为本申请提供的编码端的编码示意图；
图4为本申请提供的解码端的解码示意图；
图5为本申请提供的编码端的编码示意图；
图6为本申请一实施例提供的正在编码的图像的示意图；
图7为本申请一实施例提供的参考像素模板的示意图；
图8A和图8B为本申请一实施例提供的Planar模式的示意图；
图9为本申请一实施例提供的33种角度预测模式的具体方向示意图；
图10为本申请一实施例提供的图像下采样示意图；
图11为本申请一实施例提供的图像上采样示意图；
图12为本申请一实施例提供的图像上采样示意图；
图13为本申请一实施例提供的一种图像处理方法的流程图；
图14为本申请一实施例提供的4邻域像素的示意图；
图15为本申请一实施例提供的8邻域像素的示意图；
图16为本申请一实施例提供的图像上采样示意图；
图17为本申请另一实施例提供的图像上采样示意图；
图18为本申请一实施例提供的图像上采样示意图；
图19为本申请另一实施例提供的图像上采样示意图；
图20为本申请一实施例提供的当前待重建图像块与参考重建图像块的示意图；
图21为本申请一实施例提供的当前待重建图像块与参考重建图像块的示意图；
图22为本申请一实施例提供的当前待重建图像块与参考重建图像块的示意图；
图23为本申请一实施例提供的当前待重建图像块与参考重建图像块的示意图；
图24为本申请另一实施例提供的一种图像处理方法的流程图；
图25为本申请一实施例提供的一种图像处理设备的结构示意图；
图26为本申请另一实施例提供的一种图像处理设备的结构示意图；
图27为本申请提供的一种图像处理系统的结构示意图。
具体实施方式
以下，对本申请中的部分专业用语进行解释说明，以便于本领域技术人员理解。
数字视频是以数字形式记录的视频。数字视频由多帧数字图像组成。通常原始视频包括的数据量较大，不适合存储和传输，需要利用高效的视频压缩编码技术来压缩原始数据。
视频压缩技术是通过消除视频冗余来达到压缩的目的。视频冗余主要包括如下几项：空间冗余，时间冗余，视觉冗余和信息熵冗余。
空间冗余：它是静态图像存在的最主要的数据冗余。它是指在一幅图像中，相邻的像素的幅值都比较相近，这种空间连贯性被称为空间相关或空间冗余。空间冗余主要是通过帧内预测方法来消除，帧内预测方法是指利用视频空间域的相关性，使用参考重建图像块的像素预测当前图像块的像素，以达到去除视频空间冗余的目的。
时间冗余：它是视频序列中经常包含的冗余，由于视频的相邻图像往往包含相同或类似的背景和运动物体，只是运动物体所在的空间位置略有不同，这种相邻图像间的数据的高度相关性就称为时间冗余。时间冗余主要是通过帧间预测技术来消除，帧间预测技术是指利用时间上相邻图像的像素来预测当前像素。
视觉冗余：人眼视觉系统对图像细节的变化不敏感，这些细微变化信息即使丢失，人眼也感受不到。而在记录原始视频数据时，通常假定视觉系统是对各种内容的敏感度是一致的，这样就产生了比理想编码更多的数据，称为视觉冗余。视觉冗余主要是通过变换，量化技术来消除，该变换技术是指将图像信号变换到频率域进行处理，根据不同频率信号对视觉质量的贡献大小进行数据表达和比特再分配，这样可以纠正空间域上均匀采样的不合理表达。同时在比特再分配过程中融合考虑去除视觉冗余的需要，通过量化操作，省略过分精细的高频分量表达，实现有效压缩。
信息熵冗余：由信息论可知，为表示图像数据的一个像素，只要按其信息熵的大小分配相应比特数即可，而对于图像数据的每个像素，在图像获取时很难得到它的信息熵，因此一般是对每个像素采用相同的比特数来表示，这样必然存在冗余。信息熵冗余主要是通过熵编码技术来消除熵编码技术是通过统计系数的信息熵分布，为具有不同信息熵的数据分配不同的比特数。
当前主流的视频压缩编码架构是混合编码架构，针对上述冗余，采取不同的技术来消除冗余，并将这些技术结合在一起，形成了视频编码的混合架构。如图3所示，编码端接收到视频之后，对于构成视频的每帧图像，将该图像划分成待编码图像块。对于当前待编码图像块，首先通过参考重建图像块对当前待编码图像块进行预测，得到当前待编码图像块的预测信号；用当前待编码图像块的原始信号减去预测信号，得到残差信号。经过预测后，残差信号的幅值远小于原始信号。将残差信号进行变换和量化操作。经过变换量化后，得到变换量化系数，再通过熵编码技术编码量化系数以及编码中的其他指示信息，得到码流。进一步地，编码端还需要重建当前待编码图像块，以实现对后续待编码图像块的编码提供参考像素。具体地，在得到当前待编码图像块的变换量化系数之后，编码端需要对当前待编码图像块的变换量化系数进行反量化和反变换，得到重建的残差信号，将重建的残差信号与当前待编码图像块对应的预测信号相加，得到当前待编码图像块的重建信号，根据该重建信号得到重建图像块。
如图4所示，解码端获取到码流之后，首先对码流进行熵解码，得到当前待重建图像块的变换量化系数，然后对变换量化系数进行反量化和反变换，得到当前待重建图像 块的重建的残差信号。通过参考重建图像块对当前待重建图像块进行预测，得到当前待重建图像块的预测信号，然后将预测信号和上述重建的残差信号相加，得到当前待重建图像块的重建信号，然后根据该重建信号得到当前待重建图像块对应的当前重建图像块。
为了降低编解码复杂度，编码端对每帧图像先进行下采样处理，如图5所示，编码端对整幅图像采用下采样处理，然后对经过下采样处理后的图像中的每个待编码图像块进行编码，得到码流。其中，每个待编码图像块对应的重建图像块的分辨率为下采样分辨率。相应的，解码端解析码流，每个待重建图像块的分辨率均为下采样分辨率，得到对应的重建图像块的分辨率也为下采样分辨率，解码端需要对重建图像块采用上采样处理，以得到原始分辨率的重建图像块。
其中，在编码端和解码端均涉及通过参考重建图像块对当前图像块(当前待编码图像块或者当前待重建图像块)进行预测，得到当前图像块的预测信号。在本申请中，对当前图像块的预测模式(主要是帧内预测方法)可以采用现有技术，具体如下：
例如：图6为本申请一实施例提供的正在编码的图像的示意图，如图6所示，该图像包括多个图像块，其中，该图像的编码顺序是：从上到下，从左到右。在图6中，图像块C、B、D、E和A表示已经完成重建的重建图像块，图像块F为当前待编码图像块，该图像中的其他区域为未编码的图像区域。
这里以H.265标准来说明帧内预测方法的具体过程，H.265支持将当前待编码图像块划分成更小的子图像块进行预测操作。子图像块的划分结构为四叉树结构，即一个图像块可以划分成四个子图像块，每个子图像块可以继续划分成四个子图像块。如图6所示，假设当前待编码图像块被划分成7个子图像块进行预测操作，当前待编码图像块也可以划分成更多的子图像块进行预测操作。对于每个子图像块，先进行预测操作，获得预测信号，然后根据预测信号获得子图像块的残差信号，进一步对残差信号进行变换、量化和熵编码。对于预测操作，每个子图像块可选的帧内预测方法有35种，包括Planar模式、DC模式以及33种角度预测模式。所有预测模式都使用相同的参考像素模板(由多个参考像素构成)，图7为本申请一实施例提供的参考像素模板的示意图，如图7所示，P 1， 1，P 2，1……P N，1……P 1，N，P 2，N……P N，N这些像素构成待编码的子图像块，例如：该待编码的子图像块可以是图6中的子图像块1。如图7所示，除了待编码的子图像块，其他参考像素R 0，0，R 1，0……R 2N+1，0……R 0，2N构成参考像素模板，假设该待编码的子图像块是图6中的子图像块1，这种情况下，这些参考像素中的一部分像素为参考重建图像块B的最后一行的像素，另一部分像素为参考重建图像块A的最右侧一列的像素。对于其他标准，这些参考像素中的一部分像素为参考重建图像块B包括的下方多行的像素，另一部分像素为参考重建图像块A包括的右侧多列的像素。即本申请对参考像素模板不做限制。
Planar模式
Planar模式适用于像素值缓慢变化的区域，图8A和图8B为本申请一实施例提供的Planar模式的示意图，如图8所示，使用水平和竖直方向的两个线性滤波器，分别得到两个预测值
Figure PCTCN2018081678-appb-000001
和
Figure PCTCN2018081678-appb-000002
并将
Figure PCTCN2018081678-appb-000003
和
Figure PCTCN2018081678-appb-000004
的平均值作为像素(x,y)的预测信号。
DC模式
DC模式适用于大面积平坦区域，当前待编码的子图像块的预测信号可由其左侧和上方的参考像素的平均值得到，如图7所示，待编码的子图像块中各个像素的预测信号可 以通过R 0,1,…,R 0,N,R 1,0,…,R N,0的平均值得到。
角度模式
H.265/HEVC规定了33种角度预测模式，以更好地适应视频内容中不同方向的纹理。图9为本申请一实施例提供的33种角度预测模式的具体方向示意图，如图9所示，33种角度预测模式分为水平类模式(2～17)和竖直类模式(18～34)。其中V0(模式26)和H0(模式10)分别表示竖直和水平方向，其余角度预测模式的预测方向都可以看作是在竖直或水平方向上做一个角度偏移。这里以竖直方向V0(26)为例说明角度预测过程，竖直方向预测是使用当前待编码的子图像块上方相邻的一行参考像素来预测当前待编码的子图像块，当前待编码的子图像块内每个像素的预测信号等于该像素所在列对应的参考像素的像素值，即P x,y＝R y,0。对于其他角度预测模式，会与水平或竖直方向有一个角度偏移，根据这个角度偏移可以计算参考像素的位置。该参考像素的位置可能是两个相邻参考像素之间的位置，若是这种情况，则需要在两个参考像素之间根据计算出来的位置插值得到一个参考像素。通过得到的参考像素生成预测信号。
需要说明的是，上述帧内预测方法同样适用于解码端，本申请在此不再赘述。
进一步地，本申请还涉及到图像下采样处理和图像上采样处理。
其中，图像下采样处理涉及三个方面的信息：1、下采样比例；2、下采样位置；3、下采样所使用的滤波器。
下采样比例是指原图像与下采样后的图像的比例，可以分别在水平方向和竖直方向来描述。例如可以对图像信号进行水平方向2:1下采样，竖直方向4:1下采样；或者水平方向不下采样，竖直方向2:1下采样；或者水平和竖直方向都进行2:1下采样等。
下采样位置是指下采样点与原采样点的位置关系，例如，下采样点的位置可以与部分原采样点位置一样，或者下采样点落在几个原采样点之间。
下采样滤波器可以是3-lobe Lanczos滤波器，Bilinear滤波器，Bicubic，Gauss滤波器等。
下面以分辨率为16*16的图像块(实际的图像会比这大很多，例如1920*1080)为例来说明下采样过程。图10为本申请一实施例提供的图像下采样示意图，假设水平方向和竖直方向的采样比例均为2:1，水平方向上，下采样点的位置落在两个原采样点的左侧原采样点位置，竖直方向上，下采样点落在两个原采样点的上方原采样点位置。如图10所示，框出来的圆圈表示下采样点的位置，下采样的滤波器如下：
Figure PCTCN2018081678-appb-000005
该滤波器是简单的低通滤波器，该低通滤波器可以被看作是一个二维滤波器，也可以被看作是两个一维滤波器。如果当作一个二维滤波器，可以在一次滤波操作中同时完成水平和竖直方向上的下采样。如图10所示，对下采样点A进行下采样时，用到了邻近的8个原采样点(用三角形框起的圆圈)，根据上面滤波器计算出该下采样点A的像素值。如果当作是两个一维滤波器，需要先完成水平或者竖直方向的下采样，再对已完成水平或者竖直方向下采样的结果进行竖直或水平方向下采样。如图10所示，对下采样点A进行下采样时，先利用下采样点A左右各一个原采样点进行水平下采样，然后对下采样后的结果利用下采样点A上下各一个原采样点进行竖直方向上的下采样，根据上述滤 波器计算出下采样点A的像素值。采用相同的方法，对整个16*16的图像块进行下采样处理，最终下采样的结果如图10所示，各个下采样点的位置如框出来的圆圈所示，下采样点的像素值为通过滤波器操作后的数值。如图10所示，下采样后的图像块的分辨率为8*8。
通常编码端或者解码端需要对经过下采样后的图像进行上采样处理，目的是为了得到原分辨率的图像。上采样处理涉及三个方面的信息：1、上采样比例；2、上采样位置；3上采样所使用的滤波器。
上采样比例是指上采样前的图像与上采样后的图像的比例，可以分别在水平方向和竖直方向来描述。例如可以对上采样才的图像信号进行水平方向1:2上采样，竖直方向1:4上采样；或者水平方向不上采样，竖直方向1:2上采样；或者水平和竖直方向都进行1:2上采样等。
上采样位置是指上采样后的采样点与上采样前的采样点的位置关系，例如，图11为本申请一实施例提供的图像上采样示意图，如图11所示，在第一行，采用水平1:2上采样比例，上采样后的采样点的位置可以在上采样前的采样点的右侧，其中×表示上采样后的采样点位置，圆圈表示上采样前的采样点位置。在第二行，采用水平1:2上采样比例，上采样后的采样点的位置可以在上采样前的采样点的左侧，其中×表示上采样后的采样点位置，圆圈表示上采样前的采样点位置。需要说明的是，上采样后的采样点的位置应与上述下采样点的位置选取相对应，例如：在进行下采样时，选择下采样点的位置为它的左侧原采样点的位置，那么在进行上采样时，选择上采样后的采样点的位置为它的右侧上采样前的采样点(下采样点)的位置。
上采样滤波器可以是DCTIF滤波器，双线性插值滤波器，sinc滤波器等。下面以分辨率为8*8的图像块(即上述下采样后的图像块)为例来说明上采样过程。假设水平方向和竖直方向的上采样比例均为1:2，在水平方向上，上采样后的采样点的位置为右侧上采样前的采样点的位置，在竖直方向上，上采样后的采样点的位置为下方上采样前的采样点的位置，这里以水平方向和竖直方向分别上采样为例，以及以DCTIF滤波器为例来说明上采样处理过程。DCTIF滤波器为(-1,4,-11,40,40,-11,4,-1)，假设当前要进行水平方向上采样，在图11中，假设需要插入B3采样点，则采用如下公式确定B3的像素值：
B 3＝(-A 0+4*A 1-11*A 2+40*A 3+40*A 4-11*A 5+4*A 6-A 7)＞＞6
对于其他位置的插值采样点，例如B7，需要用到B7右边的四个像素，这些像素目前不可得，实际中一般会将A7重复4次，以用于计算B7的像素值。在竖直方向上的上采样与在水平方向上的上采样类似，在此不再赘述。也可以先进行竖直方向上的上采样，再进行水平方向上的上采样。图12为本申请一实施例提供的图像上采样示意图，如图12所示，×表示上采样后的采样点，圆圈表示上采样前的采样点。
现有技术中，编码端对整幅图像都采用下采样处理，然而，整幅图像包括的各个图像块的特性可能不同，例如，有些图像块可能较为平坦，编码端适合对其采用下采样处理；有些图像块可能细节比较多，这种情况下，下采样处理会损失掉这些细节。这将导致编码端的编码效果不佳。相应的，这将导致解码端得到的有些重建图像块比较模糊，即解码端得到的重建图像块效果不佳。
为了解决上述技术问题，本申请提供一种图像处理方法、设备及系统。本申请基于 图3和图5的编码示意图，如图3和图5所示，一幅图像包括的待编码图像块的编码方式可以是如图3所示的原分辨率编码方式或者是如图5所示的下采样编码方式。其中，原分辨率编码方式是指对当前待编码图像块直接进行编码操作。下采样编码方式是指对当前待编码图像块先进行下采样处理，然后对下采样后的当前待编码图像块进行编码操作。一般情况下，纹理图像块采用原分辨率编码方式，平滑图像块采用下采样编码方式。编码端需要标记每个待编码图像块使用的编码方式，并将该标记写入码流中。使得解码端，根据该标记对待重建图像块进行相应的操作。本申请的主旨思想在于：解码端根据当前待重建图像块的编码方式、当前待重建图像块对应的M个参考重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素，根据多个参考像素生成预测信号，进而重建待重建图像块。相应的，编码端根据当前待编码图像块的编码方式、当前待编码图像块对应的M个参考重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待编码图像块的多个参考像素，根据多个参考像素生成预测信号，进而得到残差信号，对残差信号进行编码。
具体地，图13为本申请一实施例提供的一种图像处理方法的流程图，如图13所示，该方法包括：
步骤S1301：解析码流，以获取当前图像的当前待重建图像块的编码方式、当前待重建图像块的残差信号、当前待重建图像块对应的每个参考重建图像块中的像素；
其中，解码端解析码流，该码流中携带有当前待重建图像块的编码方式对应的标记、以及当前待重建图像块的变换量化系数。解码端可以对变换量化系数进行反量化和反变换，得到当前待重建图像块的残差信号。
当前待重建图像块对应M个参考重建图像块，M为大于或者等于1的正整数。参考重建图像块用于确定待重建图像块的多个参考像素，其中，多个参考像素用于生成当前重建图像块的预测信号。实际上，参考重建图像块具体是哪个重建图像块和解码端采用的预测模式有关。当采用上述35中预测模式中的任一种时，可以参考如图7所示的参考像素模板。
步骤S1302：根据当前待重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素。
其中，若参考重建图像块与当前待重建图像块的分辨率相同，则直接在该参考重建图像块中确定至少一个参考像素；若当前待重建图像块是原分辨率，参考重建图像块的分辨率为下采样分辨率，则从该参考重建图像块中获取对当前重建图像块进行重建时所需的至少一个像素，并对当前重建图像块进行重建时所需的至少一个像素进行上采样处理，以得到当前待重建图像块的至少一个参考像素；若当前待重建图像块是下采样分辨率，参考重建图像块的分辨率为原分辨率，则从该参考重建图像块中获取对所述当前待重建图像块进行重建时所需的至少一个像素，对这些像素进行下采样处理，以得到当前待重建图像块的至少一个参考像素。
步骤S1303：根据多个参考像素生成当前待重建图像块的预测信号；
步骤S1304：根据预测信号和残差信号生成当前待重建图像块的重建信号，并根据重建信号重建当前待重建图像块，得到当前重建图像块。
其中，根据多个参考像素生成当前待重建图像块的预测信号，可以采用上述35中预测模式中的任一种预测模式，当然也可以采用现有技术中其他的预测模式，本申请对此 不做限制。最后，将重建的残差信号与预测信号相加，得到当前待重建图像块的重建信号，并根据重建信号重建当前待重建图像块，得到当前重建图像块。
综上，本申请考虑到图像块具有各自的特性，编码端对它们所采用的编码方式也不尽相同，基于此，解码端将当前待重建图像块的编码方式这个因素考虑在内，以重建当前待重建图像块。从而使得解码端得到的重建图像块效果更佳。
进一步地，若当前重建图像块的编码方式为下采样编码方式，则图像处理方法还包括：基于当前重建图像块进行上采样处理时所需的相邻重建图像块的像素，对当前重建图像块进行上采样处理。需要说明的是，所需的相邻重建图像块的像素主要用于对当前重建图像块的部分边界进行上采样处理，对于当前重建图像块中除上述部分边界之外的部分，均采用自己的像素进行上采样处理。其中，当解码端采用滤波器对当前重建图像块进行上采样处理时，针对不同的滤波器，当前重建图像块所需的相邻重建图像块也不同。假设滤波器是基于离散余弦变换的插值滤波器(Discrete Cosine Transform-Based Interpolation Filter，DCTIF)，这种情况下，当前重建图像块所需的相邻重建图像块具体如下：图14为本申请一实施例提供的4邻域像素的示意图，如图14所示，当前重建图像块所需的相邻重建图像块包括：当前重建图像块的上边图像块、下边图像块、左边图像块和右边图像块。假设滤波器是卷积神经网络(Convolutional Neural Network，CNN)滤波器。这种情况下，当前重建图像块所需的相邻重建图像块具体如下：图15为本申请一实施例提供的8邻域像素的示意图，如图15所示，当前重建图像块所需的相邻重建图像块包括：当前重建图像块的上边图像块、下边图像块、左边图像块、右边图像块、左上图像块、左下图像块、右上图像块和右下图像块。按照目前的编码顺序(从上至下，从左至右的编码顺序)，目前当前重建图像块的下边图像块、右边图像块、左下图像块和右下图像块都还没有完成重建，现有技术是通过复制当前重建图像块自己的像素以实现上采样处理，但是这种方式将导致经过上采样处理后的当前重建图像块存在右边界和下边界不连续的问题。为了解决这一问题，本申请提供如下四种可选方式：
方式一、上采样处理是在当前重建图像块进行上采样处理时所需的所有相邻重建图像块都重建完成后进行的；相应的，码流包括：M个参考重建图像块中的每个参考重建图像块的编码方式；根据当前待重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素，包括：根据当前待重建图像块的编码方式、M个参考重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素。
方式二、上采样处理是在当前图像的所有图像块都重建完成后进行的；相应的，码流包括：M个参考重建图像块中的每个参考重建图像块的编码方式；根据当前待重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素，包括：根据当前待重建图像块的编码方式、M个参考重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素。
方式三、根据所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对当前重建图像块进行一次上采样处理；若所需的相邻重建图像块中的当前未完成重建的另一部分相邻重建图像块已完成重建，则根据另一部分相邻重建图像块对当前重建图像块的部分边界进行二次上采样处理，其中，当前重建图像块的部分边界与另一部分相邻重建图像块邻接。
方式四、对当前重建图像块进行上采样处理包括：根据所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对当前重建图像块进行一次上采样处理；若当前图像的所有图像块均已完成重建，则根据在所需的相邻重建图像块中的另一部分相邻重建图像块对当前重建图像块的部分边界进行二次上采样处理，其中，另一部分相邻重建图像块是在对当前重建图像块进行第一次上采样处理时未完成重建的图像块；当前重建图像块的部分边界与另一部分相邻重建图像块邻接。
其中，所述当前重建图像块的部分边界满足条件：在对当前重建图像块进行的第一次上采样处理中，该部分边界所需的另一部分相邻重建图像块未完成重建。
可选地，当前重建图像块的部分边界为当前重建图像块的右边界和下边界。
可选地，所述一部分相邻重建图像块为所述当前重建图像块的上边图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的下边图像块和右边图像块。
或者，
所述一部分相邻重建图像块为所述当前重建图像块的左上图像块，上边图像块，右上图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的右边图像块，左下图像块，下边图像块和右下图像块。
针对方式一进行详细说明：
具体地，针对不同的滤波器，当前重建图像块所需的相邻重建图像块也不同。例如：如图14所示，当前重建图像块所需的相邻重建图像块包括：当前重建图像块的上边图像块、下边图像块、左边图像块和右边图像块。如图15所示，当前重建图像块所需的相邻重建图像块包括：当前重建图像块的上边图像块、下边图像块、左边图像块、右边图像块、左上图像块、左下图像块、右上图像块和右下图像块。
其中，可以采用现有技术的上采样处理方法对当前重建图像块进行处理。例如：图16为本申请一实施例提供的图像上采样示意图，如图16所示，当前重建图像块B所需的相邻重建图像块1、相邻重建图像块2、相邻重建图像块3和相邻重建图像块4均已完成重建。基于此，对当前重建图像块B进行上采样处理，如图16所示，其中B中的圆圈表示上采样前的采样点，×表示上采样后的采样点。对B进行上采样处理时，可以先对B进行水平方向上的上采样，再对上采样后的信号进行竖直方向上的上采样；或者，可以先对B进行竖直方向上的上采样，再对上采样后的信号进行水平方向上的上采样。
特别的，若当前重建图像块本身是一幅图像的边界图像块，这种情况下，即使所需的相邻重建图像块都重建完成，在进行上采样处理时，还是需要复制当前重建图像的像素。例如：如图14所示，当当前重建图像块是一幅图像的最右侧的一个图像块时，它的右边图像块是不存在的，因此可以对当前重建图像块包括的最右侧一列的像素进行复制，以实现上采样处理。当然，也可以采用其他方法进行上采样处理，本申请对此不做限制。
进一步地，由于当前重建图像块所需的各个相邻重建图像块的编码方式可以是下采样编码方式，也可以是原分辨率编码方式，因此在对当前重建图像块进行上采样处理时，具体分为以下两种情况：
1、若某相邻重建图像块的编码方式为下采样编码方式，则可以直接根据该相邻重建图像块中的像素对当前重建图像块进行上采样处理。
2、若某相邻重建图像块的编码方式为原分辨率编码方式，则可以获取该相邻重建图像块的像素中上采样处理所需的至少一个像素，对这些像素进行下采样处理，并根据下 采样处理后的至少一个像素对当前重建图像块进行上采样处理。
具体地，相邻重建图像块主要用于对当前重建图像块的部分边界进行上采样处理(该部分边界根据滤波器的不同而不同)，例如：如图16所示，相邻重建图像块3采用下采样编码方式，这种情况下，可以直接利用相邻重建图像块3包括的像素对当前重建图像块B的右边界进行上采样处理。而相邻重建图像块4采用原分辨率编码方式，则需要对相邻重建图像块4包括的上采样处理所需的像素进行下采样处理，或者对相邻重建图像块4采用下采样处理，并根据下采样处理后的像素对当前重建图像块B的下边界进行上采样处理。其中，对相邻重建图像块4采用下采样处理，具体可以是直接取图16中被框圈出的像素作为下采样后的采样点。或者是对相邻重建图像块4进行竖直方向上的下采样处理。图17为本申请另一实施例提供的图像上采样示意图，如图17所示，这种是8邻域像素的情况，解码端对当前重建图像块C采样的上采样处理方法与4邻域像素情况下的方法类似，在此不再赘述。
需要说明的是，为了避免对当前重建图像块的重复上采样处理，可以对当前重建图像块进行上采样处理后，标识当前重建图像块已完成上采样处理。或者，按照一定的规则对当前重建图像块进行上采样处理。当上采样处理是基于4邻域像素的情况时，当前重建图像块的下边图像块一旦完成重建，则可以对当前重建图像块进行上采样处理。当上采样处理是基于8邻域像素的情况时，当前重建图像块的右下图像块一旦完成重建，则可以对当前重建图像块进行上采样处理。
针对方式二进行详细说明：
当当前图像的所有图像块都已经完成重建之后，则对于每个重建图像块来讲，它所需的相邻重建图像块都已经完成重建，基于此，对任一采用下采样编码的重建图像块都可以进行上采样处理。具体上采样处理过程类似于上述方式一的方式，本申请在此不再赘述。
针对方式三进行详细说明：
在方式三中，对当前重建图像块进行的上采样处理包括两次上采样处理过程。第一次上采样处理过程为：根据所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对当前重建图像块进行一次上采样处理。第二次上采样处理过程为：若所需的相邻重建图像块中的当前未完成重建的另一部分相邻重建图像块已完成重建，则根据另一部分相邻重建图像块对当前重建图像块的部分边界进行二次上采样处理，其中，当前重建图像块的所述部分边界与另一部分相邻重建图像块邻接。
可选地，在对当前重建图像块进行第一次上采样处理之前，保存下采样编码方式的当前重建图像块，为后续对其他待重建图像块进行预测时提供参考像素。
具体地，假设采用从上至下、从左至右的编码顺序，对于基于4邻域像素或者8邻域像素进行上采样处理的情况，在对当前重建图像块进行第一次上采样处理时，它的右边图像块、下边图像块、左下图像块和右下图像块都还没有完成重建。这种情况下，解码端可以复制当前重建图像块包括的最右侧一列或者多列的像素，得到插值像素。通过这些插值像素对当前重建图像块包括的右边界进行上采样处理。解码端还可以复制当前重建图像块包括的最下方一行或者多行的像素，得到插值像素。通过这些插值像素对当前重建图像块包括的下边界进行上采样处理。
图18为本申请一实施例提供的图像上采样示意图，如图18所示，假设当前重建图 像块为图像块A，在对当前重建图像块A进行第二次上采样处理时，假设目前采用DCTIF滤波器进行上采样处理，当前重建图像块A已完成第一次上采样处理，如上面所述使用DCTIF滤波器进行上采样处理时，需要用到左右各四个像素，那么在第一次上采样处理时，当前重建图像块A的右边四列×(×表示第一次上采样处理后的采样点)所需的四个参考像素均是不全的。例如，对于最右侧的一列×，每个×所需要的右边四个参考像素都不存在。若相邻重建图像块C完成了重建，则根据相邻重建图像块C对当前重建图像块的右边界进行第二次上采样处理。这里的上采样处理方法与上述上采样处理方法相同，在此不再赘述。
同样地，假设当前重建图像块为图像块B，在对当前重建图像块B进行第二次上采样处理时，假设目前采用DCTIF滤波器进行上采样处理，当前重建图像块B已完成第一次上采样处理，如上面所述使用DCTIF滤波器进行上采样处理时，需要用到上下各四个像素，那么在第一次上采样处理时，当前重建图像块B的下方四行×(×表示第一次上采样处理后的采样点)所需的四个参考像素均是不全的。例如，对于最下方的一行×，每个×所需要的下方四个参考像素都不存在。若相邻重建图像块C完成了重建，则根据相邻重建图像块C对当前重建图像块的下边界进行第二次上采样处理。这里的上采样处理方法与上述上采样处理方法相同，在此不再赘述。
图19为本申请另一实施例提供的图像上采样示意图，如图19所示，这种是8邻域像素的情况，解码端对当前重建图像块A的右边界以及当前重建图像块B的下边界采用的上采样处理方法与4邻域像素情况下的方法类似，在此不再赘述。
需要说明的是，为了避免对当前重建图像块的重复上采样处理，可以对当前重建图像块完成第二次上采样处理后，标识当前重建图像块已完成上采样处理。或者，按照一定的规则对当前重建图像块进行上采样处理。当上采样处理是基于4邻域像素的情况时，当前重建图像块的下边图像块一旦完成重建，则可以对当前重建图像块进行第二次上采样处理。当上采样处理是基于8邻域像素的情况时，当前重建图像块的右下图像块一旦完成重建，则可以对当前重建图像块进行第二次上采样处理。
针对方式四进行详细说明：
当当前图像的所有图像块都已经完成重建之后，则对于每个重建图像块来讲，它所需的相邻重建图像块都已经完成重建，基于此，对任一已完成第一次上采样处理后的重建图像块都可以进行第二次上采样处理。具体上采样处理过程类似于上述方式三的方式，本申请在此不再赘述。
可选地，在对当前重建图像块进行第一次上采样处理之前，保存下采样编码方式的当前重建图像块，为后续对其他待重建图像块进行预测时提供参考像素。
综上，通过上述四种方式对当前重建图像块进行上采样处理时，当前重建图像块的部分边界都是通过所需的相邻重建图像块进行上采样处理的，而现有技术中，当前重建图像块的部分边界都是通过复制当前重建图像块的像素进行上采样处理的，因此，本申请提供的方法可以避免当前重建图像块边界不连续的问题。
更进一步地，基于上述方式一和方式二，对步骤S1302进行详细说明：
其中步骤S1302具体包括如下四种情况：
情况1、若当前待重建图像块的编码方式为原分辨率编码方式，且M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则在编码方式为原分辨率编 码方式的参考重建图像块的像素中确定当前待重建图像块的参考像素。
情况2、若当前待重建图像块的编码方式为原分辨率编码方式，且M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则从编码方式为下采样编码方式的参考重建图像块的像素中获取对当前待重建图像块进行重建时所需的像素，并对当前待重建图像块进行重建时所需的像素进行上采样处理，以得到当前待重建图像块的参考像素。
情况3、若当前待重建图像块的编码方式为下采样编码方式，且M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则在编码方式为下采样编码方式的参考重建图像块的像素中确定当前待重建图像块的参考像素。
情况4、若当前待重建图像块的编码方式为下采样编码方式，且M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则从编码方式为原分辨率编码方式的参考重建图像块的像素中获取对当前待重建图像块进行重建时所需的像素，对当前待重建图像块进行重建时所需的像素进行下采样处理，以得到当前待重建图像块的参考像素。
对情况1进行说明：图20为本申请一实施例提供的当前待重建图像块与参考重建图像块的示意图，如图20所示，假设本申请实施例使用如图7所示的参考像素模板，则当前待重建图像块E对应的M个参考重建图像块为参考重建图像块A、B、C和D，实际上，按照如图7所示的参考像素模板，图像块E还需要使用图像块A下方的图像块，但是按照目前的编码顺序，图像块A下方的图像块还未完成重建，现有技术通过复制图像块A的方式，实现对图像块E的预测。本申请对此不做详细说明。对于参考重建图像块A、B、C和D，由于它们的编码方式均为原分辨率编码方式，而当前待重建图像块E的编码方式也是原分辨率编码方式，因此，可以直接从参考重建图像块A、B、C和D中确定当前待重建图像块的参考像素。例如：如图20所示，被框中的像素为参考像素。
对情况2进行说明：图21为本申请一实施例提供的当前待重建图像块与参考重建图像块的示意图，如图21所示，假设本申请实施例使用如图7所示的参考像素模板，则当前待重建图像块E对应的M个参考重建图像块为参考重建图像块A、B、C和D，其中，当前待重建图像块E的编码方式为原分辨率编码方式，参考重建图像块B的编码方式为下采样编码方式，这种情况下，需要获取参考重建图像块B中的最下方一行的像素，然后对获取的像素进行上采样处理，得到当前重建图像块E的参考像素。如图21所示，被框中的像素为所有的参考像素。需要说明的是，还可以根据参考重建图像块B中的多行像素确定当前重建图像块E的参考像素等，本申请对此不做限制。
对情况3进行说明：图22为本申请一实施例提供的当前待重建图像块与参考重建图像块的示意图，如图22所示，假设本申请实施例使用如图7所示的参考像素模板，则当前待重建图像块E对应的M个参考重建图像块为参考重建图像块A、B、C和D，其中，当前待重建图像块E的编码方式为原分辨率编码方式，参考重建图像块A、B和D的编码方式均为原采样编码方式，这种情况下，可以直接从参考重建图像块A、B、C和D中确定当前待重建图像块的参考像素。例如：如图22所示，被框中的像素为参考像素。
对情况4进行说明：图23为本申请一实施例提供的当前待重建图像块与参考重建图像块的示意图，如图23所示，假设本申请实施例使用如图7所示的参考像素模板，则当前待重建图像块E对应的M个参考重建图像块为参考重建图像块A、B、C和D，其中， 当前待重建图像块E的编码方式为下采样编码方式，参考重建图像块B的编码方式为原分辨率编码方式，这种情况下，需要获取参考重建图像块B中的最下方一行的像素，然后对获取的像素进行下采样处理，得到当前重建图像块E的参考像素。如图23所示，被框中的像素为所有的参考像素。
上述四种情况所涉及的下采样处理和上采样处理都可以采用现有技术的方法，本申请对此不做限制。
需要强调的是，由于在方式三和方式四中，对于任何一个采用下采样编码方式的待重建图像块，在对待重建图像块重建完之后，都要进行第一次上采样处理，因此任何一个重建图像块的分辨率都是原分辨率。
基于此，针对方式三和方式四情况，步骤S1302包括：若当前待重建图像块的编码方式为原分辨率编码方式，则在M个参考重建图像块的像素中确定当前待重建图像块的参考像素；若当前待重建图像块的编码方式为下采样编码方式，则在M个参考重建图像块的像素中获取对当前待重建图像块进行重建时所需的像素，并对当前待重建图像块进行重建时所需的像素进行下采样处理，以得到当前待重建图像块的参考像素；或者，若当前待重建图像块所需的某相邻重建图像块采用下采样编码方式，则保存该相邻重建图像块在进行第一次上采样处理之前的像素，若当前待重建图像块的编码方式为下采样编码方式，则可以直接在之前保存的像素中确定当前待重建图像块的参考像素。
综上，通过上述方法可以有效的确定当前重建图像块的参考像素，进而实现对当前重建图像块的重建。
上面主要介绍了解码端的图像处理方法，下面将介绍编码端的图像处理方法。
具体地，图24为本申请另一实施例提供的一种图像处理方法的流程图，如图24所示，该方法包括：
步骤S2401：获取当前图像的当前待编码图像块的编码方式、当前待编码图像块对应的每个参考重建图像块中的像素；
步骤S2402：根据当前待编码图像块的编码方式和M个参考重建图像块中的像素，确定当前待编码图像块的多个参考像素；
步骤S2403：根据多个参考像素生成当前待编码图像块的预测信号；
步骤S2404：获取当前待编码图像块的编码信号，其中，当当前待编码图像块的编码方式是原分辨率编码方式时，编码信号为当前待编码图像块的原始信号，当当前待编码图像块的编码方式是下采样编码方式时，编码信号为当前待编码图像块的原始信号经过下采样处理后得到的信号；
步骤S2405：根据预测信号和编码信号生成当前待编码图像块的残差信号；
步骤S2406：对残差信号进行编码。
其中，编码方式为原分辨率编码方式或者下采样编码方式，当前待编码图像块对应M个参考重建图像块，M为大于或者等于1的正整数。
其中，若参考重建图像块与当前待重建图像块的分辨率相同，则直接在该参考重建图像块中确定至少一个参考像素；若当前待重建图像块是原分辨率，参考重建图像块的分辨率为下采样分辨率，则从该参考重建图像块中获取对当前重建图像块进行重建时所需的至少一个像素，并对当前重建图像块进行重建时所需的至少一个像素进行上采样处理，以得到当前待重建图像块的至少一个参考像素；若当前待重建图像块是下采样分辨 率，参考重建图像块的分辨率为原分辨率，则从该参考重建图像块中获取对所述当前待编码图像块进行重建时所需的至少一个像素，对这些像素进行下采样处理，以得到当前待重建图像块的至少一个参考像素。
实际上，参考重建图像块具体是哪个重建图像块和解码端采用的预测模式有关。当采用上述35中预测模式中的任一种时，可以参考如图7所示的参考像素模板。根据多个参考像素生成当前待重建图像块的预测信号，可以采用上述35中预测模式中的任一种预测模式，当然也可以采用现有技术中其他的预测模式，本申请对此不做限制。最后，根据预测信号和编码信号生成当前待编码图像块的残差信号；对残差信号进行编码，包括：将残差信号进行变换和量化操作。经过变换量化后，得到变换量化系数，再通过熵编码技术编码量化系数以及编码中的其他指示信息，得到码流。
综上，本申请考虑到待编码图像块具有各自的特性，编码端对它们所采用的编码方式也不尽相同，基于此，编码端将当前待重建图像块的编码方式这个因素考虑在内，以对当前待编码图像块进行编码。从而使得编码端的编码效果更佳。
进一步地，上述方法还包括：生成当前待编码图像块的重建信号，并根据重建信号重建当前待编码图像块，得到当前重建图像块；若当前重建图像块的编码方式为下采样编码方式，则基于当前重建图像块进行上采样处理时所需的相邻重建图像块的像素，对当前重建图像块进行上采样处理。
其中，生成当前待编码图像块的重建信号包括：对当前待编码图像块的残差信号进行变换、量化，得到当前待编码图像块的量化系数，将量化系数进行反量化，反变换，得到重建的残差信号，将重建的残差信号与当前待编码图像块的预测信号相加，得到当前待编码图像块的重建信号。
对当前重建图像块进行上采样处理，目的是为了获得原分辨率的当前重建图像块，从而为后续帧间预测技术做准备。
其中，对当前重建图像块进行的上采样处理存在四种可选方式：
方式一、上采样处理是在当前图像的所有图像块都重建完成后进行的；相应的，根据当前待编码图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素，包括：获取M个参考重建图像块中的每个参考重建图像块的编码方式；根据当前待编码图像块的编码方式、M个参考重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待编码图像块的多个参考像素。
方式二、上采样处理是在当前重建图像块进行上采样处理时所需的所有相邻重建图像块都重建完成后进行的；相应的，根据当前待编码图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素，包括：获取M个参考重建图像块中的每个参考重建图像块的编码方式；根据当前待编码图像块的编码方式、M个参考重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待编码图像块的多个参考像素。
方式三、对当前重建图像块进行上采样处理包括：根据所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对当前重建图像块进行一次上采样处理；若所需的相邻重建图像块中的当前未完成重建的另一部分相邻重建图像块已完成重建，则根据另一部分相邻重建图像块对当前重建图像块的部分边界进行二次上采样处理，其中，当前重建图像块的部分边界与另一部分相邻重建图像块邻接。
方式四、对当前重建图像块进行上采样处理包括：根据所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对当前重建图像块进行一次上采样处理；若当前图像的所有图像块均已完成重建，则根据在所需的相邻重建图像块中的另一部分相邻重建图像块对当前重建图像块的部分边界进行二次上采样处理，其中，另一部分相邻重建图像块是在对当前重建图像块进行第一次上采样处理时未完成重建的图像块；当前重建图像块的部分边界与另一部分相邻重建图像块邻接。
上述四种方式与解码端的四种方式完全相同，对应内容和效果在此不再赘述。
可选地，所述一部分相邻重建图像块为所述当前重建图像块的上边图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的下边图像块和右边图像块。
或者，
所述一部分相邻重建图像块为所述当前重建图像块的左上图像块，上边图像块，右上图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的右边图像块，左下图像块，下边图像块和右下图像块。
可选地，所述当前重建图像块的部分边界为所述当前重建图像块的右边界和下边界。
可选地，所需的相邻重建图像块包括所述当前重建图像块的上边图像块，下边图像块，左边图像块和右边图像块；或者，所需的相邻重建图像块包括所述当前重建图像块的上边图像块，下边图像块，左边图像块，右边图像块，左上图像块，左下图像块，右上图像块和右下图像块。
可选地，基于上述方式一和方式二，对步骤S2402具体包括如下四种情况：
情况1、若当前待编码图像块的编码方式为原分辨率编码方式，且M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则在编码方式为原分辨率编码方式的参考重建图像块的像素中确定当前待编码图像块的参考像素。
情况2、若当前待编码图像块的编码方式为原分辨率编码方式，且M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则从编码方式为下采样编码方式的参考重建图像块的像素中获取对当前重建图像块进行重建时所需的像素，并对当前待编码图像块进行重建时所需的像素进行上采样处理，以得到当前待编码图像块的参考像素。
情况3、若当前待编码图像块的编码方式为下采样编码方式，且M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则在编码方式为下采样编码方式的参考重建图像块的像素中确定当前待编码图像块的参考像素。
情况4、若当前待编码图像块的编码方式为下采样编码方式，且M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则从编码方式为原分辨率编码方式的参考重建图像块的像素中获取对当前待编码图像块进行重建时所需的像素，对当前待编码图像块进行重建时所需的像素进行下采样处理，以得到当前待编码图像块的参考像素。
上述四种情况与解码端的四种情况类似，对应内容和效果在此不再赘述。
针对方式三和方式四，上述步骤S2402包括：若当前待编码图像块的编码方式为原分辨率编码方式，则在M个参考重建图像块的像素中确定当前待编码图像块的参考像素；若当前待编码图像块的编码方式为下采样编码方式，则在M个参考重建图像块的像素中获取对当前待编码图像块进行重建时所需的像素，并对当前待编码图像块进行重建时所 需的像素进行下采样处理，以得到所述当前待编码图像块的参考像素。
可选地，对当前重建图像块进行上采样处理后，上述方法还包括：标识当前重建图像块已完成上采样处理。或者，按照一定的规则对当前重建图像块进行上采样处理。当上采样处理是基于4邻域像素的情况时，当前重建图像块的下边图像块一旦完成重建，则可以对当前重建图像块进行上采样处理。当上采样处理是基于8邻域像素的情况时，当前重建图像块的右下图像块一旦完成重建，则可以对当前重建图像块进行上采样处理。
可选地，步骤S2401中获取当前待编码图像块的编码方式，包括：确定当前待编码图像块采用原分辨率编码方式时的第一编码代价；确定当前待编码图像块采用下采样编码方式时的第二编码代价；将第一编码代价和第二编码代价中较小的编码代价对应的编码方式作为当前待编码图像块的编码方式。
其中，编码端可以采用率失真优化(Rate Distortion Optimization，RDO)方法计算当前编码图像块的编码代价。RDO方法是一种提升视频压缩性能的最优化方法。其原理是对视频的有损(画面质量)与比特率(编码所需的数据量)同时进行最优化，以求达到一个最佳的平衡点。虽然此算法一开始是在视频压缩的编码器中被使用，但也可以用于各种多媒体编码包含视频、视频、音频等等，只要编码时会同时考虑到质量及文件大小皆可使用。
基于此，本申请中，待编码图像块采用的编码方式为编码代价较小的编码方式，从而降低编码端的编码复杂度，进而提高编码端的编码效率。
图25为本申请一实施例提供的一种图像处理设备的结构示意图，如图25所示，该设备包括：解析模块2501，用于解析码流，以获取当前图像的当前待重建图像块的编码方式、所述当前待重建图像块的残差信号、所述当前待重建图像块对应的每个参考重建图像块中的像素，其中，所述编码方式为原分辨率编码方式或者下采样编码方式，所述当前待重建图像块对应M个参考重建图像块，M为大于或者等于1的正整数；确定模块2502，用于根据当前待重建图像块的编码方式和M个参考重建图像块中的像素，确定当前待重建图像块的多个参考像素；生成模块2503，用于根据多个参考像素生成当前待重建图像块的预测信号；重建模块2504，用于根据预测信号和残差信号生成当前待重建图像块的重建信号，并根据重建信号重建当前待重建图像块，得到当前重建图像块。
可选地，还包括：处理模块2505。其中，处理模块2505用于若当前重建图像块的编码方式为所述下采样编码方式，则基于当前重建图像块进行上采样处理时所需的相邻重建图像块的像素，对当前重建图像块进行上采样处理。
可选地，处理模块2505具体用于：在所述当前图像的所有图像块都重建完成后进行所述上采样处理。相应的，码流包括：所述M个参考重建图像块中的每个参考重建图像块的编码方式；所述确定模块2502具体用于：根据所述当前待重建图像块的编码方式、所述M个参考重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待重建图像块的多个参考像素。
可选地，处理模块2505具体用于：在当前重建图像块进行上采样处理时所需的所有相邻重建图像块都重建完成后进行上采样处理。相应的，码流包括：所述M个参考重建图像块中的每个参考重建图像块的编码方式；所述确定模块2502具体用于：根据所述当前待重建图像块的编码方式、所述M个参考重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待重建图像块的多个参考像素。
可选地，确定模块2502具体用于：若当前待重建图像块的编码方式为原分辨率编码方式，且M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则在编码方式为原分辨率编码方式的参考重建图像块的像素中确定当前待重建图像块的参考像素；若当前待重建图像块的编码方式为原分辨率编码方式，且M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则从编码方式为下采样编码方式的参考重建图像块的像素中获取对当前待重建图像块进行重建时所需的像素，并对当前待重建图像块进行重建时所需的像素进行上采样处理，以得到当前待重建图像块的参考像素；若当前待重建图像块的编码方式为下采样编码方式，且M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则在编码方式为下采样编码方式的参考重建图像块的像素中确定当前待重建图像块的参考像素；若当前待重建图像块的编码方式为下采样编码方式，且M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则从编码方式为原分辨率编码方式的参考重建图像块的像素中获取对当前待重建图像块进行重建时所需的像素，对当前待重建图像块进行重建时所需的像素进行下采样处理，以得到当前待重建图像块的参考像素。
可选地，处理模块2505具体用于：根据所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对所述当前重建图像块进行一次上采样处理；若所需的相邻重建图像块中的当前未完成重建的另一部分相邻重建图像块已完成重建，则根据另一部分相邻重建图像块对所述当前重建图像块的部分边界进行二次上采样处理，其中，当前重建图像块的部分边界与所述另一部分相邻重建图像块邻接。
可选地，处理模块2505具体用于：根据所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对当前重建图像块进行一次上采样处理；若当前图像的所有图像块均已完成重建，则根据在所需的相邻重建图像块中的另一部分相邻重建图像块对当前重建图像块的部分边界进行二次上采样处理，其中，另一部分相邻重建图像块是在对当前重建图像块进行第一次上采样处理时未完成重建的图像块；当前重建图像块的部分边界与另一部分相邻重建图像块邻接。
可选地，确定模块2502具体用于：若当前待重建图像块的编码方式为原分辨率编码方式，则在M个参考重建图像块的像素中确定所述当前待重建图像块的参考像素；若当前待重建图像块的编码方式为所述下采样编码方式，则在所述M个参考重建图像块的像素中获取对所述当前待重建图像块进行重建时所需的像素，并对当前待重建图像块进行重建时所需的像素进行下采样处理，以得到所述当前待重建图像块的参考像素。
可选地，所述一部分相邻重建图像块为所述当前重建图像块的上边图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的下边图像块和右边图像块。
或者，
可选地，所述一部分相邻重建图像块为所述当前重建图像块的左上图像块，上边图像块，右上图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的右边图像块，左下图像块，下边图像块和右下图像块。
可选地，所述当前重建图像块的部分边界为所述当前重建图像块的右边界和下边界。
可选地，所需的相邻重建图像块包括所述当前重建图像块的上边图像块，下边图像块，左边图像块和右边图像块；或者，所需的相邻重建图像块包括当前重建图像块的上边图像块，下边图像块，左边图像块，右边图像块，左上图像块，左下图像块，右上图 像块和右下图像块。
可选地，处理模块2505还用于：在对所述当前重建图像块进行上采样处理后，标识所述当前重建图像块已完成上采样处理。
本申请提供的图像处理设备可以执行上述图13对应的图像处理方法以及该方法的可选方式，其实现原理和技术效果类似，此处不再赘述。
图26为本申请另一实施例提供的一种图像处理设备的结构示意图，如图26所示，该设备包括：
获取模块2601，用于获取当前图像的当前待编码图像块的编码方式、所述当前待编码图像块对应的每个参考重建图像块中的像素，其中，所述编码方式为原分辨率编码方式或者下采样编码方式，所述当前待编码图像块对应M个参考重建图像块，M为大于或者等于1的正整数。
确定模块2602，用于根据所述当前待编码图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待编码图像块的多个参考像素。
生成模块2603，用于根据所述多个参考像素生成所述当前待编码图像块的预测信号。
所述获取模块2601，还用于获取所述当前待编码图像块的编码信号，其中，当所述当前待编码图像块的编码方式是所述原分辨率编码方式时，所述编码信号为所述当前待编码图像块的原始信号，当所述当前待编码图像块的编码方式是所述下采样编码方式时，所述编码信号为所述当前待编码图像块的原始信号经过下采样处理后得到的信号。
所述生成模块2603，还用于根据所述预测信号和所述编码信号生成所述当前待编码图像块的残差信号。
编码模块2604，用于对残差信号进行编码。
可选地，还包括处理模块2605。其中，生成模块2603还用于生成所述当前待编码图像块的重建信号，并根据所述重建信号重建所述当前待编码图像块，得到当前重建图像块；处理模块2605，用于若当前重建图像块的编码方式为所述下采样编码方式，则基于当前重建图像块进行上采样处理时所需的相邻重建图像块的像素，对当前重建图像块进行上采样处理。
可选地，所述处理模块2605具体用于：在所述当前图像的所有图像块都重建完成后进行所述上采样处理；相应的，所述确定模块2602具体用于：获取所述M个参考重建图像块中的每个参考重建图像块的编码方式；根据所述当前待编码图像块的编码方式、所述M个参考重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待编码图像块的多个参考像素。
可选地，所述处理模块2605具体用于：在当前重建图像块进行上采样处理时所需的所有相邻重建图像块都重建完成后进行上采样处理。
可选地，确定模块2602具体用于：若当前待编码图像块的编码方式为原分辨率编码方式，且M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则在编码方式为原分辨率编码方式的参考重建图像块的像素中确定当前待编码图像块的参考像素；若当前待编码图像块的编码方式为原分辨率编码方式，且M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则从编码方式为下采样编码方式的参考重建图像块的像素中获取对当前重建图像块进行重建时所需的像素，并对当前待编码图像块进行重建时所需的像素进行上采样处理，以得到当前待编码图像块的参考像素； 若当前待编码图像块的编码方式为下采样编码方式，且M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则在编码方式为下采样编码方式的参考重建图像块的像素中确定当前待编码图像块的参考像素；若当前待编码图像块的编码方式为下采样编码方式，且M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则从编码方式为原分辨率编码方式的参考重建图像块的像素中获取对当前待编码图像块进行重建时所需的像素，对当前待编码图像块进行重建时所需的像素进行下采样处理，以得到当前待编码图像块的参考像素。
可选地，所述处理模块2605具体用于：根据所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对当前重建图像块进行一次上采样处理；若所需的相邻重建图像块中的当前未完成重建的另一部分相邻重建图像块已完成重建，则根据另一部分相邻重建图像块对当前重建图像块的部分边界进行二次上采样处理，其中，当前重建图像块的部分边界与另一部分相邻重建图像块邻接。
可选地，所述处理模块2605具体用于：根据所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对当前重建图像块进行一次上采样处理；若当前图像的所有图像块均已完成重建，则根据在所需的相邻重建图像块中的另一部分相邻重建图像块对当前重建图像块的部分边界进行二次上采样处理，其中，另一部分相邻重建图像块是在对当前重建图像块进行第一次上采样处理时未完成重建的图像块；当前重建图像块的部分边界与另一部分相邻重建图像块邻接。
可选地，确定模块2502具体用于：若当前待编码图像块的编码方式为原分辨率编码方式，则在M个参考重建图像块的像素中确定所述当前待编码图像块的参考像素；若所述当前待编码图像块的编码方式为所述下采样编码方式，则在所述M个参考重建图像块的像素中获取对所述当前待编码图像块进行重建时所需的像素，并对所述当前待编码图像块进行重建时所需的像素进行下采样处理，以得到所述当前待编码图像块的参考像素。
可选地，所述一部分相邻重建图像块为所述当前重建图像块的上边图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的下边图像块和右边图像块。
或者，
所述一部分相邻重建图像块为所述当前重建图像块的左上图像块，上边图像块，右上图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的右边图像块，左下图像块，下边图像块和右下图像块。
可选地，所述当前重建图像块的部分边界为所述当前重建图像块的右边界和下边界。
所述所需的相邻重建图像块包括所述当前重建图像块的上边图像块，下边图像块，左边图像块和右边图像块；或者，所述所需的相邻重建图像块包括所述当前重建图像块的上边图像块，下边图像块，左边图像块，右边图像块，左上图像块，左下图像块，右上图像块和右下图像块。
可选地，所述处理模块2605还用于：在对所述当前重建图像块进行上采样处理后，标识所述当前重建图像块已完成上采样处理。
可选地，所述获取模块2601具体用于：确定当前待编码图像块采用原分辨率编码方式时的第一编码代价；确定当前待编码图像块采用下采样编码方式时的第二编码代价；将第一编码代价和第二编码代价中较小的编码代价对应的编码方式作为所述当前待编码图像块的编码方式。
本申请提供的图像处理设备可以执行上述图24对应的图像处理方法以及该方法的可选方式，其实现原理和技术效果类似，此处不再赘述。
图27为本申请提供的一种图像处理系统的结构示意图，如图27所示，该系统包括：上述解码端的图像处理设备2701，以及编码端的图像处理设备2702。
本申请提供一种图像处理设备，该设备包括：处理器和用于存储所述处理器的可执行指令的存储器；其中该处理器可以执行图13对应的图像处理方法以及该方法的可选方式。其实现原理和技术效果类似，此处不再赘述。
本申请提供一种图像处理设备，该设备包括：处理器和用于存储所述处理器的可执行指令的存储器；其中该处理器可以执行图24对应的图像处理方法以及该方法的可选方式。其实现原理和技术效果类似，此处不再赘述。
本申请提供的图像处理系统包括的解码端的图像处理设备可以执行上述图13对应的图像处理方法以及该方法的可选方式，包括的编码端的图像处理设备可以执行上述图24对应的图像处理方法以及该方法的可选方式，其实现原理和技术效果类似，此处不再赘述。

Claims (51)

  1. 一种图像处理方法，其特征在于，包括：
    解析码流，以获取当前图像的当前待重建图像块的编码方式、所述当前待重建图像块的残差信号、所述当前待重建图像块对应的每个参考重建图像块中的像素，其中，所述编码方式为原分辨率编码方式或者下采样编码方式，所述当前待重建图像块对应M个参考重建图像块，M为正整数；
    根据所述当前待重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待重建图像块的多个参考像素；
    根据所述多个参考像素生成所述当前待重建图像块的预测信号；
    根据所述预测信号和所述残差信号生成所述当前待重建图像块的重建信号，并根据所述重建信号重建所述当前待重建图像块，得到当前重建图像块。
  2. 根据权利要求1所述的方法，其特征在于，若所述当前重建图像块的编码方式为所述下采样编码方式，则所述方法还包括：
    基于所述当前重建图像块进行上采样处理时所需的相邻重建图像块的像素，对所述当前重建图像块进行上采样处理。
  3. 根据权利要求2所述的方法，其特征在于，所述上采样处理是在所述当前图像的所有图像块都重建完成后进行的；
    相应的，所述码流包括：所述M个参考重建图像块中的每个参考重建图像块的编码方式；
    所述根据所述当前待重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待重建图像块的多个参考像素，包括：
    根据所述当前待重建图像块的编码方式、所述M个参考重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待重建图像块的多个参考像素。
  4. 根据权利要求2所述的方法，其特征在于，所述上采样处理是在所述当前重建图像块进行上采样处理时所需的所有相邻重建图像块都重建完成后进行的；
    相应的，所述码流包括：所述M个参考重建图像块中的每个参考重建图像块的编码方式；
    所述根据所述当前待重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待重建图像块的多个参考像素，包括：
    根据所述当前待重建图像块的编码方式、所述M个参考重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待重建图像块的多个参考像素。
  5. 根据权利要求3或4所述的方法，其特征在于，所述根据所述当前待重建图像块的编码方式、所述M个参考重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待重建图像块的多个参考像素，包括：
    若所述当前待重建图像块的编码方式为原分辨率编码方式，且所述M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则在所述编码方式为原分辨率编码方式的参考重建图像块的像素中确定所述当前待重建图像块的参考像素；
    若所述当前待重建图像块的编码方式为原分辨率编码方式，且所述M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则从所述编码方式为下采样编码方式的参考重建图像块的像素中获取对所述当前待重建图像块进行重建时所需的像素， 并对所述当前待重建图像块进行重建时所需的像素进行上采样处理，以得到所述当前待重建图像块的参考像素；
    若所述当前待重建图像块的编码方式为下采样编码方式，且所述M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则在所述编码方式为下采样编码方式的参考重建图像块的像素中确定所述当前待重建图像块的参考像素；
    若所述当前待重建图像块的编码方式为所述下采样编码方式，且所述M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则从编码方式为原分辨率编码方式的参考重建图像块的像素中获取对所述当前待重建图像块进行重建时所需的像素，对所述当前待重建图像块进行重建时所需的像素进行下采样处理，以得到所述当前待重建图像块的参考像素。
  6. 根据权利要求2所述的方法，其特征在于，所述对所述当前重建图像块进行上采样处理包括：
    根据所述所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对所述当前重建图像块进行一次上采样处理；
    若所述所需的相邻重建图像块中的当前未完成重建的另一部分相邻重建图像块已完成重建，则根据所述另一部分相邻重建图像块对所述当前重建图像块的部分边界进行二次上采样处理，其中，所述当前重建图像块的所述部分边界与所述另一部分相邻重建图像块邻接。
  7. 根据权利要求2所述的方法，其特征在于，所述对所述当前重建图像块进行上采样处理包括：
    根据所述所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对所述当前重建图像块进行一次上采样处理；
    若所述当前图像的所有图像块均已完成重建，则根据在所述所需的相邻重建图像块中的另一部分相邻重建图像块对所述当前重建图像块的部分边界进行二次上采样处理，其中，所述另一部分相邻重建图像块是在对所述当前重建图像块进行第一次上采样处理时未完成重建的图像块；所述当前重建图像块的所述部分边界与所述另一部分相邻重建图像块邻接。
  8. 根据权利要求6或7所述的方法，其特征在于，所述根据所述当前待重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待重建图像块的多个参考像素，包括：
    若所述当前待重建图像块的编码方式为原分辨率编码方式，则在所述M个参考重建图像块的像素中确定所述当前待重建图像块的参考像素；
    若所述当前待重建图像块的编码方式为所述下采样编码方式，则在所述M个参考重建图像块的像素中获取对所述当前待重建图像块进行重建时所需的像素，并对所述当前待重建图像块进行重建时所需的像素进行下采样处理，以得到所述当前待重建图像块的参考像素。
  9. 根据权利要求6至8任一项所述的方法，其特征在于，所述一部分相邻重建图像块为所述当前重建图像块的上边图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的下边图像块和右边图像块；或者，
    所述一部分相邻重建图像块为所述当前重建图像块的左上图像块，上边图像块，右 上图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的右边图像块，左下图像块，下边图像块和右下图像块。
  10. 根据权利要求6至9任一项所述的方法，其特征在于，所述当前重建图像块的部分边界为所述当前重建图像块的右边界和下边界。
  11. 根据权利要求2至10任一项所述的方法，其特征在于，所述所需的相邻重建图像块包括所述当前重建图像块的上边图像块，下边图像块，左边图像块和右边图像块；或者，
    所述所需的相邻重建图像块包括所述当前重建图像块的上边图像块，下边图像块，左边图像块，右边图像块，左上图像块，左下图像块，右上图像块和右下图像块。
  12. 根据权利要求2至11任一项所述的方法，其特征在于，所述对所述当前重建图像块进行上采样处理后，所述方法还包括：
    标识所述当前重建图像块已完成上采样处理。
  13. 一种图像处理方法，其特征在于，包括：
    获取当前图像的当前待编码图像块的编码方式、所述当前待编码图像块对应的每个参考重建图像块中的像素，其中，所述编码方式为原分辨率编码方式或者下采样编码方式，所述当前待编码图像块对应M个参考重建图像块，M为大于或者等于1的正整数；
    根据所述当前待编码图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待编码图像块的多个参考像素；
    根据所述多个参考像素生成所述当前待编码图像块的预测信号；
    获取所述当前待编码图像块的编码信号，其中，当所述当前待编码图像块的编码方式是所述原分辨率编码方式时，所述编码信号为所述当前待编码图像块的原始信号，当所述当前待编码图像块的编码方式是所述下采样编码方式时，所述编码信号为所述当前待编码图像块的原始信号经过下采样处理后得到的信号；
    根据所述预测信号和所述编码信号生成所述当前待编码图像块的残差信号；
    对所述残差信号进行编码。
  14. 根据权利要求13所述的方法，其特征在于，还包括：
    生成所述当前待编码图像块的重建信号，并根据所述重建信号重建所述当前待编码图像块，得到当前重建图像块；
    若所述当前重建图像块的编码方式为所述下采样编码方式，则基于所述当前重建图像块进行上采样处理时所需的相邻重建图像块的像素，对所述当前重建图像块进行上采样处理。
  15. 根据权利要求14所述的方法，其特征在于，所述上采样处理是在所述当前图像的所有图像块都重建完成后进行的；
    相应的，所述根据所述当前待编码图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待重建图像块的多个参考像素，包括：
    获取所述M个参考重建图像块中的每个参考重建图像块的编码方式；
    根据所述当前待编码图像块的编码方式、所述M个参考重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待编码图像块的多个参考像素。
  16. 根据权利要求14所述的方法，其特征在于，所述上采样处理是在所述当前重建图像块进行上采样处理时所需的所有相邻重建图像块都重建完成后进行的；
    相应的，所述根据所述当前待编码图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待重建图像块的多个参考像素，包括：
    获取所述M个参考重建图像块中的每个参考重建图像块的编码方式；
    根据所述当前待编码图像块的编码方式、所述M个参考重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待编码图像块的多个参考像素。
  17. 根据权利要求15或16所述的方法，其特征在于，所述根据所述当前待编码图像块的编码方式、所述M个参考重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待编码图像块的多个参考像素，包括：
    若所述当前待编码图像块的编码方式为原分辨率编码方式，且所述M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则在所述编码方式为原分辨率编码方式的参考重建图像块的像素中确定所述当前待编码图像块的参考像素；
    若所述当前待编码图像块的编码方式为原分辨率编码方式，且所述M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则从所述编码方式为下采样编码方式的参考重建图像块的像素中获取对所述当前重建图像块进行重建时所需的像素，并对所述当前待编码图像块进行重建时所需的像素进行上采样处理，以得到所述当前待编码图像块的参考像素；
    若所述当前待编码图像块的编码方式为下采样编码方式，且所述M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则在所述编码方式为下采样编码方式的参考重建图像块的像素中确定所述当前待编码图像块的参考像素；
    若所述当前待编码图像块的编码方式为所述下采样编码方式，且所述M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则从编码方式为原分辨率编码方式的参考重建图像块的像素中获取对所述当前待编码图像块进行重建时所需的像素，对所述当前待编码图像块进行重建时所需的像素进行下采样处理，以得到所述当前待编码图像块的参考像素。
  18. 根据权利要求14所述的方法，其特征在于，所述对所述当前重建图像块进行上采样处理包括：
    根据所述所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对所述当前重建图像块进行一次上采样处理；
    若所述所需的相邻重建图像块中的当前未完成重建的另一部分相邻重建图像块已完成重建，则根据所述另一部分相邻重建图像块对所述当前重建图像块的部分边界进行二次上采样处理，其中，所述当前重建图像块的所述部分边界与所述另一部分相邻重建图像块邻接。
  19. 根据权利要求14所述的方法，其特征在于，所述对所述当前重建图像块进行上采样处理包括：
    根据所述所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对所述当前重建图像块进行一次上采样处理；
    若所述当前图像的所有图像块均已完成重建，则根据在所述所需的相邻重建图像块中的另一部分相邻重建图像块对所述当前重建图像块的部分边界进行二次上采样处理，其中，所述另一部分相邻重建图像块是在对所述当前重建图像块进行第一次上采样处理时未完成重建的图像块；所述当前重建图像块的所述部分边界与所述另一部分相邻重建 图像块邻接。
  20. 根据权利要求18或19所述的方法，其特征在于，所述根据所述当前待编码图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待编码图像块的多个参考像素，包括：
    若所述当前待编码图像块的编码方式为原分辨率编码方式，则在所述M个参考重建图像块的像素中确定所述当前待编码图像块的参考像素；
    若所述当前待编码图像块的编码方式为所述下采样编码方式，则在所述M个参考重建图像块的像素中获取对所述当前待编码图像块进行重建时所需的像素，并对所述当前待编码图像块进行重建时所需的像素进行下采样处理，以得到所述当前待编码图像块的参考像素。
  21. 根据权利要求18至20任一项所述的方法，其特征在于，所述一部分相邻重建图像块为所述当前重建图像块的上边图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的下边图像块和右边图像块；或者，
    所述一部分相邻重建图像块为所述当前重建图像块的左上图像块，上边图像块，右上图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的右边图像块，左下图像块，下边图像块和右下图像块。
  22. 根据权利要求18至21任一项所述的方法，其特征在于，所述当前重建图像块的部分边界为所述当前重建图像块的右边界和下边界。
  23. 根据权利要求14至22任一项所述的方法，其特征在于，所述所需的相邻重建图像块包括所述当前重建图像块的上边图像块，下边图像块，左边图像块和右边图像块；或者，
    所述所需的相邻重建图像块包括所述当前重建图像块的上边图像块，下边图像块，左边图像块，右边图像块，左上图像块，左下图像块，右上图像块和右下图像块。
  24. 根据权利要求14至23任一项所述的方法，其特征在于，所述对所述当前重建图像块进行上采样处理后，所述方法还包括：
    标识所述当前重建图像块已完成上采样处理。
  25. 根据权利要求13至24任一项所述的方法，其特征在于，所述获取当前待编码图像块的编码方式，包括：
    确定所述当前待编码图像块采用原分辨率编码方式时的第一编码代价；
    确定所述当前待编码图像块采用下采样编码方式时的第二编码代价；
    将所述第一编码代价和所述第二编码代价中较小的编码代价对应的编码方式作为所述当前待编码图像块的编码方式。
  26. 一种图像处理设备，其特征在于，包括：
    解析模块，用于解析码流，以获取当前图像的当前待重建图像块的编码方式、所述当前待重建图像块的残差信号、所述当前待重建图像块对应的每个参考重建图像块中的像素，其中，所述编码方式为原分辨率编码方式或者下采样编码方式，所述当前待重建图像块对应M个参考重建图像块，M为大于或者等于1的正整数；
    确定模块，用于根据所述当前待重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待重建图像块的多个参考像素；
    生成模块，用于根据所述多个参考像素生成所述当前待重建图像块的预测信号；
    重建模块，用于根据所述预测信号和所述残差信号生成所述当前待重建图像块的重建信号，并根据所述重建信号重建所述当前待重建图像块，得到当前重建图像块。
  27. 根据权利要求26所述的设备，其特征在于，还包括：处理模块；
    所述处理模块，用于若所述当前重建图像块的编码方式为所述下采样编码方式，则基于所述当前重建图像块进行上采样处理时所需的相邻重建图像块的像素，对所述当前重建图像块进行上采样处理。
  28. 根据权利要求26所述的设备，其特征在于，所述处理模块具体用于：在所述当前图像的所有图像块都重建完成后进行所述上采样处理；
    相应的，所述码流包括：所述M个参考重建图像块中的每个参考重建图像块的编码方式；
    所述确定模块具体用于：
    根据所述当前待重建图像块的编码方式、所述M个参考重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待重建图像块的多个参考像素。
  29. 根据权利要求26所述的设备，其特征在于，所述处理模块具体用于：在所述当前重建图像块进行上采样处理时所需的所有相邻重建图像块都重建完成后进行所述上采样处理；
    相应的，所述码流包括：所述M个参考重建图像块中的每个参考重建图像块的编码方式；
    所述确定模块具体用于：
    根据所述当前待重建图像块的编码方式、所述M个参考重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待重建图像块的多个参考像素。
  30. 根据权利要求28或29所述的设备，其特征在于，所述确定模块具体用于：
    若所述当前待重建图像块的编码方式为原分辨率编码方式，且所述M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则在所述编码方式为原分辨率编码方式的参考重建图像块的像素中确定所述当前待重建图像块的参考像素；
    若所述当前待重建图像块的编码方式为原分辨率编码方式，且所述M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则从所述编码方式为下采样编码方式的参考重建图像块的像素中获取对所述当前待重建图像块进行重建时所需的像素，并对所述当前待重建图像块进行重建时所需的像素进行上采样处理，以得到所述当前待重建图像块的参考像素；
    若所述当前待重建图像块的编码方式为下采样编码方式，且所述M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则在所述编码方式为下采样编码方式的参考重建图像块的像素中确定所述当前待重建图像块的参考像素；
    若所述当前待重建图像块的编码方式为所述下采样编码方式，且所述M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则从编码方式为原分辨率编码方式的参考重建图像块的像素中获取对所述当前待重建图像块进行重建时所需的像素，对所述当前待重建图像块进行重建时所需的像素进行下采样处理，以得到所述当前待重建图像块的参考像素。
  31. 根据权利要求26所述的设备，其特征在于，所述处理模块具体用于：
    根据所述所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像 素，对所述当前重建图像块进行一次上采样处理；
    若所述所需的相邻重建图像块中的当前未完成重建的另一部分相邻重建图像块已完成重建，则根据所述另一部分相邻重建图像块对所述当前重建图像块的部分边界进行二次上采样处理，其中，所述当前重建图像块的所述部分边界与所述另一部分相邻重建图像块邻接。
  32. 根据权利要求26所述的设备，其特征在于，所述处理模块具体用于：
    根据所述所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对所述当前重建图像块进行一次上采样处理；
    若所述当前图像的所有图像块均已完成重建，则根据在所述所需的相邻重建图像块中的另一部分相邻重建图像块对所述当前重建图像块的部分边界进行二次上采样处理，其中，所述另一部分相邻重建图像块是在对所述当前重建图像块进行第一次上采样处理时未完成重建的图像块；所述当前重建图像块的所述部分边界与所述另一部分相邻重建图像块邻接。
  33. 根据权利要求31或32所述的设备，其特征在于，所述确定模块具体用于：
    若所述当前待重建图像块的编码方式为原分辨率编码方式，则在所述M个参考重建图像块的像素中确定所述当前待重建图像块的参考像素；
    若所述当前待重建图像块的编码方式为所述下采样编码方式，则在所述M个参考重建图像块的像素中获取对所述当前待重建图像块进行重建时所需的像素，并对所述当前待重建图像块进行重建时所需的像素进行下采样处理，以得到所述当前待重建图像块的参考像素。
  34. 根据权利要求31至33任一项所述的设备，其特征在于，所述一部分相邻重建图像块为所述当前重建图像块的上边图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的下边图像块和右边图像块；或者，
    所述一部分相邻重建图像块为所述当前重建图像块的左上图像块，上边图像块，右上图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的右边图像块，左下图像块，下边图像块和右下图像块。
  35. 根据权利要求31至34任一项所述的设备，其特征在于，所述当前重建图像块的部分边界为所述当前重建图像块的右边界和下边界。
  36. 根据权利要求27至35任一项所述的设备，其特征在于，所述所需的相邻重建图像块包括所述当前重建图像块的上边图像块，下边图像块，左边图像块和右边图像块；或者，
    所述所需的相邻重建图像块包括所述当前重建图像块的上边图像块，下边图像块，左边图像块，右边图像块，左上图像块，左下图像块，右上图像块和右下图像块。
  37. 根据权利要求27至36任一项所述的设备，其特征在于，所述处理模块还用于：
    在对所述当前重建图像块进行上采样处理后，标识所述当前重建图像块已完成上采样处理。
  38. 一种图像处理设备，其特征在于，包括：
    获取模块，用于获取当前图像的当前待编码图像块的编码方式、所述当前待编码图像块对应的每个参考重建图像块中的像素，其中，所述编码方式为原分辨率编码方式或者下采样编码方式，所述当前待编码图像块对应M个参考重建图像块，M为大于或者等 于1的正整数；
    确定模块，用于根据所述当前待编码图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待编码图像块的多个参考像素；
    生成模块，用于根据所述多个参考像素生成所述当前待编码图像块的预测信号；
    所述获取模块，还用于获取所述当前待编码图像块的编码信号，其中，当所述当前待编码图像块的编码方式是所述原分辨率编码方式时，所述编码信号为所述当前待编码图像块的原始信号，当所述当前待编码图像块的编码方式是所述下采样编码方式时，所述编码信号为所述当前待编码图像块的原始信号经过下采样处理后得到的信号；
    所述生成模块，还用于根据所述预测信号和所述编码信号生成所述当前待编码图像块的残差信号；
    编码模块，用于对所述残差信号进行编码。
  39. 根据权利要求38所述的设备，其特征在于，还包括：处理模块；
    所述生成模块，还用于生成所述当前待编码图像块的重建信号，并根据所述重建信号重建所述当前待编码图像块，得到当前重建图像块；
    所述处理模块，用于若所述当前重建图像块的编码方式为所述下采样编码方式，则基于所述当前重建图像块进行上采样处理时所需的相邻重建图像块的像素，对所述当前重建图像块进行上采样处理。
  40. 根据权利要求39所述的设备，其特征在于，所述处理模块具体用于：在所述当前图像的所有图像块都重建完成后进行所述上采样处理；
    相应的，所述确定模块具体用于：
    获取所述M个参考重建图像块中的每个参考重建图像块的编码方式；
    根据所述当前待编码图像块的编码方式、所述M个参考重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待编码图像块的多个参考像素。
  41. 根据权利要求39所述的设备，其特征在于，所述处理模块具体用于：在所述当前重建图像块进行上采样处理时所需的所有相邻重建图像块都重建完成后进行所述上采样处理；
    相应的，所述确定模块具体用于：
    获取所述M个参考重建图像块中的每个参考重建图像块的编码方式；
    根据所述当前待编码图像块的编码方式、所述M个参考重建图像块的编码方式和所述M个参考重建图像块中的像素，确定所述当前待编码图像块的多个参考像素。
  42. 根据权利要求40或41所述的设备，其特征在于，所述确定模块具体用于：
    若所述当前待编码图像块的编码方式为原分辨率编码方式，且所述M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则在所述编码方式为原分辨率编码方式的参考重建图像块的像素中确定所述当前待编码图像块的参考像素；
    若所述当前待编码图像块的编码方式为原分辨率编码方式，且所述M个参考重建图像块包括编码方式为下采样编码方式的参考重建图像块，则从所述编码方式为下采样编码方式的参考重建图像块的像素中获取对所述当前重建图像块进行重建时所需的像素，并对所述当前待编码图像块进行重建时所需的像素进行上采样处理，以得到所述当前待编码图像块的参考像素；
    若所述当前待编码图像块的编码方式为下采样编码方式，且所述M个参考重建图像 块包括编码方式为下采样编码方式的参考重建图像块，则在所述编码方式为下采样编码方式的参考重建图像块的像素中确定所述当前待编码图像块的参考像素；
    若所述当前待编码图像块的编码方式为所述下采样编码方式，且所述M个参考重建图像块包括编码方式为原分辨率编码方式的参考重建图像块，则从编码方式为原分辨率编码方式的参考重建图像块的像素中获取对所述当前待编码图像块进行重建时所需的像素，对所述当前待编码图像块进行重建时所需的像素进行下采样处理，以得到所述当前待编码图像块的参考像素。
  43. 根据权利要求39所述的设备，其特征在于，所述处理模块具体用于：
    根据所述所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对所述当前重建图像块进行一次上采样处理；
    若所述所需的相邻重建图像块中的当前未完成重建的另一部分相邻重建图像块已完成重建，则根据所述另一部分相邻重建图像块对所述当前重建图像块的部分边界进行二次上采样处理，其中，所述当前重建图像块的所述部分边界与所述另一部分相邻重建图像块邻接。
  44. 根据权利要求39所述的设备，其特征在于，所述处理模块具体用于：
    根据所述所需的相邻重建图像块中的当前已完成重建的一部分相邻重建图像块的像素，对所述当前重建图像块进行一次上采样处理；
    若所述当前图像的所有图像块均已完成重建，则根据在所述所需的相邻重建图像块中的另一部分相邻重建图像块对所述当前重建图像块的部分边界进行二次上采样处理，其中，所述另一部分相邻重建图像块是在对所述当前重建图像块进行第一次上采样处理时未完成重建的图像块；所述当前重建图像块的所述部分边界与所述另一部分相邻重建图像块邻接。
  45. 根据权利要求43或44所述的设备，其特征在于，所述确定模块具体用于：
    若所述当前待编码图像块的编码方式为原分辨率编码方式，则在所述M个参考重建图像块的像素中确定所述当前待编码图像块的参考像素；
    若所述当前待编码图像块的编码方式为所述下采样编码方式，则在所述M个参考重建图像块的像素中获取对所述当前待编码图像块进行重建时所需的像素，并对所述当前待编码图像块进行重建时所需的像素进行下采样处理，以得到所述当前待编码图像块的参考像素。
  46. 根据权利要求43至45任一项所述的设备，其特征在于，所述一部分相邻重建图像块为所述当前重建图像块的上边图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的下边图像块和右边图像块；或者，
    所述一部分相邻重建图像块为所述当前重建图像块的左上图像块，上边图像块，右上图像块和左边图像块，所述另一部分相邻重建图像块为所述当前重建图像块的右边图像块，左下图像块，下边图像块和右下图像块。
  47. 根据权利要求43至46任一项所述的设备，其特征在于，所述当前重建图像块的部分边界为所述当前重建图像块的右边界和下边界。
  48. 根据权利要求39至47任一项所述的设备，其特征在于，所述所需的相邻重建图像块包括所述当前重建图像块的上边图像块，下边图像块，左边图像块和右边图像块；或者，
    所述所需的相邻重建图像块包括所述当前重建图像块的上边图像块，下边图像块，左边图像块，右边图像块，左上图像块，左下图像块，右上图像块和右下图像块。
  49. 根据权利要求39至48任一项所述的设备，其特征在于，所述处理模块还用于：
    在对所述当前重建图像块进行上采样处理后，标识所述当前重建图像块已完成上采样处理。
  50. 根据权利要求39至49任一项所述的设备，其特征在于，所述获取模块具体用于：
    确定所述当前待编码图像块采用原分辨率编码方式时的第一编码代价；
    确定所述当前待编码图像块采用下采样编码方式时的第二编码代价；
    将所述第一编码代价和所述第二编码代价中较小的编码代价对应的编码方式作为所述当前待编码图像块的编码方式。
  51. 一种图像处理系统，其特征在于，包括：如权利要求26至37任一项所述的图像处理设备，以及如权利要求38至50任一项所述的图像处理设备。
PCT/CN2018/081678 2017-07-13 2018-04-03 图像处理方法、设备及系统 Ceased WO2019011011A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710572139.7A CN109257608B (zh) 2017-07-13 2017-07-13 图像处理方法、设备及系统
CN201710572139.7 2017-07-13

Publications (1)

Publication Number Publication Date
WO2019011011A1 true WO2019011011A1 (zh) 2019-01-17

Family

ID=65000880

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/081678 Ceased WO2019011011A1 (zh) 2017-07-13 2018-04-03 图像处理方法、设备及系统

Country Status (3)

Country Link
CN (1) CN109257608B (zh)
TW (1) TWI672941B (zh)
WO (1) WO2019011011A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111641831B (zh) * 2019-03-01 2023-09-01 杭州海康威视数字技术股份有限公司 帧内预测方法、装置、设备及存储介质
CN112235568B (zh) * 2019-07-15 2024-03-26 华为技术有限公司 图像编码方法、解码方法、装置和存储介质
CN111586410B (zh) * 2020-06-02 2022-04-19 浙江大华技术股份有限公司 视频编码方法、解码方法及其相关装置
CN113329228B (zh) * 2021-05-27 2024-04-26 杭州网易智企科技有限公司 视频编码方法、解码方法、装置、电子设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105430417A (zh) * 2014-09-22 2016-03-23 中兴通讯股份有限公司 编码方法、解码方法、装置及电子设备
CN105556963A (zh) * 2013-10-14 2016-05-04 联发科技（新加坡）私人有限公司 用于hevc范围扩展的残差差分脉冲编码调制方法
US20170064323A1 (en) * 2011-10-26 2017-03-02 Intellectual Discovery Co., Ltd. Scalable video coding method and apparatus using intra prediction mode
WO2017069419A1 (ko) * 2015-10-22 2017-04-27 엘지전자 주식회사 비디오 코딩 시스템에서 인트라 예측 방법 및 장치

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RS61146B1 (sr) * 2011-10-24 2020-12-31 Innotive Ltd Postupak i aparat za dekodiranje slike
US9998727B2 (en) * 2012-09-19 2018-06-12 Qualcomm Incorporated Advanced inter-view residual prediction in multiview or 3-dimensional video coding

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170064323A1 (en) * 2011-10-26 2017-03-02 Intellectual Discovery Co., Ltd. Scalable video coding method and apparatus using intra prediction mode
CN105556963A (zh) * 2013-10-14 2016-05-04 联发科技（新加坡）私人有限公司 用于hevc范围扩展的残差差分脉冲编码调制方法
CN105430417A (zh) * 2014-09-22 2016-03-23 中兴通讯股份有限公司 编码方法、解码方法、装置及电子设备
WO2017069419A1 (ko) * 2015-10-22 2017-04-27 엘지전자 주식회사 비디오 코딩 시스템에서 인트라 예측 방법 및 장치

Also Published As

Publication number Publication date
TW201909635A (zh) 2019-03-01
CN109257608B (zh) 2021-11-30
TWI672941B (zh) 2019-09-21
CN109257608A (zh) 2019-01-22

Similar Documents

Publication Publication Date Title
TWI681670B (zh) 影像處理方法、設備及系統
TWI801087B (zh) 仿射線性加權畫面內預測技術
JP2019216466A (ja) 省略符号化を用いた映像符号化及び復号化装置及びその方法
KR20200044653A (ko) 딥 뉴럴 네트워크를 이용한 영상의 ａｉ 부호화 및 ａｉ 복호화 방법, 및 장치
TWI575932B (zh) 混成視訊解碼器、混成視訊編碼器及資料串流
TWI681672B (zh) 圖像處理方法、設備及系統
JP7802870B2 (ja) ビデオ画像成分予測方法および装置、コンピュータ記憶媒体
KR20230108286A (ko) 전처리를 이용한 비디오 인코딩
TWI672941B (zh) 影像處理方法、設備及系統
WO2013158669A1 (en) Method and apparatus of quantization matrix coding
KR20200050284A (ko) 영상 적응적 양자화 테이블을 이용한 영상의 부호화 장치 및 방법
TWI890043B (zh) 在4:4:4色度格式及單一樹狀結構情況下針對所有通道之基於矩陣的內部預測技術
US12615367B2 (en) Application of super resolution
US20240137517A1 (en) Super Resolution Position and Network Structure
KR20230107627A (ko) 후처리 제어를 이용한 비디오 디코딩
US20240137519A1 (en) Super resolution downsampling
CN114827614A (zh) 一种实现lcevc视频编码优化的方法
US20250267298A1 (en) Down-sampling methods and ratios for super-resolution based video coding
CN120602671A (zh) 运动矢量的确定方法、装置、设备、存储介质及产品
Tsang et al. Encoding of colour images using adaptive decimation and interpolation
CN118433404A (zh) 视频解码方法、视频编码方法及译码设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18832344

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18832344

Country of ref document: EP

Kind code of ref document: A1